What title would you prefer? "Your Python mock might not work, but it could still be a good idea if you do it right, and here I will explain how"? :)
I didn't mean to imply that mocking is bad. Is that what you took from it? Why would I explain how to get mocks to work if I thought people shouldn't use mocks?
I’ve recently learned that some actually do consider mocking harmful.
Where I work right now, there’s a really outdated and unfashionable fight over the benefit of unit testing in general. Existing engineers don’t see value in test driven development, exhaustive testing, unit testing, and mocks/spies are thrown in... and we’re a python shop. I’m utterly confused. I, too, grew concerned after reading–will I be hearing this cited/twisted as further evidence against investing in our dreadful testing situation?
I'm a died in the wool TTDer, but I actually think that mocking unnecessarily is usually harmful. I use it as a technique of last resort.
I should define my terms before I explain, because many people use the term "mock" to mean things it didn't originally mean. Test objects that are used in place of domain level objects were traditionally known as "fakes". A fake that represented a fixed known value was called a "stub". A fake that included an assertion that a function was called (or that collected data on function calling) was called a "mock". It's a bit confusing for me that many people use the term "mock" to mean "fake". I found it weird that the original article pointed to an article on faking and then used the term "mock" without referred to the original meaning of the that word (which makes me wonder what they mean when they say "fake").
Anyway, usually you want a fake when you don't have access to some part of the system to test it directly. Sometimes that's because it's a completely different service. You can fake out that service so that you can see if the code that interacts with that service is working, without having to actually set up the service.
A stub is useful in situations where you need to know that your code is working with specific values of data inputs. So you might have an object that you pass to a function and you want to know what happens if one of the properties on the object is null. It might be hard to set that up, so you can stub it out.
A mock on the other hand, basically tests if a function is called. A good example where you might legitimately need a mock is where you pass an object to a function and you are expecting that a callback on that object will be called. It's really hard to test that without a mock.
Where mocks can be dangerous is when you completely mock out any interfaces and stub the return values. You pass a fake object as a collaborator to your function and you test that your function works. The problem is that your fake object may not necessarily represent a real object in the system.
If you ever want to refactor the code, your tests will no longer tell you that a property is missing, or that a function is missing because all of your test code is using fake objects with mocked and stubbed methods. Ideally a unit test that uses an interface should fail when you change that interface. This allows you simply to change an interface somewhere and have your tests tell you exactly what you need to do to make that change work.
Where you end up getting a lot of conflict WRT testing strategy is that some people believe very strongly that unit tests should test things in isolation. Secondly people believe that unit testing should be a black box testing strategy. So you should test through your public interfaces only and any collaborators that adheres to the interface contract should work as expected.
In this style of testing, you are often encouraged to mock anything and everything at the interface boundary. This has many advantages. First, it means that your test objects can be very simple, so writing tests is very quick -- even if the code in the system is complex (because you aren't using any of that code). Second, because you are testing the public interface only and there is minimal setup, your tests become documentation of the interface contracts. Third, because it is black box, if you change the implementation of your "unit", you don't have to change your tests.
Despite these benefits, I'm not a big fan of this style. I like white box testing using real collaborators. My goal is not to define interfaces and nail them up -- quite the opposite. I want to be able to change interfaces fluidly. I value ease of refactoring over just about anything else. Second, I want to use real collaborators almost because it is painful. If your collaborator is awkward and brittle to set up in tests, it is also awkward and brittle to set up in production code. My goal is to remove that and to simply the code. Again, my highest value is my ability to refactor the code. I want the code to become easier to work with over time, not harder and more complex. Finally, I want to code to break at a "white box" level, not a "black box" level when I change behaviour. Ideally, I want my tests to say, "On the third line of that function, we're going to have a problem because that function is different now". I don't want to be aware of problems at a larger scope "Somewhere in function A that calls function B which calls function C and D there is something wrong because it does something weird".
In the end I write small functions that are tested directly with real collaborators. I avoid private functions because it hides my implementation details. I test at a low level so that I avoid test complexity from excess branching. I get incredible specificity from failing tests, when end up essentially giving me a TODO list for what I need to do when refactoring code.
Hope that gives you some idea of at least why one person avoids mocking -- although, you do need it sometimes. And to be fair, sometimes I'll do a London School, outside in, mock the world implementation if I'm not sure what I'm building. However, I throw away all my mocks and re-TDD once I know what I'm building.
> some people believe very strongly that unit tests should test things in isolation. Secondly people believe that unit testing should be a black box testing strategy. So you should test through your public interfaces only and any collaborators that adheres to the interface contract should work as expected.
I think much of the confusion and talking-past-each-other comes from ambiguous language. I actually agree with all the things in the above quote (isolation, black-box, public-only, relying only on specified interfaces). Where I've differed from co-workers is that I consider the appropriate "unit" to be a feature/piece-of-functionality (e.g. "logging in"), whereas they consider the appropriate "unit" to be a piece of code (e.g. a method or class).
I think Michael Feathers explained it the best. He likened unit testing to clamping a piece of woodwork while you are working on it. The bits you are working on need to be in motion because you are working on them. The bits you are not working on them need to be clamped in place -- you don't want those things moving while you are working on some other bit. A "unit" is anything you might want to be clamped in place. It can be a function. It can be an object. It can be a subsystem. You want to unit test at different levels of abstraction so that you can "clamp" those levels of abstraction down.
One of the things I've found people get confused with is that they see unit testing and integration testing as orthogonal. They think a unit test should exercise a small piece of code in isolation and an integration test should test examples of real collaborators. Frequently they mock out all their unit tests and write a few integration tests. Then their unit tests become brittle and annoying and so they delete them, leaving only a few integration tests. This leads a lot of people with the impression that only integration tests are useful. If we can back up and redefine "unit test", then the problem disappears.
I read your "rant". Don't even get me started on BDD :-) Originally people had problems understanding the purpose of TDD because the word "test" had them confused. They would think, "I need to write tests to ensure that this is working". They didn't think about it in terms of clamping the behaviour so that it doesn't change when you are working on another part of the system. For that reason, a lot of people discussed changing the word "test" to something else that truly embodied what TDD was all about. Many people hit on the word "behavior" -- you want to document the current behaviour of your "units" (at different levels of abstraction). Somehow this got totally confused with automated acceptance testing! Now we have things like cucumber (which I don't actually hate, but it accomplishes a completely different goal than TDD!)
What really frustrates me is when I talk to people about this stuff and they think I'm a complete lunatic :-)
Ian Cooper reminds what was Kent's original proposition on TDD, what misunderstandings occurred along the way,
and suggests a better approach to TDD, one that supports development rather impeding it.
I'm not sure what your objection is. People try to use mocks, and they don't work because they've mocked the wrong name. I explain this in the article. The article is about why their mock didn't work. How is this a misleading title? How is this a title I don't agree with?
Are we talking about the same piece and the same title? You seem to be under the impression that I am trying to tell people not to use mocks. Have you read the piece?
I notice that the title on HN is "Why a mock doesn't work," which could be interpreted as "Mocks don't work." My title was (and still is) "Why your mock doesn't work." I don't know if that is the source of the confusion.
Now you seem to be willfully misunderstanding. He states clearly that your article is specifically about mocking in python. But mocking as a concept need not relate to python at all. Hence the confusion.
A better title for your article would be "Why a mock doesn't work in python"
I can see why adding "Python" would help here on Hacker News. The original complaint seemed to go deeper than that, I think because the title here is different than the title on my site.
Personally I've actually come to prefer such titles—where the thesis is in the title, and I can judge at once whether I should read for details. I'd aim for about 200 characters. HN is already full of uninformative links.
Since we're in this discussion, it'd be fun to change the title to “Why Python tests may fail to mock imported module functions, and a better approach to doing that”.
I didn't mean to imply that mocking is bad. Is that what you took from it? Why would I explain how to get mocks to work if I thought people shouldn't use mocks?