There’s a pyramid for a reason. It only takes a couple of tests to make sure that your plumbing connects all the way through. You inspect all the bits when they are going in but in the end you still check that things end up where they are supposed to.

I’ve been doing automated testing for a while. It’s hard to learn, there aren’t many people to emulate. Well, there are people to emulate but the winning strategies are conterintuitive, so your gut fights you the entire time. It took me 8 years to feel confident in tests and my own test code routinely makes me sad because I find antipatterns and I should know better. Also other people copy all of my mistakes :/

I’ve seen a number independent groups two or more years into their testing adventure and the failure modes are not that different. Everyone pretty much makes the same mistakes I do, and it’s frustrating watching everyone go through the pain before they accept that something has to change and it’s probably them.

The best strat I know of for testing is to use inductive reasoning and sampling to verify. If you don’t like the plumbing analogy then this is the Logic version of the same thing. If A -> B and B -> C then A -> C. Only a couple of your tests should verify A -> C and the bulk should check every kind of A [edit] and every kind of B.

If you want to do things like this without making your code not ‘say’ anything (a huge pet peeve of mine, so I can empathize with your concerns) then there are a couple of things to do there. One is an old trick from Bertrand Meyer: split code that makes decisions from code that acts upon them. Beth is split leaves the code more legible, not less.

Most of the boundary conditions are in the decisions. And this code is side effect free you can test the hell out of it with no mocks. Getting every permutation is straightforward and you can count your tests and your conditional branches to figure out if you are done.

Once your code looks like this, adding and removing new rules to the system later is a snap. Even much later.

> There’s a pyramid for a reason.

Sorry, but I am still unconvinced people got that reason correctly.

Let's say you have that A -> B; B -> C pipeline. How many tests you should have on each step (and on the origin) depends completely on how much freedom that steps grants you. It is not something one can say generalities about.

For example, if you are writing an enterprise CRUD application, almost your entire freedom resides on the data mapping. That means that your tests should be equally divided between data validation and data storage/retrieval. And the second can only be done at the integration or E2E levels.

If you are writing a multi-client statefull server (like a threaded web server), the freedom concentrated on launching and reloading it is so large that you can't even reasonably test for it. You'd better design your software around proving this is correct and let testing for less problematic stuff.

My biggest issue with the unity test pushing isn't even that it forces a bad structure into the code (what it does), or that it's pushes for fragile and valueless code (what it also dies). It is that it's wrong at the larger level, oversimplifying stuff and letting people get out of the hook without thinking for themselves.