Tests need to be faster than you probably think
Newsletterers! Hello! Welcome! I've heard that a few of you wish you could register with me that you've been enjoying this newsletter. First– thank you! Second– I have good news– you can reply to this e-mail– and it's free! I always enjoy hearing from readers, so please do write in.
I'm Nat Bennett, and this is Simpler Machines, a weekly letter about computer programs and the people who make them.
One of the things that keeps software consultants like me employed is that there are a bunch of practices where there are some key things that aren't obvious until you've actually seen them in person. Pairing, for instance, works dramatically better when you have the right workstation set up (with a screen, mouse, and keyboard for each member of the pair) but lots of people try pairing (and have a bad time) without that.
Another big one is test speed. How long should unit tests take to run? In the companies I've worked at that weren't hardcore test-driven development (TDD) shops, anywhere from 5 to 20 minutes wasn't unusual. Often, tests take longer than that.
Serious TDD practitioners and teachers tend to consider tests "slow" if they take longer than 15 seconds. Ideally they're faster than that, if you're going to run them on every code change.
The reason for this is, basically, human attention span. 15 seconds about the maximum amount of time that most people can pay attention to something without getting distracted. For a programmer, this is the length of time where a test run goes from "unnoticeable" to "slightly annoying." Exactly how long this takes depends a lot on the person and how you define "attention" but there's some evidence that active attention in an untrained person lasts for as little as 8 seconds.
There's a kind of state change the happens when tests cross this threshold and if you've ever worked with tests that fast you know what I'm talking about, especially if you've got them set up so that they automatically run on code changes. The tests are just... there? As soon as you look at them, the results are in. You don't even think about it. They're more like invariant assertions than tests. It's nice. It feels like having programmer super powers.
There's another threshold around the "cup of coffee" length that's more like "time until there's a temptation to pick up a second task." You don't want your "fast" tests that you run all the time to be this length, but a second set of tests that you run when you check in code can be about 15 minutes. More than that, though, and they'll still be running when you get back from that coffee break – and then you're going to pick up another story, and then you'll be juuuuust starting to get into something else if there's a failure in those tests that you need to investigate.
Context switches are productivity poison. They also just tend to make people miserable. Be kind to yourself! Do one thing at a time!
If you're really interested in the mechanics here you'll probably enjoy The Mind Illuminated, which significantly influenced my model of the mind. One of the points that book makes that's relevant here is that attention feels continuous but it's not. Attention is more like a string of beads, each sensory impression or thought is an independent moment. So your mind is constantly flickering across a range of ideas and impressions, and the feeling of "paying attention" is created when a particular idea or impression is present in many or most of those individual disconnected moments. For most people, objects that aren't constantly re-upping "I'm important!" fade from awareness really rapidly because they get outcompeted by new information.
I have a personal theory entirely unsupported by evidence that for various reasons programmers as a population do not have normal attention spans and that research on non-programmers is therefore of limited use in describing the attention behavior of the median programmer but this is probably crankery.