Incremental improvement works

Hello there! If you're new– welcome. If you're not– welcome back! I'm Nat Bennett, a software engineer/consultant, writer, and occasional wandering photographer, and you're reading Simpler Machines, a usually-weekly newsletter about software, design, and systems.

I've been on a contract, and will be through the rest of the year so the newsletter will probably continue to be less regular than usual. Then we'll wrap up Season 2 entirely, take a break, and launch Season 3.

Mostly what I've been working on is a React application, and deploy scripts for that application.

Until a few weeks ago I didn't know any React. Have been figuring it out all on the job. My secret weapon? Taking tiny steps. Specifically, with the Mikado Method.

I learned how property threading and state hooks work in React just by doing this:

• Take a look at some code that's very confusing.
• Make a guess at how to make the change I want happen. Copy some code naively.
• Look at what it actually did in the browser. It didn't do what I wanted. I don't understand why what happened instead happened.
• git stash save "a description of what I was trying to do"
• Look at the code again. Identify a refactor that would make it easier to understand the code. Maybe I take two parts of the codebase that seem like they should be identical and make them actual identical. Maybe I extract a new component. Maybe I rename something.
• Look at what it actually did in the browser. Did it do what I wanted? If so, git stash pop. If not, stash again, and try an even smaller refactor.

Repeat until the part of the codebase I'm working on makes enough sense that I can make the change I'm actually trying to make.

I like this technique because it helps me do the right amount of refactoring. Most refactoring that most developers do is a waste of time because it's making code more changeable that they're not actually going to change, or it's making changes easier that they'll never actually make.

Where small steps really came in handy was with the deploy process.

This particular project uses Lightsail. Lightsail is somewhere between "devops resistant" and "devops hostile." By default it thinks you should deploy by clicking in a GUI.

That obviously wasn't acceptable but I also didn't want to do the "right" thing and spend a bunch of time writing Cloud Formation. We're pretty sure we'll be migrating off of Lightsail soon and frankly I just hate writing Cloud Formation. Most of the work in writing infrastructure templates is correcting errors, so you spend 90% of the time "almost done." (When anyone says they're "almost done" you should mentally translate that into "I don't know how long this will take.) And the feedback cycles are long. It's frustrating.

So instead I wrote a dumb bash script that automated everything that was easy to automate, because we were already doing it on the command line, and deployed a few times. Then I wrote another Bash script, that ran a single command using output from the first script, that was copy and pasted by a human into the right form, and deployed a few times. Then I wrote a third Bash script, that parsed the output of the first one and built the right structure for the inputs to the second script.

Finally, I copy-pasted them all together. Now the deploy is one command, plus checking that the deploy worked.

The next step will be putting that deploy script into Github actions or similar, but doing it by hand for a while let me incrementally make the process easier and more reliable in between other work.

My point is: Incremental improvement works. It works really well. It takes discipline but it's worth it. There's no awful sucking swamp of "the new system isn't ready and the old system still sucks" to wade through. There's no product manager hassling me for an estimate on a refactor. There's no "and then we spent three months getting all the bugs out of the rewrite."

Instead: Steady, continuous progress.