Here’s my advice for keeping a lab notebook as a computer scientist:

Use a text file

There’s no need to use anything fancier. And if you’re writing code all day, you can just keep your log file open in your editor of choice, so that you don’t need to leave your hacking environment to make a note of something. I’m an emacs junkie and use org-mode to keep all my notes, but use whatever you’re comfortable with.

You could use a physical notebook, but unless you expect to want documentation of what you were working on for some legal reason, I don’t see the point. And most programmers I know hate writing things out by hand.

Treat your file as append-only

One of the downsides of just using a text file is that you need to enforce some discipline on yourself. It’s tempting to go back and fix up things that have changed, or to keep editing a desgin description as it evolved. But creating a perfect finished piece of documentation is a different task. The point here is to have a log of everything you do, not a single finished document. You can work on one side-by-side with the (dated or even timestampped) append-only log.

Keep automatic backups.

Do this any way you want. I keep my log files in git, and have a cron job that commits and pushes to a server every five minutes. This provides at least two benefits. First, it applies an automatic timestamp (to within five minutes) on anything I type, and second, it serves to make sure that a recent version is available on my server, in case I switch machines and want to keep working on a project.

Train yourself to make regular entries.

Keeping this nice text file, regularly backed up, is useless if you don’t record relevant things into it. Getting yourself into the habit of writing enough down is tricky. One strategy I’ve found helpful is to write down a list of “triggers” for situations that you promise to always record. With an explict list of these that you’ve decided to always record, you can get good at making a note whenever you perform one of these tasks, which gets you into the habit of writing things down. Here’s a partial list of some of the sorts of things I make a point of writing down:

• Whenever I take a measurement. That is, any time I run time, or look at top to figure out how much memory a program under development is using, or similar, I’ll write down the number and the circumstances under which it was recorded. I’ve found that I almost always would like to have a better record of such things later.
• Whenever I do a non-trivial search and find a result. This can be anything from googling for a solution to some problem where it takes me several tries to find the right keywords, or grepping a large codebase searching for the function that performs some task, or tracking down some API function in documentation. If I wanted it once, I’ll often want it later, and if it was hard to find, I’ll feel stupid if I didn’t write it down.
• Whenever I make an explicit design decision. If I’m thinking about how to design some function, system, process, or whatever, I’ll stop and take a moment to write down the options I was considering, and why I made the choice I did. This ensures I can go back and remember why I built a system the way it is.
• Whenever I wish I had written something down last time. This should be obvious, but if you find yourself thinking “I know I did this before; Why didn’t I take notes?”, you should always make a point of writing it down this time, no matter what excuse you come up with for why you really won’t need that information a third time.

These categories are deliberately broad, but at the same time well-defined. It’s fairly easy to train yourself to notice whenever any of these points triggers, and remember to stop and write something down. And that’s the first step to getting in the habit of writing down everything that you’ll later wish you had written down.

Software engineers, as a rule, suck at writing things down. Part of this is training – unlike chemists and biologists who are trailed to obsessively document everything they do in their lab notebooks, computer scientists are taught to document the end results of their work, but aren’t, in general, taught to take notes as they go, and document the steps they take in building a system. 6.005, MIT’s new introductory software engineering class, attempted to require its students to keep lab notebooks for a few semesters, and was met with near-universal complaints and ridicule from the students (“Lab notebooks? For a software engineering class? What the hell?”). (To be fair, I suspect they did a horrible job of it, but I’m not sure that students would have been any less confused at the idea).

Part of the reason is probably also the nature of software that makes it very easy to record certain things as part of our tools, and that makes experiments cheaply reproducible. Version control lets us document the process of developing a piece of code, so it feels superfluous to be taking additional notes on the side. Computers are mostly deterministic, and cycles are cheap, so why bother meticuously recording the results of a test run somewhere when you can just run it again later, any time you want? Computers feel much neater and simpler than messy bio or chem labs, and software is much simpler than complicated biology experiments or chemical syntheses, and so no one feels the need to be nearly as careful.

However, I am increasingly of the opinion that most software engineers’ total inability to work in a lab-notebook style, where you meticulously document your work, is unfortunate and often seriously detrimental to their work. While it’s true that things like commit logs do a good job of documenting certain processes, here are some types of situations where I’ve found working with meticulous notes at every step can be invaluable:

Debugging subtle problems

Debugging is very much a problem of gathering data and making and testing hypotheses. For subtle bugs in large programs, the amount of state you need to keep track of can rapidly get out of hand. And good luck when a bug is tricky enough that your debugging gets spread across multiple days, or even across a lunch break.

If you’ve ever found yourself wondering “Wait – did I see the bug after I made $CHANGE to my code or test environment?”, you should have been writing more things down.

This is especially important for non-deterministic bugs, such as rare race conditions. If it takes you half a hour on average to reproduce a bug, and you are experimenting with a dozen different variables in your test environment that might affect the bug, you can’t afford to forget the results of a single test, or to forget a single detail of what you did to test. This is the point at which you should be writing down every single command you type in any relevant prompts, and every single code change (or, since we have technology, obsessively saving the output of `history`, making commits to test branches, and recording the correlation between them).

Profiling and optimization

This is a process similar in many ways to debugging, but even more data-driven. When you’re done with a session of optimizing a piece of code or a system, if you can’t show documented evidence of exactly how much faster you’ve made it, where that speedup came from, and all the things you tried and how much they helped, perhaps you should be writing more things down. And if you (or ideally, even someone else) can’t go back and reproduce the experiments you did, with approximately the same results, you probably haven’t been documenting your work well enough.

Even if you’re happy with the performance improvements you’ve made, you may need to come back and wring even more performance out of the system, and it’d be nice not to start from scratch. Or maybe future testing will reveal that or more of your optimizations was invalid, and you need to go back and consider alternate options.

This is critically important when you’re optimizing not just a piece of code, but some kind of system with lots of configuration and setup, that you’ll later have to duplicate somewhere else, instead of just checking the result into source control.

Understanding a new project’s code or documentation

Whenever I’m first diving into a large code-base or first playing with a large new API, I find it invaluable to take notes as I go about what I look for and where I find it. I’ll often need to look up half a dozen different API calls or pieces of code to understand something, often too many to keep in my head as I go and dive through more and more pieces of code or docs.

And when the documentation is ambiguous, I’ll often drop into a REPL or build test programs to make various calls and understand what happens. Again, after more than two or three of these, it’s vital that I’ve been writing down my findings.

This is one example where a chronological style documenting exactly in what order I found things is less critical, but that detailed notes as I go are still vital.

Designing things

Whenever you’re designing something – be it an API, a protocol, an interface, some kind of system, or something else – it’s worth taking notes on the process you took to get to your final decisions, and the choices you considered and rejected, and why.

You’ll presumably end up a producing a piece of code or a design document that indicates what you ended up deciding, but understanding why you made the decisions you made is often important to understanding how your system is supposed to work, and how to best use or extend it in the future. Hopefully, when you’re done, you’d do this writeup in brief somewhere anyways, but the best way to make sure you don’t forget is to take good notes as your thought process happens.

And nothing in software is ever complete. If you have to revise the design for some reason, because someone points out problems or new requirements come up, you’ll probably want to remember the other possibilities you came up with – maybe one of them is now more right.

So, if you’re a software engineer, I strongly encourage you to try to get better at writing things down. In a future post, I’ll hopefully write up the techniques I’ve started using to take notes as I code, debug, and design, but in the meanwhile, I encourage you to just grab a text editor or a physical lab notebook, whichever is more comfortable for you, and start taking more notes on what you’re doing.