I’ve finally settled on a solution that I like using git, and so this is a writeup of my workflows for working with my dotfiles in git, in the hopes that someone else might find it useful. You will not find any scripts here, only a description of a workflow: It’s simple enough that I have not felt the need to script any of the pieces, even though I potentially could.

On the machines

On each machine, I have the dotfiles directory checked out into ~/.dotfiles/, and a symlink farm from the actual files in ~/ into ~/.dotfiles. If I need to edit files, I just edit them in place and commit in ~/.dotfiles. When adding new dotfiles, I just manually create them in the checkout and create the symlink — no fancy scripts. I do this rarely enough that I find it doesn’t bug me.

Branches

I maintain one branch for each machine or group of machines I keep dotfiles on. For instance, my laptop has a branch, my desktop has a branch, and all of the machines I have accounts on at work share a branch. In addition there is a “master” branch, which contains a prototypical set of dotfiles without any machine-specific customizations.

In an ideal world, any change to my dotfiles would go to either master (if it is common to all machines) or to a specific machine branch, and I could then merge master into each machine branch to sync the state of my dotfiles around. Unfortunately, one of my main desiderata is that I can edit and commit dotfiles in place. And committing to a non-checked-out branch is awkward at best, and checking out master in the ~/.dotfiles/ working copy is undesirable, since that might disrupt other programs that are using my dotfiles at the time. So in practice, I make commits to the branch of whichever machine I made a given change on, and then push them to that branch in the master repository on my server.

Synchronizing

Periodically, I fetch that repository into another working copy, and synchronize the branches. I check out master, and git cherry-pick any commits off of each per-machine branch that should be shared among all the machines. Once that’s done, I check out each per-machine branch in turn, and git merge master back into it. With that done, I push the branches back, and pull them on each machine.

It’s very important that this process — including the merging back from master — be done in a separate working copy. Otherwise, a conflicted merge results in conflict markers in your dotfiles. It was particularly fun when I did this once and had a conflict in .gitconfig, as git then immediately stopped working, and so I couldn’t reset --hard out of the merge or inspect history until I either resolved the merge or moved the broken file aside.

In practice, this sequence happens maybe once a month, and takes half an hour or so. I could probably script some of that away, but I find it mostly acceptable as is.

Final thoughts

I find the “commit and cherry-pick” workflow somewhat unfortunate, in that it results in two copies of every commit, and it makes “synchronize my dotfiles” a sufficiently expensive operation (in terms of effort) that it doesn’t happy continually. It’s conceivable that I would be better served by being disciplined about, after testing any change, immediately copying the files to another repository and comitting onto master and merging into the appropriate branch. However, I haven’t experimented with that because I intensely dislike any workflow that turns a common, simple operation (“make a stupid fix to my dotfiles”) into a more complex one. The current workflow does include a complex operation (the cherry-pick and merge step), but it’s a batch job, so I don’t mind scheduling it periodically, and it is an additional task I perform on top of the normal work of “messing with dotfiles”, which has specific additional benefit (“keeping my dotfiles in sync”), so I find it much more palatable.

Learning to use git in this way requires two steps. First, you need to understand the git object model, so that you know how to express whatever you are trying to do in terms of operations on that object model. In general, I tend to recommend Git for Computer Scientists as an excellent quick introduction to the git object model for hackers who are basically familiar with concepts like graphs and trees. I’ve also given a talk for SIPB entitled Understanding Git, which attempt to explain git starting from an understanding of the object model. I recommend either or both of those documents to anyone who wants to understand the backend git object model.

The second step, of course, is learning the git commands so that you can actually carry out the operations on the git objects and working copy that you want to accomplish. Git for Computer Scientists falls short on this point, although I think this is mostly because that’s not its goal. This blog post is an attempt to provide a quick graphical reference for a number of common git commands, in the hopes of helping users who have gotten comfortable with the git model, but occasionally find themselves groping for the command they want to carry out some operation. I hope that the first image especially will also be useful as a reference for those already comfortable with git.

Working Trees, Indexes, and HEADs, Oh My!

Personally, I find that the commands that often trip me up the most in git are the various commands for moving data between the working tree, index, and HEAD. I can always remember the basic ones like “add” and “commit”, of course, but if I, say, want to update some files in the index to match some commit, it often takes a minute to remember I want git reset. The following image attempts to diagram the working tree, the index, and HEAD, the last commit, and show the common commands for moving data between any pair of them:

The command above each pair of arrows moves data from left to right, and the one on the bottom from right to left. As you can see from the diagram, moving data directly between the working tree and the latest commit generally also updates the index, which is what makes it possible to mostly pretend that the index doesn’t exist during normal usage, if you so choose. Also, of course, anywhere in this diagram where a command accepts HEAD, you can instead specify a an arbitrary commit object.

Working with branches and commits

The next pair of diagrams, will both be starting from the following image, showing part of a git repository that contains two branches (master and topic), which refer to some commits within the git object store:

Starting from this image, we can adjust master by adding new commits in several ways. The following diagram shows how the picture would change after each of git commit, git commit --amend or git merge topic:

Note that these three commands aren’t quite parallel — the first two, assuming you’ve staged some files into the index, will both result in the same tree object, whereas the third will execute a merge and construct its tree that way.

Instead of committing to the repository, we might want to create new branches, or move refs around. The fourth diagram, below, shows various commands we could execute to create a branch, move to another branch, or move around the master pointer:

Hopefully someone finds some of these useful as a quick reference, or perhaps feels inspired to do something similar but better — my graphic design abilities are limited at best. Let me know if you find any of these helpful!

I love git, and use it everywhere. One of the things I particularly love about git is the fact that I understand essentially every element of git’s data model and how it works, which means that I can invent odd workflows or script git to do things that I never could have with SVN or even SVK. Part of the reason for this is that git, internally, really is fundamentally simple in a way subversion isn’t. There’s some complexity (which I don’t and don’t have to understand) behind implementing its model efficiently, but I can describe the basic git data model in about a single slide (And in fact I have).

At the same time, though, I consistently find that people who come to git from subversion find it insufferably complex and hard to learn, and spend quite a while wrangling with it, before they eventually fight it to a draw where they only need to ask their local git expert to dig them out of a hole every once in a while, or else until they just give up and declare git to be space-alien and go back to something they “understand”.

The problem

Having helped enough of these people, I’ve begun to understand the problem. Fundamentally, the way (most) people learn and think about subversion is different from the way git experts think about git. subversion’s internal model is fairly complex, but you are not expected to understand it. You just have to know the half dozen commands you’ll ever need, and you use them, and everything is fine.

Git’s model is fundamentally fairly simple (a DAG of immutable commit objects where branches are named mutable pointers into it), but you are expected to understand it fully to use git effectively.

A subversion user who wants to get the latest updates from the server just knows to run svn up and go back to their life. A git user has to (in general) think about whether they want to rebase their local changes onto the remote changes or merge them, and which remote branch they want (this is all, of course, expressed essentially in terms of the git data model), and then maps that back into the commands they want to perform this task.

Subversion users aren’t used to thinking like this, and so when they ask a git user for help, they get one of two classes of answers, neither of which they like. Either they just get back a list of half a dozen possible commands they could choose (since the git user has mapped their request into the git object model, and can think of a bunch of ways to implement that operation, depending on your mood and the phase of the moon), or else they get a “What are you really trying to?”, since their request has multiple interpretations in the language of git objects. Neither answer is a single command that will always solve their problem, which is what they want, so they can just go back to whatever they were trying to do, and so they end concluding that git is needlessly strange and confusing.

I think there are at least two reasons for this different perspective from git users. One is historical — git was designed to work this way. Linus wrote git as a dumb content tracker, and intended other people to write various UIs on top of it, but not really for anyone to use it as-is. So fundamentally, git is designed to just implement this basic model, and not to export a single way to do anything. The other, perhaps more fundamental reason, is that git is designed to be infinitely flexible, and so it’s fairly rare that you can give an answer for “How do I do X with git?”, since the answer will often depend on, “Well, what are your project’s conventions?”

What do we do about this?

So, as git users, hackers, and evangelists, what can we do to improve this situation? I think that basically, we need to understand what’s going on here, and try to have better answers for what the subversion users regard as simple questions. We should encourage people to get to know and love the git object model, but realize that most of the time, they just want to get work done, and we need to try to have simple answers to simple questions.

To that end, I think there are three main remaining areas in the git UI that I think new users find confusing, and that we need to figure out how to improve. These are pulling and pushing changes, and the whole issue of the git index. Here are my thoughts on what’s wrong, and what we can improve on:

• ”git-push”: This command is a disaster in a number of ways. matching is utterly the wrong default for push.default (I believe 1.7 is going to fix this). The git push $REMOTE $BRANCH syntax is not optimized for the common “subversion-like” case of a single remote, and leads countless users to attempt to git push master, at which point they get the less than totally helpful error message:

  fatal: 'master' does not appear to be a git repository
  fatal: The remote end hung up unexpectedly
  

  In general, the error messages are pretty bad. I believe it was @defunkt who nominated “src refspec does not match any” as the “worst error message of all time”, and it’s hard not to see his point.

• ”git-pull” is also problematic. The vast majority of users in a project live at the edges, pushing code inwards, and they almost never actually want to create a merge commit. I almost always end up telling users “No, really, you always just want pull --rebase“, which leaves them with a poor impression of git’s UI (if you always want that flag, why isn’t it just mandatory?), and which is hard to explain, because the whole concept of “rebasing” is difficult to really grok without really understanding the commit DAG.

  branch.<name>.rebase and branch.autosetuprebase are a partial solution, but is there a reason I can’t just have a repository wide default option that makes --rebase the default? With the former solution, I’m never confident I’ve actually set it on all of my branches, or that it will get set if some script creates a branch without using git-branch or git-checkout.

  Similarly, it sucks that you can’t pull if you have any uncommitted changes at all, especially for subversion users who are used to this Just Working.

• Finally, we really need to improve the story with the index. As an experienced git user, I love the index, and use it for many reasons all the time. However, as anyone who has tried to explain it (or even to learn git) probably knows, it’s a confusing idea at first.

  Generally, when someone is starting out with git, I try to ignore the index, and suggest they just always use git commit with a path. I then have to handwave, of course, over why they need git commit -a instead of git commit, but that’s a mostly minor problem.

  The bigger issue comes when a user finds themselves in a merge conflict, and finds that it’s basically impossible to do anything without fully understanding the index, and all of the weird commands for moving content between the index, the working tree, and HEAD. Just the other day, three of us git experts had to stop and think for a while about how to answer the question “So, if I’m in a conflicted merge, and I want to just conditionally take “their” changes, what the hell do I do?”. That’s not a good sign for new users being able to figure out what to do here.

  I don’t fully know what the right answer is, but I think it’s clear the tools need some help here.