Comments on: A Very Subtle Bug

By: Chris Lesniewski-Laas

Chris Lesniewski-Laas — Thu, 11 Mar 2010 04:33:18 +0000

In 6.033, "complex systems fail for complex reasons" does refer to the subtle interactions between components, but it also is meant to serve as a reminder that technical root causes have deeper organizational and societal root causes: the "Keep Digging" principle.

In fact, the Rogers Report on the Space Shuttle Challenger disaster is a classic example of the Keep Digging principle. The Columbia disaster does not have the same classic status, but NASA applied similar thoroughness in its analysis of that event.

By: tranto

tranto — Tue, 02 Mar 2010 09:51:13 +0000

nelhage :
The short version is that there’s a race condition — the kernel will buffer some amount of data, so it’s possible for gzip to write the remaining data into the kernel pipe buffer (without tar reading it), and exit before tar closes the pipe, in which case there’s no SIGPIPE.

Surely this can only succeed with fairly short .tar files then? I mean, if you want an entry from the beginning or the middle of a large archive, there's no way the contents following the entry will fit in a pipe buffer.

By: lacos / lbzip2

lacos / lbzip2 — Mon, 01 Mar 2010 17:22:16 +0000

Followup from the GNU tar maintainer here.

By: nelhage

nelhage — Mon, 01 Mar 2010 15:22:00 +0000

The short version is that there's a race condition -- the kernel will buffer some amount of data, so it's possible for gzip to write the remaining data into the kernel pipe buffer (without tar reading it), and exit before tar closes the pipe, in which case there's no SIGPIPE.

In response to this post and reddit discussion, a friend of mine did some tar source-diving and figured out some more about what's going on, which you can read on his blog.

By: Thanassis

Thanassis — Mon, 01 Mar 2010 13:24:47 +0000

I understand the complex interactions now that you’ve described them – what I don’t understand is how you sometimes have the command succeed, and sometimes fail… With SIGPIPE set to SIG_IGN, shouldn’t you ALWAYS get an EPIPE, and therefore an error at the Python level?

By: ALB

ALB — Sun, 28 Feb 2010 09:33:08 +0000

"Complex systems fail for complex reasons." I agree, this is a quotation that does not accurately capture reality. In fact, complex systems fail for simple reasons. However, the complexity makes it far more difficult to find the simple reason, in part because there are so many possible simple reasons.

A classic example of this is the Columbia shuttle disaster, in which a simple piece of falling foam doomed the mission.