Output Waits in the Pipe: Why stdbuf Often Isn't Enough

You run a script in your terminal and the lines scroll past. You then add | grep error
to filter it, and the screen stays empty. The program did not hang and did not get stuck;
it is running, and it may well have finished. It just isn't talking.

The thesis of this article: your program does not decide when its output becomes visible.
The destination decides — a terminal, a pipe, or a plain file. And stdbuf, the tool
everyone reaches for to undo that decision, worked in only two of the seven runtimes
measured below. The rest keep their own buffer and never ask libc about it.

This blog has no shortage of "I turned the knob and the system ignored me" pieces. The
difference here: there is no knob to turn. Nothing is configured, no default is changed —
the same binary, without a line altered, behaves like two different programs depending on
what it is attached to.

Everything below was measured on my own machine (MacBook Pro M4 Pro, macOS 26 / Darwin
25.6.0, arm64) and, for comparison, inside Docker on Debian 13 / glibc 2.41 / aarch64.
I ran every line three times; the table values are the median of three runs — and on this
topic I got lucky, because all three runs came out identical.

How I set up the measurement

To measure the claim "output disappears," you have to make the loss visible. The method:
a program writes a fixed number of lines with printf, never calls fflush, then waits
forever. While it waits, I kill it with SIGKILL. SIGKILL cannot be caught, so whatever
sits in the buffer dies there. The number of bytes that reach the other end of the pipe
is the part that was genuinely flushed. Every byte that does not arrive is a log line you
would have lost in real life.

/* prod2.c — every line is exactly 10 bytes, no flush */ #include <stdio.h> #include <stdlib.h> #include <unistd.h> int main(int argc, char **argv) { long n = atol(argv[1]); for (long i = 0; i < n; i++) printf("%09ld\n", i); pause(); /* whatever is in the buffer stays there */ return 0; } 

Pinning the line length at 10 bytes made things easier: 1,638 lines is exactly 16,380 bytes,
four bytes under the 16 KiB boundary. Those four bytes will show up later in a place I did
not expect.

The reading side is a small hand-written Python script: it reads the pipe without blocking,
records when the first byte arrives, counts the total, and kills the child when time is up.
One subtlety: the measuring tool itself must not be buffered, or you end up
measuring your own noise. I read from the raw file descriptor with os.read, not through
sys.stdin.

The destination makes the call

I attached the same compiled binary to two different destinations; the plain-file case gets
its own table shortly. The program did not change; not one line of code differs.

That 18,018 looks odd at first because it exceeds what was written. The reason is innocent:
a pty turns every \n into \r\n, which is 1,638 extra bytes for 1,638 lines.
16,380 + 1,638 = 18,018. So attached to a terminal, everything got through. Attached
to a pipe, nothing did.

The difference comes from two separate decisions libc makes: the buffer's mode and its size.

The mode decision is written down in POSIX. The
stdin/stdout/stderr definition
says stdin and stdout shall be fully buffered "if and only if the file descriptor
associated with the stream is determined not to be associated with an interactive device."
So a terminal flushes line by line and a pipe waits until it is full. The standard leaves
the definition of "interactive device" to the implementation.

The size decision is not in the documentation at all. FreeBSD's
setvbuf(3) page only says
an "optimally-sized buffer is obtained" and gives no number. To learn the number you have
to read Apple's own libc source,
makebuf.c.
The line that sets the size is this:

fp->_blksize = *bufsize = st.st_blksize > MAXBUFSIZE ? MAXBUFSIZE : st.st_blksize; 

The buffer size is the destination's st_blksize, read via fstat. If fstat fails or
st_blksize is zero, it falls back to BUFSIZ. Line buffering is enabled only for
character devices where isatty returns true — in the source the couldbetty check is
(st.st_mode & S_IFMT) == S_IFCHR, so a pipe (a FIFO) can never pass through that gate.

This is easy to confirm. Here is the st_blksize of each destination:

A caveat on st_blksize: when I asked statx about O_DIRECT alignment, the result was that
the story this same field tells does not hold across five filesystems.
Here I am not using it as a promise about alignment, but as the number libc reads when
choosing a buffer size — and in that role it matches exactly.

And measuring the quantum right at the boundary lines up exactly:

### written to a PIPE, then SIGKILL written= 16380 bytes -> arrived= 0 written= 16390 bytes -> arrived= 16384 written= 32760 bytes -> arrived= 16384 written= 32770 bytes -> arrived= 32768 written= 50000 bytes -> arrived= 49152 ### written to a PLAIN FILE, then SIGKILL written= 4090 bytes -> in file= 0 written= 4100 bytes -> in file= 4096 written= 8190 bytes -> in file= 4096 written= 8200 bytes -> in file= 8192 written= 16380 bytes -> in file= 12288 

There are those four bytes. Writing 16,380 sent zero bytes into the pipe; writing 16,390
sent 16,384. The difference is ten bytes, one single line. That is how fragile the
relationship is between how much a program says and how much the reader hears.

One note: MAXBUFSIZE in the source is defined at two different values — 4 KiB if
FEATURE_SMALL_STDIOBUF is compiled in, 16 MiB otherwise. Since the buffer on a pipe came out at 16,384
bytes, the small variant is not enabled in the build Apple ships. I did not read
that from the source; I inferred it from the measurement, and I should say that it is an
inference.

stdbuf fixed two of seven runtimes

So far this was C. But nobody writes their log-producing job in C. I ran the same test —
16,380 bytes, to a pipe, no flush, then SIGKILL — across seven different runtimes, and
repeated each one with stdbuf -o L added. You will see nine rows in the table, because
Python was measured in two versions and Go through two different write paths.

macOS has /usr/bin/stdbuf, which surprised me; it is not GNU coreutils' stdbuf but
FreeBSD's version. The man page on my machine says so outright: the utility "first
appeared in FreeBSD 8.4", the idea came from Padraig Brady in GNU coreutils, and Jeremie
Le Hen did the FreeBSD implementation. GNU's own documentation says the tool is installed
only "on platforms that use the Executable and Linkable Format (ELF) and support the
constructor attribute" — macOS, being Mach-O, does not fit that description, so Apple
took the BSD equivalent in its shell_cmds package.

There are a few things in that table.

First: stdbuf only worked for C and awk — it had no effect in four of the six buffered
runtimes I added stdbuf -o L to.

To understand why, I looked at what the tool actually does, and did not find what I
expected. FreeBSD's man page
says it relies on a preload library called libstdbuf(3); on macOS there is no such
library. Comparing the environment of a process launched under stdbuf -o L turned up
exactly one difference: _STDBUF_O=L. No DYLD_INSERT_LIBRARIES, no
/usr/lib/libstdbuf.dylib.

The thing that reads that variable is inside the very file quoted above: __senvbuf() in
makebuf.c calls getenv("_STDBUF_O"), and __smakebuf() invokes it immediately after
__swhatbuf(). So on macOS stdbuf injects nothing; it just leaves an environment
variable behind, and libc itself makes the decision.

That also explains the limit in one sentence: the variable only changes the initial
decision libc makes for the stream. Python, Perl, Ruby and Go's bufio do not get their
buffer from libc in the first place — they build their own layer, so the variable never
reaches them. The GNU coreutils documentation states the same limit from its side: dd is
unaffected because it does not use FILE streams, and tee because it adjusts its own
streams' buffering.

Second: there are five different answers for the same 16,380 bytes — 0, 8,192, 8,200,
12,288, 16,380. One machine, one pipe, one byte count. Perl flushes at 8 KiB and Go's
bufio at 4 KiB (the Go source has defaultBufSize = 4096 and NewWriter passes it
directly — 12,288 = 3 × 4,096). None of them uses the 16,384 that libc picked for the pipe.

Third: Go's os.Stdout and Node are unbuffered. Both delivered everything. So "the
buffering problem" is not universal; it is a property of your language choice.

Let me also be open about the counting, because "two of seven" looks the way it does partly
because the denominator is one of my own picking. Counted as languages there are seven (C, awk, Python, Perl,
Ruby, Go, Node); the table has nine rows because Python appears in two versions and Go
through two write paths. I added stdbuf to the six that buffer, and it worked on two. But
honesty requires the other half: stdbuf was only ever designed for programs using libc
stdio, and within that set it did not fail — in the third section it goes two for two on
grep and cut. So "two of seven" measures not the quality of the tool but the fact that
most of a 2026 log pipeline is no longer C. Read the title's "often isn't enough" that way:
the tool is not broken, its scope is narrow.

Python has a fix, and that fix is not stdbuf. The
-u flag or the PYTHONUNBUFFERED environment variable settles it: the same test under
python3 -u delivered all 16,380 bytes. The Python documentation says -u will "force the
stdout and stderr streams to be unbuffered" and notes that since 3.7 the text layer is
unbuffered as well. That word "layer" is not idle: in Python sys.stdout is a
TextIOWrapper with a BufferedWriter underneath it. Two buffers stacked. That is why
Python's flush boundary is not a clean power of two, and why it also shifts with the
destination:

### macOS, python3, to a PIPE — threshold found by bisection written= 16390 bytes -> arrived= 0 written= 24570 bytes -> arrived= 0 written= 24580 bytes -> arrived= 16380 <-- first delivery written= 65530 bytes -> arrived= 49140 ### macOS, python3, to a PLAIN FILE written= 8190 bytes -> in file= 0 written= 8200 bytes -> in file= 8190 

Rather than guess the threshold I bisected for it: at 24,570 bytes into a pipe the output is
still zero, and the first delivery lands at 24,580. That number should look familiar —
24,576 is exactly 8 KiB + 16 KiB. Python's 8,192-byte text layer and the 16,384 libc chose
for this pipe appear to be stacked on top of each other. I say "appear" because I did not
trace the interaction of the two layers through the CPython source; what I have is a
measurement and an addition that works out. That is not enough to assert the mechanism, but
it is enough to say where the threshold sits.

The real culprit is the middle of the pipeline

The above is the "what if the source misbehaves" scenario. The far more annoying case in
real life is this one: the source is perfect and the filter in the middle is to blame.

To measure that I wrote a producer that calls fflush on every line and emits 20 lines per
second. There is no flaw on the source side. Then I put a single filter in the middle, waited
two seconds, and counted how many lines reached the end of the pipeline. A healthy value
is 38.

Six filters passed not a single line in two seconds. grep, sed, cut, tr, awk, jq
— that is every tool you would reach for to filter a log. The producer was flushing each
line immediately; the filter poured them into its own 16 KiB container and waited. I tested
jq separately with a JSON-producing source, since that is the standard way to filter
structured logs; the result was the same.

That cat and tee pass things through is no coincidence. The GNU documentation already
told us tee adjusts its own buffering, and here we see it in the measurement too. If you
reach for tee to both watch a log on screen and write it to a file, you have picked the
right tool without knowing it.

The fix side is cheap, fortunately, and the bottom half of the table shows it. Most tools
have their own flag: grep --line-buffered, sed -l (BSD), fflush() inside awk,
jq --unbuffered. All four went through the same test and all four passed the 38 lines. I checked the GNU
side separately in a Linux container: there the flag is sed -u, and it works the same way
— BSD's -l and GNU's -u do the same job with different letters, and you have to notice
that when writing a portable script.

For tools without their own flag, like cut, stdbuf -o L steps in — because those use
plain stdio, which is exactly stdbuf's area of effect. The last two rows of the table
confirm it: stdbuf works here, while a section ago it did nothing for Python. A tool
working in one place and failing in another is not random; the boundary is whether the
program left its buffering to libc.

Let me also record a mistake of my own here, because the correction was instructive. In my
first attempt the wait was done with system("sleep 30"), and awk delivered all 16,380
bytes as though it used no buffer at all. My first thought was "so awk is clean"; then I
realised my waiting method was doing the flushing — the awk on my machine empties stdout
before running system(). I had assumed this was a standard requirement, but POSIX's
awk page contains no
such rule: the only flushing requirement on that page belongs to the fflush function, and
the system() entry says only that the command is executed. So this is the behaviour of the
awk on this machine, not a mandate of the standard. Once I changed the waiting method, awk went
to zero bytes like everyone else.

The general lesson: your measuring tool can change the thing it measures. Ask your own test
rig the same question.

"It worked on Linux" is not evidence

I copied the same source files into Debian 13 inside Docker and compiled them with gcc.
glibc 2.41, aarch64. Not one line of code changed.

Same program, same byte count, same pipe: zero on macOS, 12,288 on Linux. The cause is a
single number — Linux reports a pipe's st_blksize as 4,096 and macOS as 16,384. libc
applies the same rule on both; the rule is identical and the outcome still differs fourfold.

I think the most practical conclusion from that table is this: if your CI runs on Linux and
you develop on a Mac, "I can see the logs locally" and "the logs show up in CI" are two
independent facts. The reverse holds too. Personally I would put running this test on both
platforms on the acceptance checklist.

Node's separate trap: no buffer, but still a loss

Node came out clean on the buffering side of this article. But it has a hole of its own, and
it is documented. The process.stdout note says writes
are synchronous or asynchronous depending on the destination: files are synchronous on both
platforms, terminals are synchronous on POSIX, and pipes and sockets are asynchronous on
POSIX. The warning follows immediately — synchronous writes avoid problems such as output
"not written at all if process.exit() is called before an asynchronous write completes."

I tried this, and my first attempt failed. Writing 10 MB line by line in a loop and then
calling process.exit(0) lost nothing at all; all 10,485,760 bytes arrived. I could not
reproduce what the documentation warned about.

To produce the loss I had to change the shape of the write. Giving the same 10 MB in a
single write call and exiting immediately:

 1,048,576 small writes in a loop + exit -> 10,485,760 bytes COMPLETE one 10,485,760-byte write + exit -> 65,536 bytes LOST 

The 65,536 bytes that arrived are exactly 64 KiB: as much as the kernel accepted into the
pipe buffer in one go. The rest was waiting in Node's queue, and process.exit() did not
wait for it. That is a 99.4% loss.

The loop version survives because each small write can drain as the reader consumes it. So
to produce this bug the write has to be too large to fit into the pipe at once. I am writing
this down because you are likely to try the same thing, and if you fail to reproduce it on
the first attempt I do not want you concluding there is no problem. The documentation is
right; its condition is just narrow.

The state of my own infrastructure

While writing this I asked my own blog repository the same question, because you should look
at your own house before handing out advice. The results:

• I have 17 GitHub Actions workflows. In eleven of them, across 31 lines, a grep, sed, awk, cut or tr is fed from a pipe. But that first count of 31 was misleading; what matters is where the filter sits. The number of lines with another pipe after the filter — genuinely in the middle of the pipeline — is 10: cache-purge.yml 1, content-generate.yml 6, deploy-only.yml 1, translate-archive.yml 2. In the other 21 the filter is the last link in the chain.
• Searching .github/, scripts/ and deploy/ for stdbuf, --line-buffered, sed -u/-l and PYTHONUNBUFFERED: zero hits. No precaution anywhere the pipelines are actually built. (The repository as a whole has exactly one PYTHONUNBUFFERED, inside an example systemd unit quoted in an older article — prose, not code.)

The situation is not alarming, and honesty requires writing down why. The 21 last-link lines
fall outside this article's subject: there is no reader waiting behind the filter, the output
goes into a variable or a file, the stream closes when the process ends, and closing flushes.
Buffering only turns into data loss when the job is cut off partway through — and being
cut off, as you will see below, is harsher than I assumed.

The one place that caught my attention was translate-archive.yml. There a job with a
60-minute timeout captures the output of a long translation loop with 2>&1 | tee translate.log.
If it times out, GitHub cancels the job. According to the table above, that pipeline is
safe: tee is one of the two tools that pass data through, and Node is unbuffered anyway.
But had I written grep there instead of tee, the log file would have been empty the
moment a 60-minute job was cut off — which is exactly the moment I would need to know why
it was cut off.

This is also a reminder that log loss is not a new subject for me. I previously measured
losing 33,879 lines in the gap that copytruncate dismisses as "a very small moment".
The mechanism in that article was completely different — there the loss came from logrotate
truncating the file, here it comes from libc's buffer. What they share is this: in both
cases, the component causing the loss reports no error at all. I have also measured the pipe
itself from another angle:
a neighbouring container filling the quota, so my new pipes came back at 8 KiB instead of 64 KiB,
happened just as silently. The 65,536 bytes below are that same 64 KiB.

"What if I stop it gently?" — no

Here I had to test an assumption of my own. When a job is cancelled, SIGTERM arrives before
SIGKILL, and it seems reasonable to think the polite signal would shut things down properly
and flush the buffer. The measurement says otherwise:

16,380 bytes written to a pipe, then a signal: SIGTERM -> arrived=0 SIGINT -> arrived=0 SIGKILL -> arrived=0 SIGHUP -> arrived=0 

All four are zero. The reason is simple but easy to miss: a signal's default disposition
is to terminate the process, not to call exit(). Because exit() is never called, atexit
handlers do not run and stdio buffers are not flushed. So what saves the buffer is not the
politeness of the signal but the program exiting properly under its own control. A process
that catches SIGTERM and leaves via exit() keeps its data; one that does not catch it
never does.

That leaves my translate-archive.yml concern standing: when GitHub cancels the job, I
cannot expect a polite signal to rescue me.

In a container, stdout is already a pipe

If you are reading this thinking "but I don't use pipes," look again. Inside a container
stdout is a pipe to the logging driver — what you see with docker logs or
kubectl logs is the far end of it. So the runtime table above applies as-is to every
containerised application: a service written in Python goes silent in a container unless it
is given -u, and that is the classic case where hours get burned on "the logs aren't
coming through." My own blog is served from a container too.

There is also a free way out: stderr. POSIX says stderr shall not be fully buffered when
opened, and the measurement agrees — writing the same program's output to stderr delivered
all 16,380 bytes immediately. Sending diagnostic lines to stderr is the cheapest fix
available, and it needs no flag at all.

A decision framework

If output is delayed in a pipeline, check these in order:

1. Identify the destination. Checking st_blksize with fstat is not overkill: a pipe is 16 KiB (macOS) or 4 KiB (Linux), a plain file 4 KiB. The delay you should expect comes from there.
2. Look for the culprit in the middle of the pipeline. Before changing the source, pull the filters out one at a time and retest. In my measurement the source was flawless and grep was the blockage.
3. If the tool has its own flag, use it. The ones verified here: grep --line-buffered, sed -u (GNU) / sed -l (BSD), fflush() inside awk, jq --unbuffered, -u or PYTHONUNBUFFERED=1 in Python, $| = 1 in Perl. In Go, calling Flush() on a bufio.Writer is your responsibility anyway. These are more reliable than stdbuf because they touch the tool's own buffer.
4. Trust stdbuf only for tools that use stdio. It works on classic filters like cut, tr and grep. It does not work for Python, Perl, Ruby, Go or dd, and it is unnecessary for tee.
5. If you cannot change the source and it has no flag, fake a terminal for it. This article's first measurement is already the proof: the same binary gave 18,018 bytes on a pty and zero on a pipe. script -q /dev/null <command> or expect's unbuffer attaches a pseudo-terminal, libc sees isatty as true, and the stream switches to line buffering. For third-party binaries you cannot open, this is usually the only lever you have.
6. Remember that turning the buffer off has a price. I wrote 10,485,760 bytes to /dev/null and timed it (five runs, median): 0.05 s with the default full buffering, 0.41 s with stdbuf -o L, 0.42 s with stdbuf -o 0. Line buffering is roughly eight times more expensive here. Do not carry that number into production as-is: it was measured against a /dev/null with no reader, and the figures change behind a real consumer. What does transfer is the conclusion — line buffering is not free, so apply it where you watch output live, not everywhere.
7. Do not use process.exit() in Node. Set process.exitCode and let the process exit naturally. The loss recorded here was 99.4%.
8. Do not count on the signal being gentle. SIGTERM, SIGINT, SIGHUP, SIGKILL — all four killed the process without flushing. The only thing that saves the data is the program catching the signal and exiting under its own control.

One last diagnostic habit: when you see a process not talking, question the pipeline before
you question the process. I got the same prod2 binary to produce 18,018 bytes and zero
bytes within the same minute, without changing it. The program was innocent; it had done the
same thing both times. The only thing that differed was who its words were going to.

Buffering is a well-meant invention: a few large writes are cheaper than many small ones.
The price of that good intention is not that good news gets held back — it is that bad news
does. The last twenty lines explaining why a job crashed may still be sitting in a buffer at
the exact moment of the crash.

Official Sources

• Apple Libc — stdio/FreeBSD/makebuf.c — the buffer size coming from st_blksize, the BUFSIZ fallback, the isatty line-buffering decision and the TTYBUFSIZE cap are all in this file.
• Python — Command line and environment (-u, PYTHONUNBUFFERED) — documents that -u makes stdout/stderr unbuffered and that since 3.7 this covers the text layer too.
• Node.js — process and the process I/O note — the table of synchronous/asynchronous writes per destination, and the process.exit() warning.
• Go — the bufio package — documents that NewWriter returns a buffer of "the default size"; the numeric value (defaultBufSize = 4096) is in the package source.