You use grep, wc, jq and tree every day without thinking about them. They look simple: search for a pattern, count lines, filter JSON, print a directory. Writing a rough version of each takes an afternoon.
Writing a version whose output is exactly the same as the real tool is a different exercise. The moment you compare bytes, you find rules you never noticed: how wide wc makes its columns, what + means in a basic regular expression, how jq prints 1e2, how a nested .gitignore overrides its parent.
unix-tools is my learning project built on that idea. It contains four small Go binaries, and a test suite that runs each one next to the real tool on the same input and requires byte-identical stdout and the same exit status.
What it does
The repository builds four separate binaries:
| binary | does | flags |
|---|---|---|
cmd/grep | GNU-grep-compatible search | -i -v -n -r -c, plus -E -F -q -H -e PAT |
cmd/wc | line, word, char and byte counts | -l -w -m -c |
cmd/jqlite | a subset of jq | -c -r -n |
cmd/tree | directory tree | -a -d -L N --gitignore --noreport |
The reference versions are GNU grep 3.12, GNU wc 9.x, jq 1.7.1 and tree 2.2.1.
jqlite understands a useful part of the jq language: paths such as .a.b, .[0], .[-1] and .[], the recursive .., the ? operator, pipes, ,, //, and, or, comparisons, arithmetic, array and object construction ({a, b: .x, (.k): .v}), and the builtins select map keys keys_unsorted length has type not add sort reverse tostring tonumber to_entries empty.
What I built from scratch, and what it teaches
Each tool hid its own lesson.
grep: basic regular expressions are not what you think. By default grep uses POSIX basic regular expressions. In that dialect + ? | ( ) { } are literal characters, and the operators are written \+ \| \( \). Go's regexp package uses RE2 syntax, where it is the other way round. So the tool has a translator from BRE to RE2. Matching grep also means getting the small output rules right: the filename prefix appears when there are several files or with -r, the leading ./ is dropped when -r has no operand, and the exit code is 0 for a match, 1 for no match and 2 for an error.
wc: the column width is computed. GNU wc does not use a fixed width. The width is the number of digits in the total size of the regular files. It is at least 7 when any input is a pipe, and 1 when a single number is printed for a single input. You only learn this when your output differs from the real one by a few spaces.
jqlite: a parser and a generator-style evaluator. jq's grammar has a precedence order: pipes bind looser than ,, which binds looser than //, then or, and, comparisons and arithmetic. I wrote a recursive-descent parser for it. The evaluator treats every filter as something that returns zero or more outputs, which is how .[] and select fit together naturally. Objects keep their key order. Numbers were the hardest part: jq 1.7.1 prints input literals in decNumber's canonical form (1e2 becomes 1E+2, 1.0 stays 1.0), while computed doubles use the shortest digits that round-trip (1e+17, 3.3333333333333335).
tree: Git's ignore rules. The --gitignore flag needs the real rules: **, negation with !, directory-only patterns, anchoring by a slash, and nested .gitignore files that override their parents. tree 2.x also has its own quirks: the root counts as a directory in the report, and the vertical bar is padded with two no-break spaces, not normal spaces.
How to use it
Prebuilt binaries for Linux, macOS and Windows (amd64 and arm64) are on the releases page of the repository. Each archive contains the four binaries, the README and the license, and checksums.txt has the SHA-256 sums.
With Go installed, you can build from source:
go install github.com/AbdeldjalilChougui/unix-tools/cmd/...@latest One warning from the README: the binaries are named grep, wc and tree. If $(go env GOPATH)/bin comes first on your PATH, they may shadow the system tools.
Or clone the repository and try them from a local bin/ folder:
go build -o bin/ ./cmd/... bin/grep -rn TODO . printf 'a b\nc\n' | bin/wc bin/jqlite -c '.items[] | select(.price > 10) | {name, price}' data.json bin/tree --gitignore -L 2 How the tests work
The interesting part of this project is the test suite:
go test -race ./... compare_test.go builds a fixture tree in a temporary directory: code, notes, logs, nested .gitignore files, UTF-8 text and a JSON document. Then it runs about 70 cases, with two checks:
- If the real tool is available (
grep,wc,jqortreeonPATH, or$JQ_BIN/$TREE_BIN), its output is the expectation. - In every case the output must also match a golden file in
testdata/golden/, recorded from the real tools. So the suite still means something on a machine where jq or tree is not installed.
The tools run with LC_ALL=C (C.UTF-8 for wc -m and tree). The output of grep -r is sorted before comparing, because real grep walks directories in filesystem order. There are also unit tests for the BRE translator, the gitignore rules, the argument parser and jqlite's evaluator and number formatting. A small script, scripts/tree-docker.sh, runs tree 2.2.1 from a Debian container for machines that do not have it.
Things to know
These are learning-sized subsets, not replacements for the real tools. The README lists the gaps clearly:
- grep: no context options (
-A/-B/-C), no-o,-l,-w,-x, no colour, and no backreferences, because RE2 has none. Character classes follow Go, so locale-specific collation differs.-rvisits files in sorted order, and binary detection is a simple NUL-byte check. - wc: no
-Lor--files0-from. It always decodes UTF-8 for-m, while GNU wc counts bytes in the C locale. - jqlite: no variables (
as $x),reduce/foreach, function definitions, string interpolation, slices, assignment operators,--argor-S. Results are fully materialised, with no lazy streams. Error messages are close to jq's but not identical. - tree: no
-I,-P, sorting options, sizes, permissions or JSON output. Symlinked directories are listed but not followed. Only.gitignorefiles are read, not.git/info/excludeor the global excludes file.
What I learned
The main lesson is about testing. "It works on my examples" and "it prints the same bytes as the reference on 70 cases" are far apart, and the distance between them is where the learning is. A differential test against a real implementation finds the rules you did not know existed, and it tells you exactly which byte is wrong.
The second lesson is that small tools carry a lot of accumulated decisions. A column width, a no-break space, an exit code: each has a reason, and scripts depend on them. Rebuilding a tool is a good way to read its specification closely.
Get the code
The source, the test fixtures and the golden files are on GitHub: https://github.com/AbdeldjalilChougui/unix-tools. It is MIT licensed. If you want a good exercise, pick one of the missing flags from the limitations list and make the comparison test pass.
Originally published at abdeldjalilchougui.biz.





