jq — Trivia & Interesting Facts¶
Surprising, historical, and little-known facts about jq, the command-line JSON processor.
jq is essentially a programming language disguised as a CLI tool¶
jq has variables, functions, conditionals, recursion, try/catch, string interpolation, reduce, and user-defined functions. It is Turing-complete. People have implemented Conway's Game of Life, Brainfuck interpreters, and Mandelbrot set renderers in jq. Most users only ever use .field and | select().
Stephen Dolan created jq as a one-person project¶
Stephen Dolan, a computer science researcher at the University of Cambridge, created jq in 2012. He designed the entire language, wrote the C implementation, and created the manual. The language design shows strong influence from Haskell and Icon (a language with generator-based evaluation), reflecting Dolan's academic background.
The dot (.) is jq's identity operator¶
The seemingly simple . in jq is actually an identity filter that passes its input unchanged. This functional programming concept makes jq's pipeline model work: .foo | .bar chains two filters, each transforming the stream. The entire language is built on filter composition, which is why jq '.' is the standard way to pretty-print JSON.
jq's streaming parser can handle files larger than RAM¶
The --stream flag switches jq into a streaming mode that processes JSON token-by-token instead of loading the entire document into memory. A 50 GB JSON file can be processed on a machine with 4 GB of RAM. This is rarely taught but invaluable for processing API dumps and log archives.
The @base64 and @uri formatters are hidden gems¶
jq has built-in format strings: @base64, @base64d, @uri, @csv, @tsv, @html, and @json. These handle encoding/decoding that would otherwise require piping through external tools. jq -r '.token | @base64d' decodes base64 inline without needing base64 -d.
null propagation makes jq surprisingly forgiving¶
Accessing a field that does not exist returns null instead of an error. Chaining .foo.bar.baz on an object without .foo just returns null. This null propagation (similar to optional chaining in JavaScript) makes jq scripts resilient to missing fields but can also hide bugs when you misspell a field name.
jq has had a single major version for over a decade¶
jq 1.0 was released in 2012, and jq 1.7 (released in 2023) was the first major release in over 5 years. Despite this slow release cadence, jq became a de facto standard because nothing else filled its niche. The project nearly stalled until a community effort revived active development in 2023.
gojq, xq, and yq prove jq's language design was right¶
jq's filter language has been reimplemented multiple times: gojq (Go, by itchyny), xq (Python, wraps jq for XML), yq (Go/Python, for YAML), and jaq (Rust). The fact that multiple independent projects chose to replicate jq's syntax rather than invent new syntax validates Dolan's original language design.
The ? operator suppresses errors silently¶
Adding ? after an expression (like .foo?) suppresses any errors instead of crashing. .[] | .name? will skip array elements that are not objects instead of failing. This operator is powerful but dangerous — it can mask data quality issues in production pipelines by silently dropping malformed records.
jq can generate JSON, not just transform it¶
jq is often used purely for reading JSON, but it can construct JSON from scratch: jq -n '{name: "test", count: 42}' generates a JSON object without any input. The -n (null input) flag is the key. Combined with --arg for injecting shell variables, jq becomes a safe JSON generator that handles escaping correctly.
The reduce function exists but almost nobody uses it¶
jq has reduce .[] as $item (init; update), a full reduce/fold operation. It can compute running totals, build objects from arrays, and implement any aggregation. Most jq users reach for external tools or awkward workarounds because reduce's syntax is unfamiliar, even though it is documented in the manual.