Parsing untrusted XML in production has always been risky. Between entity expansion attacks, parser differential bugs, and memory vulnerabilities in legacy C libraries, handling incoming XML payloads can feel a little daunting.
libexpat has been the default workhorse for XML parsing since 1998. It powers everything from Python's standard library to system-level service brokers. However, maintaining and securing decades-old C code against arbitrary, hostile input is an ongoing struggle.
We built lexpat to tackle the security problem. By porting libexpat's XML 1.0 parsing semantics to Lean 4, backed by machine-checked proofs and a bounded C interface, we can mathematically prove it safe.
Why XML Still Matters
JSON won the public API war, but XML still powers the infrastructure where failures are most expensive:
Single Sign-On (SSO): SAML assertions are pure XML. A parser flaw or canonicalisation bug in an identity provider can lead directly to authentication bypass.
Fintech and Enterprise Integrations: Financial networks and banking systems (such as ISO 20022 and legacy SOAP services) process millions of untrusted XML documents daily.
Document and Configuration Pipelines: Office formats (DOCX, XLSX) and deployment manifests rely on deeply nested XML trees.
When these systems accept XML from external sources, they expose a large attack surface. Traditional mitigations usually involve defensive wrapping, strict entity disabling, and continuous fuzzing. We wanted something stronger: mathematical guarantees.
Why Lean 4 for Systems Software?
Lean 4 is widely recognised as an interactive theorem prover, but it is also an efficient, purely functional systems programming language. It compiles directly to C, handles memory through reference counting without a heavy runtime garbage collector, and allows developers to write code and prove properties about that code in the same workspace.
Rather than writing code in C and hoping a fuzzer catches every edge case, Lean lets us state the invariant we want, prove that our parser logic satisfies it, and compile down to fast, predictable binaries.
How lexpat Works
lexpat is not a line-by-line mirror of the entire legacy libexpat codebase. Instead, it isolates the core XML 1.0 parsing semantics needed to consume untrusted data safely:
A Machine-Checked UTF-8 Decoder
Text encoding is a frequent source of parser vulnerabilities. Invalid byte sequences, overlong encodings, and surrogate halves can desynchronise parsers or hide malicious strings. In lexpat, the UTF-8 decoder is mathematically verified against the specification. Valid sequences decode predictably; invalid sequences fail immediately.
A Bounded C Surface
To work with existing systems, lexpat exposes a C interface that downstream projects can link against. Unlike traditional C libraries that might attempt to process malformed or ambiguous input, lexpat's surface is strictly bounded. If a call falls outside the implemented and proved semantics, it fails closed. It does not guess, and it avoids undefined behaviour.
Protection Against Denial of Service
Attacks like the Billion Laughs rely on recursive entity expansion to exhaust server memory. By restricting how entity references are resolved and keeping state transitions explicit, lexpat prevents memory runaway by construction.
What It Is Not
It helps to be clear about what lexpat is not trying to be:
It is not an unconditional drop-in replacement: libexpat carries twenty-five years of historical quirks, obsolete build configurations, and permissive error handling. lexpat adopts its core parsing semantics for XML 1.0, rather than replicating legacy quirks that compromise security.
It does not require users to write Lean: Because Lean compiles down to C, applications written in Rust, Go, Python, or C can consume lexpat through standard foreign function interfaces (FFI).
Looking Ahead
lexpat serves as a practical demonstration of verified systems programming. By combining Lean's proof capabilities with standard compilation targets, it is possible to remove entire classes of parser exploits while keeping performance competitive.
Take a look at the implementation and proofs on GitHub:
GitHub Lexpat Repo
Lexpat website write up
Lexpat is open sourced, licensed under the Apache License, Version 2.0. Give the repo a star if you like it :)
