I built BookDNA, an AI book recommender that finds books by trope, setting, and tone, and checks every pick against Open Library so nothing is made up. I’d already spent a lot of time on speed: I timed every stage, tested 10 variants, and cut the time to the first result from about 30 seconds to 3.
So I thought I knew where the time went. Then I added tracing with OpenTelemetry and SigNoz, and within minutes it showed me two things I’d missed.
What I set up
I used OpenTelemetry’s zero-code instrumentation for FastAPI and httpx, sent traces to SigNoz Cloud, and added a few custom spans by hand: one for each Open Library lookup, each Claude call, and each recommendation shelf.
Two decisions mattered most:
No prompts in my traces. Common LLM auto-instrumentation records prompts and responses by default. I only record the model, token counts, and the finish reason, never the text or my API key.
One trace per search. A single search makes three separate requests from the browser. I generate one trace ID per search in the frontend and send it on all three, so SigNoz shows the whole search as one timeline. (SigNoz shows a “Missing Span” at the top, because the browser never sends its own span. That’s expected.)
Finding 1: my homepage could take 26 seconds
The “For You” shelf on my homepage blends a reader’s recent searches into personal picks. I had never timed it. The trace showed the request took about 26 seconds, and one Claude call took 23 of them. It’s an older code path: it waits for 18 books before returning anything, while my search page already streams results as they’re ready.
Finding 2: a 13-second gap in every search
My first shelf arrives in about 5 seconds, which is what my speed work was about. But the trace showed a quiet gap of about 13 seconds before the other three shelves arrived. They all come from one coordinated Claude call (so the shelves don’t repeat each other), and the validation checks under each shelf were fast. The real wait was Claude writing about 24 books with reasons.
My timing logs measured the first shelf. The trace showed the whole experience.
What tripped me up
I kept a friction log while setting it up. A few highlights:
Keeping the ingestion key in a .env file isn’t covered in the docs. The instrumentation reads settings before my app loads its .env file, so I start the server with dotenv run.
On Windows, opentelemetry-instrument returns the prompt immediately while the server keeps running in the background. Confusing the first time.
The setup command wanted about 20 instrumentation packages. I only needed FastAPI and httpx, so I pinned those.
What’s next
Streaming the For You shelves one at a time, the way search already works, then comparing the before-and-after traces in SigNoz.
I built this with Claude Code, making the decisions and reviewing every change.
