Some applied LLM interviews skip "explain temperature" and start from a production symptom: answers are cut off, the bill doubled, behavior changed overnight. The answer shape that works: name the likely causes, say how you'd tell them apart, then say what you'd change.

Six of the 15 scenarios from the full guide, condensed:

1. A fact in the middle of a long prompt gets ignored

Name the effect: the "Lost in the Middle" study (Liu et al., 2023) found performance is often highest when relevant information sits at the beginning or end of the context and degrades when it's in the middle. Reproduce it by moving the fact to the edges. The fix is less context: fewer, better passages, the most relevant ones at the edges, and the question restated after long documents.

2. Answers are cut off, but only in production

Read the stop reason before guessing. OpenAI's finish_reason is length when the token limit was reached, and Anthropic's stop_reason is max_tokens. "Only in production" points to longer real inputs, a lower limit in the prod config, or a stop sequence that appears in real text. With reasoning models the cap includes reasoning tokens, so it can be hit before any visible output.

3. About 2% of structured responses fail to parse

Triage before fixing. Sort failures into buckets: truncated JSON (check the stop reason), valid JSON with the wrong shape, extra prose around the JSON, or a refusal. Each has a different fix: a bigger limit, schema-constrained output or one repair retry, or a separate refusal path. Count the buckets before and after each change so you know which fix moved the 2%.

4. The bill doubled, and traffic didn't

Break cost into requests × tokens per request × price, by endpoint and model; one of those moved. Usual suspects: a retry loop, an agent calling tools more often, a prompt or context that grew, chat history no longer trimmed, traffic routed to a pricier model, or prompt-cache reads that fell. Then add budgets: alerts on tokens per request and a cap on agent steps.

5. The prompt cache hit rate dropped to near zero after a deploy

Caching works on an exact prefix. Anthropic's docs say cache hits require 100 percent identical prompt segments, built in the order tools, system, messages, and a change at one level invalidates everything after it. Diff the prompt: a timestamp or user name moved into the system prompt, or tools reordered, is the classic cause. Also check timing, because the default cache lifetime is 5 minutes, so a quiet endpoint can miss even with an unchanged prefix.

6. Behavior changed overnight, and nothing was deployed

List what isn't in your deploy: the model behind the ID you call, retrieved data, upstream tools and the traffic itself. Check whether you call an alias or a pinned snapshot, rerun your eval set and compare with the last stored run. Prevention: pin model versions, run evals on a schedule, and log the model version returned with each response.

The pattern

Every scenario rewards the same habit: find a signal that separates the causes (stop reason, chunk timestamp, cache-read tokens, per-request cost) before changing anything. If you can say what you'd log so the next incident takes minutes, you're answering at the mid-to-senior level.

All 15 scenarios, with levels, follow-ups and links to the vendor docs: LLM Interview Questions (2026) on StackChief.

This post was written with the help of AI, as a summary of the StackChief article above.