Your JSON Mode Is Not a Schema: Four Structured-Output Myths and a Validation Gate

A free model answered my extraction prompt fast. The JSON parsed cleanly. My pipeline still crashed ten minutes later.

The ticket id came back as an integer. The hours field was the string "three". JSON mode did its job. My schema did nothing at all.

I keep relearning this lesson. So I built a gate that fails in one place instead of five.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Myth 1: "JSON mode means my schema is enforced"

This is the most expensive myth on the list. JSON mode buys you syntax. It does not buy you shape.

A model can return {"hours": "three"} and be perfectly valid JSON. Your parser is happy. Your database write is not.

So what actually enforces your contract? A validator. Not the model, not the SDK, not the vibe.

Corrected mental model: JSON mode is a formatting hint. Your schema is the contract. Only your validator can refuse delivery.

Myth 2: "Retrying the same prompt fixes a bad shape"

It sometimes works. That is exactly why people believe it.

But blind retries teach you nothing. You resend the same text, get a different sample, and move on. The failure class stays invisible.

Here is the better question. Does your retry include the validation error?

A repair retry sends back the exact complaint: hours: 'three' is not of type 'number'. That is a different request with different information. It converges far faster than a coin flip.

Corrected mental model: Retry with evidence, not with hope. Cap the repair budget at two and fail closed after that.

Myth 3: "A parser exception is my failure signal"

Most bad outputs never throw. They coerce.

Watch these three moves:

1. "3" becomes 3 inside your ORM.
2. A missing key becomes None inside your template.
3. An extra field gets silently dropped by your dataclass.

Nothing logs. Nothing alerts. The wrong value lands in the table and looks normal.

So no, a clean parse is not a pass. It is the absence of one specific failure mode.

Corrected mental model: Count failures per field, not per request. A pipeline at 97% request success can still be 60% correct on one field.

Myth 4: "Free endpoints are too flaky for structured work"

This one is backwards, and it hides a design mistake.

Structured extraction is the best fit for cheap model access. The task is narrow. The output is checkable. Failure is machine-detectable at zero cost.

Compare that with open-ended chat, where nobody knows if the answer was right.

So the flakiness is real, but it is measurable. That is a very different situation from unmeasurable.

Corrected mental model: Free access plus a hard validator beats expensive access plus trust.

The artifact: one gate, three outcomes

I run this against an OpenAI-compatible chat completions endpoint. Swap the URL and model name for whatever you use.

pip install jsonschema 

# schema_gate.py — one place where bad shapes die. import json, os, re, sys, urllib.request import jsonschema ENDPOINT = os.environ["MODEL_ENDPOINT"] # full URL, e.g. https://host/v1/chat/completions MODEL = os.environ["MODEL_NAME"] KEY = os.environ.get("MODEL_API_KEY", "none") SCHEMA = { "type": "object", "additionalProperties": False, "required": ["ticket_id", "severity", "hours"], "properties": { "ticket_id": {"type": "string", "pattern": "^TCK-[0-9]{4}$"}, "severity": {"enum": ["low", "medium", "high"]}, "hours": {"type": "number", "minimum": 0, "maximum": 40}, }, } FENCE = re.compile(r"^``` (?:json)?\s*|\s* ```$", re.MULTILINE) def call_model(prompt: str) -> str: body = json.dumps({ "model": MODEL, "temperature": 0, "messages": [{"role": "user", "content": prompt}], }).encode() req = urllib.request.Request( ENDPOINT, data=body, headers={"Content-Type": "application/json", "Authorization": f"Bearer {KEY}"}, ) with urllib.request.urlopen(req, timeout=60) as resp: return json.load(resp)["choices"][0]["message"]["content"] def strip_fence(text: str) -> str: return FENCE.sub("", text).strip() def errors_for(text: str) -> list[str]: try: payload = json.loads(strip_fence(text)) except json.JSONDecodeError as e: return [f"parse: {e.msg} at line {e.lineno}"] validator = jsonschema.Draft202012Validator(SCHEMA) return [f"{'/'.join(map(str, e.path)) or '<root>'}: {e.message}" for e in validator.iter_errors(payload)] PROMPT = """Extract exactly one ticket. Return JSON only, no prose. Fields: ticket_id (string like TCK-0001), severity (low|medium|high), hours (number between 0 and 40). Text: {text}""" def gate(text: str, budget: int = 2) -> dict: prompt = PROMPT.format(text=text) errs: list[str] = [] for attempt in range(budget + 1): raw = call_model(prompt) errs = errors_for(raw) if not errs: return {"ok": True, "payload": json.loads(strip_fence(raw)), "attempts": attempt + 1} print(json.dumps({"attempt": attempt + 1, "errors": errs}), file=sys.stderr) prompt = (PROMPT.format(text=text) + "\nYour previous answer failed validation:\n- " + "\n- ".join(errs) + "\nReturn corrected JSON only.") return {"ok": False, "errors": errs, "attempts": budget + 1} if __name__ == "__main__": result = gate(" ".join(sys.argv[1:])) print(json.dumps(result, indent=2)) sys.exit(0 if result["ok"] else 1) 

Run it against one messy sentence:

python schema_gate.py "Printer on floor 3 dead since Monday, roughly 2 hours of work" 

You get one of three outcomes. Accept, repaired, or rejected. Never a silent coercion.

Reading the failure log

Every rejection writes one JSON line to stderr. Stop guessing which field breaks.

python schema_gate.py "..." 2>> fails.jsonl jq -r '.errors[]' fails.jsonl | cut -d: -f1 | sort | uniq -c | sort -rn 

That count is your real metric. It tells you which field to tighten and which prompt sentence to rewrite.

Decision table: repair, degrade, or fail closed

Print that table and tape it to the review. Half the arguments disappear.

Where a free server actually helps

The gate is a small, stateless function. It holds no session and stores no state.

That makes it a decent fit for the free server option. I run the validator as a separate sidecar, not inside my main app. When it dies, extraction stops. Nothing corrupts.

Two limits I accept. A free server is not my latency-critical path. And I never assume any availability guarantee I have not been given in writing.

That is the honest shape of the trade. Cheap and checkable, not cheap and trusted.

Limitations and who should skip this

This workflow is not universal. Be blunt with yourself about the fit.

• If your output is free text, a schema gate buys you little. You need an evaluator.
• If your schema changes weekly, the repair loop will chase you. Freeze the contract first.
• If you need sub-second p99, add a fast local model or a queue. Do not blame the endpoint.
• If a wrong value is unrecoverable, no gate is enough. Add a human approval step.
• If you cannot log rejections, you cannot measure anything. Fix logging first.

And please do not ship this gate as your only test. It checks shape, not truth. A perfectly typed wrong number passes.

The short version

JSON mode formats. Your validator decides.

Retry with the error text appended. Cap the repairs. Count failures per field.

If you want a cheap place to practice this loop, try it against MonkeyCode's free model access. Then keep the gate and drop everything else. The validator is the part that survives the vendor.