How to use this file. It is the Dev.to-ready copy of the post that is already live at
https://flintapi.ai/blog/model-markup-leaks-dsml-grammar-deadlock. published: false keeps it a draft
until someone flips it. canonical_url points at our own copy so search engines credit the original.
Publish via the Dev.to editor (paste body below the frontmatter) or via the API if a key is configured.

Both of these bugs look like "the model output is broken." Neither is. In both cases the model did something legal, and the layer that was supposed to interpret it disagreed.

One leaks markup into the visible answer. The other turns a legal token into an HTTP 500. They share a root: the boundary between reasoning, content, and tool calls is a token, and every layer that touches it assumes a different vocabulary.

Two public reports cover this: OmniRoute #13930 (DeepSeek V4 native DSML markup leaking into message content) and vLLM #61574 (structured-output requests terminating with grammar rejected tokens).

Case 1 — markup that arrives in content

DeepSeek V4 emits native markup around tool invocations, using fullwidth vertical bars (U+FF5C) rather than ASCII |:

<||DSML|| calls> <||DSML|| invoke name="Read"> <||DSML|| parameter name="path">docs/parser.md</||DSML|| parameter> 

A parser that only knows <tool> and <tool_call> matches nothing, so the envelope is treated as ordinary text and streamed to the user verbatim. That is the OmniRoute report.

The detail I find most useful: | and | are visually identical in most terminals and editors. The bytes are ef bd 9c, not 7c. So debug the bytes, not the log.

Minimal repro

curl -sN https://<your-endpoint>/v1/chat/completions \ -H 'content-type: application/json' \ -d '{"model":"deepseek-v4-flash","stream":true, "messages":[{"role":"user","content":"Read docs/parser.md and summarise it."}], "tools":[{"type":"function","function":{"name":"Read", "parameters":{"type":"object","required":["path"], "properties":{"path":{"type":"string"}}}}}]}' \ | tee raw.sse # is the leak a dialect gap or a real text answer? grep -c $'\xef\xbd\x9c' raw.sse # >0 => DSML bytes present grep -o '"content":"[^"]\{0,80\}' raw.sse | head 

Then re-run the same conversation with the tools array removed.

  • Markers vanish with no tools registered → the tags came from your prompt template or an upstream transform, not from the model's tool-call channel.
  • Markers still appear → it is the parser's vocabulary, which is the OmniRoute case.

Three candidate causes, and how to tell them apart

Candidate Discriminator
Dialect gap — parser only knows <tool> / <tool_call> Hexdump shows ef bd 9c; the tag never appears in the tool_calls field either
Dropped envelope on long context — model skips <invoke> and emits bare <parameter> blocks Same prompt at ~5k vs ~30k input tokens; the failure appears only on the long one
Demux ordering — content is assembled from the byte stream before tool-call extraction runs A tool_calls object arrives and the text still contains the envelope (duplication, not misclassification)

Those three need different fixes: a parser patch, schema-based matching on nameless blocks, and extraction before any content delta is emitted.

Case 2 — 649 five-hundreds in eight hours

The vLLM report is the cleanest write-up of this class I have seen. A GLM-5.3 deployment logged 649 grammar rejected tokens terminations in 8 hours, and 625 of them ended on <tool_call> (token 154843). Non-streaming requests terminated this way return HTTP 500.

The sequence: a pure JSON-schema response_format; an unconstrained bitmask row while reasoning; the model ending reasoning implicitly with <tool_call> instead of </think>; that token fed to a grammar that cannot start with it; Failed to advance FSM → 500.

Minimal repro

curl -s -o /dev/null -w '%{http_code}\n' https://<your-endpoint>/v1/chat/completions \ -H 'content-type: application/json' \ -d '{"model":"glm-5.3","temperature":0,"max_tokens":96, "messages":[ {"role":"user","content":"Read docs/parser.md, then summarise it as JSON."}, {"role":"assistant","content":"","tool_calls":[{"id":"c1","type":"function", "function":{"name":"Read","arguments":"{\"path\":\"docs/parser.md\"}"}}]}, {"role":"tool","tool_call_id":"c1", "content":"GLM wraps calls in <tool_call>...<arg_key>k</arg_key><arg_value>v</arg_value>"}], "response_format":{"type":"json_schema","json_schema":{"name":"s","schema":{ "type":"object","properties":{"summary":{"type":"string"}}, "required":["summary"],"additionalProperties":false}}}}' 

A tool result that quotes the model's own tags is what makes it sample <tool_call> mid-reasoning. Plain-worded tool results usually do not reproduce it.

Three candidate causes, and how to tell them apart

Candidate Discriminator
Unconstrained reasoning row — implicit end sampled where the grammar cannot start with it Engine log has one Failed to advance FSM ... tokens 154843 per 500; 200 comes back when tool_choice:"auto" or a tool is required
Template injects the token — chat template puts <tool_call> into the prompt before generation Render the template and inspect it; the literal token appears before sampling starts, on every seed
Backend difference — xgrammar vs a different structured-output backend Same seeded request against each backend; only one returns 500

The first is fixed upstream by masking implicit reasoning-end tokens in unconstrained rows. The other two are configuration problems on your side, and no upstream patch will find them.

What we changed on our side

We run a gateway for these models, so both bugs land on us first. Four rules came out of it.

Count latency at the first frame, not the first content delta. On reasoning models the gap is large: one measured request had a first token at 2.13s and a total of 7.22s. Timing to first content measures the model's thinking, not your network.

Stream through, never buffer and replay. We measured our own gateway at a first-token-to-total ratio of 0.9977 when it buffered the upstream response: first token and completion arrived together, which makes every latency number a lie. After per-frame forwarding the ratio is 0.389, with about 5.3 ms of overhead.

Separate reasoning by field, never by regex over content. Five field spellings are stripped; a normal answer is byte-identical afterwards. A turn that produced only reasoning returns a fixed fallback string plus a status field, instead of a 200 with empty content — the failure mode that makes this class invisible.

Report tokens you did not receive as null, not zero. If the upstream omits reasoning_tokens, the field is null. A fabricated zero would make an unmeasured bill look measured.

None of this is exotic. It is just the unglamorous part: pick one vocabulary for the boundary, enforce it at the edge, and make the unmeasurable case look different from the zero case.

If you are chasing either bug, hexdump the frame before you read the logs.