Originally published on my blog — read it there with the full context, code and updates.

A single self-hosted OpenAI-compatible gateway served every model tier: long-context reading, refactors, cheap background jobs. One key, one model picker, one dashboard where spend was visible in one place. Six independent workflow seats — repo indexing, CI triage, interactive refactors — sat behind it, each authenticating with its own subscription OAuth credential.

Then all six came back disabled inside one short detection window. None of them had been doing anything unusual. What tripped was a scheduled credential check, and what it found was a block already applied upstream.

The interesting part was not the block. It was the ten minutes I spent rotating credentials that were never wrong.

The error that sent me the wrong way

For the first ten minutes, every consumer of the gateway's error reported a credential failure — orchestrator, shell probe, CLI — because that is what the body said: 401 invalid_api_key. The upstream had actually returned a policy decision.

Three probes separated the layers, and they are worth keeping for any "the model stopped answering" ticket: confirm the gateway answers its own health endpoint; confirm a model from an unrelated provider succeeds on the same key; then loop the subscription models and compare the error codes. One model failing means a broken model. Every model on one provider failing identically, while other providers succeed on the same key, means the provider relationship is closed — and no key rotation moves that.

The real defect is the error contract, not the outage:

# Before: policy decisions disguised as auth failures if upstream.status == 403: return jsonify(error="invalid_api_key"), 401 # After: distinct classes, distinct client behaviour if "Terms of Service" in upstream.text or "disabled in this account" in upstream.text: return jsonify(error="upstream_policy_block", upstream_status=403, alert="credential-policy"), 502 return jsonify(error="invalid_api_key"), 401

A 401 that means "your key is wrong" and a 403 that means "the vendor closed this seat" must never collapse into the same response. Flattening them costs the responder the first ten minutes of every incident.

Why a shared token is not a shared key

An API key is a bearer credential: whoever holds it can use it. An OAuth token is bound to a client identity — client id, consent record, scopes. When a different process presents that token, the request keeps the claims and loses the behaviour: headers, TLS fingerprint, request cadence.

Two properties of the initial gateway topology made the pattern obvious to any detection system:

  • One egress. Every seat left through one container on a single public address.
  • One fingerprint. The gateway normalised headers and stripped client telemetry, so six workflow seats looked like one client at six times the volume.

Simultaneous failures across independent workflows are not six human reviews. They are one deterministic cluster rule firing. The vendor's terms had said it in advance: scripting the official CLI is acceptable, pooling seats or automating rotation across them is not.

Task dispatch instead of routing

The lesson is not "use fewer seats". It is that a subscription is a seat, not an API. The moment its token leaves the vendor's client, it becomes a shared credential.

So the gateway keeps serving models paid for by API key, and subscription work moved to task dispatch: the orchestrator runs isolated, single-task invocations of the official CLI bound to a dedicated workflow seat. No pooled credentials, no hijacked persistent session, no rotation. The token never leaves the vendor's binary.

orchestrator -> task dispatcher -> official CLI (owns the token) -> vendor | +- cwd, permission scope, timeout, tree-kill, no retries

Four dispatcher properties mattered more than any prompt:

  1. Explicit working directory — each task starts inside one repository, never a user home directory or a filesystem root, because an agent with edit rights inherits the whole shell.
  2. Closed stdin — interactive prompts in a headless run become errors instead of silent hangs. One forgotten -y otherwise blocks the parent forever.
  3. Process-group kill on timeout — killing the parent leaves grandchildren running; kill the group and return a timeout code, not a half-finished diff.
  4. No automatic retries — an inner agent loop times an outer retry policy is hundreds of upstream calls in minutes, the exact cadence that got the previous design flagged. 
proc = subprocess.Popen( argv, cwd=repo, stdin=subprocess.DEVNULL, # no interactive hangs  stdout=subprocess.PIPE, stderr=subprocess.PIPE, start_new_session=True, # own process group ) try: out, err = proc.communicate(timeout=task_timeout) except subprocess.TimeoutExpired: os.killpg(proc.pid, signal.SIGKILL) # kill the whole tree  raise

Context stopped travelling in prompts as well: the delegate reads repository instruction files and queries a self-hosted memory service over MCP, so a brief stays six lines instead of a transcript dump. Delegate writes are tagged status: draft and final archiving stays with the orchestrator.

The economics survived, because the workload splits

Nine turns in ten are routine orchestration — tool loops, retrieval, cron, log triage, short edits. Those run on a curated open-weight plan: $10/month carrying a $100 monthly allowance on the fast model, with per-model pools beside it. At $0.15 per million input tokens and $0.60 per million output, a 4k-in/500-out turn costs about $0.0009 — under a tenth of a cent.

The remaining tenth is heavy engineering: multi-file refactors, legacy archaeology, anything with a real build-and-test contract. That dispatches to an authorised CLI seat, where the vendor's client owns the credential and the allowance belongs to the subscription rather than to a pooled bucket.

Pay for frequency with API-key models, pay for depth with authorised seats, and never let the two share a token.

💬 Let's Compare Notes

If a credential is bound to a client, where should a platform team draw the execution boundary when metered APIs and client-bound CLI tools have to coexist: standardise on pure APIs and pay the premium, or run a dispatcher that governs official binaries? I have moved to the second, and I would like to hear from anyone who has run the first at team scale and made it survive an audit.

Two related posts from the same infrastructure work:

I write about self-hosted infrastructure, DevOps and the failures behind them at blog.hoelee.com.