Originally published on tamiz.pro.

The past two years of AI coding agents have been defined by a paradox. The models are commoditizing — Claude, GPT, and open-weight models increasingly converge on the same chat-completion core — but the agents aren't. Claude Code, OpenAI Codex, Cursor, and a dozen open-source clones each ship their own CLI, their own session model, their own prompt templates, their own MCP client, and, most painfully, their own provider contract. The result is lock-in not to a model, but to a vendor's implementation of an agent.

That's changing. In 2026, the winning pattern for platform teams is a BYOK (Bring Your Own Key) control plane: a thin, self-hosted layer that owns the provider keys, the routing, the cost governance, and the session state, while the agents themselves become interchangeable executors behind a unified interface. You bring the keys; the control plane handles the rest.

This is a deep-dive into designing that control plane. We'll walk through the architecture, the provider abstraction, capability-aware routing, session normalization, secret handling, fallback strategies, and observability — with runnable TypeScript you can adapt to your own stack.

  • 1. Why the Agent Layer Is the New Lock-In
  • 2. What a BYOK Control Plane Actually Is
  • 3. Architecture Overview
  • 4. The Provider Abstraction
  • 5. Capability-Aware Routing
  • 6. Session and Context Normalization
  • 7. Cost, Quota, and Rate-Limit Governance
  • 8. Secret Handling and Key Rotation
  • 9. Serverless Local Models as a First-Class Provider
  • 10. Fallback, Retries, and Degraded Modes
  • 11. Observability and Tracing
  • 12. Production Best Practices
  • 13. Conclusion
  • Frequently Asked Questions

1. Why the Agent Layer Is the New Lock-In

In 2023, the lock-in problem was simple: you couldn't swap models without rewriting prompts. That's largely solved. Anthropic and OpenAI both expose OpenAI-compatible endpoints, and open-weight models run on the same protocol. A chat.completions call is a chat.completions call.

But the agent layer is a different story. An agent isn't a model call — it's a system: a prompt template, a tool schema, a loop that calls the model, parses tool calls, executes them, and feeds results back. Each vendor implements that loop differently, with different conventions for:

  • Session format — how conversation history is stored and rehydrated.
  • Tool-calling protocol — the JSON schema for tool definitions and the parsing of tool_calls.
  • Prompt caching — how context is marked for cache reuse, and how cache invalidation works.
  • Configuration — ~/.claude/, .codex/, environment variables, and CI hooks.
  • Hooks and lifecycle — pre-tool, post-tool, and error hooks, each with their own contract.

Switch from Claude Code to Codex and you're not just changing a model — you're rewriting your CI pipeline, your hooks, your prompt cache, your session recovery, and your team's muscle memory. That's the lock-in.

The BYOK control plane inverts the dependency. Instead of your workflow depending on a specific agent, the agent depends on a generic provider contract. Your CI calls the control plane; the control plane decides which provider to hit, with whose keys, under what budget, and with what fallback. Swap providers and your workflow doesn't change.

2. What a BYOK Control Plane Actually Is

A BYOK control plane is a self-hosted service (or embedded library) that sits between your developers/CI and the actual model providers. It exposes a single unified agent API and translates requests to provider-specific implementations. "BYOK" means the user supplies their own API keys, which the control plane stores securely and uses on their behalf — never exposing them to the developer's machine or to CI logs.

Why build one? Four reasons:

Reason What it buys you
Portability Swap Claude for Codex or a local model without touching workflows
Governance Centralized cost tracking, quotas, and audit trails
Security Keys never live in developer configs, .env files, or CI artifacts
Intelligence Capability-aware routing, prompt caching, and automatic fallback

The control plane isn't an agent. It doesn't write code or run tools. It's a dispatcher and governor: it knows which providers exist, what each can do, what each costs, and how much budget each team has left. The actual agent logic (the loop, the tools, the file edits) stays in the client — the CLI, the IDE extension, or the CI job.

3. Architecture Overview

Here's the high-level topology:

+-------------------+ +--------------------------+ +----------------------+ | Developer / CI | | Control Plane | | Providers | | | | | | | | - Agent CLI | --> | - Auth & Key Vault | --> | - Claude (Anthropic) | | - Agent SDK | | - Routing Engine | | - Codex (OpenAI) | | - MCP Client | | - Session Store | | - Local / Serverless | | | | - Cost Governor | | - Open-weight models | | | | - Circuit Breaker | | | +-------------------+ +--------------------------+ +----------------------+ 

The control plane is composed of five core components:

  1. Auth & Key Vault — authenticates the client, retrieves the user's provider keys, and injects them into provider calls without exposing them to the client.
  2. Routing Engine — selects a provider and model based on the request's requirements and the provider's capabilities.
  3. Session Store — normalizes conversation state across providers so a session can be rehydrated by any agent.
  4. Cost Governor — enforces budgets, quotas, and rate limits before a call is dispatched.
  5. Circuit Breaker — tracks provider health and routes around failing providers.

Everything is observable: every request, every provider call, every token, every dollar is traced and attributed.

4. The Provider Abstraction

The heart of the control plane is a single interface. Every provider — Anthropic, OpenAI, local, serverless — implements it. This is the contract that makes providers interchangeable.

// types.ts export type ProviderId = "anthropic" | "openai" | "local" | "serverless"; export interface FileContext { path: string; content: string; language: string; } export interface ToolSpec { name: string; description: string; parameters: Record<string, unknown>; } export interface AgentRequest { sessionId: string; prompt: string; files?: FileContext[]; tools?: ToolSpec[]; model?: string; maxTokens?: number; temperature?: number; metadata?: Record<string, unknown>; } export interface TokenUsage { inputTokens: number; outputTokens: number; cacheReadTokens?: number; cacheWriteTokens?: number; } export interface AgentResponse { sessionId: string; provider: ProviderId; model: string; content: string; toolCalls?: ToolCall[]; usage: TokenUsage; latencyMs: number; raw: unknown; } export interface AgentProvider { readonly id: ProviderId; readonly capabilities: ProviderCapabilities; execute(req: AgentRequest, ctx: ProviderContext): Promise<AgentResponse>; stream?(req: AgentRequest, ctx: ProviderContext): AsyncIterable<AgentChunk>; } 

The ProviderContext carries the decrypted API key, the request ID, and the trace span — everything a provider needs to make the call without knowing why it's being called.

export interface ProviderContext { apiKey: string; requestId: string; traceParent?: string; signal?: AbortSignal; } 

Each provider implementation is thin. It translates an AgentRequest into the provider's native API shape and maps the response back into the unified AgentResponse. For example, an Anthropic implementation maps max_tokens, system, and tools to Anthropic's API, while an OpenAI implementation maps to chat.completions.

The key insight: the interface is the product. Once you have this contract, adding a new provider is a small, well-defined task — implement execute, declare capabilities, register it in the registry. It's the same pattern as a database driver or a cloud provider SDK.

5. Capability-Aware Routing

Not all providers are equal. A local Llama 3.3 model might be free but have a 128K context window and no prompt caching. Claude Opus 4.5 might be expensive but excellent at complex refactors. Codex might be best for multi-file edits but lack long-context support. The routing engine needs to know each provider's capabilities to make a smart choice.

export interface ProviderCapabilities { maxContextTokens: number; supportsTools: boolean; supportsStreaming: boolean; supportsImages: boolean; supportsPromptCaching: boolean; costPerMillionInput: number; costPerMillionOutput: number; latencyP50Ms: number; reliabilityScore: number; // 0-1, updated by circuit breaker } 

Routing is a two-phase process: filter, then score.

Phase 1 — Filter by hard requirements. A provider is eligible only if it satisfies the non-negotiable constraints of the request:

function isEligible(p: ProviderCapabilities, req: AgentRequest): boolean { const needsTools = req.tools && req.tools.length > 0; const contextTokens = estimateContextTokens(req); return ( (!needsTools || p.supportsTools) && p.maxContextTokens >= contextTokens && (!req.metadata?.requiresImages || p.supportsImages) ); } 

Phase 2 — Score by policy. Among eligible providers, compute a weighted score based on cost, latency, and capability fit. The weights come from a routing policy, which can be per-team, per-project, or per-request.

export interface RoutingPolicy { weightCost: number; weightLatency: number; weightReliability: number; weightCapabilityFit: number; maxCostPerRequest?: number; preferProvider?: ProviderId; } function scoreProvider( p: ProviderCapabilities, req: AgentRequest, policy: RoutingPolicy ): number { const estTokens = estimateContextTokens(req); const estCost = (estTokens / 1_000_000) * p.costPerMillionInput + (req.maxTokens ?? 4096) / 1_000_000 * p.costPerMillionOutput; const costScore = 1 / (1 + estCost * 100); // normalize const latencyScore = 1 / (1 + p.latencyP50Ms / 1000); const reliabilityScore = p.reliabilityScore; const fitScore = p.supportsPromptCaching ? 1.2 : 1.0; return ( policy.weightCost * costScore + policy.weightLatency * latencyScore + policy.weightReliability * reliabilityScore + policy.weightCapabilityFit * fitScore ); } 

The engine then picks the highest-scoring provider, optionally respecting a preferProvider pin for testing or compliance. The result is that a cheap, fast local model wins for simple edits, while a premium model wins for complex, multi-file refactors — automatically, with no workflow changes.

6. Session and Context Normalization

The hardest part of cross-provider portability isn't the API — it's the session state. Claude Code stores sessions as a directory of JSON files with a specific prompt-cache layout. Codex uses a SQLite database. A local agent might use a simple transcript. If your control plane stores sessions in a vendor-specific format, you're back to lock-in.

The solution is a normalized session model that all providers can hydrate from and serialize to.

export interface SessionMessage { role: "system" | "user" | "assistant" | "tool"; content: string; toolCalls?: ToolCall[]; toolResult?: ToolResult; timestamp: number; provider?: ProviderId; // which provider produced this message } export interface SessionState { sessionId: string; projectPath: string; messages: SessionMessage[]; fileSnapshots: Map<string, string>; // path -> content hash createdAt: number; updatedAt: number; metadata: Record<string, unknown>; } 

When a request comes in, the control plane loads the SessionState, converts it to the target provider's expected format (e.g., Anthropic's system + messages array, or OpenAI's messages array), and executes. When the response comes back, it converts the provider's output back into SessionMessage entries and persists them.

This normalization is also where prompt caching gets handled. Different providers have different cache mechanisms — Anthropic uses cache_control blocks, OpenAI uses automatic prefix caching. The control plane decides which messages are cacheable and marks them appropriately, so you get cache hits even when you switch providers.

function toAnthropicMessages(session: SessionState): AnthropicMessage[] { return session.messages.map((m) => ({ role: m.role, content: m.content, ...(m.toolCalls && { tool_calls: m.toolCalls }), ...(isCacheable(m) && { cache_control: { type: "ephemeral" } }), })); } 

The key benefit: a developer can start a session with Claude, hit a rate limit, and the control plane can transparently rehydrate the session with Codex — without losing context, tool results, or file state.

7. Cost, Quota, and Rate-Limit Governance

BYOK means the user pays, but the platform still needs to govern how they pay. The cost governor is a pre-flight gate: before a request is dispatched, it checks whether the caller has budget, whether the provider's rate limit is respected, and whether the request fits within the team's quota.

export interface BudgetLedger { teamId: string; userId: string; projectId: string; monthlyBudgetUsd: number; spentUsd: number; requestsThisMinute: number; tokensThisHour: number; } export class CostGovernor { constructor(private store: LedgerStore) {} async authorize(req: AgentRequest, provider: ProviderCapabilities): Promise<Decision> { const ledger = await this.store.getLedger(req.metadata?.teamId, req.metadata?.userId); const estCost = this.estimateCost(req, provider); if (ledger.spentUsd + estCost > ledger.monthlyBudgetUsd) { return { allowed: false, reason: "BUDGET_EXCEEDED" }; } if (ledger.requestsThisMinute >= 60) { return { allowed: false, reason: "RATE_LIMITED" }; } return { allowed: true, estimatedCostUsd: estCost }; } async recordUsage(req: AgentRequest, usage: TokenUsage, provider: ProviderCapabilities) { const cost = this.calculateCost(usage, provider); await this.store.incrementSpend(req.metadata?.teamId, req.metadata?.userId, cost); } } 

This is where the control plane earns its keep in an enterprise: centralized cost attribution. Every request is tagged with team, user, project, and provider, so finance gets a clean ledger instead of a pile of individual provider invoices. You can enforce "no more than $500/month per developer" or "local models first for CI" — and the governor enforces it automatically.

8. Secret Handling and Key Rotation

The "BYOK" part is the security-critical piece. Keys must never appear in:

  • Developer .env files or shell history.
  • CI logs, build artifacts, or error traces.
  • The agent's own config files.

Instead, the control plane holds keys in a secret store — Vault, AWS Secrets Manager, or a KMS-backed encrypted database — and injects them per-request into the ProviderContext.

export interface SecretStore { getKey(userId: string, provider: ProviderId): Promise<string>; rotateKey(userId: string, provider: ProviderId, newKey: string): Promise<void>; revokeKey(userId: string, provider: ProviderId): Promise<void>; } export class KeyVault { constructor(private secrets: SecretStore) {} async resolve(userId: string, provider: ProviderId): Promise<string> { // Decrypt on read, never log, never cache in plaintext const encrypted = await this.secrets.getKey(userId, provider); return decrypt(encrypted); } } 

Key rotation is a first-class concern. When a user rotates their Anthropic key, the control plane needs to update its store and invalidate any cached sessions that depend on the old key. The pattern is:

  1. User submits a new key via the control plane's admin API.
  2. The control plane encrypts it and writes it to the secret store.
  3. The old key is marked deprecated but retained for 24 hours to allow in-flight requests to complete.
  4. After the grace period, the old key is deleted.

This gives you a clean rotation path without breaking active sessions.

9. Serverless Local Models as a First-Class Provider

The most interesting 2026 trend is the rise of serverless local models: open-weight models (Qwen, DeepSeek, Llama) running on ephemeral GPU workers via Modal, RunPod, or Cloudflare Workers AI, or on a developer's own machine via Ollama. These aren't just "cheaper" — they change the economics of the control plane.

A local provider has a different cost profile:

Metric Claude Opus Serverless Local
Cost per 1M input tokens ~$15 ~$0 (compute already paid)
Cost per 1M output tokens ~$75 ~$0
Context window 200K 128K–256K
Prompt caching Yes Sometimes
Latency P50 ~2s ~4–8s
Privacy Data leaves VPC Data stays in VPC

The control plane treats "local" as just another provider, but with two special rules:

  1. Privacy-first routing: if the request's metadata includes sensitive: true (e.g., proprietary code), the router must prefer the local provider, regardless of cost.
  2. Capability ceiling: local models may not support complex tool-calling or long-context reasoning, so the router only sends tasks within their capability envelope.
export class LocalProvider implements AgentProvider { readonly id = "local" as const; readonly capabilities: ProviderCapabilities = { maxContextTokens: 131_072, supportsTools: true, supportsStreaming: true, supportsImages: false, supportsPromptCaching: false, costPerMillionInput: 0, costPerMillionOutput: 0, latencyP50Ms: 6000, reliabilityScore: 0.9, }; constructor(private endpoint: string) {} async execute(req: AgentRequest, ctx: ProviderContext): Promise<AgentResponse> { // Call Ollama / vLLM / Modal endpoint with the unified request const res = await fetch(`${this.endpoint}/v1/chat/completions`, { method: "POST", headers: { Authorization: `Bearer ${ctx.apiKey}` }, body: JSON.stringify(toOpenAIShape(req)), }); return fromOpenAIShape(await res.json(), this.id); } } 

The strategic value: for CI workloads, code review bots, and sensitive-code tasks, the local provider becomes the default, and premium providers become the exception — a dramatic cost reduction with zero workflow changes.

10. Fallback, Retries, and Degraded Modes

Providers fail. Rate limits hit. Models get deprecated. A control plane without resilience is just a single point of failure. The fallback chain is the mechanism that keeps the system alive.

export class FallbackChain { constructor( private providers: AgentProvider[], private router: RoutingEngine, private breaker: CircuitBreaker ) {} async execute(req: AgentRequest, ctx: ProviderContext): Promise<AgentResponse> { const candidates = this.router.rank(req, this.providers); let lastError: Error | undefined; for (const provider of candidates) { if (this.breaker.isOpen(provider.id)) ...
Source:https://dev.to/tamizuddin/the-end-of-per-provider-agent-lock-in-designing-a-byok-coding-agent-control-plane-for-claude-4hbk