If you have ever built an app that talks to more than one AI model, you know the pain. OpenAI has its SDK and key. Anthropic has another. Google has a third. Each one has different request shapes, streaming formats, error codes, rate limits and billing dashboards. Before you have shipped a single feature, you are maintaining three integrations and rotating three sets of secrets.
The problem with juggling providers
- Multiple SDKs to install, upgrade and keep compatible
- Multiple API keys to store, rotate and monitor
- Different response formats that leak into your business logic
- Separate invoices, quotas and outages to track
One OpenAI-compatible endpoint
The idea is simple: put a single OpenAI-compatible endpoint in front of many models. Your code keeps using the familiar openai client. You change only the base_url and the model string, and the same request works across providers.
from openai import OpenAI client = OpenAI( api_key="YOUR_APIMODELS_KEY", base_url="https://api.apimodels.app/v1", ) for model in ["gpt-4o", "claude-3-5-sonnet"]: resp = client.chat.completions.create( model=model, messages=[{"role": "user", "content": "Explain REST in one sentence."}], ) print(model, "->", resp.choices[0].message.content) Note: check the docs for exact model names, since they can change over time.
Tips for comparing cost and latency
- Use the same prompt set. Run 20-50 real prompts from your app against every model, not just a toy example.
- Measure latency yourself. Record time to first token and total time. Averages hide tail latency, so look at the p95 too.
- Track tokens, not just price per call. Different models use different amounts of output for the same task, so compare cost per successful result.
- Score quality. Cheap and fast does not help if the answer is wrong. Even a simple pass/fail check helps.
- Add a fallback. If one model is slow or down, switch the
modelstring and retry. That is a one-line change with a single endpoint.
Wrapping up
Once switching models is a one-line change, experimenting gets cheap, and you stop being locked into whichever provider you picked first.
I built APIMODELS to make this one key, one endpoint.



