Panduan API Rate‑Limit: Membangun Local Proxy dengan Token Bucket

Banyak layanan API menerapkan rate limit untuk melindungi server dari overload. Ketika aplikasi Anda melebihi batas, respons 429 Too Many Requests muncul dan proses terhenti. Solusi klasik adalah retry dengan exponential backoff, tetapi pendekatan itu masih membiarkan burst request sampai ke upstream.

Pendekatan yang lebih baik: local proxy yang menampung request, mengatur keluar secara ritmis, dan menyimpan cache respons. Artikel ini menunjukkan arsitektur dan implementasi minimal menggunakan Python (FastAPI + httpx).

Arsitektur Proxy Lokal 

<svg width="500" height="250" xmlns="http://www.w3.org/2000/svg"> <rect x="20" y="80" width="100" height="50" rx="5" fill="#4a90d9"/> <text x="70" y="110" text-anchor="middle" fill="white" font-size="14">Client</text> <path d="M120 105 L180 105" stroke="#666" stroke-width="2" marker-end="url(#arrow)"/> <rect x="180" y="50" width="140" height="110" rx="5" fill="#f5a623"/> <text x="250" y="75" text-anchor="middle" fill="white" font-size="14">Local Proxy</text> <text x="250" y="95" text-anchor="middle" fill="white" font-size="11">Token Bucket</text> <text x="250" y="115" text-anchor="middle" fill="white" font-size="11">Request Queue</text> <text x="250" y="135" text-anchor="middle" fill="white" font-size="11">Retry Manager</text> <text x="250" y="155" text-anchor="middle" fill="white" font-size="11">Cache Store</text> <path d="M320 105 L380 105" stroke="#666" stroke-width="2" marker-end="url(#arrow)"/> <rect x="380" y="80" width="100" height="50" rx="5" fill="#7ed321"/> <text x="430" y="110" text-anchor="middle" fill="white" font-size="14">Upstream API</text> <defs> <marker id="arrow" markerWidth="10" markerHeight="10" refX="8" refY="3" orient="auto" markerUnits="strokeWidth"> <path d="M0,0 L0,6 L9,3 z" fill="#666"/> </marker> </defs> </svg>

Penjelasan alur:

  1. Client mengirim request ke Local Proxy (bukan langsung ke upstream).
  2. Proxy memeriksa Token Bucket: apakah token tersedia? Jika ya, request dilepaskan ke upstream; jika tidak, request masuk Request Queue.
  3. Retry Manager menangani respons 429 dari upstream dengan membaca header Retry-After dan menambahkan jitter.
  4. Cache Store menyimpan respons GET yang berhasil; request identik berikutnya disajikan dari cache tanpa menghitung ke bucket.

Implementasi Minimal (FastAPI + httpx) 

# proxy.py import asyncio, time, hashlib, json from collections import deque from fastapi import FastAPI, Request, Response, HTTPException import httpx app = FastAPI() UPSTREAM = "https://api.example.com" RATE = 10 # request per second BURST = 20 # bucket capacity CACHE_TTL = 300 # seconds  class TokenBucket: def __init__(self, rate, burst): self.rate = rate self.burst = burst self.tokens = burst self.last = time.monotonic() self.lock = asyncio.Lock() async def take(self): async with self.lock: now = time.monotonic() self.tokens = min(self.burst, self.tokens + (now - self.last) * self.rate) self.last = now if self.tokens >= 1: self.tokens -= 1 return True return False bucket = TokenBucket(RATE, BURST) queue = deque() cache = {} async def forward(req: Request): key = f"{req.method}:{req.url.path}:{req.query_params}" if req.method == "GET" and key in cache: cached, ts = cache[key] if time.time() - ts < CACHE_TTL: return Response(content=cached, media_type="application/json") async with httpx.AsyncClient(base_url=UPSTREAM, timeout=10) as client: while True: if await bucket.take(): break await asyncio.sleep(0.1) upstream_req = client.build_request(req.method, req.url.path, params=req.query_params, headers=dict(req.headers), content=await req.body()) resp = await client.send(upstream_req) if resp.status_code == 429: retry = int(resp.headers.get("Retry-After", "1")) await asyncio.sleep(retry + 0.1 * (hash(key) % 10)) continue if req.method == "GET" and resp.status_code == 200: cache[key] = (resp.content, time.time()) return Response(content=resp.content, status_code=resp.status_code, media_type=resp.headers.get("content-type")) @app.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE"]) async def proxy(path: str, request: Request): return await forward(request) if __name__ == "__main__": import uvicorn uvicorn.run(app, host="0.0.0.0", port=8000)

Menjalankan Proxy 

pip install fastapi httpx uvicorn uvicorn proxy:app --port 8000

Sekarang arahkan client ke http://localhost:8000/... sebagai pengganti upstream asli.

Checklist Produksi

  • [ ] Tambahkan metrics (Prometheus) untuk mengamati bucket & queue.
  • [ ] Implementasikan circuit breaker jika upstream terus gagal.
  • [ ] Gunakan Redis untuk cache & bucket terdistribusi.
  • [ ] Tambahkan authentication (API key) pada proxy.
  • [ ] Tulis unit & integration test untuk retry logic.

Kesimpulan

Dengan local proxy token‑bucket, aplikasi Anda tidak lagi perlu khawatir soal 429. Proxy menyerap burst, mengatur throughput, dan mengurangi beban upstream lewat caching. Pola ini cocok untuk microservices, scraper, dan integrasi pihak ketiga.

Artikel ini dipublikasikan otomatis melalui pipeline Writer → Designer → Reviewer → Publisher.