A long model request should not force your app to keep the first HTTP connection open. OGAD (Off Grid AI Desktop) supports asynchronous API requests: submit the job, keep its request ID, then read the result through a separate request.

Download OGAD

OGAD

The core local API is free. Asynchronous here means separate submission and polling. It is not a durable queue that survives an app restart.

Submit a small request first

Keep OGAD running with a local text model loaded. The following Python 3 example uses only the standard library. It chooses a local chat model from the gateway inventory, submits a request, and polls for up to five minutes.

import json import time from urllib.request import Request, urlopen BASE = "http://127.0.0.1:7878" def request(path, body=None): data = None if body is None else json.dumps(body).encode() req = Request(BASE + path, data=data, headers={"Content-Type": "application/json"}) with urlopen(req, timeout=30) as response: return json.load(response) models = request("/v1/models")["data"] model = next((m["id"] for m in models if m.get("kind") in ("chat", "vision") and not m.get("remote")), None) if model is None: raise SystemExit("Load a local text model in OGAD first.") job = request("/v1/chat/completions?async=true", { "model": model, "messages": [{"role": "user", "content": "Write a short checklist for reviewing a project brief."}], "max_tokens": 180, "stream": False }) print("Request:", job["request_id"]) poll_path = "/v1/requests/" + job["request_id"] deadline = time.monotonic() + 300 while time.monotonic() < deadline: state = request(poll_path) if state["status"] == "completed": print(json.dumps(state["result"], indent=2)) break if state["status"] == "failed": raise SystemExit(str(state.get("error", "Request failed"))) time.sleep(2) else: raise SystemExit("Polling timed out. The server job may still be running.")

Use the actual gateway port if it differs. Save this as local_async.py and run it with python3 local_async.py. The inventory can contain more than one model; for a specific task, replace the selected ID with the local model you intend to use.

Know which state you received

The initial submission returns HTTP 202 and a request resource. Queued and running mean the result is not ready. Completed carries the result. Failed carries error information.

Do not resubmit the same task every time a poll says running. Keep the original ID and poll that resource. A client-side timeout also does not prove the server stopped working.

Keep the lifetime limit in your design

Request records live in the running gateway's memory. Restarting the app loses them, and the store is bounded to 500 records. Save the completed result in your own app when you need to retain it. Do not use the request ID as a permanent document link.

Prepare the local model before offline use. Async submission changes how you wait; it does not remove network needs from a remote model or a web-dependent task.

The API reference describes this request-resource flow. Try OGAD with one short job, then use the same lifecycle for work that takes longer.