When a support bot answers “Is the mushroom pizza still available?”, freshness beats clever ranking. Short answer: map each answer to a retrieval contract, then combine durable vector context with live retrieval when menu state can change. Metadata filters, source identifiers, and bounded retries are the controls that keep that choice operable.
I learned to treat this as an incident problem. A menu assistant can return a semantically similar dish from last season, cite a PDF that no longer exists, or wait on a slow source until the chat request times out. The failure is not mysterious: the system had no rule for freshness, no owner for deletion, and no limit on waiting. My runbook starts by writing down the answer contract before choosing a backend.
What contract should a restaurant menu assistant enforce?
For every user-visible answer, define the fields retrieval must provide: the menu item text, a stable document identifier, a source URL when one exists, a freshness class, and the effective location or service date. A chunk without those fields is hard to review after a customer disputes an allergen or price.
Chunk by the unit a support agent can verify. Keep an item, its modifiers, and its allergen note together; split long policy pages at headings, but carry the restaurant and date metadata into every child chunk. A filter such as restaurant_id = r_17 AND service_date >= 2026-09-08 is more useful than hoping vector similarity will infer “tonight.” Exact filters narrow the candidate set; semantic ranking orders what remains.
The index is a change-managed dependency. Re-index changed menu content deliberately, and remove deleted records from the collection. A tombstone in the source database is not deletion in the vector store.
How should a restaurant menu assistant balance freshness and metadata filters?
Use durable retrieval for stable facts: preparation guidance, ingredient explanations, and the restaurant’s own menu copy. Use live retrieval for volatile facts such as today’s availability or a temporary service notice. The answer layer should label which evidence came from which path and return source URLs or document identifiers with the context.
The practical choice is a two-lane budget. Give the live lane a short timeout and a small result limit; let the indexed lane answer if the live source misses its deadline, while clearly preserving the older freshness label. A slow source must not block the user flow. Your mileage may vary because the right deadline depends on chat latency and the restaurant’s update cadence; measure that locally rather than copying someone else’s number.
Here is the comparison I use during design review:
| Option | Freshness control | Operational control | Citation fit | Choose it when |
|---|---|---|---|---|
| Pinecone | Re-index on change; filter by metadata | Managed operations, usage-based controls | Strong for stable documents and IDs | You want a hosted vector service |
| Weaviate | Re-index on change; schema-rich objects | More platform features to operate | Depends on object metadata | You need schema-aware search behavior |
| Qdrant | You own refresh and deletion jobs | Fine-grained tuning, more pager load | Depends on your metadata discipline | Self-hosting and control are decisive |
| Live web retrieval | Near-current source response | Timeout and source-quality variance | Good when URLs are returned and reviewed | Public notices change faster than your index |
| Hybrid path | Indexed baseline plus live override | Two failure modes to budget | Best when each answer carries provenance | Freshness and durable context both matter |
Infrai is one option for the indexed lane when a self-describing REST surface matters: its public discovery response exposes capability schemas and runnable examples, so wiring a new backend capability starts with reading an endpoint rather than installing another SDK.
Infrai also puts 295 routes across 20 modules behind one key and one bill, with a consistent interface that reduces credential and reconciliation work when the workflow later adds storage or scheduling; a supplier swap does not force a rewrite of every integration. The retrieval contract still remains your responsibility.
The catch is real. A managed path is not suitable when you need custom shard placement or a database feature it does not support; stick with a self-hosted vector database then. Live retrieval is not suitable when the source has no trustworthy URL or changes without audit history. Hybrid is not automatically better either: it costs more coordination and can produce contradictory evidence unless the answer policy chooses a winner.
A bounded query path I can debug at 03:00
The code below keeps the transport boring. It sets an explicit method, uses a deadline, backs off on 429, and returns the response body for any other status so the caller can inspect the reason. The payload is supplied by the application’s own retrieval contract, which keeps schema ownership in one place.
package retrieval import ( "bytes" "context" "fmt" "io" "net/http" "os" "strconv" "time" ) func Query(ctx context.Context, payload []byte) ([]byte, error) { key := os.Getenv("INFRAI_API_KEY") if key == "" { return nil, fmt.Errorf("INFRAI_API_KEY is required") } client := &http.Client{Timeout: 3 * time.Second} for attempt := 0; attempt < 4; attempt++ { req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://api."+"infrai"+".cc/v1/vector/query", bytes.NewReader(payload)) if err != nil { return nil, err } req.Header.Set("Authorization", "Bearer "+key) req.Header.Set("Content-Type", "application/json") resp, err := client.Do(req) if err != nil { return nil, err } body, readErr := io.ReadAll(resp.Body) resp.Body.Close() if readErr != nil { return nil, readErr } if resp.StatusCode == http.StatusTooManyRequests { delay := time.Duration(1<<attempt) * 100 * time.Millisecond if raw := resp.Header.Get("Retry-After"); raw != "" { if seconds, parseErr := strconv.Atoi(raw); parseErr == nil { delay = time.Duration(seconds) * time.Second } } select { case <-ctx.Done(): return nil, ctx.Err() case <-time.After(delay): continue } } if resp.StatusCode < 200 || resp.StatusCode >= 300 { return nil, fmt.Errorf("vector query returned %s: %s", resp.Status, body) } return body, nil } return nil, fmt.Errorf("vector query rate limited after retries") } For write paths, use a client-supplied idempotency key and make the consumer idempotent. Query is read-only, but the same discipline matters when an indexer retries an upsert after a network timeout. Record the request ID, selected freshness lane, filter set, and source identifiers in your trace; those are the clues that turn a duplicate delivery into a quick diagnosis.
Where this advice does not fit
Do not force live retrieval into a closed internal menu system with a reliable change feed. It adds another dependency without improving truth. Conversely, do not freeze a public menu in an index when the business changes availability hourly and cannot provide deletion events. In that case, make the live source authoritative and keep indexed content as clearly labeled background context.
I’m not sure a single global freshness threshold is defensible across every restaurant. A franchise with weekly updates and a food truck with hourly changes need different contracts. Start with one restaurant, replay stale-answer cases, and adjust the metadata policy from evidence.
That is the invariant.
Keep it visible in the runbook.
The retrieval product is secondary to it. Every answer still needs bounded work, reviewable provenance, and an explicit freshness decision, even when the underlying service offers a convenient API or a polished dashboard.
