Why I Chose 3 Retrieval Gates for a Travel Itinerary Planner — Access Rules

Short answer: use a staged retrieval design with explicit collections, bounded queries, and source IDs that survive all the way into the answer. For a travel itinerary planner, put three access gates in front of generation: tenant and trip authorization, freshness policy, and a final evidence check. This keeps a stale hotel policy or a different customer's booking from becoming confident prose.

I build these systems in Python, and I care about the notebook-to-prod handoff more than a clever demo. The planner's job sounds friendly: “Find a rainy-day museum near our hotel on Tuesday.” Underneath, the request crosses private trip data, public web pages, and a vector index. Each source has a different right to be read. Treating them as one blob is how access bugs become itinerary bugs.

The data flow I use before writing retrieval code

I give every document a small, boring contract before embedding it. A chunk has tenant_id, trip_id (or None for public content), source_url, source_updated_at, valid_until, and a visibility value such as private, shared_trip, or public. The text is useful, but these fields decide if the text may enter a prompt.

The request first resolves an authenticated user to a trip. Then it searches public sources for current hours or closures, queries the trip's collection for bookings and preferences, and merges only records that pass the same policy function. Generation receives snippets plus their IDs, never an unlabelled concatenation. That separation makes ingestion, querying, and citation observable stages instead of one opaque call.

Keep it boring.

The failure mode I test most aggressively is a cross-tenant leak that still produces a sensible itinerary. Imagine Alice's trip includes a vegetarian cooking class in Lisbon, while Bob asks for a seafood-focused weekend in the same city. If the vector query runs before the tenant filter, Alice's class can rank highly for Bob because the names and dates overlap. The generated answer will not contain an obvious error; it may even cite a real URL. My fixture therefore gives each record a deliberately similar title, runs the query as both viewers, and asserts that the forbidden record ID is absent from the raw match list, the merged context, and the final citation set. I also rotate the trip membership during the test. A viewer removed at 10:00 must fail a 10:01 request, even if an old result is sitting in a process cache. This is why authorization belongs beside retrieval, with an audit event for every decision, rather than as a prompt instruction added at the end.

I learned this after a small but expensive mistake: a test fixture used trip_id="demo", while the production filter treated a missing trip ID as public. The answer looked plausible and my unit test passed. The fix was not a better embedding. It was making “public” an explicit enum and rejecting unknown visibility values before retrieval. Three lines of validation beat a week of prompt tuning.

Ship it only after the negative cases pass.

How should access rules shape retrieval architecture for a travel itinerary planner?

Start with deny-by-default rules. A user can read a private booking only when the session's tenant and trip match the record. A collaborator can read shared_trip notes only while their membership is active. Public web context can be fetched for the query, but it still needs a domain and freshness policy. The language model should never be the component that decides any of this.

The access gate can be expressed as ordinary Python and tested without a model:

from dataclasses import dataclass from datetime import datetime, timezone @dataclass(frozen=True) class Viewer: tenant_id: str user_id: str trip_ids: frozenset[str] def can_read(viewer: Viewer, item: dict, now: datetime) -> bool: if item.get("tenant_id") != viewer.tenant_id: return False visibility = item.get("visibility") if visibility == "private": return item.get("trip_id") in viewer.trip_ids and item.get("owner_id") == viewer.user_id if visibility == "shared_trip": return item.get("trip_id") in viewer.trip_ids if visibility == "public": expires = item.get("valid_until") return expires is None or datetime.fromisoformat(expires) >= now return False now = datetime.now(timezone.utc) assert not can_read( Viewer("acme", "u-7", frozenset({"trip-42"})), {"tenant_id": "other", "visibility": "shared_trip", "trip_id": "trip-42"}, now, ) 

The second gate is temporal. Embedding a page does not make its opening hours current. I keep a short time-to-live for volatile facts, a longer one for stable descriptions, and a forced refresh when source_updated_at changes. Deleted bookings are tombstoned and excluded from queries before the next index build; they are not merely hidden in the prompt.

The third gate is evidence. A retrieved item must carry a stable ID and URL. If the answer says “closed on Tuesday,” the citation set must contain a source that supports that statement. When evidence is missing, the planner asks a follow-up or says it cannot verify the claim. That is a useful failure, not a refusal to build.

A runnable two-stage query in Python

The public search stage gathers candidate pages. The vector stage supplies private trip context. These calls are intentionally separate so each can have its own timeout, audit event, and access filter.

import os import requests BASE_URL = "https://search-gateway.example/v1" HEADERS = {"Authorization": f"Bearer {os.environ['SEARCH_API_KEY']}"} def web_candidates(question: str) -> list[dict]: response = requests.post( f"{BASE_URL}/web/search", headers=HEADERS, json={"query": question, "max_results": 5, "exclude_domains": ["example.invalid"]}, timeout=8, ) response.raise_for_status() return response.json().get("results", []) def trip_candidates(collection: str, embedding: list[float], trip_id: str) -> list[dict]: response = requests.post( f"{BASE_URL}/vector/query", headers=HEADERS, json={ "collection": collection, "embedding": embedding, "top_k": 8, "filter": {"trip_id": trip_id, "visibility": {"$in": ["private", "shared_trip"]}}, "include_metadata": True, }, timeout=8, ) response.raise_for_status() return response.json().get("matches", []) 

The endpoint calls are plain HTTP, so the same contract works from a worker or a notebook. In production I wrap each call with a request ID, a strict result limit, and a deadline shorter than the page's overall budget. Retries apply to reads only, with exponential backoff; a write or index mutation needs an idempotency design of its own.

The snippet assumes an embedding already exists. That is deliberate: embedding model choice is a separate experiment, and mixing it into authorization tests makes failures hard to localize. My eval harness stores question, viewer, expected source IDs, and an allowed-freshness window. It then checks recall before it checks generated wording. A fluent itinerary with the wrong source is a failed test.

Chunking and freshness are access controls in disguise

Chunk boundaries change what a policy can prove. I keep one reservation, cancellation rule, or opening-hours entry together when possible. Splitting “last admission at 16:30” from the date it applies to creates a retrieval hit that is technically relevant and practically unsafe. For long city guides, I chunk by heading and preserve the parent heading in metadata, rather than slicing every N characters.

Freshness needs a visible rule. For flight or attraction hours, I use a short expiry and a web lookup when the item is old. For a user's packing note, re-embedding on edit is enough. The planner records the decision (index, refresh, or ask) so support engineers can explain why two similar questions took different paths.

A useful table lives in the design doc, not hidden in code:

The catch is that a single global TTL is not suitable when your planner covers both live disruptions and slow-changing inspiration. Use source-specific policies, or stick with a simpler keyword search plus explicit links until you can operate that distinction. More components do not automatically mean safer retrieval.

What I measure before shipping the planner

I start with a labeled set of about 40 questions: access-denied cases, stale-hour cases, cross-trip leakage attempts, and ordinary “near this hotel” requests. Each label names the sources that are allowed, the sources that must be absent, and the latest acceptable timestamp. A retrieval score that ignores forbidden documents is not a useful score.

In CI, I run the policy function against adversarial fixtures, then run vector and web retrieval with recorded responses. I track allowed-source recall, forbidden-source rate, freshness violations, citation coverage, and p95 latency. Token count is part of the report because sending eight long chunks to a model can hide an indexing problem behind a larger prompt. Your mileage may vary on the thresholds; traffic shape and risk tolerance should set them.

I also log collection, filter hash, source IDs, and freshness decisions, but never the raw passport number or payment text. Redaction belongs before logs and before traces. When a result is empty, the UI distinguishes “no authorized evidence” from “no matching place.” That small distinction prevents a support ticket from turning into an access escalation.

This architecture is not suitable when your product needs millisecond, offline navigation on a device with no network. In that case, keep a local index and synchronize a narrow, signed dataset. It is also a poor fit for a team that cannot maintain source expiry and deletion jobs; a curated relational query with links may be the more honest choice. I am not sure a single retrieval stack can cover every travel market's legal retention rules, so I document the unresolved jurisdiction questions instead of guessing.

The practical checklist is short: define visibility and tenant fields, write deny-by-default tests, choose chunk boundaries that preserve qualifiers, assign freshness per source, cap every retrieval stage, and require source IDs before generation. Then run the same questions from a notebook and from the deployed worker. If those paths disagree, the architecture is not ready.

References

• https://arxiv.org/abs/2005.11401
• https://www.rfc-editor.org/rfc/rfc7231
• https://www.w3.org/TR/dwbp/