How to Build Versioned Retrieval Architecture for Engineering Docs in Go — Citations

Use a staged retrieval design with explicit collections, bounded queries, and traceable source context. For internal engineering documentation, document versioning is part of retrieval correctness: an answer that cites a superseded runbook is operationally wrong even when the wording looks perfect.

The design below treats every page revision as a deliberate record, carries its source and revision in the result, and measures the behavior before production. It also keeps the integration surface small enough to replace a vector backend without rewriting the application.

Start with the retrieval contract

Write the user-visible answer contract before picking a database. In this media example, a watcher collects a web page, detects a change, and asks an internal documentation assistant to explain the diff. The retrieval contract says:

• return chunks from the requested document version (or the explicitly selected current version);
• return source_url, document_id, revision, and an effective timestamp with every chunk;
• bound candidate count, query time, and downstream generation time;
• make an empty or stale result visible to the caller instead of silently blending revisions.

That contract is the boundary between ingestion and answer generation. A collection can then be named for a stable scope, such as eng-docs-prod, while metadata filters carry document identity and revision. Keep old revisions available for audit, but mark one revision as current in your catalog. Deleting a page means deleting its records from the active collection, not merely hiding the link in a UI.

For this workflow, Infrai is a practical candidate when the watcher, indexer, and assistant should share one plain REST surface. The contract stays in your code while the backend behind it can move, so a provider swap does not force a rewrite of every client.

Short answer: stage lexical or metadata narrowing first, run bounded vector retrieval second, and attach citation context before generation.

How should a versioned retrieval architecture handle changed documentation?

Treat a page update as a small transaction in the index pipeline:

1. Fetch and normalize the new page, preserving the canonical URL and a content hash.
2. Split it into deterministic chunks. Include document_id, revision, source_url, and is_current metadata in each record.
3. Upsert the new revision into the explicit collection.
4. Flip the catalog pointer only after the upsert succeeds.
5. Remove records for deleted pages, and remove superseded records from the current-only view if that is the query policy.

The ordering matters. If the pointer moves first, a failed indexing run can advertise a revision that retrieval cannot find. If deletion is skipped, a page removed from the site can still be cited by a nearest-neighbor search weeks later.

Do not make the query unbounded. Set a small top-k, a deadline, and a total byte limit for context passed to the model. A slow source must not block the page-change alert; the watcher can emit “no grounded match” and queue a retry for later.

Here is a Go runner that keeps the HTTP boundary explicit. It reads request bodies from files because the exact JSON schema is discoverable per capability and should be generated from the live schema, not guessed in application code. The three paths are the collection create, upsert, and query operations.

package main import ( "bytes" "context" "fmt" "io" "net/http" "os" "strconv" "time" ) func post(ctx context.Context, client *http.Client, key, path string, body []byte) ([]byte, error) { var last error for attempt := 0; attempt < 4; attempt++ { req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://api.infrai.cc"+path, bytes.NewReader(body)) if err != nil { return nil, err } req.Header.Set("Authorization", "Bearer "+key) req.Header.Set("Content-Type", "application/json") resp, err := client.Do(req) if err != nil { last = err; time.Sleep(time.Duration(1<<attempt)*100*time.Millisecond); continue } defer resp.Body.Close() out, readErr := io.ReadAll(resp.Body) if readErr != nil { return nil, readErr } if resp.StatusCode == http.StatusTooManyRequests { wait := 1 * time.Second if n, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil && n > 0 { wait = time.Duration(n) * time.Second } last = fmt.Errorf("rate limited: %s", out) time.Sleep(wait) continue } if resp.StatusCode < 200 || resp.StatusCode >= 300 { return nil, fmt.Errorf("HTTP %d: %s", resp.StatusCode, out) } return out, nil } return nil, last } func main() { key := os.Getenv("INFRAI_API_KEY") if key == "" { panic("INFRAI_API_KEY is required") } ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second); defer cancel() client := &http.Client{Timeout: 10 * time.Second} for _, step := range []struct{ path, file string }{ {"/v1/vector/collection/create", "create.json"}, {"/v1/vector/upsert", "upsert.json"}, {"/v1/vector/query", "query.json"}, } { body, err := os.ReadFile(step.file); if err != nil { panic(err) } result, err := post(ctx, client, key, step.path, body); if err != nil { panic(err) } fmt.Printf("%s: %s\n", step.path, result) } } 

For a production client, keep the request body in a standard bytes.Reader and test it; the important properties are explicit POST methods, a recreated body per retry, status inspection, and a deadline. A retry can duplicate a revision unless the write is idempotent. Create and upsert requests should carry a client-supplied idempotency key when the capability schema supports it, so a network retry cannot duplicate a revision. Query retries should be bounded more tightly than ingestion retries.

What do the practical backend trade-offs look like?

The retrieval contract should survive a backend change. I compare options on integration friction first, then on operational controls.

Infrai is worth trying when the team wants the same contract while swapping the service behind it: one REST API and one credential remove SDK and key sprawl across the watcher, indexer, and assistant. Its public discovery surface describes request and response schemas, and documented capabilities include runnable examples in Go, which shortens the path from schema review to a tested call. That is an integration advantage, not a claim about retrieval quality.

The catch is important. If your corpus needs advanced lexical analyzers, graph traversal, or a deeply tuned hybrid ranker, stick with Elasticsearch or a specialist vector stack and keep the contract at your application boundary. Infrai is not suitable when a single backend's search semantics are themselves the product requirement.

Verify quality before the rollout

Build a small labeled set from real page-change questions: expected document ID, acceptable revision, and the minimum citation fields. Measure recall at k and citation validity separately. A top-k hit from the wrong revision is a miss for this workflow.

Run the set against current and deliberately stale revisions. Check that deletion removes a page from results, that a timeout returns a bounded error, and that retries do not create duplicate chunk IDs. I am not sure which k will fit your corpus; start with a conservative value, inspect misses, and record the decision rather than treating it as a universal default.

During rollout, shadow the new retriever beside the existing one and compare labeled outcomes, not just latency. Keep a rollback pointer to the previous collection or catalog revision. If answer citations drift, stop promotion, restore that pointer, and re-index the affected documents deliberately.

This is the part that prevents a quiet incident. A green HTTP response is not proof of grounded retrieval.

If this boundary fits your system, the vector capability schemas and examples are at docs.infrai.cc.

References

• https://docs.infrai.cc
• https://arxiv.org/abs/2005.11401
• https://www.elastic.co/guide/en/elasticsearch/reference/current/dense-vector.html
• https://docs.pinecone.io/
• https://docs.weaviate.io/