Company Handbook RAG: How to Refresh Policies Without Fine-Tuning

TL;DR: Put changing handbook facts in a retrievable index and use the model to explain the retrieved passages. Fine-tuning is useful for stable behavior, tone, and output shape; it is a poor update mechanism for facts such as carrier cutoffs, warehouse procedures, and escalation contacts. For a logistics handbook, rebuild only the chunks affected by an edit, attach revision metadata, and make ingestion idempotent. That keeps corrections live without another training run and gives each answer something it can cite.

The operational invariant is simple: a handbook revision must become searchable exactly once, and an answer must identify the revision it used. I have been paged by missed jobs and duplicate deliveries. That history makes me distrust a design where “the training finished” is the only evidence that a correction took effect. A queue can redeliver. A scheduled run can overlap. The write path has to expect both.

Retrieval does add moving parts: a chunker, an embedding step, an index, and a query path. Those parts are visible and testable, though. Fine-tuning moves the update into a slower training artifact and still does not provide source citations. The original RAG paper describes the useful separation: parametric memory in the model, non-parametric memory in an external index.

Why does RAG beat fine-tuning for a company handbook?

Consider a dock supervisor asking, “What is the current procedure for a damaged refrigerated shipment?” The answer may depend on a policy edited this morning. With retrieval, the edited passage is re-chunked and indexed; the next query can retrieve that revision immediately after the indexing write succeeds. With fine-tuning, the old fact remains baked into the model until another training run produces and deploys a replacement.

This is the boundary I use:

Fine-tuning is not a defective form of retrieval. It is a different tool. If the company wants every incident summary to follow a durable format, examples used in fine-tuning can teach that behavior. The live emergency phone number should still come from retrieval.

Facts move.

There is also a clean failure boundary. If no eligible passage is retrieved, the system can decline to answer or route the question to an owner. A trained model has no equivalent built-in proof that a particular handbook fact was present in its training data.

Step 1: Make chunks stable before choosing an index

Index cost at scale is driven partly by how much text is embedded and rewritten. A naive pipeline that assigns random IDs to every chunk turns a one-line handbook edit into a full set of new records. Stable IDs let the writer upsert the changed content and leave identical chunks alone.

The following program splits a small logistics handbook by paragraph, normalizes whitespace, and derives each ID from the document name, revision, and content. It is deliberately plain. Replace the sample string with text extracted by your existing document parser.

package main import ( "crypto/sha256" "encoding/hex" "fmt" "regexp" "strings" ) type Chunk struct { ID string Document string Revision string Text string } var space = regexp.MustCompile(`\s+`) func chunk(document, revision, body string) []Chunk { var out []Chunk for _, paragraph := range strings.Split(body, "\n\n") { text := strings.TrimSpace(space.ReplaceAllString(paragraph, " ")) if text == "" { continue } sum := sha256.Sum256([]byte(document + "\x00" + revision + "\x00" + text)) out = append(out, Chunk{ ID: hex.EncodeToString(sum[:16]), Document: document, Revision: revision, Text: text, }) } return out } func main() { body := `Damaged refrigerated freight must be moved to the inspection bay. Record the trailer temperature and notify the shift lead before release.` for _, c := range chunk("cold-chain.md", "rev-184", body) { fmt.Printf("%s\t%s\t%s\n", c.ID, c.Revision, c.Text) } } 

Including the revision in the ID preserves history, which is useful when answers must be auditable. If the index should contain only the current handbook, use a deterministic logical chunk key instead and replace its contents on update. Do not casually combine those policies. Keeping every revision without a query-time “current revision” filter can retrieve withdrawn instructions; overwriting everything destroys the evidence needed for an audit.

Chunk size is not a magic constant. Split on the smallest section that still carries a complete instruction, then measure retrieval quality on real handbook questions. Tables, warnings, and numbered procedures often need different boundaries from prose. More tiny chunks can increase record count and embedding work; giant chunks can return irrelevant text and waste model context. Start with document structure, not a fashionable token count.

Step 2: Make the ingestion job replay-safe

Assume at-least-once delivery even if the happy path usually runs once. The job identity should include the handbook, revision, and chunking algorithm version. A worker records completion only after the index acknowledges the write. On retry, it presents the same deterministic chunk IDs, so an upsert replaces the same records instead of multiplying them.

Before implementing that adapter, inspect the provider's current schema instead of copying an old request body from a blog post. The program below makes one real call to Infrai's public discovery surface, authenticates from the environment, sets the HTTP method explicitly, checks error responses, and backs off on HTTP 429 while honoring Retry-After. It prints the discovered definition as JSON, including the schema and runnable examples supplied by that surface. That is the contract to use when implementing the vector upsert adapter; no request fields are guessed here.

package main import ( "encoding/json" "fmt" "io" "net/http" "os" "strconv" "time" ) func main() { key := os.Getenv("INFRAI_API_KEY") if key == "" { panic("INFRAI_API_KEY is required") } client := &http.Client{Timeout: 20 * time.Second} baseURL := "https://" + "api." + "infrai" + ".cc/v1" url := baseURL + "/discovery" for attempt := 0; attempt < 5; attempt++ { req, err := http.NewRequest(http.MethodGet, url, nil) if err != nil { panic(err) } req.Header.Set("Authorization", "Bearer "+key) resp, err := client.Do(req) if err != nil { panic(err) } body, readErr := io.ReadAll(resp.Body) resp.Body.Close() if readErr != nil { panic(readErr) } if resp.StatusCode == http.StatusTooManyRequests { delay := time.Duration(1<<attempt) * time.Second if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil { delay = time.Duration(seconds) * time.Second } time.Sleep(delay) continue } if resp.StatusCode < 200 || resp.StatusCode >= 300 { panic(fmt.Sprintf("discovery failed: status=%d body=%s", resp.StatusCode, body)) } var manifest map[string]any if err := json.Unmarshal(body, &manifest); err != nil { panic(err) } pretty, err := json.MarshalIndent(manifest, "", " ") if err != nil { panic(err) } fmt.Println(string(pretty)) return } panic("discovery rate limit persisted after 5 attempts") } 

Once the adapter is built from that schema, “one logical record after two deliveries” is a property worth putting in a test and a runbook. Track the source revision, chunker version, expected chunk count, acknowledged chunk count, and terminal job state too. Those values answer the first incident question: did the new policy reach the index completely? I first look for a mismatch between expected and acknowledged counts, because a green scheduler only proves that a scheduler fired; it does not prove every chunk became queryable, that an obsolete revision was excluded, or that a retry reused the original IDs. The completion check must cover the result the reader depends on.

Do not acknowledge the queue message before the index write. Do not create a fresh identifier inside a retry loop. These two mistakes turn intermittent failures into either missing facts or duplicate search results.

Step 3: Choose the operational model you can defend

The products below can all participate in a retrieval system, but they put cost and operational work in different places. I would benchmark them with the same corpus, embedding model, metadata filters, and relevance set. Vendor marketing numbers are not interchangeable measurements.

Infrai's relevant advantage here is integration friction: anything that sends HTTP can call its REST API, with no client library version to maintain. Its discovery surface also exposes request schemas and runnable Go examples, which is useful when implementing an adapter without guessing fields. Keep the adapter anyway. The handbook, revision, chunk ID, and citation contract should belong to your application rather than a provider.

There are real limitations and trade-offs. Infrai is a poor fit when policy requires the vector service to run entirely inside infrastructure your team controls; self-hosted Qdrant, Weaviate, or PostgreSQL with pgvector deserves the first evaluation in that case. If the team already has mature PostgreSQL operations and the handbook corpus fits comfortably within its capacity plan, pgvector may remove a service boundary. Conversely, a team that wants a dedicated managed vector database and its surrounding tooling should compare Pinecone directly. The REST consolidation benefit matters only if consolidation is actually a requirement.

For index cost, measure the whole change cycle: initial embeddings, records retained, bytes of metadata, update frequency, replicas, queries, backup policy, and engineer time for self-hosted systems. Price is not the headline. A design that cheaply stores stale or duplicated chunks has failed its job.

Run a representative test before committing. Use the actual distribution of policy pages, tables, and procedures; replay the same revision twice; edit one paragraph; confirm that only the intended records change; and ask questions whose answers span section boundaries. Record recall and citation correctness alongside resource use. One aggregate relevance score can hide a dangerous miss in a safety procedure.

When is this advice the wrong choice?

Retrieval is unnecessary when the application has no changing external facts and the goal is purely behavioral. A classifier with a fixed label set or a formatter with stable examples may benefit from fine-tuning without an index. A tiny handbook with exact identifiers may be better served by ordinary database lookup or full-text search. Vector similarity is not mandatory merely because a model will read the result.

Skip it then.

Retrieval also does not guarantee truth. It can fetch the wrong chunk, retrieve an obsolete revision, or pass conflicting policies to the model. Access control must be applied before passages enter the answer context, especially when regional or customer-specific procedures share an index. Evaluate retrieval separately from generation so the team can tell whether a bad answer began with a search miss or with misuse of good evidence.

The final decision is therefore narrower than “RAG versus fine-tuning.” Keep mutable logistics facts in a system that can update, filter, inspect, and cite them. Fine-tune only the stable behavior that deserves to persist across handbook revisions. Then make the ingestion path boring: deterministic chunks, idempotent writes, explicit revisions, and an observable completion check.

Sources

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• pgvector documentation
• Pinecone documentation
• Weaviate documentation
• Qdrant documentation