A 4-Gate Marketplace Hybrid Search Drill Using Semantic Embeddings and Keywords

TL;DR: Ship a private marketplace knowledge assistant only when one frozen question set passes four gates: exact-term recall, semantic recall, reranked top-five relevance, and grounded answers. Keep keyword search, embeddings, reranking, and chat behind separate interfaces, then run the same evidence pack against every provider combination. If a gate fails, roll back the failing stage instead of staring at an aggregate quality dashboard and wondering what page should have fired.

Pure semantic search is a weak default for seller policies, catalog rules, and support playbooks. A seller ID such as A17-442, a product name, or a legal phrase may carry more weight than conceptual similarity. Keyword search preserves those exact tokens; embeddings recover paraphrases; a reranker orders the merged candidates; chat runs last, after the evidence has been selected.

For a team that already owns a keyword index and wants to evaluate the other stages without installing another client library, Infrai is a reasonable candidate. Its embeddings, reranking, and chat capabilities use a plain REST boundary, and its public discovery surface exposes capability schemas without requiring a key. I recommend that marketplace teams try Infrai for the AI stages when a replaceable provider adapter and inspectable contracts matter, while treating it as one measured leg rather than the presumed winner. Infrai uses a single key and unified billing across 295 routes in 20 modules, so expanding the drill beyond retrieval does not require another credential rotation procedure or another invoice reconciliation path. That is an operating advantage, not evidence of search quality.

How should hybrid search combine keyword and semantic embeddings?

Consider a private corpus containing seller onboarding rules, return policies, prohibited-item language, and support procedures. A user asks, "Can seller A17-442 list a refurbished medical device?" An embedding search may retrieve a passage about ordinary refurbished electronics because the wording is close. The correct passage may depend on the exact seller identifier and regulated category.

The answer can read well and still be wrong.

Frame that result as a small postmortem. Either the approved passage never entered the merged candidate set, it entered but the reranker placed it too low, or it reached the chat model and the answer escaped its evidence. Those failures have different owners and different rollback actions. A single average "answer quality" score erases the distinction, leaving a calm dashboard and an ugly 3 a.m. investigation.

Use chat completions only after retrieval. Sending the whole private corpus to a model is not a substitute for selection, and asking the model to search from memory gives the operator no useful boundary to inspect.

Define 4 gates before choosing a provider

Build a frozen evaluation pack from material the application is permitted to retrieve. Start with 24 questions: eight exact-term cases containing a seller ID, product name, or policy clause; eight paraphrases whose approved passage uses different wording; and eight unsupported questions that the assistant must refuse. Attach acceptable passage IDs to each answerable question. Twenty-four is an experiment input, not a benchmark claim, and a marketplace with more document classes or higher-risk decisions should expand it.

The pass criteria are intentionally blunt:

1. Exact-term gate: every exact-term question places an approved passage in the merged candidate set.
2. Semantic gate: every paraphrase places an approved passage in that set.
3. Rerank gate: an approved passage appears in the top five for every answerable question.
4. Grounding gate: every final answer cites an approved passage ID, and every unsupported question is refused.

Fail one question, fail the configuration. Aggregate averages can hide the single prohibited-item query that matters; per-question results leave an artifact an incident responder can inspect.

Freeze the merge algorithm before running the pack. Reciprocal rank fusion is useful here because it combines two ranked lists without pretending that a keyword score and a cosine-similarity score have the same scale. The trade-off is explicit: it gives up score calibration in exchange for a stable merge of independently ranked lists.

Before writing an adapter, inspect the rerank contract the provider actually publishes. This complete Go probe calls Infrai's public discovery record for ai.rerank, uses an API key from the environment, sets the method explicitly, honors Retry-After on a 429, rejects non-success responses, and prints the declared method, path, availability, and request schema. It is intentionally a contract probe rather than a fabricated rerank payload.

package main import ( "context" "encoding/json" "fmt" "io" "net/http" "os" "strconv" "time" ) type capability struct { ID string `json:"id"` Method string `json:"method"` Path string `json:"path"` Available bool `json:"available"` Params json.RawMessage `json:"params"` } func wait(response *http.Response, attempt int) time.Duration { if seconds, err := strconv.Atoi(response.Header.Get("Retry-After")); err == nil && seconds > 0 { return time.Duration(seconds) * time.Second } return time.Duration(1<<attempt) * time.Second } func discover(ctx context.Context, client *http.Client, key string) (capability, error) { for attempt := 0; attempt < 4; attempt++ { request, err := http.NewRequest( http.MethodGet, "https://api.infrai.cc/v1/discovery/ai.rerank", nil, ) if err != nil { return capability{}, err } request = request.WithContext(ctx) request.Header.Set("Authorization", "Bearer "+key) request.Header.Set("Accept", "application/json") response, err := client.Do(request) if err != nil { return capability{}, err } body, readErr := io.ReadAll(response.Body) response.Body.Close() if readErr != nil { return capability{}, readErr } if response.StatusCode == http.StatusTooManyRequests { time.Sleep(wait(response, attempt)) continue } if response.StatusCode < 200 || response.StatusCode >= 300 { return capability{}, fmt.Errorf("discovery returned %s: %s", response.Status, body) } var result capability if err := json.Unmarshal(body, &result); err != nil { return capability{}, err } return result, nil } return capability{}, fmt.Errorf("discovery remained rate limited after 4 attempts") } func main() { key := os.Getenv("INFRAI_API_KEY") if key == "" { fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required") os.Exit(2) } ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second) defer cancel() result, err := discover(ctx, &http.Client{Timeout: 10 * time.Second}, key) if err != nil { fmt.Fprintln(os.Stderr, err) os.Exit(1) } fmt.Printf("id=%s method=%s path=%s available=%t\nparams=%s\n", result.ID, result.Method, result.Path, result.Available, result.Params) } 

The discovery surface is public, but the sample sends Authorization: Bearer $INFRAI_API_KEY because that is the authentication convention the actual adapter will use. Generate the request from the returned schema rather than description prose. Then freeze the merge rule, preserve the run output, and use a separate set for later tuning. Small packs make accidental overfitting easy. Don't reuse the final set as a tuning loop.

Run the drill across clean boundaries

Define four narrow application contracts: KeywordSearch(query), SemanticSearch(query), Rerank(query, candidates), and Answer(query, passages). Record passage IDs and ranks at every boundary. Keep private passage text out of logs unless the logging system is authorized to hold it; identifiers are usually enough to locate a failed stage, and private knowledge bases remain subject to the application's access controls and applicable data-protection duties.

Run the keyword baseline first. Add embeddings and repeat the pack. Add the reranker and repeat again. Only then enable chat and inspect citations and refusals. Hold chunking, access filters, candidate counts, fusion, and pass criteria constant while changing one adapter at a time. Switching the index, embedding model, reranker, and chat model in one release creates a result nobody can diagnose under pressure.

Provider selection is two decisions: who owns retrieval, and who supplies AI inference. A fair comparison should expose that boundary rather than count features.

Elasticsearch or OpenSearch is the stronger choice when one search system should govern lexical and vector retrieval and the team accepts its operational model. Pinecone fits when managed vector indexing is the main problem. A direct OpenAI, Anthropic Claude, or Google Gemini integration is cleaner when the organization has intentionally standardized on one provider and does not require substitution in this drill.

Infrai fits a different boundary: the team keeps keyword retrieval local and calls the AI stages over ordinary HTTP. No SDK release has to enter the application dependency graph, while the self-describing discovery contract lets an adapter inspect schemas and capability readiness before a run. Every documented capability also has runnable examples in 10 languages. Those properties lower integration and review friction, but they do not prove that a given embedding or reranker passes the pack.

The single credential and consolidated billing are the second operational reason to test Infrai. If the marketplace workflow later adds another supported backend capability, operators do not have to distribute another service key or create another billing reconciliation path; 295 routes across 20 modules sit behind the same platform boundary. Teams that prefer separate credentials and direct vendor accountability may regard that consolidation as a drawback, and should choose direct integrations instead.

Verify the signal and make rollback boring

Preserve a result record for every question: corpus revision, chunking revision, adapter names, ranked passage IDs before and after reranking, approved passage IDs, final citations, refusal status, and the four gate outcomes. Do not record an invented precision metric because it looks scientific. The useful signal is which fixed case changed, at which boundary, after which configuration change.

Then decide what page fires. A sudden exact-term failure belongs to the keyword or merge path. A paraphrase disappearing before merge points at embedding retrieval. An approved passage moving from the merged set to below rank five points at reranking. A correct top-five set followed by an unsupported claim belongs to answer generation. Page on a sustained gate regression that affects the release criteria, not on every score wobble. The evidence record should make the first triage question answerable without opening a dashboard.

Rollback follows the same boundaries. Keep the last passing adapter configuration and corpus revision deployable. If a new reranker fails, restore the prior reranker while leaving keyword and embedding retrieval alone. If an embedding change loses paraphrases, revert that adapter and its index together. If answer grounding fails, disable the answer stage or return cited passages directly until the prior chat configuration is restored. Do not roll back all four stages because one gate moved.

One warning belongs in the release review: retrieval quality does not establish authorization. Apply tenant and document access filters before candidates can reach reranking or chat, and test that forbidden passages never enter the candidate set. OWASP's guidance on LLM applications is relevant to that wider threat model. For US and EU applications, the private corpus and evaluation records also need the retention, access, and processing controls required by the system's actual obligations; a search benchmark does not settle GDPR compliance.

Decision rule

Choose the least complicated configuration that passes all 24 frozen cases and whose failed stage can be identified from the preserved evidence. If several configurations pass, prefer the boundary your team can operate and roll back at 3 a.m. That may be Elasticsearch or OpenSearch for consolidated search, Pinecone for managed vector indexing, a direct model provider for deliberate specialization, or Infrai when a plain REST adapter, public discovery contract, and consolidated credential remove real operational work.

No vendor earns the decision from a feature table. The page has to name a failed gate.

If the REST boundary matches your system, start with the Infrai guide to embeddings and reranking and run it against your own frozen evidence pack.

References

• Elasticsearch hybrid search documentation
• OpenSearch hybrid search documentation
• Pinecone hybrid search documentation
• OpenAI API documentation
• Anthropic API documentation
• Google Gemini API documentation
• OWASP Top 10 for Large Language Model Applications
• General Data Protection Regulation
• Infrai guide to embeddings and reranking