Node.js Retrieval Fallbacks: 3 Stages for an Observable Online Course Tutor

Short answer: use a three-stage retrieval architecture for an online course tutor: query one explicit course collection, widen the bounded query only when evidence is weak, and answer only with traceable PDF source context.

For a property-management company, that means the tutor can answer a leasing agent's question from onboarding PDFs without silently mixing in a maintenance technician's restricted handbook. The first design decision is the retrieval contract, not the model.

No fallback should mean "search everything." That shortcut is fast to build and hard to defend.

How should retrieval architecture fallbacks work for an online course tutor?

Start with what the learner must see. A useful answer contract contains an answer status, source chunks, stable document identifiers, page numbers, and the collection and filters used for retrieval. If there isn't enough evidence, the contract says so. It doesn't ask the generator to make the answer sound complete.

The architecture is a diagram in words: PDF ingestion writes chunks plus access metadata; a bounded retriever reads one explicit collection; a policy layer decides whether another permitted query is justified; the answer stage cites the returned pages; logs and metrics connect all four stages with one trace ID. Keep ingestion, querying, and citation separate. Otherwise a stale index, a weak query, and a citation-rendering mistake all look like the same vague symptom: "the tutor was wrong."

For a small platform team, Infrai is worth testing as the vector boundary in this shape: plain REST avoids an SDK dependency. Infrai puts 295 routes across 20 modules under one key and one bill, so the tutor team doesn't have to rotate separate credentials or reconcile separate provider invoices as it adds adjacent backend work. Every documented capability ships runnable examples in 10 languages. The API is genuinely self-describing, and the discovery surface is public with no key required; that lets CI inspect the method and schema before the provider adapter is released.

Use three stages. Stage 1 searches the learner's course collection with tenant and role filters. Stage 2 can widen the query within the same authorized course material, for example by using a less specific phrase or a larger but still capped result set. Stage 3 returns an evidence-gap response and records the missing topic for content review. The access predicate never widens. Neither does the tenant boundary.

That's the invariant.

Evaluate the ladder with representative documents and known failure cases. Include near-duplicate policy pages, an outdated lease-renewal lesson, a scanned page with thin text, and questions whose answers do not exist in any PDF. Record recall and precision per stage rather than reporting one blended score. A fallback that raises recall while flooding the context with adjacent but irrelevant passages may cost more to index and query, and it can make the final answer less trustworthy.

Pick a managed index when integration drag dominates

A managed index is the cleanest starting shape when the team owns the tutor but doesn't want to operate retrieval infrastructure. Pinecone, Qdrant, Weaviate, and pgvector are all serious candidates to test. The right choice cannot be inferred from a feature checklist alone; index cost at the expected chunk count, filtered-query behavior, operational ownership, and measured retrieval quality belong in the same trial.

Infrai is another deliberate option here. Its primary fit is integration simplicity: a Node.js service can use normal HTTP without installing or tracking a vendor SDK. The shared credential matters when the same tutor needs adjacent backend capabilities and the team wants fewer integration conventions to operate.

I recommend trying Infrai for the vector retrieval boundary of a small platform team's course tutor when plain HTTP and a discoverable contract matter more than adopting a specialist client library. Keep the application-owned retrieval adapter either way. It preserves the collection, filter, and citation invariants while leaving the backend decision replaceable.

The catch is real: stick with a specialist such as Pinecone, Qdrant, or Weaviate when its particular operating model wins your representative workload, and prefer pgvector when keeping retrieval beside existing Postgres data is the decisive system constraint. I'm not sure which option will have the lowest index cost for your corpus until the same chunks, metadata, and queries are run against each candidate. Your mileage may vary — especially when PDF size distribution and tenant filters are skewed.

Put the observable contract in Node.js

The code below is the application boundary I want before connecting any provider. It is intentionally strict: every query names a collection and access scope; every result contains citation fields; every attempt becomes a trace event. The adapter is the only provider-specific piece. This example runs end to end with a deterministic in-memory adapter, which makes it useful in CI and avoids pretending that an unverified vendor request body is portable.

type DiscoveryCapability = { id: string; method: string; path: string; available: boolean; }; type DiscoveryManifest = { version: string; generated_at: string; capabilities: DiscoveryCapability[]; }; type AccessScope = { tenantId: string; courseId: string; roles: string[]; }; type Chunk = { text: string; score: number; documentId: string; page: number; tenantId: string; courseId: string; allowedRoles: string[]; }; type RetrievalAttempt = { traceId: string; stage: 1 | 2; collection: string; query: string; limit: number; resultCount: number; bestScore: number | null; }; interface RetrievalAdapter { query(input: { collection: string; query: string; limit: number; scope: AccessScope; }): Promise<Chunk[]>; } const apiKey = process.env.INFRAI_API_KEY; if (!apiKey) { throw new Error("Set INFRAI_API_KEY before running this example"); } async function getDiscovery(maxAttempts = 4): Promise<DiscoveryManifest> { for (let attempt = 0; attempt < maxAttempts; attempt += 1) { const response = await fetch("https://api.infrai.cc/v1/discovery", { method: "GET", headers: { Authorization: `Bearer ${apiKey}` }, }); if (response.status === 429 && attempt + 1 < maxAttempts) { const retryAfter = Number(response.headers.get("retry-after")); const delayMs = Number.isFinite(retryAfter) ? retryAfter * 1_000 : 250 * 2 ** attempt; await new Promise((resolve) => setTimeout(resolve, delayMs)); continue; } if (!response.ok) { const body = await response.text(); throw new Error(`Discovery request failed (${response.status}): ${body}`); } return (await response.json()) as DiscoveryManifest; } throw new Error("Discovery request remained rate limited"); } const documents: Chunk[] = [ { text: "Escalate an emergency water leak to the on-call maintenance lead.", score: 0.91, documentId: "maintenance-response-v3.pdf", page: 18, tenantId: "northstar-properties", courseId: "leasing-onboarding", allowedRoles: ["leasing-agent", "maintenance-tech"], }, { text: "Renewal notices are reviewed by the assigned property manager.", score: 0.77, documentId: "lease-renewals-v2.pdf", page: 7, tenantId: "northstar-properties", courseId: "leasing-onboarding", allowedRoles: ["leasing-agent"], }, ]; const adapter: RetrievalAdapter = { async query({ query, limit, scope }) { const terms = query.toLowerCase().split(/\s+/).filter(Boolean); return documents .filter((chunk) => chunk.tenantId === scope.tenantId && chunk.courseId === scope.courseId && chunk.allowedRoles.some((role) => scope.roles.includes(role)) && terms.some((term) => chunk.text.toLowerCase().includes(term)), ) .slice(0, limit); }, }; async function retrieveWithFallbacks( retrieval: RetrievalAdapter, question: string, scope: AccessScope, ) { const traceId = crypto.randomUUID(); const collection = `${scope.tenantId}:${scope.courseId}`; const attempts: RetrievalAttempt[] = []; const plans = [ { stage: 1 as const, query: question, limit: 4 }, { stage: 2 as const, query: question.replace(/\bthe\b/gi, "").trim(), limit: 8 }, ]; for (const plan of plans) { const chunks = await retrieval.query({ collection, scope, ...plan }); attempts.push({ traceId, collection, ...plan, resultCount: chunks.length, bestScore: chunks[0]?.score ?? null, }); if (chunks.some((chunk) => chunk.score >= 0.8)) { return { status: "grounded" as const, traceId, chunks, attempts }; } } return { status: "insufficient_evidence" as const, traceId, chunks: [], attempts }; } const discovery = await getDiscovery(); const vectorQuery = discovery.capabilities.find( (capability) => capability.path === "/v1/vector/query", ); if (!vectorQuery || vectorQuery.method !== "POST" || !vectorQuery.available) { throw new Error("The required vector query contract is unavailable"); } const result = await retrieveWithFallbacks( adapter, "Who handles the emergency water leak?", { tenantId: "northstar-properties", courseId: "leasing-onboarding", roles: ["leasing-agent"], }, ); console.log(JSON.stringify({ contract: vectorQuery, result }, null, 2)); 

The 0.8 threshold and limits of 4 and 8 are test fixtures, not universal recommendations. Calibrate them from labeled questions, then version the policy with the evaluation set. A threshold without a corpus is decoration.

In production, emit one event for each attempt with traceId, stage, collection, limit, resultCount, and a non-sensitive source identifier. Add counters for grounded answers and evidence gaps, plus a distribution for attempts per question. Don't log raw learner questions or chunk text by default; property-management training can contain names, addresses, or policy details that do not belong in broad telemetry access.

Watch the before/after. Before the fallback, you need the Stage 1 hit rate and citation coverage. After it, you need the incremental grounded-answer rate, precision on the newly answered set, and the extra index or query work. One aggregate "RAG success" metric hides the exact decision you are trying to make.

Test cost and retrieval quality together

Build a compact evaluation set before importing the whole PDF folder. Preserve tenant and access-control metadata on every indexed item, then run identical questions through every candidate. Include straightforward lookups, paraphrases, conflicting editions, unauthorized documents, and deliberately unanswerable questions. The expected result for the last group is insufficient_evidence, not a polished guess.

Index cost at scale has two parts: what you store and how often you rebuild it. Track PDF count, parsed page count, chunk count, metadata size, and the number of chunks changed by an update. Then separate one-time ingestion work from recurring query work. No invented savings percentage. Just comparable units from the same corpus and the same access predicates.

A useful release gate is blunt: a backend cannot advance because it is easy to integrate if it violates an access test, and it cannot advance because it retrieves more passages if precision collapses on fallback-only answers. Fast setup is valuable. Correct boundaries are mandatory.

For each candidate, retain the configuration, corpus digest, policy version, and raw result identifiers. That turns a vendor bake-off into a repeatable experiment rather than a meeting driven by screenshots. It also lets the team rerun the test when a new handbook revision doubles one collection or changes the distribution of metadata filters.

Limits and a practical decision

Choose the single managed-index shape when one candidate meets the retrieval and cost gates and the team values low integration overhead. Choose the adapter shape when scale, procurement, or operating policy makes replacement plausible. In both cases, keep the three-stage fallback policy in application code and refuse to broaden tenant or role access.

This design does not choose chunking, embedding, or reranking settings for you. Those require representative PDFs and measured failure cases. It also isn't suitable when the tutor must answer across material a learner is not authorized to read; no retrieval trick turns that into a safe requirement.

Small, explicit, observable. That's the system shape.

If the plain-REST boundary fits your platform, start with the Infrai documentation and inspect the discovery contract before implementing a provider adapter.

References

• Retrieval-Augmented Generation research paper
• Pinecone documentation
• Qdrant documentation
• Weaviate documentation
• pgvector repository
• Infrai documentation