Is Your RAG Answer Turning a Hearing Into a Verdict?

Your chunk boundary can turn a hearing notice into a verdict. You're indexing regulatory histories, and the answer draft says “found responsible” where the retrieved passage only describes an allegation.

The catch arrives before tokenization: the current GCC profile supplies a free-text action description and a decision link. That link doesn't put the decision's contents in your index.

Lab note 107: we want to answer a debugging question. Is the retrieval unit preserving a published action, or quietly manufacturing an outcome label?

Is the passage a hearing or a finding?

Consider the temptation to turn every regulatory-history entry into a classification target. There are fields named finding and outcome in the older field inventory. Convenient names. Very inconvenient assumptions.

Those older structured fields are no longer published and are omitted. The Actor captures the current narrative through description and decisionUrl.

Example synthetic fixture: this fictional notice gives the serializer something to process without implying anything about a real practitioner.

example_records = [{ "registrationNumber": "00000", "fullName": "Example Practitioner", "registrationStatus": "Registered - Practising", "sourceUrl": "https://example.com/sourceurl/1", "scrapedAt": "2026-10-01T12:00:00Z", "hasRegulatoryAction": True, "regulatoryActionCount": 1, "regulatoryActions": [{ "title": "Example hearing notice", "actionDate": "2026-10-01", "description": ( "Example only: a hearing is scheduled to consider an allegation. " "This notice does not state a finding." ), "decisionUrl": "https://example.com/decisionurl/1" }] }] 

A classifier could assign that description a label. The source hasn't supplied one.

We'd rather leave the field out than let a convenient schema become an accusation.

The action is the evidence unit

A published regulatory action is a narrative evidence unit, not a ready-made outcome label.

The corpus should preserve its title, action date, description, practitioner identity, and decision pointer together. Allegations, procedural steps, findings, and sanctions can appear in the same narrative; their proximity doesn't make them equivalent.

Here's the boundary we'd use:

Keep current registration status out of the historical action's embedding text. Otherwise, a current profile label can become the apparent outcome of an older hearing.

For supervised labeling, attach an annotation to the complete action and its evidence version. A truncated chunk is a poor place to decide what the source formally established.

Keep the parent outside the chunk boundary

The source structure is a practitioner with published actions. Preserve that relationship even when the retrieval index needs smaller passages.

Practitioner: registrationNumber └─ Published action: title + date + complete description ├─ Retrieval chunk + repeated identity and action context └─ Retrieval chunk + repeated identity and action context Profile citation → sourceUrl Decision pointer → decisionUrl 

hasRegulatoryAction means at least an action is published. It doesn't encode misconduct, severity, or a formal outcome.

Likewise, a profile without action entries contributes no action documents to this corpus. That's an ingestion decision, not a “cleared” label.

Keep the parent key in metadata rather than relying on the model to recognize the practitioner's name. Names are text; registration numbers are the join.

Deduplicate narratives, not people

Repeated exports shouldn't multiply the same evidence. Different actions belonging to the same practitioner shouldn't disappear into a person-level deduplication key, either.

The following builder normalizes whitespace, fingerprints the attributed narrative, and retains distinct source observations. A changed description or decision pointer becomes a different evidence version.

The identifiers, observations, audit counters, and split labels below are fields we create for the corpus.

import hashlib import json import tiktoken encoding = tiktoken.get_encoding("cl100k_base") TOKEN_BUDGET = 512 SPLIT_BUCKETS = 10 def clean(value): return " ".join(str(value or "").split()) def digest(value): payload = json.dumps(value, ensure_ascii=False, sort_keys=True) return hashlib.sha256(payload.encode("utf-8")).hexdigest() def build_actions(records): documents = {} audit = {"profiles_without_actions": 0, "empty_descriptions": 0} for profile in records: actions = profile.get("regulatoryActions") or [] if not actions: audit["profiles_without_actions"] += 1 continue parent = clean(profile.get("registrationNumber")) source = clean(profile.get("sourceUrl")) if not parent or not source: raise ValueError("Action evidence needs a parent and profile citation.") parent = parent.zfill(5) for action in actions: description = clean(action.get("description")) if not description: audit["empty_descriptions"] += 1 continue evidence = { "registrationNumber": parent, "fullName": clean(profile.get("fullName")), "title": clean(action.get("title")), "actionDate": clean(action.get("actionDate")), "description": description, "decisionUrl": clean(action.get("decisionUrl")), } action_id = digest(evidence) if action_id not in documents: bucket = int(digest(parent), 16) % SPLIT_BUCKETS documents[action_id] = { "action_id": action_id, **evidence, "split": "test" if bucket == 0 else "train", "observations": [], } observation = { "sourceUrl": source, "scrapedAt": profile.get("scrapedAt"), "registrationStatus": profile.get("registrationStatus"), } observations = documents[action_id]["observations"] if observation not in observations: observations.append(observation) return list(documents.values()), audit 

Notice what isn't a duplicate key: scrapedAt. A later collection can observe unchanged evidence.

Also notice what is retained: practitioner identity. Identical boilerplate attributed to different people remains different evidence. Global text deduplication would save space by deleting attribution. An efficient way to build the wrong corpus.

The audit distinguishes absent action entries from descriptions that can't supply narrative text. Neither becomes a negative training example.

Fit the context inside the token budget

Keep the complete action in an action store. Build retrieval chunks from it, repeating identity, title, and date whenever the description needs splitting.

The budget applies to the entire embedding string, including that repeated context. The encoding should match your embedding pipeline; the encoding and budget here are explicit choices.

This writer preserves contiguous spans of the normalized description. It prefers a whitespace boundary when that boundary still fits and raises rather than truncating oversized context.

def action_chunks(document): prefix = ( f"Practitioner: {document['fullName']} " f"[{document['registrationNumber']}]\n" f"Action: {document['title'] or '[not published]'}\n" f"Date: {document['actionDate'] or '[not published]'}\n" "Description:\n" ) if len(encoding.encode(prefix)) >= TOKEN_BUDGET: raise ValueError("Action context exceeds the token budget.") body = document["description"] start = 0 while start < len(body): low, high, stop = start + 1, len(body), start while low <= high: middle = (low + high) // 2 candidate = prefix + body[start:middle] if len(encoding.encode(candidate)) <= TOKEN_BUDGET: stop = middle low = middle + 1 else: high = middle - 1 if stop == start: raise ValueError("No description span fits with its action context.") if stop < len(body): boundary = body.rfind(" ", start + 1, stop) if boundary != -1: proposed = boundary + 1 if len(encoding.encode(prefix + body[start:proposed])) <= TOKEN_BUDGET: stop = proposed text = prefix + body[start:stop] yield { "chunk_id": digest([document["action_id"], start, stop]), "action_id": document["action_id"], "registrationNumber": document["registrationNumber"], "split": document["split"], "char_start": start, "char_end": stop, "text": text, "token_count": len(encoding.encode(text)), } start = stop def write_corpus(records, stem): actions, audit = build_actions(records) with ( open(f"{stem}.actions.jsonl", "w", encoding="utf-8") as action_file, open(f"{stem}.chunks.jsonl", "w", encoding="utf-8") as chunk_file, ): for document in actions: action_file.write(json.dumps(document, ensure_ascii=False) + "\n") for chunk in action_chunks(document): chunk_file.write(json.dumps(chunk, ensure_ascii=False) + "\n") return {"actions": len(actions), **audit} example_audit = write_corpus(example_records, "example") 

Character offsets refer to the whitespace-normalized description stored in the action file. They aren't offsets into the source page.

Whitespace boundaries also aren't semantic boundaries. A long allegation can still cross chunks. Use the retrieved action_id to recover the complete action for answer generation, with a separate generation budget. If that evidence won't fit, the answer needs to acknowledge incomplete context.

What the answer layer still can't claim

A decision URL is a retrieval pointer. Until a separate ingestion step fetches and processes that document, the answer can cite the profile narrative but can't claim to have read the decision.

Give the answer layer an explicit claim-type instruction:

Treat retrieved descriptions as attributed regulatory narratives. Distinguish allegations, scheduled proceedings, findings, and sanctions only where the supplied text explicitly supports them. Do not infer an outcome from a title, current registration status, or decision URL. State when decision text is unavailable.

That instruction helps express the evidence boundary. It doesn't enforce it. Evaluate answers for unsupported upgrades from “alleged” or “scheduled” to “found.”

Privacy needs another boundary. The allowlist excludes practice addresses, phone numbers, and email fields, but the narrative itself may contain personal information. Names and registration numbers also remain deliberately identifying.

Review descriptions for patient details and other unnecessary identifiers before sharing a corpus. Don't call this serializer an anonymizer.

Keep related actions in the same split

Random chunk splitting leaks the action straight into evaluation. Action-level splitting can still leak a practitioner's other actions or an earlier version of the same narrative.

The code assigns partitions using the parent key, so a practitioner's actions and revisions stay together. That supports evaluating retrieval on held-out practitioners; it doesn't make every evaluation independent.

Shared decision documents and copied narrative passages can cross practitioner boundaries. For a stricter evaluation, group those related records before assigning partitions, and inspect text overlap.

Temporal evaluation needs collection history too. actionDate describes the action; it doesn't establish when the narrative first became available to your system. A snapshot collected later can't recreate an earlier knowledge state.

The worked example is the General Chiropractic Council Register Scraper. Its fetchDetails option opens profiles for regulatory-action data; list-only records can't supply this corpus.

Our verdict: index the attributed narrative, and make every stronger claim earn its evidence.

Reproduce the corpus build

Install apify-client and tiktoken, then execute the cells above to serialize the synthetic fixture. Inspect example_audit and both generated JSONL files.

For live collection, set APIFY_TOKEN in your environment and run this cell. It writes separate action and retrieval files and leaves linked decisions untouched.

Example collection input: these settings include non-practising and removed or suspended profiles within the emitted-record cap.

import os from apify_client import ApifyClient run_input = { "mode": "fullRegister", "includeNonPractising": True, "includeRemovedSuspended": True, "fetchDetails": True, "maxItems": 50, } client = ApifyClient(os.environ["APIFY_TOKEN"]) run = client.actor( "vexaloom-labs/general-chiropractic-council-register-scraper" ).call(run_input=run_input) if run["status"] != "SUCCEEDED": raise RuntimeError(f"Collection ended with status {run['status']}") items = client.dataset(run["defaultDatasetId"]).list_items().items corpus_audit = write_corpus(items, "gcc") 

Inspect the audit before indexing. Embed only each chunk's text; resolve its action_id against the action file for citations and complete context.

Example exact Actor input for the collection cell:

{ "mode": "fullRegister", "includeNonPractising": true, "includeRemovedSuspended": true, "fetchDetails": true, "maxItems": 50 }