Your competitor report can invent an equal-demand race

Three competitors can each hit 100 without having equal search interest.

You're wiring up a competitor report. Retrieval finds a peak score for each brand, the model calls the race a tie, and every citation points to a valid record. That's the annoying part: the evidence can be individually correct while the comparison is wrong.

Lab note 053: we wanted to know where a pipeline should stop that conclusion. The main caveat comes first: independently requested terms have independently normalized scores. Putting those records in the same prompt doesn't create a shared measurement scale.

Why do identical peaks invent a tie?

Independent keyword requests can't support a cross-brand score ranking; that ranking needs evidence from the same shared comparison row.

Maximum normalization preserves a series' shape while discarding the magnitude needed to compare it with another independently normalized series. Your model can know what normalization means and still treat adjacent numbers as comparable. Retrieval has quietly supplied the wrong relationship.

For this worked example, the Actor has separate collection paths. Independent mode returns a row per term. Comparison mode places up to five terms on a shared 0-100 scale in a single row.

Neither path supplies absolute search counts. Even a valid comparison supports a statement about relative search interest within its settings, rather than market share or purchasing demand. We'd rather leave “demand leader” out than dress it in citations.

Give every row a scale identity

We'd put provenance around the returned row before indexing it.

The wrapper below belongs to your application. Its scale_id identifies a source row, and its manifest records the requested cohort and measurement settings. Neither is an extra Actor output field.

Example: this request defines a synthetic competitor cohort; the wrapper also rejects inputs whose effective settings it can't safely establish.

import json import math import os from dataclasses import dataclass from apify_client import ApifyClient run_input = { "mode": "interest", "searchTerms": ["term alpha", "term beta", "term gamma"], "compareMode": True, "geo": "", "timeRange": "today 12-m", "category": "0", "searchType": "", "maxItems": 1, } @dataclass(frozen=True) class Evidence: scale_id: str manifest: str row: dict def wrap(scale_id, request, row): if request.get("startUrls"): raise ValueError("Resolve URL overrides before creating evidence") if request.get("mode", "interest") != "interest": raise ValueError("This pipeline accepts interest requests") terms = request.get("searchTerms", []) shared = request.get("compareMode", False) if ( not isinstance(terms, list) or not terms or any(not isinstance(term, str) or not term.strip() for term in terms) or len(set(terms)) != len(terms) or not isinstance(shared, bool) ): raise ValueError("Expected distinct terms and a boolean compareMode") if shared and len(terms) > 5: raise ValueError("Comparison cohort would be trimmed") if not shared and len(terms) != 1: raise ValueError("Submit independent terms separately for attribution") defaults = { "mode": "interest", "geo": "", "timeRange": "today 12-m", "customDateRange": "", "category": "0", "searchType": "", } manifest = {key: request.get(key, default) for key, default in defaults.items()} manifest.update(searchTerms=list(terms), compareMode=shared) return Evidence(scale_id, json.dumps(manifest, sort_keys=True), row) 

The URL rejection matters. Settings in startUrls override the input fields, so copying the surrounding request into metadata can attach a convincing but incorrect measurement context. This implementation declines that path until you resolve those overrides.

The independent-request restriction is also our ingestion policy. Submitting each term separately gives the wrapper unambiguous attribution without relying on dataset position.

Rejecting oversized comparison requests prevents another quiet error: additional terms are trimmed. A manifest claiming an entire requested cohort would then describe more than the shared row actually represents.

Block the ranking before the prompt

A prompt saying “be careful with normalized data” leaves the model to infer provenance from prose. We'd make admission mechanical.

The gate permits score comparisons only when every candidate comes from the same shared source row and manifest.

Example: these synthetic records define independent controls and a shared comparison control.

def can_rank(group): if not group: return False first = group[0] return all( json.loads(e.manifest)["compareMode"] is True and e.scale_id == first.scale_id and e.manifest == first.manifest for e in group ) independent = [ wrap( f"example-independent-{term}", {**run_input, "searchTerms": [term], "compareMode": False}, {"recordType": "interest", "peakInterest": 100}, ) for term in run_input["searchTerms"] ] shared = wrap( "example-shared-row", run_input, { "recordType": "interest", "interestOverTime": [ { "date": "2026-10-01", "values": [100, 48, 24], "isPartial": False, } ], }, ) assert not can_rank(independent) assert can_rank([shared]) 

The assertions encode the boundary. Identical independent peaks must fail admission. A shared row can proceed to point validation.

Matching geography, window, category and search surface doesn't rescue independent scales. Those settings are necessary context, but they don't supply the missing normalization relationship.

Overlapping comparison batches also remain separate. A common term can be useful in a carefully justified calibration method, but overlap alone doesn't authorize stitching batches into a larger leaderboard.

Missing points aren't losing brands

A shared scale fixes comparability, not coverage.

Sparse terms can have empty values because points are returned only where sufficient search data exists. Unservable queries are skipped, and empty fields are omitted. Replacing absence with a numeric score converts “we lack evidence” into “this term lost.”

The timeline also marks the current period with isPartial. Including an unfinished period in a closed-period comparison can make a brand appear to decline simply because the bucket isn't complete.

This filter keeps complete comparison vectors from finished periods and leaves missing evidence missing.

def usable_points(evidence): if not can_rank([evidence]): return [] points = evidence.row.get("interestOverTime") if not isinstance(points, list): return [] width = len(json.loads(evidence.manifest)["searchTerms"]) accepted = [] for point in points: if not isinstance(point, dict): continue if point.get("isPartial", False) is not False: continue date = point.get("date") values = point.get("values") if not isinstance(date, str) or not date: continue if not isinstance(values, list) or len(values) != width: continue if not all( isinstance(value, (int, float)) and not isinstance(value, bool) and 0 <= value <= 100 and math.isfinite(value) for value in values ): continue accepted.append(point) return accepted 

Dropping an incomplete vector is deliberately conservative: it gives every retained comparison point the same cohort coverage. Don't silently turn those retained points into a window-wide average. That would describe the surviving observations, which may differ from the requested window.

For independent trend-shape analysis, use a separate retrieval path that keeps each term's scale explicit. This filter serves comparative reports.

Every chunk needs its cohort

A perfectly wrapped document can still lose its meaning during chunking.

A header-only manifest is fragile. Retrieval may return a timeline fragment without the header, or a later summarizer may preserve the numbers while dropping their scope.

request settings + returned row | provenance wrapper | complete comparison point | cohort + settings + source identity + values | retrieval excerpt 

We'd serialize the context into every excerpt and reject an oversized excerpt as a unit. Slicing the text until it fits can remove the very metadata that makes its scores interpretable.

This filter also keeps unrelated payloads out of the ranking index, so related-query text and region breakdowns don't consume the evidence budget for a timeline question.

def serialize_excerpt(evidence, point, max_chars): if not can_rank([evidence]): return None payload = { **json.loads(evidence.manifest), "exploreUrl": evidence.row.get("exploreUrl"), "interestOverTime": [point], } text = ( f"Comparison row: {evidence.scale_id}\n" + json.dumps(payload, ensure_ascii=False, separators=(",", ":")) ) return text if len(text) <= max_chars else None 

The character ceiling is an excerpt-size policy, not a token guarantee. Apply your model's tokenizer to the assembled prompt when enforcing its actual budget.

For deduplication, retain source identity and measurement context. Hashing only a score or a stripped timeline can merge evidence whose numbers match but whose meanings differ. Preserve cohort order too; alphabetizing labels separately from a value vector is a remarkably efficient way to misattribute evidence.

Admit complete evidence, then let the model write

Here's where the filters meet the Google Trends Interest & Related Query Scraper.

The combined path collects the shared cohort, checks the run status, and builds index entries only from admissible points.

client = ApifyClient(os.environ["APIFY_TOKEN"]) run = client.actor( "vexaloom-labs/google-trends-interest-and-related-query-scraper" ).call(run_input=run_input) if run["status"] != "SUCCEEDED": raise RuntimeError(f"Collection ended with {run['status']}") items = client.dataset(run["defaultDatasetId"]).list_items().items index_entries = [] for offset, row in enumerate(items): if row.get("recordType") != "interest": continue evidence = wrap( f"{run['defaultDatasetId']}:{offset}", run_input, row ) for point in usable_points(evidence): excerpt = serialize_excerpt(evidence, point, max_chars=4000) if excerpt is not None: index_entries.append( (evidence.scale_id, point["date"], evidence.manifest, excerpt) ) 

Persist the wrapper metadata alongside the indexed text. When retrieval assembles a comparative answer, apply can_rank again to the source evidence it selected. Otherwise, independently valid shared cohorts can still arrive together and tempt the model into a broader ranking.

If admission leaves no comparable evidence, the report should say that comparison is unavailable. A reassuring sentence with incompatible citations is still wrong.

Our verdict: shared comparison rows can support relative-interest comparisons within their context. Independent peaks can't support a competitor ranking, however tidy the table looks.

Reproduce the boundary

• Execute the synthetic controls and inspect the admission conditions. Change the shared row's identity and confirm that candidates from different rows can't form a comparison group.
• Supply APIFY_TOKEN through the environment and execute the collection cell.
• Inspect the retained excerpts for the complete cohort, measurement settings and source identity. Use exploreUrl to spot-check the underlying view.
• Keep the requested comparison scope when writing the report. An empty evidence set should remain an explicit gap.

Example: use this exact Actor input for the collection path.

{ "mode": "interest", "searchTerms": ["term alpha", "term beta", "term gamma"], "compareMode": true, "geo": "", "timeRange": "today 12-m", "category": "0", "searchType": "", "maxItems": 1 }