How We Built a Fake-Review Detector Using Wilson Confidence Intervals and Bayesian Inference

When we started building GoBuy, we had one question: can math tell you whether those 4.9-star reviews are real?

The answer is: not with certainty, but with confidence. And the difference between certainty and confidence is where the interesting engineering lives.

The Problem

Amazon has over 300 million products. Conservative estimates suggest 30-40% of reviews are fake, AI-generated, or incentivized. The problem is getting worse, not better — LLMs can generate thousands of convincing reviews in minutes.

Existing tools (Fakespot, ReviewMeta) use proprietary black-box methods. We wanted something transparent: a scoring system where every number is derivable from first principles.

The Stack: Three Mathematical Layers

Layer 1: Wilson Score Lower Bound

The foundation of every GoBuy Smart Score is the Wilson Score Interval — specifically, the lower bound of a binomial proportion confidence interval.

If a product has 500 reviews and 480 are positive (96%), the naive average is 0.96. But how confident are we that the true positive rate is 0.96? The Wilson interval answers this:

Lower bound = (p + z²/(2n) - z·√(p(1-p)/n + z²/(4n²))) / (1 + z²/n) 

Where:

• p = observed positive proportion (0.96)
• n = sample size (500)
• z = z-score for confidence level (1.96 for 95% confidence)

For our example: the Wilson lower bound is 0.938, not 0.96. That's a meaningful difference when comparing products.

Why this matters: A product with 10 reviews at 100% positive has a Wilson lower bound of 0.69 — barely better than a coin flip. A product with 1,000 reviews at 95% positive has a Wilson lower bound of 0.935. The math naturally penalizes small samples.

Layer 2: Bayesian Beta Posterior

Wilson gives us a snapshot. But reviews accumulate over time, and we want to update our beliefs incrementally. Enter the Beta-Binomial model.

We model each product's true quality as a draw from a Beta distribution:

Quality ~ Beta(α, β) 

Where α = positive reviews + 1 (prior pseudo-count) and β = negative reviews + 1.

The Beta posterior is conjugate to the Binomial likelihood, which means we can update it in closed form as new reviews arrive — no MCMC, no approximations, just arithmetic.

The posterior mean is α / (α + β) — which naturally shrinks toward 0.5 (uncertainty) when review counts are low, and converges to the true positive rate as data accumulates.

We use the 5th percentile of the posterior (the lower credible bound) as our quality estimate. This is more conservative than the posterior mean and provides the same "benefit of the doubt goes to skepticism" principle as the Wilson bound.

Layer 3: Temporal Anomaly Detection

A product with 2,000 genuine reviews and 500 fake reviews added in a 48-hour window should raise red flags — even if the overall positive rate looks fine.

We compute a review velocity signature: the rolling rate of review arrival over time, segmented by verified/unverified status and rating value. Sustained spikes in 5-star reviews from unverified purchasers are the strongest signal of coordinated manipulation.

We use a simple exponential weighted moving average (EWMA) of review sentiment over time. Products whose recent reviews are systematically more positive than their historical baseline receive a temporal penalty.

Putting It Together: The Smart Score

The final GoBuy Smart Score (0-100) is a weighted combination:

Every component is auditable. Every score is reproducible. No black boxes.

The MCP Layer for AI Agents

Here's where it gets interesting for developers. GoBuy exposes an MCP (Model Context Protocol) server that lets AI agents query trust scores in real-time:

result = await mcp_client.call_tool("gobuy_check_trust", { "product_url": "https://amazon.com/dp/B08X4ABCD" }) 

This means any AI shopping agent can make trust-aware decisions — filtering out suspicious products before recommending them to users.

What We Learned

1. Transparency beats complexity. Our scoring is simpler than Fakespot's, and that's a feature. Users and developers can understand exactly why a product scores 34/100.

2. Small samples are the enemy. The biggest mistake in review analysis is trusting products with few reviews. Wilson and Beta both handle this naturally — you don't need a complex model to know that 10 reviews aren't enough.

3. Temporal signals are critical. The pattern of when reviews arrive matters as much as what they say. A 48-hour 5-star burst is never organic.

4. AI agents need trust scores more than humans do. A human can read reviews critically. An AI agent consuming a product database cannot — it needs a pre-computed trust signal.

GoBuy is live at gobuy.ai. Check any Amazon product's trust score, or integrate our MCP server into your AI agent. API docs here.