People have started telling ChatGPT, Gemini or Claude "find me X under $Y and order it." The agent reads your site, compares you with the others, and decides. If it can't find your price or tell whether something is a subscription, it skips you, and you never learn you lost a customer.

This isn't a someday problem. Traffic from AI sources to US retail sites rose 393% year over year in Q1 2026, and those visits converted 42% better than other traffic (Adobe, via CMOtech). Morgan Stanley expects AI agents to make 10% to 20% of US online shopping spending by 2030 (Digital Commerce 360).

So at HackPrinceton Fall 2026 we built Will Agents Buy?: paste a website, and real AI shopping agents plus AI customers with real preferences open it in a real browser, try to buy, talk about it, and tell the owner what to fix.

Two layers, two levels of trust

The sharpest feedback we got at the hackathon was that AI personas aren't people. That's right, and we don't pretend otherwise. So the product has two layers:

  • AI shopping agents. One each from Gemini, Claude and GPT, sent on the errand a real user would give them. When the shopper is an AI agent, testing with the actual agents isn't a simulation. It's the measurement.
  • AI customers. A price-sensitive student, a skeptic, a first-timer, each with a budget, a goal and a share of the customer base. This layer is a fast first pass, and we grade it against real data (more below).

On our demo listing (a training mannequin head from the public Amazon Reviews 2023 dataset), the Gemini and GPT agents bought it. The Claude agent refused: "the checkout page showed no shipping, tax, return, or delivery terms, so I couldn't verify the real total." That's a fix an owner can ship today.

How a shopper works

Every shopper is a LangChain agent with its own model and its own Chromium browser (Playwright). Each step it gets a screenshot plus a list of things it can click, thinks in character, and picks one action: click, scroll, back, search, add to cart, or decide. Every step is saved with the screenshot, the thought and the exact prompt, so you can trace any decision.

When a director agent runs the test (a LangChain Deep Agents create_deep_agent), each shopper becomes a subagent, several shop at once through the task tool, and the run pauses for the owner to approve the panel with an interrupt.

Bug 1: every site said 0 of 7 would buy

Our first runs said nobody would buy, on every website. It looked like a damning insight. It was our bug.

Most sites render their nav twice: once for desktop, once hidden for mobile. We handed the model every <a> on the page. It picked "Shop", Playwright's locator resolved to the hidden mobile copy, the click timed out, and every customer circled the homepage until they gave up.

The fix: only offer what a person can actually see, screen first.

const els = [...document.querySelectorAll('a[href], button, label, summary, [role=button], [role=link], [role=tab], [role=checkbox], input[type=submit]')]; // what's on screen first, then the rest of the page els.sort((a, b) => { const ra = a.getBoundingClientRect(), rb = b.getBoundingClientRect(); return (ra.top > innerHeight || ra.bottom < 0) - (rb.top > innerHeight || rb.bottom < 0); }); for (const e of els) { const r = e.getBoundingClientRect(), st = getComputedStyle(e); if (r.width < 2 || r.height < 2 || st.visibility === 'hidden' || st.display === 'none' || st.opacity === '0') continue; // ...keep its text, and its href so a link goes where it actually points }

If the chosen text maps to a real link, we goto the href instead of clicking. We also follow links that open a new tab, the way a person would.

Bug 2: bot walls blamed the business

Plenty of sites show "verify you are human" to automated browsers and quietly empty their carts. Our customers then wrote angry reviews of stores that did nothing wrong. Now we detect bot walls, exclude those steps, and tell the shopper an empty cart is our test setup, not the store.

Bug 3: what counts as a sale

Models hedge. One customer picked a size, opened the cart, and then said "maybe later." First we counted any add-to-cart click as a sale. That inflated everything. Now the outcome is strict:

  • On a real website our shoppers never pay, so a sale is a final decision of "buy".
  • On a store we generate from a product description, checkout needs no payment, so a sale means actually placing the order.
  • Cart, then "maybe later", is an abandoned cart, not a sale.

Bug 4: Apple's configurator

On apple.com every shopper got stuck clicking "256GB" over and over. Storage is a disabled radio button until you pick a color, so the click just timed out, and the model only heard "it didn't work." Now, when a click fails, we check whether the target is disabled and say why:

DISABLED_JS = """t => { t = t.toLowerCase(); const hit = [...document.querySelectorAll('label, button, [role=radio], [role=option], a')] .filter(e => (e.innerText || '').toLowerCase().includes(t)); return hit.some(e => { const i = e.control || e.querySelector('input') || (e.htmlFor && document.getElementById(e.htmlFor)); return e.disabled || e.getAttribute('aria-disabled') === 'true' || (i && i.disabled); }); }"""

The shopper hears: "it's greyed out, it unlocks after an earlier choice, pick the color first." A failed-click message that names the reason beats a generic error in any LLM tool loop.

Checking ourselves against real reviews

We wanted a number we didn't tune to look good. We took that mannequin head (881 reviews), split the reviews by date, rebuilt the listing without any reviews, and let our customers shop it.

  • Cold, they named 3 of the 6 themes real reviewers wrote about.
  • After learning from the older half, they named 5 of 6, scored only on the newer reviews they never saw.
  • Their star ratings stayed 1.4 to 2 stars too harsh. We report that instead of hiding it.

So the honest pitch is "they find what your real buyers will complain about", not "they predict your star rating". Not yet.

What spreads

After shopping, customers post about their visit, then reply, repost or scroll past each other's posts. We build the who-reacted-to-whom graph and run PageRank (damping 0.85) to find whose opinion travels furthest. On the mannequin head, the doubt that spread was whether the hair is really human, so the top fix became "state one hair material and add proof."

Gemini then looks at the exact screenshots where customers got stuck and draws boxes around the part of the page that caused it.

Stack

  • LangChain + Deep Agents for the shoppers, the director and its subagents, with long-term customer memory in a LangGraph store
  • Playwright for one real browser per shopper
  • Gemini, Claude and GPT as shoppers, with Gemini also running the screenshot lens
  • MongoDB Atlas with Vector Search for memory that carries across businesses
  • NetworkX for the word-of-mouth graph
  • Photon so an owner can run a test over iMessage
  • LangSmith tracing, Vultr for deploy, React + TypeScript for the app

What we learned

  • Simulated customers are pessimistic by default. Most of the work was removing reasons to say no that a real person would never hit: hidden links, bot walls, dead buttons, configurators.
  • What an AI agent needs is different from what a person needs. A page can win people over with photos and still lose the agent because the total isn't written down.
  • Measuring accuracy changed our pitch, for the better.

Built at HackPrinceton Fall 2026 with Sarah and Jayden.
Devpost: https://devpost.com/software/will-agents-buy