Teaching Sidekick to say no: automated data curation with LLM judge consensus

Shopify Engineering2026-06-16 03:300 views

Production training data only captures successful queries; it can't teach a model when to say no. We built an automated curation pipeline using LLM judge consensus to close that gap.

Source：https://shopify.engineering/sidekick-curation

My favorites

PreviousClustering billions of products for agentic commerce with Catalog API NextQuick: An internal hosting platform for the AI era

Как ChatGPT подтолкнул Боуэн Вань отказаться от лекарств и почему чат-боты соглашаются с пользователем

Иск родителей Боуэн Вань к OpenAI показывает, чем может обернуться медицинский совет чат-бота. Из материалов дела и пере

0dev.to AI

ViKernel: VIDRAFT's Weight-Native LLM OS with Built-In Compute Circuits and a Kill Switch

ViKernel: VIDRAFT's Weight-Native LLM OS with Built-In Compute Circuits and a Kill Switch TL;DR: VIDRAFT has open-demoed

0dev.to AI

Your LLM Costs Are Growing. Here's What to Fix First.

LLM costs can look manageable during development. Then the application goes into production, more users arrive, and one

0dev.to DevOps

Sources: Jensen Huang had an extramarital relationship with Nvidia's VP of healthcare, Kimberly Powell, who reports to him; it was discussed inside the company (New York Times)

New York Times : Sources: Jensen Huang had an extramarital relationship with Nvidia's VP of healthcare, Kimberly Powell,

0Techmeme

Sources: Nvidia's board is preparing to oust Jensen Huang; it plans to meet as soon as Wednesday to determine his future (Wall Street Journal)

Wall Street Journal : Sources: Nvidia's board is preparing to oust Jensen Huang; it plans to meet as soon as Wednesday t

0Techmeme

Sources: Scott Bessent confronted Howard Lutnick last week over the Commerce Dept. not adding Megaspeed to the Entity List; Commerce denies the confrontation (Axios)

Axios : Sources: Scott Bessent confronted Howard Lutnick last week over the Commerce Dept. not adding Megaspeed to the E

0Techmeme

Teaching Sidekick to say no: automated data curation with LLM judge consensus

Related posts

Как ChatGPT подтолкнул Боуэн Вань отказаться от лекарств и почему чат-боты соглашаются с пользователем

ViKernel: VIDRAFT's Weight-Native LLM OS with Built-In Compute Circuits and a Kill Switch

Your LLM Costs Are Growing. Here's What to Fix First.

Sources: Jensen Huang had an extramarital relationship with Nvidia's VP of healthcare, Kimberly Powell, who reports to him; it was discussed inside the company (New York Times)

Sources: Nvidia's board is preparing to oust Jensen Huang; it plans to meet as soon as Wednesday to determine his future (Wall Street Journal)

Sources: Scott Bessent confronted Howard Lutnick last week over the Commerce Dept. not adding Megaspeed to the Entity List; Commerce denies the confrontation (Axios)