TL;DR: building a basic AI detector is easy, building one that works is difficult. we realised the need for an adversarial testing pipeline that constantly tests our detector with new patterns, and retrains adapter heads to guard against fresh humanizer patterns.
At first when we built t9xt.com, we trained our model to detect output from frontier models + popular open-source models. We used over 10m text samples from a variety of contexts and writing styles. We hit nearly 100% accuracy on OOF data really fast. But something didn’t seem right - it started bothering us, surely it couldn’t be that easy? and it wasn’t…
The more we tested our model on the raw web, the more it humbled us. Turns out we missed an entire category of text - humanized text. Now I’m sure most of you already know what that is - obscured AI text using techniques like substitution, elongation and superimposition. Often marketed as “beautifying writing”, it doesn’t sound like human writing (to humans) but most AI detectors failed here. So we went in to “crisis mode” and started generating and adding samples from all the humanizers we found, suddenly the water was muddy and the model accuracy nosedived to less than a coin toss ~ 45%. Something was definitely wrong. We had to redesign our model - took months of tinkering but we finally went back above 98% on OOF data (this deserves a separate post so I will spare you the details here). It was great, until it wasn’t - a few weeks later we got sent a sample by one of our early users, and sure enough this humanizer came back as human - and funny enough, the writing didn’t make sense when you read it - jumbled up prose and idiosyncratic text arrangement, it was novel and it hit the mark. When I asked chat to tell me what it meant, and it actually understood it really well - which got me thinking, maybe the writing was not meant for humans in the first place… and the best customers for humanizers, are probably SEO blog SaaS providers that want 500k words to push up their aggregate text on search engines.
A fresh search found over 30 new humanizers, where a few of them still came back as human. This seemed like a never ending loop (we added more data and retrained our model) - then it hit me, why wait for new humanizers? why not just build your own? So I started working on building humanizers for our training data…
The first one we managed to build, got past 60% of the time and the next one 30% - but more importantly we had found an automated way to generate new humanizers. We isolated 20 key base conjugations, wrote an orchestration script (when I say we I meant ChatGPT & I) that bounced off the output against our model in perpetuity studying probabilities and tweaking the pattern mix. We managed to identify 5 unique pattern pipes that went through 40-60% of the time. I was curious so I ran the output against most leading models - and surely enough, it came back as human/likely human over 50% of the time. Some models almost always got it back as human, while others that use a similar training technique to us - flagged it accurately only ~ 50% (which includes part of the text being flagged as human).
Not only did this exercise validate our hypothesis, it also helped emphasise the importance of finding new patterns that might slip through. As a result, we now do this supervised exercise every week and retrain attention heads if accuracy drops more than 3% on any given pattern.
The model of the story: don’t just build, constantly iterate slightly ahead of the curve.
Taken from: t9xt.substack.com/p/how-we-ended-up-building-a-set-of
Comments URL: https://news.ycombinator.com/item?id=50545764
Points: 1
# Comments: 0

