On DEV, we measure posts in hearts and comments.
AI assistants measure them in something else entirely, and I suspect the two lists barely overlap.

This post is a hypothesis and a method, not a result. I want to know if you think the method is sound before anyone spends a weekend on it.

The hypothesis

The DEV posts with the most reactions are not, mostly, the ones AI assistants cite when answering related questions.

Why I think so:

  • Reactions reward timing and community. A post that lands on a busy day, from an author with followers, about something everyone is feeling this week.
  • Citations reward answerability. A post that contains a clear, self-contained answer to a question someone will type months later.

Opinion posts and discussions (the format that dominates the top of DEV) are great for conversation and hard to quote as an answer. Narrow how-tos with a boring title may be the opposite.

If that's true, writers optimizing only for reactions are optimizing for one audience and ignoring a growing second one.

Step 1: Sample posts with the public API

The Forem API (the software behind DEV) has a public articles endpoint that can return top posts for a period, with reaction and comment counts. No key needed for reading published articles.

Plan: take the top 100 posts by reactions over a fixed period, and a comparison set of 100 posts from the same tags with low reactions. Freeze the list and save the raw response with a date.

Step 2: Turn each post into a question

For each post, write one question a real person might ask an assistant, without using the post's title or distinctive phrases. For a post about debouncing in React, the question might be "how do I stop a search box from firing on every keystroke in React?".

This is the most subjective step. Two mitigations:

  • Have two people write questions independently and keep both.
  • Publish the full question list.

Step 3: Ask the assistants

For each question, on each assistant with web access:

  • fresh session, same wording, search enabled
  • three runs, on different days
  • record every cited URL

Then mark whether the original DEV post (or any DEV post) appears among the citations.

Step 4: Compare the two rankings

For each post we'd have two numbers: reactions, and how many of the runs cited it. The simplest useful comparison:

  • Rank correlation (Spearman) between reactions and citation count. Near zero would support the hypothesis.
  • High-reaction vs low-reaction set: citation rate in each. If the low-reaction set is cited about as often, reactions aren't predicting citations.
  • Format breakdown: discussion, opinion, how-to, list, experiment. Which formats get cited?

What would falsify it

If the top-reaction posts are cited clearly more often than the low-reaction set, the hypothesis is wrong: popularity (or what causes it) also drives citation. That would be a useful result too, and I'd write it up exactly as clearly.

Confounders I can already see

  • Age: older posts have had more time to be indexed and linked.
  • Backlinks: popular posts get linked elsewhere, which may drive citations independently.
  • Question bias: my questions might unintentionally match some posts better.
  • Assistant differences: results may differ heavily per assistant; report them separately.

What writers can do in the meantime

Even without results, there's a low-cost hedge: write posts that work for both audiences.

  • Keep the hook and the opinion for humans.
  • Add one section that answers a specific question in a self-contained way, under a heading phrased like the question.
  • Put a one-sentence summary near the top that someone could quote without the rest of the post.

You keep the comments, and you give an assistant something to lift.

This is also just good writing. A reader who lands on your post from a search six months later has the same need as the assistant: they want the answer to their question quickly, and then, if you've earned it, they'll stay for the opinion. Writing for the late reader is writing for both.

A minimal script for step 1

To show how low the barrier is, here's roughly what sampling looks like. It reads public data only and saves the raw response so the sample can't drift:

const res = await fetch("https://dev.to/api/articles?top=30&per_page=100"); const posts = await res.json(); const rows = posts.map((p) => ({ id: p.id, title: p.title, url: p.url, tags: p.tag_list, reactions: p.public_reactions_count, comments: p.comments_count, reading_time: p.reading_time_minutes, })); await Deno.writeTextFile("sample.json", JSON.stringify(rows, null, 2));

(Swap the last line for your runtime's file API.) Be polite with the API: one request is enough for this, and there's no reason to hammer it.

The low-reaction comparison set takes a bit more care, because you want posts from the same tags and period. Filtering a larger tag-specific sample works fine.

Takeaways

  1. Reactions measure the community's response this week; citations measure answerability over time.
  2. The Forem API makes a reproducible sample possible for free.
  3. Write questions without using the post's own words, and publish them.
  4. Compare high- and low-reaction sets, not just a single ranking.
  5. Hedge now: a quotable section under a question-shaped heading costs nothing.

Your turn

What's a post of yours with almost no reactions that you suspect quietly gets traffic from search or AI? And what's the biggest hole in this method?

I write about practice, feedback and the AI-readable web while building NEXO, a daily learning game.