Your article engagement rate measures the announcement

The engagementRate beside an article title won't tell you whether anyone read its text.

You're ranking articles for the weekly report, and that column looks ready to sort. The awkward part comes when your manager asks what the percentage actually divides.

engagementRate measures announcement-post likes, retweets and replies divided by known author followers, expressed as a percentage. It measures neither article reading nor every engagement metric in the row, and its availability depends on knowing the follower count.

A perfectly valid percentage can still answer the wrong question.

The announcement is the observation

Dataspryva's article collector combines article metadata, announcement-post metrics and an author profile. Those belong to different analytical units, even when they arrive together.

Start from the post whose card opens the article. Raw article links can't be mapped without signing in and produce an articleLinkUnmapped status row.

This cell requests profile enrichment and collects every dataset item through the streaming iterator. Example input:

import os import pandas as pd import duckdb from apify_client import ApifyClient client = ApifyClient(os.environ["APIFY_TOKEN"]) run_input = { "tweetUrls": [ "https://x.com/vxunderground/status/1979885711579865307" ], "includeAuthorProfile": True, "includeAnnouncementPost": True, "includeArticleBody": False, } run = client.actor( "dataspryva/x-article-announcement-post-metadata-collector" ).call(run_input=run_input) if run["status"] != "SUCCEEDED": raise RuntimeError(f"Collection ended with {run['status']}") items = [] for item in client.dataset(run["defaultDatasetId"]).iterate_items(): items.append(item) 

A successful run isn't a promise that every input became an article row. Status rows describe collection outcomes, and a bodyUnavailable status can accompany retained article metadata.

Counting dataset items as articles would therefore mix observations with messages about observations. The spreadsheet will accept that arrangement without complaint. Spreadsheets are accommodating like that.

Which fields belong in the ranking?

Keep the selected fields in these compartments before calculating anything:

article identity | +-- announcement post --> reactions and post views | +-- author profile ----> follower denominator collection timestamp ------> when this observation was collected 

The compact map below separates the returned values from the pandas types we'll use. These are storage choices for analysis: the field list doesn't specify a wire type for every metric.

Keep identifier strings intact. Arithmetic on an article ID has no analytical meaning, and a numeric round trip can damage the key you need for a join.

The count fields also aren't interchangeable ingredients. Adding other recorded reactions creates a different numerator. That may suit your research question, but it needs its own label.

An absent count isn't a quiet audience

Empty fields are omitted rather than returned as null, empty strings, empty lists or empty objects.

That makes absence meaningful, but it doesn't identify its cause. An omitted reaction count doesn't establish that nobody reacted. A missing follower count doesn't describe an author with no followers.

For most fields in this comparison, absence alone doesn't supply a more specific explanation. Keep that uncertainty rather than assigning a convenient story.

There's another distinction worth preserving: a supplied rate and an independently reproducible rate aren't the same population. The sample includes a rate without every numerator component alongside it.

You can't recover the individual missing components by rearranging the formula. Several combinations could fit, and rounding adds uncertainty.

Our audit therefore requires a complete numerator and a positive denominator. That's a stricter eligibility rule than the condition for a supplied rate.

Give missing values a place to stay

This loader separates row types, creates missing columns explicitly and parses timestamps as UTC. It rejects malformed numeric values instead of quietly converting them into apparent collection gaps.

article_records = [ item for item in items if item.get("type") == "article" ] status_records = [ item for item in items if item.get("type") == "status" ] identity_fields = ["articleId", "title", "sourceTweetId"] count_fields = [ "authorFollowers", "likeCount", "replyCount", "retweetCount", "quoteCount", "bookmarkCount", "viewCount", ] selected_fields = ( identity_fields + count_fields + ["engagementRate", "collectedAt"] ) df = pd.DataFrame(article_records).reindex(columns=selected_fields) for field in identity_fields: df[field] = df[field].astype("string") for field in count_fields: df[field] = pd.to_numeric( df[field], errors="raise" ).astype("Int64") if df[field].lt(0).any(): raise ValueError(f"Negative count in {field}") df["engagementRate"] = pd.to_numeric( df["engagementRate"], errors="raise" ).astype("Float64") df["collectedAt"] = pd.to_datetime( df["collectedAt"], utc=True, errors="raise" ) status_df = pd.DataFrame(status_records).reindex( columns=["status", "message", "tweetUrl", "tweetId", "collectedAt"] ) 

The nullable types preserve the distinction between an unavailable measurement and a measured zero.

The timestamp parser handles offset-bearing strings explicitly. The percentage stays in percentage units; treating it as a fraction would change its scale.

Don't deduplicate by title to make the ranking look cleaner. Across repeated collections, the same article and announcement can legitimately recur with a different observation time. Decide whether your report compares snapshots or distinct articles before removing rows.

Reconstruct the rate without completing the evidence

The audit below leaves incomplete numerators missing. Its rounding tolerance is an explicit analyst setting in percentage points, not a guarantee about the returned rate's precision.

numerator_fields = ["likeCount", "retweetCount", "replyCount"] usable_denominator = df["authorFollowers"].gt(0).fillna(False) complete_numerator = df[numerator_fields].notna().all(axis=1) eligible = usable_denominator & complete_numerator reaction_total = df[numerator_fields].sum( axis=1, min_count=len(numerator_fields) ) df["recomputed_pct"] = ( reaction_total.where(eligible) .div(df["authorFollowers"].where(eligible)) .mul(100) ).astype("Float64") rounding_tolerance_pp = 0.0001 comparable = eligible & df["engagementRate"].notna() df["rate_disagrees"] = pd.Series( pd.NA, index=df.index, dtype="boolean" ) df.loc[comparable, "rate_disagrees"] = ( df.loc[comparable, "recomputed_pct"] .sub(df.loc[comparable, "engagementRate"]) .abs() .gt(rounding_tolerance_pp) ) 

Using min_count matters here: a partial numerator mustn't become a smaller, apparently complete total.

An unavailable comparison remains unavailable in rate_disagrees. Calling those rows matches would make the audit look reassuring precisely where it lacks evidence.

A discrepancy calls for inspection. Agreement only establishes consistency with the available arithmetic; it doesn't validate a claim about readers.

Every average needs its population beside it

Before publishing an average, expose the rows it can represent. This DuckDB query separates missing followers, unusable denominators and incomplete numerators.

coverage = duckdb.sql(""" SELECT CASE WHEN authorFollowers IS NULL THEN 'missing followers' WHEN authorFollowers <= 0 THEN 'non-positive followers' WHEN recomputed_pct IS NULL THEN 'incomplete numerator' ELSE 'recomputable' END AS coverage_group, COUNT(*) AS article_snapshot_rows, COUNT(engagementRate) AS supplied_rates FROM df GROUP BY ALL ORDER BY coverage_group """).df() eligible_rows = df.loc[eligible] summary = pd.Series({ "eligible_article_snapshot_rows": int(eligible.sum()), "mean_row_rate_pct": eligible_rows["recomputed_pct"].mean(), "pooled_row_ratio_pct": ( 100 * reaction_total.loc[eligible].sum(min_count=1) / eligible_rows["authorFollowers"].sum(min_count=1) ), }) 

The summary deliberately uses the same eligible rows for both calculations. Otherwise, a difference between figures could reflect a population change as well as a weighting change.

The mean gives each eligible announcement row equal weight. The pooled ratio weights those rows by follower count.

Neither is automatically an author-level average. Repeated appearances by an author repeat their contribution, including their follower denominator in the pooled calculation. Overlapping followers also remain overlapping people.

Report the row unit, eligibility rule and coverage alongside whichever statistic answers your question. Keep supplied-rate coverage visible separately from reconstruction coverage.

Where the readership claim runs out

The announcement links you supply to X Article Announcement Post Metadata Collector define the starting population. It can't enumerate an account's complete article list without signing in, so your ranking doesn't establish that you compared everything an author published.

The follower denominator introduces another boundary. It isn't an exposure count, so the resulting percentage isn't an exposure-based conversion rate. A changed follower count can also change the rate while the reaction numerator stays fixed.

Article text can support analysis of subject matter. Post metrics can support analysis of the announcement's recorded response. Neither supplies evidence of who opened the text, how far they read or whether they understood it.

Sort the column if it answers your question. A tidy ranking still can't turn followers into readers.