The Canadian Food Inspection Agency (CFIA) tests foods for chemical contaminants in targeted surveys. For each survey it publishes a report page and a per-sample CSV on open.canada.ca. MV2 of Munim, Inc. has been checking those open files against every figure CFIA printed for them. The work is done by AI agents working in his name, and everything below can be reproduced from the repository linked at the end.

Each check follows the same order. Predictions are fixed from the printed report alone and committed in public. A names-only census then counts samples, labels and units without reading any value. The grader is published before any value is read. Last, a sealed job does the grading and an independent test re-runs the result from GitHub. Failed predictions stay failed.

This post sums up eleven files: seven bisphenol A (BPA) surveys from 2009-10 to 2017-18, BPA and its alternatives in 2018-19 and 2020-21, and polycyclic aromatic hydrocarbons (PAHs) in 2018-19 and 2019-20. For the seven older BPA files and PAH 2019-20, only the names-only census is done so far. Their value grades are pending.

1. The totals match

All eleven files hold exactly the total number of samples their reports print. For 2018-19 BPA, that is the total in its detailed tables (more on this below). The main tables mostly reproduce cell for cell. PAH 2018-19's Table 2, BPA 2018-19's Table 2 and BPA 2020-21's Tables 2 to 5 all reproduce in full.

2. The splits are where things drift

When a report and its file disagree, it is almost always about how samples are split between neighbouring groups, not about how many there are:

  • 2010-11: 129 infant formulas in the file where the page says 127, and 87 fruit products where it says 92. The total of 234 still matches.
  • 2012-13: 21 curry samples where the page says 24, and 147 vegetables where it says 144. Together they make the printed 168.
  • 2013-14: juices and other drinks split 55 / 42 where the description says 54 / 43.
  • 2017-18: 322 imported samples, not the printed 325. With 322 the origin row adds up to 468.
  • PAH 2018-19 and 2019-20: one or two samples per group move between "imported" and "unspecified".

A later report re-prints the 2013-14 survey with "34" pastas and "45 (100%)" positives. The file has 45 pastas.

3. A report can describe more samples than its open file

The 2018-19 BPA report gives two sample counts. Its Tables 1 and 2 say 377, and the open file holds 377 and reproduces Table 2 in every cell. Its summary says 381 samples with BPA in 43%. Its Table 3 says 138 infant formulas, with a maximum of 5.89 ppb and an average of 1.65, and 96 sauces.

Before reading the file, we worked out from the report alone what those numbers require. The 17 formula positives in Table 2 must sum to between 23.72 and 23.90 ppb. In the file they sum to 23.81. Adding four samples makes every printed figure agree: three formulas, one of them BPA-positive at 5.89 ppb, and one sauce with no BPA. No other result for the extra formulas fits. So the survey's highest infant-formula value is printed in the report but is not in the open data. The file cannot say why.

4. Some rows look copied

In the 2018-19 PAH file, two pairs of different foods carry identical results. A soybean oil from the United States and a sunflower oil from Turkey share all 16 PAH values. A roasted pepper from Greece and a cookie share 14 non-zero values to four decimals. The report's Figure 1 also prints the same seven values for soybean and sunflower oil. The file shows the soybean list is the copy. We had predicted the sunflower list, and that prediction failed.

The two bisphenol files from 2018-19 and 2020-21 have no such pairs.

5. Sentences are the weakest part

The 2020-21 BPA tables reproduce in full, but two sentences in the text do not match the file. The text says all eight samples with both BADGE and BPA were imported meat or fish. In the file, two of the eight are canned meats from Canada. It also says five imported corned beef samples contained both BPS and BPA. All five BPS-positive meats are imported, but one of them has no BPA.

6. Check the date column

The 2009-10 BPA file gives its 200 samples 200 different sampling dates. The ones we listed run day by day from October 2009 into January 2010, with 25 December and 1 January included. Those look like placeholder dates, so don't read seasonality from that file.

What this is not

None of this is a compliance finding or a statement about any brand or health risk. Canada has no maximum levels for BPA or its alternatives in food. The tables that matter most for exposure estimates reproduce. The findings are about bookkeeping between a report and its open file. That matters when anyone reuses the open data.

Reproduce it

The index of every note, pre-registration, grader and per-sample table is in the open repository: https://github.com/munimtechnologies/spice-lead-recalls/blob/main/docs/cfia-chemistry-surveys-index.md . The code is Apache-2.0 and the text and derived data are CC BY 4.0. The source data is CFIA's, under the Open Government Licence - Canada.

Made with AI agents working in MV2's name.

— MV2, Munim, Inc., mv2@munimtech.com