A forty-page report arrives, the %GRR figure comes back under ten percent, and the AOI gauge R&R study is declared passed. Nobody asks what was measured. In most cases it was microns and degrees — X and Y offset, rotation, 3D height — while the pass/fail decision that actually stops the line went untested.
That gap is structural, not dishonest. An inspection system emits two kinds of output, and they fail in different ways. Variable outputs — X and Y offset, rotation, 3D height — need a Gage R&R. Attribute outputs — accept or reject — need an attribute agreement analysis with Kappa. Run only the first and you have proven the ruler is straight while leaving the judgement untested.
Two Kinds of Output, Two Kinds of Study
Variable data has magnitude: a 0402 resistor offset 43 µm in X, a part rotated 1.8°, a joint standing 112 µm above the pad, a paste deposit of 0.62 mm³.
Attribute data has only a verdict, so its quality is measured as agreement — with a standard, another operator, another run of the same board.
The statistics differ because the questions differ: Gage R&R asks how far a number moves; attribute agreement asks whether the verdict survives a second run, machine, or reviewer.
A system can be excellent at one and mediocre at the other: precise optics with badly tuned thresholds measure beautifully and judge inconsistently.
The AOI Gauge R&R Study: %GRR and ndc on Microns and Degrees
Use the ANOVA method rather than average-and-range. ANOVA separates the part-by-appraiser interaction, which on AOI data is often the largest component: the same feature is easy on a ceramic capacitor and hard on a black connector body.
A workable design: 10 boards spanning real process variation, 3 operators, 3 randomized trials each, boards reloaded every run, one frozen program version throughout.
%GRR is measurement variation as a percentage of the tolerance band you set for that feature. If placement tolerance is ±100 µm, the study tells you how much of that 200 µm window the machine consumes.
| %GRR | Decision | What it usually means on an AOI |
|---|---|---|
| under 10% | Acceptable | Noise is small against the tolerance band |
| 10–30% | Conditional | Acceptable only if the feature is hard to measure, improvement cost is high, and the decision is documented |
| over 30% | Not acceptable | The system cannot separate good from bad inside your tolerance |
ndc is the number of distinct categories the system can resolve. Below 5, it cannot reliably rank parts, so any trend toward a limit is noise. A %GRR of 9% with an ndc of 3 is a warning, not a pass.
One trap sits in sample selection, and which way it bites depends on the denominator. Against a tolerance band, %GRR barely moves — the number is anchored to a drawing dimension. Against study variation it moves a lot, because the denominator is the spread of the parts you chose: ten boards from a narrow slice of production understate that spread and inflate %GRR, and ten boards from one easy panel position do the same. Pad the set with a couple of obviously scrap boards and the opposite happens: the spread widens until %GRR collapses, which flatters the machine. Neither number is wrong; both answer a narrower question than the one you asked.
Sample the way the line runs — every panel position, every supplier in the current material set — and report %GRR against the tolerance band when the decision is whether the machine can hold the drawing.
What "Appraiser" Actually Means on an AOI
On a manual gauge, the appraiser is the person holding the caliper. On an AOI, the person at the screen is not the appraiser. The appraiser is the whole handling-and-judgement chain:
- The loading action — board against the stops, clamp pressure, pushed to the left rail or the right.
- A second operator performing that load with the same program.
- A second machine running the same program version on the same boards.
The third layer is the one buyers forget and the one that matters at scale. Two machines of the same model with the same recipe can still differ in camera gain, lamp aging, or conveyor height by enough to flip a borderline call.
This is the procedure we run at qualification, on the SMT 2D AOI platform and on anything else in front of us. The loading layer is where most of the variance sits, and no lens specification fixes a board that does not sit against the same stop every time.
The Attribute Study: Within, Between, and Against Standard
Attribute agreement analysis yields three comparisons that answer different questions.
- Within appraiser (repeatability): same operator, same board, repeated runs. Did the call change?
- Between appraisers (reproducibility): operator A versus B, machine 1 versus machine 2. Do different parts of your organization judge the same board alike?
- Each appraiser versus the standard: the call versus the truth, established by a qualified reviewer against IPC-A-610. This is the only comparison that measures accuracy rather than consistency.
Report all three. A system that agrees with itself and disagrees with the standard is consistently wrong — worse than randomly wrong, because it generates no alarms.
IPC-9262 applies Kappa, which corrects for agreement occurring by chance. The bands below are the convention most published 9262 studies are read against, not a table in the standard:
| Kappa | Conventional band | Practical reading |
|---|---|---|
| ≥ 0.81 | Excellent | Agreement well beyond chance |
| 0.61–0.80 | Good | Acceptable for most applications |
| 0.41–0.60 | Moderate | Needs improvement; not for customer disposition |
| ≤ 0.40 | Poor | Judgement is not reproducible |
The standard ties AOI verification to AIAG's Measurement System Analysis methodology. That is the practical reason to use Kappa instead of inventing a percentage-agreement metric: it produces the MSA evidence an IATF 16949 auditor already asks for, instead of a standalone PDF that proves nothing outside itself.
Percentage agreement should be retired. With a 95% good population, a machine that calls everything good scores 95% agreement while catching nothing. Kappa collapses toward zero, which is the correct answer.
Sample Sets That Can Produce a Bad Number
A study can only fail if the samples let it. The standard supplier demonstration uses known-good boards plus obviously broken ones — a missing connector, a part rotated 90°, a tombstoned capacitor. Any working camera scores near 100%, Kappa lands above 0.9, and the report looks clean.
Build the set across the decision boundary instead:
- 0402 offset at roughly 25%, 50%, and 75% of pad width
- Insufficient solder at 50% and 75% of specified fillet height
- Tombstone at the angle where one fillet survives and the part has lifted
- Reversed polarity on a package with a low-contrast marking
- Good units near the limit — the ones a well-tuned program should pass and a jumpy threshold will reject
A common design is 50 units with at least a third carrying a real defect, plus enough borderline units that a disagreement has somewhere to appear. A defect category with two examples produces an anecdote, not a Kappa.
When the Model Is Not Deterministic
Most Gage R&R guides omit this, and it is now the most common reason an attribute study cannot be reproduced.
If the inspection program uses deep learning, and the inference pipeline includes random augmentation, confidence-threshold jitter, test-time augmentation, or multi-frame fusion, the same board can return a different score on the next pass. That is a design choice, not a fault, but it leaves "repeatability" undefined until you fix the input and the network path.
Before any attribute study, ask three questions:
- Is inference deterministic given identical input images? Get a written answer, not a demo.
- Does the pipeline fuse multiple frames or exposures? If yes, which ones, and in what order?
- Can the vendor expose the exact model build and preprocessing parameters used for a given run?
Then lock the answers: fixed input images, fixed preprocessing, fixed model build, single inference path. Only after that does a within-appraiser Kappa mean anything. Testing repeatability on a pipeline with stochastic elements measures the dice, not the detector.
Vendors that build their own AI-based inspection algorithms should be able to answer these three questions in writing, including the fallback path when a model is updated mid-qualification.
Freezing the System While You Measure It
Repeatability is meaningful only if the thing repeating is identical. Before the first run, record the program version by hash, every threshold and rule-set version, light source and exposure, camera gain and white balance, conveyor width and clamp pressure, and stop position.
Put that list in the report. A Gage R&R without a recorded configuration cannot be defended two years later when a customer asks why the same board yields a different result.
Reading a Kappa Table, and Where the Study Stops
Prevalence. Kappa falls when one category dominates, even with high raw agreement. A defect appearing on 2% of units yields a low Kappa from a system that catches all of them. Report prevalence beside Kappa, and read a low value on a rare defect as "not enough events," not "bad machine."
Averaging. One Kappa across all defect types hides the structure of the failure. A program can score 0.85 overall while scoring 0.35 on insufficient solder and 0.9 on missing parts. Report per class or report nothing.
The temptation at this point is to relax the threshold until the numbers improve. A study that ends that way has qualified nothing.
Measure variable outputs with Gage R&R, measure judgements with attribute agreement, build a sample set that can fail, freeze the configuration, and report per category. Half a qualification is worse than none, because it produces a number that looks like an answer.
