What is a good Kappa value?

Kappa 0.75. These thresholds are guidelines — context matters.

Why not just use percent agreement?

Percent agreement doesn't account for agreement that occurs by chance. If defect rate is low (e.g., 5%), inspectors who always say "pass" get 95% agreement despite being useless. Kappa adjusts for this.

How many parts do I need?

At least 50 parts, ideally including 25–50% that are borderline or defective. If your defect rate is low, deliberately include known defective parts to test detection capability.

Can I evaluate more than 2 inspectors?

Yes. Use Fleiss' Kappa for multiple raters making categorical decisions. You can also compute pairwise Kappa between each pair of inspectors to identify specific disagreement patterns.

What if inspectors disagree on borderline samples?

This is expected and reveals opportunities. Create clearer criteria, boundary samples, and comparison standards. Re-evaluate after training. Borderline disagreement is the norm, not the exception.

How does attribute agreement relate to escape rate?

Low agreement with the standard means high escape rates (defective parts passed as good) and high false reject rates (good parts rejected). Quantifying agreement helps predict these operational impacts.

Attribute Agreement Analysis Calculator

Calculate attribute agreement percentages and Cohen's Kappa for pass/fail inspection decisions. Evaluate inspector consistency and accuracy.

Study Type

Total Parts Evaluated

True Positives (correct pass)

False Positives (false pass)

False Negatives (missed defect)

True Negatives (correct reject)

Number of Trials

% Agreement with Standard

93.00%

93 of 100 decisions correct

Cohen's Kappa (κ)

0.838

Excellent

Detection Rate (Sensitivity)

92.90%

65 of 70 actual defects detected

Escape Rate (Miss Rate)

7.10%

5 defects missed — passed as good

False Alarm Rate

6.70%

2 good parts incorrectly rejected

Precision (PPV)

97.00%

Of parts called pass, % actually good

Specificity

93.30%

Correct rejection rate for truly bad parts

Repeatability (est.)

96.00%

Estimated from 2 trials

Kappa Rating

κ = 0.838 — Excellent

0 (chance)0.400.600.751.0 (perfect)

Confusion Matrix

	Inspector: Pass	Inspector: Fail	Total
Actual: Pass	TP = 65	FN = 5	70
Actual: Fail	FP = 2	TN = 28	30
Total	67	33	100

Kappa Interpretation Guide

Kappa Range	Interpretation	Action Required
0.75 – 1.00	Excellent agreement	System is capable. Monitor periodically.
0.60 – 0.74	Good agreement	Acceptable for most applications. Clarify borderlines.
0.40 – 0.59	Moderate agreement	Needs improvement. Review criteria and retrain.
0.20 – 0.39	Fair agreement	Unreliable. Overhaul inspection criteria.
< 0.20	Poor / no agreement	Not better than chance. Redesign process.

Recommended Sample Sizes

Defect Rate	Min Total Parts	# Defective Parts	# Good Parts	Trials
< 5%	100	20–25	75–80	3
5 – 15%	75	15–20	55–60	2–3
15 – 30%	50	15	35	2
> 30%	50	25	25	2

Planning notes, formulas, and examples

About the Attribute Agreement Analysis Calculator

Attribute Agreement Analysis evaluates the consistency and accuracy of inspectors making categorical decisions — pass/fail, accept/reject, or defect classification. Unlike variable Gage R&R which measures continuous data, attribute agreement addresses the binary or categorical inspection decisions that are common in visual inspection, go/no-go gaging, and defect sorting.

The analysis measures agreement within each inspector (repeatability), between inspectors (reproducibility), and between each inspector and the known standard (accuracy). Cohen's Kappa statistic adjusts for chance agreement, providing a more rigorous measure than simple percent agreement.

This calculator takes the total number of decisions and the number of matching decisions to compute percent agreement and Cohen's Kappa, helping you determine if your attribute inspection process is reliable.

This analytical approach aligns with lean manufacturing principles by replacing waste-generating guesswork with efficient, fact-based processes that directly support value creation and cost reduction. By calculating this metric accurately, production managers gain actionable insights that drive continuous improvement efforts and strengthen overall operational performance across the shop floor.

When This Page Helps

Visual and attribute inspections are subjective and variable. Attribute agreement analysis exposes inconsistency and bias, enabling targeted training that makes pass/fail decisions more reliable and defensible.

How to Use the Inputs

Have multiple inspectors independently evaluate the same set of parts (at least 50 samples).
Record each inspector's decision alongside the known correct answer.
Count total decisions and matching decisions.
Enter the values into the calculator.
Review percent agreement and Kappa statistic.
If Kappa < 0.75, investigate boundary samples and provide additional training.

Formula used

% Agreement = (Matching Decisions / Total Decisions) × 100

Cohen's Kappa (κ) = (P_o − P_e) / (1 − P_e)

where:
• P_o = observed agreement (proportion)
• P_e = expected agreement by chance
• P_e = (a/n × d/n) + (b/n × c/n) for 2×2 confusion matrix

Example Calculation

Result: 88% Agreement, κ = 0.76

Out of 100 decisions, 88 matched the standard. Simple agreement is 88%. Adjusting for the 50% expected by chance: κ = (0.88 − 0.50) / (1 − 0.50) = 0.76, indicating substantial agreement beyond chance.

Tips & Best Practices

Use at least 50 samples with a known standard (master classification) for statistical significance.
Include borderline samples — they reveal the most about inspector judgment consistency.
Blind the study — don't let inspectors know they are being evaluated or they may be more careful than normal.
Run repeat evaluations (2–3 trials) to assess within-inspector repeatability.
Kappa accounts for chance agreement — use it instead of simple % agreement for rigorous analysis.
Provide clear defect standards (limit samples, comparison photos) to improve agreement.

The 2×2 Confusion Matrix

For pass/fail decisions against a known standard, the confusion matrix shows: true accepts, false accepts, true rejects, and false rejects. Each cell informs a different quality metric — escape rate, false alarm rate, and overall accuracy.

Improving Attribute Agreement

Common interventions include: creating limit sample boards with clear pass/fail boundaries, standardizing lighting and viewing conditions, implementing inspector certification programs, and rotating inspectors to prevent fatigue-related degradation.

Attribute Agreement in Automotive

IATF 16949 clause 7.1.5.1.1 requires attribute MSA for all subjective inspection processes referenced in the control plan. Third-party auditors specifically check for documented attribute agreement studies.

Sources & Methodology

Last updated: February 9, 2026

Frequently Asked Questions

Kappa < 0.40 is poor agreement. 0.40–0.60 is moderate. 0.60–0.75 is good. Above 0.75 is excellent. For critical quality decisions, aim for κ > 0.75. These thresholds are guidelines — context matters.