Reliability
Agreement coefficient (Cohen’s and Fleiss’ κ)
Measures chance-corrected agreement when several raters assign the same objects to categories.
Variables
- RatersNominal · Ordinal — at least 2
Options
- WeightsUnweighted · Linear weights · Quadratic weights
What it reports
Measures chance-corrected agreement when several raters assign the same objects to categories: Cohen’s κ for two raters, Fleiss’ κ for three or more.
statsmodels · formula · Open the accuracy report
When does this apply?
Use when the same objects are rated by several raters on a nominal or ordinal scale; each rater is a separate column, one row is one object. The coefficient follows from the number of raters: two columns give Cohen’s κ, three or more give Fleiss’ κ; the choice comes from the study design, not from the data. Weights are available only for an ordinal scale and two raters. κ is reported with its confidence interval (Cohen’s κ) and a z test. No quality verdict is given.