Stata Help

Kappa: 2+ Raters, Two Possible Outcomes

If you have more than two raters, the math does not work as if the same 2+ raters did the rating. Thus if you have only two raters, but not the same two raters for every case, you will want to use these instructions. For the data below, 3-5 imaginary raters were asked to assign an overall mood (happy or depressed) to imaginary subjects.

Since this is frequency data, I am going to use the kappa command. Frequency data helps to emphasize that, in this case, the identity of the rater is irrelevant. The command is kappa happy depressed and it yields the following output:

Note: Stata tells you precisely what type of kappa it is calculating before showing you the kappa value (in this instance, two-outcomes with multiple raters).