Skip to content
Prev 386964 / 398502 Next

Help with Krippendorff's Alpha

Hello!

I am requesting help for a data set in which I have 6 evaluators (I've called them 'Trainers'), each of which took a questionnaire with 180 questions. The possible answers to the questions are ordinal numbers. I am looking to test inter-rater reliability of the questionnaire using Krippendorff's alpha. I have already run this in R, however when I get the output it seems the matrix is inverted. While it seems like it should be an easy problem to fix I don't see how I would be able to create my matrix the other way without typing out all 180 scores manually. For reference, I'm using the irr package to use kripp.alpha().
Krippendorff's alpha
Subjects = 6
Raters = 180
alpha = -0.00149

While I'm getting an output - shouldn't my raters be 6 and my subjects be 180?

Appreciate the help!

JR