An Illustration of a Latent Class Analysis for Interrater Agreement: Identifying Subpopulations with Different Agreement Levels
Articles > Article · 2023
Authors: Ömer Emre Can Alagöz, Yılmaz Orhun Gürlük, Mediha KORKMAZ, Gizem Cömert
Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, Journal Indexed in ESCI, International, Scopus, 2023, vol 14, pp 492-507
Summary
This study proposes a latent class analysis (LCA) approach to investigate interrater agreement based on rating patterns. LCA identifies which subjects are rated similarly or differently by raters, providing a new perspective for investigating agreement. Using an empirical dataset of parents and teachers evaluating pupils, the study found two latent classes of respondents, one belonging to a moderate agreement pattern and one belonging to low agreement pattern. We calculated raw agreement coefficient (RAC) per behaviour in the whole sample and each latent class. When RAC was calculated in the whole sample, many behaviour had low/moderate RAC values. However, LCA showed that these items had higher RAC values in the high agreement and lower RAC values in the low agreement class.