Group Sequential Designs for Intraclass Correlation Coefficients in Reliability Studies Open Access
Downloadable ContentDownload PDF
The intraclass correlation coefficient (ICC) has a lengthy history of application in several different fields of research. It is a widely recognized index of reliability among measurements. Two-stage group sequential designs are proposed to lead to savings in sample size, time and cost when compared with standard fixed sample procedures. We present two-stage test that is optimal in the sense that the average sample number (ASN) is minimized when the measurements have low reliability subject to constraints upon the size of the type I and type II errors. In addition, the proposed method is extended to the design for two-stage tests of the ICC derived from a two-way ANOVA model. Inter-rater reliability studies are conducted to investigate the reproducibility and level of agreement on assessments among different raters. The analysis of reliability is a common feature in research and practice since most measurements involve measurement errors, particularly those made by humans. Measurement error can seriously affect statistical analysis and interpretation; it therefore becomes important to assess the amount of such error by calculating a reliability index.Using two-way analysis of variance to model raters and subjects as random effects, we derive group sequential testing procedures for the design and analysis of reliability studies in which multiple raters evaluate multiple subjects. Compared with the conventional fixed sample procedures, the group sequential test has smaller average sample number. The performance of the proposed technique is examined using simulation studies and critical values are tabulated for a range of two-stage design parameters.