Table 1

Results from the Generalisability analysis with relative contribution of variance

Results from the G-study showing the contribution of each source of variance
Source of variance VDescriptionRelative contributionInterpretation of results
Participant, Vpa Variation among participants77.0%Most of the variance comes from the various skill levels among the participants
The procedures, Vp Variability among the two procedures0.5%Almost none of the variance comes from the procedures due to the standardised testing setup
Rater, Vr Variability among the raters3.2%A small contribution to the variance indicates a high level of agreement among the raters
Interaction between the participant and the procedures, Vpa*p If the participants consistently show a difference in the two procedures2.8%The small contribution to the variance from this interaction indicates a small learning by testing effect and a successful blinding
Interaction between the rater and the participant, Vpa*r If a rater assesses a particular participant differently2.3%A small variance contribution indicating a successful blinding
Interaction between the rater and the procedure, Vp*r If a rater shows a consistent difference in rating the two procedures0.0%No variance contribution indicates a successful blinding
Interaction between participant, rater and procedures, Vpa*r*p The remaining variability14.2%Unavoidable, unexplained error