Multilevel Composite Reliability

This is a short blog post on computing multilevel reliability based on Lai (2021).

Load Packages

library(lavaan)
## This is lavaan 0.6-12
## lavaan is FREE software! Please report any bugs.
library(semTools)
## 
## ###############################################################################
## This is semTools 0.5-6
## All users of R (or SEM) are invited to submit functions or ideas for functions.
## ###############################################################################

First, use demo data from lavaan

data("Demo.twolevel", package = "lavaan")

Then source a script I wrote here implementing the procedures in the paper, which defines a function multilevel_alpha():

source("https://github.com/marklhc/mcfa_reliability_supp/raw/master/multilevel_alpha.R")

Example syntax in R and Mplus for the paper can be found in https://github.com/marklhc/mcfa_reliability_supp. Going back to the multilevel demo data, we can consider the reliability for between-level and within-level composite scores when summing y1, y2, and y3. This can be done using

multilevel_alpha(Demo.twolevel[c("y1", "y2", "y3")],
                 id = Demo.twolevel$cluster)
## Loading required package: psych
## 
## Attaching package: 'psych'
## The following objects are masked from 'package:semTools':
## 
##     reliability, skew
## The following object is masked from 'package:lavaan':
## 
##     cor2cov
## Parallel analysis suggests that the number of factors =  NA  and the number of components =  1 
## Parallel analysis suggests that the number of factors =  NA  and the number of components =  1
## $alpha
##   alpha2l    alphab    alphaw 
## 0.7523324 0.7613526 0.6682291 
## 
## $alpha_ci
##              2.5%     97.5%
## alpha2l 0.7280449 0.7736249
## alphab  0.7145704 0.7968058
## alphaw  0.6420312 0.6936817
## 
## $omega
##   omega2l    omegab    omegaw 
## 0.7669377 0.7837490 0.6761273 
## 
## $omega_ci
##              2.5%     97.5%
## omega2l 0.7428390 0.7866943
## omegab  0.7401762 0.8168061
## omegaw  0.6485975 0.7008100
## 
## $ncomp
##  within between 
##       1       1

Note that the \(\omega\) indices assume unidimensionality, local independence, and cross-level invariance. There were a total of six indices: three \(\alpha\)s and three \(\omega\)s, corresponding to raw composite (“2l”), between-level composite (“b”), and within-level composite (“w”). The 95% CIs were obtained using the Monte Carlo method by simulating from the asymptotic distribution of the model parameters. In addition, the function also implemented a parallel analysis procedure for determining the number of factors/components at each level, as discussed in this paper.

Using semTools::compRelSEM()

Some of the above procedures were implemented in the semTools package. To use that, one needs to fit a two-level CFA in lavaan first (see ?compRelSEM:

mcfa_mod <- '
  level: 1
    f =~ y1 + L2*y2 + L3*y3
  level: 2
    f =~ y1 + L2*y2 + L3*y3
'
mcfa_fit <- cfa(mcfa_mod, data = Demo.twolevel, cluster = "cluster")

Then call the function

compRelSEM(mcfa_fit, config = c("f"), shared = c("f"))
## $config
## $config$f
##   omega_W  omega_2L 
## 0.6732269 0.7705154 
## 
## 
## $shared
## $shared$f
##   omega_B       IRR 
## 0.7972928 0.8361785
Hok Chio (Mark) Lai 黎學昭
Hok Chio (Mark) Lai 黎學昭
Assistant Professor of Psychology (Quantitative Methods)

My research interests include statistics, multilevel and latent variable models, and psychometrics.

comments powered by Disqus

Related