Multilevel Composite Reliability

This is a short blog post on computing multilevel reliability based on Lai (2021).

Load Packages

library(lavaan)

This is lavaan 0.6-17
lavaan is FREE software! Please report any bugs.

library(semTools)

###############################################################################

This is semTools 0.5-6

All users of R (or SEM) are invited to submit functions or ideas for functions.

###############################################################################

First, use demo data from lavaan

data("Demo.twolevel", package = "lavaan")

Then source a script I wrote here implementing the procedures in the paper, which defines a function multilevel_alpha():

source("https://github.com/marklhc/mcfa_reliability_supp/raw/master/multilevel_alpha.R")

Example syntax in R and Mplus for the paper can be found in https://github.com/marklhc/mcfa_reliability_supp. Going back to the multilevel demo data, we can consider the reliability for between-level and within-level composite scores when summing y1, y2, and y3. This can be done using

multilevel_alpha(Demo.twolevel[c("y1", "y2", "y3")],
                 id = Demo.twolevel$cluster)

Loading required package: psych

Attaching package: 'psych'

The following objects are masked from 'package:semTools':

    reliability, skew

The following object is masked from 'package:lavaan':

    cor2cov

Parallel analysis suggests that the number of factors =  NA  and the number of components =  1 
Parallel analysis suggests that the number of factors =  NA  and the number of components =  1 

$alpha
  alpha2l    alphab    alphaw 
0.7523324 0.7613526 0.6682291 

$alpha_ci
             2.5%     97.5%
alpha2l 0.7278728 0.7731307
alphab  0.7140770 0.7964839
alphaw  0.6396391 0.6925092

$omega
  omega2l    omegab    omegaw 
0.7669377 0.7837490 0.6761273 

$omega_ci
             2.5%     97.5%
omega2l 0.7425671 0.7871950
omegab  0.7390618 0.8168503
omegaw  0.6480075 0.7006861

$ncomp
 within between 
      1       1 

Note that the \(\omega\) indices assume unidimensionality, local independence, and cross-level invariance. There were a total of six indices: three \(\alpha\)s and three \(\omega\)s, corresponding to raw composite (“2l”), between-level composite (“b”), and within-level composite (“w”). The 95% CIs were obtained using the Monte Carlo method by simulating from the asymptotic distribution of the model parameters. In addition, the function also implemented a parallel analysis procedure for determining the number of factors/components at each level, as discussed in this paper.

Using semTools::compRelSEM()

Some of the above procedures were implemented in the semTools package. To use that, one needs to fit a two-level CFA in lavaan first (see ?compRelSEM:

mcfa_mod <- '
  level: 1
    f =~ y1 + L2*y2 + L3*y3
  level: 2
    f =~ y1 + L2*y2 + L3*y3
'
mcfa_fit <- cfa(mcfa_mod, data = Demo.twolevel, cluster = "cluster")

Then call the function

compRelSEM(mcfa_fit, config = c("f"), shared = c("f"))

$config
$config$f
  omega_W  omega_2L 
0.6732269 0.7705154 


$shared
$shared$f
  omega_B       IRR 
0.7972928 0.8361785