# Multilevel Composite Reliability

Statistics
Author

Mark Lai

Published

November 13, 2022

This is a short blog post on computing multilevel reliability based on Lai (2021).

library(lavaan)
This is lavaan 0.6-17
lavaan is FREE software! Please report any bugs.
library(semTools)
 
###############################################################################
This is semTools 0.5-6
All users of R (or SEM) are invited to submit functions or ideas for functions.
###############################################################################

First, use demo data from lavaan

data("Demo.twolevel", package = "lavaan")

Then source a script I wrote here implementing the procedures in the paper, which defines a function multilevel_alpha():

source("https://github.com/marklhc/mcfa_reliability_supp/raw/master/multilevel_alpha.R")

Example syntax in R and Mplus for the paper can be found in https://github.com/marklhc/mcfa_reliability_supp. Going back to the multilevel demo data, we can consider the reliability for between-level and within-level composite scores when summing y1, y2, and y3. This can be done using

multilevel_alpha(Demo.twolevel[c("y1", "y2", "y3")],
id = Demo.twolevel$cluster) Loading required package: psych  Attaching package: 'psych' The following objects are masked from 'package:semTools': reliability, skew The following object is masked from 'package:lavaan': cor2cov Parallel analysis suggests that the number of factors = NA and the number of components = 1 Parallel analysis suggests that the number of factors = NA and the number of components = 1  $alpha
alpha2l    alphab    alphaw
0.7523324 0.7613526 0.6682291

$alpha_ci 2.5% 97.5% alpha2l 0.7278728 0.7731307 alphab 0.7140770 0.7964839 alphaw 0.6396391 0.6925092$omega
omega2l    omegab    omegaw
0.7669377 0.7837490 0.6761273

$omega_ci 2.5% 97.5% omega2l 0.7425671 0.7871950 omegab 0.7390618 0.8168503 omegaw 0.6480075 0.7006861$ncomp
within between
1       1 

Note that the $$\omega$$ indices assume unidimensionality, local independence, and cross-level invariance. There were a total of six indices: three $$\alpha$$s and three $$\omega$$s, corresponding to raw composite (“2l”), between-level composite (“b”), and within-level composite (“w”). The 95% CIs were obtained using the Monte Carlo method by simulating from the asymptotic distribution of the model parameters. In addition, the function also implemented a parallel analysis procedure for determining the number of factors/components at each level, as discussed in this paper.

## Using semTools::compRelSEM()

Some of the above procedures were implemented in the semTools package. To use that, one needs to fit a two-level CFA in lavaan first (see ?compRelSEM:

mcfa_mod <- '
level: 1
f =~ y1 + L2*y2 + L3*y3
level: 2
f =~ y1 + L2*y2 + L3*y3
'
mcfa_fit <- cfa(mcfa_mod, data = Demo.twolevel, cluster = "cluster")

Then call the function

compRelSEM(mcfa_fit, config = c("f"), shared = c("f"))
$config$config$f omega_W omega_2L 0.6732269 0.7705154$shared
$shared$f
omega_B       IRR
0.7972928 0.8361785