# Multilevel Composite Reliability

This is a short blog post on computing multilevel reliability based on Lai (2021).

library(lavaan)
## This is lavaan 0.6-12
## lavaan is FREE software! Please report any bugs.
library(semTools)
## 
## ###############################################################################
## This is semTools 0.5-6
## All users of R (or SEM) are invited to submit functions or ideas for functions.
## ###############################################################################

First, use demo data from lavaan

data("Demo.twolevel", package = "lavaan")

Then source a script I wrote here implementing the procedures in the paper, which defines a function multilevel_alpha():

source("https://github.com/marklhc/mcfa_reliability_supp/raw/master/multilevel_alpha.R")

Example syntax in R and Mplus for the paper can be found in https://github.com/marklhc/mcfa_reliability_supp. Going back to the multilevel demo data, we can consider the reliability for between-level and within-level composite scores when summing y1, y2, and y3. This can be done using

multilevel_alpha(Demo.twolevel[c("y1", "y2", "y3")],
id = Demo.twolevel$cluster) ## Loading required package: psych ## ## Attaching package: 'psych' ## The following objects are masked from 'package:semTools': ## ## reliability, skew ## The following object is masked from 'package:lavaan': ## ## cor2cov ## Parallel analysis suggests that the number of factors = NA and the number of components = 1 ## Parallel analysis suggests that the number of factors = NA and the number of components = 1 ##$alpha
##   alpha2l    alphab    alphaw
## 0.7523324 0.7613526 0.6682291
##
## $alpha_ci ## 2.5% 97.5% ## alpha2l 0.7280449 0.7736249 ## alphab 0.7145704 0.7968058 ## alphaw 0.6420312 0.6936817 ## ##$omega
##   omega2l    omegab    omegaw
## 0.7669377 0.7837490 0.6761273
##
## $omega_ci ## 2.5% 97.5% ## omega2l 0.7428390 0.7866943 ## omegab 0.7401762 0.8168061 ## omegaw 0.6485975 0.7008100 ## ##$ncomp
##  within between
##       1       1

Note that the $$\omega$$ indices assume unidimensionality, local independence, and cross-level invariance. There were a total of six indices: three $$\alpha$$s and three $$\omega$$s, corresponding to raw composite (“2l”), between-level composite (“b”), and within-level composite (“w”). The 95% CIs were obtained using the Monte Carlo method by simulating from the asymptotic distribution of the model parameters. In addition, the function also implemented a parallel analysis procedure for determining the number of factors/components at each level, as discussed in this paper.

## Using semTools::compRelSEM()

Some of the above procedures were implemented in the semTools package. To use that, one needs to fit a two-level CFA in lavaan first (see ?compRelSEM:

mcfa_mod <- '
level: 1
f =~ y1 + L2*y2 + L3*y3
level: 2
f =~ y1 + L2*y2 + L3*y3
'
mcfa_fit <- cfa(mcfa_mod, data = Demo.twolevel, cluster = "cluster")

Then call the function

compRelSEM(mcfa_fit, config = c("f"), shared = c("f"))
## $config ##$config$f ## omega_W omega_2L ## 0.6732269 0.7705154 ## ## ##$shared
## $shared$f
##   omega_B       IRR
## 0.7972928 0.8361785
##### Hok Chio (Mark) Lai 黎學昭
###### Assistant Professor of Psychology (Quantitative Methods)

My research interests include statistics, multilevel and latent variable models, and psychometrics.