skip to main content skip to footer

A Monte Carlo Study of the Effects of Model Misspecification on HLM Estimates

Donoghue, John R.; Jenkins, Frank
Publication Year:
Report Number:
ETS Research Report
Document Type:
Subject/Key Words:
Estimation (Mathematics), Mathematical Models, Models, Monte Carlo Methods, Simulation, Statistical Analysis


Monte Carlo methods were used to investigate the effect of mis- specification of the second-level in a two-level hierarchical linear model. Sample composition, heterogeneity of the group size, level of intraclass correlation, and correlation between second-level predictors were manipulated. Twenty data sets were generated for each condition. Each data set was analyzed nine times with the HLM program, corresponding to the correct model and eight types of misspecification. The error variance sigma squared was estimated accurately for all model specifications. For the correct second-level model, regression parameters gamma and the covariance matrix of the second-level errors tau were estimated accurately, although estimates were slightly biased for some levels of the sample composition manipulation. Misspecifications that failed to include a predictor had a larger effect than those which erroneously included an effect. Reactivity was assessed by examining parameters of equations which were correctly specified, although another equation in the model was misspecified. HLM showed little reactivity. Only one of the misspecified models yielded gamma estimates that were significantly farther, on average, from the generating parameters than were those of the correctly specified model. Similar results were found for the root mean square residual between the generating and estimated tau.

Read More