Linear mixed model
Verification
Scope
Coefficients, covariance matrices, standard errors, z statistics, p-values, confidence intervals, conditional and marginal fitted values and residuals, random effects, variance components, log likelihood, and AIC of one Gaussian random intercept fitted by ML. Identical design matrices, groups, and missing-data exclusions across implementations.
References
- Reference
- R lme4 lmer(REML = FALSE, bobyqa rhoend = 1e-10)
- Versions
- R 4.6.1, lme4 2.0.6, statsmodels 0.15.0, PHP 8.5.10, Python 3.14.7
Data
- Inputs
- Saved lme4 fixtures: simulated schools with correlated predictors, one predictor, no fixed intercept, intercept only, a singleton group, and a zero group-variance boundary, plus missing values in predictors, outcomes, and group IDs.
Method
Each case is fitted by lme4, statsmodels, and phit, and every quantity is compared with lme4 under the case's stored tolerance. Each case's status is recomputed from the saved values.
- Check
- Comparison with a reference implementation
- Recorded
- 2026-11-01, PHP 8.5.10, macOS 27.0.1 (arm64), commit 066e2e1
Tolerances
|x − r| ≤ 1e-7 + 1e-6 × |r|
Applies to: All reported quantities
x: package value; r: reference value. Absolute error: |x − r|. Max errors are over the values a row compares. statsmodels: that implementation’s error against the same reference.
Results
126 comparisons in 7 cases: 126 pass, 0 fail, 0 skipped.
By case
| Case | Comparisons | Largest abs. error | Status |
|---|---|---|---|
| Simulated schools | 18 | 4.8e-7 | pass |
| One predictor | 18 | 1.5e-6 | pass |
| No fixed intercept | 18 | 1.6e-4 | pass |
| Intercept only | 18 | 4.1e-7 | pass |
| Singleton groups | 18 | 5.8e-7 | pass |
| Zero group variance (boundary) | 18 | 7.1e-15 | pass |
| Listwise deletion of missing values | 18 | 1.1e-6 | pass |
All 126 comparisons
| Case | Quantity | Criterion | Values | Package value | Reference value | Max abs. error | statsmodels | Status |
|---|---|---|---|---|---|---|---|---|
| Simulated schools | Coefficients | |x − r| ≤ 1e-7 + 1e-6 × |r| | 3 | 2.5e-9 | 2.5e-9 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-7 + 1e-6 × |r| | 9 | 3.6e-8 | 3.6e-8 | pass | |||
| Standard errors | |x − r| ≤ 1e-7 + 1e-6 × |r| | 3 | 8.2e-9 | 8.2e-9 | pass | |||
| z statistics | |x − r| ≤ 1e-7 + 1e-6 × |r| | 3 | 7.7e-8 | 7.7e-8 | pass | |||
| p-values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 3 | 5.3e-17 | 5.3e-17 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 6 | 1.9e-8 | 1.9e-8 | pass | |||
| Fitted values (fixed effects only) | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 1.4e-9 | 1.4e-9 | pass | |||
| Fitted values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 2.1e-8 | 2.0e-8 | pass | |||
| Residuals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 2.1e-8 | 2.0e-8 | pass | |||
| Residuals (fixed effects only) | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 1.4e-9 | 1.4e-9 | pass | |||
| Random effects | |x − r| ≤ 1e-7 + 1e-6 × |r| | 12 | 2.0e-8 | 2.0e-8 | pass | |||
| Residual variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 25.35542936 | 25.35542933 | 3.1e-8 | 3.1e-8 | pass | |
| Random-intercept variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 18.10513853 | 18.10513901 | 4.8e-7 | 4.8e-7 | pass | |
| Residual standard deviation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 5.035417497 | 5.035417493 | 3.1e-9 | 3.1e-9 | pass | |
| Intraclass correlation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 0.4165877118 | 0.4165877185 | 6.7e-9 | 6.7e-9 | pass | |
| Log likelihood | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | -775.3741989 | -775.3741989 | 0 | 1.1e-13 | pass | |
| AIC | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 1560.748398 | 1560.748398 | 0 | 2.3e-13 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 247 | 247 | 0 | 0 | pass | |
| One predictor | Coefficients | |x − r| ≤ 1e-7 + 1e-6 × |r| | 2 | 1.9e-9 | 1.9e-9 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-7 + 1e-6 × |r| | 4 | 1.2e-7 | 1.2e-7 | pass | |||
| Standard errors | |x − r| ≤ 1e-7 + 1e-6 × |r| | 2 | 4.1e-8 | 4.1e-8 | pass | |||
| z statistics | |x − r| ≤ 1e-7 + 1e-6 × |r| | 2 | 1.0e-6 | 1.0e-6 | pass | |||
| p-values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 2 | 2.6e-109 | 2.6e-109 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 4 | 8.2e-8 | 8.1e-8 | pass | |||
| Fitted values (fixed effects only) | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 1.7e-9 | 1.6e-9 | pass | |||
| Fitted values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 6.2e-8 | 6.2e-8 | pass | |||
| Residuals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 6.2e-8 | 6.2e-8 | pass | |||
| Residuals (fixed effects only) | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 1.7e-9 | 1.6e-9 | pass | |||
| Random effects | |x − r| ≤ 1e-7 + 1e-6 × |r| | 12 | 6.1e-8 | 6.0e-8 | pass | |||
| Residual variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 28.97253836 | 28.97253825 | 1.1e-7 | 1.1e-7 | pass | |
| Random-intercept variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 19.0025852 | 19.00258669 | 1.5e-6 | 1.5e-6 | pass | |
| Residual standard deviation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 5.382614454 | 5.382614444 | 9.9e-9 | 9.8e-9 | pass | |
| Intraclass correlation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 0.3960924703 | 0.3960924899 | 2.0e-8 | 1.9e-8 | pass | |
| Log likelihood | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | -791.5680446 | -791.5680446 | 1.1e-13 | 2.3e-13 | pass | |
| AIC | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 1591.136089 | 1591.136089 | 2.3e-13 | 4.5e-13 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 248 | 248 | 0 | 0 | pass | |
| No fixed intercept | Coefficients | |x − r| ≤ 1e-7 + 1e-6 × |r| | 2 | 2.8e-9 | 2.9e-9 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-7 + 1e-6 × |r| | 4 | 8.9e-11 | 9.1e-11 | pass | |||
| Standard errors | |x − r| ≤ 1e-7 + 1e-6 × |r| | 2 | 2.8e-10 | 2.8e-10 | pass | |||
| z statistics | |x − r| ≤ 1e-7 + 1e-6 × |r| | 2 | 3.5e-8 | 3.6e-8 | pass | |||
| p-values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 2 | 4.7e-17 | 4.8e-17 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 4 | 3.4e-9 | 3.4e-9 | pass | |||
| Fitted values (fixed effects only) | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 9.7e-8 | 9.9e-8 | pass | |||
| Fitted values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 4.1e-8 | 4.2e-8 | pass | |||
| Residuals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 4.1e-8 | 4.2e-8 | pass | |||
| Residuals (fixed effects only) | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 9.7e-8 | 9.9e-8 | pass | |||
| Random effects | |x − r| ≤ 1e-7 + 1e-6 × |r| | 12 | 7.3e-8 | 7.5e-8 | pass | |||
| Residual variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 25.37650817 | 25.37650807 | 9.6e-8 | 9.9e-8 | pass | |
| Random-intercept variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 2125.62581 | 2125.62597 | 1.6e-4 | 1.6e-4 | pass | |
| Residual standard deviation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 5.037510116 | 5.037510107 | 9.6e-9 | 9.8e-9 | pass | |
| Intraclass correlation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 0.9882024729 | 0.9882024738 | 9.2e-10 | 9.5e-10 | pass | |
| Log likelihood | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | -803.6695094 | -803.6695094 | 1.1e-13 | 0 | pass | |
| AIC | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 1615.339019 | 1615.339019 | 2.3e-13 | 0 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 248 | 248 | 0 | 0 | pass | |
| Intercept only | Coefficients | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 6.6e-11 | 6.6e-11 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 3.4e-8 | 3.4e-8 | pass | |||
| Standard errors | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 1.2e-8 | 1.2e-8 | pass | |||
| z statistics | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 4.0e-7 | 4.0e-7 | pass | |||
| p-values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 0 | 0 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 2 | 2.3e-8 | 2.3e-8 | pass | |||
| Fitted values (fixed effects only) | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 6.6e-11 | 6.6e-11 | pass | |||
| Fitted values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 3.3e-8 | 3.3e-8 | pass | |||
| Residuals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 3.3e-8 | 3.3e-8 | pass | |||
| Residuals (fixed effects only) | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 6.6e-11 | 6.6e-11 | pass | |||
| Random effects | |x − r| ≤ 1e-7 + 1e-6 × |r| | 12 | 3.3e-8 | 3.3e-8 | pass | |||
| Residual variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 85.24170907 | 85.24170914 | 6.9e-8 | 6.9e-8 | pass | |
| Random-intercept variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 21.32255517 | 21.32255476 | 4.1e-7 | 4.1e-7 | pass | |
| Residual standard deviation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 9.232643666 | 9.23264367 | 3.7e-9 | 3.7e-9 | pass | |
| Intraclass correlation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 0.2000910467 | 0.2000910435 | 3.2e-9 | 3.2e-9 | pass | |
| Log likelihood | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | -921.3206334 | -921.3206334 | 0 | 1.1e-13 | pass | |
| AIC | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 1848.641267 | 1848.641267 | 0 | 2.3e-13 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 249 | 249 | 0 | 0 | pass | |
| Singleton groups | Coefficients | |x − r| ≤ 1e-7 + 1e-6 × |r| | 3 | 1.6e-9 | 1.6e-9 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-7 + 1e-6 × |r| | 9 | 4.1e-8 | 4.1e-8 | pass | |||
| Standard errors | |x − r| ≤ 1e-7 + 1e-6 × |r| | 3 | 9.4e-9 | 9.4e-9 | pass | |||
| z statistics | |x − r| ≤ 1e-7 + 1e-6 × |r| | 3 | 9.2e-8 | 9.2e-8 | pass | |||
| p-values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 3 | 7.5e-17 | 7.5e-17 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 6 | 2.0e-8 | 2.0e-8 | pass | |||
| Fitted values (fixed effects only) | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 4.8e-9 | 4.8e-9 | pass | |||
| Fitted values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 5.0e-8 | 5.1e-8 | pass | |||
| Residuals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 5.0e-8 | 5.1e-8 | pass | |||
| Residuals (fixed effects only) | |x − r| ≤ 1e-7 + 1e-6 × |r| | 250 | 4.8e-9 | 4.8e-9 | pass | |||
| Random effects | |x − r| ≤ 1e-7 + 1e-6 × |r| | 13 | 4.7e-8 | 4.7e-8 | pass | |||
| Residual variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 25.27706709 | 25.27706713 | 4.0e-8 | 4.0e-8 | pass | |
| Random-intercept variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 18.03945864 | 18.03945806 | 5.8e-7 | 5.8e-7 | pass | |
| Residual standard deviation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 5.027630365 | 5.027630369 | 3.9e-9 | 3.9e-9 | pass | |
| Intraclass correlation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 0.4164567295 | 0.4164567213 | 8.2e-9 | 8.2e-9 | pass | |
| Log likelihood | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | -775.2271157 | -775.2271157 | 0 | 1.1e-13 | pass | |
| AIC | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 1560.454231 | 1560.454231 | 0 | 2.3e-13 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 247 | 247 | 0 | 0 | pass | |
| Zero group variance (boundary) | Coefficients | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 6.4e-17 | 7.1e-19 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 6.9e-18 | 0 | pass | |||
| Standard errors | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 2.8e-17 | 0 | pass | |||
| z statistics | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 2.7e-16 | 3.0e-18 | pass | |||
| p-values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 2.2e-16 | 0 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 2 | 1.1e-16 | 1.7e-16 | pass | |||
| Fitted values (fixed effects only) | |x − r| ≤ 1e-7 + 1e-6 × |r| | 12 | 6.4e-17 | 7.1e-19 | pass | |||
| Fitted values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 12 | 6.4e-17 | 7.1e-19 | pass | |||
| Residuals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 12 | 1.1e-16 | 7.1e-19 | pass | |||
| Residuals (fixed effects only) | |x − r| ≤ 1e-7 + 1e-6 × |r| | 12 | 1.1e-16 | 7.1e-19 | pass | |||
| Random effects | |x − r| ≤ 1e-7 + 1e-6 × |r| | 4 | 0 | 0 | pass | |||
| Residual variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 0.6666666667 | 0.6666666667 | 0 | 0 | pass | |
| Random-intercept variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 0 | 0 | 0 | 0 | pass | |
| Residual standard deviation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 0.8164965809 | 0.8164965809 | 0 | 0 | pass | |
| Intraclass correlation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 0 | 0 | 0 | 0 | pass | |
| Log likelihood | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | -14.59447175 | -14.59447175 | 3.6e-15 | 1.8e-15 | pass | |
| AIC | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 35.1889435 | 35.1889435 | 7.1e-15 | 7.1e-15 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 11 | 11 | 0 | 0 | pass | |
| Listwise deletion of missing values | Coefficients | |x − r| ≤ 1e-7 + 1e-6 × |r| | 3 | 6.3e-9 | 6.3e-9 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-7 + 1e-6 × |r| | 9 | 8.5e-8 | 8.5e-8 | pass | |||
| Standard errors | |x − r| ≤ 1e-7 + 1e-6 × |r| | 3 | 1.9e-8 | 1.9e-8 | pass | |||
| z statistics | |x − r| ≤ 1e-7 + 1e-6 × |r| | 3 | 1.8e-7 | 1.8e-7 | pass | |||
| p-values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 3 | 1.6e-16 | 1.6e-16 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 6 | 4.4e-8 | 4.4e-8 | pass | |||
| Fitted values (fixed effects only) | |x − r| ≤ 1e-7 + 1e-6 × |r| | 247 | 3.4e-9 | 3.4e-9 | pass | |||
| Fitted values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 247 | 4.8e-8 | 4.8e-8 | pass | |||
| Residuals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 247 | 4.8e-8 | 4.8e-8 | pass | |||
| Residuals (fixed effects only) | |x − r| ≤ 1e-7 + 1e-6 × |r| | 247 | 3.4e-9 | 3.4e-9 | pass | |||
| Random effects | |x − r| ≤ 1e-7 + 1e-6 × |r| | 12 | 4.6e-8 | 4.6e-8 | pass | |||
| Residual variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 25.41820757 | 25.41820764 | 7.5e-8 | 7.5e-8 | pass | |
| Random-intercept variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 18.1432932 | 18.14329208 | 1.1e-6 | 1.1e-6 | pass | |
| Residual standard deviation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 5.041647307 | 5.041647314 | 7.4e-9 | 7.4e-9 | pass | |
| Intraclass correlation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 0.4164983502 | 0.4164983344 | 1.6e-8 | 1.6e-8 | pass | |
| Log likelihood | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | -766.4872381 | -766.4872381 | 2.3e-13 | 3.4e-13 | pass | |
| AIC | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 1542.974476 | 1542.974476 | 4.5e-13 | 6.8e-13 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 244 | 244 | 0 | 0 | pass |
Limitations
Exclusions
- REML, random slopes, and several grouping terms are covered by the R compatibility record.
Known differences
- statsmodels estimates are combined with plug-in GLS covariance to match lme4; statsmodels' own joint-Hessian covariance is a different convention.
- The profile search uses a finite log-ratio range and grid; agreement does not prove a global optimum.
Reproduction
From the repository root
cargo build -p phit --release --locked
.context/notebooks-venv/bin/python docs/verifications/compare_lmm.py- Raw record
- site/verifications/lmm.json
- Full results
- docs/verifications/lmm-results.json
- Input data
- docs/data/schools.csv