Linear mixed model, published data
Verification
Scope
The quantities of the linear mixed model record on sleepstudy, Orthodont, Machines, and Hsb82, fitted by ML, with nlme as a second reference.
References
- Reference
- R lme4 lmer(REML = FALSE, bobyqa rhoend = 1e-10) and nlme lme()
- Versions
- R 4.6.1, lme4 2.0.6, statsmodels 0.15.0, PHP 8.5.10, Python 3.14.7, nlme 3.1.169
Data
- Inputs
- Four published data sets: sleepstudy, Orthodont, Machines, and Hsb82. Their provenance and licenses are listed with the data.
Method
Each case is fitted by lme4, statsmodels, nlme, and phit, and every quantity is compared with lme4 under the case's stored tolerance. Each case's status is recomputed from the saved values.
- Check
- Comparison with a reference implementation
- Recorded
- 2026-11-01, PHP 8.5.10, macOS 27.0.1 (arm64), commit 066e2e1
Tolerances
|x − r| ≤ 1e-7 + 1e-6 × |r|
Applies to: All other reported quantities
|x − r| ≤ 1e-6 + 1e-6 × |r|
Applies to: Fitted values (fixed effects only), fitted values, residuals, residuals (fixed effects only)
|x − r| ≤ 1e-7 + 1e-10 × |r|
Applies to: Log likelihood, AIC
x: package value; r: reference value. Absolute error: |x − r|. Max errors are over the values a row compares. nlme, statsmodels: each implementation’s error against the same reference.
Results
72 comparisons in 4 cases: 72 pass, 0 fail, 0 skipped.
By case
| Case | Comparisons | Largest abs. error | Status |
|---|---|---|---|
| sleepstudy | 18 | 1.5e-5 | pass |
| Orthodont | 18 | 5.8e-8 | pass |
| Machines | 18 | 3.6e-7 | pass |
| Hsb82 | 18 | 2.7e-6 | pass |
All 72 comparisons
| Case | Quantity | Criterion | Values | Package value | Reference value | Max abs. error | nlme | statsmodels | Status |
|---|---|---|---|---|---|---|---|---|---|
| sleepstudy | Coefficients | |x − r| ≤ 1e-7 + 1e-6 × |r| | 2 | 1.3e-12 | 1.2e-12 | 4.8e-13 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-7 + 1e-6 × |r| | 4 | 8.3e-7 | 6.3e-7 | 8.4e-7 | pass | |||
| Standard errors | |x − r| ≤ 1e-7 + 1e-6 × |r| | 2 | 4.4e-8 | 3.3e-8 | 4.4e-8 | pass | |||
| z statistics | |x − r| ≤ 1e-7 + 1e-6 × |r| | 2 | 1.2e-7 | 9.2e-8 | 1.2e-7 | pass | |||
| p-values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 2 | 6.1e-46 | 4.6e-46 | 6.2e-46 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 4 | 8.6e-8 | 6.5e-8 | 8.6e-8 | pass | |||
| Fitted values (fixed effects only) | |x − r| ≤ 1e-6 + 1e-6 × |r| | 180 | 1.3e-12 | 1.2e-12 | 4.8e-13 | pass | |||
| Fitted values | |x − r| ≤ 1e-6 + 1e-6 × |r| | 180 | 7.0e-8 | 5.2e-8 | 7.0e-8 | pass | |||
| Residuals | |x − r| ≤ 1e-6 + 1e-6 × |r| | 180 | 7.0e-8 | 5.2e-8 | 7.0e-8 | pass | |||
| Residuals (fixed effects only) | |x − r| ≤ 1e-6 + 1e-6 × |r| | 180 | 1.3e-12 | 1.2e-12 | 4.8e-13 | pass | |||
| Random effects | |x − r| ≤ 1e-7 + 1e-6 × |r| | 18 | 7.0e-8 | 5.2e-8 | 7.0e-8 | pass | |||
| Residual variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 954.5278346 | 954.5278335 | 1.2e-6 | 8.7e-7 | 1.2e-6 | pass | |
| Random-intercept variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 1296.87004 | 1296.870056 | 1.5e-5 | 1.2e-5 | 1.5e-5 | pass | |
| Residual standard deviation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 30.89543388 | 30.89543386 | 1.9e-8 | 1.4e-8 | 1.9e-8 | pass | |
| Intraclass correlation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 0.5760288107 | 0.5760288139 | 3.2e-9 | 2.4e-9 | 3.2e-9 | pass | |
| Log likelihood | |x − r| ≤ 1e-7 + 1e-10 × |r| | 1 | -897.0393215 | -897.0393215 | 1.1e-13 | 2.3e-13 | 1.1e-13 | pass | |
| AIC | |x − r| ≤ 1e-7 + 1e-10 × |r| | 1 | 1802.078643 | 1802.078643 | 2.3e-13 | 4.5e-13 | 2.3e-13 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 178 | 178 | 0 | 0 | 0 | pass | |
| Orthodont | Coefficients | |x − r| ≤ 1e-7 + 1e-6 × |r| | 4 | 1.2e-13 | 1.3e-13 | 1.8e-13 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-7 + 1e-6 × |r| | 16 | 8.3e-10 | 5.1e-9 | 8.3e-10 | pass | |||
| Standard errors | |x − r| ≤ 1e-7 + 1e-6 × |r| | 4 | 3.1e-10 | 2.0e-9 | 3.1e-10 | pass | |||
| z statistics | |x − r| ≤ 1e-7 + 1e-6 × |r| | 4 | 2.7e-8 | 1.7e-7 | 2.7e-8 | pass | |||
| p-values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 4 | 2.1e-10 | 1.3e-9 | 2.1e-10 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 8 | 6.2e-10 | 3.8e-9 | 6.2e-10 | pass | |||
| Fitted values (fixed effects only) | |x − r| ≤ 1e-6 + 1e-6 × |r| | 108 | 3.6e-14 | 4.3e-14 | 6.4e-14 | pass | |||
| Fitted values | |x − r| ≤ 1e-6 + 1e-6 × |r| | 108 | 1.3e-8 | 7.9e-8 | 1.3e-8 | pass | |||
| Residuals | |x − r| ≤ 1e-6 + 1e-6 × |r| | 108 | 1.3e-8 | 7.9e-8 | 1.3e-8 | pass | |||
| Residuals (fixed effects only) | |x − r| ≤ 1e-6 + 1e-6 × |r| | 108 | 3.6e-14 | 4.3e-14 | 6.4e-14 | pass | |||
| Random effects | |x − r| ≤ 1e-7 + 1e-6 × |r| | 27 | 1.3e-8 | 7.9e-8 | 1.3e-8 | pass | |||
| Residual variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 1.874596661 | 1.874596671 | 9.8e-9 | 6.1e-8 | 9.8e-9 | pass | |
| Random-intercept variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 3.030561693 | 3.030561636 | 5.8e-8 | 3.6e-7 | 5.8e-8 | pass | |
| Residual standard deviation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 1.369159107 | 1.369159111 | 3.6e-9 | 2.2e-8 | 3.6e-9 | pass | |
| Intraclass correlation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 0.6178315713 | 0.6178315655 | 5.7e-9 | 3.5e-8 | 5.7e-9 | pass | |
| Log likelihood | |x − r| ≤ 1e-7 + 1e-10 × |r| | 1 | -214.319529 | -214.319529 | 8.5e-14 | 1.7e-13 | 1.1e-13 | pass | |
| AIC | |x − r| ≤ 1e-7 + 1e-10 × |r| | 1 | 440.639058 | 440.639058 | 1.7e-13 | 3.4e-13 | 2.3e-13 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 104 | 104 | 0 | 0 | 0 | pass | |
| Machines | Coefficients | |x − r| ≤ 1e-7 + 1e-6 × |r| | 3 | 7.8e-14 | 8.9e-15 | 7.1e-14 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-7 + 1e-6 × |r| | 9 | 6.0e-8 | 1.6e-7 | 5.9e-8 | pass | |||
| Standard errors | |x − r| ≤ 1e-7 + 1e-6 × |r| | 3 | 1.5e-8 | 3.9e-8 | 1.4e-8 | pass | |||
| z statistics | |x − r| ≤ 1e-7 + 1e-6 × |r| | 3 | 1.8e-7 | 4.9e-7 | 1.8e-7 | pass | |||
| p-values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 3 | 6.8e-22 | 1.8e-21 | 6.8e-22 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 6 | 2.9e-8 | 7.7e-8 | 2.8e-8 | pass | |||
| Fitted values (fixed effects only) | |x − r| ≤ 1e-6 + 1e-6 × |r| | 54 | 8.5e-14 | 1.4e-14 | 7.1e-14 | pass | |||
| Fitted values | |x − r| ≤ 1e-6 + 1e-6 × |r| | 54 | 7.4e-9 | 2.0e-8 | 7.4e-9 | pass | |||
| Residuals | |x − r| ≤ 1e-6 + 1e-6 × |r| | 54 | 7.4e-9 | 2.0e-8 | 7.4e-9 | pass | |||
| Residuals (fixed effects only) | |x − r| ≤ 1e-6 + 1e-6 × |r| | 54 | 8.5e-14 | 1.4e-14 | 7.1e-14 | pass | |||
| Random effects | |x − r| ≤ 1e-7 + 1e-6 × |r| | 6 | 7.4e-9 | 2.0e-8 | 7.4e-9 | pass | |||
| Residual variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 9.579513889 | 9.579513908 | 1.9e-8 | 5.0e-8 | 1.9e-8 | pass | |
| Random-intercept variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 21.93366512 | 21.93366476 | 3.6e-7 | 9.8e-7 | 3.6e-7 | pass | |
| Residual standard deviation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 3.095078979 | 3.095078982 | 3.0e-9 | 8.2e-9 | 3.0e-9 | pass | |
| Intraclass correlation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 0.6960156294 | 0.6960156254 | 3.9e-9 | 1.1e-8 | 3.9e-9 | pass | |
| Log likelihood | |x − r| ≤ 1e-7 + 1e-10 × |r| | 1 | -146.8516279 | -146.8516279 | 2.8e-14 | 0 | 2.8e-14 | pass | |
| AIC | |x − r| ≤ 1e-7 + 1e-10 × |r| | 1 | 303.7032558 | 303.7032558 | 5.7e-14 | 0 | 5.7e-14 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 51 | 51 | 0 | 0 | 0 | pass | |
| Hsb82 | Coefficients | |x − r| ≤ 1e-7 + 1e-6 × |r| | 4 | 5.4e-9 | 9.8e-9 | 5.4e-9 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-7 + 1e-6 × |r| | 16 | 1.2e-8 | 2.2e-8 | 1.2e-8 | pass | |||
| Standard errors | |x − r| ≤ 1e-7 + 1e-6 × |r| | 4 | 1.6e-8 | 2.9e-8 | 1.6e-8 | pass | |||
| z statistics | |x − r| ≤ 1e-7 + 1e-6 × |r| | 4 | 2.7e-6 | 4.9e-6 | 2.7e-6 | pass | |||
| p-values | |x − r| ≤ 1e-7 + 1e-6 × |r| | 4 | 4.2e-11 | 7.7e-11 | 4.2e-11 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-7 + 1e-6 × |r| | 8 | 3.6e-8 | 6.6e-8 | 3.6e-8 | pass | |||
| Fitted values (fixed effects only) | |x − r| ≤ 1e-6 + 1e-6 × |r| | 7,185 | 6.5e-9 | 1.2e-8 | 6.5e-9 | pass | |||
| Fitted values | |x − r| ≤ 1e-6 + 1e-6 × |r| | 7,185 | 2.4e-7 | 4.4e-7 | 2.4e-7 | pass | |||
| Residuals | |x − r| ≤ 1e-6 + 1e-6 × |r| | 7,185 | 2.4e-7 | 4.4e-7 | 2.4e-7 | pass | |||
| Residuals (fixed effects only) | |x − r| ≤ 1e-6 + 1e-6 × |r| | 7,185 | 6.5e-9 | 1.2e-8 | 6.5e-9 | pass | |||
| Random effects | |x − r| ≤ 1e-7 + 1e-6 × |r| | 160 | 2.4e-7 | 4.4e-7 | 2.4e-7 | pass | |||
| Residual variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 37.01821718 | 37.0182171 | 7.5e-8 | 1.4e-7 | 7.5e-8 | pass | |
| Random-intercept variance | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 2.306016959 | 2.306017246 | 2.9e-7 | 5.2e-7 | 2.9e-7 | pass | |
| Residual standard deviation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 6.084259789 | 6.084259783 | 6.2e-9 | 1.1e-8 | 6.2e-9 | pass | |
| Intraclass correlation | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 0.05864111558 | 0.05864112255 | 7.0e-9 | 1.3e-8 | 7.0e-9 | pass | |
| Log likelihood | |x − r| ≤ 1e-7 + 1e-10 × |r| | 1 | -23274.1597 | -23274.1597 | 7.3e-12 | 3.6e-12 | 7.3e-12 | pass | |
| AIC | |x − r| ≤ 1e-7 + 1e-10 × |r| | 1 | 46560.3194 | 46560.3194 | 1.5e-11 | 7.3e-12 | 1.5e-11 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-7 + 1e-6 × |r| | 1 | 7,181 | 7,181 | 0 | 0 | 0 | pass |
Limitations
Exclusions
- REML, random slopes, and several grouping terms are covered by the R compatibility record.
Known differences
- statsmodels estimates are combined with plug-in GLS covariance to match lme4; statsmodels' own joint-Hessian covariance is a different convention.
- The profile search uses a finite log-ratio range and grid; agreement does not prove a global optimum.
- nlme is a second reference for the same ML fits.
Reproduction
From the repository root
cargo build -p phit --release --locked
.context/notebooks-venv/bin/python docs/verifications/compare_lmm.py --public- Raw record
- site/verifications/lmm-public.json
- Full results
- docs/verifications/lmm-public-results.json
- Input data
- docs/data/lmm