phit

Benchmarks and verification

The same random-intercept ML model in R, Python and PHP

The math walkthrough derives the marginal Gaussian likelihood, group whitening, variance profiling and shrunken conditional group estimates. All runtimes fit the same fixed columns, retained observations and grouping IDs.

from pathlib import Path
import sys
import pandas as pd
ROOT=next(p for p in [Path.cwd(),*Path.cwd().parents] if (p/'Cargo.toml').exists())
sys.path.insert(0,str(ROOT/'docs/verifications'))
from compare_lmm import cases,compare_case
results=[compare_case(c) for c in cases()]
first=results[0]
print(first['versions'])
rows=[]
for implementation in ['R','Python','php-stats']:
    fit=first[implementation]
    for j,label in enumerate(['intercept','study hours','prior score']):
        rows.append(dict(implementation=implementation,term=label,estimate=fit['coefficients'][j],SE=fit['std_errors'][j],z=fit['z_statistics'][j],p=fit['p_values'][j],lower95=fit['conf_int'][j][0],upper95=fit['conf_int'][j][1]))
display(pd.DataFrame(rows))
display(pd.DataFrame([dict(case=r['name'],Python_backend=r['python_diagnostics']['backend'],
                           max_absolute_difference_vs_R=max(e for values in r['max_absolute_errors_vs_R'].values() for e in values.values()),
                           PHP_boundary=r['php-stats']['boundary'],missing_warnings=len(r['warnings'])) for r in results]))
{'R': 'R version 4.6.1 (2026-06-24)', 'lme4': '2.0.6', 'statsmodels': '0.15.0', 'PHP': '8.5.10', 'Python': '3.14.7'}
implementation term estimate SE z p lower95 upper95
0 R intercept 46.655187 2.204067 21.167768 1.893116e-99 42.335294 50.975079
1 R study hours 1.613908 0.160247 10.071362 7.394689e-24 1.299829 1.927987
2 R prior score 0.245431 0.041713 5.883790 4.009775e-09 0.163675 0.327187
3 Python intercept 46.655187 2.204067 21.167768 1.893113e-99 42.335294 50.975079
4 Python study hours 1.613908 0.160247 10.071362 7.394690e-24 1.299829 1.927987
5 Python prior score 0.245431 0.041713 5.883790 4.009775e-09 0.163675 0.327187
6 php-stats intercept 46.655187 2.204067 21.167768 1.893113e-99 42.335294 50.975079
7 php-stats study hours 1.613908 0.160247 10.071362 7.394690e-24 1.299829 1.927987
8 php-stats prior score 0.245431 0.041713 5.883790 4.009775e-09 0.163675 0.327187
case Python_backend max_absolute_difference_vs_R PHP_boundary missing_warnings
0 simulated_schools statsmodels MixedLM (ML) 4.783827e-07 False 0
1 simple statsmodels MixedLM (ML) 1.490081e-06 False 0
2 no_intercept statsmodels MixedLM (ML) 1.641136e-04 False 0
3 intercept_only statsmodels MixedLM (ML) 4.142385e-07 False 0
4 singleton_groups statsmodels MixedLM (ML) 5.790743e-07 False 0
5 boundary NumPy exact zero-random-variance limit 7.105427e-15 True 0
6 listwise_missing statsmodels MixedLM (ML) 1.126244e-06 False 1

Comparisons include every fitted value/residual, fixed-effect inference, group effects, variance components, ICC and marginal likelihood at tolerance 1e-7 + 1e-6 × abs(reference). Group labels, deletion positions and counts are checked exactly. Optimizer evaluation counts are not expected to match.

Equivalent model calls

library(lme4)
students <- read.csv('docs/data/schools.csv')
fit <- lmer(score ~ study_hours + prior_score + (1|school),
            data=students, REML=FALSE,
            control=lmerControl(optimizer='bobyqa',optCtrl=list(rhoend=1e-10)))
fixef(fit)
vcov(fit)
ranef(fit)
import statsmodels.api as sm
students = pd.read_csv('docs/data/schools.csv')
x = sm.add_constant(students[['study_hours','prior_score']])
fit = sm.MixedLM(students.score, x, groups=students.school).fit(
    reml=False, method='bfgs', gtol=1e-10, maxiter=1000)
fit.fe_params, fit.scale, fit.cov_re
$x = $y = $groups = [];
foreach ($students as $student) {
    $x[] = [1.0, $student['study_hours'], $student['prior_score']];
    $y[] = $student['score'];
    $groups[] = $student['school'];
}
$result = stats_lmm(x: $x, y: $y, groups: $groups);

These PHP records are already numeric; examples/lmm.php supplies the complete CSV conversion. No formulas or implicit intercepts are interpreted by the native call. The comparison runner gives R the exact design columns with formula y ~ 0 + x0 + ... + (1|group).

Conventions that must match

ML is explicit: R and statsmodels otherwise support/default to REML, which fits a different variance criterion. Likelihood here integrates over group effects. Fixed-effect standard errors use plug-in GLS covariance (X’ V^-1 X)^-1, matching lme4. statsmodels’ reported joint observed-Hessian covariance can differ, so the Python runner explicitly recalculates the declared GLS convention using its fitted variance components; it does not claim its unmodified bse_fe matches every case. Normal z-based Wald intervals are constructed explicitly for every reference.

Our df_residual=n-p is informational. R mixed-model df methods may include the two variance parameters; no finite-sample t degrees of freedom are claimed here. AIC counts p+2 parameters in every case, including a zero variance boundary.

The exact tau²=0 fixture is checked against lme4 and PHP, with a separately labeled NumPy OLS-limit reference. It is not represented as a converged statsmodels mixed fit. At this boundary, variance is RSS/n and standard errors differ from OLS’s unbiased-scale inference. This explicit check avoids treating an approximate positive-variance optimizer solution as proof of exact boundary behavior.

keys=['residual_variance','random_intercept_variance','icc','log_likelihood','aic']
display(pd.DataFrame([dict(implementation=name,**{k:first[name][k] for k in keys}) for name in ['R','Python','php-stats']]))
print('Python fit warnings:', first['python_diagnostics']['warnings'])
missing=results[-1]
print('Dropped original rows:',missing['php-stats']['dropped_rows'])
print('Missing-data warning:',missing['warnings'])
print('Schools and effects:',list(zip(first['php-stats']['group_labels'],first['php-stats']['random_effects'])))
implementation residual_variance random_intercept_variance icc log_likelihood aic
0 R 25.355429 18.105139 0.416588 -775.374199 1560.748398
1 Python 25.355429 18.105139 0.416588 -775.374199 1560.748398
2 php-stats 25.355429 18.105139 0.416588 -775.374199 1560.748398
Python fit warnings: []
Dropped original rows: [1, 4, 8]
Missing-data warning: [{'severity': 2, 'message': 'stats_lmm(): stats_lmm dropped 3 of 250 observations with missing values (listwise deletion)'}]
Schools and effects: [(107, 0.13413303204091775), (114, 1.6086975147685796), (121, -1.7846684515271147), (128, -5.564364724188796), (135, 0.24907018967277064), (142, 2.2564389505860314), (149, 10.331238035927292), (156, -5.104699176036904), (163, 2.102747547050482), (170, 0.5690811113270031), (177, 0.07903267727850147), (184, -4.87670670689903)]

See the verification record for rejection tests, finite-search limitations and reproduction instructions. A converged numerical fit does not establish that the model assumptions or small-group Wald inference are adequate for an application.

From phpstats/phit@76b22f0.