Probit regression
Verification
Scope
Coefficients, covariance matrices, standard errors, z statistics, p-values, confidence intervals, fitted values, residuals, log likelihood, deviance, null deviance, and AIC. Identical design matrices and missing-data exclusions across implementations.
References
- Reference
- R glm(family = binomial(link = "probit"))
- Versions
- R 4.6.1, statsmodels 0.15.0, numpy 2.5.4, Python 3.14.7, PHP 8.5.10
Data
- Inputs
- Six saved R fixtures: the simulated learning sample, one predictor, no intercept, intercept only, correlated columns, and imbalanced outcomes, plus a listwise-deletion case with two dropped rows.
Method
Each design is fitted by R, statsmodels, and phit's matrix function, with the intercept supplied as a column. Every quantity is compared with R, and each case's status is recomputed from the saved values.
- Check
- Comparison with a reference implementation
- Recorded
- 2026-11-01, PHP 8.5.10, macOS 27.0.1 (arm64), commit 066e2e1
Tolerances
|x − r| ≤ 1e-9 + 1e-7 × |r|
Applies to: Numerical results
exact equality
Applies to: Row indices, counts, and warnings
x: package value; r: reference value. Absolute error: |x − r|. Max errors are over the values a row compares. statsmodels: that implementation’s error against the same reference.
Results
91 comparisons in 7 cases: 91 pass, 0 fail, 0 skipped.
By case
| Case | Comparisons | Largest abs. error | Status |
|---|---|---|---|
| Simulated learning sample | 13 | 6.4e-10 | pass |
| One predictor | 13 | 2.7e-10 | pass |
| No intercept | 13 | 5.6e-11 | pass |
| Intercept only | 13 | 2.0e-12 | pass |
| Correlated predictors | 13 | 2.7e-10 | pass |
| Imbalanced outcomes | 13 | 1.4e-10 | pass |
| Listwise deletion of missing values | 13 | 3.7e-10 | pass |
All 91 comparisons
| Case | Quantity | Criterion | Values | Package value | Reference value | Max abs. error | statsmodels | Status |
|---|---|---|---|---|---|---|---|---|
| Simulated learning sample | Coefficients | |x − r| ≤ 1e-9 + 1e-7 × |r| | 3 | 1.9e-10 | 1.3e-10 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-9 + 1e-7 × |r| | 9 | 1.7e-10 | 1.5e-10 | pass | |||
| Standard errors | |x − r| ≤ 1e-9 + 1e-7 × |r| | 3 | 1.2e-10 | 1.1e-10 | pass | |||
| z statistics | |x − r| ≤ 1e-9 + 1e-7 × |r| | 3 | 6.4e-10 | 5.8e-10 | pass | |||
| p-values | |x − r| ≤ 1e-9 + 1e-7 × |r| | 3 | 1.9e-10 | 1.8e-10 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-9 + 1e-7 × |r| | 6 | 4.2e-10 | 3.3e-10 | pass | |||
| Fitted values | |x − r| ≤ 1e-9 + 1e-7 × |r| | 120 | 3.3e-11 | 3.5e-11 | pass | |||
| Residuals | |x − r| ≤ 1e-9 + 1e-7 × |r| | 120 | 3.3e-11 | 3.5e-11 | pass | |||
| Log likelihood | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | -72.92760486 | -72.92760486 | 1.4e-10 | 1.4e-14 | pass | |
| Deviance | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 145.8552097 | 145.8552097 | 2.7e-10 | 0 | pass | |
| Null deviance | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 162.2990865 | 162.2990865 | 8.5e-14 | 5.7e-14 | pass | |
| AIC | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 151.8552097 | 151.8552097 | 2.7e-10 | 2.8e-14 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 117 | 117 | 0 | 0 | pass | |
| One predictor | Coefficients | |x − r| ≤ 1e-9 + 1e-7 × |r| | 2 | 5.6e-11 | 1.5e-13 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-9 + 1e-7 × |r| | 4 | 4.2e-12 | 1.3e-12 | pass | |||
| Standard errors | |x − r| ≤ 1e-9 + 1e-7 × |r| | 2 | 7.1e-12 | 2.2e-12 | pass | |||
| z statistics | |x − r| ≤ 1e-9 + 1e-7 × |r| | 2 | 8.9e-11 | 3.3e-11 | pass | |||
| p-values | |x − r| ≤ 1e-9 + 1e-7 × |r| | 2 | 2.5e-14 | 7.2e-15 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-9 + 1e-7 × |r| | 4 | 7.0e-11 | 4.5e-12 | pass | |||
| Fitted values | |x − r| ≤ 1e-9 + 1e-7 × |r| | 120 | 1.2e-11 | 3.1e-14 | pass | |||
| Residuals | |x − r| ≤ 1e-9 + 1e-7 × |r| | 120 | 1.2e-11 | 3.1e-14 | pass | |||
| Log likelihood | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | -72.92761153 | -72.92761153 | 1.4e-10 | 0 | pass | |
| Deviance | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 145.8552231 | 145.8552231 | 2.7e-10 | 2.8e-14 | pass | |
| Null deviance | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 162.2990865 | 162.2990865 | 8.5e-14 | 5.7e-14 | pass | |
| AIC | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 149.8552231 | 149.8552231 | 2.7e-10 | 0 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 118 | 118 | 0 | 0 | pass | |
| No intercept | Coefficients | |x − r| ≤ 1e-9 + 1e-7 × |r| | 2 | 7.7e-12 | 2.8e-15 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-9 + 1e-7 × |r| | 4 | 1.3e-13 | 1.9e-15 | pass | |||
| Standard errors | |x − r| ≤ 1e-9 + 1e-7 × |r| | 2 | 1.2e-12 | 1.8e-14 | pass | |||
| z statistics | |x − r| ≤ 1e-9 + 1e-7 × |r| | 2 | 5.6e-11 | 1.3e-12 | pass | |||
| p-values | |x − r| ≤ 1e-9 + 1e-7 × |r| | 2 | 2.0e-14 | 6.7e-16 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-9 + 1e-7 × |r| | 4 | 1.0e-11 | 3.7e-14 | pass | |||
| Fitted values | |x − r| ≤ 1e-9 + 1e-7 × |r| | 120 | 1.7e-11 | 5.2e-15 | pass | |||
| Residuals | |x − r| ≤ 1e-9 + 1e-7 × |r| | 120 | 1.7e-11 | 5.2e-15 | pass | |||
| Log likelihood | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | -74.53280174 | -74.53280174 | 7.9e-12 | 4.3e-14 | pass | |
| Deviance | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 149.0656035 | 149.0656035 | 1.6e-11 | 5.7e-14 | pass | |
| Null deviance | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 166.3553233 | 166.3553233 | 0 | 0 | pass | |
| AIC | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 153.0656035 | 153.0656035 | 1.6e-11 | 8.5e-14 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 118 | 118 | 0 | 0 | pass | |
| Intercept only | Coefficients | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 8.3e-17 | 1.9e-16 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 6.9e-18 | 1.0e-17 | pass | |||
| Standard errors | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 2.8e-17 | 4.2e-17 | pass | |||
| z statistics | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 0 | 2.7e-15 | pass | |||
| p-values | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 2.0e-12 | 2.9e-16 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-9 + 1e-7 × |r| | 2 | 2.2e-16 | 3.6e-16 | pass | |||
| Fitted values | |x − r| ≤ 1e-9 + 1e-7 × |r| | 120 | 5.6e-17 | 5.6e-17 | pass | |||
| Residuals | |x − r| ≤ 1e-9 + 1e-7 × |r| | 120 | 5.6e-17 | 1.1e-16 | pass | |||
| Log likelihood | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | -81.14954326 | -81.14954326 | 5.7e-14 | 4.3e-14 | pass | |
| Deviance | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 162.2990865 | 162.2990865 | 1.1e-13 | 8.5e-14 | pass | |
| Null deviance | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 162.2990865 | 162.2990865 | 8.5e-14 | 5.7e-14 | pass | |
| AIC | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 164.2990865 | 164.2990865 | 1.1e-13 | 8.5e-14 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 119 | 119 | 0 | 0 | pass | |
| Correlated predictors | Coefficients | |x − r| ≤ 1e-9 + 1e-7 × |r| | 3 | 5.6e-11 | 3.2e-11 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-9 + 1e-7 × |r| | 9 | 2.2e-11 | 3.9e-11 | pass | |||
| Standard errors | |x − r| ≤ 1e-9 + 1e-7 × |r| | 3 | 1.2e-11 | 2.2e-11 | pass | |||
| z statistics | |x − r| ≤ 1e-9 + 1e-7 × |r| | 3 | 1.8e-10 | 1.3e-10 | pass | |||
| p-values | |x − r| ≤ 1e-9 + 1e-7 × |r| | 3 | 4.5e-11 | 2.0e-11 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-9 + 1e-7 × |r| | 6 | 7.6e-11 | 7.6e-11 | pass | |||
| Fitted values | |x − r| ≤ 1e-9 + 1e-7 × |r| | 120 | 1.3e-11 | 4.3e-12 | pass | |||
| Residuals | |x − r| ≤ 1e-9 + 1e-7 × |r| | 120 | 1.3e-11 | 4.3e-12 | pass | |||
| Log likelihood | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | -72.86030716 | -72.86030716 | 1.4e-10 | 1.4e-14 | pass | |
| Deviance | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 145.7206143 | 145.7206143 | 2.7e-10 | 0 | pass | |
| Null deviance | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 162.2990865 | 162.2990865 | 8.5e-14 | 5.7e-14 | pass | |
| AIC | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 151.7206143 | 151.7206143 | 2.7e-10 | 2.8e-14 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 117 | 117 | 0 | 0 | pass | |
| Imbalanced outcomes | Coefficients | |x − r| ≤ 1e-9 + 1e-7 × |r| | 3 | 1.0e-10 | 6.6e-13 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-9 + 1e-7 × |r| | 9 | 5.7e-11 | 3.8e-12 | pass | |||
| Standard errors | |x − r| ≤ 1e-9 + 1e-7 × |r| | 3 | 1.7e-11 | 1.2e-12 | pass | |||
| z statistics | |x − r| ≤ 1e-9 + 1e-7 × |r| | 3 | 1.0e-10 | 1.3e-12 | pass | |||
| p-values | |x − r| ≤ 1e-9 + 1e-7 × |r| | 3 | 2.0e-11 | 9.1e-14 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-9 + 1e-7 × |r| | 6 | 1.4e-10 | 2.9e-12 | pass | |||
| Fitted values | |x − r| ≤ 1e-9 + 1e-7 × |r| | 120 | 3.8e-12 | 3.8e-14 | pass | |||
| Residuals | |x − r| ≤ 1e-9 + 1e-7 × |r| | 120 | 3.8e-12 | 3.8e-14 | pass | |||
| Log likelihood | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | -14.99359308 | -14.99359308 | 3.4e-11 | 3.6e-15 | pass | |
| Deviance | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 29.98718615 | 29.98718615 | 6.7e-11 | 7.1e-15 | pass | |
| Null deviance | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 35.07473904 | 35.07473904 | 1.6e-13 | 1.6e-13 | pass | |
| AIC | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 35.98718615 | 35.98718615 | 6.7e-11 | 7.1e-15 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 117 | 117 | 0 | 0 | pass | |
| Listwise deletion of missing values | Coefficients | |x − r| ≤ 1e-9 + 1e-7 × |r| | 3 | 5.9e-11 | 3.9e-14 | pass | ||
| Covariance matrix | |x − r| ≤ 1e-9 + 1e-7 × |r| | 9 | 1.9e-11 | 7.0e-15 | pass | |||
| Standard errors | |x − r| ≤ 1e-9 + 1e-7 × |r| | 3 | 1.3e-11 | 6.1e-15 | pass | |||
| z statistics | |x − r| ≤ 1e-9 + 1e-7 × |r| | 3 | 5.6e-11 | 1.6e-13 | pass | |||
| p-values | |x − r| ≤ 1e-9 + 1e-7 × |r| | 3 | 7.1e-12 | 5.4e-14 | pass | |||
| Confidence intervals | |x − r| ≤ 1e-9 + 1e-7 × |r| | 6 | 8.5e-11 | 4.9e-14 | pass | |||
| Fitted values | |x − r| ≤ 1e-9 + 1e-7 × |r| | 118 | 1.5e-11 | 1.2e-14 | pass | |||
| Residuals | |x − r| ≤ 1e-9 + 1e-7 × |r| | 118 | 1.5e-11 | 1.2e-14 | pass | |||
| Log likelihood | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | -71.01719103 | -71.01719103 | 1.9e-10 | 1.4e-14 | pass | |
| Deviance | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 142.0343821 | 142.0343821 | 3.7e-10 | 2.8e-14 | pass | |
| Null deviance | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 159.4569404 | 159.4569404 | 2.3e-13 | 2.3e-13 | pass | |
| AIC | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 148.0343821 | 148.0343821 | 3.7e-10 | 2.8e-14 | pass | |
| Residual degrees of freedom | |x − r| ≤ 1e-9 + 1e-7 × |r| | 1 | 115 | 115 | 0 | 0 | pass |
Limitations
Exclusions
- Separated and quasi-separated designs are rejected; their tests are in the test-suite record, not compared with R.
Known differences
- R intervals come from confint.default() (Wald), not profile-likelihood confint().
- phit reports the covariance at the converged coefficients; R glm() evaluates it at the last IRLS iterate. With the tight fitting tolerances used here the difference is negligible. See the R compatibility page for details.
- The covariance is the inverse expected information, as in glm(); a discrete-model probit fit with observed-Hessian covariance can differ.
Reproduction
From the repository root
cargo build -p phit --release --locked
.context/notebooks-venv/bin/python docs/verifications/compare_probit.py- Raw record
- site/verifications/probit.json
- Full results
- docs/verifications/probit-results.json
- Input data
- docs/data/learning.csv