Verification

Routine checks

From the repository root:

cargo fmt --all --check
cargo test --workspace --locked
cargo build -p phit --locked
# macOS; use libphit.so on Linux
export STATS_EXTENSION="$PWD/target/debug/libphit.dylib"
php -n -d "extension=$STATS_EXTENSION" -m
php -n -d "extension=$STATS_EXTENSION" -r 'echo phpversion("phit"), PHP_EOL;'
php -n -d "extension=$STATS_EXTENSION" tests/ols.php
php -n -d "extension=$STATS_EXTENSION" tests/logistic.php
php -n -d "extension=$STATS_EXTENSION" examples/ols.php
php -n -d "extension=$STATS_EXTENSION" examples/logistic.php

cargo test verifies the independent Rust engine. The extension crate disables its Rust test harness; explicitly building it and running PHP integration tests are separate required checks for binding changes. For a release/deployment check, build with –release and run the PHP scripts against target/release instead. For documentation-only changes, check links, commands and claims against source; rerun numerical tests when behavior or implementation changes.

Probability and descriptive APIs

The offline release gate runs tests/distributions.php, tests/descriptive.php and examples/distributions.php, along with the Rust distribution/descriptive suites. Distribution checks include 8,618 reference comparisons, 25 independent mpmath adjudications and 43 documented boundary differences. Descriptive tests add 145 R quantities, type-7 scalar/batch identities, strict/explicit-drop missing policies, pairwise alignment, weight definitions and arithmetic overflow errors. The distribution record describes provenance, reference limitations and operation-specific tolerances. The utility contract documents undefined correlation separately from numerical failure. tests/smoothing.php compares 15,639 density, bandwidth, interpolation, and ECDF quantities with R at relative 1e-12.

Refresh PHP error evidence without changing independent expected values:

PHP=/path/to/matching/php .context/v02/python/bin/python docs/verifications/compare_distributions.py --stored-references
python3 scripts/reference-manifest.py
python3 scripts/distribution-docs.py

Reference regeneration with R or 60-digit mpmath is intentional and separate; see the comparison page. Distribution scalar/batch/Rust/R/SciPy benchmarks and the seven-model pre-v0.2 preservation comparison run separately from the gate, with numerical agreement recorded before distribution timings. Seeded sampling adds 25 independently generated Python/R cases: 2,000 Rust engine/quantile values and 1,616 PHP transformed references, plus repeatability, prefix, state-isolation, domain/limit and moment checks. tests/rng.php and examples/rng.php run in the release gate. The readiness assessment records remaining release steps.

Posterior-draw kernels

The release gate includes crates/stats-core/tests/draws.rs and tests/draws.php. Both consume 25 independent 80-digit Python Decimal cases, including covariance-map identities, affine round trips and log determinants. Strict validation includes zero weights, one effective draw and singular covariance, with explicit failures for map/affine numerical singularity. Provenance, formulas, numerical tolerances, reproduction commands and limitations are recorded in draw numerical notes. The benchmark validates complete outputs against R before recording timings; it runs separately from the offline release gate.

PHP intercept and offset contracts

tests/intercept.php checks all seven PHP fitters with predictors-only inputs against the same complete design passed with intercept: false. Those results must be identical, including covariance, likelihood, group effects, row accounting, and diagnostics. Existing fixtures retain their literal matrices and their PHP runners explicitly opt out of insertion. No-intercept and intercept-only models are exercised separately; automatic insertion never changes random-design z.

Poisson exposure checks independently calculate the intercept-only weighted MLE, its standard error, likelihood and fitted counts. They use absolute tolerances 1e-10 for the coefficient/SE and 1e-9 for likelihood/counts. Rescaling exposure must shift only the intercept and preserve counts. Default-intercept weighted/offset fits also match stored R fixtures under the existing 1e-7 absolute + 1e-6 relative tolerance. Missing offsets participate in the same warning/deletion/strict-rejection contract as other inputs. The offline release gate includes this suite and the runnable exposure example.

Reference and tolerances

tests/fixtures/ols-r.json contains deterministic results generated by base R 4.6.1 lm(), summary(), confint() and logLik(). R is not a runtime or test dependency. To regenerate: Rscript tests/generate-fixtures.R. The saved simulated housing observations in docs/data/housing.csv are shared by the PHP example, notebooks and one- and two-predictor reference fits. To regenerate that dataset intentionally, run Rscript tests/generate-learning-data.R first, then regenerate fixtures, the three-way report and both notebook outputs. The dataset generator specifies its RNG and seed; ordinary rendering and testing read the saved observations and do not generate fresh random samples. Rust and PHP tests compare noisy simple, multiple, no-intercept and correlated full-rank regressions, and include perfect/zero fits. Tolerance is 1e-9 + 1e-7 * abs(reference) for each compared quantity. Perfect fits compare coefficients, predictions, residuals and standard errors, but omit unstable t/p, R-squared and likelihood comparisons; Rust also compares finite confidence intervals. Zero-fit conventions are tested directly. R’s undefined quantities are stored as JSON null. Invalid shapes, insufficient observations and rank failure are tested separately because this implementation rejects cases that R may handle by dropping aliased coefficients.

Current coverage and limits

For ordinary fixtures, comparisons include coefficients, covariance, fitted values, residuals, standard errors, t-statistics, p-values, confidence intervals, RSS, residual variance, sigma, R-squared, adjusted R-squared and likelihood. Residual degrees of freedom is also checked. This is a small deterministic suite, not proof of correctness for every floating-point input. CI runs it on Linux and macOS, and the packaging workflow runs it on Linux x86_64 and aarch64 at PHP 8.3, 8.4 and 8.5, including a PHP-FPM request. It does not exercise extreme scales or every array-key pattern.

Maintaining evidence

When changing an estimator or inference calculation, add a case that distinguishes the intended behavior from the old behavior. Compare against R where its conventions match ours, and test deliberate differences explicitly. Keep fixture inputs, generator and expected outputs together. Inspect regenerated fixture diffs; do not simply relax a tolerance to hide a discrepancy. When reporting verification, identify which Rust/PHP checks ran and which platform was used.

The current absolute-plus-relative tolerance supports near-zero quantities as well as larger ones. It does not establish relative accuracy for p-values much smaller than the absolute tolerance. If a change depends on extreme tails, add focused reference tests and a suitable tolerance for those values.

Executed three-way evidence

The OLS verification report and verification notebook estimate the same cases afresh in R, Python statsmodels and PHP. Python/R are optional development tools; the ordinary Rust/PHP tests remain independent of them. The math notebook exposes each intermediate calculation. Follow notebook setup to regenerate the comparison report and saved notebook outputs. Each future implemented estimator should have a walkthrough and equivalent-model evidence where reference support exists.

Scale and performance checks

The OLS benchmark report records release-extension timings, input loading, whole-process time and peak resident memory against R and statsmodels. Its standard cases include 1,000, 100,000 and one million observations with three coefficients, plus 100,000 observations with eleven coefficients (intercept included). It checks all inference and scalar statistics and ten sampled fitted/residual values against R; it does not compare every observation-length output on these large cases. These checks run separately from the small unit/reference suite.

Run .context/notebooks-venv/bin/python tests/benchmarks/run.py after building the release extension with matching PHP headers. Review raw results and numerical agreement before reporting speed comparisons. Follow the report for timing boundaries, backend settings, hardware, reproduction and evidence limitations.

Binary logistic regression

tests/fixtures/logistic-r.json stores base R glm(..., family=binomial()) results for simulated students, one predictor, multiple predictors, no intercept, intercept only, highly correlated full-rank columns, and imbalanced outcomes. Regenerate intentionally with Rscript tests/generate-logistic-fixtures.R; this also regenerates the saved teaching observations in docs/data/learning.csv. Wald intervals use confint.default(), since glm’s ordinary confint() uses profile likelihood. Reference fitting uses epsilon 1e-14 to stabilize final information/covariance agreement.

Rust and PHP tests compare all statistical fields and full observation vectors with tolerance 1e-9 + 1e-7 × abs(reference). Rejection cases include malformed inputs, nonbinary outcomes, one retained class, rank deficiency, complete and quasi separation. Rust tests cover extreme logits and iteration exhaustion. PHP tests cover listwise deletion, warnings, row accounting, strict missingness and invalid values before deletion. Three-way verification also checks a sample with missing rows. These tests do not establish exhaustive separation detection.

PHP=/opt/homebrew/bin/php .context/notebooks-venv/bin/python docs/verifications/compare_logistic.py
PHP=/opt/homebrew/bin/php .context/notebooks-venv/bin/python tests/benchmarks/run.py --model logistic

See the verification record, executed comparison and release benchmark.

Probit and Poisson

tests/fixtures/probit-r.json and poisson-r.json are generated with base R glm, normal Wald inference and tolerance epsilon 1e-16. Regenerate intentionally with Rscript tests/generate-probit-poisson-fixtures.R; this also writes both shared teaching samples. Six fixtures per method cover named simulations, one predictor, multiple predictors, no intercept, intercept only, correlated columns and imbalanced/sparse outcomes. Three-way scripts add two missing rows and compare all fields/full observation vectors at 1e-9 + 1e-7 × abs(reference).

Probit comparisons use expected-information covariance in both reference libraries. Rust tests include underflowing normal log tails, failed convergence and separation. Poisson tests cover integer validation, zero counts, precision of near-zero deviance, failed convergence and selected boundary designs. PHP tests check missing warnings, strict policies, invalid outcomes before deletion, row alignment and convergence metadata. Existing OLS/logistic tests continue to cover shared binding conversion.

php -n -d "extension=$STATS_EXTENSION" tests/probit.php
php -n -d "extension=$STATS_EXTENSION" tests/poisson.php
.context/notebooks-venv/bin/python docs/verifications/compare_probit.py
.context/notebooks-venv/bin/python docs/verifications/compare_poisson.py
.context/notebooks-venv/bin/python tests/benchmarks/run.py --model probit
.context/notebooks-venv/bin/python tests/benchmarks/run.py --model poisson

See the probit comparison, Poisson comparison and their benchmark pages. These cases establish tested behavior, not exhaustive separation/boundary detection or coverage of Linux/PHP-FPM deployment.

Gaussian random-intercept ML

tests/fixtures/lmm-r.json stores six lme4 ML fits: the simulated schools, one predictor, no intercept, intercept only, an additional singleton group and an exact zero-between-group-variance boundary. The generator Rscript tests/generate-lmm-fixtures.R intentionally refreshes both fixtures and docs/data/schools.csv. lme4 is a development dependency, never a runtime requirement. R uses REML=FALSE with bobyqa rhoend=1e-10.

Rust/PHP comparisons cover fixed coefficients, plug-in GLS covariance and Wald inference, population/conditional fitted values, both residual definitions, shrunken group effects, variance components, ICC, marginal likelihood and AIC. Tolerance is 1e-7 + 1e-6 × abs(reference), reflecting independently optimized variance components. A balanced intercept-only case also has independently known ML variance and shrinkage results. Tests reject invalid/unidentifiable samples and positive-residual-variance failures, and verify units/row order, exact boundary behavior, typed group IDs and listwise deletion across x/y/groups.

The three-way comparison uses statsmodels MixedLM ML estimates with the declared GLS covariance convention. Its exact zero-variance case uses a separately labeled NumPy OLS-limit check, since a statsmodels mixed optimizer is not a reliable exact-boundary reference. It also tests a sample with missing values in all three inputs. The stored report records the Python backend, warnings and covariance convention for each case.

php -n -d "extension=$STATS_EXTENSION" tests/lmm.php
.context/notebooks-venv/bin/python docs/verifications/compare_lmm.py
.context/notebooks-venv/bin/python tests/benchmarks/run.py --model lmm

The finite profile search is not an exhaustive global-optimization proof. The original fast call remains one random intercept with Gaussian ML. The expanded general fitter now has separate REML/random-slope/multiple-term tests; finite-sample mixed-model inference adjustments remain open.

Published Gaussian mixed-model data

The published-data comparison checks sleepstudy (180 observations), Orthodont (108), Machines (54) and Hsb82 (7,185) against R lme4, R nlme and Python statsmodels. Both R packages have saved expected outputs in tests/fixtures/lmm-public-r.json, checked by Rust and PHP without runtime R/Python dependencies. The live comparison checks full output vectors, all inference, variance components, likelihood and exact group/sample metadata. CSV snapshots, original studies, export versions, checksums and explicit matrix construction are documented in data provenance.

Published near-zero observation-level differences use a documented 1e-6 absolute outcome-unit tolerance plus 1e-6 relative. Other fields use 1e-7 absolute plus 1e-6 relative; likelihood/AIC use 1e-7 absolute plus 1e-10 relative. A separate native-versus-statsmodels check uses the stricter original uniform tolerance. The notebook includes independent profile-score checks and recorded reference optimizer rounding, rather than hiding disagreements or changing model conventions. Run docs/verifications/compare_lmm.py --public in the pinned Python environment.

Expanded R compatibility

The executed audit verifies 12 weighted/offset fixed-estimator cases and 15 general Gaussian mixed cases. Both native BOBYQA and Nelder–Mead are compared against actual lme4 bobyqa, Nelder_Mead, nlminbwrap and nloptwrap reference fits, with saved controls, convergence codes and warnings. The published sleepstudy random-slope model has ML/REML statsmodels checks too. The coverage ledger documents quantity-specific tolerances, backend differences and unsupported features. New fixture tests need no R; compare_r_compatibility.py adds actual PHP and live statsmodels verification.

Binomial logit with one random intercept

tests/fixtures/logistic-mixed-r.json stores seven datasets at four integration settings (Laplace, 9, 25, 61 nodes), including published cbpp data and difficult rare-event/large-variation cases. Rust checks full output against lme4 and tests normal quadrature moments/tiny tail weights, the ordinary-GLM zero-variance reduction, predictor units, count canonicalization/limits and failure controls. PHP integration checks the actual binding, typed labels, complete row alignment, original-input validation, warnings and strict missingness.

The executed verification notebook adds independent SciPy root finding, matched-approximation likelihood evaluation and QUADPACK integration. It distinguishes R agreement from approximation accuracy and retains lme4’s raw likelihood convention plus the disclosed grouped-binomial normalization correction. The full Python mixed-model reference is SciPy; statsmodels’ Bayesian binomial mixed estimator has a different objective.

See the v0.1 stabilization assessment for the offline scripts/check gate, IDE stubs, structured diagnostics, stress fixtures, provenance manifest, performance preservation and remaining release/deployment gates.