Loading approved evidence
Retrieving and verifying the requested analytical release. Result data loads before optional map boundaries and source checks.
Your filters and release stay in the address. Navigation remains available.
Retrieving and verifying the requested analytical release. Result data loads before optional map boundaries and source checks.
Your filters and release stay in the address. Navigation remains available.
SOCIAL FACTORS · PERSISTENCE & ROBUSTNESS · V3
Hold model parameters fixed, change one assumption at a time, and inspect where predictions lose accuracy.
What do these findings mean for insurance? Read the executive interpretation and finding register →
Phase 6 diagnostics complete · Original findings preserved
These outcomes have already been seen. The follow-up was designed after inspecting v2. It explains sensitivity and transfer failures; it is not fresh confirmation, and an improved alternative does not promote the original hypotheses. The original thresholds and unsuccessful results remain visible.
With the 2023 parameters held fixed, deprivation and qualifications together improve later-year deviance by 12.2% beyond collision history. The regional conclusion changes under alternative count-family, deprivation-vintage and regional-intercept assumptions.
In the original authority transfer test, 24 of 33 London authorities lie outside at least one training predictor range. Burglary’s added value remains sensitive to specification. These diagnostics support further model development, with fresh validation still required.
SAME OBSERVATIONS · ONE ASSUMPTION CHANGED
Positive gain means lower Poisson deviance against the same paired comparator. It does not mean fewer collisions or lower insurance claims. The cohort, mileage offset and holdout membership remain unchanged.
| Specification | Temporal gain | Temporal 95% interval | Temporal O/E | Unseen-region gain |
|---|---|---|---|---|
| Original v2 | +10.97% | +5.99% to +15.02% | 1.030 | -4.35% |
| Fixed 2023 parameters | +12.20% | +7.12% to +16.36% | 1.025 | Temporal test only |
| Poisson count family | +7.52% | +1.39% to +12.70% | 1.003 | +6.51% |
| Fixed 2015 deprivation | +5.54% | +1.21% to +11.15% | 1.030 | +3.90% |
| Without regional intercepts | +11.74% | +2.58% to +17.93% | 1.039 | +7.11% |
O/E = observed / expected collisions; 1 is aggregate calibration. Intervals use nine region blocks and are conditional on fitted predictions, without model-refitting or input measurement uncertainty. These multiple diagnostic comparisons are not adjusted significance tests.
The coefficients, scalers, history prior and regional coding are fixed from the 2023 model. Each later year still uses its own two-year-lagged inputs and realised exposure. This tests parameter persistence with updated inputs, not a three-year forecast issued in 2023.
| Assessment year | Gain | Observed / expected | Observed rate / 100k | Predicted rate / 100k |
|---|---|---|---|---|
| 2023 | +5.70% | 1.011 | 0.03664 | 0.03625 |
| 2024 | +12.24% | 1.010 | 0.03465 | 0.03430 |
| 2025 | +16.36% | 1.056 | 0.03408 | 0.03228 |
These are separate sensitivities, not a search across combinations for a winning model. Income and employment are highly correlated, so individual coefficients should not become separate causal explanations.
| Year | Gain | Observed / expected |
|---|---|---|
| 2023 | +1.68% | 0.990 |
| 2024 | +8.34% | 0.988 |
| 2025 | +12.25% | 1.034 |
| Year | Gain | Observed / expected |
|---|---|---|
| 2023 | +1.87% | 1.009 |
| 2024 | +5.15% | 1.018 |
| 2025 | +8.49% | 1.064 |
| Year | Gain | Observed / expected |
|---|---|---|
| 2023 | +10.25% | 1.013 |
| 2024 | +13.23% | 1.026 |
| 2025 | +11.48% | 1.081 |
ORIGINAL V2 REGIONAL TEST
London makes the largest absolute contribution to this comparison: -6.46 percentage points of the combined gain. All regions stay in the primary result.
| Region | Gain contribution | Observed / expected | Mileage outside training ranges | Gain if omitted from scoring |
|---|---|---|---|---|
| East Midlands | -1.25 pp | 1.224 | 31.7% | -3.32% |
| East of England | 2.53 pp | 1.292 | 0.0% | -8.31% |
| London | -6.46 pp | 1.287 | 63.9% | +3.38% |
| North East | -2.51 pp | 0.758 | 0.0% | -1.90% |
| North West | -0.45 pp | 0.936 | 0.0% | -4.24% |
| South East | 0.89 pp | 1.170 | 5.0% | -6.04% |
| South West | 1.24 pp | 1.095 | 0.0% | -5.90% |
| West Midlands | 1.38 pp | 1.055 | 18.5% | -6.13% |
| Yorkshire and The Humber | 0.30 pp | 1.065 | 0.0% | -4.73% |
The final column recalculates the score with one region omitted; it does not refit the models or justify excluding that region. Outside-range flags use training-only continuous-feature ranges, excluding calendar year and regional indicators. No flagged row is removed or clipped.
| Region | Flagged / assessed areas | Out-of-range features |
|---|---|---|
| East Midlands | 3 / 8 | 2011 share under 25 (1); Lagged car/taxi mileage share (1); Credibility-adjusted collision history (1) |
| East of England | 0 / 11 | None |
| London | 24 / 33 | 2011 population density (19); 2011 share under 25 (2); 2011 share aged 65+ (9); Lagged car/taxi mileage share (1); Lagged traffic per road kilometre (2); Credibility-adjusted collision history (15); 2011 no-qualifications share (7) |
| North East | 0 / 10 | None |
| North West | 0 / 22 | None |
| South East | 3 / 18 | 2011 share aged 65+ (1); Income deprivation (1); Employment deprivation (2) |
| South West | 0 / 7 | None |
| West Midlands | 3 / 12 | 2011 population density (2); 2011 no-qualifications share (1) |
| Yorkshire and The Humber | 0 / 12 | None |
A row inside every separate feature range may still be an unfamiliar combination. Range checks do not replace road-context validation.
Five bands are defined from each expanded model’s training predicted rates, weighted by mileage. Assessment outcomes do not set the boundaries. Both models are assessed on the same rows in each band.
| Risk band | Assessment rows | Regions | Billion miles | Observed rate / 100k | Expanded model rate / 100k | Expanded O/E | Comparator O/E |
|---|---|---|---|---|---|---|---|
| 1 · lowest | 71 | 7 | 271.80 | 0.02016 | 0.01722 | 1.171 | 1.201 |
| 2 | 53 | 8 | 115.13 | 0.02396 | 0.02272 | 1.054 | 1.069 |
| 3 | 62 | 7 | 125.07 | 0.02845 | 0.02880 | 0.988 | 0.997 |
| 4 | 51 | 9 | 93.21 | 0.03831 | 0.03661 | 1.046 | 1.060 |
| 5 · highest | 162 | 8 | 113.95 | 0.08671 | 0.08962 | 0.968 | 0.962 |
Rows are area-years, not independent people. Band boundaries differ between training folds and are archived. Sparse force bands and nine region blocks limit precision; these descriptive bands do not have independent-row confidence intervals.
View live source checks, maintenance status and remaining gaps →
| Phase | Delivery | Current position |
|---|---|---|
| 1 | Define tests | Original protocol v1 frozen |
| 2 | Build matched data | Frozen panel, source archives and QA complete |
| 3 | Test deprivation | Exploratory models complete; full v1 specification still blocked |
| 4 | Test crime | Burglary extension complete; broader local crime remains unresolved |
| 5 | Education and combinations | Earlier-Census exploratory comparisons complete |
| 6 | Persistence and geographic validation | These diagnostics complete; fresh confirmation still needed |
| 7 | Publish findings and interpretations | Lens reading, shared finding register and website integration complete; no validated pricing finding |
| 8 | Maintain evidence | Source monitoring and monthly delivery-integrity checks installed; model refresh needs a new version |
The next evidence requirement is stronger validation. Historical joint road class/urban-rural controls, source-vintage reconciliation and fresh outcomes or independent data are still needed. The 2025 results cannot become a new unseen holdout after these diagnostics.
The retrospective complete-history cohort also uses knowledge of later data availability. Pandemic lagged predictors remain present even when pandemic outcome years are excluded from training.
504 new training fits, plus reuse of frozen 2023 parameters. Protocol and source/code hashes were fixed before these diagnostics ran. No human actuarial review is recorded.
Study social-robustness-2026-10-10-v3 · computed 2026-10-10 · Original exploratory results v2 · Original protocol v1.