ML research · UK personal lines insurance

Your GBM outperforms.
Your GLM is still live.

On the forefront of machine learning and data science research in UK personal lines insurance. Helping teams adopt best practice, best-in-class tooling, and Databricks.

Training course → Browse on GitHub
shap_relativities_demo.py
48 Libraries
4,300+ Tests
54 Articles
12 Course modules
The problem we solve

The missing piece is not technical skill. It is tooling that bridges the two.

Most UK pricing teams have adopted GBMs but are still taking GLM outputs to production. The GBM sits on a server outperforming the production model, but the outputs are not in a form that a rating engine, regulator, or pricing committee can work with. The model never makes it to rates.

Each library here solves one specific problem in the pricing workflow. Actuarial tests are included. Outputs use the formats pricing teams already recognise: factor tables, Lorenz curves, A/E ratios, movement-capped rate changes.

sklearn-compatible where it matters. Documented by people who have sat in the same sign-off meetings you have.

See it in practice

Three lines to a factor table. Five to validated splits.

Real API calls from the libraries. Not wrappers around wrappers. Each one does the specific thing a pricing team needs.

from shap_relativities import SHAPRelativities sr = SHAPRelativities(model, X_train) factors = sr.fit_transform(X_test) # Returns multiplicative factor tables in GLM format # Same structure as exp(beta) from your Emblem model factors.head() # vehicle_age relativity ci_lower ci_upper # 0 1.000 0.982 1.018 # 1 0.912 0.901 0.923 print(f"Reconstruction R² = {sr.reconstruction_r2:.4f}") # Reconstruction R² = 0.9973
Factor tables, confidence intervals, exposure weighting, reconstruction validation. Output goes straight into a pricing committee pack.
from insurance_cv import InsuranceTemporalCV from sklearn.model_selection import cross_val_score cv = InsuranceTemporalCV( n_splits=5, ibnr_buffer_months=6 ) scores = cross_val_score( model, X, y, cv=cv, scoring="poisson_deviance" ) # Walk-forward splits - no future data leaks into training folds # IBNR buffer prevents immature periods contaminating validation print(f"CV deviance: {scores.mean():.4f} ± {scores.std():.4f}")
Walk-forward splits with configurable IBNR buffers. Temporally correct: no future data leaks into training folds. sklearn-compatible API.
from rate_optimiser import RateOptimiser opt = RateOptimiser( current_rates, technical_rates, exposure ) result = opt.optimise( max_movement=0.10, target_lr_improvement=0.03 ) # Efficient frontier as a linear programme # Respects ±10% movement cap per segment print(f"LR improvement: {result.lr_delta:.1%}") # LR improvement: 2.8% (within movement constraints)
Formulates the efficient frontier as a linear programme. Respects movement caps per segment, targets aggregate loss ratio improvement.
Who this is for

Built for people who know the problem from the inside

These libraries assume you understand insurance pricing. They do not explain what a GLM is.

PA
Pricing actuaries moving from Emblem or Radar to Python

You know the techniques. These libraries give you Python equivalents that produce outputs in the same formats you already use: factor tables, A/E ratios, Lorenz curves.

DS
Data scientists joining an insurance pricing team

You have the ML skills but lack the actuarial context. These libraries encode that context: correct cross-validation for IBNR, credibility-weighted factors, fairness tests that map to FCA requirements.

PM
Pricing managers evaluating modern tooling

You need to know what is production-ready and what is a research prototype. Each library here has actuarial tests, a clear scope, and outputs a pricing team lead can explain to a committee.

AR
Academic researchers working on insurance pricing methods

We implement recent literature: Manna et al. (2025) on conformal prediction, BYM2 spatial models, variance-weighted non-conformity scores. Reproducible, documented, testable.

Open-source libraries

The full pricing workflow, covered

Each library solves one well-defined problem. Actuarial tests included. sklearn-compatible where it matters.

📉
Data
insurance-cv ins-synthetic ins-datasets
→
🧠
Model
credibility bayesian-pricing ins-multilevel ins-spatial ins-trend ins-anam ins-dispersion ins-experience
→
🔍
Interpret
shap-relativities ins-interactions ins-causal
→
✓
Validate
ins-validation ins-conformal ins-monitoring
→
⚖
Compliance
ins-fairness ins-causal-policy ins-mrm
→
📈
Deploy
rate-optimiser ins-elasticity ins-deploy
🔍 Model interpretation
shap-relativities Extract multiplicative rating factor tables from CatBoost models using SHAP values. Same output format as exp(β) from a GLM: factor tables, confidence intervals, exposure weighting, reconstruction validation. SHAP CatBoost factor tables
✓ Validation & deployment
insurance-cv Temporally-correct cross-validation for insurance pricing models. Walk-forward splits with configurable IBNR buffers, Poisson and Gamma deviance scorers, sklearn-compatible API. walk-forward sklearn IBNR insurance-conformal Distribution-free prediction intervals for insurance GBMs. Variance-weighted non-conformity scores produce narrower intervals than the naive approach with identical coverage guarantees. conformal prediction GBM insurance-monitoring Three-layer model monitoring for deployed pricing models. Exposure-weighted PSI/CSI, segmented A/E ratios with IBNR adjustment, and a formal Gini z-test to distinguish recalibration from refit signals. PSI/CSI A/E ratios Gini z-test insurance-validation PRA SS1/23 compliant model validation reports. Bootstrap Gini CI, Poisson A/E CI, double-lift charts, renewal cohort test, 9-section HTML/JSON output ready for model governance. PRA SS1/23 model governance HTML report insurance-calibration Balance property testing, auto-calibration curves, Murphy score decomposition. Detect and correct systematic bias in predicted frequencies and severities before deployment. calibration Murphy score reliability insurance-deploy Champion/challenger pricing framework. Shadow mode deployment, SHA-256 deterministic routing, SQLite quote log, bootstrap likelihood ratio comparison, ENBP audit trail for ICOBS 6B.2. champion/challenger shadow mode audit trail
⚙ Techniques
credibility Buhlmann-Straub credibility in Python, with mixed-model equivalence checks. Practical for capping thin segments, stabilising NCD factors, and blending a new model with an incumbent rate. Buhlmann-Straub Polars bayesian-pricing Hierarchical Bayesian models for thin-data pricing segments. Partial pooling across risk groups, with credibility factor output in a format that maps back to traditional actuarial review. hierarchical Bayes partial pooling insurance-interactions Tools for detecting, quantifying, and presenting interaction effects in insurance pricing models: the effects a main-effects-only GLM cannot see. interaction effects GLM diagnostics insurance-causal Causal inference methods for insurance pricing. Separating genuine risk signal from confounded association, relevant wherever rating factors are correlated with distribution channel or policyholder behaviour. causal inference deconfounding insurance-spatial Spatial territory ratemaking using BYM2 models. Geographically smoothed relativities that borrow strength across adjacent areas, particularly useful for postcode-level home and motor models with thin data. BYM2 spatial smoothing postcode insurance-multilevel Two-stage CatBoost + REML random effects for high-cardinality group factors like broker, scheme, or fleet. Credibility-weighted adjustments with ICC diagnostics. mixed effects CatBoost credibility insurance-survival Survival models for insurance retention. Cure rate models, CLV, lapse tables, insurance exposure handling, MLflow wrapper. Wraps lifelines with insurance-specific gaps filled. lapse retention CLV insurance-causal-policy Synthetic difference-in-differences for causal rate change evaluation. Event study, HonestDiD sensitivity, FCA evidence pack. Did your rate change actually cause the movement you see? SDID causal inference rate change insurance-trend Loss cost trend analysis with structural break detection. Identifies regime changes in claims frequency and severity, producing annualised trend factors with confidence intervals for reserving and pricing review. trend analysis structural breaks loss cost insurance-anam Actuarial Neural Additive Model for interpretable deep learning in insurance pricing. Shape functions per rating factor give the transparency of a GLM with the predictive power of a neural network. neural additive model interpretable ML deep learning insurance-glm-cluster Automated GLM factor level clustering for insurance pricing. R2VF algorithm collapses 500 vehicle makes or 350 occupation codes into pricing bands. Ridge ranking for nominals, fused lasso for fusion, unpenalised GLM refit. BIC lambda selection. Monotonicity enforcement. fused lasso categorical banding GLM insurance-experience Individual Bayesian posterior experience rating. Four model tiers: static Buhlmann-Straub, dynamic Poisson-gamma state-space, IS-based surrogate posteriors for non-conjugate models, and deep attention credibility. All four produce a multiplicative credibility factor; the balance property holds at portfolio level. 125 tests. Bayesian credibility experience rating attention
📈 Commercial
rate-optimiser Constrained rate change optimisation for UK personal lines. Formulates the efficient frontier between loss ratio improvement and movement cap constraints as a linear programme. linear programming movement caps insurance-demand Demand and conversion modelling for insurance pricing. Price elasticity curves, own-price and cross-price effects, integration with rate optimisation. price elasticity conversion insurance-uplift Uplift modelling for UK personal lines retention. Per-customer CATE via CausalForestDML, Qini curves, four-customer taxonomy (Persuadable / Sure Thing / Lost Cause / Do Not Disturb), PolicyTree discount targeting, ENBP constraint, Consumer Duty fairness audit. 127 tests. uplift CATE retention experience-rating NCD/bonus-malus systems, experience modification factors, and schedule rating for UK insurance pricing. Principled adjustment of technical rates based on individual risk history. NCD bonus-malus schedule rating insurance-elasticity Causal price elasticity via DML. CausalForestDML and DR-Learner from EconML, ENBP-constrained optimiser, ICOBS 6B.2 audit trail. Segment-level elasticity curves for renewal pricing. price elasticity CausalForestDML ICOBS
📊 Tail risk & distributions
insurance-quantile Actuarial tail risk quantile/expectile regression. QuantileGBM wrapping CatBoost MultiQuantile, per-risk TVaR, large loss loading, ILF tables, exceedance curves, isotonic crossing fix. quantile GBM TVaR tail risk insurance-distributional Distributional GBMs for insurance. TweedieGBM, GammaGBM, ZIPGBM, NegBinomialGBM. Smyth-Jørgensen double GLM for dispersion. Per-risk volatility scoring. Tweedie distributional volatility insurance-synthetic Vine copula synthetic portfolio generation with insurance-specific marginals. Exposure-aware Poisson, constraint enforcement, fidelity reporting with TSTR Gini gap. vine copula synthetic data fidelity insurance-datasets Synthetic UK motor insurance datasets with known data generating process. Useful for model validation benchmarks, library demos, and training exercises. synthetic data UK motor DGP insurance-distributional-glm GAMLSS for Python: model ALL distribution parameters as functions of covariates — mean, dispersion, shape, zero-inflation. Seven families, RS algorithm with backtracking. Pure NumPy/SciPy. GAMLSS distributional GLM NumPy/SciPy insurance-dispersion Double GLM for joint mean-dispersion modelling. Standard GLMs assign a single scalar phi to all observations — fleet brokers and personal lines get identical volatility assumptions. DGLM fits a second regression for phi so dispersion varies by risk segment. Alternating IRLS, REML correction, overdispersion LRT, actuarial factor tables. 79 tests. double GLM dispersion REML insurance-composite Composite severity regression with spliced body/tail distributions, covariate-dependent thresholds, ILF estimation, and TVaR. The body and tail obey different physics; this fits them separately while letting the threshold vary by covariate. 106 tests. spliced distributions ILF TVaR
⚖ Compliance
insurance-fairness Proxy discrimination detection for insurance pricing models. Measures of disparate impact, fairness-accuracy trade-off analysis, FCA Consumer Duty documentation support. FCA Consumer Duty proxy discrimination insurance-fairness-ot Discrimination-free pricing via optimal transport. Lindholm marginalisation, causal path decomposition, Wasserstein barycenter for multi-attribute fairness. Built for FCA EP25/2 compliance and the Equality Act Section 19 proportionate justification test. optimal transport FCA EP25/2 discrimination-free insurance-fairness-diag Proxy discrimination diagnostics for fitted pricing models. D_proxy scalar (LRTW 2026), Owen 2014 Shapley attribution of discrimination to individual rating factors, per-policyholder vulnerability scores. HTML/JSON audit reports for FCA EP25/2 and Consumer Duty evidence. Shapley effects proxy discrimination Consumer Duty insurance-mrm Model risk management framework for insurance pricing. ModelCard, ModelInventory, RiskTierScorer, and GovernanceReport produce PRA SS1/23-aligned documentation that survives a model governance review. model governance PRA SS1/23 risk register
🚗 Telematics
insurance-telematics End-to-end pipeline from raw 1Hz GPS/accelerometer data to GLM-compatible risk scores. HMM driving state classification (cautious/normal/aggressive), Buhlmann-Straub credibility aggregation to driver level, Poisson GLM integration. TripSimulator for synthetic fleet generation. HMM telematics UBI
📊 Sensitivity analysis
insurance-sensitivity Global sensitivity analysis for insurance pricing models. Song 2016 Shapley effects give correct variance attribution under correlated rating factors. Rabitti-Tzougas 2025 CLH subsampling handles large portfolios in minutes. Exposure weighting, Sobol indices, fitted-model interface for GLM and GBM. Shapley effects variance decomposition Consumer Duty
📊 Pricing / Reserving Bridge
insurance-nowcast ML-EM nowcasting for claims reporting delays. Wilsens/Antonio/Claeskens algorithm produces completion factors and IBNR counts conditioned on your actual risk mix — not aggregate triangle factors. Gives frequency GLMs a fully-developed view of the most recent 6–24 months of accident year data. IBNR nowcasting EM algorithm
Training course

Modern Insurance Pricing with Python and Databricks

Twelve modules written for pricing actuaries and analysts at UK personal lines insurers. Every module covers a real pricing problem, not a generic data science tutorial adapted to insurance. You work through real Databricks notebooks, on synthetic data that behaves like the real thing.

Full course + all tools
Free
All 12 modules · all future updates · all Burning Cost tools
Open access — no signup required
Start the course →
  • 01 Databricks for pricing teams
  • 02 GLMs in Python: the bridge from Emblem
  • 03 GBMs for insurance pricing
  • 04 SHAP relativities
  • 05 Conformal prediction intervals
  • 06 Credibility and Bayesian pricing
  • 07 Constrained rate optimisation
  • 08 End-to-end pipeline capstone
  • 09 Demand modelling and price elasticity
  • 10 GBM-to-GLM distillation and interactions
  • 11 Model monitoring and drift detection
  • 12 Spatial territory rating
From the blog

Practitioner articles on insurance pricing

11 Mar 2026
Causal Elasticity Estimation for Renewal Pricing
Standard renewal demand models overestimate price sensitivity for bad risks and underestimate it for good ones — because risk drives both premium and lapse. Double Machine Learning removes this structural confound to produce causal elasticity estimates that hold up under FCA Consumer Duty scrutiny.
Read article →
11 Mar 2026
Survival Models for Insurance Retention
Why standard churn models fail for UK personal lines, and what to do instead. Introducing insurance-survival: covariate-adjusted cure models, survival-adjusted CLV, actuarial lapse tables, and MLflow deployment — the four things lifelines does not do for insurance.
Read article →
11 Mar 2026
Your Retention Campaign Has No Targeting Rule
Every insurer runs discount campaigns at renewal. Most target by propensity to lapse — who will leave? The correct question is who will respond to a discount. Those are not the same people. insurance-uplift estimates per-customer treatment effects on renewal probability, classifies the portfolio into Persuadables, Sure Things, Lost Causes, and Do Not Disturbs, and clips recommendations to ENBP. 127 tests.
Read article →
11 Mar 2026
Your Age-25 Premium Cliff Has No Causal Evidence
UK motor insurers charge under-25s roughly three times the premium of 25-30 drivers. The risk discontinuity at age 25 is assumed, not demonstrated. insurance-rdd brings Regression Discontinuity Design to Python for the first time with insurance-specific defaults: exposure weighting, Poisson outcomes, geographic territory boundaries, and FCA Consumer Duty output.
Read article →
11 Mar 2026
Individual Experience Rating Beyond NCD: From Bühlmann-Straub to Neural Credibility
NCD is a crude proxy for individual risk. insurance-experience implements actuarially correct posterior experience rating at policy level: static Bühlmann-Straub, dynamic Poisson-gamma state-space (Ahn/Jeong/Lu/Wüthrich 2023), surrogate Bayesian posteriors (Calcetero/Badescu/Lin 2024), and deep attention credibility (Wüthrich 2024). All four models produce a multiplicative credibility factor that slots into existing GLM rating engines.
Read article →
11 Mar 2026
Double GLM for Insurance: Every Risk Gets Its Own Dispersion
R has had the dglm package since 1998. Python has had nothing. insurance-dispersion is a Double GLM implementation that gives every policy its own dispersion parameter — enabling risk-adequate margin loading, Solvency II variance at policy level, and Tweedie segments that move independently.
Read article →
11 Mar 2026
Detecting Threshold Gaming in Insurance Portfolios
Policyholders game rating thresholds. Mileage declarations cluster just below 10,000. Claimed ages spike just above 25. Declared sums bunch at £100k, £200k, £300k. insurance-bunching is the first Python implementation of bunching estimators — adapted from public economics (Saez 2010, Kleven 2016) to detect exactly this.
Read article →
11 Mar 2026
The Attention Head That Is Also a Credibility Weight
The CLS token's self-attention weight in the Credibility Transformer is not an analogy for Bühlmann-Straub credibility — it is mathematically identical to it. insurance-credibility-transformer implements base CT and ICL-CT in PyTorch. 1,746 parameters. Beats CAFTT (27K parameters). 85 tests.
Read article →
11 Mar 2026
Distribution-Free Solvency Capital from Conformal Prediction
insurance-conformal v0.2 adds SCRReport for distribution-free 99.5% upper bounds, LocallyWeightedConformal for intervals ~24% narrower than standard split conformal, model-free prediction via Hong (2025), and extended diagnostics for checking conditional coverage by rating segment.
Read article →
All articles →
Start the course →
✕