Measuring the Effects of Population-level Interventions Using Healthcare Claims Data

2026 Symposium of Causal Inference in the Health Sciences
Fribourg

Author

Aurélien Sallin, SWICA and HSG

Published

March 18, 2026

Population-level interventions with administrative data: a common but challenging setting

Need for causal analysis…
  • Who: public health authorities, researchers, and insurers
  • Why: accountability, quality improvement, and policy design
… in an imperfect world…
  • No control group: population-level policies affect everyone simultaneously, ruling out randomized experiments and difference-in-differences designs.
  • Routine data: claims data collected for billing/administrative purposes, not purpose-built for research

🎤 This talk

  • How we think about the problem from the perspective of SWICA
  • One empirical illustration (Vitamin D testing and low-value care)

“This is the problem I face in my work.”

A credible control group is rarely available

Problem: in many real-world settings, population-level policies affect all units simultaneously

National changes in clinical guidelines
Changes in physician compensation
Public health campaigns

Pragmatic solution: interrupted time series (ITS)

  • Idea: exploits the pre-intervention trend to construct a counterfactual

  • Challenge: strong causal assumptions (no concurrent interventions, trend continuity) and correct functional form specification*

Problem: No staggered intervention, no control group, no Diff-in-Diff. No CREDIBLE control group. No untreated units: forecast the counterfactual.

Example: Congenital syphilis (mother-to-child transmission or vertical transmission of syphilis) causes damage to the fetus if the mother’s infection is not detected and properly treated during pregnancy. October of 2016: “Syphilis No!” Project in Brazil.

Gold standard: A randomized experiment is ideal, but rarely feasible when a policy rolls out across an entire population at once.

Second best: Quasi-experimental designs such as difference-in-differences require untreated units as a comparison group, which a universal policy eliminates by construction.

Our approach: Interrupted time series exploits the pre-intervention trend to construct a counterfactual and measures the deviation from it once the policy takes effect.

Claims data support policy evaluation with no control group

  • Coverage: entire Swiss population enrolled under mandatory health insurance (>130 million invoices/year)
  • Administrative records: byproduct of the billing and reimbursement process (not purpose-built for research).
  • Strengths: large number of units (physicians, patients, consultations) and services billed observed over time.
  • Limitations: longitudinal tracking complicated by switchers; regional fragmentation of insurers; no clinical information.
  • ⚡Key advantage for ITS: cross-sectional variation to predict each unit’s counterfactual trajectory.

ITS used with administrative data presents many advantages, but…

ITS is widely used but implemented with high risk of bias

“The risk of bias for ITS studies was high for 53.3% and very high for 19.2%.”
Hategeka et al. (2020), N = 120, 1990-2020

  • Concurrent changes confound the intervention.
  • Time series techniques misapplied.
  • Trends and time-dependent covariates misspecified.
  • …

Sound implementation guidance is needed.

Here is how we address these challenges

  • User’s guide: recommendations for ITS implementation, identifying assumptions, and modeling choices.
  • Open-access software (itscausal): tailored to wide panel data (many units, few periods), with a rolling-window ML forecasting approach.
  • Approach validation: simulations across different data-generating processes, benchmarking against published research.
  • Ongoing work: 2024 co-payment reform and generic drug substitution in Switzerland (together with ZHAW).

A visualization of a simulation study for itscausal

Use machine learners and ensemble methods to train models on pre-intervention data. Forecast onto the post-intervention period. Compare actual outcomes with predicted counterfactuals to estimate causal effects. Aggregate results for the desired population and time period.

  • ML learners (random forest, gradient boosting, neural networks, catboost, lstm) with hyperparameters tuning.
  • Rolling-window approach for post-intervention forecasts with time-variant and invariant predictors.
  • Effect computed as the difference between the observed and the forecasted values.
  • Simulations of different data-generating processes show good performance.

An example 💉

💉 Two policies targeted the same low-value practice

  • Context: Vitamin D testing rates nearly doubled between 2013 and 2020 in Switzerland.
  • Problem: Clinical guidelines advised against routine testing (low-value care, ~30% of healthcare in high-income countries)
  • Intervention 1 (April 2021): The Smarter Medicine initiative added Vitamin D testing to its list of procedures to avoid.
  • Intervention 2 (July 2022): The Federal Office of Public Health restricted reimbursement, shifting costs to out-of-pocket.
  • Results: …

Testing rates kept rising despite low clinical value

Warning: Using `size` aesthetic for lines was deprecated in ggplot2 3.4.0.
ℹ Please use `linewidth` instead.
Warning in geom_text(aes(x = -26, y = 1.8, label = "COVID19 \nLockdown"), : All aesthetics have length 1, but the data has 66 rows.
ℹ Please consider using `annotate()` or provide this layer with data containing a single row.
Warning: Removed 4 rows containing missing values or values outside the scale range
(`geom_line()`).
Warning: Removed 39 rows containing missing values or values outside the scale range
(`geom_line()`).
Warning: Removed 4 rows containing missing values or values outside the scale range
(`geom_line()`).

Trend: Vitamin D test prescriptions rose by 20% between 2018 and 2021

Clinical recommendations reduced testing by 6%

Warning in geom_text(aes(x = -26, y = 1.8, label = "COVID19 \nLockdown"), : All aesthetics have length 1, but the data has 66 rows.
ℹ Please consider using `annotate()` or provide this layer with data containing a single row.
Warning: Removed 4 rows containing missing values or values outside the scale range
(`geom_line()`).
Warning: Removed 39 rows containing missing values or values outside the scale range
(`geom_line()`).
Warning: Removed 4 rows containing missing values or values outside the scale range
(`geom_line()`).

  • Intervention: Smarter Medicine recommendation.
  • Results: the clinical recommendation reduced tests by 6% in 12 months.*

* Estimates adjusted for patient and physician characteristics. Robust to specifications with physician fixed effects. Results available in Sallin et al., 2025.

Financial incentives reduced testing by 58% within six months

Warning in geom_text(aes(x = -26, y = 1.8, label = "COVID19 \nLockdown"), : All aesthetics have length 1, but the data has 66 rows.
ℹ Please consider using `annotate()` or provide this layer with data containing a single row.
Warning: Removed 4 rows containing missing values or values outside the scale range
(`geom_line()`).
Warning: Removed 39 rows containing missing values or values outside the scale range
(`geom_line()`).
Warning: Removed 4 rows containing missing values or values outside the scale range
(`geom_line()`).

  • Intervention: Federal Office of Public Health restricted coverage: patients pay out-of-pocket for routine tests
  • Results: The number of tests per consultation dropped by 58% in 6 months
  • Savings: Healthcare system savings were 15.65 million CHF in 2022 alone (1.8 CHF per Swiss resident)

We can do a lot with population data and without control group

No control group is not the end of the world

Interrupted time series can estimate causal effects by leveraging the pre-intervention trend to construct a counterfactual, especially when rich panel data are available.

We build a practical solution

itscausal provides a guide for sound ITS implementation in real-world evidence.

🔗 itscausal on GitHub
🌐 www.aureliensallin.ch

💊 Co-payment reform shifted purchasing from brand-name to generic drugs (ongoing)

  • Context: Switzerland among the lowest generic uptake rates in comparable countries.
  • Intervention (January 2024): co-payment for brand-name drugs from 20% to 40% when a generic substitute exists, while generics remain at 10%.
  • Data: drugs in categories with available generic substitutes.
  • Results:
    • +8% share of patients for generics, -40% share of patients for brand-name drugs
    • +12% costs for generics, -47% costs for brand-name drugs

Identifying assumptions for ITS

The following assumptions must hold (see Cerqua et al. (2024)):
  1. There are no hidden forms of treatment leading to different potential outcomes (weak SUTVA).
  2. Additivity: the treatment effect adds on top of what would have happened anyway, independently of the level of the counterfactual. It rules out multiplicative or interactive effects and it assumes the policy doesn’t affect the variance of outcomes, only their level. Analogous to the “additive separability” assumption in difference-in-differences; parallel assumption is stated in levels, not ratios.
  3. No anticipation and no confounding
    • Absence of anticipatory effects of the intervention on the covariates and the potential outcomes
    • Future covariates do not affect current potential outcomes
    • Covariates remain unaffected by the policy in the post-intervention period (post-treatment exogeneity of the covariates)
  4. Dynamic potential outcomes model: the potential outcomes absent the policy (the “counterfactual”) can be predicted using lagged values of the outcome and of the covariates.
  5. Post-intervention non-linear multi-step-ahead model: the counterfactual can be predicted for multiple periods ahead using lagged values of the outcomes until the intervention, conditional expectations of the outcome after the intervention, and the covariates.

References

Bernal, J Lopez, S Soumerai, and Antonio Gasparrini. 2018. “A Methodological Framework for Model Selection in Interrupted Time Series Studies.” Journal of Clinical Epidemiology 103: 82–91.
Brodersen, Kay H, Fabian Gallusser, Jim Koehler, Nicolas Remy, and Steven L Scott. 2015. Inferring Causal Impact Using Bayesian Structural Time-Series Models.
Cerqua, Augusto, Marco Letta, and Fiammetta Menchetti. 2024. “Causal Inference and Policy Evaluation Without a Control Group.” arXiv Preprint arXiv:2312.05858.
Chernozhukov, Victor, Kaspar Wüthrich, and Yinchu Zhu. 2021. “An Exact and Robust Conformal Inference Method for Counterfactual and Synthetic Controls.” Journal of the American Statistical Association 116 (536): 1849–64.
Hategeka, Celestin, Hinda Ruton, Mohammad Karamouzian, Larry D Lynd, and Michael R Law. 2020. “Use of Interrupted Time Series Methods in the Evaluation of Health System Quality Improvement Interventions: A Methodological Systematic Review.” BMJ Global Health 5 (10): e003567.
Lopez Bernal, James, Steven Cummins, and Antonio Gasparrini. 2018. “The Use of Controls in Interrupted Time Series Studies of Public Health Interventions.” International Journal of Epidemiology 47 (6): 2082–93.
Sallin, Aurélien, Daniel Ammann, Caroline Bähler, et al. 2025. “The Impact of Choosing WiselyTM Recommendations and Insurance Coverage Restrictions on the Provision of Low-Value Care: An Interrupted Time Series Analysis of Vitamin d Tests.” BMC Health Services Research 25 (1): 1359.
Turner, Simon L, Amalia Karahalios, Andrew B Forbes, Monica Taljaard, Jeremy M Grimshaw, and Joanne E McKenzie. 2021. “Comparison of Six Statistical Methods for Interrupted Time Series Studies: Empirical Evaluation of 190 Published Series.” BMC Medical Research Methodology 21: 134. https://doi.org/10.1186/s12874-021-01306-w.

\[\begin{aligned} {y_{it}} &= {\beta _0} + {\beta _1}{t_{it}} + {\beta _2}{x_t} + {\beta _3}\left( {{t_{it}} \times {x_t}} \right) \\&\quad+ {\beta _4}{Z_{it}} + {\alpha _i} + {\gamma _t} + { \in _{it}} \end{aligned}\]

Linear model with time variable, treatment variables, individual controls and seasonality patterns

● Grey line = Actual vitamin D tests prescribed monthly

● Red line = Statistical model predictions

This is our econometric model for the interrupted time series analysis. The model includes time trends, intervention effects, and controls for both patient and physician characteristics.