Measuring the Effects of Population-level Interventions Using Healthcare Claims Data

2026 Symposium of Causal Inference in the Health Sciences
Fribourg

Aurélien Sallin, SWICA and HSG

2026-03-18

Population-level interventions with administrative data: a common but challenging setting

Need for causal analysis…
  • Who: public health authorities, researchers, and insurers
  • Why: accountability, quality improvement, and policy design
… in an imperfect world…
  • No control group: population-level policies affect everyone simultaneously, ruling out randomized experiments and difference-in-differences designs.
  • Routine data: claims data collected for billing/administrative purposes, not purpose-built for research

🎤 This talk

  • How we think about the problem from the perspective of SWICA
  • One empirical illustration (Vitamin D testing and low-value care)

A credible control group is rarely available

Problem: in many real-world settings, population-level policies affect all units simultaneously

National changes in clinical guidelines
Changes in physician compensation
Public health campaigns

Pragmatic solution: interrupted time series (ITS)

  • Idea: exploits the pre-intervention trend to construct a counterfactual

  • Challenge: strong causal assumptions (no concurrent interventions, trend continuity) and correct functional form specification*

Claims data support policy evaluation with no control group

  • Coverage: entire Swiss population enrolled under mandatory health insurance (>130 million invoices/year)
  • Administrative records: byproduct of the billing and reimbursement process (not purpose-built for research).
  • Strengths: large number of units (physicians, patients, consultations) and services billed observed over time.
  • Limitations: longitudinal tracking complicated by switchers; regional fragmentation of insurers; no clinical information.
  • ⚡Key advantage for ITS: cross-sectional variation to predict each unit’s counterfactual trajectory.

ITS used with administrative data presents many advantages, but…

ITS is widely used but implemented with high risk of bias

“The risk of bias for ITS studies was high for 53.3% and very high for 19.2%.”
Hategeka et al. (2020), N = 120, 1990-2020

  • Concurrent changes confound the intervention.
  • Time series techniques misapplied.
  • Trends and time-dependent covariates misspecified.
  • …

Sound implementation guidance is needed.

Here is how we address these challenges

  • User’s guide: recommendations for ITS implementation, identifying assumptions, and modeling choices.
  • Open-access software (itscausal): tailored to wide panel data (many units, few periods), with a rolling-window ML forecasting approach.
  • Approach validation: simulations across different data-generating processes, benchmarking against published research.
  • Ongoing work: 2024 co-payment reform and generic drug substitution in Switzerland (together with ZHAW).

A visualization of a simulation study for itscausal

An example 💉

💉 Two policies targeted the same low-value practice

  • Context: Vitamin D testing rates nearly doubled between 2013 and 2020 in Switzerland.
  • Problem: Clinical guidelines advised against routine testing (low-value care, ~30% of healthcare in high-income countries)
  • Intervention 1 (April 2021): The Smarter Medicine initiative added Vitamin D testing to its list of procedures to avoid.
  • Intervention 2 (July 2022): The Federal Office of Public Health restricted reimbursement, shifting costs to out-of-pocket.
  • Results: …

Testing rates kept rising despite low clinical value

Trend: Vitamin D test prescriptions rose by 20% between 2018 and 2021

Clinical recommendations reduced testing by 6%

  • Intervention: Smarter Medicine recommendation.
  • Results: the clinical recommendation reduced tests by 6% in 12 months.*

Financial incentives reduced testing by 58% within six months

  • Intervention: Federal Office of Public Health restricted coverage: patients pay out-of-pocket for routine tests
  • Results: The number of tests per consultation dropped by 58% in 6 months
  • Savings: Healthcare system savings were 15.65 million CHF in 2022 alone (1.8 CHF per Swiss resident)

We can do a lot with population data and without control group

No control group is not the end of the world

Interrupted time series can estimate causal effects by leveraging the pre-intervention trend to construct a counterfactual, especially when rich panel data are available.

We build a practical solution

itscausal provides a guide for sound ITS implementation in real-world evidence.

🔗 itscausal on GitHub
🌐 www.aureliensallin.ch

💊 Co-payment reform shifted purchasing from brand-name to generic drugs (ongoing)

  • Context: Switzerland among the lowest generic uptake rates in comparable countries.
  • Intervention (January 2024): co-payment for brand-name drugs from 20% to 40% when a generic substitute exists, while generics remain at 10%.
  • Data: drugs in categories with available generic substitutes.
  • Results:
    • +8% share of patients for generics, -40% share of patients for brand-name drugs
    • +12% costs for generics, -47% costs for brand-name drugs

Identifying assumptions for ITS

The following assumptions must hold (see Cerqua et al. (2024)):
  1. There are no hidden forms of treatment leading to different potential outcomes (weak SUTVA).
  2. Additivity: the treatment effect adds on top of what would have happened anyway, independently of the level of the counterfactual. It rules out multiplicative or interactive effects and it assumes the policy doesn’t affect the variance of outcomes, only their level. Analogous to the “additive separability” assumption in difference-in-differences; parallel assumption is stated in levels, not ratios.
  3. No anticipation and no confounding
    • Absence of anticipatory effects of the intervention on the covariates and the potential outcomes
    • Future covariates do not affect current potential outcomes
    • Covariates remain unaffected by the policy in the post-intervention period (post-treatment exogeneity of the covariates)
  4. Dynamic potential outcomes model: the potential outcomes absent the policy (the “counterfactual”) can be predicted using lagged values of the outcome and of the covariates.
  5. Post-intervention non-linear multi-step-ahead model: the counterfactual can be predicted for multiple periods ahead using lagged values of the outcomes until the intervention, conditional expectations of the outcome after the intervention, and the covariates.

References

Bernal, J Lopez, S Soumerai, and Antonio Gasparrini. 2018. “A Methodological Framework for Model Selection in Interrupted Time Series Studies.” Journal of Clinical Epidemiology 103: 82–91.
Brodersen, Kay H, Fabian Gallusser, Jim Koehler, Nicolas Remy, and Steven L Scott. 2015. Inferring Causal Impact Using Bayesian Structural Time-Series Models.
Cerqua, Augusto, Marco Letta, and Fiammetta Menchetti. 2024. “Causal Inference and Policy Evaluation Without a Control Group.” arXiv Preprint arXiv:2312.05858.
Chernozhukov, Victor, Kaspar Wüthrich, and Yinchu Zhu. 2021. “An Exact and Robust Conformal Inference Method for Counterfactual and Synthetic Controls.” Journal of the American Statistical Association 116 (536): 1849–64.
Hategeka, Celestin, Hinda Ruton, Mohammad Karamouzian, Larry D Lynd, and Michael R Law. 2020. “Use of Interrupted Time Series Methods in the Evaluation of Health System Quality Improvement Interventions: A Methodological Systematic Review.” BMJ Global Health 5 (10): e003567.
Lopez Bernal, James, Steven Cummins, and Antonio Gasparrini. 2018. “The Use of Controls in Interrupted Time Series Studies of Public Health Interventions.” International Journal of Epidemiology 47 (6): 2082–93.
Sallin, Aurélien, Daniel Ammann, Caroline Bähler, et al. 2025. “The Impact of Choosing WiselyTM Recommendations and Insurance Coverage Restrictions on the Provision of Low-Value Care: An Interrupted Time Series Analysis of Vitamin d Tests.” BMC Health Services Research 25 (1): 1359.
Turner, Simon L, Amalia Karahalios, Andrew B Forbes, Monica Taljaard, Jeremy M Grimshaw, and Joanne E McKenzie. 2021. “Comparison of Six Statistical Methods for Interrupted Time Series Studies: Empirical Evaluation of 190 Published Series.” BMC Medical Research Methodology 21: 134. https://doi.org/10.1186/s12874-021-01306-w.