Allen Tran, Aurélien Bibaut, and Nathan Kallus
Experiments are often much shorter than the decisions they are meant to inform. A platform may test a product change for a few weeks but care about its effect over years. A clinical trial may observe a new treatment briefly even though patients would take it indefinitely.
For a one-time intervention, researchers sometimes bridge this gap with a surrogate: measure an early outcome, then use its known relationship with the later outcome. But a continuing treatment creates a harder problem. Tomorrow’s dose, recommendation, or product experience can have its own direct effect. The long run is not merely the delayed echo of what happened during the experiment.
Our paper treats this as a sequential decision problem. The experiment reveals how treatment changes the next state of the world: a person’s behavior, health, or other relevant condition. Observational data can then help describe how those states evolve over longer horizons. Put together, this begins to look like offline reinforcement learning, except the goal is causal measurement rather than choosing a policy.
This connection gives us a way to estimate the effect of continual treatment from short-term experimental observations. We use doubly robust estimators, which combine a model of outcomes with a model of treatment assignment. If either component is correctly specified, the estimator can still be consistent. We also develop confidence intervals, because a long-range point estimate without a credible account of uncertainty is not much of an answer.
No method gets the future for free. The approach relies on assumptions about how the measured state carries relevant information forward and about what can be learned from the available experimental and observational data. The virtue is that these assumptions address continual exposure directly, instead of quietly treating a long-term treatment like a one-time shock.
The broader idea is simple: a short experiment may not show us the distant outcome, but it can show us the machinery that produces it. If we can model that machinery carefully, we can reason about effects that take longer to unfold than any practical experiment can run.