|
|
Across a wide array of disciplines, many researchers use machine
learning (ML) algorithms to identify a subgroup of individuals who
are likely to benefit from a treatment the most (``exceptional
responders'') or those who are harmed by it. A common approach to
this subgroup identification problem consists of two steps. First,
researchers estimate the conditional average treatment effect (CATE)
using an ML algorithm. Next, they use the estimated CATE to select
those individuals who are predicted to be most affected by the
treatment, either positively or negatively. Unfortunately, CATE
estimates are often biased and noisy. In addition, utilizing the
same data to both identify a subgroup and estimate its group average
treatment effect results in a multiple testing problem. To address
these challenges, we develop uniform confidence bands for estimation
of the group average treatment effect sorted by generic ML algorithm
(GATES). Using these uniform confidence bands, researchers can
identify, with a statistical guarantee, a subgroup whose GATES
exceeds a certain effect size, regardless of how this effect size is
chosen. The validity of the proposed methodology depends solely on
randomization of treatment and random sampling of
units. Importantly, our method does not require modeling assumptions
and avoids a computationally intensive resampling procedure. A
simulation study shows that the proposed uniform confidence bands
are reasonably informative and have an appropriate empirical
coverage even when the sample size is as small as 100. We analyze a
clinical trial of late-stage prostate cancer and find a relatively
large proportion of exceptional responders.
The open-source
software package is available for implementing the
proposed methodology. |
Imai, Kosuke and Michael Lingzhi
Li. (2023). ``Experimental
Evaluation of Individualized Treatment Rules.''
Journal of the American Statistical Association,
Vol. 118, No. 541, pp. 242-256. |
Li, Michael Lingzhi and Kosuke
Imai. (2024). ``Neyman Meets Causal
Machine Learning: Experimental Evaluation of Individualized
Treatment Rules.'' Journal of Causal
Inference, Vol 12, No. 1, pp. 1-20. Special Issue on Neyman
(1923) and its influences on causal inference. |
Imai, Kosuke and Michael Lingzhi
Li. (2025). ``Statistical Inference
for Heterogeneous Treatment Effects Discovered by Generic Machine
Learning in Randomized Experiments.'' Journal of
Business & Economic Statistics, Vol. 43, No. 1,
pp. 256-268. |
Jia, Zeyang, Kosuke Imai, and Michael
Lingzhi Li. ``Cramming Contextual
Bandits for On-policy Statistical
Evaluation.'' |
Li, Michael Lingzhi and Kosuke Imai. ``evalITR:
Evaluating Individualized Treatment Rules.'' available
through The Comprehensive R
Archive Network and GitHub |