|
|
Researchers are increasingly turning to
machine learning (ML) algorithms to investigate causal heterogeneity
in randomized experiments. Despite their promise, ML algorithms may
fail to accurately ascertain heterogeneous treatment effects under
practical settings with many covariates and small sample size. In
addition, the quantification of estimation uncertainty remains a
challenge. We develop a general approach to statistical inference for
heterogeneous treatment effects discovered by a generic ML
algorithm. We apply the Neyman's repeated sampling framework to a
common setting, in which researchers use an ML algorithm to estimate
the conditional average treatment effect and then divide the sample
into several groups based on the magnitude of the estimated
effects. We show how to estimate the average treatment effect within
each of these groups, and construct a valid confidence interval. In
addition, we develop nonparametric tests of treatment effect
homogeneity across groups, and rank-consistency of within-group
average treatment effects. The validity of our methodology does not
rely on the properties of ML algorithms because it is solely based on
the randomization of treatment assignment and random sampling of
units. Finally, we generalize our methodology to the cross-fitting
procedure by accounting for the additional uncertainty induced by the
random splitting of data. The open-source
software package is available for implementing the
proposed methodology. |
Imai, Kosuke and Michael Lingzhi
Li. (2023). ``Experimental
Evaluation of Individualized Treatment Rules.''
Journal of the American Statistical Association,
Vol. 118, No. 541, pp. 242-256. |
Li, Michael Lingzhi and Kosuke
Imai. ``Statistical Performance
Guarantee for Subgroup Identification with Generic Machine
Learning.'' |
Jia, Zeyang, Kosuke Imai, and Michael
Lingzhi Li. ``The Cram Method for
Efficient Simultaneous Learning and
Evaluation.'' |
Li, Michael Lingzhi and Kosuke Imai. ``evalITR:
Evaluating Individualized Treatment Rules.'' available
through The Comprehensive R
Archive Network and GitHub |