|
|
Classical statistical decision theory
evaluates treatment choices based solely on observed
outcomes. However, by ignoring counterfactual outcomes, it cannot
assess the quality of decisions relative to feasible
alternatives. For example, the quality of a physician's decision may
depend not only on patient survival, but also on whether a less
invasive treatment could have produced a similar result. To address
this limitation, we extend standard decision theory to incorporate
counterfactual losses--criteria that evaluate decisions using all
potential outcomes. The central challenge in this generalization is
identification: because only one potential outcome is observed for
each unit, the associated risk under a counterfactual loss is
generally not identifiable. We show that under the assumption of
strong ignorability, a counterfactual risk is identifiable if and
only if the counterfactual loss function is additive in the
potential outcomes. Moreover, we demonstrate that additive
counterfactual losses can yield treatment recommendations that
differ from those based on standard loss functions, provided that
the decision problem involves more than two treatment
options. |
Imai, Kosuke, Zhichao Jiang, D. James
Greiner, Ryan Halen, and Sooahn Shin. (2023). ``Experimental Evaluation of Algorithm-Assisted
Human Decision-Making: Application to Pretrial Public Safety
Assessment.'' (with discussion) Journal of the
Royal Statistical Society, Series A (Statistics in Society),
Vol. 186, No. 2 (April), pp. 167-189. Read before the Royal
Statistical Society. |
Imai, Kosuke and Zhichao
Jiang. (2023). ``Principal
Fairness for Human and Algorithmic Decision-Making.''
Statistical Science, Vol. 38, No. 2 (July),
pp317-328. |
Ben-Michael, Eli, Kosuke Imai, and Zhichao
Jiang. (2024). ``Policy
Learning with Asymmetric Counterfactual Utilities.''
Journal of the American Statistical Association,
Vol. 119, No. 548, pp. 3045-3058. |
Ben-Michael, Eli, D. James Greiner,
Melody Huang, Kosuke Imai, Zhichao Jiang, Sooahn Shin. ``Does AI help humans make better decisions? A
statistical evaluation framework for experimental and observational
studies.'' |
Shin, Sooahn, Zhichao Jiang, and Kosuke
Imai. ``aihuman:
Experimental Evaluation of Algorithm-Assisted Human Decision-Making
.'' available through The Comprehensive R
Archive Network and GitHub.
|