|
|
Algorithmic recommendations and decisions
have become ubiquitous in today's society. Many of these and other
data-driven policies, especially in the realm of public policy, are
based on known, deterministic rules to ensure their transparency and
interpretability. For example, algorithmic pre-trial risk
assessments, which serve as our motivating application, provide
relatively simple, deterministic classification scores and
recommendations to help judges make release decisions. How can we
use the data based on existing deterministic policies to learn new
and better policies? Unfortunately, prior methods for policy
learning are not applicable because they require existing policies
to be stochastic rather than deterministic. We develop a robust
optimization approach that partially identifies the expected utility
of a policy, and then finds an optimal policy by minimizing the
worst-case regret. The resulting policy is conservative but has a
statistical safety guarantee, allowing the policy-maker to limit the
probability of producing a worse outcome than the existing
policy. We extend this approach to common and important settings
where humans make decisions with the aid of algorithmic
recommendations. Lastly, we apply the proposed methodology to a
unique field experiment on pre-trial risk assessment instruments. We
derive new classification and recommendation rules that retain the
transparency and interpretability of the existing instrument while
potentially leading to better overall outcomes at a lower
cost. (Last updated in February 2022) |
Imai, Kosuke, Zhichao Jiang, D. James
Greiner, Ryan Halen, and Sooahn Shin. (2023). ``Experimental Evaluation of Algorithm-Assisted
Human Decision-Making: Application to Pretrial Public Safety
Assessment.'' (with discussion) Journal of the
Royal Statistical Society, Series A (Statistics in Society),
Vol. 186, No. 2 (April), pp. 167-189. Read before the Royal
Statistical Society. |
Zhang, Yi, Eli Ben-Michael, and Kosuke
Imai. ``Safe Policy Learning under
Regression Discontinuity Designs with Multiple Cutoffs..''
|
Jia, Zeyang, Eli Ben-Michael, and Kosuke
Imai. ``Bayesian Safe Policy
Learning with Chance Constrained Optimization: Application to
Military Security Assessment during the Vietnam
War..'' |
Imai, Kosuke and Zhichao Jiang. (2023). ``Principal Fairness for Human and
Algorithmic Decision-Making.'' Statistical
Science, Vol. 38, No. 2 (July), pp317-328. |
Ben-Michael, Eli, Kosuke Imai, and Zhichao
Jiang. ``Policy Learning with
Asymmetric Counterfactual Utilities.'' Journal of the American
Statistical Assciation, Forthcoming. |
Ben-Michael, Eli, D. James Greiner, Melody
Huang, Kosuke Imai, Zhichao Jiang, Sooahn Shin. ``Does AI help humans make better decisions? A
methodological framework for experimental
evaluation.'' |