Why you may predict distribution shifts in generalization, and how
About me
Hi! I’m Ying. I am currently a Wojcicki-Troper Postdoctoral Fellow at Harvard Data Science Initiative, where I have the fortune to be mentored by Professor José Zubizarreta and also work with Professor Marinka Zitnik.
Starting July 2025, I will be an Assistant Professor in the Department of Statistics and Data Science at the Wharton School, University of Pennsylvania. I obtained my PhD in Statistics from Stanford University in 2024, advised by Professors Emmanuel Candès and Dominik Rothenhäusler. Prior to that, I studied Mathematics at Tsinghua University. Here are my CV, Github and Google scholar pages.
I currently help organize the Online Causal Inference Seminar.
Research interests
I work on statistical problems related to two main themes:
- Uncertainty quantification
I develop methods to quantify and control the uncertainty of black-box AI models for their confident deployment in critical domains. My recent work integrates selective inference ideas into conformal prediction, with the goal of enabling trusted decision-making based on uncertainty principles. One example is Conformal Selection using multiple testing ideas to find unlabeled instances with “good” labels, such as active drugs and trustable LLM outputs, which can then be acted upon with confidence. Related projects:- Methodologies in conformal selection [Conformal Selection, weighted CS, optimize CS, post-selection conformal inference]
- Applications in AI-driven drug discovery [slides]
- Methods & applications in selective trust of AI models [Conformal alignment]
- Automatic active scientific discovery with LLM agents [Popper]
- Generalizability and robustness
I am interested in understanding the generalization and robustness of statistical findings across datasets, populations, and contexts. My recent works study the empirical nature of distribution shifts in large-scale replication projects. I also work on methods that address distribution shifts in generalizing treatment effects, learning causal decision rules, and combining datasets. Related projects:- Empirical distribution shifts in causal effect generalization [paper 1, paper 2, blog]
- Robust design and analysis of observational studies [conformal sensitivity analysis, f-sensitivity]
- Data integration and efficient transfer [data fusion for prediction, finite-pop transfer, A/B test efficiency]
- Pessimism for offline learning from imbalanced data [RL, causal policy learning]
These questions lead me to the fields of conformal prediction, causal inference, and multiple testing.
News
- Feb 2025: Imagine LLM agents for scientific discovery—agents that autonomously gather knowledge by creative reasoning and flexible tool use. How to ensure the soundness of what they acquire? We propose Popper, a framework where LLM agents design sequential experiments, collect data, and accumulate statistical evidence to validate a free-form hypothesis with error control!
- Dec 2024: Recent empirical investigations have challenged the sufficiency of covariate shift adjustment for generalization under distribution shift. How to address what remained unexplained? Analyzing two large-scale multi-site replication projects, our new paper suggests a predictive role of covariate shift: it informs the strength of unknown conditional shift, which helps generalization!
- Nov 2024: Excited to share Optimized Conformal Selection for optimizing/selecting conformity scores to pick as many “interesting” instances as possible (e.g., active drugs, trustable LLM outputs) while maintaining FDR control – even without sample splitting!
- Sept 2024: Outputs from black-box foundation models must align with human values before use. For example, can we ensure only human-quality AI-generated medical reports are deferred to doctors? Our paper Conformal Alignment is accepted to NeurIPS 2024!
- Sept 2024: I’m thrilled to receive the 2025 IMS Lawrence D. Brown PhD student award!
- Sept 2024: My paper on optimal variance reduction in online experiments (2021 internship project at LinkedIn) receives the 2024 Jack Youden Prize for the best expository paper in Technometrics! Thank you, ASQ/ASA!
- March 2024: How to quantify the uncertainty for an “interesting” unit picked by a complicated, data-driven process? Check out JOMI, our framework for conformal prediction with selection conditional coverage!
- Sept 2023: I’ll be giving a seminar at Genentech on leveraging Conformal Selection [1, 2] for reliable AI-assisted drug discovery.
- Sept 2023: Scientists often refer to distribution shifts when effects from two studies differ, e.g. in replicability failure. Do they really contribute? See our preprint for a formal diagnosis framework. Play with our live app, or explore our data repository! I gave an invited talk about it in the Causality in Practice Conference.
Beyond academics, I love traveling and photography in my free time. See my photography gallery!
Education
- Ph.D, Statistics, Stanford University, 2019 - 2024.
- B.S. Mathematics, Tsinghua University, 2015 - 2019.
- B.A. Ecomonics (Finance), Tsinghua University, 2015 - 2019.