Melody Huang

Melody Huang (Yale) | Applied Statistics Workshop Gov 3009 Breadcrumbs

Weds at 12:00 ET: #Yale assistant professor @melodyyhuang.bsky.social presents "Relative Bias Under Imperfect Identification in Observational #CausalInference" at this week's #AppliedStatistics workshop. #politicalscience #statistics
appliedstatsworkshopgov3009.hsites.harvard.edu/event/melody...

But have you seen Love Never Dies?? (It gets so much worse...)

Congrats to BART rider and Oakland legend Alysa Liu on winning a gold medal at the Olympics and making the Bay Area proud!

I keep coming back to @cdsamii.bsky.social essay on the “problem-centered” over “puzzle-centered” research paradigm and I can’t help but feel like so many problems with social science methods boil down to this cyrussamii.com?p=3682

Comic. Conjecture: It’s possible to construct a convincing proof without words, pictures, or content of any kind. Proof: [empty box] [caption] Proofs without words are cool, but we can go further.

Proof Without Content

xkcd.com/3201/

A blog post giving a more thorough take on survey experiments and the credibility revolution: cyrussamii.com?p=4168

To me this is a depressing theme in modern academia.

There is so much work being produced, and so many competing demands on our time, that people rarely seem able to just closely read work and frankly say "yes, I believe this" or "no, I don't."

If we aren't doing this, what _are_ we doing?!

new paper by Sean Westwood:

With current technology, it is impossible to tell whether survey respondents are real or bots. Among other things, makes it easy for bad actors to manipulate outcomes. No good news here for the future of online-based survey research

Excellent report on experiences of @guidoimbens.bsky.social & Mary Wootters co-teaching "Causality, Decision Making, and Data Science" to undergrads at Stanford fall 2024: hdsr.mitpress.mit.edu/pub/uynpjlow... Course material here: stanford-causal-inference-class.github.io

Does AI help humans make better decisions? A statistical evaluation framework for experimental and observational studies | PNAS The use of AI, or more generally data-driven algorithms, has become ubiquitous in today’s society. Yet, in many cases and especially when stakes ar...

Excited to see our paper on evaluating whether AI can help humans make better decisions is now out in @pnas.org!
www.pnas.org/doi/10.1073/...

Going to include this on my slides about partial identification

The Yale Population Studies Workshop is live! Check out our fall lineup here: isps.yale.edu/population-s... Speakers span sociology, economics, medicine, and political science. Both Yale and non-Yale folks are welcome to sign up for the email listserv!

senseweight: Sensitivity Analysis for Weighted Estimators Provides tools to conduct interpretable sensitivity analyses for weighted estimators, introduced in Huang (2024) &lt;<a href="https://doi.org/10.1093%2Fjrsssa%2Fqnae012" target="_top">doi:10.1093/jrsssa/qnae012</a>&gt; and Hartman and Huang (2024) &lt;<a href="https://doi.org/10.1017%2Fpan.2023.12" target="_top">doi:10.1017/pan.2023.12</a>&gt;. The package allows researchers to generate the set of recommended sensitivity summaries to evaluate the sensitivity in their underlying weighting estimators to omitted moderators or confounders. The tools can be flexibly applied in causal inference settings (i.e., in external and internal validity contexts) or survey contexts.

New on CRAN: senseweight (0.0.1). View at https://CRAN.R-project.org/package=senseweight

Could that be @melodyyhuang.bsky.social's new #rstats package for sensitivity analyses? 👀

Lots of different ways to do observational causal inference—IV, proximal inference, etc. What if you could compare those strategies more directly?

New preprint w/ @melodyyhuang.bsky.social tries to do just that. Here's one cool figure—we're able to visualize bias of 3 estimators on the same plot

We show that the act of choosing an identification strategy implicitly expresses a belief about the degree of violations that must be present in alternative identification strategies. (4/4)

To help compare the sensitivities across the different identification strategies, we propose a set of sensitivity tools and an augmented bias contour plot visualizes the relationship between these strategies. (3/n)

We develop bias expressions for IV and proximal inference that show how violations of their respective assumptions are actually *amplified* by any unmeasured confounding in the outcome variable. (2/n)

Relative Bias Under Imperfect Identification in Observational Causal Inference To conduct causal inference in observational settings, researchers must rely on certain identifying assumptions. In practice, these assumptions are unlikely to hold exactly. This paper considers the bias of selection-on-observables, instrumental variables, and proximal inference estimates under violations of their identifying assumptions. We develop bias expressions for IV and proximal inference that show how violations of their respective assumptions are amplified by any unmeasured confounding in the outcome variable. We propose a set of sensitivity tools that quantify the sensitivity of different identification strategies, and an augmented bias contour plot visualizes the relationship between these strategies. We argue that the act of choosing an identification strategy implicitly expresses a belief about the degree of violations that must be present in alternative identification strategies. Even when researchers intend to conduct an IV or proximal analysis, a sensitivity analysis comparing different identification strategies can help to better understand the implications of each set of assumptions. Throughout, we compare the different approaches on a re-analysis of the impact of state surveillance on the incidence of protest in Communist Poland.

New working paper with @corymccartan.com on comparing the relative bias of different identification approaches! We compare the bias of selection-on-observables, instrumental variables, and proximal inference estimates under violations of their identifying assumptions.
arxiv.org/abs/2507.23743

4-panel comic. (1) [Person 1 with ponytail flanked by person with short hair and another person speaking into microphone at podium] PERSON 1: In the early 2010s, researchers found that many major scientific results couldn’t be reproduced. (2) PERSON 1: Over a decade into the replication crisis, we wanted to see if today’s studies have become more robust. (3) PERSON 1: Unfortunately, our replication analysis has found exactly the same problems that those 2010s researchers did. (4) [newspaper with image of speakers from previous panels] Headline: Replication Crisis Solved

Replication Crisis

xkcd.com/3117/

I have a new R package, 'bases,' out on CRAN today!

'bases' provides a number of basis expansions that you can use inside any modeling formula. This means you can fit nonparametric regressions with lm() or glmnet() easily!

Bases includes random Fourier features, approximate BART, and more!

We are pleased to announce an experiment intended to stimulate academic discussion and exchange, centered on papers published in Econometrica 1/5

Screenshot of the linked Quarto website, with input checkboxes to change different conditions for a regression model that predicts economic performance based on US political party, with a reported p-value

I’ve long used FiveThirtyEight’s interactive “Hack Your Way To Scientific Glory” to illustrate the idea of p-hacking when I teach statistics. But ABC/Disney killed the site earlier this month :(

So I made my own with #rstats and Observable and #QuartoPub ! stats.andrewheiss.com/hack-your-way/

*Please repost* @sjgreenwood.bsky.social and I just launched a new personalized feed (*please pin*) that we hope will become a "must use" for #academicsky. The feed shows posts about papers filtered by *your* follower network. It's become my default Bluesky experience bsky.app/profile/pape...

Does drinking this much coffee lower the risk of omitted variable bias?

Congratulations Anton!!!

Congratulations Guilherme!

Key features:
- Closed-form solution: The robust CATE is an interpretable weighted average of site-specific CATE models.
- Flexibility: Allows the use of off-the-shelf single-site CATE estimation methods.
- Privacy-preserving: Avoids sharing individual-level data across sites. (4/n)

We propose a minimax-regret framework for generalizing CATEs (conditional average treatment effects) across multisite data. We propose minimizing the worst-case regret over a class of target populations whose CATE can be represented as convex combinations of site-specific CATEs. (3/n)

Researchers often want to estimate HTEs that are consistent across different populations & contexts. When we have multiple source sites, we run into challenges:
(1) Site-specific models lack external validity.
(2) Pooled models risk bias if site heterogeneity is ignored. (2/n)