Stephanie T. Wang

Demographic cues (eg, names, dialect) are widely used to study how LLM behavior may change depending on user demographics. Such cues are often assumed interchangeable.

🚨 We show they are not: different cues yield different model behavior for the same group and different conclusions on LLM bias. 🧵👇

title: Cheap science, real harm: the cost of replacing human participation with synthetic data author: Abeba Birhane abstract: Driven by the goals of augmenting diversity, increasing speed, reducing cost, the use of synthetic data as a replacement for human participants is gaining traction in AI research and product development. This talk critically examines the claim that synthetic data can “augment diversity,” arguing that this notion is empirically unsubstantiated, conceptually flawed, and epistemically harmful. While speed and cost-efficiency may be achievable, they often come at the expense of rigour, insight, and robust science. Drawing on research from dataset audits, model evaluations, Black feminist scholarship, and complexity science, I argue that replacing human participants with synthetic data risks producing both real-world and epistemic harms at worst and superficial knowledge and cheap science at best

I wrote this brief talk on why “augmenting diversity” with LLMs is empirically unsubstantiable, conceptually flawed, and epistemically harmful and a nice surprise to see the organisers have made it public

synthetic-data-workshop.github.io/papers/13.pdf

Auditing Google's AI Overviews and Featured Snippets: A Case Study on Baby Care and Pregnancy Google Search increasingly surfaces AI-generated content through features like AI Overviews (AIO) and Featured Snippets (FS), which users frequently rely on despite having no control over their presen...

Google AI overviews now reach over 2B users worldwide. But how reliable are they on high stakes topics - for instance, pregnancy and baby care?

We have a new paper - led by Desheng Hu, now accepted at @icwsm.bsky.social - exploring that and finding many issues

Preprint: arxiv.org/abs/2511.12920
🧵👇

How public involvement can improve the science of AI | PNAS As AI systems from decision-making algorithms to generative AI are deployed more widely, computer scientists and social scientists alike are being ...

Can public involvement in AI evaluation improve the science? Or does it compromise quality, speed, cost?

In @pnas.org, Megan Price & I summarize challenges of AI evaluation, review strengths/weaknesses, & suggest how participatory methods can improve the science of AI
www.pnas.org/doi/10.1073/...

A large-scale field experiment on participatory decision-making in China - Nature Human Behaviour Wu et al. show that involving citizens in local decision-making (participatory budgeting) improves civic engagement in a Chinese context.

Sherry Jueyu Wu showed that when people participate in collective decision-making, they are more willing to express that the gov needs improvement. Interesting to think about in the context of participation and accountability on online platforms...
🔗: www.nature.com/articles/s41...

had a lovely time at the News Futures workshop and my first CHI conf with some amazing folks 🌸

Large Language Models in Qualitative Research: Uses, Tensions, and Intentions | Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems

Excited to be presenting "LLMs in Qualitative Research: Uses, Tensions, and Intentions" with @mariannealq.bsky.social at #CHI2025 today!
🆕 paper: dl.acm.org/doi/10.1145/...

I'm at #CHI2025! 🇯🇵 🌸
Presenting our LBW "Traceable Texts and Their Effects".
We studied how phrase-level links from AI summaries to their sources influence the reading of complex texts.
🕥 April 30 at 10:30a & 3:40p 📍North 1F
Interested in text augmentation or improving source transparency? Drop by!

What’s Political on TikTok? A User-Centered Approach Using audience perspectives and LLMs to map out the political landscape on TikTok.

LLMs show a lot of utility for analyzing content at scale. Here's @stephanietwang.bsky.social's write-up for GAIN on how she and collaborators used LLMs to examine political content on TikTok: generative-ai-newsroom.com/whats-politi...

Deeply evil

This @acm-cscw.bsky.social paper confirms the findings of the recent Nature paper on chrono feeds. Algorithmic feeds lead to more centrist, trustworthy content, but have little impact on user behavior. An independent audit of platforms w/o access to internal data!

dl.acm.org/doi/10.1145/...

Last week at CSCW, Catherine Han presented our work on journalists' unmet needs for protecting against harassment online. While the work targeted Twitter/X, it surfaces several nuances in users' needs that span future platforms as well (e.g., not wanting to filter out threats or visibly block users)

spent a day in San José trying many delicious Costa Rican fruits at Mercado Borbón 😋

stopping by every coffee farm on the road is not helping my bean addiction ☕️

On the Use of Proxies in Political Ad Targeting | Proceedings of the ACM on Human-Computer Interaction Detailed targeting of advertisements has long been one of the core offerings of online platforms. Unfortunately, malicious advertisers have frequently abused such targeting features, with results that...

Pretty blown away by this paper from Piotr Sapiezynski and team at Northeastern and Princeton, presented at #CSCW2024: dl.acm.org/doi/10.1145/...
For lots of good legal and moral reasons, Facebook doesn't allow advertisers to explicitly target ads to "White Republicans" or "Black Democrats"..

hello world; participating in cscw 2024 + elections aftermath has finally convinced me to make a jump