LLMs are increasingly used as agents for decisions under uncertainty, e.g. medical diagnosis. But do they act like rational agents with coherent beliefs and preferences? Much of the difficulty is telling whether a model's response to.a prompt ("What is the probability of X?") is a "real" belief.
Paper here: arxiv.org/abs/2602.06286. Led by my excellent PhD student Khurram Yamin
In applications based on medical diagnosis, the answer is...sometimes! In some settings, we can prove that no rational agent could hold beliefs expressed by the model. But in others, particularly for stronger models, outputs are close to consistent with rational belief
We give a framework to test whether the model's stated belief functions *as if it were* a rational agent's subjective probability by comparing with its decisions. We give empirically checkable conditions that don't require any assumptions about the model's "utility function".
You might think that models don't have coherent beliefs at all. Or, you might think that they don't report truthfully in response to any given prompt. How could we possibly tell?
LLMs are increasingly used as agents for decisions under uncertainty, e.g. medical diagnosis. But do they act like rational agents with coherent beliefs and preferences? Much of the difficulty is telling whether a model's response to.a prompt ("What is the probability of X?") is a "real" belief.
Totally agree! I think the fundamental distinction is more between people using AI in their own work vs AI being in a decision-making role that everyone is subject to
As UKRI explores using LLMs to review grants, it's a good time to revisit Bryan Wilder's excellent blog post.
There are a lot of naive reasons to oppose AI review ("you'll never automate human intuition!"). But there are also good reasons, including the *load-bearing role of human disagreement.*
Come talk to me and Angela at NeurIPS on Friday! We argue that "AI for social impact" needs to get more rigorous about evaluating deployments of AI, but also that there are many other forms of impact that get overlooked right now
Based on joint work led by @yewonbyun.bsky.social, with @donskerclass.bsky.social. See our NeurIPS paper, arxiv.org/abs/2508.06635, for more!
I gave talks at MIT and Harvard this week about "Science with synthetic data". How can generative models help us learn about the actual world (e.g., social systems) in a principled way? Lots of interesting conversations -- more convinced than ever that there's nuanced issues to navigate here.
I’m recruiting students this upcoming cycle at UIUC! I’m excited about Qs on societal impact of AI, especially human-AI collaboration, multi-agent interactions, incentives in data sharing, and AI policy/regulation (all from both a theoretical and applied lens). Apply through CS & select my name!
We're in the process of selecting the location for next year's ACM EAAMO conference! If you're interested in bringing the EAAMO community to your institution, please check out the open call here and get in touch. conference.eaamo.org/call_for_loc...
How can synthetic data from LLMs be used, e.g. for social science, in a principled way? Check out Emily's thread on our NeurIPS paper! Generating paired real-synthetic samples and using both in a method-of-moments framework enables valid inference that benefits when synthetic data is informative.
Are you a researcher using computational methods to understand cities?
@mfranchi.bsky.social @jennahgosciak.bsky.social and I organize an EAAMO Bridges working group on Urban Data Science and we are looking for new members!
Fill the interest form on our page: urban-data-science-eaamo.github.io
Screenshot of paper abstract, with text: "A core ethos of the Economics and Computation (EconCS) community is that people have complex private preferences and information of which the central planner is unaware, but which an appropriately designed mechanism can uncover to improve collective decisionmaking. This ethos underlies the community’s largest deployed success stories, from stable matching systems to participatory budgeting. I ask: is this choice and information aggregation “worth it”? In particular, I discuss how such systems induce heterogeneous participation: those already relatively advantaged are, empirically, more able to pay time costs and navigate administrative burdens imposed by the mechanisms. I draw on three case studies, including my own work – complex democratic mechanisms, resident crowdsourcing, and school matching. I end with lessons for practice and research, challenging the community to help reduce participation heterogeneity and design and deploy mechanisms that meet a “best of both worlds” north star: use preferences and information from those who choose to participate, but provide a “sufficient” quality of service to those who do not."
New piece, out in the Sigecom Exchanges! It's my first solo-author piece, and the closest thing I've written to being my "manifesto." #econsky #ecsky
arxiv.org/abs/2507.03600
Submit an abstract to present a poster at EAAMO, deadline July 25! EAAMO is one of my favorite conferences, and a great place for anyone working on ML/algorithms/optimization in social settings. The conference is in Pittsburgh this November.
conference.eaamo.org/cfp/call_for...
ACM EAAMO, which is coming to Pitt this Fall, has two events for students: a doctoral consortium and a poster session, both of which are due July 25th
- poster session conference.eaamo.org/cfp/call_for...
- doctoral consortium
conference.eaamo.org/cfp/call_for...
My takeaway is that algorithm designers should think more broadly about the goals for algorithms in policy settings. It's tempting to just train ML models to maximize predictive performance, but services might be improved a lot with even modest alterations for other goals.
Using historical data from human services, we then look at how severe learning-targeting tradeoffs really are. It turns out, not that bad! We get most of the possible targeting performance while giving up only a little bit of learning compared to the ideal RCT.
We introduce a framework for designing allocation policies that optimally trade off between targeting high-need people and learning a treatment effect as accurately as possible. We give efficient algorithms and finite-sample guarantees using a duality-based characterization of the optimal policy.
A big factor is that randomizing conflicts with the targeting goal: running a RCT means that people with high predicted risk won't get prioritized for treatment. We wanted to know how sharp the tradeoff really is: does learning treatment effects require giving up on targeting entirely?
These days, public services are often targeted with predictive algorithms. Targeting helps prioritize people who might be most in need. But, we don't typically have good causal evidence about whether the program we're targeting actually improves outcomes. Why not run RCTs?
Excited to share that our paper "Learning treatment effects while treating those in need" received the exemplary paper award for AI at EC 2025! This paper grew out collaborations with Allegheny County's human services department and my co-author Pim Welle (at ACDHS).
arxiv.org/abs/2407.07596
CMU is hosting a workshop on Human-AI Complementarity for Decision Making this September! Abstract submissions due July 15, travel will be covered for accepted presenters.
www.cmu.edu/ai-sdm/resea...
Excited to have this work out at ICML this year! Do LLMs make correlated errors? Yes, and those by the same company, and also more accurate/later generations are more correlated -- increasing algorithmic monoculture
arxiv.org/abs/2506.07962
Still thinking about this post. The broader point, which should resonate way beyond the specific issue of "peer review," is that human disagreement is not friction and waste. It's a load-bearing, functional part of social and intellectual systems.
I don't know one way or another, but it's at least a clearer capability to benchmark. And, if a LLM *could* summarize well enough on existing papers, arxiv using it for lower-bar moderation decisions wouldn't distort paper-writing in the future.
Thoughtful take on one aspect of the increasing problem of LLMs leading to “centralization” of thought/writing/etc.
The arXiv summarization use case sounds a lot more sensible. Clear value judgment specified up-front, not outsourced to the LLM: papers should have easily summarized claims and evidence. Resulting incentives for authors seem ok (making sure LLMs can at least parse the paper probably isn't bad).
