As a result, models are ranked by latent quality. This positive result demonstrates that benchmarks need not set bad incentives, even though current evaluations do.
Interested in the challenges and future of AI benchmarking? Sign up now to attend our workshop: buff.ly/hS3vSDQ
This result explains why current practice leads to unreliable leaderboards, incentivizing model providers to strategize in opaque ways. However, she proves that under mild conditions, a recently proposed evaluation protocol, "train-before-test", induces a benchmark with a unique Nash equilibrium.
Each competitor has a model of unknown latent quality and can inflate its observed score by allocating resources to benchmark-specific improvements. First, she shows that current benchmarks yield games with no Nash equilibrium between model builders.
Dr. Chen initiates a study of the incentive structure that benchmarks induce. She models benchmarking as a Stackelberg game in which a benchmark leaderboard designer chooses an evaluation protocol and multiple model providers compete simultaneously in a subgame determined by the designer’s choice.
The title of her talk is "Why LLM leaderboard is wrong and how to fix it".
Influential benchmarks incentivize competing model builders to strategically allocate post-training resources towards improvements on the leaderboard, a notorious phenomenon dubbed "benchmaxxing" or training on the test task.
Portrait of Dr. Yatong Chen
Meet our fourth speaker: Dr. Yatong Chen, a group leader in the Social Foundations of Computation Department at the Max Planck Institute for Intelligent Systems. Her work focuses on the social aspects of ML, considering human decision subjects' responses when developing algorithmic systems.
As biological computing moves toward commercialization, major legal and ethical questions arise around fairness, accountability, and transparency. The talk explores how the AI Act and GDPR shape OI development.
Join us to enjoy this interesting talk: buff.ly/hS3vSDQ
Dr. Wernick will present her talk titled "The Law and Ethics of Organoid Intelligence". Organoid Intelligence (OI) uses lab-grown neural tissue for computing, enabling tasks like voice recognition and music generation with high energy efficiency.
Her group is part of the CZS Institute for Artificial Intelligence and Law, and is also an associated member of @ml4science.bsky.social. They explore the dynamic relationship between artificial intelligence (AI) and legal frameworks, focusing on their broader social and societal impacts.
Portrait of Dr. Alina Wernick
The 4th workshop of TWiML features four speakers, and today, we are excited to introduce our third speaker: Dr. Alina Wernick, who leads the Law, AI, and Society Group.
Join us to learn more about privacy and safety in AI Agents: buff.ly/hS3vSDQ
Dr. Abdelnabi introduces a framework grounded in contextual integrity to prevent oversharing, combining data minimization, structured privacy reasoning, reinforcement-learning-based norm learning, and agent communication firewalls, enabling personalized AI without leaking sensitive data.
The title of her talk at the 4th TWiML workshop is "Preserving Privacy in the Age of AI Agents." LLM agents gain access to rich personal data; therefore, contextual privacy becomes critical.
She works at the intersection of AI, security, safety, and sociopolitical aspects, focusing on identifying model risks and failures, designing robust mitigations and controls, and applying AI agents to advance science and societal good.
We are delighted to introduce our next speaker: Dr. Sahar Abdelnabi, who leads the COMPASS research group at the @ellisinsttue.bsky.social, Max Planck Institute for Intelligent Systems, and @tuebingen-ai.bsky.social.
We are excited to hear her talk!
🔗Register here: docs.google.com/forms/d/e/1F...
But how can models be personalized, and what can they reveal about brain function and dysfunction? In this talk, she will present recent discoveries and surprises from her team’s work on developing personalized models of brain activity for psychiatry.
AI-enhanced personalized modeling offers a promising way to study and optimize treatments through simulations of each patient’s brain, enabling individualized therapeutic recommendations.
For example, different brain regions are affected in female and male patients with autism spectrum disorder, which is more commonly diagnosed in boys than girls. This highlights the need for personalized approaches to understanding and treating such conditions.
Over one billion people worldwide live with a mental disorder. Developing widely effective treatments is challenging due to substantial inter-individual variability in symptoms, neural substrates, and causes.
Dr. Nghiem's talk at the workshop will be on "Promises and pitfalls in personalized psychiatry: (mis)adventures in modeling brain (dys)function."
Photo of Dr. Trang-Anh Nghiem.
Get to know our 4th workshop speakers!
Our first speaker is Dr. Trang-Anh Nghiem, group leader of Computational Neuropsychiatry at @hertie-ai.bsky.social. Her research aims to apply computational cognitive neuroscience, neuro-AI, and biophysics to psychiatry.
Outstanding posters will receive prizes 🎁
Looking forward to seeing you there!
@ml4science.bsky.social @cyber-valley.bsky.social @hertie-ai.bsky.social @tuebingen-ai.bsky.social @maxplanckcampus.bsky.social
Expect invited talks, interactive sessions, a poster session, and lots of discussion + networking opportunities.
Have research or ideas to share? Submit a poster! We especially encourage women and individuals from underrepresented groups. Recycle old posters if you like — no need to print new ones 🌱
We’re excited to invite you to the 4th Tübingen Women in Machine Learning Workshop!
🧠 From Benchmarks to Brains: Incentives and Responsibility in Modern AI
📅 March 6, 2026
📍 Maria von Linden Str. 1, Tübingen
🔗 Register: lnkd.in/ddYYhc36
🌟 Everyone is welcome 🌟
👉 Tentative schedule: lnkd.in/ge-YvHJx
@tuebingen-ai.bsky.social @ml4science.bsky.social @maxplanckcampus.bsky.social @hertie-ai.bsky.social
🚀 Exciting news! The 4th TWiML Workshop is coming soon
📅 March 6, 2026
Incredible speakers, insightful discussions, and great social moments ahead.
More details coming soon—save the date!
The #EurIPS Salon des Refusés posters are now online!
If you want your rejected NeurIPS paper featured, dont worry! You can still apply, but posters will be accepted on a rolling basis until the session is filled or Friday the 14th Nov, whichever comes first.
eurips.cc/salon-des-re...
The Tübingen AI Research Building, where the Cluster of Excellence "Machine Learning: New Perspectives for Science" is based: A four-story, beige-colored building with numerous window fronts. In front of it stand four trees with autumn leaves.
We are looking for several postdoctoral researchers to join our Norms & Practices Lab! This is an exciting opportunity for scholars of #philosophy, #law, #culturalanthropology, and (qualitative) #socialscience: uni-tuebingen.de/en/128980#c2... 1/2
🚀 The new call for Principal Investigators as Hector Endowed Fellows at the ELLIS Institute Tübingen is now open!
These positions offer the exciting possibility of co-appointments with the @mpi-is.bsky.social and the @tuebingen-ai.bsky.social .
📌 Apply here: institute-tue.ellis.eu/en/jobs/PI-c...
