🧠🤖 Join us for the launch of the Center for Philosophy of AI (University of Copenhagen)!
📅 Sept 3, 13:00-17:00
📍CPH Conference
Keynotes on philosophy of LLMs by @parismarx.com, Ellie Pavlick, @dcm.social.sunet.se.ap.brid.gy, Tom Sterkenburg & @zhijingjin.bsky.social
Register: cpai.ku.dk
🤔Do Vision and Language Models Share Concepts? 🚀
We present an empirical evaluation and find that language models partially converge towards representations isomorphic to those of vision models. #EMNLP
📃 direct.mit.edu/tacl/article...
🙋🏽♀️🙋🏽♀️
6/🧵 We hope that the connection we provide to epistemology can inform and motivate better evaluations and claims regarding knowledge in LLMs 🚀
5/🧵 We also ask philosophers and computer scientists for their opinion on this matter: can we even say that an LLM knows a fact? We find some intriguing discrepancies between the two groups 🌗
4/🧵Using the definitions, we identify gaps in how current NLP research conceptualizes knowledge compared to epistemology 🔍. We also highlight open questions and new research opportunities to improve knowledge evaluations.
3/🧵Philosophers have thought and discussed about “knowledge” for a long time. So we look to epistemological research for definitions of knowledge 🧐 and interpret them in the context of LLMs 📝
2/🧵 However, inconsistencies may arise with paraphrases or with logically related facts 💥. Would it really know that “Lionel Messi plays for Inter Miami” if it fails to predict where Lionel Messi resides? ⚽️
1/🧵 LLMs seem to know a lot of facts about the world, GPT-4 correctly completes “The capital of Germany is ___” with Berlin, so it must know that, right?
Heading to Miami! To present 👇🌴
Does GPT-4 truly “know” that the Earth is round? 🤔
First, some definitions 💡
“Defining Knowledge: Bridging Epistemology and Large Language Models”
📄 arxiv.org/abs/2410.02499
🗣️ #EMNLP2024
Hey Maria! Thanks for the list, could you add me? 😀
