In our newest work (led by the amazing
@sunnytqin.bsky.social , w/ @emalach.bsky.social, Samy Jelassi), we investigate a core question for LLMs: "๐ก๐ ๐๐๐๐๐ก๐๐๐๐ ๐๐ ๐๐๐ก ๐ก๐ ๐๐๐๐๐ก๐๐๐๐" in two prototypical logic-heavy puzzles: CountDown and Sudoku.
Will be presenting this work at #NeurIPS2024, today 11am, poster #2311. Come visit us!
Heading to NeurIPS tomorrow โ๏ธ
Will be presenting a few papers during the week. Ping me if you want to chat!
I defended my PhD dissertation back in May. I didn't have time to share it widely then (newborn baby), but I think some of you might enjoy it, especially the opening chapters: benjaminedelman.com/assets/disse...
Just put together a starter pack for Deep Learning Theory. Let me know if you'd like to be included or suggest someone to add to the list!
go.bsky.app/2qnppia
How does test loss change as we change the training data? And how does this interact with scaling laws?
We propose a methodology to approach these questions by showing that we can predict the performance across datasets and losses with simple shifted power law fits.
