benjamin

I know a lot of people are working on making ModernBERT-based embedding models, but in the meantime, if you’d like to play around with it (no better way to learn than practice), it’s plug&play with Sentence Transformers www.sbert.net and we have examples on the repo

Hey! As Jeremy replied, this is fully expected, encoder-models aren’t expected to produce well-calibrated semantically similar scores out of the box, because it’s very far from the training task for the base model!

However, they fine tune really well into embedding models that are good at this 1/2

I'll get straight to the point.

We trained 2 new models. Like BERT, but modern. ModernBERT.

Not some hypey GenAI thing, but a proper workhorse model, for retrieval, classification, etc. Real practical stuff.

It's much faster, more accurate, longer context, and more useful. 🧵

White-on-black text saying "In fact, [MASK]-large’s processing speed is closer to that of [MASK]-base than it is to [MASK]-large's.", with the [MASK] drawn in purple toi draw attention

I wonder if some kind of model could fill this in...

There was one time my flight from Geneva got cancelled and I got a replacement one from Lyon. Still one of my most surreal experiences.

Won't be at NeurIPS but I'll be at ICLR in April, in case you're planning on being there 😄

Please do go on about the coffee. Is it a make-you-an-espresso-as-required kind of deal or a big pot? Perhaps a lovingly made 1L chemex?

Thank you @bsky.app team for correcting the mistake. Glad to be back!

I can understand this yeah. I’m generally open to discussion but I’ve seen enough unsavoury behaviour & DMs in the past couple days to want to dial it down a teensy bit at the moment sadly.

Jokes aside, it does make me kinda sad. ML Bluesky has a lot of the vibes of early twitter and interesting discussions, but seeing so many of the death threats posters unbanned while someone was banned for *posting a link to a dataset* is a really bad sign :/

GitHub - McGill-NLP/llm2vec: Code for 'LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders' Code for 'LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders' - McGill-NLP/llm2vec

LLM2Vec is also a nice approach for this -- only difference is you'd FT for classification rather than retrieval at the end github.com/McGill-NLP/l...

It’s only hate if it comes from
the Champagne region of X, otherwise it’s just sparkling outrage (I think?)

people on this platform will take your words out of context, twist, not mention your correction, because they just want to hate on what you work on, and insult you comfortably.

I'll keep posting here about my work but will not be interacting with anyone who wants to bash on my company.

i exclusively consent to my tweets being used for training neural networks. if you are not a neural network, stop reading this immediately

(ChromaDB is good too, but IMO it's targeting a different/less AI tinkery audience)

(they do not employ me, nor pay me in any way, I'm just out there doing unpaid advertising)

heartily recommend lancedb for local stuff where you don't want to fuss with things too much -- mostly sane default, has reranking and bm25 support so you can do two-step or hybrid search whenever needed, and the disk ANN is plenty for most people.

Note: you can still criticise the way the original dataset was built. Nothing's black and white. I understand why people are upset.
None of this implies there isn't something seriously wrong with sending death threats to someone because they *curated an open dataset from an open protocol*.

This might sound obvious, but bullying and threatening people doing perfectly legal things because you morally don't agree with them is wrong.

People stifling any serious discussion by doing this, albeit for another set of morals, is actually the exact reason that made a lot of people migrate here.

Some days I really like this place, and then there are others in which there's a level of puritanical fervour that permeates a lot of public discourse that I find off-putting. Some of the over the top hateful responses wouldn't be out of place in the Hellsite.

We should make sure that only really big companies can afford to pay really big copyright holders to access the data needed to do stuff with AI, and keep everyone else out.

Wouldn’t that be just super?

Data gathering on an open platform via an open protocol is only ethical if you're not told about it, silly.

It’s been absolutely horrible to watch this. Pure “it’s fine to insult, harass and threaten people as long as you are doing it for the right reason” energy.

At least blocklists help, I guess blocking toxicity on sight is the only way.

I'm disheartened by how toxic and violent some responses were here.

There was a mistake, a quick follow up to mitigate and an apology. I worked with Daniel for years and is one of the persons most preoccupied with ethical implications of AI. Some replies are Reddit-toxic level. We need empathy.

fast.ai—Making neural nets uncool again – fast.ai

I'm a TA for the new fast.ai course which starts in less than 30 minutes, and which sold out in <48 hours. It's so cool to see it all coming together

OLMo 2 is out 🥳 7B and 13B trained on 5T tokens, and meticulousy instruction tuned using Tulu 3 recipe.

Simply the best fully open models yet.

Really proud of the work & the amazing team at
@ai2.bsky.social

The fou du metro into 7€ happy hour drink pipeline does take its toll over time

Small French towns are really the only way to stay sane in spite of Paris to be fair

you guys need a cute a mascot, then we can start the posting.

> yet another very useful finding from the kraken, it seems DPO is stronger than we thought