Will Whitney

I’m so ready for the robot X Games

Likewise, voice mode is a qualitatively different interaction

Oh I love that! Having an always-on voice channel for meta-interactions is a clear part of what I’m envisioning

If you're around NeurIPS and want to chat about this stuff, hit me up.

Under the hood, the model will interpret every click and update the UI. In the limiting case, your whole computer is just a large model: it generates pixels, reads your clicks and taps, sends some API calls, then generates the next frame. Generative UI is a whole different computing paradigm.

Models that generate UI will build tools that help us communicate with them. DALL-E generates sliders for your image that control its outputs. A coding model generates a WYSIWYG editor for the web page it built. Instant feedback and rich interaction.

Right now we're a little stuck thinking about AI as a person, which comes with the baggage of how we interact with people. But large models don't have the same limitations as humans.

The future of AI is models that generate graphical interfaces. Instead of the linear, low-bandwidth metaphor of conversation, models will represent themselves to us as computers: rich visuals, direct manipulation, and instant feedback.

willwhitney.com/computing-in...

Seems like a close relative of the Janus problem, but I was always suspicious that guidance was the root cause of that. I don’t expect guidance to have a preference for left/right orientation in general… I bet big-ish models do fine on this

I shared this on the Bad Twitter, but now that I'm on The Good Twitter...

Just open sourced Brush🖌️, a new, super cross-platform Gaussian Splatting engine! It can train and render on any platform, or even straight in your Browser

github.com/ArthurBrusse...