brendan chambers

Why NVIDIA builds their own open models | Nemotron w/ Bryan Catanzaro YouTube video by Interconnects AI

For people who are just learning about Nemotron with the awesome Nemotron 3 Super drop, recommend you watching this interview I did with one of the leads Bryan Catanzaro -- Nemotron as a project is a LONG time coming.
www.youtube.com/watch?v=Y3Vb...

But in the backward pass, the story is much worse. Gradients get compressed via projection onto a D-dimensional subspace, and most of the training signal simply vanishes.

Common Corpus just breaking 1M downloads: it took some time but open data in ai is actually popular.

A little offended Grammarly didn't make a sloppelganger of me

“autoresearch” micro teaching repo from Karpathy

readme edits seem like such a nice dx for open ended hparam tuning, and maybe other kinds of hill climbing too, so much less painful than the old days

github.com/karpathy/aut...

It was all about spying on Americans: www.theatlantic.com/technology/2...

FlashSampling: Fast and Memory-Efficient Exact Sampling

Paper: flashsampling.github.io/FlashSamplin...

We analyzed 250K+ queries & 430K+ clickstream interactions from Asta, our AI-powered research assistant—and today we're releasing the full dataset. How do researchers actually use AI science tools? Here's what we found. 🧵

Permissioned Data Diary 2: Buckets The second in a series of posts building up a solution to permissioned data on atproto. We introduce buckets: a new protocol primitive for creating a shared social context.

new blog post on permissioned data in atproto! this one introduces "buckets", the protocol-level primitive for shared access control. I walk through two approaches that don't quite work and land on something that I think does

let me know your thoughts!

tldr iiuc we are once again enclosing the commons and industrializing craft, dispossessing laborers while apotheosizing capital, and to slow down this doomloop we need to innovate new collectives and public goods

The Geometry of Prompting: Unveiling Distinct Mechanisms of Task Adaptation in Language Models Decoder-only language models have the ability to dynamically switch between various computational tasks based on input prompts. Despite many successful applications of prompting, there is very limited...

This has a very cool result on in-context learned classification tasks, where they disentangle representational quality (how well-separated concept labels are) and readout alignment (how good it is at reading out its own inner labels). Adding demo examples helps through readout, not representations!

Designing around the tight bottleneck on latency and throughput that separates local and cloud compute is such an interesting problem. Significant challenges though

Anti-homeless benches in Pokemon Legends ZA

why is there anti-homeless architecture in pokemon

Data Centers Ditching the Power Grid, Mark Carney's Viral Speech, and Some Joy Here are some trends I'm following

A year ago, data center developers were focused on connecting to the grid. Today roughly 1/3 of all planned capacity is onsite power - and 72% of that planned capacity is fossil gas. Homer City PA's data center project could soon be one of the largest single sources of carbon emissions in the US.

CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data Language identification (LID) is a fundamental step in curating multilingual corpora. However, LID models still perform poorly for many languages, especially on the noisy and heterogeneous web data of...

Announcing our latest paper: CommonLID

In collaboration with @commoncrawl.bsky.social @mlcommons.org @jhu.edu we built a LID benchmark on actual Common Crawl text covering 109 languages. Existing evaluations overestimate how well LangID works on web data.

arxiv.org/abs/2601.18026

warning: earnestpost
thanks Caleb

extremely poor safekeeping of a student’s private data

as a tech worker I think it’s very disturbing to see Google endangering its own users

working on a seven thousand layer model of extended claugenition

Are LLMs Smarter Than Chimpanzees? An Evaluation on Perspective Taking and Knowledge State Estimation Cognitive anthropology suggests that the distinction of human intelligence lies in the ability to infer other individuals' knowledge states and understand their intentions. In comparison, our closest ...

New work by my former PhD student, Boyang Li

His team produced 500 stories of less than 100 words. LLMs were basically chance-level at answering binary questions about the stories

arxiv.org/abs/2601.12410

This is a real banger of a paper. The example of a model being weirdly focused on jasmine (lol) makes me increasingly think that single-point-of-access models don't really consider who their audience is. Jasmine is a super legible cultural marker for people outside, but is so, _so_ generic.

From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence Can we learn more from data than existed in the generating process itself? Can new and useful information be constructed from merely applying deterministic transformations to existing data? Can the le...

This was a colossal multi-year effort driven by an incredible team that gave this everything: Marc Finzi, Shikai Qiu, Yiding Jiang, Pavel Izmailov, Zico Kolter. Much more in the paper! arxiv.org/abs/2601.03220 7/7

Emergent temporal abstractions in autoregressive models enable hierarchical reinforcement learning Large-scale autoregressive models pretrained on next-token prediction and finetuned with reinforcement learning (RL) have achieved unprecedented success on many problem domains. During RL, these model...

Well this is exciting: arxiv.org/abs/2512.20605

the reason I'd follow Cat Hicks into hell is this unswerving humanist conviction that actually

people are going to do the best they can

we can help them do even better

and neither avenue is served by thinking less of people

39C3 - From Silicon to Darude Sand-storm: breaking famous synthesizer DSPs YouTube video by media.ccc.de

i think we are about to experience an explosion of the possibilities in reverse engineering

we’re at a fascinating moment where I am still ~better at programming than Claude at a medium-horizon difficulty task, but Claude has me absolutely beat in terms of cognitive fatigue so we’re able to ship so much more stuff I never would’ve gotten around to before

Great list of models in 2025 👏🏽

arXiv AI/ML Catch-Up Was your New Year's resolution to keep up with arXiv AI/ML preprints? Browse the past week's new uploads in 30 mins.

I uh, made this. It was supposed to be a joke / concept-art thing that scrolls through the torrent of new AI/ML arXiv uploads too fast to read. But I think I iterated too much and made it almost usable.

Everyone’s favorite feed is running on one person’s gaming system. I love how hackable this site is, it makes it much more fun.

If you’re working on a non-fiction research/writing project that isn’t journalism and you don’t have an academic affiliation, how do you find other people who are doing the same thing? Ideally locally (I’m in NY).