Huh. Looks like Plato was right.
A new paper shows all language models converge on the same "universal geometry" of meaning. Researchers can translate between ANY model's embeddings without seeing the original text.
Implications for philosophy and vector databases alike. arxiv.org/pdf/2505.12540
AI Agents vs. Agentic #AI: A Conceptual Taxonomy, Applications and Challenges (preprint) arxiv.org/abs/2505.10468
BREAKING NEWS: The White House has begun the process of looking for a new secretary of defense, according to a U.S. official who was not authorized to speak publicly.
They show LMs can synthesize their own thoughts for more data-efficient pretraining, bootstrapping their capabilities on limited, task-agnostic data. They call this new paradigm βreasoning to learnβ.
PapersChat β Chat with Research Papers
PapersChat provides an agentic AI interface for querying papers, retrieving insights from ArXiv & PubMed, and structuring responses efficiently.
github.com/AstraBert/Pa...
Show HN: Evolving Agents Framework
https://github.com/matiasmolinas/evolving-agents
https://news.ycombinator.com/item?id=43310963
French Senator Claude Malhuret:
"Washington has become Neroβs court, with an incendiary emperor, submissive courtiers and a jester high on ketamine... We were at war with a dictator, we are now at war with a dictator backed by a traitor."
A few words on DeepSeek new releases. Links are:
- github.com/deepseek-ai/...
- github.com/deepseek-ai/...
- github.com/deepseek-ai/...
and the Ultra-Scale Playbook at huggingface.co/spaces/nanot...
Just read the s1: Simple Test-Time Scaling paper. Super interesting approach to improving reasoning models!
TL;DR:
1. SFT on 1k curated examples w/ reasoning traces.
2. Control response length w/ budget forcing:
"Wait" tokens β longer reasoning/self-correction.
"Final Answer:" β enforce stopping.
Maybe a hot take, but what about the following advice to the next gen:
Don't get an AI degree; the curriculum will be outdated before you graduate. Instead, study math, stats, or physics as your foundation, and stay current with AI through code-focused books, blogs, and papers.
A herd of bison stretching off into the distance on a snowy prairie.
Bison should be allowed to roam free and cattle should be restricted to private land.
All abandoned barbed wire should be removed from public land.
The money today being wasted on public lands grazing should go into building wildlife overpasses and installing wildlife safe guide fencing.
no pun intended but βAttention is all you needβ
Not one VC would ever fund a startup to do the kind of hardcore optimization work that DeepSeek did.
Every VC firm should be asking themselves why.
Havenβt we been doing the same to Google and Facebook for the past 15 years?
Finally finally finally some scaling curves for imitation learning in the large-scale-data regime: arxiv.org/abs/2411.04434
And here's a great reader project who trained a tokenizer from scratch on Nepali: github.com/rasbt/LLMs-f...
Nice and fresh content to understand how Large Language Models work: arxiv.org/abs/2501.09223 #LLM #NLP
This is a wonderfully simple blog on how tensors flow through a transformer model.
Covering:
- Tokenize
- Embed
- Positional Encoding
- Decoder
- Multi-Head Attention
- Add and normalize
- Feed-Forward
- Model Head
- Cross-Attention
Blog:
Free Our Feeds! What is it! @freeourfeeds.com
F.O.F. is an independent group with the goal of running THISπ social network totally outside of Bluesky.
It's not us. It's a fully independent version of the network. All the same users and posts. Running cooperatively with us and others.
If youβre an AI startups, or interviewing w/ one ask:
What are you the best in the world at?
Do you offer a service, formula, or delivery method you invented?
Is there something you do thatβs patentable or a unique user experience?
Have you identified and isolated a market segment?
If not, walk
Happy new year 2025
Very interesting paper by Ananda Theertha Suresh et al.
For categorical/Gaussian distributions, they derive the rate at which a sample is forgotten to be 1/k after k rounds of recursive training (hence π¦π¨πππ₯ ππ¨π₯π₯ππ©π¬π happens more slowly than intuitively expected)
lol wait until they realize Vivek is Indian as well
Releasing a dataset of 40 million Bluesky posts!
Collected using the Firehose API, I hope people do some cool ML with it.
Anonymized with a data removal mechanism and includes text, language predictions, and image data.
#ai #ml #NLP
huggingface.co/datasets/Ara...
only bay area residents have an exclusive right to refer to Sf as the city
A short list of tips for keeping a clean, organized ML codebase for new researchers: eugenevinitsky.com/posts/quick-...
Hey all, I've been a bit quiet the last couple of weeks as I am recovering from an accident & injury.
Unfortunately, I couldnβt write my yearly AI research review this year, but hereβs at least a list of bookmarked papers you might find useful: magazine.sebastianraschka.com/p/llm-resear...
Title card: Alignment Faking in Large Language Models by Greenblatt et al.
New work from my team at Anthropic in collaboration with Redwood Research. I think this is plausibly the most important AGI safety result of the year. Cross-posting the thread below:
it really depends on the type of spice , cumin or coriander pre-sautΓ©, if itβs Garam masala post sautΓ© to preserve the aroma
LLMs might secretly be world models of the internet!
By treating LLMs as simulators that can predict "what would happen if I click this?" the authors built an AI that can navigate websites by imagining outcomes before taking action, performing 33% better than baseline. arxiv.org/pdf/2411.06559
