A common trend across recent research in using reinforcement learning to train reasoning models is that the clipping operation within a trust region (core to PPO, adopted by GRPO) is squashing rare tokens that are key to clever behaviors like verification or backtracking.
Tried creating an AI chatbot on Instagram
A high-level summary diagram taken from the slides linked below. It shows the interplay of two main components: a probabilistic model and decision maker or planner.
Probabilistic predictions of an underfitting polynomial classifier on a noisy XOR task and the corresponding under-confident calibration curve.
Probabilistic predictions of an overfitting polynomial classifier and the resulting overconfident calibration curve on the same noisy XOR problem.
Simulation study to show the relative lack of stability of hyperparameter tuning when using hard metrics such as Accuracy or soft yet not probabilistic metrics such as ROC AUC compared to a strictly proper scoring rule such as the log-loss.
I recently shared some of my reflections on how to use probabilistic classifiers for optimal decision-making under uncertainty at @pydataparis.bsky.social 2024.
Here is the recording of the presentation:
www.youtube.com/watch?v=-gYn...
Bringing structure and recommended practices to Machine Learning projects can be challenging. Even experienced data scientists struggle with it.
That's why we built skore – your companion when modeling with scikit-learn. Check it out and let us know what you think!
github.com/probabl-ai/s...
Couldn’t get much opinions here but to conclude this post, got a lot of insights from threads for anyone looking.
www.threads.net/@a.i.slayer/...
Which setup would you choose for running large language models (LLMs) locally ?
Option 1:
• Apple M4 Max
• 14-core CPU, 32-core GPU
• 36 GB unified memory
• 1 TB SSD
Option 2:
• Apple M4 Pro
• 14-core CPU, 20-core GPU
• 48 GB unified memory
• 1 TB SSD
Everything you always wanted to ask about entropy but didn't know whom by John Baez.
arxiv.org/abs/2409.09232
@bsky.app The translate takes us to a tab and does the conversion. Is there an update which makes it English in place?
Love the Starter Pack by @bsky.app .. brilliant idea! Quickly finds all your X & Threads follows in one go!
We're always updating the pydata & scipy project starter pack:
go.bsky.app/6HkrMcp
Hello @scikit-learn.bsky.social , @networkx.bsky.social , @scipyconf.bsky.social
One of my fav projects: LeanRL, a simple RL library that provides recipes for fast RL training using torch.compile and cudagraphs.
Using these, we got >6x speed-ups compared to the original CleanRL implementations.
github.com/pytorch-labs...
www.anthropic.com/research/sta...
This is an excellent attempt (blog & paper) at bringing more statistical rigor to evaluation of ML models (this is specifically focused on LLM evals).
I feel like we need to have similar clear standards for many types of predictive models in biology. 1/
👋
Just put together a list of papers to highlight 4 interesting things about transformers & LLMs.
Including a discussion on why the original transformer architecture figure is wrong, and a related approach published in 1991!
https://magazine.sebastianraschka.com/p/why-the-original-transformer-figure
The Llama 3.2 1B and 3B models are my favorite LLMs -- small but very capable.
If you want to understand how the architectures look like under the hood, I implemented them from scratch (one of the best ways to learn): github.com/rasbt/LLMs-f...
How do I get myself added here?
