📕 And if you’re looking for a physical copy, grab yours here: www.nannyml.com/metrics?via=...
11.03.2025 15:43
👍 1
🔁 0
💬 0
📌 0
📕 And if you’re looking for a physical copy, grab yours here: www.nannyml.com/metrics?via=...
What’s the best way to track the progress of my book, The Little Book of ML Metrics?
1️⃣ Visit the book’s repo: github.com/NannyML/The-...
2️⃣ Download the latest digital WIP version.
3️⃣ Start reading while I keep writing.
It gets updated every time I push new changes.
It's happening!
Join us next week to ask Sebastian Raschka anything!
📅 Date: February 11th
⏰ Time: 10:00 AM – 11:00 AM EST
📍 Register: lu.ma/evqa4rct
Performance Estimation methods are a step forward in solving the real problem. Proud to be part of this team!
Big kudos to my colleagues Jakub and Wojtek for their work on this!
I’ll be honest, even at the risk of sounding biased. There are many ML monitoring companies out there, but none of them are solving the real problem. Most monitor the data, not the models.
Super proud to work at a place that values open science.
Four years ago, at NannyML, we invented the first version of Confidence-Based Performance Estimation. Today, a paper about it was published in JAIR.
JAIR: jair.org/index.php/ja...
ArXiv: arxiv.org/abs/2407.08649
Took me over an hour to fully understand the computation behind the Pair Confusion Matrix.
Hopefully, it’ll take you a lot less after reading my explanation in "The Little Book of ML Metrics"
www.nannyml.com/metrics?via=...
You can just do many things.
Yesterday was my first day at culinary school!
Don’t overthink it. Embrace the cringe.
If people don’t think what you do is cringe, then you’re not pushing hard enough.
Every person you admire was once considered cringe by someone.
A Writer, YouTuber, Founder, Musician, you name it. They all got to where they are because they constantly shared their work with the world. Constantly.
We’re deciding what book to read next in the "AI from Scratch" study group.
So far, we have these two:
1. AI Engineering by Chip Huyen
2. Hands-On Generative AI with Transformers and Diffusion Models by Omar Sanseviero and gang
Any other suggestions?
• Work hard
• Keep learning
• Cherish loved ones
• Find people who inspire you
• Be kind & egoless
• Eat healthy, exercise, sleep well
• Read & write
• Practice gratitude & meditate
• Be present
• Enjoy food & nature
• Don’t sweat the small stuff
• Smile =)
First AI from Scratch session of 2025!
A big thanks to @carloscapote.bsky.social and Michael Erasmus for their excellent explanations in today's meeting.
Forgot to share the news, but here it is:
Our NannyML open-source package reached 2,000 GitHub stars! 🌟
Slowly but steadily 💪
Another one from the book.
Log Loss (aka cross-entropy loss)!
---
If you're interested in more metric descriptions like this one, check out the book I'm writing: The Little Book of ML Metrics.
GitHub Repo: github.com/NannyML/The-...
Pre-order the book:https://www.nannyml.com/metrics
Ideally before the end on Q2 2025
Please recommend me your recommender system metric 😂
Here are the ones I have so far:
• MRR: Mean Reciprocal Rank
• ARHR@k: Average Reciprocal Hit-Rank at K
• nDCG@K: Normalized Discounted Cumulative Gain
• Precision@k
• Recall@k
• F1@K
• Average Recall@k
• Average Precision@k
• MAP: Mean Average Precision
Which ranking metrics am I missing?
In the coming weeks, I'll be working on the ranking chapter for "The Little Book of ML Metrics", and I want to make sure I'm not missing any popular ranking/recsys metrics.
😂
Every time you say "garbage in, garbage out" an ML model dies.
6. "Large Language Models: A Deep Dive—Bridging Theory and Practice" (Kamath et al., 2024): amzn.to/3VZO6ct
This one is probably too long and expensive, but I want to get it haha.
Anything else to add?
5. "Fundamentals of Data Engineering: Plan and Build Robust Data Systems" (Reis, Housley, 2022): amzn.to/41TJWq9
Not ML-related, but still relevant.
4. "Pen & Paper Exercises in Machine Learning"(Gutmann, 2022): arxiv.org/pdf/2206.13446
I’m not sure if I’ll go through the whole book, but it looks fun—maybe also for a study group?
3. "Writing for Developers: Blogs That Get Read" (Sarna and Dunlop, 2025): amzn.to/3ZXKmJl
Not ML-related, but still relevant.
2. "Alice’s Adventures in a Differentiable Wonderland: A Primer on Designing Neural Networks (Volume I)"(Scardapane, 2024): amzn.to/3DKo3iU
Looks sweet and short, and I’ve been wanting to read it for a while.
ML Books I'll Be Reading in 2025 📚
1. "AI Engineering: Building Applications with Foundation Models" (Huyen, 2024): amzn.to/4gtQgJo
We’ll probably read it in the study group "AI from Scratch."
7. "Be able to dedicate myself to personal projects and make money doing so."Okay, maybe not a lot of money yet, but I’ve done some.
8. "Be creative and empathetic."
9. "Work at Webflow." At the time, Webflow was my dream company.
10. "Be grateful for every step."
11. "Exercise."
