Today at #EuroScipy2025, @glemaitre58.bsky.social and I presented a tutorial on pitfalls of machine learning for imbalanced classification problems.
We discussed what (not) to do when fitting a classifier and obtaining degenerate precision or recall values.
probabl-ai.github.io/calibration-...
A small update on the retrospective and future priorities of the open source team at @probabl.bsky.social for the next 6 months or so.
Sometimes you think you are right by doing everything "by the book." But sometimes the book is just a tiny part of the full story. Keep digging and writing a new chapter with more insights is actually fun...
New podcast episode! This one is about imbalanced-learn and how the maintainer looks back with some lessons learned.
If you are dealing with imbalanced classification use-cases, like fraud, you'll want to listen in on this one!
youtu.be/npSkuNcm-Og
OK it is an interesting feedback. We could support older versions. We saw that up-to-now, we don't have any code that we are eager to drop quickly. I understand about the runtime dependencies and on our side, the idea is only depending on scikit-learn. But agreed that it is one more dependency.
We are working on a small package to ease developer life: github.com/glemaitre/sk.... The idea is that recurrent work could be centralized in a single package. Once we have a minimal version, we will do a first release to support scikit-learn 1.2 to 1.6
A high-level summary diagram taken from the slides linked below. It shows the interplay of two main components: a probabilistic model and decision maker or planner.
Probabilistic predictions of an underfitting polynomial classifier on a noisy XOR task and the corresponding under-confident calibration curve.
Probabilistic predictions of an overfitting polynomial classifier and the resulting overconfident calibration curve on the same noisy XOR problem.
Simulation study to show the relative lack of stability of hyperparameter tuning when using hard metrics such as Accuracy or soft yet not probabilistic metrics such as ROC AUC compared to a strictly proper scoring rule such as the log-loss.
I recently shared some of my reflections on how to use probabilistic classifiers for optimal decision-making under uncertainty at @pydataparis.bsky.social 2024.
Here is the recording of the presentation:
www.youtube.com/watch?v=-gYn...
3rd-party library maintainers might find it cumbersome to handle the transition to the new estimator tags while keeping backward compatibility with older scikit-learn versions. We will devise a way to smooth out the transition before releasing 1.6.0 final:
github.com/scikit-learn...
Please help us test the first release candidate for scikit-learn 1.6: pip install scikit-learn==1.6.0rc1
Changelog: scikit-learn.org/1.6/whats_ne...
In particular, if you maintain a project with a dependency on
scikit-learn, please let us know about any regression.
With Artefact, we are delighted to invite data leaders to an exclusive Paris masterclass: ✨Aligning Probabilistic Classification with Business Decisions using @scikit-learn.bsky.social ✨ 🚨Limited seats available! Secure your spot now 👉🏻 lu.ma/fopoglzo #MachineLearning #Advanced #AI #Masterclass
