4/ This perspective is basically a friendly nudge to the field: let’s build benchmarks that matter in the real world, not just on leaderboards.
Huge thanks to the committee for pulling this together!
Read the letter: polarishub.io/guidelines/m...
3/ Few highlights:
— Many benchmarks reward models for finding what we already know
— Drug discovery needs models that generalize, identify new biology, and flag problems early
— Simple baselines still beat fancy models
— We need clearer tasks, cleaner data, and shared frameworks
2/ Multiomics AI is moving fast, but the benchmarks aren’t keeping up with what drug discovery actually needs.
The letter lays out the gap clearly and makes the case for why our community needs better, more realistic ways to measure progress.
1/ Exciting update from the Multiomics Steering Committee on Polaris!
Read the letter: polarishub.io/guidelines/m...
4/ Make sure to checkout our Open Access paper, associated code and thoughts on implementation 👇
📄 Open Access paper: pubs.acs.org/doi/full/10....
💻 Code on GitHub: github.com/polaris-hub/...
💭 Thoughts on implementation: polarishub.io/blog/imagini...
3/ We're especially thrilled that this paper is open access thanks to the JCIM editor's choice program.
We hope for the broad adoption of these guidelines to help close the gap between perceived progress and real-world impact in drug discovery.
2/ The initial preprint was refined with invaluable feedback:
✅ Community: Unique open-feedback period hosted on Github: github.com/polaris-hub/....
✅ Practical Learnings: Lessons learned from applying these guidelines in the ASAP Antiviral challenge.
✅ Peer Review: Formal review process at JCIM.
1/ 🚀 Our updated paper on practically significant ML method comparison is now published in JCIM!🚀
How do we ensure a model that tops a leaderboard also delivers a practical, real-world impact in drug-discovery?
This paper offers a clear framework to address the field's replicability crisis.
(3/3) Ready to explore the findings?
Now’s your chance. Analyze the data, test your ideas, and drop your insights in the Discord.
Join the Polaris community on Discord to dive deeper into the challenge results and share what you discover: discord.com/invite/cS6Nf...
(2/3) 📢 In addition, ASAP is hosting a virtual showcase tomorrow, April 30 at 11AM ET / 5PM BST featuring top performers from the challenge.
Speakers will present their methods, background, and post-hoc analyses now that the test data is live.
Register for the link: docs.google.com/forms/d/e/1F...
(1/3) 🎉 Exciting Updates: Antiviral Unblinded Datasets! 🎉
Following the release of the ASAP Discovery x OpenADMET antiviral challenge leaderboard, the evaluation code is now open-sourced and the unblinded datasets are available on Polaris.
Unblinded Datasets: polarishub.io/datasets?acc...
Stay updated on the Multiomics Steering Committee's work by subscribing to our mailing list: polarishub.io#subscribe-to...
and Rob Moccia (formerly Pfizer), Cuong Q. Nguyen (formerly GSK), Nate Robichaud (Nomic), William J. Godinez (Novartis), Tommaso Biancalani (Genentech), and Kristina Ulicna (Valence Labs). 🧵
Led by experts from academia and industry, the committee includes: Oren Kraus (Recursion), Judith Mueller (Merck), Ali Denton (Valence Labs), Fabian J. Theis (Helmholtz Munich), Paula A. Marin Zapata (Bayer), Anne E. Carpenter (Broad Institute), Ian Barrett (AstraZeneca)... 🧵
Multiomics data holds the potential to unravel biological complexity by providing complementary insights into cellular states of disease and their modulation through novel therapeutics.
We aim to propose clear definitions, identify benchmarking tasks, and encourage alignment on best practices. 🧵
We're thrilled to announce the formation of the new Multiomics Steering Committee!
By "multiomics" we refer to the integration of high-dimensional data modalities at the cellular level, encompassing molecular (DNA, RNA, protein) and phenotypic (imaging-based, multi-cell-line viability) readouts. 🧵
🔔 Stay Updated!
For future challenges & collaborative opportunities, subscribe to our mailing list.
✉️ polarishub.io#subscribe-to...
📢 We value your feedback!
Help us improve future challenges, share your thoughts.
📄 Feedback Form: share-eu1.hsforms.com/23XmDlumfQBu...
📝 Next Steps:
ASAP Discovery & OpenADMET are writing a paper summarizing insights from the challenge.
All participants are invited to contribute. Join us on Discord: discord.gg/cS6Nfj58Gf
For detailed metrics, leaderboards, and further information, see the challenge page: antiviral.polarishub.io
🚀 Antiviral Challenge Results! 🚀
After weeks of competition, and nearly 400 submissions, the antiviral challenge has concluded!
Thank you to all of our participants. Each submission came with a report — check out the full leaderboard and find more details 👇
Thanks for the feedback, @greglandrum.bsky.social! We're always looking for ways to improve the accessibility and value of the platform so we'll take this into consideration as we continue to build. 🙏
📢 Antiviral Challenge Deadline Extended! 🦠
We’ve extended the deadline for the Antiviral Challenge until March 14th at 12:00 PM ET!
💡 To be eligible for the final leaderboard, you’ll need to share a short (~1 page) report about your method.
Submit here: polarishub.io/competitions
We also want to give a huge shoutout to the community for sharing feedback on the challenge! Let’s keep the conversation going.
Join us on Discord: discord.gg/VaCbn82CeD
🚀 Leaderboard Update: Antiviral Challenge! 🚀
The latest rankings are in—check out where you stand: antiviral.polarishub.io
Missed this update? No worries, you still have time to compete! The challenge runs until March 10th at midnight.
Get started today: polarishub.io/competitions
Did you miss the cutoff for the last intermediate leaderboard of the antiviral challenge?
We’re doing one last leaderboard update before the challenge ends on March 10th! Submit something by midnight, Feb 26th, to see how you place.
Submit your results today: polarishub.io/competitions
Thanks to everyone who joined us last week! You can now watch the recording of the webinar: youtu.be/_ykP_tEVrOk
Reminder - this webinar is happening today at 11 AM ET! Join us live in ~40 minutes: us06web.zoom.us/webinar/regi...
Polaris is on a mission to bring innovators and practitioners closer together to develop methods that matter.
We’re excited to welcome Nicola, Michael, and Nils, who will work with the existing SC to craft guidelines around benchmarking for the community.
Learn more: polarishub.io/guidelines
Today, we’re excited to welcome Nicola Richmond (@recursionpharma.bsky.social), Michael Reutlinger (Roche), and Nils Weskamp (@boehringerglobal.bsky.social) to the Small Molecule Steering Committee at Polaris!
Learn more about what the steering committee is up to: polarishub.io/guidelines
