Kevin Jablonka

Congrats! 🎊

Congrats!

Congratulations to Dr. Hiep Le for her PhD at @ciqus.bsky.social! I am very proud of her 😁

congrats!

Evaluating chemical reasoning capabilities of LLMs with ChemBench As large language models (LLMs) sweep across scientific domains, chemistry confronts a critical question: can they rival—or even surpass—trained chemists in core reasoning tasks?

I just published a Substack post highlighting ChemBench by Mirza and coauthors, which evaluates how well LLMs perform on chemistry tasks. From strengths in structural analysis to key limitations, it’s a timely look at AI’s role in chemical reasoning.

open.substack.com/pub/bravoaba...

Work by Nawaf Alampara and Mara Schilling-Wilhelmi.

Support by @carlzeissstiftung.bsky.social, Intel & Merck, OpenPhilanthropy, @fairmat.bsky.social

Work done at the Friedrich-Schiller University Jena and HIPOLE Jena.

As a first step, we propose "eval cards" github.com/lamalab-org/... which hopefully encourage developers of eval methods to slow down (another thing I hope we do as a field: kjablonka.com/blog/posts/t...) and craft evals that really help move the field forward.

We hope to find ways to make it more transparent what assumptions go into building evaluations and to be more thoughtful when designing measurement instruments.

This article ended up including inspiration from many different fields and is again written in a (probably) unconventional form: But I really hope that some of the ideas in there spark a discussion as the field really needs those discussions.

And add this on top of the fact that most of our measurements in machine learning are actually defined by the act of the measurement itself - which is very unintuitive for a physical scientist.

To me, there is really a need for a "science of evals".

When we build benchmarks or other methods for testing machine learning systems, we make many assumptions - too often hidden - that can drastically change what we measure (see the parallel coordinates plots).

Our team has been spending a lot of time building evaluations for machine learning systems.

We have learned some lessons and wrote them down arxiv.org/abs/2503.10837

🚀ChemBench just leveled up!
We’re thrilled to announce the latest release of ChemBench—now smarter and smoother! Dive into benchmarking any chemistry AI model with our revamped framework, designed for flexibility and ease.
#ChemistryAI #MachineLearning #OpenScience #Innovation

Many of our benchmarks underwent a large revision in the last weeks. We now host HuggingFace spaces for them and the dataset.

The revised article and post below give more details

From text to insight: large language models for chemical data extraction The vast majority of chemical knowledge exists in unstructured natural language, yet structured data is crucial for innovative and systematic materials design. Traditionally, the field has relied on m...

From @kjablonka.com, @mvictoriagil.bsky.social, @pepe-marquez.bsky.social and colleagues.

'From text to insight: large language models for chemical data extraction'

#OpenAccess 🔓

pubs.rsc.org/en/content/a...

what is wrong with the link?

We at @digital-discovery.bsky.social are very happy to announce a new paper type called "Commit". Inspired by version control systems such as git, the idea is that if you have an update on a short and pointed publication, you can send it as a commit. We envision commits to be co-cited with the

I've been thinking about how reasoning models will change AI applied to science. The recent papers from Deepseek/AI2/MoonShotAI are showing that we can exceed humans on reasoning tasks and I've written up some reflections on the consequences

diffuse.one/p/d1-007

Interested in reasoning agents?

t.co/Ax9RtOLDCG

@andrew.diffuse.one

BioToP stands for "Biomolecular Technology of Proteins" and aims in interdisciplinary training of PhD students.

Super excited that we are able to advertise 12(!) fully funded PhD positions in the field of biomolecular technology of protein interactions.
Check out the exciting projects, spread the word and join BioToP.

#biotechnology #compchem
@bokuvienna.bsky.social @fwf-at.bsky.social

biotop.boku.ac.at

More details: www.dropbox.com/scl/fi/mlew5...

It would help us a lot if you could support us in spreading the word about the openings in our team! 🙏🏼

We strongly encourage candidates of all different backgrounds and identities to apply. Each new hire is an opportunity for us to bring in a different perspective, and we are always eager to diversify our team further.

In particular, we are looking for PostDocs to support our efforts in building and testing frontier models.

Join LAMA Made with Fillout, the best way to make forms, surveys and quizzes your audience will answer.

If you are excited about ML for materials science and chemistry, we might have good news for you.

We are still hiring on all levels. Simply connect by submitting your application via forms.fillout.com/t/eoGA7AhnAKus.

😂

Our data extraction tutorial is now online in Chem. Soc. Rev. The notebooks can be run using the #jupyter4nfdi service from #base4nfdi.

📝 Paper: pubs.rsc.org/en/content/a...
💻 JupyterHub: t1p.de/matextract-cpu
📚 Online book: matextract.pub

📽️ intro from @kjablonka.com! 👇

Doing good science is 90% finding a science buddy to constantly talk to about the project.

wow, congrats 🙏🏼

Steering towards safe self-driving laboratories The past decade has witnessed remarkable advancements in autonomous systems, such as automobiles that are evolving from traditional vehicles to ones capable of navigating complex environments without ...

10 minutes ago

I am excited to share a perspective on the much-needed topic of hashtag#safety for hashtag#selfdrivinglaboratories. As the field progresses, understanding the challenges and gaps in building safe setups will be crucial for scaling up this technology!

doi.org/10.26434/che...