Florent Delgrange

If you want to know more about this vision, check out the paper or come talk to me at AAMAS in May!

The framework unifies reinforcement learning, formal verification, and reactive synthesis to deliver world models that can be checked and queried, enabling agents to synthesize verifiable programs, learn new policies quickly, and maintain correctness while adapting to novelty.

This 'Blue Sky Idea' paper lays out a research agenda for foundation world models: persistent and compositional world models designed to support verification and adaptation of learning agents in a single, principled loop.

Glad to share that my paper,
“Foundation World Models for Agents that Learn, Verify, and Adapt Reliably Beyond Static Environments”
has been accepted at AAMAS 2026!

📄 Paper: arxiv.org/abs/2602.23997

📢 Deadline Extended!
The submission deadline for the Adaptive and Learning Agents (ALA) Workshop at #AAMAS2026 (Paphos, Cyprus 🇨🇾) has been extended!
🗓️ Feb 26, 2026
alaworkshop2026.github.io

Looking forward to discussing this at ICLR 🇧🇷!

We study safe policy improvement in general state spaces by combining world models, representation learning, and careful policy updates. We link representation quality and model prediction loss to safe updates, and introduce DeepSPI, an on-policy algorithm with strong empirical performance.

RL relies on representation learning to handle complex observations. Alongside policies & value functions, agents may learn world models to support planning or improve sample efficiency. They are learned jointly and evolve, so it is crucial to ensure improvement with policy updates.

🎉 Excited to share that our paper, “Deep SPI: Safe Policy Improvement via World Models,” has been accepted to ICLR 2026 🇧🇷!

📄 full paper: arxiv.org/abs/2510.12312
🤝 with @raphael.avalos.fr and @willemropke.bsky.social

🧵

Really looking forward to ALA @ AAMAS 2026! Glad to be co-organizing this edition. If you’re working on adaptive & learning agents, I hope to see you in Paphos!

Mind the GAP!

we've had a few works proposing techniques for enabling scaling in deep rl, such as MoEs, tokenization, & sparse training.
ghada sokar and i looked further & found a bit more clarity into *what* enables scaling, leading us to simpler solutions (see GAP in figure)!
1/

PhD Position in Causal Agent-based Modelling of Complex Social Systems Join this exciting interdisciplinary research project at the Centre for Complex Systems Studies and study causal agent-based modelling!

🎓 PhD position available!

Join our interdisciplinary research project on causal agent-based modelling!

🔍 Looking for curious minds with a MSc degree (or near to completing one) in CS/AI/related fields.
📍 Location: Utrecht University, NL
🗓️ Deadline: 16 June 2025
📩 Info: www.uu.nl/en/organisat...

5/ This work is a first step towards improving the reliability of learning agents by unifying RL and reactive synthesis.

I'm very grateful to my co-authors for this great collaboration!
Check my blogpost for more insights!

I'll present the paper in a few weeks at @aamasconf.bsky.social

4/ This approach allows for
- a separation of concerns
- formal guarantees through (PAC) bounds on both the world model quality and policy performance
- reusability and scaling to domains where synthesis was not applicable

3/ Given the map, the learned low-level models/policies, and a formal specification describing what the agent should do or not, we apply reactive synthesis to obtain a high-level planner.

2/ We consider scenarios where a "map" describing the environment's high-level structure can be provided as a graph. Each vertex is a "room," where we apply RL to get low-level policies. In addition, we learn a world model of each room that can be formally verified.

1/ RL enables agents to learn efficient policies in complex domains, but lacks formal guarantees — a challenge in high-stakes scenarios. In contrast, when the environment model is accessible, reactive synthesis offers formal guarantees, but struggles to scale.

Composing Reinforcement Learning Policies, with Formal Guarantees | Florent Delgrange Synthesizing controllers in large domains from verified world models and reinforcement learning policy composition.

Happy to share our new paper (AAMAS 2025)!
We combine reinforcement learning 🤖🧠 & reactive synthesis ⚙️ for learning scalable safe policies in complex tasks with formal guarantees.

📑paper: arxiv.org/abs/2402.13785
✍️blogpost: delgrange.me/post/composi...

A thread🧵⤵️

Still 7 days to submit your work to the ALA workshop at AAMAS! We welcome full papers, work in progress, and 2-page abstracts of recently published journal papers. All the info is available at ala-workshop.github.io.

📌

Thanks!

Hi, I'd be pleased if you could add me too if there's still room :-)

Thanks!

Hey, I’d love to be added!

Empirical Design in Reinforcement Learning Empirical design in reinforcement learning is no small task. Running good experiments requires attention to detail and at times significant computational resources. While compute resources available p...

Another must read for reinforcement learning. Answers many key questions for researchers;
-Do I need multiple training runs?
-How do I report model confidence?
-And a great section on common mistakes to fend off reviewer 2
🧪
#DRL
#reinforcementlearning
#AI
arxiv.org/abs/2304.01315

In addition to the Deep Learning Theory starter pack, I've also put together a starter pack for Reinforcement Learning Theory. Let me know if you'd like to be included or suggest someone to add to the list!

go.bsky.app/LWyGAAu

hey! working on RL and formal verification, do you mind adding me? :-)

If you're an RL researcher or RL adjacent, pipe up to make sure I've added you here!
go.bsky.app/3WPHcHg