Hey @araffin.bsky.social sorry, I missed your question. "Coming soon" was a but optimistic, I got caught up in other projected which I did not expect... We will release the XQC code in the next few weeks! We won't do a seperate CrossQ+WN code release though.
🎉 Really excited, our paper "XQC: Well-conditioned Optimization Accelerates Deep Reinforcement Learning" has been accepted at #ICLR2026.
If you are interested in reinforcement learning, sample-efficiency, compute-efficiency go check it out. See you in Rio!
I'm super excited to have been named an #NVIDIA Graduate Fellowship Finalist! 💚
Huge thanks to my supervisor @jan-peters.bsky.social and all my collaborators.
Can't wait to join the NVIDIA Seattle Robotics Lab for my internship next summer! 🤖
blogs.nvidia.com/blog/graduat...
Had a really great time presenting our #NeurIPS paper at the poster session today. Thanks to everyone who stopped by.
If you are interested in sample-efficient #RL, check out our work:
Scaling Off-Policy Reinforcement Learning with Batch and Weight Normalization
Read the full preprint here:
👉 arxiv.org/pdf/2509.25174
Code coming soon.
We’d love feedback & discussion! 💬
Key takeaway:
Well-conditioned optimization > raw scale.
XQC proves principled architecture choices can outperform larger, more complex ones
📊 Results across 70 tasks (55 proprioception + 15 vision-based):
⚡️ Matches/outperforms SimbaV2, BRO, BRC, MRQ, and DRQ-V2
🌿~4.5× fewer parameters and 1/10 FLOP/s of SimbaV2
💪Especially strong on the hardest tasks: HumanoidBench, DMC Hard & DMC Humanoids from pixels
This leads to XQC, a streamlined extension of Soft Actor-Critic with
✅ only 4 hidden layers
✅ BN after each linear layer
✅ WN projection
✅ CE critic loss
Simplicity + principled design = efficiency ⚡️
🔑 Insight: A simple synergy—BatchNorm + WeightNorm + Cross-Entropy loss—makes critics dramatically more well-conditioned.
➡️Result: Stable effective learning rates and smoother optimization.
Instead of "bigger is better," we ask:
Can better conditioning beat scaling?
By analyzing the Hessian eigenspectrum of critic networks, we uncover how different architectural choices shape optimization landscapes.
🚀 New preprint! Introducing XQC— a simple, well-conditioned actor-critic that achieves SOTA sample efficiency in #RL
✅ ~4.5× fewer parameters than SimbaV2
✅ Scales to vision-based RL
👉 arxiv.org/pdf/2509.25174
Thanks to Florian Vogt @joemwatson.bsky.social @jan-peters.bsky.social
Thanks to my co-authors Florian Vogt, @joemwatson.bsky.social @jan-peters.bsky.social
@hessianai.bsky.social @ias-tudarmstadt.bsky.social @dfki.bsky.social @cs-tudarmstadt.bsky.social
#RL #ML #AI
If you're working on RL stability, plasticity, or sample efficiency, this might be relevant for you.
We'd love to hear your thoughts and feedback!
Come talk to us at RLDM in June in Dublin (rldm.org)
📚 TL;DR: We combine BN + WN in CrossQ for stable high-UTD training and SOTA performance on challenging RL benchmarks. No need for network resets, no critic ensembles, no other tricks... Simple regularization, big gains.
Paper: t.co/Z6QrMxZaPY
⚖️ Simpler ≠ Weaker: Compared to SOTA baselines like BRO our method:
✅ Needs 90% fewer parameters (~600K vs. 5M)
✅ Avoids parameter resets
✅ Scales stably with compute.
We also compare strongly to the concurrent SIMBA algorithm.
No tricks—just principled normalization. ✨
🔬 The Result: CrossQ + WN scales reliably with increasing UTD—no more resets, no critic ensembles, no other tricks.
We match or outperform SOTA on 25 continuous control tasks from DeepMind Control Suite & MyoSuite, including dog 🐕 and humanoid🧍♂️tasks across UTDS.
➡️ With growing weight norm, the effective learning rate decreases, and learning slows down/stops.
💡Solution: After each gradient update, we rescale parameters to the unit sphere, preserving plasticity and keeping the effective learning rate stable.
🧠Key Idea: BN improves sample efficiency, but fails to reliably scale with complex tasks & high UTDs due to growing weight norms.
However, BN regularized networks are scale invariant w.r.t. their weights; yet, the gradient scales inversely proportional (Van Laarhoven 2017)
🔍 Background: Off-policy RL methods like CrossQ (Bhatt* & Palenicek* et al. 2024) are sample-efficient but struggle to scale to high update-to-data (UTD) ratios.
We identify why scaling CrossQ fails—and fix it with a surprisingly effective tweak: Weight Normalization (WN). 🏋️
🚀 New preprint "Scaling Off-Policy Reinforcement Learning with Batch and Weight Normalization"🤖
We propose CrossQ+WN, a simple yet powerful off-policy RL for more sample-efficiency and scalability to higher update-to-data ratios. 🧵 t.co/Z6QrMxZaPY
#RL @ias-tudarmstadt.bsky.social
Check out our latest work, where we train an omnidirectional locomotion policy directly on a real quadruped robot in just a few minutes based on our CrossQ RL algorithm 🚀
Shoutout @nicobohlinger.bsky.social, Jonathan Kinzel.
@ias-tudarmstadt.bsky.social @hessianai.bsky.social
