FBK - NLP Research Group

🚀 Big news from #CLiCit2025!
Our PhD student Davide Testa presented MAIA 🎞️🧠🇮🇹 — the first Italian benchmark to test Vision-Language Models on multimodal reasoning & robustness.
📄 You can check the Paper in the CLiC-it pre-processing: clic2025.unica.it/wp-content/u...

#AI #NLProc #VLMs

🎉Congrats to our PhD student Sofia Brenna for presenting her poster at #SigDIAL2025 in Avignon 🇫🇷 about "Investigating Proactivity in Task-Oriented Dialogues", recently published in Dialogue & Discours journal.

Paper here: journals.uic.edu/ojs/index.ph...

#NLProc

All-in-one: Understanding and Generation in Multimodal Reasoning with the MAIA Benchmark We introduce MAIA (Multimodal AI Assessment), a native-Italian benchmark designed for fine-grained investigation of the reasoning abilities of visual language models on videos. MAIA differs from other...

🎉 Thrilled to share that our paper “All-in-one: Understanding and Generation in Multimodal Reasoning with the MAIA Benchmark” has been accepted at the EMNLP 2025 conference! 🔥

📄 Preprint here: arxiv.org/abs/2502.16989

See you in Suzhou next November!!! 🇨🇳🚀
#EMNLP2025 #NLP #Multimodality

🚀 Can Vision-Language Models plan effectively?

We introduce ViPlan, a benchmark comparing:
🔹 VLM-as-planner
🔹 VLM-as-grounder

🏠 Home robotics
🧱 Visual Blocksworld

Spoiler alert:❌ Visual-Reasoning

📍 Pre-print here: arxiv.org/abs/2505.13180

#VLM #AI #NLProc

Evalita Llm Leaderboard - a Hugging Face Space by evalitahf Duplicate this leaderboard to initialize your own!

🚀 Exciting News! The Evalita-LLM Leaderboard is now live on Hugging Face! Explore the performance of over 40 Large Language Models on native Italian tasks. Dive in here: huggingface.co/spaces/evali...

@FondazioneBrunoKessler
@igenius
@diunito.bsky.social

#NLProc #LLM #AI #Italian #Benchmarking

Don't miss out ‼️ Join us at Wired Health 2025 #WH25💭 Our Head Unit, Bernardo Magnini will take the stage to discuss "Artificial Intelligence and Clinical Data: The Future of Emergency Medicine." 🔥
Full program here: lnkd.in/eKi44biT
@wired.com

#AI #healthcare #Innovation

Collab with Cimec (UniTrento)- @raffagbernardi.bsky.social , ItalianNLP_Lab - @alessiomiaschi.bsky.social and UniPisa (Alessandro Lenci, Lucia Passaro and Alessandro Bondielli)

All-in-one: Understanding and Generation in Multimodal Reasoning with the MAIA Benchmark We introduce MAIA (Multimodal AI Assessment), a native-Italian benchmark designed for fine-grained investigation of the reasoning abilities of visual language models on videos. MAIA differs from other...

Welcome to MAIA! 🚀 Our new benchmark for evaluating multimodal reasoning in Vision-LMs on videos fully in Italian! 🇮🇹 MAIA tests understanding & generation with fine-grained reasoning categories and a brand-new evaluation metric! 🔎🔥Discover MAIA here: arxiv.org/abs/2502.16989
#NLProc #evaluation #AI

Thank you for supporting this project! 💪🏻

Evalita-LLM: Benchmarking Large Language Models on Italian We describe Evalita-LLM, a new benchmark designed to evaluate Large Language Models (LLMs) on Italian tasks. The distinguishing and innovative features of Evalita-LLM are the following: (i) all tasks ...

🚀 **Exciting News!** 🎉 Evalita-LLM is here! 🇮🇹 A new benchmark for evaluating LLMs—offering native Italian tasks, generative challenges, and fair multi-prompt evaluations. Now also available in lm-evaluation harness by @eleutherai.bsky.social !
ArXiv: arxiv.org/abs/2502.02289
#NLProc #LLM #Evaluation

Our group leader took the stage at the FBK plenary session to showcase our research interests, ongoing projects, challenges and future plans. An exciting moment to share our vision and push the boundaries of NLP even further!
Here’s a glimpse of the event! 📸✨
#NLProc #AI #Research #FBK #Innovation

A big congratulations to Sofia Lugli, student of our Carlo Strapparava, for receiving a special mention for her thesis at
CLiC-it conference 2 weeks ago! 🎉👏
We are incredibly proud of her achievement. Can’t wait to see what she accomplishes next. Well done Sofia!🌟
#NLP #AI @ailc-nlp.bsky.social

🚀 #CALAMITA has officially kicked off - and we’re on board!
Fondazione Bruno Kessler proudly participated in this 1st #evaluation #campaign for #LLMs with our Andrea Zaninello together with @fbk-mt.bsky.social group!🤖
Here some pictures of our calamitici!🧲
See u next year 4 other challenges! 🔥🔍

Data-LLM-Tutorial You Are what You Eat Processing Data for Training and Evaluating LLMs Giovanni Bonetta and Bernardo Magnini Fondazione Bruno Kessler, Trento, Italy {gbonetta|magnini}@fbk.eu Tutorial at CLiC-it 2024, ...

Exciting news! 🎉 If you’re curious about the opening #tutorial at CLiC-it 2024 conference made by our group on processing #data for #training and #evaluating #LLMs , here’s your chance! 📊 Explore the slides and get inspired!!! 🚀🔍✨
docs.google.com/presentation...
@ailc-nlp.bsky.social

too sweet darling!!! ❤️

Here are our group leader Bernardo Magnini and postdoc
Giovanni Bonetta kicking off CLiC-it conference in #Pisa 2 weeks ago as invited speakers with an insightful #tutorial on #data: how to collect and use it for various purposes in #AI and #NLP !💡🔍
Thanks to @ailc-nlp.bsky.social for inviting us!✨

#NLP Research group in action! Our group leader Bernardo Magnini, alongside @tizaino.bsky.social, Sofia Brenna, and Giovanni Bonetta, presenting their #paper "Are you a Good Assistant? Assessing LLM Trustability in Task-oriented Dialogues" at CLiC-it '24 #Pisa 🔍✨
@ailc-nlp.bsky.social
#NLProc #AI

What a conference! 🎉 CLiC-it in #Pisa was truly special! Returning to where it all began a decade ago! ❤️
Our group was honored to participate, share ideas and connect with such an inspiring community.
Here are some highlights from the conference!!!📸✨
#CLiCit2024 #NLP #AI @ailc-nlp.bsky.social

🎄✨ Happy Holidays from our Research Center! ✨🎄

Our director @ferruccioresta shares a warm holiday message with all @FBK_research: gratitude for a year of outstanding achievements and best wishes for a joyful festive season and a 2025 filled with groundbreaking discoveries!