Georg Heiler

Metaxy + Dagster-Slurm for Efficient Multimodal Pipelines | Georg Heiler How to combine Metaxy, Dagster-Slurm, Docling, and Ray to run incremental multimodal pipelines on sovereign AI infrastructure.

See here in action how EU sovereign HPC AI + ray allow to easily scale document processing with docling georgheiler.com/2026/02/22/m...

Metaxy + Dagster-Slurm for Efficient Multimodal Pipelines | Georg Heiler How to combine Metaxy, Dagster-Slurm, Docling, and Ray to run incremental multimodal pipelines on sovereign AI infrastructure.

Multimodal data handling is different! Especially with regards to complexity and cost. Daniel Gafni and I built Metaxy (docs.metaxy.io) to simplify Efficient Multimodal Pipelines

Read how Magenta (👋 @geoheil.com and @milicevica23.bsky.social ) uses Dagster+ to feel a bit of that joy: dagster.io/customers/ho...

Introduction - Metaxy A high level introduction to Metaxy.

And docs.metaxy.io/main/ also integrating with I.e. lance for again a different kind of versioning

Branching and Shallow Cloning in Lance: Towards a "Git for AI Data" A deep dive into how table formats handle version management for ML/AI experimentation, and how Lance unifies branching, tagging, and shallow clone on top of …

Don’t forget lance lancedb.com/blog/branchi... for multimodal fit for data

great implementation for www.stacking.dev

GitHub - cesarferreira/stax: The fastest stacked-branch workflow for Git. Interactive TUI, smart PRs, safe undo. Written in Rust. The fastest stacked-branch workflow for Git. Interactive TUI, smart PRs, safe undo. Written in Rust. - cesarferreira/stax

github.com/cesarferreir... #rust #stacking is awesome

Hannes Werthner "The Role of Computer Science in the Age of AI (or Digital Humanism?)" YouTube video by Digital Humanism

www.youtube.com/watch?v=DGA0...

An European sovereign GPU cloud does not come out of nowhere maybe this project can support making HPC systems more accessible. The recently started projects will take a long time to complete. I hope github.com/ascii-supply... will help.

Modern Architecture 101 for New Engineers & Forgetful Experts - Jerry Nixon - NDC Copenhagen 2025 YouTube video by NDC Conferences

#great talk www.youtube.com/watch?v=WRg1... on #architecture for #engineers

Scaling data pipelines @ Magenta Telekom - OSA Con 2025 Presented by Georg Heiler at OSA Con 2025. Magenta Telekom ingests many terabytes of new data every day, and every downstream consumer wants it immediately. The real bottleneck turned out not to be…

The OSACon recordings are available now www.youtube.com/watch?v=31LH...

PyTogether - Real-Time Collaborative Python IDE Collaborative Python IDE for students, educators, and teams.

#python #together pytogether.org nice

Reconstructing History with XTDB (Jeremy Taylor + James Henderson) YouTube video by CMU Database Group

interesting new #timeseries #database #xtdb xtdb.com see the great #cmu video for details www.youtube.com/watch?v=zzqD...

GitHub - mxschmitt/action-tmate: Debug your GitHub Actions via SSH by using tmate to get access to the runner system itself. Debug your GitHub Actions via SSH by using tmate to get access to the runner system itself. - mxschmitt/action-tmate

github.com/mxschmitt/ac... #tmux #action-tmate - really neat #debugging

DEF CON 33 - Exploiting Shadow Data from AI Models and Embeddings - Patrick Walsh This talk explores the hidden risks in apps leveraging modern AI systems—especially those using large language models (LLMs) and retrieval-augmented generation (RAG) workflows. We demonstrate how…

A great video about LLMs and the data they can provide to the world - even though perhaps they should not | www.youtube.com/watch?v=O7BI... - DEF CON 33 - Exploiting Shadow Data from AI Models and Embeddings - Patrick Walsh

Introducing Apache Fory™ Rust: A Versatile Serialization Framework for the Modern Age | Apache Fory™ TL;DR: Apache Fory Rust is a blazingly-fast, cross-language serialization framework that delivers ultra-fast serialization performance while automatically handling circular references, trait objects, ...

#rust #fory #serialization fory.apache.org/blog/fory_ru...

The real AI win isn't superhuman agents, it's scaled mediocrity.
Doing less with less at massive scale unlocks tasks that were once uneconomical.
The magic is in aggregate value, not perfect outputs. Empower teams with practical AI tools. 
🔗 https://dlthub.com/blog/the-real-ai-win-scaled-mediocrity

GitHub - l-mds/dsc-dach-tutorial-dagster: Introduction to using and scaling dagster Introduction to using and scaling dagster. Contribute to l-mds/dsc-dach-tutorial-dagster development by creating an account on GitHub.

#dsc-dach #data it was a. pleasure to share an introductory workshop about spark and data pipelines. Thank you Aleks for the great collaboration!

Find the workshop files here if you want to follow along github.com/l-mds/dsc-da...

DuckLake: Learning from Cloud Data Warehouses to Build a Robust “Lakehouse” (Jordan Tigani) YouTube video by CMU Database Group

#duckdb #ducklake #cmu www.youtube.com/watch?v=z2Gh...

feat: build SLURM integration for dagster by HPicatto · Pull Request #19 · ascii-supply-networks/dagster-slurm Type of Change feat: New feature fix: Bug fix docs: Documentation style: Code style refactor: Code refactor perf: Performance improvement test: Tests chore: Maintenance Description adds ...

Something about super and computing in the making anyone daring out there who wants to explore? Or folks who want to exchange ideas about SLURM, HET jobs and advanced resource management? github.com/ascii-supply...

good point. I think I only have < 1 hour so BI/vis will have to wait a bit. But otherwise it would be a great addition

#duckdb #dagster #ray #ducklake

Simple Sovereign Scalable Data Stack | Georg Heiler Tired of cloud lock-in and surprise bills? This talk shows how to build a fast, portable analytics stack around DuckDB and Dagster. Along the way of our journey to sovereignty and scale we touch on…

Simple Sovereign Scalable Data Stack georgheiler.com/event/tdwi-2... precursor: pypi.org/project/dags... github.com/dagster-io/c... if you want to see this in action join in Nürnberg or Vienna for some sovereign, scalable data talks in the coming weeks

When the duck quacks: Multimodal querying with FlockMTL YouTube video by DuckDB

#duckdb #multimodal #rag www.youtube.com/watch?v=2qSZ... blobs.duckdb.org/events/duckd...

Katharine Jarmul - Anonymization: Why is it so hard? (PyData Prague #27) YouTube video by PyData

#compliance #anonymization #python www.youtube.com/watch?v=EqQd...

#gis #medium-data #sedona #rust #datafusion sedona.apache.org/latest/blog/...

📈 DuckDB 1.4.0 is out! This is our first LTS release which comes with *one year of community support*. It also supports database encryption, the MERGE SQL statement and Iceberg writes.

For more details, read the announcement blog post at
duckdb.org/2025/09/16/a...

Home | Data inconsistencies Data inconsistencies, architecuture and real world stories

A living Elo leaderboard for analytics/OLAP engines. Public benchmarks (TPC-DS/H, SSB, vendor & community posts) becomes a “match.” Upsets + context matter. Browse the board & poke holes: rebrand.ly/ey6y7hf

Multi-Agentic system Threat Modeling Guide v1.0 This guide builds on the OWASP Agentic AI – Threats and Mitigations publication, our master agentic threat taxonomy, by applying its threat taxonomy to real-world multi-agent systems (MAS). These…

#owasp now gearing up for #llm and #genai - Multi-Agentic system Threat Modeling Guide v1.0 genai.owasp.org/resource/mul...

so we go back to faster than s3 alternatives?