We designed DSL kernels in Blosc2 so that they can be used in the main platforms; nowadays this necessarily includes Web Assembly (WASM) in the browser.
We also implemented a new JIT compiler for WASM specially meant for Blosc2 DSL kernels.
Play with it at: cat2.cloud/demo/roots/@...
Have fun!
New DSL kernels in Python-Blosc2, being parallelizable and JIT-compiling capable, can accelerate code quite a bit π
For example, linspace() has been rewritten to use DSL kernels, and performance boost is well beyond expectations.
Blosc2 4.1 Release!
We've packed this minor release: optimised compression and funcs for unicode arrays; cumulative reductions; memory-map support for store containers like `DictStore` ; and a DSL kernel functionality for faster, compiled, user-defined funcs!ποΈ
Notebook - github.com/Blosc/python...)
Super-efficient DSL kernels are coming with forthcoming Python-Blosc 4.1.0.
A DSL kernel is a function that takes ndarray objects as input and returns a ndarray object as output.
Performance with classic Mandelbrot is kind of mind-blowing.
See github.com/Blosc/python... for full notebook.
The new engine in Python-Blosc2 4.0 delivers quite awesome performance results. Processing a 400 MB array can be up to 9.6x faster than NumPy π€―
The best part is that it fully supports NumPy's array and ufunc interfaces. High performance with zero friction! ποΈπ¨
ironarray.io/blog/miniexp...
Enjoy!
Python-Blosc2 just became a compute powerhouse. β‘οΈ
v4.0 integrates `miniexpr`: computing on cache-friendly blocks instead of RAM-heavy chunks.
π Result? Massive speedups for memory-bound tasks (up to 4.5x in Cat2Cloud).
See benchmarks:
ironarray.io/blog/miniexp...
#Python #HPC #DataEngineering
π’ Blosc2 π€ OpenZL π
Use the OpenZL compression ποΈ library from Blosc2: github.com/Blosc/blosc2...!
πΜ²πΜ²πΜ²βΜ²πΜ²πΜ²πΜ²πΜ²πΜ²πΜ²πΜ²βΜ²πΜ²πΜ²πΜ²πΜ²πΜ²πΈΜ²βΜ²πΜ²πΜ²πΜ²πΜ²π£Μ²πΜ²
and just like that, OpenZL compression + Blosc2 compute engine!
Check out the OpenZL project openzl.org.
Inside story on the plugin β©οΈ blosc.org/posts/openzl...
π Our #PyDataGlobal 2025 tutorial on modern #Blosc2 & #Caterva2 features is out!
Learn how compression boosts array performance & enables cloud computing without downloads. βοΈ
Watch here: π www.youtube.com/watch?v=tUvS...
#Python #HPC #DataScience #BigData
π€ Can a library that computes on compressed data actually outperform performance heavyweights like NumPy and NumExpr?
Find some surprising answers π www.blosc.org/posts/roofli...
During the walk, I'm also introducing a funny anecdote back when NumPy was a πΆ
Enjoy!
#Blosc2 #HPC #NumPy #Numexpr
To round off our presentation of the shiny new Cat2Cloud product, we have a final π ironPill πon data handling from the web client π .
See how to move β‘οΈ , copy Β©οΈ and perform other file management operations easily from the prompt box!
Stop downloading, start doing! π¨
ironArray is on a roll π
!
With Cat2Cloud released this week we continue our πironPill series with another video, showing you how to browse your cloud-hosted data βοΈ in the bespoke web client!
Visualise and manage all kinds of data formats - from arrays to jpgs πΌοΈ, markdown to PDFs π !
Excited to be part of this data story! β€οΈ
Tired of downloading large datasets? Cat2Cloud is now live! π
Bring computation directly to your data. Host, run, and share workflows with incredible speed. β‘
Read the announcement: ironarray.io/blog/cat2clo...
#Cat2Cloud #BigData #DataScience #Python #HPC
Stop downloading. Start computing.
Everything changes on November 18, 2025.
#Cat2Cloud is coming.
Our Blosc2 tutorial for @euroscipy.bsky.social 2025 is up! Learn how compression boosts tensor data handling, sharing & computing.
www.youtube.com/watch?v=BdpT... #DataScience #BigData #HPC #OpenSource
π’π¨Blosc2 3.11.0 Released! π¨ π’
Blosc2 is part of the NumPy array library ecosystem: numpy.org
In v3.11, Blosc2 has become even more flexible; all functions can accept basically any array object:
ππππππΈ.ππππππ(π°, π±)
for π°, π± PyTorch, Jax, Tensorflow, Zarr... arrays!
#Compute
#Tensor
#PyTorch
πIronPill 2π
See how Blosc2 powers heavy-duty linear algebra (100GB!) workflows
β‘1.5-2x faster than PyTorch + h5py!
𧱠optimised chunking for your cache hierarchy
π one line syntax ππππππΈ.ππππππ(π°, π±, πππππππ='πππ.ππΈππ')
See blog here: ironarray.io/blog/la-blosc
#Blosc2
#Data
#LinearAlgebra
π Want to boost performance in your complex calculations? Avoiding temporary arrays is critical!
Check out our new video π¬ to see how #Blosc2 eliminates these temporariesβ‘
Compress Better, Compute Larger
π’Blosc2 3.10.2 Released! π’
Lazy evaluation for ALL Blosc2 functions in version 3.10!
Blosc2 powers:
- Zarr/HDF5 computations (see image and blosc.org/posts/tensor...)
- Server compuation via Cat2Cloud (demo site here cat2.cloud/demo/)!
#Zarr
#HDF5
#Python
There is no data wider than the one offered by the Milky Way! And with the right tools, you can do it all on your laptop.
This is really exciting news! π
It's also worth highlighting that the Blosc2 internal compute engine brings several advanced features to the table:
π‘ Broadcasting
π― Reductions
π€ Superior type compatibility with NumPy
βοΈ Support for User-Defined Functions (UDFs)
Compress Better, Compute Bigger!
π Boost your Python computations on NumPy arrays with @blosc.jit! Let Blosc2's compute engine accelerate your code without changing containers. β‘
For ultimate speed, native Blosc2 containers are still king. ποΈ
Code: gist.github.com/FrancescAlte...
#Python #DataScience #NumPy #Performance #Blosc
π£οΈ Announcing Python-Blosc2 3.8.0 π
A step closer to compliance with the array-api standard: data-apis.org/array-api!
This is an effort across all array-based libraries so that your code works (e.g. for both blosc2 and NumPy) by simply changing the import statement below!
#arrayapi #Blosc2
π£οΈ Announcing C-Blosc2 2.21.2
This is a maintenance release, with a few fixes and some optimizations. Thanks to Barak Ugav and Preeyan Parmar.
For more info, see the release notes in:
github.com/Blosc/c-blos...
Compress Better, Compute Bigger
Enjoy!
Struggling to get performant code from LLMs? π€
We can help! πͺ We've spent countless hours profiling and micro-benchmarking Blosc2 for you. A thing that LLMs cannot do (yet).
Ready to dive in? π
πΉ EuroSciPy Talk: www.blosc.org/docs/2025-Eu...
πΉ Tutorial: github.com/Blosc/EuroSc...
#HPC #Blosc2
π£Great to see the community building powerful tools on Blosc2! π
Check out compress-image: a new C++/Python library for working with compressed images directly in memory.
Kudos to Emil Dohne for this fantastic work!
Project here: github.com/EmilDohne/co...
#Blosc2 #FOSS #ImageProcessing
The slides of my talk about _Blosc2: Compress Better, Compute Bigger_ for the recent @euroscipy.bsky.social:
www.blosc.org/docs/2025-Eu...
EuroSciPy 2025 was great! A human sized event with lots of interesting talks and people ready to help you. Thanks to the organizers for such a fantastic event!
π We're thrilled to announce *TreeStore*, a new class in Python-Blosc2! Endow your datasets with a hierarchical structure! β‘οΈ
π We've blogged about it:
www.blosc.org/posts/new-tr...
It's in beta, and available in Python-Blosc2 v3.7.2. Enjoy!
#Python #Blosc2 #TreeStore #DataScience #OpenSource
Python-Blosc2 comes with an extensively tested partition sizes algorithm for automatically setting the best chunk and block sizes for you.
But you can always bypass this mechanism for more appropriate fine-tuning in different scenarios π
www.blosc.org/python-blosc...
#Compression #Performance
