Talking of fast data loaders, we replaced Apache "commons-csv" with hand-coded CSV parser in DFLib, and suddenly all my log processing is 6x faster! (about 4.7s for an 8GB CSV vs 29s before) 🚀
And we have a chance to make it 12x or better based on early parallelization POCs
Oh nice. We just finished a set of Parquet optimizations in DFLib based on parquet-java. Looks like our reader is due for another facelift
I'll see what I can do. The dataset is 50% PII / 50% company finances :) But I can probably share the code once I figure more complex UI scenarios, like nested dialog editors and such
Just hacked a simple table viewer for one of the company datasets with Tamboui / DFLib / Bootique. Looks promising to build internal business automation on the cheap with a single Java stack.
This sounds pretty cool. I guess JJava kernel won’t work in the browser yet (WebAssembly and all), but I am curious what it would take to make it work
A new milestone of JJava Jupyter kernel is out. We had a community request for a more extensible lib to build custom extensions, distros or even kernels. We did just that. Thanks to @maxandersen.xam.dk
for providing feeback and PRs through the process! github.com/dflib/jjava/...
DFLib 2, milestone 4 is out. Massive QL improvements. I finally switched all of my own projects to QL from the API-based expressions. Also some new ECharts stuff github.com/dflib/dflib/...
Matches my experience
DFLib just got a parsable expression grammar. Those expressions can be thought of as parts of SQL for the purpose of column transformation, filtering and (soon) DataFrame sorting
github.com/dflib/dflib/...
Wow, just got a piece of decent code out of Gen AI - a deep clone method for an Open API descriptor tree. Still had to go line by line, adding missing properties, but the code was generally clean and meaningful. Unfortunately such success is a rare occasion
My first time at @lu.voxxeddays.com. Pretty exciting!
In the meantime, working on getting the next DFLib milestone out of the door with the new parser of SQL-like expressions, making expressions more readable compared to the current Java DSL
Just updated my Java dataframe test & benchmark. @andrus.adamchik.org contributed an implementation for DFLib, which is now the fastest Java dataframe (not counting @duckdb.org which is much faster, but doesn't give you the dataframe syntax)
github.com/mathijs81/ja...
The a5 version of the Jupyter kernel is officially released. Looking forward to feedback on the new Maven engine. I feel like adding a flag equivalent to "mvn -U" for snapshot refreshing may be the next logical step.
github.com/dflib/jjava/...
Request for name suggestions:
I am building a language to define data schema (similarly to what xsd and json-schema do) that is format independent, and thus it could be used to validate data serialized in multiple different formats.
It’s optimized for ergonomics, as I find current schema […]
Try googling (or whatever the term is for ChatGPT 🙂) for "Jupyter keyboard shortcuts". Notebook keyboard navigation is awesome and makes you very productive (inserting cells above and below, changing cell type between code and markdown, etc.)
The "TAB" key should do it.
Been testing the soon-to-be-released DFLib JJava Jupyter kernel v1.0-a5. No more Ivy there, just pure Maven. Makes dependency management so much more reliable.
Ok, the first version of the docs just got published at the link above.
Every 10-20 years, a breakthrough technology promises non-developers to finally create software without needing to hire programmers.
So far, every such technology resulted in the need for more devs… expert in this NEW technology (or knowing how to fix it up).
Every. Time.
Thanks! Yeah, we'll prioritize filling this gap in the docs. For now, look up the class called "Excel" and its static methods. E.g.:
// load all sheets
Map<String, DataFrame> data = Excel.load("my.xlsx");
// load a single sheet
DataFrame df = Excel.loadSheet("my.xlsx", "Sheet 1");
Enjoyed the closing keynote at @geecon.bsky.social by @jbaru.ch and Leonid Igolnik. They tried to envision an "enterprise" version of vibe coding (that's not how they worded it 🙂) Though futuristic, it came out rather convincing speaking.jbaru.ch/DVCzoZ/back-...
Today I learned that my position at Microsoft was discontinued, amongst many others. Please reach out if it makes sense for us to explore something new together. My bio is always up-to-date. reza-rahman.me/about/
Andrus Adamchik presenting DFLib in a Jupyter notebook in front of a screen
Tonight, @andrus.adamchik.org is giving us an overview of DFLib!
Server management approach is often described as "pets vs cattle". I think the same analogy might apply to data representation in apps: objects - "pets" 🐕, DataFrames - "cattle" 🐄 . Too brutal?
Very nice! I resisted the urge to make “$” a method name in DFLib for expression shortcuts, but here it looks very natural 🙂
Ever been annoyed with how complex it is to execute processes in Java and handle stdin, stdout and stderr?
Me too - so Introducing JBang Jash - github.com/jbangdev/jba...
Join us on April 30, 2025 at Deutsche Bank to see @andrus.adamchik.org give a talk on DataFrame!
www.meetup.com/triangle-jav...
Just published DFLib (#java DataFrame library) v1.3.0. This is a backwards-compatible maintenance release with a number of bug fixes, new statistical expressions, and an optimized vConcat operation github.com/dflib/dflib/...
#JAlba is happening again in 3 months, registration is open! It will be our 6th edition, don't miss out on attending the #Java #unconference in Edinburgh Scotland, created on the model of #JCrete. Come and meet #Java champions and experts. Register now: jalba.scot
