Soon to be, it looks like: youtu.be/zeonmOO9jm4?...
Otherwise, there is no point of using Parquet instead of their DuckDB native format. I’m glad they didn’t ignore the “industry standards”
Is there any plan to support data compaction to data lake when data inlining is used?
I was worried about Iceberg being ignored in favor of DuckLake but looks like you fixed Iceberg’s biggest problems and still kept the compatibility. Super exciting!
Turns out the implementation wasn’t WAL but they had a new Iceberg compatible data lake extension. I like the direction they are going!
I have this one but they might have soon to be public extension to use the WAL to keep the data in sync with data lake: github.com/duckdb/duckd...
Is @duckdb.org cooking native data lake integration with streaming support through WAL? That could enable DuckDB to have a multi-user mode..
After not using Facebook for years, wanted to try out Marketplace. Apparently you can send messages to people on the website but you can only see messages are sent to you on their Messenger app. I guess this is their definition of “connecting people”.
That’s a good analogy, might steal it. :) However; when the destination path is not clear (which is usually case as you need to experiment and iterate anyways) smashing can help accelerate finding the destination as you learn where not to go.
Ironically the number of stale documents in our company is increased dramatically thanks to LLM.
Oh I lost count of how much time I waste trying to infer the column names from random CSV files without a header. This is very handy!
Just found out that Databricks hired Snowflake’s Polaris (Iceberg) lead PM. It’s crazy how aggressive these guys with the competition!
Great to see Amazon implementing Iceberg REST Catalog layer for Glue! It enables read/write support on S3Tables from any Iceberg client, now everybody as a free Iceberg catalog via AWS Glue. aws.amazon.com/blogs/storag...
Exactly! I think Flight will get more popular over time as it's the most efficient implementation, but this approach can help existing RESTFul apps to adopt SQL integrations before switching over to GRPC.
The main inspirations are github.com/PostgREST/po... and @qxip.bsky.social 's DuckDB webmacro extension: duckdb.org/community_ex...
Released an experimental @fastapi.tiangolo.com integration with @duckdb.org today, which enables REST APIs to have bidirectional read/write support in SQL. github.com/buremba/duck...
Pretty common but if one of these languages is the “main” one, it might be more desirable to generate JSONSchema from Pydantic/TS and generate the models for other language from JSONSchema. It’s more about where you want the source of truth should be.
I had the exact same thought..
"think twice before you speak."
Thanks. I'm also a fan of your creative extensions! Quackpipe was one of the inspirations. :)
Today I had to explain my partner what @duckdb.org is because “I will fly to Amsterdam for a day to meet ducks” didn’t make any sense to her. Excited to meet with the contributors! duckdb.org/events/2025/...
One here! 🍻
It's interesting to see many seed-stage, well-funded startups trying to "re-write X in Rust." as a business model.
WarpStream, ScyllaDB, and Redpanda are successful because they're either 10x efficient or make the maintenance much easier than their alternative, not because they're written in C++
I couldn't figure out how to insert a table into an S3 Table without Spark. I tried to use the API but it requires me to create the files and update the metadata. PyIceberg can't write to S3 Tables through its S3 integration yet so I had to stick to Spark. boto3.amazonaws.com/v1/documenta...
If AWS is serious about S3 Tables, they should support Iceberg REST Catalog in it. Right now we can only create tables with Spark.
Qlik's Upsolver acquisition shows the importance of adopting new technologies as a potential acquisition target for bigger companies. It's a 10-year-old company, and they raised a ton, so I'm not sure how good the deal was for the co-founders.
dbt acquiring SDF Labs shows how important it is to have a good relationship with your competitors. SQLMesh might be more ambitious, but I'm sure it was a good exit for SDF founders in only 2 years!
It's a good day to be acquired in the data space.
For the record I checked if Motherduck notebooks ahave it but doesn’t seem to be the case, at least yet.
Look great! I would love to try out, Where is this going to be available?
People say LLM is killing low-code platforms such as Retool and Bubble, but they seem to hire more people + raise even more funding. They're better positioned to leverage LLM maybe.
The AI tools like bolt.new and v0.dev work best with Next + Shacdn combination after all, so I wouldn't be surprised.
