“Gradually, then suddenly.” That’s how adoption works when you’re building something new. Opendata is still "gradually" but 100 stars with $0 spent on marketing is a good start. Back to building! github.com/opendata-oss...
Yes! With larger payloads compression is almost always worth it. Thanks for sharing
I implemented prefix compression for SlateDB & noticed benchmarks looked "worse".
Fell down a rabbit hole. Turns out I was thinking about compression backwards... Wrote up my learning: www.bitsxpages.com/p/the-mathem...
Can you beat 180KB? I created a challenge to reduce a dataset as much as possible 🏆
my approach uses delta encoding & prefix compression + zstd(22) compression to reduce 25MB -> 180KB
github.com/agavra/bit-g...
by the way, it's pronounced like "tweaker"
I fixed the problem with reviewing code written by Claude in "accept edits on" mode: github.com/agavra/tuicr
would love to know what you think, and if you want to contribute to an OSS rust project there's a bunch of open issues to pick up!
I'll die on this hill: Sorted String Tables (SSTs) are the single most important data structure for modern DBs. They lean in to the limitations of SSDs and Object Storage, making them (and similar layouts) the best choice for many databases.
Blogged in detail: www.bitsxpages.com/p/sorted-str...
thanks for the kind words! I use monodraw.helftone.com -- not sure if planetscale/turbopuffer use the same but I was definitely inspired by their design style
The inner join between sets of people who build databases, write, and draw? Low cardinality. I'm in that set, so I'm starting a blog!
Here's my first post: www.bitsxpages.com/p/frameworks...
Sometimes the best solution is "do nothing", but it's always more fun to play with tools.
Calling database nerds in SF! I'm covering SlateDB at the systems meetup next Wednesday (10/29). If you're around, I'd love to meet you in person (that way you'll have proof I'm not just an AI bot).
👉 luma.com/e7feg2i6
I wonder why it hasn't made it's way to the US! The only time I get it is when I make it at home.
Beans on toast is underrated.
I recently implemented Gorilla encoding (www.vldb.org/pvldb/vol8/p...) for a SlateDB PR. Pretty cool stuff - easy to understand but really powerful. Here it is, explained by a gorilla.
I guess the answer to that is “technically, yes.”
It's true. Every company eventually becomes a database company.
Despite using so many new technologies, I somehow never learn my lesson: read the docs sooner and read the docs thoroughly.
Reading the tokio-rs async documentation makes me feel like...
that's so cool - I love contraptions that are just there to show that we can do something!
No shame in appreciating bona fide nerd humor 😉
One day I'll open a coffee shop dedicated to the not-insignificant intersection between database nerds and coffee snobs. Until then, enjoy this comic.
Maybe... just maybe, adding more features and complexity into stream processors is NOT what we need?
Fact.
The new electric Caltrain cars have WiFi. 🙏
SF bay area has finally entered the 21st century (on this dimension of public transit only).
Today marks SlateDB’s one year anniversary! It’s been a lot of fun. Thanks to @rohanpd.bsky.social @flaneur2024.bsky.social @almog.ai @vigneshc.bsky.social @paulbutler.org Jason Gustafson, David Moravek, and many others for joining the project. 😀
This got me thinking: Should MCP itself formally distinguish between request vs response context?
It could let LLMs be more intelligent about when to pull extra info. 🤔
[6/6]
I built a demo agent for hotel booking. My setup:
Inject request context into the prompt (user prefs, budget, pulled from account)
Fetch response context live using MCP tools (available hotels that match search params)
Result: the agent felt way smarter.
[5/N]
Back when we only had prompts, you could frame the task (request context).
Today, tools like MCP change the game. But should you use them?
[4/N]
Example: a hotel booking agent 🛌
Request context: "Book a hotel in Barcelona on April 28 under $400"
Response context: A list of hotels available in Barcelona that day, within budget.
[3/N]
⚖️ You need to balance two types of context:
Request Context = frames the task
Response Context = info needed to complete it
[2/N]
