getting ready to release distributed python environments @isidentical.bsky.social has been working on for fal.ai
it is incredible, we are for sure writing a blog post on it
Growing list of data people to follow@spite.vc
@jakthom.com
@lloydtabb.bsky.social
@grk.bsky.social
@teej.bsky.social
@randyau.bsky.social
@launchit.ai @ryxcommar.bsky.social
@thebbennett.bsky.social
@beeonaposy.bsky.social
@skamille.bsky.social
@seanjtaylor.com
@emilyriederer.bsky.social
Ok hot take; llm projects are packaged very well, you dont even need docker to productize them
who are these people?
typescript is wildly successful because it not a special js runtime and produces js code that can work with existing libraries 100% of the time
but then there will be differences in behavior that impacts how you thought your program ran.. things still wont work at the end lots of libraries need to be re-written in mojo. and that's ok.
its not going to just work if you try to use your existing python program.. after a lot of new extra work the existing programs will start working and they might even have 100% coverage in terms of python functionality on paper..
just to be clear mojo is not a python superset yet! it aspires to be one, its a long term goal not something out of the box
you mean duck
👋
have you guys seen creator of LLVM just released a new programming language? why would, world's richest man, bernard arnault create a new programming language?
just point FalServerlessRunner to your local dbt directory and it takes care of the rest. syncs the project to the cloud, prepares an environment if necessary, runs the dbt command and releases cloud resources
new addition to fal, FalServerlessRunner - https://github.com/fal-ai/fal/pull/842
best way to run a whole dbt project in fal-serverless
works especially well if you use dbt-duckdb, and keep a duckdb file in the fal-serverless persistent filesystem
this is coming along nicely, bonus: also running a small `dbt run` with each event and only storing the transformed table
who is excited to use polars in python dbt models?
Do I know anyone who uses databricks notebooks at work? How do you manage python environments?
at least one pr to the docsite repo after every customer call
“This meeting should have been an offsite”
30
You can follow along here:
https://github.com/fal-ai/fal-events
The beautiful part of this is that the simplest version of this implementation is really simple and good enough for a lot people
Most certainly this will become a problem. I dont expect duckdb to handle a large amount of writes. All the events will be stored in a single duckdb file, that wont scale forever at that point we can swap up storage with open source delta lake (delta-rs) or iceberg
I expect this project to grow as our company grows. First we'll be sending all events directly to a duckdb instance and everything will be stored there
This will be a reference implementation to how do build such a system using fal-serverless
Previously we were using posthog and snowflake. We don't have a lot data but still snowflake was charging us 1k a month
We are going to move all our internal user events and metrics to duckdb hosted by fal-serverless
Coalesce proposal submitted! 🤸♂️
Thanks for the invite @davidgasquez.bsky.social
I made it!
