pgflow

how it compares to Manjaro?

Tagging @supabase.com so it does not get missed :-)

Example code snippet: // Provide a type for the input of the Flow type Input = { url: string; }; export const AnalyzeWebsite = new Flow<Input>({ slug: 'analyze_website', maxAttempts: 3, baseDelay: 5, timeout: 10, }) .step( { slug: 'website' }, async (input) => await scrapeWebsite(input.run.url) ) .step( { slug: 'sentiment', dependsOn: ['website'], timeout: 30, maxAttempts: 5 }, async (input) => await analyzeSentiment(input.website.content) ) .step( { slug: 'summary', dependsOn: ['website'] }, async (input) => await summarizeWithAI(input.website.content) ) .step( { slug: 'saveToDb', dependsOn: ['sentiment', 'summary'] }, async (input) => { const results = await saveToDb({ websiteUrl: input.run.url, sentiment: input.sentiment.score, summary: input.summary.aiSummary, }); return results.status; } );

A flowchart showing data from a website being processed through both sentiment analysis and summary generation, with the results from both paths being saved to a database.

Just merged the new Flow DSL in pgflow! 🚀

Next up: Edge Worker refactor to make it run flows defined with the DSL.

Check out the snippet for a simple AI web-scraping flow. 🤖

I also talked through millions of LLM tokens (I brainstorm a lot) and I have concrete implementation plans for:
- fanout steps
- conditional steps (every step type will have conditions)
- subflows and fanout subflows
(but those will be included after the initial MVP is released).

#Postgres #SQL

It is a productive week so far for my @supabase.com
workflow engine:

- defining flows, steps, and dependencies
- starting flows, queuing stuff
- polling for tasks
- completing tasks
- dependency resolution
- 1000 lines of tests

What models do you prefer?

For brainstorming and planning I often use ones with reasoning - either Deepseek R1 or O3 mini.

Also - anyone having troubles with images on bluesky? When i paste one it becomes black (tested on firefox, chromium and chrome)....

Balancing the simplicity of the low-level SQL functions with the flexibility demanded by high-level pipeline types will be critical for the long-term success of pgflow.

However, careful attention must be paid to maintaining clean abstraction boundaries, preparing for schema evolution, and designing robust integrations with external workers and (potentially) UI layers.

- Simplified, auditable, and maintainable low-level primitives.
- The ability for rapid MVP delivery while keeping the door open for sophisticated pipelines (e.g., foreach, parallel, and conditional processing).
- Enhanced developer experience through strong typing and better tooling in TypeScript.

Produced by o3-mini from a huge prompt and lot of SQL migrations in context:

5. Conclusion

The proposed design of pgflow—using a minimal, declarative SQL orchestration layer as building blocks in combination with a high-level, expressive TypeScript DSL—offers significant advantages:

Brainstorming & viability checks with LLMs - anyone else doing this? 🤔 Curious to see how common it is! (recent conclusion in the reply!)

#LLM #buildinpublic #AI #supabase #postgres

Pros:
✅ Faster queries
✅ Automatic updates
✅ Data consistency

Cons:
❌ Extra storage space
❌ Slower writes
❌ Limited to simple expressions

🐘 Postgres Tip: 𝗚𝗘𝗡𝗘𝗥𝗔𝗧𝗘𝗗 𝗰𝗼𝗹𝘂𝗺𝗻𝘀

Generated columns are a powerful PostgreSQL feature that automatically update based on changes to underlying data, ensuring your calculated values always stay current.

Useful for storing frequently computed values.

#Postgres #PostgreSQL #Supabase

I spent last month on:

✅ Learning how to tweet, gaining 130+ followers (thank you 🙇🏻)
✅ Refining Edge Worker
✅ Improving docs and SEO
✅ Testing big, continuous workloads on hosted @supabase.com
✅ Suffering through a massive flu 🤧

This was enough chores - excited for main part now! ✨

✂️ Time to extract pgflow! 🚀

I'm starting March with a laser focus: 𝗦𝗵𝗶𝗽𝗽𝗶𝗻𝗴 𝗽𝗴𝗳𝗹𝗼𝘄 𝗮𝗹𝗽𝗵𝗮 𝗔𝗦𝗔𝗣! 🚢

I also plan to post some technical findings, snippets and ideas, mostly around queues and SQL, so if you are interested in this kind of stuff, watch this space! 🔭

#supabase #buildinpublic

Flu's been "in production" for days. Constantly hitting CPU clock limits (zero focus), and my runny nose desperately needs horizontal scaling for tissues. Needing an emergency patch ASAP! 🤢

🚀 Edge Worker 𝗽𝗿𝗼𝗰𝗲𝘀𝘀𝗲𝗱 𝟳.𝟯𝟳 𝗠𝗜𝗟𝗟𝗜𝗢𝗡 𝗷𝗼𝗯𝘀 𝗶𝗻 𝟭𝟲 𝗵𝗼𝘂𝗿𝘀!

I'm testing production deployments and continuous operations on the @supabase.com FREE tier:

🔥 𝟯 𝗮𝗰𝘁𝗶𝘃𝗲 𝘄𝗼𝗿𝗸𝗲𝗿𝘀 (kept up by pg_cron)
⚙️ 𝟭𝟭,𝟬𝟬𝟬 unique Edge Function instances spun up
⚡ 𝟭𝟮𝟴 𝗷𝗼𝗯𝘀/𝘀𝗲𝗰 average throughput

It runs perfectly fin

🚀 𝗘𝗱𝗴𝗲 𝗪𝗼𝗿𝗸𝗲𝗿 𝘃𝟬.𝟬.𝟰 𝗶𝘀 𝗼𝘂𝘁!

v0.0.3 had a pretty bad bug—production deployments didn't work because I mistakenly tested a development version in the cloud, instead of the JSR package.

It's fixed now (live on JSR). If you run into issues, please try again!

I appreciate everyone’s interest—your eng

Tough one... The feature is already validated, and landing page will probably get validated only when ready - I would vote for the feature!

Adding a second example that is more straight forward with the types, thanks for the heads up!

We all love using #TypeScript with @supabase.com ! ✨

Flow DSL makes it super simple to define flows:
- Type for Flow params (e.g "string")
- Build flow step by step
- Steps have name, optional deps & handler func
- Dependencies determine handler payload

It couldn't be simpler - change my mind! 🚀

Yes! There was a bug with starting root steps (ones without dependencies).

Kudos for noticing!

I'm building pgflow to seamlessly integrate with @supabase
's building blocks.

That's why the flow shapes are kept in the database - this allows streaming their progress using Supabase Realtime! ⚡ How cool is that?

November prototype (can anyone spot a little bug?):

Those are good times to be building stuff!

That must feel extremely satisfying! Hope to reach that place at some point myself. Good luck!

EdgeRuntime.waitUntil(new Promise(() => {}));

pgflow is not an empty promise #supabase

Excited to share Edge Worker PoC! Built on top of @supabase.com Background Tasks to make your PGMQ queue processing easier with automatic retries, concurrency limits and observability. Learn more at pgflow.dev ✨

#postgres #sql #pgmq #supabase