Ribhu

The “Jupiter Greedy” discourse is quite literally sending me into outer space

Introduce yourself with:

One Book 📚
One Movie 🎥
One Album 💿
One TV Show 📺

Every NBA season I learn of a new way to spell the name "Jaylen"

While these are just 3 thoughts I had while reading the above blog, what do y'all think about the "simple parts connected by clean interfaces" bit in the context of LLM-based systems today? What are some patterns and anti-patterns you have noticed?

Unlike deterministic APIs, LLMs can return valid JSON that's semantically wrong. Strict schemas catch this at the interface, not in your business logic.

We NEED this in the modern AI stack.

More specifically the structure of it. What I mean is that when you start architecting an app, one of the first tasks we do is to create API contracts. Creation of a schema to communicate between components is what allows things to independently grow without any fear of breakdown of operability.

Coming back to the thread, what makes any full stack application work well, even with hundreds of components, modules, microservices, and so on, is the reliability of the information.

They operate at the logit layer of any open-weights model to ensure that the specified schema is "almost" deterministically followed.

3️⃣ Lack of reliability in outputs. Which is kind of the point Remy is making because that's what @dottxtai.bsky.social does so well. I would urge folks to try out the outlines library if you haven't already.

Adding simpler flows from the beginning saves us this back-and-forth of adding validation checks and output parsers

A single complex prompt that needs 3 retries costs more than 3 simple prompts that work first time.

Unfortunately, what follows later on is having to add flows to retry, or post-process the output to refine it in a way where we get the output in the desired form and fidelity. Which leads us to more prompts anyway.

2️⃣ More prompts = More tokens = More cost. This is the mind-killer. This mental model is what leads engineers and products people to fit everything into less prompts.

These are your initial set of sub-tasks, which you can later refine as needed.

Now, you go about your task the way you – the master – would, and note down all the steps you needed to get it done. The first thought would be to combine a few of them. DON'T

A mental model that I find helpful with this is the master-apprentice model. Here, you are the master and the LLM (you pretending to be one) is the apprentice.

The ONLY way to get better at this is to train this muscle of breaking down tasks into the absolute singular task that is simple and stateless

And this trickles down to designing prompts as well.

The reason why engineering and product managers exist. When given a goal, a lot of us sub-optimally break it down, based on our cognitive affordances.

1️⃣ Breaking down a task into simpler sub-tasks. The hardest of the 3.

And this simplification of work is by no means easy. There are 3 main factors that make it harder:

Agentic patterns that can alleviate this, can sometimes be worse, where an agent can have instructions on pursuing multiple things at the same time.

There is almost a sort of pride that prompting folks take in being able to do everything in "one-shot" and not having to rely on multiple turns.

Software engineering was built on this principle, but it's something I don't often see AI engineers follow (myself included). Many times there is a tendency to add every single instruction and decision point in a single prompt.

I was prompted to structure these thoughts after reading this blog by @remilouf.bsky.social:

blog.dottxt.ai/do-one-thing...

"Complex systems should emerge from simple parts connected by clean interfaces"

The principle based on which Unix was founded, and which guides building any software systems with a degree of complexity.

Can this be replicated in AI systems? Here's some thoughts I had on the same 👇

Shouldn't the number of donuts be proportional to the calories burnt?

The amount of AI Slop on Xitter is crazy!

Its "godfathers of AI" all the way down