Felix

Why are embeddings so cheap? or a lesson in profiling and FLOPS per dollar

www.tensoreconomics.com/p/why-are-em...

Why does embedding the entire Wikipedia only cost a few dollars? Deep dive blog post, link below

Hi 👋

LLM Inference Economics from First Principles The main product LLM companies offer these days is access to their models via an API, and the key question that will determine the profitability they can enjoy is the inference cost structure.

www.tensoreconomics.com/p/llm-infere...

deep dive on how LLM inference works (link in the post below)

the new RTX 5000 Blackwell seems like a quite good deal compared to a modded 4090 48gb

Singapore is very impressive :D

made it to Singapore :)

i'll be attending ICLR. DM me if you would like to chat :)

what do you think is a reasonable time to post here? (my posts are naturally more in the evening of EU time - so morning time in the US)

I'll be leaving London on Saturday afternoon

but I'm continuing to do it anyway 🙃

I’ll be in London starting Sunday evening (tomorrow). I want to meet people! :D (please DM me, my DMs are open)

Posting here feels far more like shouting in a void than on X/twitter 🥲

I finally got around to putting some stickers on my laptop 😊

I met someone yesterday who works in LLM safety (of closed models) but yet he didn’t know the term LLM inference and didn’t understand the concept without an in depth explanation 🙃

Why are system prompts for image diffusion models not a thing?

Happy new year everyone 🎆 2024 was a really good year for me, I hope 2025 will be even better 🚀

I’m having fruity tasting coffee ☕️ #38c3

It being physically small and very low power consumption at idle probably also plays a role (vs having a desktop computer with 2 4090s idling)

I’m attending the #38c3 in Hamburg. DM me to meet up :)

who remembers GPT-3 Davinci pricing?

the o3 announcement feels very analogous to the GPT-3 announcement in early 2020: "here is new tech, it's very powerful, but it's super expensive to run and we're super selective with who we give access to"

I’ll be at NeurIPS starting Tuesday morning. I want to meet people! :D (my DMs are open)

I'll be in SF starting this weekend. I'd love to meet cool people :D (My DMs are open)

Then choose the Jupyter way, you’ll have a web browser and openssh installed anyway

The vscode ssh functionality works pretty well or alternatively tunnelling web traffic through ssh for Jupyter is also neat

I have the same problem - I hope people just follow me first so that narrows down the search space

How do I do that?