Route between OpenAI and Anthropic with one API key without an account for either.
ngrok's AI Gateway now has its own API keys and prepaid credits. One key covers both, and you can just set your model to "ngrok/auto" if you don't even want to pick.
ngrok.com/blog/ai-gate...
ngrok's AI gateway now supports the Anthropic SDK natively. Change your base_url, keep your prompt caching, extended thinking, all of it.
The diff is one line & there's no step two.
ngrok.com/blog/native-...
What do LLMs see?
I wrote a lil' tool that extracts the attention matrices out of open models and creates this typing visual, with each token's opacity changing according to its average attention score as the prompt progresses. Dimmer words are considered less important to the model.
This week on Overcommitted, we got to sit down with Bluesky's favorite tech blogger @samwho.dev and it did not disappoint!
Sam makes some of the coolest tech content on the internet, and if you haven't heard from him yet, you should! Full episode out now: overcommitted.dev/interactive-...
We're hiring a full-time video creator. Looking for someone who always aims for quality above cadence, is eager to deep-dive on AI and networking, wants to be part of a small and taste-obsessed team.
Details: job-boards.greenhouse.io/ngrokinc/job...
Did you know that SWE-bench only tests how good models are at Python? Or the big drama around the FrontierMath benchmark? We didn't either! But now you will, because @samwho.dev put together an extensive explanation of 14 popular AI benchmarks.
ngrok.com/blog/ai-benc...
These all started as product feedback from folks who couldn't use ngrok without them or felt one papercut too many.
Give one or all of em a *hey i want this please* click on our early access page in the dash.
dashboard.ngrok.com/developer-pr...
Three ships coming soon we're excited about:
◆ region pinning for data residency
◆ similarly, dedicated IPs instead of our default range of multi-tenant IPs
◆ faster & more reliable certificate provisioning for ngrok/custom/wildcard domains ↓
A favorite beat on the new ngrok.com: We put ngrok on top of ngrok.com so you can ngrok while you learn about ngrok. If we're going to ask you to use us for anything, we better do it ourselves first, in prod, and also show our work.
And we finally made a homepage that says that, and much more, with some real *gusto*.
ngrok is one gateway for all your traffic.
So, time to introduce the new ngrok.com
For years we've helped you secure, transform, and route to services running anywhere. localhost to prod, APIs to AI models, devices in the field to databases in customer networks.
We heard you like paging through dozens or hundreds of objects in API responses to find the one you want.
Just kidding, nobody's ever said that, so thank goodness that server-side filtering of the ngrok API is now GA.
ngrok.com/blog/api-fil...
An enjoyable look at how LLMs work under the hood from @samwho.dev. The whole chain of tokenizers (text chunks), embeddings (vectorized text chunks), transformers (the "T" in ChatGPT), caching (reused chunks), and all the math that goes into returning a response.
ngrok.com/blog/prompt-...
Learned more about LLM's under the hood from this post than my stupid $30 online course
ngrok.com/blog/prompt-...
@samwho.dev doing @samwho.dev things 🙌
Privacy is at our core! 🛡️
New post! ✨ Prompt caching ✨
My first big project post for @ngrok.com. 5 weeks, 12217 lines of code, 195 commits. I poured a lot into this one, and learned a lot in the process.
I really hope you enjoy it ❤️
Yesterday we launched ngrok.ai into early access. Today we're bringing you a deep dive into LLM internals with beautiful visuals crafted by our very own @samwho.dev. Discover exactly what gets cached to offer you 10x cheaper input tokens.
ngrok.com/blog/prompt-...
One gateway for every AI model. Change your baseURL and we handle routing, failover, key rotation, and more. Early access is open—and we're building the roadmap with you.
🔗 ngrok.com/blog/ngrok-a...
At ngrok, we embedded the OWASP Core Rule Set and Coraza into ngrok's own Traffic Policy engine, and then dogfooded it across 300M+ requests to ngrok.com. 🛡️
In this post, Ben Chan walks you through how we built (and battle-tested) ngrok’s WAF! ⚔️
🔗 ngrok.com/blog/ngrok-w...
Shape #1: The Database Gateway
Gateways aren’t just for APIs. This one lives between your users and the database you need to (securely) put online, offering mTLS, rate limiting, query filtering, and secrets support, all at the edge.
🔗 ngrok.com/docs/univers...
1/5
ngrok Documentation: The many shapes of API Gateways and where ngrok fits in
To celebrate the many shapes of gateways (🔗 RE: ngrok.com/blog/api-gat...), this week we'll walk through a few of our favorite gateway patterns; how they work, what they solve, and where ngrok fits in each one!
ngrok Blog: Sync secrets from external sources to ngrok with Kubernetes External Secrets
If your secrets live on a platform like HashiCorp Vault, AWS Secrets Manager, or Google Secret Manager, you may want to sync them into ngrok Vaults via the External Secrets Operator so your Traffic Policies reference a single, rotated source of truth. 🔑
🔗 Check: ngrok.com/blog/kuberne...
ngrok Blog: Why didn't ngrok go down in last week's AWS outage?
Last week’s AWS outage took down half the internet. ngrok stayed up, and a bunch of people asked "HOW?" 🤔
Our Senior Customer Engineer, Peter Yoakum, covers just that in the latest on our blog. 🕸️
🔗 Read here: ngrok.com/blog/dont-us...
ngrok Blog: What are AI gateways, and do you even need them?
“The AI gold rush has led to a very modern problem: too many shovels, too little gold.”
AI gateways are how teams keep their models, costs, and chaos under control. 💸
Here’s a guide on how they work and whether you actually need one...
🔗 ngrok.com/blog/ai-gate...
ngrok Blog: Self-hosting with and without ngrok
Start from scratch or skip straight to simple. 💯
In this blog post, @samwho.dev helps you build up the moving parts of self-hosting your app first, then shows how ngrok makes it all effortless.
🔗 Read here: ngrok.com/blog/self-ho...
ngrok Blog: Self-hosted local AI workflows with Docker, n8n, Ollama, and ngrok
Spin up n8n, Ollama, and ngrok in Docker Desktop and self‑host your AI workflows, sharing them securely, and skipping all the networking headaches. Very shrimple. 🦐
Here’s your guide to building local AI automations that actually scale!
🔗 ngrok.com/blog/self-ho...
ngrok Blog: How Rinsed builds data pipelines in car washes: the lessons learned
Rinsed is building real-time data pipelines for car washes, the ones with backroom servers running Windows XP, locked-down networks, & operators who’ve never heard of port forwarding. 🤯
Read how they meet the network exactly where it is and how ngrok helps them do it! 🧼
🔗 ngrok.com/blog/rinsed-...
We'll put our dogfood where our blog is; built, powered and served proudly by ngrok itself.
Take a peek at our glow up 👇
🔗 ngrok.com/blog
AI Agents hallucinate, leak PII, ignore instructions, and break in weird ways all the time. So how do you trust them before shoving to production? 🤔
We tested an AI agent with Vijil AI and ngrok and what we learned about agent failure (& trust) surprised us.
🔗 Read more: ngrok.com/blog-post/ai...
Some years ago, API gateways were just monoliths guarding THE prod.
Now? They’re everywhere: dev boxes, CI previews, LLM stacks, even your database. 🤷
We broke down 8 modern gateway "shapes" and where ngrok fits into each one.🔮
🔗 Read more: ngrok.com/blog-post/ap...
