How it works: I prompt Gemini with the first frame of video to give me (x, y) coordinates with descriptions. Then I use optical flow to track those positions over time. Of course optical flow has limitations, but it was a fast lightweight way to prototype the idea quickly.
I made it with Gemini spatial intelligence + opencv.js in Google AI Studio. Link here: ai.studio/apps/drive/1... It's been fun testing on old videos from my photo library as Gemini is able to uncover new details.
Visualizing Gemini intelligence with things in my life 🔊🧵
Gemini spatial understanding + opencv.js 🔊
Try it here WIP link ai.studio/apps/drive/1...
I really like this labeled format. Feels like a napkin sketch stream of consciousness where I can follow what Gemini's thinking as it draws.
Testing streaming vector shapes with Gemini 3 Flash. Really fast. Realtime screencap⚡🔊
Yes, right metaphor is hard because a single bit of text can represent so much. For my own prototypes, I am using text to represent the feature (I want it to do [x]), design (exact tone.js sounds, hex colors, etc), documentation (how it was implemented).
I always ask Gemini to discuss ideas with me before coding, so I can craft the details I really care about - the sine wave, wobbly-ness, the amplitude ... 🥁
Video has sound 🔊
Sketch ➡️ animation ✏️ Multimodal prompting has been really powerful for this drum machine prototype ... 🧵
As a result I have a growing collection of docs that represent my favorite sounds, colors, and other random interaction design choices. It feels like documentation (or source code?) for my own universe of personal apps that can be built at any time on the fly.
I provide it without the original source code, as I think it helps model flexibly adapt it into my new code base with "fresh eyes" unbiased by the original code.
I've been saying things like"Write a concise spec for [x] ..." to port features between apps. Gemini gives me a readable spec (which I enjoy reading for my own understanding of the code), which I use to prompt a re-implementation in my new app.
Porting features with language ... 🧵
Vibe coding an interface inspired by this fun 1960s drum machine with Gemini.
If you make something, feel free to share the song link here. Code is open-source, built with Tone.js. Forkable here on Google AI Studio ➡️ ai.studio/apps/drive/1...
Polyrhythms are especially easy and fun to make. Song link: alexanderchen.github.io/typeloop/?so...
Every song is represented as a string of text, so you can share it by just copy-pasting or making a link. Here are all the sounds you can try. 🥁
Type Loop 🎵⌨️ Create and share music by typing! Play here: alexanderchen.github.io/typeloop/ 🔊 Open-source. Built w/ Gemini. 🧵 #genuary #genuary10
Drawing + code. ✏️ Love these experiments our Creative Lab collaborator @szymonkaliski.com is doing with Gemini 3.
You can learn so much by just moving a shape around and bringing it to life. I wanted to make it really easy to discover the joy of animation. Easing, speed, scale, squash, sound. I also found the curve editor became a really neat way to simply visualize time. ⏱️
Link + code here: ai.studio/apps/drive/1...
Dot Motion 🔴🎵 A simple way to explore motion, time, and sound. Code is open-source. Video has 🔊 Link in 🧵 #genuary #genuary2
You can click "+" to add more agents. (Fun to watch the parallel conversations all unfold)
Not a new idea, lots of great research in the space of cooperative agents in past years. But the speed and low cost of Gemini 3 Flash and vibe coding environments like AI Studio make experimenting in this space so much more accessible.
Each agent remembers past conversations and uses memory to decide on goal.
Information is passed through back-and-forth conversations like this one.
TinyTown 🙂🍄 Lightweight multi-agent social simulation built w/ Gemini 3 Flash. Agents look for 🍄. When one finds it, it tells coordinates to others. Not sure where I'm taking this prototype next (open to ideas!) Code is open-source ➡️ ai.studio/apps/drive/1...
