cool ideas! i've been slowing teaching myself FPGA programming so i can run neural nets at audio rates ( e.g. matpalm.com/blog/wavenet... ). i am iirationally chasing nano second inference times!
code is all there if you want to mess around; pretty old though so definitely won't run as is...
At Imagine Innovators, we just announced our Developer Plan, offering expanded access to Edge Impulse’s powerful tools, completely free!
Get GPU training, longer runtimes, more projects, and production-ready licensing, all at zero cost.
Details: www.edgeimpulse.com/blog/introdu...
have moved mixing out of the rack to the model1. such a great performance focussed mixer! #improv #techno
It's about time I updated my recommended #FPGA links. Send me a suggestion this #FPGAFriday. 😊 projectf.io/recommended-...
i wish this was always an option...
still "finding" ( i.e. stepping on ) screws from the great M3 drawer spill of Sept '24
I'll take "n^2 - n cursed images" for $1000, Alec
_finally_ genAI doing something useful. about time.
Great interview with @jascha.sohldickstein.com about diffusion models! This is the first in a series: similar interviews with Yang Song and yours truly will follow soon.
(One of these is not like the others -- both of them basically invented the field, and I occasionally write a blog post 🥲)
Making LLMs run efficiently can feel scary, but scaling isn’t magic, it’s math! We wanted to demystify the “systems view” of LLMs and wrote a little textbook called “How To Scale Your Model” which we’re releasing today. 1/n
"In what part of the world do you currently live?" lol
it's a super clever reparameterisation. though the tensors might need to be bigger, i feel it'll be paid for by the simpler streaming implementation on an FPGA. hacking / time will tell!
keep coming back to "Scalable MatMul-free Language Modeling" arxiv.org/abs/2406.02528
full of fun ideas! am slowly mapping my FPGA audio rate cached casual dilated convolutions network to operate using their mat mul free GLU version of self attention.
slowly getting there!
same with jerky!! got some the other day that expired a few weeks after opening ???? wasn't that the entire purpose of jerky?!?!
ooffff! the "DeepSeek-V3 Technical Report" is info dense!
arxiv.org/abs/2412.19437
still my fav "at doom's gate" cover!
( i'm probably 1K of those 2M views )
www.youtube.com/watch?v=TZG1...
this looks like it's going to be a good read!
www.amazon.com/AI-Robotics-...
went camping in an area with lots of eucalyptus and was surrounded by koalas! so cute!
that _was_ a weird one...
nice one! super excited to see what comes out of this lab!
no problem at all! will post blog when i'm done
( the next generation of this matpalm.com/blog/wavenet... intending to run on matpalm.com/blog/wavenet... )
first logic gate waveshaper results tonight! this is network represented purely by combinational logic ( see arxiv.org/abs/2210.08277 ) i.e. just look up tables; no mults or adds. trained in jax ( 3 nested levels of vmap! ) & inference in amaranth. inference speed should be ~10s nanoseconds.
my largest category of splurge spend for last month was "bandcamp" :D #winning
and here i am trying to debug why param_set((2,3,16), seed=1) vs param_set((2,3,16), seed=2) isn't working... ::/
welcome @jeffdean.bsky.social !
i'm porting github.com/matpalm/cach... to use github.com/Felix-Peters... so will post progress along the way!
