The one thing I love is how Opus drafts its plan. Data flow is a masterpiece - and yes, I started to carefully read the details of what it's going to proceed, because A) it makes mistakes or misunderstands, B) it's highly educational.
- Data Preview: Have data but unsure what's inside? Explore it directly!
- Verifier View: evaluate generated data, remove duplicates, assign ratings
and many more!
github.com/mkurman/synt...
- Multi-turn Support: pass one DEEP run, let the model ask follow-up questions, and choose who should respond using SYNTH-like thinking
- Firebase/Firestore: download your data directly as a JSONL file or upload it to your Firestore (production mode)
- Generator: create your own dataset from scratch
- Converter: use existing datasets (Hugging Face support) with reasoning traces to match our SYNTH style
- DEEP Mode: multiple agents working together in various configurations
Would you like to build your own SYNTH-like datasets or contribute to the development of SYNTHLabs?
Now you can. SYNTHLabs is fully open and waiting for your contribution!
What's inside?
I'm back here after a huge break to check out how bsky works now π
It has a minor bug that requires further fine-tuning (sometimes it starts with the <|python_tag|> instead of <Thought>.
Here is my experimental Llama 3.2 3B with o1-like thinking. It utilizes Thoughts when needed, so don't be surprised when it's not.
Enjoy!
Give some likes to make me feel better π
huggingface.co/mkurman/llam...
storm.genie.stanford.edu - A great tool from Stanford for creating articles. For me, a stronger Gemini with Deep Thinking. Definitely worth trying!
Deepseek MTP is something you should definitely look at
Predicting the next token as a learning objective is insufficient for optimal LLM training.
I will definitely give it a try!
HDIC - How Do I Contribute?
A new technique we are working on seems to have a huge impact on language models' generative capabilities, allowing the layers to self-esteem their contribution to the final prediction.
RIP JetBrains subscription β οΈ after six years, it became too heavy to use as a daily IDE. Iβm now on the VS Code team.
What research tools would you recommend for searching and analyzing scientific papers?
PS. You can find the model mentioned here: huggingface.co/meditsolutio...
License: Apache 2.0ββββββββββββββββ 4/4
the model achieved better results in IFEval and a higher overall average score in Open LLM Leaderboard
I consider this a big success π, since surpassing the original in metrics is often very time-consuming, generates high costs, and doesn't always work out. 3/4
In total, not much really, since we don't have the original trained under the same conditions as our upscale. However...
1. We scaled up the model without losing its quality
2. We confirmed that the method we devised works
3. After extremely short fine-tuning, 2/4
We built a new small language model SmolLM2-MedIT-Upscale-2B, based on SmolLM2-1.7B-Instruct from Hugging Face. The premise was simple - increasing the vector in attention layers would positively impact the model's capabilities.
What did we prove? 1/4
This video clip excellently demonstrates the exceptional uses of AI.
youtu.be/MMryYio0v6k?...
It looks like we can scale up any model not only in-depth but also in width. Insane.
#llm
Polish has a nice set of open data from the SpeakLeash initiative.
Link: speakleash.org
