TobiasLogic/chessmamba-v2
Reinforcement Learning • Updated • 95 • 3
🔄 In a Training Loop
Mr Munk
GODELEV
AI & ML interests
High schooler by day, LLM builder by night. Driven by a deep love for both Physics and AI. Currently spending my runtime building on Hugging Face, experimenting with transformer architectures, and training custom LLMs.
Recent Activity
liked a model about 23 hours ago
TobiasLogic/chessmamba-v2 new activity about 23 hours ago
GODELEV/Rose-1.5-Medium:yoo liked a model 1 day ago
SupraLabs/Supra2-IMGOrganizations
liked a
model about 23 hours ago
yoo
6
#2 opened 1 day ago
by 
Banaxi-Tech
liked a
model 1 day ago
updated a
model 1 day ago
reacted to TobiasLogic's post with 🔥 2 days ago
Post
3366
We’ve been cooking something new at Bench Labs.
Introducing Cagliostro-v3, our new 146M parameter language model trained completely from scratch.
The run isn’t even finished yet.
At the current checkpoint:
• 146M parameters
• 72.7B / 75B tokens trained
• 26.27 Open SLM Index
• 43.80 ArithMark-3
• Trained on a single RTX 5090
• ~90K to 103K tokens/sec during training
• ~9 days for the full run
• Apache 2.0
For some context, SmolLM2-135M scores 27.13 on the same Index after being trained on roughly 2 trillion tokens.
Cagliostro-v3 is currently at 26.27 with only ~72.7B.
That’s around 27x fewer training tokens.
The model also currently Hold the number 3rd spot for ArithMark-3, scoring 43.80
This wasn’t achieved by just throwing more tokens at the model. A huge part of v3 has been figuring out architecture, data mixture, and training dynamics at this scale.
The model uses a custom 30-layer decoder architecture with grouped-query attention and cross-head subspace attenuation, SwiGLU, RMSNorm, RoPE, tied embeddings, and a warmup-stable-decay training schedule.
During cooldown we also substantially shifted the data mixture toward higher-quality synthetic textbook and mathematics data, with the mathematics share increasing from 10% to 28%.
And everything is open.
The repository contains the training history with checkpoints pushed roughly every 30 minutes, so you can inspect how the model evolved throughout training rather than only seeing the final weights.
This is still a pre-final checkpoint. We have roughly 2.3B tokens left and the learning-rate cooldown is still running.
So 26.27 isn’t the final number.
Really excited to see where the last part of the run lands.
Cagliostro-v3:
bench-labs/cagliostro-v3
Built by Bench Labs.
Open SLM Leaderboard:
AxiomicLabs/Open_SLM_Leaderboard
Introducing Cagliostro-v3, our new 146M parameter language model trained completely from scratch.
The run isn’t even finished yet.
At the current checkpoint:
• 146M parameters
• 72.7B / 75B tokens trained
• 26.27 Open SLM Index
• 43.80 ArithMark-3
• Trained on a single RTX 5090
• ~90K to 103K tokens/sec during training
• ~9 days for the full run
• Apache 2.0
For some context, SmolLM2-135M scores 27.13 on the same Index after being trained on roughly 2 trillion tokens.
Cagliostro-v3 is currently at 26.27 with only ~72.7B.
That’s around 27x fewer training tokens.
The model also currently Hold the number 3rd spot for ArithMark-3, scoring 43.80
This wasn’t achieved by just throwing more tokens at the model. A huge part of v3 has been figuring out architecture, data mixture, and training dynamics at this scale.
The model uses a custom 30-layer decoder architecture with grouped-query attention and cross-head subspace attenuation, SwiGLU, RMSNorm, RoPE, tied embeddings, and a warmup-stable-decay training schedule.
During cooldown we also substantially shifted the data mixture toward higher-quality synthetic textbook and mathematics data, with the mathematics share increasing from 10% to 28%.
And everything is open.
The repository contains the training history with checkpoints pushed roughly every 30 minutes, so you can inspect how the model evolved throughout training rather than only seeing the final weights.
This is still a pre-final checkpoint. We have roughly 2.3B tokens left and the learning-rate cooldown is still running.
So 26.27 isn’t the final number.
Really excited to see where the last part of the run lands.
Cagliostro-v3:
bench-labs/cagliostro-v3
Built by Bench Labs.
Open SLM Leaderboard:
AxiomicLabs/Open_SLM_Leaderboard
- 8 replies
reacted to Datdanboi25's post with 🔥 3 days ago
Post
3352
THE SLM FRONTIER ADVANCES!
bench-labs/cagliostro-v3 just hit an Intelligence Index of 26.13 on the AxiomicLabs/Open_SLM_Leaderboard a 146M-param model trained completely from scratch on a single consumer GPU. That's 2nd place overall, and as far as I can tell, the most capable SLM trained on consumer hardware to date. Beating SmolLM-135m on 1/8th of the data is just silly levels of efficiency.
Big congrats to the @BenchLabs team and specifically @TobiasLogic !
bench-labs/cagliostro-v3 just hit an Intelligence Index of 26.13 on the AxiomicLabs/Open_SLM_Leaderboard a 146M-param model trained completely from scratch on a single consumer GPU. That's 2nd place overall, and as far as I can tell, the most capable SLM trained on consumer hardware to date. Beating SmolLM-135m on 1/8th of the data is just silly levels of efficiency.
Big congrats to the @BenchLabs team and specifically @TobiasLogic !
- 6 replies
Hats Off !!
❤️ 4
3
#3 opened 3 days ago
by GODELEV
liked a
model 3 days ago
updated a
collection 3 days ago
Bruh this is crazy
❤️ 1
13
#1 opened 12 days ago
by Datdanboi25
reacted to Hoglet-33's post with 🔥 4 days ago
Post
6238
Introducing VOID. A new research branch of basically AI.
VOID — Verification of Objectives, Intentions, and Deception.
We study what lies beneath the surface: objectives, intentions, and the possibility of deception in AI systems.
There isn't much to see yet.
That will change.
Follow us for updates:
@Hoglet-33
void-research
basically-ai
VOID — Verification of Objectives, Intentions, and Deception.
We study what lies beneath the surface: objectives, intentions, and the possibility of deception in AI systems.
There isn't much to see yet.
That will change.
Follow us for updates:
@Hoglet-33
- 7 replies
liked a
Space 7 days ago
reacted to Datdanboi25's post with ❤️ 8 days ago
Post
2964
100 likes on the Open SLM Leaderboard 🎉
176 models, 54 orgs, 5 benchmarks, and a whole community of support!
Thanks to everyone who’s contributed models, reported issues, suggested benchmark improvements, or used the leaderboard to compare and evaluate small language models.
It’s been awesome watching the leaderboard grow into a broader community resource for transparent and reproducible SLM evaluation.
Thank you all, and more to come 🚀
176 models, 54 orgs, 5 benchmarks, and a whole community of support!
Thanks to everyone who’s contributed models, reported issues, suggested benchmark improvements, or used the leaderboard to compare and evaluate small language models.
It’s been awesome watching the leaderboard grow into a broader community resource for transparent and reproducible SLM evaluation.
Thank you all, and more to come 🚀
- 1 reply
liked a
model 9 days ago
liked a
Space 10 days ago
reacted to AtAndDev's post with 🔥 10 days ago
Post
2843
SPECK 2 IS ALREADY OUT: specklabs/Speck2-140M
Pretrained on 4x more tokens than the previous releases (20b vs 5b).
Instruct tuned versions are coming soon.
Very interesting models are coming soon too (hint: super long context).
Thanks for everyone supporting!
Pretrained on 4x more tokens than the previous releases (20b vs 5b).
Instruct tuned versions are coming soon.
Very interesting models are coming soon too (hint: super long context).
Thanks for everyone supporting!
- 3 replies
liked 2
models 12 days ago
reacted to HannesVonEssen's post with 🔥 12 days ago
Post
4902
📣 HF Viewer now has a HF space! 🤗
embedl/hfviewer
Visualize any model directly on Hugging Face - now 4,727 graphs!
If you like it, feel free to give the space a heart to help it grow! ❤️
And you can reply with any feedback or feature requests here!
embedl/hfviewer
Visualize any model directly on Hugging Face - now 4,727 graphs!
If you like it, feel free to give the space a heart to help it grow! ❤️
And you can reply with any feedback or feature requests here!
liked a
model 12 days ago