🪶 HobbyLM Playground
A from-scratch 500M sparse Mixture-of-Experts model family (+ a 333M image DiT), trained on a hobby budget. Chat with any variant, ask questions about an image with the multimodal Omni model, or generate a 1024px image. Models: rootxhacker on Hugging Face · code: GitHub.
These are tiny research models — fluent and fun, with the capability ceiling of a 500M model.
| Message | Model | Max new tokens | Temperature (0 = greedy) |
|---|
Upload an image and ask the HobbyLM-Omni multimodal model about it.
Text-to-image with HobbyLM-Image (1024px DiT in DC-AE latent space). Strongest on single objects and cinematic scenes.
| Prompt | Negative prompt | Steps | Guidance (CFG) | Seed |
|---|
HobbyLM is a sparse Mixture-of-Experts: each MoE layer holds 36 little expert networks, but a router sends every token through only its top-6 (plus 1 always-on shared expert). So a 500M model does the compute of a much smaller one per token. Type some text and watch the router decide — which experts each token uses, and how the load spreads across all 36.
A sparse autoencoder (SAE) trained on HobbyLM-Base's layer-8 residual stream pulls apart each activation into a handful of interpretable features from a 12,288-entry dictionary. Type text and see which concepts light up on each token — words, synonym clusters, syntax, formatting. This is mechanistic interpretability: looking at what the model actually represents inside.