Coincidentally I've been playing with small (~30B) self-hostable models for coding tasks today -- specifically plugging them into Cloudflare OS (which I work on) and asking each to build a collaborative whiteboard.
I'm finding that the Mixture-of-Experts (MoE) models (Qwen 3.6-35B, and Nemotron 3.5 Lightning) are, well, terrible at this. They just couldn't get the job done at all. Went way off the rails. They are really fast though!
Whereas ~30B dense models (not MoE) are pretty decent. I tried Muse Glimmer, Gemma 4-31B, Qwen 3.6-27B, and Laguna XS[0]. They were all able to build a working collaborative whiteboard app, without any guidance (other than feeding back error logs to the model). I also asked each to then draw a monkey by calling the API of the whiteboard it has just built. Laguna drew random scribbles but the rest all managed to produce something monkey-like.
(Frontier models in comparison will write the app in one shot with no errors at all.)
Note that both Qwen 3.6 and Gemma 4 each have both MoE and dense variants. I find this very confusing, because e.g. ollama's model index typically only distinguishes variants by their size, but MoE vs. dense makes a huge difference in how they actually perform. IMO they should use a suffix, like Qwen 3.6-moe vs. Qwen 3.6-dense, or maybe Qwen 3.6-fast vs. Qwen 3.6-smart...
[0] EDIT: Turns out Laguna XS is MoE, I misunderstood. It performed similarly to the dense models. But maybe this explains why it couldn't write code and think about monkey shapes at the same time!
How they have any credibility after they told everyone they could run Deepseek R1 on their laptop (by giving the Qwen 7B distill the `deepseek` moniker)...
As a replacement for ollama, LM Studio is probably the most appropriate. Though I haven't tried Unsloth Studio yet. You get more control using llama.cpp directly, but there's more of a learning curve.
vLLM has the best performance if you can fit your entire model into VRAM. llama.cpp is the usual go-to if you're partially loading into RAM. LM Studio is a sensible front-end to llama.cpp. ik_llama.cpp has more advanced CPU quantization strategies than llama.cpp. If you're running super large models mostly from RAM, ktransformers can sometimes be the highest performer, if it works for your model.
There is an old rule of thumb that says the quality of an MoE is equivalent to a dense model with the geometric mean of its total and active parameters. So, for example, the Qwen3.6 would be equivalent to a dense model with approximately sqrt(35×3) ≈ 10.25B parameters.
Both MoEs and dense models are always getting better, so I don't think this comparison is meaningful across generations. But still for a first approximation, this tends to hold (you wouldn't expect a lot from a 10B model in coding yet).
I've been pretty impressed with Laguna. I downloaded their coding agent and have used it for a task here and there (the larger variant). Obviously it's nothing like a frontier LLM, but it surprised me with how good it was. And I think the model personality and way it talks is pretty pleasant
Only if you know what it means. To many people these suffixes are gibberish. Also ollama doesn't actually show that suffix in its model index. Don't say "don't use ollama", you sound like a Linux user telling people not to use Mac. (I say this as a Linux user who despises Apple products myself.)
As an aside, the bigger S 2.1 runs faster than the dense Qwen 3.6 and Gemma 4 models on the same hardware (assuming the same hardware is big enough to run it) and definitely feels smarter, and more capable of long tasks, but it doesn't feel as heavily tuned for code as Qwen 3.6.
If you haven't yet, you might want to try gemma-4-E4B-it-qat or gemma-4-12B-it-qat with Structured Outputs. My main use case is tagging photos and generating headlines.
I'm finding that the Mixture-of-Experts (MoE) models (Qwen 3.6-35B, and Nemotron 3.5 Lightning) are, well, terrible at this. They just couldn't get the job done at all. Went way off the rails. They are really fast though!
Whereas ~30B dense models (not MoE) are pretty decent. I tried Muse Glimmer, Gemma 4-31B, Qwen 3.6-27B, and Laguna XS[0]. They were all able to build a working collaborative whiteboard app, without any guidance (other than feeding back error logs to the model). I also asked each to then draw a monkey by calling the API of the whiteboard it has just built. Laguna drew random scribbles but the rest all managed to produce something monkey-like.
(Frontier models in comparison will write the app in one shot with no errors at all.)
Note that both Qwen 3.6 and Gemma 4 each have both MoE and dense variants. I find this very confusing, because e.g. ollama's model index typically only distinguishes variants by their size, but MoE vs. dense makes a huge difference in how they actually perform. IMO they should use a suffix, like Qwen 3.6-moe vs. Qwen 3.6-dense, or maybe Qwen 3.6-fast vs. Qwen 3.6-smart...
[0] EDIT: Turns out Laguna XS is MoE, I misunderstood. It performed similarly to the dense models. But maybe this explains why it couldn't write code and think about monkey shapes at the same time!