I love the application. It became my default driver overnight. But the zig dependency and the odd installation paradigm make it a complete bugger to upgrade.
FYI I'm using debian trixie, fully updated, and feel like installing/updating this should be a no-brainer.
Your point is taken, and I suppose I should have been articulate; but these are all huge models, beyond the ability of most of us common folk to load up on a PC and run.
And to be honest, many of those models are not free to use from their providers.
That is, unless you have the hardware to be a provider yourself.
Do that again, and show me all the 37B and smaller LLMs that have made the scene, and I think you'll see what I mean.
Steady 12 to 14 TPS.
Not blazing, by any means, but fast enough to get meaningful personal work done, with a power consumption of 15w.
reply