It seems like everyone here knows what this is but me. I went to their homepage and I still can't tell what this is. How is this different from just giving your agent access to one of the many hosting services CLIs / MCPs? Did they just pioneer that?
val.town was built mostly pre-LLMs and it was basically lightweight typescript functions as a service. i thought they were doing ok, so i'm a bit surprised by this, but i do admit that I only ever used them for some throwaway toys
> No. The phone app talks to herdr-core, the runtime the Herdr app is built on. Herdr TUI runs its own server, so agents started there don't show up on your phone. Both can run on the same machine.
I was struggling with trying to get the herdr desktop app to connect to a TUI session on another machine, and gave up. This doesn't really make sense as a design decision to me. I hope they port the TUI to the same framework as the rest of the herdr apps.
I recommend it. I have been what I call "phone-lite" for a few years now. It lives in my sock drawer, and I only take it out if I know I'll need it for GPS when traveling. For the most part, my mobile computing is entirely done through my Daylight Computer tablet, which as a wifi-only e-paper device, I find to be minimally attention sucking.
It blows my mind how people still build their workflows around depending on services like these, with how easy it is to just self host and forget about all of this. You can buy a GPU that can run Qwen 3.8 flash next for about 9 months worth of the highest tier Claude Code subscription. You'll never have to worry about token limits, the model changing out from under you, getting banned, etc.
This really depends on your infrastructure. In this case it all depends on your hardware, and there is no secret sauce, you can buy the same GPUs or better than the big players are using. Self-hostef GLM in our company is faster than any commercial model. Of course the hardware was not cheap, but completely manageable for a medium or large company.
Speed can mean two things: rate of token generation or time to finish a task. What maters it the later and a model that takes a faster path to the result and uses fewer tokens because of that comes out ahead in practice.
Superior? Sure. But do people need the capabilities beyond what can be reasonably achieved with local models? I think the people that actually sit in that gap are vanishingly small, and if your title contains "Engineer" or "Developer" or something like that, you are not one of those people (at least for doing your job)
As for faster, maybe? It depends on the metrics and what you're doing with it. If you mean TPS, no, I get much higher TPS and even faster TTFT on my local setup than frontier models. If you mean time to complete tasks... maybe. If you're throwing whole projects at it to vibecode, yeah, a frontier model will dump slop so fast it'll make your head spin. If you're doing actual methodical development, like doing spec-driven development and actually reading the specs before spinning off agents to implement them, rather than just having another agent review it and yolo, you'll find that frontier models don't really push the needle that much, and you'll be making high-quality projects.
Qwen 3.8 (and maybe soon 4) series models are probably the best right now for local AI. Specifically, for lower end hardware like 8gb GPUs you might find a lot of use in a Qwen 3.8 variant like Underdog Saluki 27B.
Is there a model that doesn't need babysitting? Half my day is spent untangling BS that Claude and Codex hallucinate. Particularly around hallucinated product requirements. Its still faster than hand-writing code but only just. I'm seriously considering just going back to the early copilot days where I lead coding and just let it try to auto-correct. That was a great balance of AI enhancing my speed and workflows without removing any confidence in what was being shipped.
give your models a more organized context/harness if they are hallucinating product requirements... and I'm not talking about some hulking context framework. just organize your documentation in a canonical way. This product requirements hallucination is something that would bite me constantly if...they ever actually hallucinated them (and they don't, ever). This is most certainly user error.
All the time. All the frontier models love to take a requirement and bend it into a need to create an obscene amount of overhead. I needed to add a permissions for a user type. Instead of adding the type to the existing list of permitted types it created a bespoke experience for that user type complete with a full testing suite. This is without diving into the absolute insane amount of no signal tests that these things generate. It once decided the privacy policy should have its own testing suite, which basically amounted to 50 different regex tests looking for "key" words and phrases, and snuck that into a PR for something tangentially related. They love to burn tokens on creating useless shit and it happens on the daily.
We are literally in this thread because so-called frontier SOTA Claude models are shitting the bed and require babysitting. So this argument is essentially moot.
>> But do people need the capabilities beyond what can be reasonably achieved with local models?
Yes? If you're building anything reasonably complex, it is a ton of work to keep everything in your head so that different parts of the system work seamlessly and work you do in one area does not adversely affect another area.
In a workplace environment I don't think "cheaper" is necessarily the thing to optimize for. Subscription models save you a lot of time (and money) by getting things right faster, solving bigger problems, etc. A cheaper model just consumes more human time.
But with a local model with essentially free tokens, I can set it to work on tasks in the background without having to worry about any usage limits. Deferring simple bug fix tickets or other low complexity to an essentially unattended local model works just fine. And if the result occasionally fails review, it still didn't burn paid usage credits.
A local model tops 60 tokens/s max. Running it on the background means not being able to use on the task you are working on. When working on local hardware you cannot do concurrent requests. In deepseek for example you can have 100 agents running each getting at 120 tokens/s easily.
In local models you are limited by your own infrastructure.
I do concurrent requests just fine and they have no noticeable performance impact. Not all local hardware is equal. Also, humans have to sleep. Agents don't, so processing stuff over night and reviewing in the morning just gets more utilization out of what is already there.
With cloud agents I also process stuff overnight. But my gpu is limited. If I want to OCR a high number of documents or vectorize data, I can't run an LLM at the same time. I use remote models and use my compute for running commands on my desktop machine.
Today this is absolutely true. Local models have gotten very good, but they aren't there yet. The cost of a Max subscription or even API pricing is overwhelmingly worth it for most users, even if you ignore the enormous capital costs of running the competitive models locally.
But the in a million years bit, even taken as rhetorical bombast, is of course silly. The difference between local models and the frontier models is ever shrinking. Extrapolating out and eventually, sooner rather than later, the difference is going to be a wash.
Unless all of the vendors stop progressing open local models, the future is clearly there. It's why I think AI is important and valuable, but also the current leaders have a dubious financial path forward.
I sure do. And 2-4 years in the future when this insanity is over I will buy and own local llm hardware. But certainly not now with those prices and speed of progress that obsoletes everything in a year or two.
I made some assumptions and was foolish. Sadly not a new tale. But still in context I do think for every You there are several other Not Yous who are planning infrastructure who aren’t keeping an eye on the local escape hatch.
Compared to Opus local models Opus 5.5 is far superior, also for something like a dgx spark at 5k that would be 25 months ignoring power costs if you chose the $200 plan.
And that’s looking at a large model, to host the equivalent sized model as Opus 5.5 you’d need more than a single dgx spark. A fairer comparison would be to Open models hosted remotely or comparable weak models like Luna which you would get away with a $100 plan or less. That also doesn’t account for token prices continuing to go down for the same level of intelligence.
Qwen 3.8 Flash Next is 125B with 6B active, which with quantizing and system memory offloading (strata) runs nicely on 32 GB cards (I have actually run it on as low as a 16 GB GPU and 16 GB of system ram, but I don't recommend it for serious work). I was running it on a single r9700, but now I'm using 2 r9700s because I'm splitting my server with a friend, and they bought one as well.
On the single r9700 I was able to run q2 with 4 slots of 256k, and I think it was about 40 - 50 GB of system memory offloading? Got about 60 - 80 TPS per slot depending on concurrent use. With the two r9700s we are running q3 with 4 slots of 256k, using about 100 GB of system memory offloading, with mtp enabled. Getting about 70 - 100 TPS now.
i dont understand how people keep comparing local models with frontier models, they dont belong in the same discussion especially for coding usecases. they solve different problems under different constraints.
I don't understand what do you mean. They both solve the same problem. Some models solve it better, some worse. And in some cases you can only use local models, so they solve more problems.
"Frontier" (closed) models are better by definition, but if frontier models were good enough for you half a year ago, open models are good enough now.
What is the alternative? Local models? I do not think so. Maybe Kimi K3 (still not local) as this model can be served by multiple providers? However Kimi K3 sits between Sonnet/ Sol and Astra/Opus so still not the same level.
No. That is just Anthropic allowing US based companies to host it in the US and EU. You can buy a few B300 and run Kimi in your data center as a company. Not possible with frontier models from OpenAI and Anthropic.
20 euro claude sub is too good compared to a 6-10k hardware purchase to get something thats not even as good. Once the enshittification starts on full force it will be a different situation.
I honestly thought usage was going up. Now, I dont use LLMs nearly as much as a lot of people here, but I used to hit my limit very easily, and now I haven't hit it in wmquite some time.
I don't know how you thought that when real usage is down 10x since 2 months ago, they publicly announced announced several usage decreases and related issues, and its a constant point of discussion/criticism online.
i haven’t heard anyone say they downgraded their car to the children’s toy power wheels and regretted it. all the same, i’m confident that it’s a bad idea
Problem is that everyone loves to pretend their CRUD SaaS is special and requires the best of everything, so they 'need' the absolute frontier with as many tok/s as money can buy.
R9700. I walked into Microcenter on Hallow and picked one up for $1900.
I was also using a 7900 XTX before that which was reasonably capable for single user usage, but I share my server with a friend, and the 9700 is a much better fit. Also nice for doing sub agent workflows that I couldn't really do before.
I can buy a GPU that can run Qwen 3.8 Flash Next for 9 months worth of the highest tier Claude Code subscription, but then I'm running Qwen 3.8 Flash Next when I could have had nine months of unlimited Opus 5.5.
> then I'm running Qwen 3.8 Flash Next when I could have had nine months of unlimited Opus 5.5.
By choice, maybe. New models release faster than every nine months. And, evidently, you won't be running nine months of some fixed meaning of "Opus 5.5". What that means changes over time, out of your control, and without ever even being officially told.
Have there been any notable incidents where mistrusting a curl | sh was the cause? I hear people say not to do it way more than I hear people talking about actual incidents it's caused.
More just very badly made install scripts, which (in good faith) destroy some part of my operating system.
Messing up repositories added to package manager, creating dependency conflicts, overwriting config files I used. Even removing full folders - "because it's MINE" syndrome. --dry-run modes that are destructive. A lot of install scripts treat my computer as disposable machine meant for their app only.
MS Storage makes google play store look reasonable.
For Linux package management I am used to decent standards, where packages have maintainers that care about the whole ecosystem. I wish this would scale to the insane amount of software we nowadays have.
Personally wherever OS package manager is lacking, I tend to use Aqua (https://aquaproj.github.io/) as it failed me the least. Some decent work was put into this.
One coverage tracker used something like this for recommended CI setup, script was hijacked, users were advised to rotate all credentials available in CI.
it'll just print an error message because / is a directory. There's also a good chance curl starts first and fails to connect to the server that's not running yet.
If Zulip and Mattermost are in the running (no E2EE), Fluxer should be as well. Their client is a super slick discord clone, and it's very easy to self host. Federation is coming soon as well.
Why focus on that aspect? Like, yes, it needs to be started off by some other entity. That could be a human. That could be another agent.
The point is we're on track to construct such a system that's capable of sustaining itself indefinitely, and the fact that we are means there are entities driving this forward, and those entities often push buttons to start experiments. So, expect that particular button to get pushed at some point.
reply