It feels like the last mile is so long in this space. I'm a SWE with a ton of experience and I still haven't gotten over the decision paralysis of figuring out which model to run locally, and which inference platform should I use, and will my M1 Macbook Pro actually run anything well enough to be useful, and on and on.
It's kind of embarrassing to admit, but having something Just Work(tm) makes getting started so much easier.
It’s a huge new subject area and it’s completely exhausting to learn. I still feel like a total novice.
I have an M1 Max (more memory bandwidth) so I am not sure how much advice I can give about models, but I would suggest maybe trying the new LFM2.5-24B-A2B. It’s surprisingly good. If you want to mess about with multimodal work, Gemma 4 12B is really impressive. (I mostly use Muse Glimmer but it might be rather slow on an M1 Pro)
If you want to have the space to learn the terminology, tinker with what you can control etc., then you have three easy options: LM Studio, Unsloth Desktop or the llama.cpp native Mac app. But don’t get put off by the llama.cpp command line; most of these tools are running it behind the scenes, and most entries on Hugging Face have recommended parameters.
MLX is an option but so far I haven’t found it to have an edge over simple GGUF on Metal on the M1 at least; it seems it might have more advantages on the later M series chips, but it is usually slower for me.
I have explored running OpenCode inside a VM (you can “remote into it” from the OpenCode GUi desktop app). You might want to look at Paseo, which can do a similar thing but with many more coding agents.
I am quite intrigued by Osaurus; on macOS 26 and up it looks like it will manage a container for you.
You might also want to look at Orchard. Not done this myself yet but I can see the appeal.
I am an experienced web dev in my fifties. Six months ago everything I have written above would have been total gibberish to me. Learning some of the open weights stuff has made me feel much less helpless than I think I might if I just developed a Claude dependency. Nothing matters more to me than grasping the fundamentals.
I still don’t really use LLMs for hands-off coding. But I have found that a local LLM can be a useful sidekick; even at the most basic level it’s a better Google that remembers the context of queries.
Thanks for taking the time to write all this up, I'm checking out the projects you mentioned now.
I feel you on the tinkering-to-feel-less-overwhelmed front. I've been through a few ground shifts in my career so far, and even a little bit of "Oh, so that's how it works, ok..." can make it feel so much less daunting.
Mostly I think my own approach has been to try to combat my anxiety and exasperation about all of this by taking this measure of control.
While no really small local model comes super-close to a frontier model, the shape of the technology is settling and I think one can learn so much by approaching it the conventional nerdy way: what's a practical setup I can run on kit I already have? What can I build that helps me explore the problems?
I am quite cynical about AI still, including for generative coding, but I am not cynical about what I can run locally. There is no doubt that they are already helpful to me even in a minimal form.
And the really cool thing about being able to run them locally is to understand the tradeoffs.
> But by virtue of how ALPR technology works, Alaniz and other officers using these systems have access to a continuously updated location history for all vehicles caught on ALPR cameras within the network. This is a type of indiscriminate mass surveillance. It is not targeted on a single individual, as
in Carpenter. It is a tool that collects information about all vehicles that pass
by any network-connected camera at all times, and it serves up the information to law enforcement on demand.
For those saying that the headline is incorrect, I disagree. I believe that the judge's ruling was specifically that Flock's mass surveillance network is, in itself, a violation of the fourth amendment.
The dragnet nature of the data collection was given as the basis for why the officer's use of the data required a warrant or probable cause, but was not given as the thing in violation itself. The document clearly says that the problem was that the police officer searched the data without a warrant or probable cause, not that the data was collected without a warrant or probable cause.
Here's from the very final conclusion:
> "Alaniz conducted that search ... because he saw her license plate was from California. That search was not supported by probable cause, and it was done without a warrant in violation of Kyle’s Fourth Amendment rights."
And here's from earlier in the document:
> "In the Court’s view, Alaniz did not have enough information to reasonably
believe that Kyle was engaged in unlawful conduct"
The court wouldn't be talking that way if the problem were the data collection and not the circumstance of the stop.
The English language is messy as hell, and yet, here we are using it because it's a shared set of rules and symbols that pretty much everyone who wants to join in on this conversation already knows.
Same with staff notation: If you want to communicate music to other people, learning the slightly clunky but universally-known written language is generally the way to go.
The choir director at my church, Michael Dodds (UNCSA), covers the early development of musical notation, "From Modes to Keys in Early Modern Music Theory" [0]. Currently on sabbatical at Cambridge. Once in a while during rehearsal he will give an impromptu lecture on music history, always interesting.
No, English language is easy as sh... as piece of cake. A lot of it is marginalized French tbh, sorry, no offense, and the English in the USA is further simplified. You think tis difficult, because you never tried Russian, French, or Mandarin - then we discuss again. Had it not for the simplicity of English Language, it wouldn't be so omnipresent. So you are so wrong on this account.
And I really want to see you communicate wobbling bass or fast random notes. Well you can - in Studel.cc - but is impossible in this notation. Same for complex drum sections (as in dnb, yes), and I can give you fair amount of other examples where medieval notation is useless.
I don't really agree with the person replying to my original comment, but I can see where they're coming from with respect to notating something like the pitched snares in Renegade Snares (chosen specifically because it's an Amen break that I don't know how I could write down in standard rhythm notation).
It does seem a bit odd to say "We tested all of these bicycles with the same rider" when one of the test cases is actually a bare high-end frame with no components on it.
It's even weirder given that another of the test cases is essentially "We put parts on the frame (OhMyPi) and it went faster!"
Or https://github.com/FrigadeHQ/yap which is the only one I've seen so far that uses the new/improved speech-to-text API in MacOS 26 that outperforms the Whisper models. I expect most dictation software is going to start supporting the local API pretty soon, it's free real estate.
The resource itself is cool, but I'm so tired of reading generated text. It's exhausting to look at something that's supposed to be giving me info and have to wade through the extra mental load of translating from LLM-ese back to human.
thanks! i'll go back and try to re-do some of the text. do you mean the landing page or the module descriptions specifically? all the content is links to reputable sources so i'm assuming those are fine.
Yeah, just the landing page. Can't speak for other people, but for me it's one of those "first impression" things where if your initial offering is clearly human-written, seeing AI-assisted docs or other materials later isn't a big deal because it's like oh, ok, you put work into this up front.
How is the world so difficult? People live nowadays better than kings of France used to live. In the last hundred years we've been going through an unprecedented wealth creation, not just in the West but accross the world.
It's kind of embarrassing to admit, but having something Just Work(tm) makes getting started so much easier.
reply