Hacker Newsnew | past | comments | ask | show | jobs | submit | hellcow's commentslogin

Local models already crossed the threshold where they're quite good and useful for many things, even with cheap setups. I walked away really impressed with (non-coding) use of Byteshape's Qwen3.5-9B quant on my 8GB RTX 3070 for instance. The model was fast, helpful, and 100% private.

If we extrapolate over the next several years, prices for VRAM/RAM will start to come down as manufacturing capacity comes online, models will continue to become more token efficient, and intelligence will improve at every quant size. As this happens, more and more people will be running LLMs locally on hardware they already own, and there's going to be a point where local models can do everything you need to do unless you're literally trying to do breakthrough mathematics research.


I don't think "useful for many things" qualifies as "winning". It's about two orders of magnitude shy of winning.

> and there's going to be a point where local models can do everything you need to do unless you're literally trying to do breakthrough mathematics research

Maybe. But competitive markets pay exponentially more for incrementally superior services. Professional athletes are a great example of this. If you're building a product with competitors and those competitors have frontier models and you stick to local then you're probably going to lose both today and in the near-ish future.

If you're bitter lesson pilled then the alpha is entirely in having a better model, more data, and more compute. Your vision is that human ingenuity will provide more alpha. Oof. I wish. But my tastebuds are very very bitter.


> If you're building a product with competitors and those competitors have frontier models and you stick to local then you're probably going to lose both today and in the near-ish future.

i highly doubt this premise. the success of a product has very little to do with the amount of the intelligence used in producing it. you could throw millions of dollars using AI to develop a movie and it would still flop in comparison to an indie-made film.


I think the premise is confusing what the amount even is in this context.

I want to use "more intelligence" but not in terms of single higher IQ.

I want to use more intelligence in terms of a sum quantity of knowledge work being performed. A team of 10 instead of a team of 5.

This is when a local model will shine. A fixed cost and I don't have to worry about how much I am using it. I already can't tell any difference at all between Opus and Fable. It won't be that long that I won't be able to tell the difference between a local model and Opus.

There will be an infinite demand for "more" intelligence but I think many people are making a category error on the dimension in question.


Also - the “open” here doesn’t have the advantage of open source, where it can be improved by the community. At best, the community can tweak it a little. For actual improvements you need to wait for the next model which is trained by companies for billion of dollars, and could stop releasing them at any moment.

It can, through better quantizations and reinforcement learning.

Just this week, qwen 3.8-27b got two new interesting quantizations, one of which came from community. Llama.cpp also improves continuously, and novel alternatives like Strata pop up.


The point is that these are overall small improvements that the community can’t really build upon. For an illustrative example, combine all the community improvement to the best open weight model released a year ago. It won’t do much to make it competitive or even interesting.

3.8 wasn’t released a year ago. Of course base models change. But the quantization techniques carry over to new models. For instance, one of the improvements last week was someone porting the techniques used for macaq to qwen3.8-27b. It’s pretty satisfying to get a working implementation of a new technique barely one week after the original paper hit arxiv.

Also improvements to llama.cpp stay, and many small improvements stack up nicely.


you don’t even understand the lesson, oofu.

the lesson is that training a model on all input ever output will never be as good as training a model on arbitrary outputtable input.


Wasn't this exactly what happened in the LG exposé that just came out? It connects to any WiFi network it can, your will be damned.


No, no it wasn't.


My Android TV has a non-default launcher with no ads, and I can sideload apps to get things like Youtube with sponsorblock baked in. Going to Apple's locked down system seems like a huge downgrade to me.


Depends on one’s needs. The overwhelming majority of Android TV boxes and integrated Android TVs use extremely anemic hardware that will choke on any content that falls beyond hardware acceleration support for their SoCs. So fine for YouTube and Netflix probably, but not so great for a personal Plex server full of HDR content.


I have isponsorblocktv running on my home server that skips YouTube segments automatically.

Also running side-loaded Stremio with torrenting enabled so I can stream Big Buck Bunny.


Behold: https://fil-c.org/

An important improvement over rust is that "Fil-C has no unsafe statement."


As everything, there are compromises and prices to pay.

In case of fil-c, it is about 1.5-4x slower performance, and a memory overhead.

So, let's not present it as a panacea to all problems: there could good reasons to use it, but it isn't a magic trick.


Rust is slower too, and git is IO bound anyway, and routinely calls bash.


Rust is not 1.5-4x slower at all. Git is not IO bound at all, it is not saturating your IO device, it just performs IO a lot.


Google can shuttle network requests to any of their other apps (or play services, if that's running) via IPC, even with the network permission disabled in Gboard.

So if you're using Graphene for privacy from Google specifically, it's not safe to install things like Gboard even with the network permission disabled. Even if they don't use IPC in this way today, there's no telling what the Google-of-tomorrow will do.


there's no telling what the Google-of-tomorrow will do.

This is true and even vastly understated. Google is like the Walter White of companies. Starts out innocent, turned into a monster, everyone around him astonished at the dark, depraved change.

So if one thinks "nah, Google wouldn't do that", I have a bridge to sell you...


Seems like you could use an escrow to defeat this and sell domain names safely and easily.


Like I said, it's impossible to "defeat" it outright because the concept doesn't really lend itself to this sort of structure. If I buy a plaque with my name on it , therefore owning it, the only thing that could really keep me from selling it would be some centralized authority. Even then, black markets have existed for drugs for centuries.


Yes, all this talk about the language used still leads us to the same frustrating outcome: customer pays for something, and at some unspecified point in the future that same thing is taken away from them. I don't care if 6 years earlier I clicked a button that said Pay vs Buy, I care that my purchase is being removed due to some backroom licensing issue.


What alternatives do people recommend that are secure and easy to use for nontechnical people with auto-backup?


vimium is my solution to the web.


There's raylib and sdl2 eggs available. Those are a great place to start.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: