Hacker Newsnew | past | comments | ask | show | jobs | submit | try-working's commentslogin

Could you build and train a V4.1 the same way?

Surely that can be tried! of course at a small scale if you're short on GPUs. what I understand is that V4 replaces MLA with a hybrid of two compressed attention types and changes the residual connections, while keeping the mixture-of-experts design, but I haven't gone in-depth architecture :)

Bitter lesson is wrong. Large general models are outperformed by small specialized models.

Which is the way it is in life. You need a specialist. when you need to see a doctor, an engineer, a builder, a lawyer, etc, etc the world of AI will be the same. You need a specialist, a model that can focus on exactly what you need and not be cluttered up with what you don’t need.

That might mean that spending all that money on that giant master of none model might lose out to the much smaller specialist model, uh-oh…


The Bitter Lesson makes no distinction between large general models and small specialized models, assuming they are both trained from data.

The Bitter Lesson is that human-written rule models will always lose out to trained models in the end.


we will soon figure out how to update the engrams with new information


Yes, but first we need to figure out how to collect that new information. An efficient and ruthless data harvesting in one of the LLM labs secrets.


A distributed, open scraper specifically for LLMs would be interesting . Filtering for spam and useless data would be a challenge at scale, but would be cool


"Software is solved. Bugs are not yet solved"


I think I'll use the DeepSeek Search API instead and "google it" that way instead


DeepSeek.


this would be nice with an eSim and an agent application so I can control DSH on my M5U at home. would that be possible?


that's a good point, one search just to get to try it out. I guess I could add that.

What error did you see when you tried to log in? I'm not sure if it's displayed clearly enough, but the callback will always fail in the pop up and you'll have to copy the callback URL into the text field in the first modal, and let the worker acquire the credential.


same. bought the annual plan, second largest one, in around February to lock in the price. well, what did they do, cut quota by 80% when K2.7 launched and then even further when K3 and K3.1 launched.


Sales tax in China is attributed to the place of production rather than the place of sale, which is why provinces want to produce as much as possible


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: