Surely that can be tried! of course at a small scale if you're short on GPUs. what I understand is that V4 replaces MLA with a hybrid of two compressed attention types and changes the residual connections, while keeping the mixture-of-experts design, but I haven't gone in-depth architecture :)
Which is the way it is in life. You need a specialist. when you need to see a doctor, an engineer, a builder, a lawyer, etc, etc the world of AI will be the same. You need a specialist, a model that can focus on exactly what you need and not be cluttered up with what you don’t need.
That might mean that spending all that money on that giant master of none model might lose out to the much smaller specialist model, uh-oh…
A distributed, open scraper specifically for LLMs would be interesting . Filtering for spam and useless data would be a challenge at scale, but would be cool
that's a good point, one search just to get to try it out. I guess I could add that.
What error did you see when you tried to log in? I'm not sure if it's displayed clearly enough, but the callback will always fail in the pop up and you'll have to copy the callback URL into the text field in the first modal, and let the worker acquire the credential.
same. bought the annual plan, second largest one, in around February to lock in the price. well, what did they do, cut quota by 80% when K2.7 launched and then even further when K3 and K3.1 launched.
reply