Hacker Newsnew | past | comments | ask | show | jobs | submit | hazard's commentslogin

> I expect it to cost somewhere in the neighborhood of an experienced dev because that’s the alternative

But there's two ways this could be true: Frontier models costs rise, or (more likely) experience dev costs fall


It's a pretty old term, to distinguish from e.g. CPU time. This was in common usage even 30 years ago.

Example from 15 years ago: https://stackoverflow.com/questions/7335920/what-specificall...


Pangram check on the article: 94% of this text is AI


I actually think this might be a false positive. I think it got tripped up by the higher than average use of jargon (which I don't mind here because the article itself flowed well and raised good points).

Gpt-zero scores "human", and I've always found it to be a better judge


Pangram check on this comment: 142% of this text is AI


it's really obvious too


A key thing that's driving this is the fact that, currently, Gemini is kind of shit compared to the real frontier models from OpenAI and Anthropic. Add on to that the need for Google to serve an absolute insane number of these summaries and have low latency for each one - this means no real thinking time and a comparatively small models.

So it's partially AI, sure, but it's also because you're getting the summaries from what is an extremely stupid AI compared to the "good stuff" that's out there.


I worked very briefly at Baseten, and I can say that it was a perpetual annoyance (from an engineering perspective) that customers would complain about issues with their models but we couldn't actually see the inputs/outputs. I don't know about the other providers, but at Baseten they literally weren't stored anywhere.


It's a true belief, but that doesn't mean it's valid or correct. Many doomsday cults had true believers, and were able to retcon their beliefs when the predicted day came and went. AI doomers have the luxury of not having committed to a specific date or method (but see https://x.com/archerships/status/2095971206847738362)


In a moment of weakness, I joined a high-IQ group (Triple Nine Society). The whole mailing list was just people having the same political arguments that were on the rest of social media, just with a bigger vocabulary.


Put an airtag into literally every bag you check. Helps you figure out not just whether they made it on the flight, but also if they ended up at some baggage claim on the other end of the airport instead of the one they were supposed to be at


your LLM's temperature is set too high


the words are pure me.

but the temperature? that high heat about is directly from the shitcasting endlessly-complaining right. the right that actively seeks to prevent live and let live, that aggressively is trying to turn everything it can into a battleground. there's not two sides about any of this, it's just endless dark triad victimhood signalling, to rile people up. from the worst. some of that is on the left too, but at least they represent actual victims, and aren't actively trying to promote vice as the victim. https://gwern.net/doc/psychology/personality/psychopathy/202...

the monsters need to turn down the temperature and stop being fucking pathetic loser babies. grow the fuck up, get on with it, you shits. that obviously isn't working so society honestly deserves more active defenses: we need ways to shut down the monsters who can cause stinks, lay shit, flood the zone, make battles out of anything, especially those situations in which yes, some people actually did finally stop being actively systematically hurt, and no, that's not actually a problem, you absolute wankers.


I recommend reading some foundational materials on federated learning and distributed inference. The fundamental issues are (1) compute (2) latency and (3) security.

On the compute side, you can't load most individual experts into even high-end consumer-grade cards (e.g. RTX 5090) because the weights are simply too big.

On the latency side, you need to ship the activations and wait for the somewhat underpowered cards to actually do the matrix multiplications, then send the results back. This means each token takes hundreds of milliseconds or even more, which is borderline unusable.

Finally if you're running a P2P network, the only way to verify that peers aren't lying to you is to run duplicate calculations - I don't see how SHA-256 or signed model state help here unless I'm missing something (What are you calculating the hash of? What exactly is signing attesting to?).

This is one of the few places where borrowing ideas from cryptocurrencies actually makes sense - you could imagine a system where peers place bonds and forfeit them if they lie about calculations. You can look at projects like Bittensor and others for how these kind of things are currently implemented.


Thanks for the comment. Extremely clear and detailed. To write this project, I researched a lot about methods and how peers should work compared to a server, as well as security concepts, which certainly have more to add. The goal for this type of project was to move from a single-machine Colibri to multiple-machine Lumabri on a LAN to a large number of machines working together in a Napster-style P2P Lumabri. For latency and other issues, we are studying every type of method that can improve it, and we are also writing and testing other things on our test server. Thank you very much, we will continue to improve.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: