Hacker Newsnew | past | comments | ask | show | jobs | submit | zozbot234's commentslogin

> So what is happening is that models are broadening difficult-to-reason-to skills on benchmarks. ... If I'm being impolite I'd say the frontier models are trying to make sure they'll pass the "gotcha" interview questions for many jobs.

These "gotcha" skills will be needed for frontier research/scholarly tasks. It makes sense to focus on these now that traditional domains like coding or frontend design are broadly saturated (i.e. Astra- or Fable-level outcomes) even by lower-tier open models.


Real "gotcha" skills, yes (perhaps, they're gotchas because the underlying theory is not well understood. More can be achieved, certainly for superintelligence, by forcing understanding of the underlying theory rather than the gotcha). AND the incentives are totally misaligned.

"Gotcha skills", if we're going to call it that, are so popular because the AI labs are buying "tasks" to train on at huge prices, but mostly if they're gotchas.

The problem with that is that a collection of facts in the positivist sciences (and math) are always incomplete and almost preclude a full understanding of math, because it takes away the reason (to an extent) for full understanding.


Ironically, local models are much more efficient when running multiple parallel agents, provided that KV storage requirements are not excessive (not an issue with DeepSeek 4.1 Flash). It does not make any individual agent faster, but it does raise overall throughput quite a bit. The limit is then set by compute constraints rather than memory bandwidth, so something like DGX Spark or AMD Strix/Gorgon Halo can be expected to do better than the Apple-based alternative.

43 tokens is a tiny input context when compared with anything agentic. Prefill on SSD streaming is proportionally much faster with larger contexts, because expert loads can be reused across tokens and the engine can also opt to preload entire model layers into memory, without waiting for the outcome of expert routing.

AIUI more recent estimates have about half of the total training compute, or more, being used for post-training/fine-tuning. Post training a base model (even for simple chat, let alone thinking/reasoning or performance on verifiable tasks) requires reinforcement learning, which is quite hard.

Ablated models are not any more inherently knowledgeable about "forbidden" topics, either. Since such topics are not generally trained for, you just get weird creative hallucinations.

This involved drive-through service, just like the 1994 McDonalds coffee case. Perhaps businesses should just refuse to serve hot coffee via such a drive through, since it stands to reason that people who happen to be sitting in a car will be quite likely to spill their drink in a dangerous way.

or they could just not serve coffee that's so hot that it melts your flesh

> A lot more compute a lot more vram bandwidth.

But this gives you significantly more memory capacity for a compute/vram bandwidth tradeoff (i.e. arithmetic intensity) that's ultimately comparable to Blackwell, perhaps even more skewed towards compute. This seems good if you care about being able to store larger sparse models and more KV caches because that ultimately bottlenecks your ability to do inference.


> I’m not sure how much effect you get from higher TOPS with such low bandwidth though.

Raised arithmetic intensity. It would be quite good for LLM prefill, and decode would want to adapt by pushing parallelism higher (saving on memory bandwidth by reusing the same layer weights for multiple streams) or, if unable to do so (e.g. due to KV cache constraints) by resorting to speculative decoding (where the expensive 'validate' step handles tokens in parallel within a single stream). It does not make the platform inherently unviable. It would also do well at other tasks such as image/video generation, which is already quite compute bound.


With common use of n-gram tables, comparing "params" count has gotten harder. Those are exceptionally sparse and light on compute, way beyond even the sparsest MoE experts. You could host them on a slow spinning disk and still have a viable local inference platform.

So that's why I can't buy a replacement 16TB HDD for less than a 1000$ when three years ago I bought them for 229$ a pop.

I wondered why do they even need slow HDDs, like, do they store training data on them? User chats? Do they assemble crazy 12-way stripes for fast speed? Do they store what, failed attempts?


I think that’s just because hoovering up the entire worlds worth of data and saving it to a RAID multiplied by every frontier lab is a lot of fucking hard drives.

I bought one used a couple months ago for $350 on serverpartdeals.

> whether or not Ant/OAI can stay ahead of the Chinese models.

Ant/OAI/Google are way ahead when it comes to the smartest, largest-scale and most compute-intensive models. By Jevons' paradox, if you make inference for leaner models cheaper (as the Chinese models do, being broadly comparable to GPT Terra or Claude Sonnet; barely approaching Opus/Sol at their very best) that only increases demand for the complementary service of very smart inference.


Hard disagree. I own my gb200 cluster and have unlimited supply of open weight tokens. Whatever is missing from the open weights, they can make it up from speed and multi agent swarm.

Yeah: quantity is a quality all of its own. DS v4.1 Flash running at ~300tk/s makes its verbosity less of a downside, and the efficiencies the Chinese open model space has aimed for makes high concurrency easy for swarms/sub agent sets.

On the other side, I am still shocked at how good Qwen 3.8 Flash Next is on my GB10 Spark-like, even if it is only 45 tk/s decode!


Sure but routing simpler things to the open models and harder things to opus/astra gave us already a massive boost; rarely we run out of tokens on the subscription models and we recently cancelled a few seats on claude code / codex as we simply were underutilising. And also, we have users of our products ONLY using the cheap models as they benefit enough vs paying more. That was very different only months ago. I can see this gap just growing where the % of higher intelligent model use will get smaller and smaller.

> Sure but routing simpler things to the open models and harder things to opus/astra gave us already a massive boost

You have to look at the effect in the aggregate, not just what any single actor is doing. This is the core of Jevons' paradox: it's based on surfacing additional demand for the same product (here, expensive model inference) that wasn't previously viable.


> By Jevons' paradox

Jevons Paradox is not a rule its just a sometimes observed phenomenon

It says nothing one way or the other whether that would happen or not


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: