The issue, I think, is a large number of people don't realize the NIH or NIS exists or know what they do.
I think most people support them once the realize exactly what they are doing. How could you not when "They research treatments for diseases" is basically all they do.
This is obvious Claude slop writing, the author would be advised to use vale [1] with samples of their own writing as a guide.
> Performance begins with the roofline. On the M1 the engine holds about 12 fp16 TFLOP/s of compute against
a DRAM-bandwidth ceiling. The roofline has a ridge point near 141 FLOP per byte, a 2 MB working-set
threshold, a 0.23 ms floor under any single dispatch, and efficiency near 0.37 picojoules per FLOP at the
compute optimum. On a 256-channel 3x3 convolution it runs about 3.8 times faster than the same chip’s
GPU and 9 times more energy-efficient. The roofline pairs the engine’s throughput ceilings with its measured
power.
> Reaching the engine is not the same as running an arbitrary graph on it. The operations the engine executes
are distinct from the ones a capability bit only advertises. A feature attested in the hardware tables or
accepted by the compiler frontend counts only once a compile-and-run confirms it, and several advertised
operations, three-dimensional convolution among them, never lower to the engine at all. Weight compression
on the direct path cuts bandwidth, not only stored size. On the unentitled engine, int4 lookup-table weights
run about 2.37 times faster than fp16, and structured sparsity 1.55 to 1.64 times faster at 0.43 times the
bytes.
1. It uses non-idiomatic terminology in several places.
2. It repeats the same finding over and over (141 flops per byte, for example), without going deeper.
3. I stopped reading about a quarter of the way through because it felt like it was never going to stop teasing me about what it was going to tell me and actually tell me it.
4. It seems to assume the reader has a lot of context that isn't explicitly laid out (and which the reader wouldn't get just from reading the prior work, which is cited).
For example, I understand some of what it is saying because I used some similar techniques to benchmark things in the past (running at multiple scales to estimate overhead + marginal gains with a linear regression), but I wouldn't expect anyone who hasn't personally done that to follow the prose.
> 4. It seems to assume the reader has a lot of context that isn't explicitly laid out (and which the reader wouldn't get just from reading the prior work, which is cited
I've had this complaint well before LLMs were used. People writing about topics they have a lot of knowledge in the subject tend to make the assumption only other subject knowledgeable readers will read it. Or that it never edited by a real editor that would enforce rules like spelling out acronyms on first use. Or forcing additional information when too many details have been left out on the assumption it would already be known.
There's plenty of this type of writing to have trained the bots that way
You obviously haven't read it, because it is clunky garbage.
> 19.4 Pacing compiles after a failure
> A failed compile is not free of side effects on the shared compile service. A compile that fails restarts the
service, which takes a few seconds to come back, and failures that keep arriving faster than the service can
restart between them keep it from making progress, so unrelated compiles slow down until the failures stop.
The effect is a function of how fast failures arrive, not how many occur: failures spaced out past the restart
interval cause no degradation at all. On detecting a failed compile, wait at least one restart interval, roughly
15 seconds, before the next compile, so a burst of failures cannot accumulate. No hard failure-count cap is
needed.
The whole document is less nutritious than a wonderbread miracle whip sandwich.
Personally I'm not in the habit of printing and eating articles I read, but in the unlikely event that I did I find it even less likely that I would be concerned with its' nutritional content. (/s)
Was the choice of such a small model driven by a desire for high tok/sec? I ask because an m4 pro 48gb machine can run larger models (if model intelligence is the thing that would make it more useful).
> He says the problem is that they can't use Claude Code because it's the enemy, and Gemini has never been good enough to capture people's workflows like Claude has, so basically agentic coding just never really took off inside Google. They're all just plodding along, completely oblivious to what's happening out there right now.
This is a bunch of gabagoo. Wrong on so many layers, it's not even worth reading further.
a) goog has agentic coding in both antigravity & cli forms. While it is not at the level of cc + opus, it's still decent.
b) goog has their own versions of models trained on internal code
c) goog has claude in vertex, and most definitely can set it up in secure zones (like they can for their clients) so they'd be able to use claude (at cost) within their own projects.
Antigravity's workflow is so slow and buggy with permissions, it is unuseable compared to cc/codex. the only part that is nice is that it allows usage of Opus.
Gemini CLI is an absolute joke. I dont know if its the harness, or the Gemini models' poor instruction following, but more than 50% of my sessions gemini turns insane and ends up in thought loops. In another 25% it does far more updating than I asked or is reasonable.
This is why next to no-one talks about either of them. Does Antigravity's agent manager even work yet without crashing and showing zero conversations? In typical google fashion they released AG and appear to have then set it on autopilot with a skeleton crew of less than 1 developer. Clear issues have not been fixed since day 1. Some settings just do not work. Permissions are not respected.
I’m not so sure. From talking to some of my own friends at google they feel that antigravity/gemini models are handicapping them and would much rather be using claude code (which only deepmind gets to use)
Google’s businesses are very broad and durable. But Google being the only company in the world without access (except for GDM+labs) to a competent coding agent will take a toll.
We’ll see how long Google can hold out hoping for GDM to create something that is competitive.
I’m guess that within 6 months Google will give up on coding and finally let their devs use Claude/Codex.
This isn’t a security problem, this is a GDM issue with GDM’s promises being far beyond their ability.
> But Google being the only company in the world without access (except for GDM+labs) to a competent coding agent will take a toll.
I doubt it. I use Gemini CLI daily because Gemini is what work pays for, and I have a personal Claude account. The difference is not that great, especially if you're not doing full vibe-coding. It's unlikely to have the kind of effect you're describing.
I look forward to seeing that play out in the market, if true. But from what I've seen, it really isn't.
If you're talking about the ability to churn out low-stakes systems like websites, or variations on existing widely available systems, then perhaps. But once you get to more complex systems, especially large already existing systems, all LLMs today need significant ongoing assistance to prevent them from going off the rails and down rabbit holes. At that point, the advantage you're claiming tends to evaporate.
Is there a page where I could read more? What's unintuitive at a glance is that Opus 4.7 has a lower success rate than Sonnet 4.6 (90% vs 100%) while having a higher Avg Percentile (87.2% vs 70.9%).
We calculate percentiles based on successful submissions only, and then apply success rate as a separate measurement, which is incorporated into our relative rankings.
So we do penalize evals where the player failed the game, but not in the percentile measurement (success rate measures instances of playing incorrectly, did not compile, runtime errors, and other non-infrastructure related issues that can be blamed on the model). The design decision there is that percentile tells you how good the model's ideas are (when executed correctly), separately from how often it got something working correctly, but I can see how that's not great UX, at least as presented now.
But the actual score itself is a combination of percentiles and success rates with some weighting for different categories, nothing fancy.
I added a methodology page to the roadmap, thanks for pointing that out. We've converged on a benchmark methodology that should scale for a very long time, so it's time to document it better.
It's not worse, Anthropic simply has no equivalent model (if you don't consider Mythos) of GPT 5.4 Pro. Google does though: Gemini 3.1 Deep Think.
GPT 5.4 Pro is extremely slow but thorough, so it's not meant for the usual agentic work, rather for research or solving hard bugs/math problems when you provide it all the context.
I'm genuinely asking, when you say Gemini 3.1 DT is an equivalent model of GPT 5.4 Pro, is there a specific benchmark/comparison you're referring to or is this more anecdotal?
And do you mean to say that you don't really use GPT 5.4 Pro unless it's for a hard bug? Curious which models you use for system design/architecture/planning vs execution of a plan/design.
TIA! I'm still trying to figure out an optimal system for leveraging all of the LLMs available to us as I've just been throwing 100% of my work at Claude Code in recent months but would like to branch out.
So, reading the tea leaves, they're either losing subscribers for the $200 plan, or they're not following the same hockey stick path of growth they thought they were... maybe?
Edit: I wonder if this is actually compute-bound as the impetus
Nope, it's just that a lot of people (especially those using Codex) asked us for a medium-sized $100 plan. $20 felt too restrictive and $200 felt like a big jump.
Pricing strategy is always a bit of an art, without a perfect optimum for everyone:
- pay-per-token makes every query feel stressful
- a single plan overcharges light users and annoyingly blocks heavy users
- a zillion plans are confusing / annoying to navigate and change
This change mostly just adds a medium-sized plan for people doing medium-sized amounts of work. People were asking for this, and we're happy to deliver.
Did you modify the Plus plans usage recently or as part of this introduction? Given that Pro plans usage are multiples of it (5x/20x) and given reports of less Plus usage, clarification would be appreciated?
Transparency on this sort of thing is the best way to address negative company sentiment.
I'm honestly not sure, as I don't work on it. My understanding from afar is:
- There was a 2x promotion in March that ended on April 2, so limits have felt tighter since then
- We sometimes reset rate limits after bugs or milestones or because Tibo feels generous, which can make some days feel different than others (they are typically announced here: https://x.com/thsottiaux)
- Recently Plus was tweaked to have a smaller 5h limit but an increased weekly limit
- Lastly, as part of the new Pro launch, the $100 & $200 Pro tiers are getting a 2x promotion, meaning they are temporarily 10x/40x instead of 5x/20x
I've asked our team to clarify the pricing page. Agree it's not clear.
All good, I interpreted it as postulation and not accusation. :)
I do like the job! Much more organic than yanking tickets, though I'm on the model training side of things, rather than product side. Always a balance between short-term sprints patching bad behaviors for the next model vs long-term investments in infra and science that make future work easier. Sometimes the negative press gets to me a bit (it's a very different feeling than 2022 or 2023), but my goal is just to make the most useful product I can for people. It's been wild how much Codex has already changed my day-to-day work, I'm so curious to see what it looks like in 2030 or 2040.
What kind of bad behaviors? How is the whole SDLC lifecycle there? I imagine, given that this tech is kind of redefining how software is being written, it's not your standard workflow pipeline? Are there code reviews at all? Have you been in any particularly interesting meetings about how you're trying to "shape" the models?
I won't misrepresent myself, I've never spent a penny on any of these services. I am just super curious what it's like to work at one of these frontrunner companies. I bet it's pretty neat.