Hacker Newsnew | past | comments | ask | show | jobs | submit | carbocation's commentslogin

I'm very confused about why they are showing an AUC for the relationship of a continuous score with a continuous phenotype.



I don't read CT anymore but it was one of the great ones.


That was it.


The public overwhelmingly supports NIH-funded research. https://www.aacr.org/about-the-aacr/newsroom/news-releases/n...


The issue, I think, is a large number of people don't realize the NIH or NIS exists or know what they do.

I think most people support them once the realize exactly what they are doing. How could you not when "They research treatments for diseases" is basically all they do.


That is essentially how it works in the US as well thanks to public service loan forgiveness for physicians.


This scans very much as AI-written.


This is obvious Claude slop writing, the author would be advised to use vale [1] with samples of their own writing as a guide.

> Performance begins with the roofline. On the M1 the engine holds about 12 fp16 TFLOP/s of compute against a DRAM-bandwidth ceiling. The roofline has a ridge point near 141 FLOP per byte, a 2 MB working-set threshold, a 0.23 ms floor under any single dispatch, and efficiency near 0.37 picojoules per FLOP at the compute optimum. On a 256-channel 3x3 convolution it runs about 3.8 times faster than the same chip’s GPU and 9 times more energy-efficient. The roofline pairs the engine’s throughput ceilings with its measured power.

> Reaching the engine is not the same as running an arbitrary graph on it. The operations the engine executes are distinct from the ones a capability bit only advertises. A feature attested in the hardware tables or accepted by the compiler frontend counts only once a compile-and-run confirms it, and several advertised operations, three-dimensional convolution among them, never lower to the engine at all. Weight compression on the direct path cuts bandwidth, not only stored size. On the unentitled engine, int4 lookup-table weights run about 2.37 times faster than fp16, and structured sparsity 1.55 to 1.64 times faster at 0.43 times the bytes.

https://vale.sh/


Please no. The author would be advised to write their own original thoughts.


It was a joke, nothing could save this "paper". I don't think the author wrote anything. They pointed claude at a directory and said "write a paper"


Vale docs' opening line is: Learn about what Vale is (and isn't).


why?


It has many technical mistakes besides the odd writing style


1. It uses non-idiomatic terminology in several places.

2. It repeats the same finding over and over (141 flops per byte, for example), without going deeper.

3. I stopped reading about a quarter of the way through because it felt like it was never going to stop teasing me about what it was going to tell me and actually tell me it.

4. It seems to assume the reader has a lot of context that isn't explicitly laid out (and which the reader wouldn't get just from reading the prior work, which is cited).

For example, I understand some of what it is saying because I used some similar techniques to benchmark things in the past (running at multiple scales to estimate overhead + marginal gains with a linear regression), but I wouldn't expect anyone who hasn't personally done that to follow the prose.


> 4. It seems to assume the reader has a lot of context that isn't explicitly laid out (and which the reader wouldn't get just from reading the prior work, which is cited

I've had this complaint well before LLMs were used. People writing about topics they have a lot of knowledge in the subject tend to make the assumption only other subject knowledgeable readers will read it. Or that it never edited by a real editor that would enforce rules like spelling out acronyms on first use. Or forcing additional information when too many details have been left out on the assumption it would already be known.

There's plenty of this type of writing to have trained the bots that way


Cmd-F for "AI" has 1000+ hits!


The burden of proof should be with the beholder. Must be so easy to scream AI when you don’t want to read an article.


You obviously haven't read it, because it is clunky garbage.

> 19.4 Pacing compiles after a failure

> A failed compile is not free of side effects on the shared compile service. A compile that fails restarts the service, which takes a few seconds to come back, and failures that keep arriving faster than the service can restart between them keep it from making progress, so unrelated compiles slow down until the failures stop. The effect is a function of how fast failures arrive, not how many occur: failures spaced out past the restart interval cause no degradation at all. On detecting a failed compile, wait at least one restart interval, roughly 15 seconds, before the next compile, so a burst of failures cannot accumulate. No hard failure-count cap is needed.

The whole document is less nutritious than a wonderbread miracle whip sandwich.


you forgot the bologna and iceberg lettuce


Personally I'm not in the habit of printing and eating articles I read, but in the unlikely event that I did I find it even less likely that I would be concerned with its' nutritional content. (/s)


Was the choice of such a small model driven by a desire for high tok/sec? I ask because an m4 pro 48gb machine can run larger models (if model intelligence is the thing that would make it more useful).


Yes that was my goal. Also noticed a huge performance gain going from ollama to mlx. Your mileage may vary.


Last month, Steve Yegge suggested that they are not: https://xcancel.com/Steve_Yegge/status/2043747998740689171


> He says the problem is that they can't use Claude Code because it's the enemy, and Gemini has never been good enough to capture people's workflows like Claude has, so basically agentic coding just never really took off inside Google. They're all just plodding along, completely oblivious to what's happening out there right now.

This is a bunch of gabagoo. Wrong on so many layers, it's not even worth reading further.

a) goog has agentic coding in both antigravity & cli forms. While it is not at the level of cc + opus, it's still decent.

b) goog has their own versions of models trained on internal code

c) goog has claude in vertex, and most definitely can set it up in secure zones (like they can for their clients) so they'd be able to use claude (at cost) within their own projects.


Antigravity's workflow is so slow and buggy with permissions, it is unuseable compared to cc/codex. the only part that is nice is that it allows usage of Opus.

Gemini CLI is an absolute joke. I dont know if its the harness, or the Gemini models' poor instruction following, but more than 50% of my sessions gemini turns insane and ends up in thought loops. In another 25% it does far more updating than I asked or is reasonable.

This is why next to no-one talks about either of them. Does Antigravity's agent manager even work yet without crashing and showing zero conversations? In typical google fashion they released AG and appear to have then set it on autopilot with a skeleton crew of less than 1 developer. Clear issues have not been fixed since day 1. Some settings just do not work. Permissions are not respected.


Agreed, however imo there is def some problems unique to Google which is making the internal experience less than ideal.

Hoping they can figure it out sooner rather than later.


Demis Hassabis chimed in on that thread and called it what it is: clickbait.


I’m not so sure. From talking to some of my own friends at google they feel that antigravity/gemini models are handicapping them and would much rather be using claude code (which only deepmind gets to use)


Sure, but there's cavernous distance between "google = john deere" and "darn I have to use Gemini"


He was entirely correct.

He made a follow up after the pushback by GDM.

Google’s businesses are very broad and durable. But Google being the only company in the world without access (except for GDM+labs) to a competent coding agent will take a toll.

We’ll see how long Google can hold out hoping for GDM to create something that is competitive.

I’m guess that within 6 months Google will give up on coding and finally let their devs use Claude/Codex.

This isn’t a security problem, this is a GDM issue with GDM’s promises being far beyond their ability.


> But Google being the only company in the world without access (except for GDM+labs) to a competent coding agent will take a toll.

I doubt it. I use Gemini CLI daily because Gemini is what work pays for, and I have a personal Claude account. The difference is not that great, especially if you're not doing full vibe-coding. It's unlikely to have the kind of effect you're describing.


I agree with the fact that your company has quota and allocates some to you.

Gemini will conduct seconds to minutes of work before requesting aid. And it will commonly fall over.

Claude/Codex will commonly do minutes to hours of work.

The difference is one to two orders of magnitude. It is immense.


I look forward to seeing that play out in the market, if true. But from what I've seen, it really isn't.

If you're talking about the ability to churn out low-stakes systems like websites, or variations on existing widely available systems, then perhaps. But once you get to more complex systems, especially large already existing systems, all LLMs today need significant ongoing assistance to prevent them from going off the rails and down rabbit holes. At that point, the advantage you're claiming tends to evaporate.


There is value in the "eating your own dog food".

If internal staff aren't happy with the tools they build, typically that should drive improvements to their own tools


This couldn't be further from the truth


Is there a page where I could read more? What's unintuitive at a glance is that Opus 4.7 has a lower success rate than Sonnet 4.6 (90% vs 100%) while having a higher Avg Percentile (87.2% vs 70.9%).


We calculate percentiles based on successful submissions only, and then apply success rate as a separate measurement, which is incorporated into our relative rankings.

So we do penalize evals where the player failed the game, but not in the percentile measurement (success rate measures instances of playing incorrectly, did not compile, runtime errors, and other non-infrastructure related issues that can be blamed on the model). The design decision there is that percentile tells you how good the model's ideas are (when executed correctly), separately from how often it got something working correctly, but I can see how that's not great UX, at least as presented now.

But the actual score itself is a combination of percentiles and success rates with some weighting for different categories, nothing fancy.

I added a methodology page to the roadmap, thanks for pointing that out. We've converged on a benchmark methodology that should scale for a very long time, so it's time to document it better.


Neat, thank you for explaining!


The $20 Plus plan still exists, and does not give access to the pro model.

The $200 Pro plan still exists, and does give access to the pro model.

What is new is a $100 Pro plan that does give access to the pro model, with lower usage limits than the $200 Pro plan.


This is still worse than Anthropic's right? Because you get access to their top model even at the $20 price point


It's not worse, Anthropic simply has no equivalent model (if you don't consider Mythos) of GPT 5.4 Pro. Google does though: Gemini 3.1 Deep Think.

GPT 5.4 Pro is extremely slow but thorough, so it's not meant for the usual agentic work, rather for research or solving hard bugs/math problems when you provide it all the context.


I'm genuinely asking, when you say Gemini 3.1 DT is an equivalent model of GPT 5.4 Pro, is there a specific benchmark/comparison you're referring to or is this more anecdotal?

And do you mean to say that you don't really use GPT 5.4 Pro unless it's for a hard bug? Curious which models you use for system design/architecture/planning vs execution of a plan/design.

TIA! I'm still trying to figure out an optimal system for leveraging all of the LLMs available to us as I've just been throwing 100% of my work at Claude Code in recent months but would like to branch out.


Pro and DT model are equivalents because

- internally same architecture of best of N

- not available in the code harness like Codex, only in the UI (gpt has API)

- GPT-5.4 pro is extremely expensive: $30.00 input vs $180.00 output

- both DT and Pro are really good at solving math problems


So, reading the tea leaves, they're either losing subscribers for the $200 plan, or they're not following the same hockey stick path of growth they thought they were... maybe?

Edit: I wonder if this is actually compute-bound as the impetus


Nope, it's just that a lot of people (especially those using Codex) asked us for a medium-sized $100 plan. $20 felt too restrictive and $200 felt like a big jump.

Pricing strategy is always a bit of an art, without a perfect optimum for everyone:

- pay-per-token makes every query feel stressful

- a single plan overcharges light users and annoyingly blocks heavy users

- a zillion plans are confusing / annoying to navigate and change

This change mostly just adds a medium-sized plan for people doing medium-sized amounts of work. People were asking for this, and we're happy to deliver.

(I work at OpenAI.)


Did you modify the Plus plans usage recently or as part of this introduction? Given that Pro plans usage are multiples of it (5x/20x) and given reports of less Plus usage, clarification would be appreciated?

Transparency on this sort of thing is the best way to address negative company sentiment.


I'm honestly not sure, as I don't work on it. My understanding from afar is:

- There was a 2x promotion in March that ended on April 2, so limits have felt tighter since then

- We sometimes reset rate limits after bugs or milestones or because Tibo feels generous, which can make some days feel different than others (they are typically announced here: https://x.com/thsottiaux)

- Recently Plus was tweaked to have a smaller 5h limit but an increased weekly limit

- Lastly, as part of the new Pro launch, the $100 & $200 Pro tiers are getting a 2x promotion, meaning they are temporarily 10x/40x instead of 5x/20x

I've asked our team to clarify the pricing page. Agree it's not clear.


Following up - I was wrong about 10x/40x. Here's how it actually works:

$20 = 1x

$100 = 5X (but temporarily 10x for just Codex til May 31st)

$200 = 20x

We'll send out new tweets and clarify our pricing page.


Thanks for the response. I tried to phrase my postulations as just that, I didn’t intend to be an accusatory.

You like the job? How’s the day-to-day go? Yanking tickets or more organic?


All good, I interpreted it as postulation and not accusation. :)

I do like the job! Much more organic than yanking tickets, though I'm on the model training side of things, rather than product side. Always a balance between short-term sprints patching bad behaviors for the next model vs long-term investments in infra and science that make future work easier. Sometimes the negative press gets to me a bit (it's a very different feeling than 2022 or 2023), but my goal is just to make the most useful product I can for people. It's been wild how much Codex has already changed my day-to-day work, I'm so curious to see what it looks like in 2030 or 2040.


What kind of bad behaviors? How is the whole SDLC lifecycle there? I imagine, given that this tech is kind of redefining how software is being written, it's not your standard workflow pipeline? Are there code reviews at all? Have you been in any particularly interesting meetings about how you're trying to "shape" the models?

I won't misrepresent myself, I've never spent a penny on any of these services. I am just super curious what it's like to work at one of these frontrunner companies. I bet it's pretty neat.


Plenty of people wanted to spend more than $20 but less than $200 for a plan. It's long overdue IMO.


It would be very helpful if the chat logs could (optionally) be retained.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: