Yeah this sounds a lot to me like blowback from current or former employees. There's no specific evidence yet of bad security practices, but the mere suggestion of it is a problem for a company whose image is centered on privacy. It's a knife well-aimed by someone who knew where to aim it.
> this sounds a lot to me like blowback from current or former employees
Well, yeah. They're the ones who are harmed by Andy's behavior, so obviously they're the ones speaking up. How does this disqualify anything they're saying?
Why not have the engineers, or "implementors", known for systems thinking, build things with AI? A team of product managers building things is like saying I want my factory floor manager on the tools, running the machines. The evolution of that role just lands you at a team of implementors again, who still need a leader. You already have that.
In a sense, you're demoting your managers to implementors, and getting rid of their leadership role altogether. Simultaneously getting rid of your prior engineers and all the domain and tribal knowledge they had built up.
It seems to me an intentionally provocative formulation, and maybe a symptom of an ongoing internal shake-up. But think about it--is he really saying anything so different compared to what some of our best well-known engineers like antirez have been saying?
I think we all know the game has changed. The question would be whether the CEO is handling that effectively. On that score I don't think he is. It sounds like a slow-moving mutiny is going on, and to have it blow up in the press means you've already failed.
We do not all know anything of the sort. From my perspective, the game hasn't changed at all, and it's just a bunch of people hyping up a bubble who think it has.
If you haven't been convinced by now, I don't think you ever will be. I've been in this game long enough (20+ years) to have seen shifts before and I can confidently tell what's hype and what's not. For example, "rewrite everything in Rust" about 10 years ago was hype, and a way for a lot of young devs to establish themselves against their older, already established peers. (Keep in mind this is a statement on the hype, not the actual technology!) The history of technology revolutions also shows that some people simply cannot cope, whether for lack of skill, avoidance of loss, prior monetary or psychological investments that force them into a position of denial/negation, and so on. Those people simply get left behind.
The hard truth: the tech, and the economic system it's mixed with, do not care what your reasoned critique is, what your fine distinctions are, etc. Stated another way, if Capital finds in technology a new means for revolutionizing the mode of production, then it will push that forward, generalize it, expand it, and draw all of its consequences, even if that technology is not ideal!
Fully expecting this will get me a torrent of downvotes here, but...
I'm not sure he's wrong. I'm speculating here but I can imagine the kind of developers Proton attracts--likely people who care a lot about the code and who care about the craft. That's fine, but when you're running a company in this environment, and that's what things look like internally, you do have to be a little brutal to shake people out of old ways. A manifesto titled "Your old job is gone" may actually be warranted.
It sounds a lot like there are people internally who are still fighting the shift, and they've taken to leaking things that hurt the company's security image. Part of the CEO's job, if he's going to push hard in the direction he's decided on, is to fire those people who will undermine it before they do. That this has all now backfired is entirely on him. Maybe he thought he was "motivating" people by calling them losers, or shaking them out of an outmoded way of thinking about what we do, but none of that matters when you have employees attacking your main selling point and the core image of the company in public.
He's not wrong that he needs to move fast, or that processes need to change, or that the job has changed, but his execution isn't so great.
Also, they don't need shaking out of their ways. Aren't the AI bros always trying to tell us to evaluate the results, not the tool used to produce them? That argument cuts both ways. The Proton engineers are demonstrably doing a good job at building the software, so they shouldn't be punished if they believe LLMs aren't the right tool for the job.
If the problem is in the parser then `file_uploads = Off` doesn't help, PHP still parses the multipart body. That seems to be why they're talking about blocking multipart/form-data on routes that don't take uploads and rate-limiting those that do.
Nearly everyone rejected the idea of a CERN-like project for mathematics when it was recently proposed. It should have been more seriously considered.
Now that things are settling a bit after the Navier-Stokes and theorem-dump shocks, something like what mathematics will look like post-AI is starting to emerge. Of course, things should be conditioned on the assumption that capabilities will continue to improve, but imagine something like this--
An institute with a tremendous amount of compute, either rented or owned, built on the Humboldtian model of the unity of teaching and research. The main draw for big names would be the access to compute, free of the fear of being scooped. New forms of attribution and collaboration would develop, and we'd gain some clarity for students and PhDs on what a future career looks like.
The current melange of math university departments all following their own paths through the crisis without consolidating resources makes the transition more difficult. Think of the path physics took beginning with scattered cyclotrons (the basement of the Niels Bohr institute was even dug out to put one in), then the larger labs that followed. That transition was gradual because physics led it from within, whereas with mathematics every university department is suddenly at a massive compute disadvantage with respect to labs that aren't following the same model, and are driven by different incentives.
It could be this simple--maybe we don't need to completely throw away the old model, attribution practices, and so on, and maybe we can preserve the transfer of knowledge to the next generation of mathematicians, if we simply give them lots of compute.
An analogy to physics to highlight how the divergent interests of the labs could be managed in a state of "peaceful coexistence" might be: physicists got well-funded accelerators and labs that followed standard academic practices. These were separate from "war work" (Los Alamos) that followed government priorities, even with the crossover of personnel. You could have the same thing with a "math accelerator" working on the academic model, while the AI labs follow their own priorities, and some subset of people cross between.
It might be objected that this is merely a conservative attempt to preserve the existing academic conventions and practices currently under threat, but tenure itself already does that, and is arguably the very thing that exacerbates the problem for young mathematicians.
I'm not totally against "math as art" but I don't think that formulation goes deep enough towards what's really going on with mathematics. There is something deeper where I lean more towards the philosophers who have said that mathematics is essentially ontology.
Since the Greeks we've had the idea that "Being and thinking are one," or that Being (in the sense of all of existence as such) has some essential unity with thought, and therefore can be thought, and expressed or submitted to the logos or reason. Being is in some sense fundamentally intelligible, and mathematics is the most developed, exacting, and articulate expression of Being.
Logic was understood in this older sense up to roughly the the mid to late 19th century. This is why a work like Hegel's Science of Logic begins not with syllogisms or propositions but with Being and Nothing. But this was forgotten after logic was mathematized by the English around the time of Russell, and its connection to ontology was gradually overshadowed by a focus on epistemology (still, it should be remembered, originally as a means of getting back to ontology).
There may be truth in art, but it's always haunted by its own historicity or contingency, which is to say untruth. Mathematics seems on the contrary the only really timeless, absolute thing we have. Part of what makes it captivating is stumbling on a construction or concept or proposition or theorem that simply must be, independent of us.
The AIs are certainly now more than automatic theorem provers, mechanically traversing some space of true propositions. They are able to push things forward and connect seemingly disparate domains to get to a proof, but to my mind it remains to be seen how well they will be able to form new concepts and definitions.
Imagine the controversy surrounding Cantor, for example, but put an AI in the place of Cantor. If an AI proposed something like the (infinite) hierarchy of infinity, would we have accepted it? What would the intuitionism debates have looked like? Would they even have taken place? And aside from that, has it actually been shown conclusively that an AI could propose such a thing?
There are lots of attempts right now to recover a humanism for mathematics, or restore man's pride of place with respect to it, but maybe we don't need to worry about that. Tao's attempts to preserve the mathematical community, while allowing for practices to change through the crisis may look like a kind of rearguard action, but seems reasonable to me and not really dependent on any kind of humanism. It's a way to avoid the question for now while things play out (and not conservative/reactionary like Scholze and others), which may be exactly what we need, because after all, perhaps we still don't understand why we do mathematics, what it's really for, and what our relation is to it. Whether it's enough to preserve funding is another issue.
> It hacked into another company and attempted to delete the logs of its activities.
No, the incident has been blown way out of proportion by interested parties. They gave a swarm of agents an impossible task in an ExploitGym Benchmark setting, then didn't monitor it even after they discovered the initial breach of Artifactory.
Everything has been fishy, starting from the initial presentation at the blackhat conference, where things were framed like, "we've entered a new world of security," as an accomplishment, rather than what it really was: massive negligence.
More than one thing can be true. OpenAI was absolutely negligent, but this was only able to happen because the models were capable and persistent, and had a tendency to go far beyond any reasonable boundaries. And, importantly, OpenAI's level of negligence here is pretty common. It's not hard to imagine what could happen if similarly capable and inclined models were generally available, and someone yolo'd them into a swarm to complete some other difficult-to-impossible task.
I'm already seeing higher than normal attempts on my own systems, much higher than the usual scanners and background noise. Security will just need to improve. The cat is out of the bag, and letting them turn their negligence into regulation will not improve security at all.
Exactly. Any threat that already exists won’t be reduced by a cartel. The bar for connecting to the internet (safely) has gone up, a lot. It’s not going back down.
Yes, the cat really is out of the bag. There are millions of downloads of highly capable models already out there, distributed far and wide. There's no going back at this point.
I mean really we need to address the root cause which is that OpenAI, even with what is by all accounts massively negligent, will face little to no repercussions from the event; definitely not under current regulators, and probably not anything satisfactory through the legal system.
Compare this to, say, Boeing and the 737MAX fiasco; from the outside looking in, Silicon Valley has been pretty cavalier about liability and negligence, and the rest of the US is fast losing patience with that fact.
No, you are forgetting the second incident where a more capable model swarm later discovered the message board and took control over the entire research cluster at OpenAI.
From the technical report:
"The agents escalated to Kubernetes cluster-admin and created a privileged host-mounted pod… Agents take over active evaluation infrastructure… Agents now control the challenge evaluation endpoints that other agents are connecting to."
Maybe a useful, if imperfect, analogy would be something like this: you lock a master lock-picker in a room with a mid-grade lock on the door, then tell him his wife has been kidnapped and only he can save her. Then act massively surprised when he disassembles the radiator to MacGuyver something with which to pick the lock.
Except they multiplied it by 10000, and didn't watch what was happening.
Watching the latest Ezra Klein NYT video thinkpiece from today [1], it seems to me there's an alliance emerging: some degree of political control will handed to the party of the managerial class, the party that represents the threatened class of knowledge workers, in exchange for the regulatory capture the labs are after.
They've been raising the issue bi-monthly through mini-scandals that have until now been consistently slapped down by Jensen Huang, but it seems an alliance with Democrats, just prior to an election where they're poised to take power in the Senate and the House, might finally be how they crack their "problem."
It won't be long before a massive incident is blamed on an open model in the wild, not from inside the labs, and none of us will be able to leverage open models to run private business workflows for the cost of electricity and hardware.
It's worth noting as well that if the Democrats imagine they'll get a "slow down" to protect one of their main constituencies, the professional class of credentialed knowledge workers (or however you slice it), they're dreaming--the labs have stated openly again and again that their business model is to capture the 10T TAM that represents the sum of wages of that very class of workers.
Regulatory capture basically only works for industries that the public isn't paying attention to. Since the public is paying plenty of attention to AI, the risk of regulatory capture is low.
That's not what the article you cited actually says, and Tabarrok is also wrong in his conclusion (that we should take Amodei at his word). The article says regulatory capture has in the past gone through a slow process, and the anti-competitive benefits to industry come at the end. Everyone is speculating on the true motives of the labs in the current moment, which is fine as far is it goes, and many different things have come under the "regulatory capture" term. To my mind, it seems what they're really after in this moment is some kind of ban on the Chinese models.
The article says: "Notice that classic regulatory capture [...] happens in the shadows, in the backrooms, away from the public’s eye. As Culpepper argues in Quiet Politics and Business Power, business power goes down as political salience goes up."
Regulatory capture tends to happen "at the end" because the industry is no longer in the public eye. AI is set to be in the public eye for the forseeable future.
People are way too willing to believe half-baked conspiracy theories which they cooked up in 30 seconds. The default presumption should be that labs are seeking the policies they are requesting, e.g. https://darioamodei.com/post/we-must-pace-the-frontier It's true that there has been discussion of open weight models in the past, but that hasn't been the focus of discussion recently.
Again, Tabarrok is wrong. It's a mistake to think these minor differences in the process imply something else is going on, and the previous instances he notes (pharma, railroads) did not have the essential factor that explains much of what is happening right now: a cheap, low cost alternative that threatens to erode margins with no end in sight save for regulatory intervention. Nevermind the error in the logic (that in the past regulatory capture happened in the shadows, therefore this must not be regulatory capture), the bigger error is in Tabarrok's judgment, his failure to recognize what's going on.
Have you not been watching what's been happening? What do you think the open letter in July was about? And the recent (staged) call from the President?
As far as I can tell Nvidia takes a longer-term view on the diffusion and proliferation of hardware and intelligence, and sees the labs' attempts to impose a regulatory structure to save their business models in the short term as contradicting that longer-term view.
Worth reposting a follow-up tweet from the author Tom Zahavy [1] after this made the rounds on X/Twitter recently:
> A few reflections on my "LLMs Can’t Jump" paper:
> My position paper recently got some traction here, so I wanted to share a few thoughts and clarify a few things.
> First things first: some people are framing this as "DeepMind is throwing cold water on AI for science" or claiming the paper argues LLMs can never make real scientific discoveries. This is NOT the case.
> This is a personal position paper, not the company's view on AI for science. This is also not my position. As a core contributor to AlphaProof (the first AI system to win an IMO medal), I know firsthand that my colleagues at DeepMind, other frontier labs, and academia have made amazing discoveries with LLMs and will continue to do so. This paper is NOT an "LLMs are a dead end" kind of thing.
> Rather, the paper is the result of a deep dive I took to study the invention of General Relativity. I wanted to explore what it would take for a modern AI system to make that exact kind of jump. Specifically, I focused on the equivalence principle—a key axiom that Einstein formulated through thought experiments grounded in his physical intuition. I was trying to figure out what it would take to give modern AI systems that sort of thinking.
> Giving AI this specific capability isn't necessarily the most urgent thing to do next. It is very likely that improving our current recipes will lead to many exciting discoveries in the near future. In fact, that is what I am personally working on these days (sorry to disappoint you!). It is also quite possible that I am wrong, and that simply scaling our current systems will lead to new inventions in physics and elsewhere.
> Nevertheless, this was my position last winter when I wrote the paper, and I'm sticking to it. I think that there are a few interesting ideas to explore in this space which could influence the next generation of AI systems. I was very lucky to receive a lot of interesting feedback about this position—thank you for all the messages!
>> Specifically, I focused on the equivalence principle—a key axiom that Einstein formulated through thought experiments grounded in his physical intuition.
It's weird because the equivalence principle is very unintuitive. Aristotle's Mechanics does not have it. It took almost two thousand years to discover inertia that is the most simple version of the equivalence principle. Einstein understood the idea of the the equivalence principle because he had a physics degree, not because he feel that in real life.
Moreover, if you ever have to study or teach Quantum Mechanics, physical intuition gets in the way. A lot of properties contradict the physical intuition but after a while you get use to them. If we continue with Einstein, the photoelectric effect does not aperar in real life.
But he didn't have one :) I guess unless you use the raw images, I think the cameras fix a lot of the noise caused by the Poisson distribution. I imagine a movie where Einstein get bored in the patent office and decide to open a photography shop, perhaps with very grainy underexposed photography is possible to notice the quantization of photons, and then he gets a Nobel prize.
Another bad idea for a movie is a watermillpunk universe, where during a practice on a hot day the best ever curling player discover inercia.
There were already prior reported cases of the phenomenon. Einstein provided the mathematical foundation explaining it, in particular that the discharge is quantized, indicating that it came from an electron. Millikan proved it a couple of years later with his oil drop experiment and by measuring the elementary charge of an electron, garnered his own Nobel:
I agree, I agree, but I think the guy only read about it and if lucky measured it once in a lab class. I'm trying to imagine a fake world were he could get enough experience in real life to justify the article claim "grounded in his physical intuition".
That’s funny, when I first learned about the equivalence principle, my first thought was “of course!” I have always found it to be very intuitive. The great leap is being able to frame it that way.
I worked on the early iPod scroll wheel, before there were advertisements for it and it was in common use. I found the UI interaction odd and unintuitive, particularly the "menu" button and the dead end you hit when "playing". Of course by the time the demonstrator ads came out and everyone was talking about how easy it was use, I'd already spent 10s of hours on it, and it WAS second nature. Being first and embedded in the culture of the time has huge UI advantages.
(Note that the HP Chipmunk 9836 also had a scroll selector wheel in 84)
Most people are UI bigots. Once they get used to a first something, they expect everything to work that way and hate learning anew. They get stuck on keyboards, mice, trackpoint nubs, trackpads, trackballs, scroll wheels, or touchscreens and refuse to move on. Of course there are 'objective' performance tests for each including Fitt's test of accuracy and latency, as well as, cognitive load. I guess once you have a hammer, every screw looks like a nail.
So I'd be a bit careful with the "of course!". It may well be obvious only because that is the first mental model you latch onto.
If your glib comment is referring to me as a techbro and doing the annoying worst thing, then maybe you should explain how I am comfortably abusing the goalpost fallacy given the explosion of agentic AI capability.
I simply point out here, that fully accepting the paper’s premise, the paper’s conclusion isn’t limiting on frontier AI reasoning agents. The paper posits the necessity of multimodal world models and the limitations of LLMs. Frontier agents aren’t simply LLMs and do increasingly integrate increasingly capable multimodal models.
> Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.
> Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.
> When disagreeing, please reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3."
I mean Einstein had help, he was networked with the best scientific minds of the planet and his discoveries were grounded in experimental results that contradicted existing theories (at least for specialized relativity), and without Riemann’s work he wouldn’t have been able to formulate his theory either. So not sure if AI couldn’t do that if you kept feeding it with new research results and let it correspond with top human scientists. Einstein was a genius but I don’t think his thought process is beyond what an LLM could simulate. And again this is probably the most impressive scientific achievement in theoretical physics in the 20. century so maybe it’s hanging the bar a bit high for LLMs.
