> Large, complex software projects likely require multiple rounds of scanning and fixing vulnerabilities to “bottom out” latent bugs. Ideally this process is repeated with each subsequent generation of frontier models. This applies to both proprietary and open source software.
One of the most important paragraphs from the article. Real implications for software development from a security perspective for new and existing software. Every software developer and vendor has a choice... Submit your code base to a frontier model to have it reviewed and fix all the bugs. Each time a new model is released, this process must be done again. If your company is against submitting your code to the frontier model companies for the many good reasons to avoid doing it, your software will eventually become a target if it isn't already. Then your company will be forced to scan the software and deal with the fallout from getting hacked.
If the Linux kernel has this many exploits, and it has been audited more than most other software... Then it seems obvious that all other software is even less secure.
Have there been significant AI-assisted exploits found against software which is not written in memory-unsafe languages?
As models get better and porting costs get lower, maybe it's better to bite the bullet and RRiR instead of having a minor bugpocalypse every 3 months, as each subsequent generation of frontier models finds sev-high CVEs?
I am inclined to doubt any praise for rust because of its strong association with the world's largest advertising conglomerate. If codebases can be too complex to reason about, might that not be true of rust's codebase?
Security has been treated by corporations as a cost factor only and therefore ignored for years and now it starts to hurt not having taken care of in the first place.
"you're going to need to do multiple rounds of scanning so we recommend budgeting $3.6 million a year for buying tokens from us not just $1.2 million.
Dont worry just give us a few million dollars and you'll never get hacked, you can cut your security staff in half to pay for it! We definately wont cause more bugs!"
> If your company is against submitting your code to the frontier model companies for the many good reasons to avoid doing it, your software will eventually become a target if it isn't already.
Yeah yeah we're all going to be left behind if we do not jump off the same cliff everyone else is jumping off.
It is wrong to try to take your own lessons from this and do the thing but in a less harmful way. Just do exactly what the labs want you to do.
It's worth mentioning that many of these papers haven't been fully reviewed and confirmed as being correct. Several had to be retracted upon closer inspection. Too many people are assuming that passing lean checks for some of the papers means that all of them will hold up under scrutiny. It remains an open question if humans or any available system are capable of peer review for some of the more complicated proofs.
Any revolution without the privacy available at the time would have never succeeded. The ability for people to communicate without the state being able to listen presents too much of a danger to be allowed. This is true in every kind of state, democracy, theocracy, authoritarian, doesnt matter.
If AI is building all the systems and writing all the code, and AI is finding bugs and exploits in existing code and code being generated by AI... With the assumption that any code base scanned in its entirety by the latest frontier cybersecurity focused models is secure... This assumption only holds until a more advanced model is created to find bugs and exploits in the code from the now lesser advanced model... Which means even the code the top of the line model just created was never really secure...
How does this cycle get broken?? Will people ever be able to be sold the (false idea) that their devices are secure?
It is sometimes difficult to find publicly released polling for many different races, especially the down ballot ones. Other polling is done but kept internally and not released for free.
If a fake polling company could be set up to release polls for underreported races that are being traded on prediction markets then you could make a good bit of money by timing your trades to the release of your fake poll.
The goal is not just to influence the prediction markets to game a profit, it would also be to hope that once the polls influence the prediction markets, that it will feed back into the voters to actually influence the result.
It's a reasonable assumption that the cases where the NSA, etc, had a role in designing cryptographic systems, and reportedly made specific suggestions that could be forms of a backdoor, would all be revealed by AI being tasked to find it. Maybe even indirectly as part of the broader defensive security research by models unreleased to the public. If algorithmic complexity or a really obscure backdoor has existed in cryptographic systems and put there by the Gov't, it seems that current frontier models would have the best chance of revealing it.
To fully remove the national spy network from your life you would have to get rid of all other devices that will collect the same data or do already. Smartphones, smartwatches, home security systems, Wifi, any computer or device you did not build yourself, TVs, any other "smart" device, anything that has electricty going to it, any car since OnStar in the 1990s, all delivery trucks and self driving cars, your neighbors security system, devices that broadcast IDs, even bluetooth devices... So many examples..
The amount of devices in the national spy network which is passed off as just advertisement data is really providing full spectrum live video and audio to third parties that can and do share it for any reason that benefits them.
No, we would simply have to make spying illegal and require offenders criminally to pay restitution plus fines for each violation. You'd see vast improvements in folks' behavior pretty damned quick if people started getting thrown into jail AND bankrupted for spying.
One of the most important paragraphs from the article. Real implications for software development from a security perspective for new and existing software. Every software developer and vendor has a choice... Submit your code base to a frontier model to have it reviewed and fix all the bugs. Each time a new model is released, this process must be done again. If your company is against submitting your code to the frontier model companies for the many good reasons to avoid doing it, your software will eventually become a target if it isn't already. Then your company will be forced to scan the software and deal with the fallout from getting hacked.
If the Linux kernel has this many exploits, and it has been audited more than most other software... Then it seems obvious that all other software is even less secure.
reply