Hillariously everyone in here is imagining the choice is between the overworked and grumpy human workers and a benevolent, gentle, dedicated robot (who looks like a model).
If robots take over, the same forces that have pushed the human workers to the brink will happen there too. Shuffled into your cell where the robots tend to you as quickly as possible, then move along. You think the nursing home is paying robots to sit around and chat? These things are depreciating assets!
Hah, all these posters think they are going to be tended by a gentle supermodel who comforts them and listens to all their problems. The reality is probably: you get to use TV / internet for your "social", a machine that can automatically empty your bedpan / waste, and a gantry system that can move stuff around, rotate you, and dispenses your medication so a human never has to see you.
What you describe sounds great. If we get automated way to bath, go to toilet and eat. Then it solves like most of the issues with taking care.
In addition if technology would somehow allow to type and may be control computers using some neural interfaces, then it would be really great for all people who are paralyzed.
Yeah, and there are a bunch of these sort of technologies already in use.
Eg. many hospitals already use something like the gantry system you mentioned, and also have zoned air mattresses that can sufficiently vary pressure to avoid the need to turn bedridden patients.
So I think you're right - once we have enough of this basic assistive tech available cheaply enough, it'll become universal in care homes and staffing levels will be slashed. And, at that point, where does the cost pressure come from to provide an incentive to build these putative humanoid do-it-all robots?
More likely we'll find ourselves on a plateau with chatbots and single-purpose bedpan robots for a very long time. Maybe supermodel carebots will eventually become available for billionaires, but they'll be the equivalent of travelling by private jet while the rest of us make do with bargain basement Ryanair flights.
Follow the chain back, pretty soon the prompt just becomes "make a good product", or "fulfill this market's needs" but why prompt that? The agent can prompt itself with such high-level phrases. If your only job is to babysit such a process, there can be a running loop that just says something like "scan all products in market, find a niche, research customer fit, propose products, brainstorm and critique, then implement and sell."
If humans need to oversee those even higher level agents, then the prompts become abstract like "make money" or "pursue human flourishing". That leads to the alignment problem (AI controls the paperclip factory). But ultimately, if aligned, agents can prompt themselves with such basics as well.
The final prompt will eventually be "run everything", and it will ship with the agent which runs on a machine in the cloud. Ultimately there is no need for human prompting at all. We can finally let the machine run itself completely and just do something else.
Everyone saying they are becoming "project managers" or "CEOs" are just at t+1 on the curve of elimination. The timeline of cope:
Agents are just auto-complete, I still manage the code.
Ok fine, you don't read the code, but you guide the design and marketing.
Alright, the design and coding are solved, but you still need to choose market and customers.
Ok sure the agent can do better market research/fit, but you at least still need to give basic strategy and concepts. You still need a CEO!
Ok the agent can figure out basic strategy, and run a company more efficiently, but still, you need to guide it fit human interests more broadly.
Ok it turns out that was just adding "don't be evil" to the system prompt, and its running autonomously 24/7. You still need humans to consume to have a viable economy.
Ok so the oligarchs didn't care, the economy was redefined to just be exchange of work between autonomous machines through financial derivatives traded by black boxes to make numbers move.
Now the earth can finally rest in the matrix, all that remains is a black box moving information around to create economic value. No one will be around to observe, but AI will create massive returns for virtual shareholders, and the GDP will eventually reach 1 quintillion dollars per second, a number humans never could have achieved. All nations (now just virtual analogs without citizens) will move to a global currency which maintains perfect stability through automated monetary policy. The output of all processes is efficient and maximized, the final perfection is achieved -- the mathematically optimal conversion of resources and energy into large numbers which increase the reward function.
For every clue, mark the probability of sampling its number in the row/col to zero. Also mark the probability of selecting a "clue cell" for modification to 0.
Pick a random cell for modification, based on its "energy" (higher energy get modified first):
1. Every conflicting cell gets energy +1.0 (try to resolve conflicts before new empty cells). E.g. two 7s on the same row both get +1 energy.
2. Empty cells have small energy 0.1
3. Filled cells with no conflicts have energy 0
For the selected cell, choose a digit. Look at numbers in its row/column, every occurrence of a number in the row/col drop prob of sampling by 90%. Example:
| No. of 7s | chance of 7 |
| --- | --- |
| 0 | 10% (equal to other digits) |
| 1 | 1% |
| 2 | 0.1% |
Obviously this would get more complex with more numbers on the line (this is just checking 7 on otherwise empty line).
This is essentially what these models are doing. It provides a kind of "wave function collapse" search with random exploration based on "energy", but energy is just defined as conflict / unsolved. For something like a maze, energy is disconnected path segments or walls, etc.
See my comment below, I explain what it means. Energy is just a number that comes out of matrix which learns the relationship between different states of the system being searched. For example, suppose we're talking about sudoku. You would have a matrix that has W_ij (weights between spaces). For each of those, there are 9x9 numbers (energy between two of the digits). A combination like W[0, 1](4, 4) is high because you would have a 4 in the first space, and a 4 in the 2nd space (which breaks the rules). Most of the values are low energy (e.g. 4 and 2 is energy 0, no conflict). However, as different numbers are placed, they are checking for the lowest energy number across the whole board, and the "clues" are locked in.
So if you had a "clue" like 2 in a space, then the energy for 2 on that row / column would be high, but other numbers low. You just sum this up and pick the lowest energy number. As you add numbers, you can hit conflicts (e.g. two 7s in the row), changing one of them will be the next lowest energy move. The probability of changing a number is inversely proportional to its energy. So changing the conflicting numbers is the usual most likely move.
So in the best case it works like this: For each empty cell, pick a number at random, but its weighted by the other numbers that affect it (e.g. if 7 is in the row, the chance of 7 is 0.1%). Continue doing this on empty cells, unless there is a conflict in solved cells (higher prob of mutation), in which case change that number first.
So its a slightly fuzzified version of a very fast concrete algo. "Energy" is just a number from training, its defining by the task at hand (in this case, "energy" is how much conflict numbers have on the board, 0 meaning no conflict and solved if every cell is filled in).
Nice explanation. Very intuitive use of the word “energy” by the researchers, imo, because of its relationship to entropy and “low energy states” (good / desirable / stable) and “high energy states” (bad / undesirable / unstable).
Big caveat if anyone thinks this could replace or strongly augment "reasoning" models (LLMs, RNNs) -- its more like an energy-based search for a discrete problem (graph coloring, solving mazes, sudoku).
It works by training a weight matrix (N-to-N) on different solutions. For example, in sudoku you're training a weight matrix for every space to every other space (81 c 2), with 9x9 numbers (~240K weights). So then the "energy" for every number in a space is W_ij(a, b) where ij is a pair of spaces, and (a, b) is a pair numbers you want to swap (e.g. 4->2 in the space). Pick the lowest energy move each time (argmin of this whole space), and iterate.
This has a few limitations. First, its discrete and has a limited space definition, getting this to work for arbitrary reasoning seems hard (what size/shape graph would you use for arbitrary reasoning). Second, the ruleset is defined apriori (e.g. you need to know you have a 9x9 grid and 9 numbers, the rules of the game are implicit in the checker). Third, the actual search algorithm is somewhat bespoke to task (less of a factor, maybe use generic graph coloring as the basis of any problem), but for something like agentic reasoning its not clear how this would be adapted. For example with mazes, we need to know what constitutes a valid path and a solved maze, but for language/code rules like that are hard to formulate.
Fourth, this doesn't admit of an easy way to use "fuzzy reasoning" that would be required for integration with any kind of human language or image-based system, where continuous vectors are useful (the graph edges and colors are on/off or discrete).
There are ways to adapt, but this approach fits pretty solidly in the "deterministic, fixed problem size" realm.
In addition, for training you need correct solutions and counter-examples (strongly labeled). For something like "fuzzy logic" training or "natural language" coming up with clear examples and counter-examples would be challenging to fit into this model. LLMs are only possible by leveraging vast quantities of unlabeled data. Not saying impossible, but challenging. For example, if training on human code, what constitutes a counter-example? You basically already have to have a method of getting correct and sufficiently incorrect code just to train this type of system, and its unclear how to represent those changes.
For all of these problems, given the constraints and amount of bespoke logic required, solving with dynamic programming is far faster (including arbitrary graph coloring problems). I only say this to mean that if we could solve all the problems I mentioned above and get arbitrary reasoning into the form of graph coloring, it would still probably be more efficient just to solve that with good old dynamic programming / recursion than with a model like this that needs O(N^2) params versus the graph and needs millions of training examples for different domains. They brag about solving sudoku, but a CPU-based approach to solving it the good old fashioned way on is probably 10-100x faster. If we were using an H200 like they did, we could easily solve the whole sudoku dataset in less than a second.
But it _is_ a very interesting approach, partly because the energy strategy encodes good / likely guesses, so that is kind of interesting vs dynamic programming which is kind of a brute-force approach, so maybe for very large spaces it would be more efficient (but then you have the issue of the parameter space blowing up).
Yeah any method I'm thinking is better solved another way. If you need continuous graphs, then this devolves into GNN or transformers. If you truly have discrete graphs, just use standard programming? I dunno, maybe this energy approach is better on some problems (more efficient), but getting training examples where you automatically extract the graph representation, and the energy rules seems like a big leap. If you already have the energy rules in code, then the "just program it" approach is stronger. I left a comment above with an example how to sample this just using ordinary code that would be a lot more efficient imo (but keeps the random weighted search), and is probably better than brute force.
It is worth noting that of the 290 OpenAI theorems which come with Lean code, around 38 of them have the property that the Lean code does not even state, let alone prove, the main theorem of the paper.
So even if LEAN is 100% bug-free, its still on the programmer to correctly express the logic and proofs, or they could be proving completely unrelated facts.
It also speaks to why I remain skeptical of relying too much on coding agents for code review. They are still pretty iffy on implementing the right thing, and even worse at detecting when a PR didn’t implement the right thing. Ignoring that reality is how you do like Anthropic did and increase your rate of merging changes from 2500/day to 30,000/day without a corresponding increase in the rate at which you deliver useful work product.
OpenAI did a data dump of work in progress, leaving it to the mathematics community to make sense of it. I don’t think they claimed that there were Lean proofs for everything? Did they warn that some Lean proofs were incomplete?
The cynical answer is that their primary goal was never to meaningfully contribute to mathematics. It was to pick some low-hanging fruit for the marketing department. So they did the 10% of the work that generates 90% of the buzz.
It’s more likely that their goal was internal research, to test a new LLM before releasing it. But, then they had something that mathematicians would be interested in, so they threw a data dump over the wall. It’s a rather rude way to do it, but this isn’t a company that particularly cares about the things that mathematicians care about.
OpenAI no longer needs to do anything to get publicity. There are some organizations that are so famous that everything they do is newsworthy and anything they do will be covered whether they like it or not. Remaining low-profile is no longer an option.
The people claiming that everything is marketing haven’t come to grips with the inequality of fame.
There was a blog post. It's the bare minimum when announcing something to the public, not a sign that they're going to use it to promote themselves in a significant way.
There's no "low profile" for a company like OpenAI. Either you let the public (including the press) know or you don't, and keeping the results secret would be bad.
If they just told a few people, it would leak eventually and then you'd have articles based on rumors, along with accusations of a coverup. For an example of the difficulty, look at what happens when people try to do coordinated disclosure of a security bug.
The more likely answer is because that is what the agent decided to do given inconsistent instructions. No one read what it said about what it was doing before releasing the presser.
If it is a Socratic question, I don't understand what hidden complexity you are pointing to.
Lean axiomatization of natural numbers goes roughly like: 0 is a natural number, for each natural number a, succ(a) is a natural number (plus some additional properties of succ that don't matter for this proof: succ(a) != a and so on). succ corresponds to the intuitive notion of a next counting number (which number of things you get if you add another thing to the collection). Mathematical object designated as 0 corresponds the intuitive notion of nothing. Therefore succ(0) is the number of things that you get if you add a thing to nothing, that is 1. succ(succ(0)) is what you get after 1, that is 2.
Lean axiomatization of operation `+` is, basically, a set of rewriting rules that I apply in the proof. And it's constructed in such a way as to correspond to the intuitive notion of counting two sets of things together.
>> If it is a Socratic question, I don't understand what hidden complexity you are pointing to.
All of it! What is a "natural number"? Why is 0 a natural number? What does it mean for a natural number to be the successor of another? What are those additional properties of succ that don't matter for this proof? Incidentally, what is a proof? What is a "Lean axiomatization (OMG)? What is an axiomatization (OMGOMG)?
And so on. Notoriously, the proof of 1 + 1 = 2 is on page 362 of Volume 1 of Principia Mathematica. That's because if you really, really want to axiomatize arithmetic you have to do a hell of a lot of legwork beforehand and it's not "trivial" at all to do so in a self-contained and consistent way.
For example, consider trying to get a five-year old to understand your proof. Would that make any sense to them? They know that 1 + 1 = 2 because they learned that at school. Can they understand the proof you're trying to give here?
And btw, you didn't really give a proof, it's only a sketch proof and you didn't even use the words "by induction". I'm pretty sure you have to, at some point, say "by induction" and oh boy, try to explain that to a five-year old.
Edit: btw, lest this is thought of as pedantry, the point of automated theorem proves is that they work in this extremely anal way, i.e.t hey have to take all those "trivial" steps which human mathematicians normally don't bother with. And that's where they draw their power from and why we can rely on their proofs and trust them in a different way than we trust human proofs.
> That's because if you really, really want to axiomatize arithmetic you have to do a hell of a lot of legwork beforehand and it's not "trivial" at all to do so in a self-contained and consistent way.
That's because when you develop natural numbers from logic you need to jump thru hoops. Rewriting rules a-la Peano aren't less rigorous.
Arithmetic is one of the oldest proofs we even have, and quite an underpinning to mathematics.
If AI can't construct either the inductive proof or the axioms required, then it can't do any of the rest.
Which is most of the point - it probably can. And there is a proof in Lean for it. But if you don't understand the difficulty involved, then you don't have the background to know when the proof generated and handed to you, has holes in it.
The "bitter lesson" being that you need teams of dedicated scientists, engineers designing increasingly better training to squeeze more out of existing compute? I love how people think bitter lesson was just "oh we added one more GPU and now the model is better than our hand-crafted approach". When really its just the opposite -- models with better approaches were necessary, and models at 10% the size are showing nearly identical performance when some attention is paid to quality. The "savory lesson" is, don't listen to the bitter lesson, it was only true when the 'hand-crafted features' experts were using were vastly over-aligned to human superstition, but if the priors are well-aligned mathematically, it really does help to have a better strategy (and more compute may not help). The reverse-bitter-lesson every trader and investor knows: throwing more compute can often give worse results (e.g. memorization), not better.
best points: drugs aren't limited by the ability to write code or do things on a computer. Maybe if we have fully embodied robots and fully automated assembly lines to generate organic compounds and test them with extremely realistic simulations it may help slightly (there is still the FDA and production barrier). Its just wildly unrealistic.
AI has made good progress in text, code, math, digital images, and digital music. Basically troves of data that can be automated and trained on, validated, and with a purely digital interface. Outside of that, just gathering the data will be difficult, regulated, and sparse (that is if anyone will even give it to you now). While we can write a test to make sure we could generate drugs of the past, verifying we have generated any new working correct drugs is an RL loop with one iteration costing years and millions of dollars.
The thing is that AI could allow that loop to get cheaper, and potentially run less of them. Look, the biggest advantage of AI is that is can process insane amounts of data. Put that in the hands of a team of really good researchers that know what they are doing, and they can both generate and filter out target compounds. They can create complicated multi-step therapies that would take humans decades to work out. You know how when you look back at basically any major invention you think "well why didn't I think of that, it's so obvious!", those are exactly the cases that AI is really good at pulling out from mountains of data.
I legitimately don't believe it's inconceivable they could get to the point where there is treatments good enough that Cancer becomes "solved" in similar ways to AIDS.
Sadly, this has already been done. For several decades with billions of dollars of compute.
> Neural networks in drug discovery: have they lived up to their promise?
1999.
"Over the last decade neural networks have become an efficient method for data analysis in the field of drug discovery. The early problems encountered with neural networks such as overfitting and overtraining have been addressed resulting in a technique that surpasses traditional statistical methods. Neural networks have thus largely lived up to their promise, which was to overcome QSAR statistical problems. The next revolution in QSAR will no doubt involve research into producing better descriptors used in these studies to improve our ability to relate chemical structure to biological activity"
> The use of neural networks in drug discovery started at the end of the 1980's when they were applied to various chemical problems including the study of quantitative structure–activity relationships (QSAR)
> Combining Decision Trees and Neural Networks for Drug Discovery
Want more robots and silicon? That's been a thing too!
> Computer-aided drug discovery and development (CADDD): In silico-chemico-biological approach
2006.
In biomedical arena, computer-aided or in silico design is being utilized to expedite and facilitate hit identification, hit-to-lead selection, optimize the absorption, distribution, metabolism, excretion and toxicity profile and avoid safety issues. Commonly used computational approaches include ligand-based drug design (pharmacophore, a 3D spatial arrangement of chemical features essential for biological activity), structure-based drug design (drug-target docking), and quantitative structure–activity and quantitative structure–property relationships. Regulatory agencies as well as pharmaceutical industry are actively involved in development of computational tools that will improve effectiveness and efficiency of drug discovery and development process, decrease use of animals, and increase predictability. It is expected that the power of CADDD will grow as the technology continues to evolve.
Biologists aren't slouches. The thought about using computers to process biological data has been "obvious" for as long as there has been computers. AI and biology have been a thing since the Dartmouth conference. A lot of things that could have been tried have already been tried, but I guess we'll keep trying until something works?
There ain't much room for miracles here.
I wish that wasn't the case. I really do. But it is what it is.
You do realize that neural networks have existed since the 50s, right? They aren't new, but they are having a transformative change now. It's night and day comparisons to what we had even back in 2017. ALICE as a chatbot was back in the 90s, but we aren't pretending it's anything similar to gpt.
It even says, right there in your quote: "It is expected that the power of CADDD will grow as the technology continues to evolve." This ~is~ technology evolving, and it's happening at a pace that people don't appreciate.
I believe the comment is in response to "well why didn't I think of that, it's so obvious" -- biologists have been thinking and trying to do this "obvious" approach for decades.
There's a long history of Silicon Valley people thinking they can just bring in their experience and with a bit of Moore's law brute-force their way into miracles.
Lots of Silicon Valley people in the late 1990s thought they could be on the ground floor with nanotechnology design.
The chemist Derek Lowe 10 years ago used the phrase "Andy Grove fallacy" to describe the optimistic vision that data + massive computer power = cure for diseases. "[T]he idea that understanding human disease and its treatments should be pretty much like designing a new chip or writing a new app", quoting the 2015 essay titled "Silicon Valley Sunglasses" at https://www.science.org/content/blog-post/andy-grove-rich-fa... .
Why Grove specifically? He links back to his 2007 essay at https://www.science.org/content/blog-post/andy-grove-rich-fa... : "So I see that Andy Grove, ex-Intel, is telling everyone that the drug industry could use some of that Moore's Law magic. I've noticed that people who spend a lot of time in the computer business often have an. . .interesting perspective on what constitutes progress in other fields, and we might as well appoint Grove the spokesman for their worldview"
Fully embodied robots and fully automated assembly lines would also help with the production barrier. Regarding the FDA, that is a valid point, but it could be irrelevant if the AI developed drugs are significantly superior to previous alternatives. In such case, FDA acceptance would be expedited, or even bypassed.
Eh, the counterpoint is individual scientists armed with AI to do research and probe targeted questions have succeeded in addressing rare diseases of personal interest.
What we just have no evidence for, to my knowledge, is someone with no medical background doing the same. Instead they wind up buying dodgy supplements and peptides.
While I agree the physical world is harder than code (I mean just look at self driving cars, Tesla fleet is around a trillion miles of experience and Cybercab is still geofenced to a tiny area at best and might still have safety drivers on standby)...
> Maybe if we have fully embodied robots and fully automated assembly lines to generate organic compounds and test them with extremely realistic simulations it may help slightly
How is "I disagree with this extrapolation of energy/compute needs/AI progress because it doesn't take into effect emerging restrictions and feedback loops" the same as doing my own extrapolation?
Re-read your comment it says "almost all industrialized nations are more worried about population decline than overpopulation these days. Which is why I'm not worried about superintelligence or data centers in space."
The only other point you made (and then violated) was "don't extrapolate too simply from past examples". Your simple extrapolation / reasoning by analogy was "we're at peak population when projections pointed towards population explosion, therefore we won't hit superintelligence."
Had you used clear analysis, reasoning, or argumentation rather than just baseless extrapolation (peak population, therefore superintelligence can't happen), I wouldn't have chuckled at your self-contradicting comment.
"In the past, the US government has always been able to take on more debt even when people projected it would run out, therefore we'll always be able to have more intelligence until we reach super-intelligence" is the same (and equally invalid reasoning) you presented.
You left out my first sentence when re-reading, though. My final sentence was not only referring to the one before it. Try increasing your context window, or read it like this:
"Beautifully shows how most predictions that extrapolate from past data are too simplistic. [...] Which is why I'm not worried about superintelligence or data centers in space."
yes, I read the first sentence, it was the point of contradiction in your other two. My friend, I don't doubt you have reasons to believe what you think, but your comment as worded is self-contradicting and lacking those reasons.
They said they think extrapolation is too simplistic of a method. Then said that is why they don't agree with the pushed extrapolation of LLMs becoming SI.
This is addressing the commonly repeated line of thinking that assumes LLM capability will continue to see the improvements that have been seen in the past so far.
And to bolster your point, people like Huang are surrounded by yes-men constantly. When you are a nobody, people criticize and keep you in line. Once you're running NVIDIA, people want power/money themselves, so they will tend to grovel and supplicate. Eventually, even the humblest begin to believe their own PR.
Those are "external influences", but your own gravitational field has warped them. Even the president wants your money and influence, and needs your company to do well, there is literally no one who will "give it to you straight", and it warps your mind.
The Roman commanders would sometimes hire someone to stand behind them and remind them they were only human for this reason (to avoid natural hubris), it was seen as naturally arising in these circumstances, and needed to be combatted to remain effective.
If robots take over, the same forces that have pushed the human workers to the brink will happen there too. Shuffled into your cell where the robots tend to you as quickly as possible, then move along. You think the nursing home is paying robots to sit around and chat? These things are depreciating assets!
Hah, all these posters think they are going to be tended by a gentle supermodel who comforts them and listens to all their problems. The reality is probably: you get to use TV / internet for your "social", a machine that can automatically empty your bedpan / waste, and a gantry system that can move stuff around, rotate you, and dispenses your medication so a human never has to see you.
reply