Even if NVidia crashed and there ends up being an oversupply of GPUs or TSMC fab slots, this doesn't change that OpenAi and Google still need memory for their own hardware accelerators, so the price of boards wouldn't go down by much
Part of me wonders (maybe naively) if with falling cost of AI inference and improvement of models means there is an oversupply of inference infrastructure. since we may reach a point where no one needs more and everything you could want can be done by the machine and adding more compute for most users is not any kind of benefit (like it was before LLMs there was an obvious ceiling where the majority of people did not need more compute).
I know Jevons paradox is a thing and I could see it going the other way as well but the first possibility seems to be ignored and I am not sure we can. Eventually demand for training might dry up as well if are models become largely very good and only a bit of fine tuning is required for most applications.
The author is a developer working on software that scales to 1,024 cores and greater -- which is within spitting distance now thanks to 256-core AMD EPYC processors. If you count hyperthreads as a "virtual core" then a dual-socket system gets you there!
That blog made me realise that almost all software can't utilise 1,024 cores, not even close! Microsoft SQL Server, as a random example, tops out at 64 cores for any single query.
Similarly, people couldn't utilise a machine that big, not with pre-AI software, except for certain embarrassingly parallel codes such as simulations and 3D graphics rendering.
Now... NOW... there's no upper limit. None. One person(!) can cheerfully utilise millions of cores, billions even, and there's no end in sight.
If you think one conversation thread in "ChatGPT" is all we need, you're not seeing the big picture of what's possible here.
Those math proofs OpenAI produced were created with tens of thousands of agents running in parallel, orchestrated by thousands more agents, and overseen by hundreds of agents, all triggered by someone typing as little as a sentence or two to start that avalanche of compute!
With that kind of AI power at your fingertips, huge, unthinkable projects become a fun exercise for the weekend:
- Translate the entirety of Wikipedia to every human language.
- Rewrite the entire Linux kernel to Rust in one go
I am arguing that their is an quantity intercept (The demand for compute if it cost nothing would be a finite number and may go down rather then up with advancements in cheaper inference and accuracy).
Sure you can do all these things you mentioned but eventually even if it was free they would be done and doing some of these things your self would be annoying and a waist of time. Once some has formalized a proof why would some do it again and why would some one rewrite a program again, it will still take time and human effort over just using it. If you just spend 1 minute with each agent of your say 10000
I really don't think most people want to be running tens of thousands of agents and would not find a need for that many agents. if you it takes you just 1 minute of looking at output of your agents that would take you 1 year and 52 days. I think you have to admit there is a limit. Though I could find busy work for my agents, you could do the same with computing before hand such searching for primes, there is a limited thing that people want out of their machines is my point and people who would run such things would be a minority if they exist at all.
That’s not how it works. Agents coordinate agents, validate results, fan out work, coalesce work, and hand off to other agents. It’s agents all the way down.
Your other point stands tho - there might be a point where there is not much left to do. Most problems solved, and remaining require more agent hours than there is energy in the universe. Or not. Who knows.
> That’s not how it works. Agents coordinate agents, validate results, fan out work, coalesce work, and hand off to other agents. It’s agents all the way down.
True but the ratio to agents to sub-agents not going to be that high (at least that is my experience so could be wrong). I have seen gas town but that genuinely seemed crazy. My agents have never had more then 3 sub agents, but I am not a top AI power user so not sure.
At any rate it seems that that though I imagined free compute in my example's I did not assume set-up time and harness prompting and the other bits around management are time free.
What I am trying to get at is obviously we will not see free compute, but what does the demand curve look like in actual fact, how much do I actually need solved most of what I could possibly want with agents using sub-agents.
I does also still feel like human in the loop is needed for development and other tasks. All the maths proofs it produces are not elegant or easily understandable by people, huge, not decipher and difficult to know if it even triggered a subtle bug in the theorem proving language that makes it difficult to truely know if the proof is good, I am interested in mathematics but I would probably not bother with running an agent to try and do these things or even ask my Hermes agents or what ever to do it as curiosity and if I did I would probably ask it to stop if it cost me more then a coffee. For useful work to be done humans have to look at these things and understand them, and human motivation to do a lot of these things. (additionally it's still not cracked a lot of coding issues around bloating code bases and such and human intervention seem to still be needed).
Finally on researchers using this while this might be a little bit more exciting then Mersenne prime finding or protean folding, How different will the funding going to computing resources going to these projects look now and after vs how they where pre AI?
I don't see how it helps to think so narrowly about the computation needed solely for training when we are clearly headed to a future where tons of realtime applications (think robots) need to do insane amounts of processing ASAP while doing time-sensitive tasks.
IMO its copium to think that chips which have 10x more demand than supply are suddenly going to be easy to get as "AI is eating the world" like software did over the past decades.
I think there is a question of how much of that demand is real. If GPUs are sitting unused in boxes and/or if NVDA has been artificially juicing demand either through basically buying them from themselves or selling them to China through an intermediary then the 10x demand comes into question.
That leaves the real buyers who are also starting to run low on capital.
Finally, demand doesn't need to go to zero for there to be a price correction. Just dipping from insane demand to elevated levels would lead to a significant price drop.
You are definitely right, mrgordon - this isn't an infinite scaling, "one extra copy cost essentially nothing," software issue. Video cards are physical. You can't just sell more copies with almost zero marginal cost like software. Each video card is a physical good that is subjected to all those physical laws.
I was not trying to deny the corporeality of GPU's (or cpu and memory), only that there might be an oversupply given the rate of buy up of these things and the amount of start ups who have brought these things and will fail and liquidate or liquidate hardware as they pivot or go into just offering inference.
Secondly that the frontier model providers will not necessarily want to buy these up as the aforementioned lower demand for training (or costly and memory intensive training) and cost of inference goes down (and possibly requests as the models get better or at least plateau).
Further TPU's and if model weights do eventually stabilize then ASIC's might make more sense.
And lastly as models improve and make less mistakes the requirement trying things in parallel traditional compute may also go down. I think their is a quantity intercept (The demand for compute if it cost nothing would be a finite number and may go down rather then up with advancements in cheaper inference and accuracy) for compute both AI inference and traditional compute.
real time visual data processing for these applications seems to me to not be that expensive and the thinking part will be done as a swarm. There is some real quantity intercept for the demand of compute that does change over time but one can imagine it going down with better inference not to mention ASIC's.
I think there are two sides that want a particular outcome one want gpu's to get cheaper (like I admit I do), others want there investment to work out (both might want to huff copium at some point, who is huffing here is not easy to tell).
Not so easy. The problem is that someone can "just" generate a lot of documents with different or contradicting predictions or idea and then selectively release some.
Yes, you can. So maybe we'll see an adoption of that paradigm, or another. The value of a ZKP is that you can optionally prove some facts up front about the file, whereas a normal hash is all-or-nothing. You can't reveal part of the information because you need the entire contents to properly verify the hash.
> we are going to build the best onboarding experience in the programming language ecosystem
> This can only be done if we commit to unix principles
> write programs that do one thing and do it well
Rust is so approachable because Cargo does not subscribe to "do one thing".
In fact Cargo does a bunch of things. Which is very convenient for newcomers, because they only need to learn one thing instead of a bunch (and how they interact).
The same holds for a GUI like a file explorer with buttons and interactions like drag / drop or marquee select, which is a bunch simpler than chaining 3 programs together to delete 30 old files and move some folder into a subdirectory.
It also explains why the most successful tools within the land of Unix philosophy are stuff like AWK, which really don't just do one thing. Or why PowerShell is less usable than GNU. Or why Emacs is an 'operating system'.
It's also a reason why the installation experience for Haskell is so bad: there are a bunch of programs and users don't know where to start and what they need.
> Cabal and Stack: which one should I install?
> We recommend installing both.
One thing and one thing well.
Atleast it seems they've added a combined installer for these tools since the last time I've checked.
I disagree. About a decade ago I made some tooling to extract and analyse code from Haskell packages. That's easier said than done, given the use of pre-processors, TemplateHaskell, language extensions affecting the syntax/grammar, using external files like `.cabal` to specify such pre-processors/extensions, etc.
I went through several iterations, to get something that I could throw arbitrary packages at from Hackage, GitHub, etc. The result was a hacky mess containing a GHC plugin to dump Core code, a library to dynamically generate and evaluate strings of Nix to handle dependencies, shell scripts that piped `:t` commands through GHCi, and so on.
It was annoying, not because of the effort involved (I was doing weird, bespoke things; so it's no surprise it needed weird, bespoke code to be written); but because all of the functionality I needed already existed, just buried deep inside GHC/Cabal/etc.
Admittedly the GHC API has gotten much better in recent years; but there's no way the developers of "one tool to rule them all" can predict all of the different things that users will want it to do. Indeed, it's probably wise for such tools to avoid supporting/maintaining particularly weird/niche uses!
Having a collection of tools instead, which "do one thing and do it well", side-steps that problem entirely: users with weird use-cases are empowered to plug things together in a way that suits their needs, without needing the devs to add special features/flags just for them (or maintain their own forks with such flags, etc.)
Size of programs binary, memory usage, heavy runtime and the size of the compiler can be very painful and possibly non-starts for some things as well. I do like Haskell but a lot of things I could never use it for because of those issues.
> It also explains why the most successful tools within the land of Unix philosophy are stuff like AWK, which really don't just do one thing. Or why PowerShell is less usable than GNU. Or why Emacs is an 'operating system'.
I am not really sure this is true. I find my self composing Unix tools (partly because I don't know AWK that well lol) and I think 'do one thing well' when can mean easier time thinking how things interact and from primitives rather then one big opaque program. I agree for this to work things have to sit well together and adherence philosophies and respecting the boundaries of the other programs is an important thing to do.
Cabal and Stack step on each others toes constantly and it does seem confusing. I am a masochist so use nix on top of this and there a a very many ways to go when you do that for Haskell projects.
What point are you trying to make? The comment I responded to added nothing. Your snarky low value response to me is to suggest I believe expanding the nothing by use of AI would be good? Why would you make such a bad inference or worse joke?
There are like 5 gazillion books to read, even without AI. As a reader to curate your limited time you cling to every indicator of quality you can get.
And authors not bothering to write their own books or publishers not caring enough to get someone to design a good cover (which you would think would really be in their own interest...) is a good sign that the work is of low quality and that I'd be better served to just read something else
The judging for the "newest" seems a bit odd - for viral toy I answered 'Labubu' which the AI dates to 2015, which might be when they were first introduced to the market, but not when they had their viral hype.
For newest palace I answered Berliner Stadtschloss which was opened in 2020, but the AI dates it back to 1443
Both dates are actually "right" from the game's point of view: Labubu was created in 2015 (the viral hype came way later), and construction of the Berliner Stadtschloss started in 1443, the 2020 opening is the reconstruction.
In Past and Future, what counts is when the thing first came out/was created, and the dates come from the Wikipedia API. I agree it's not obvious when the question says "viral" or "newest" though, so I'll make those questions more precise. Thanks again for playing! :)
Similarly it asked for Formula 1 World Champion, in the "most recent" game, so I put in Lando Norris, who won last year, but the answer is Max Verstapen because he was born in 1999 and Lando was 1997, it took me a few minutes to even figure out what the leaderboard was saying..
The author is really missing the forest for the trees here
> When it is present, WebAuthn attestation tells the site the make and model of the authenticator, identified by its AAGUID, with a certificate chain from the vendor to back it up. It says nothing about your OS, your browser, your extensions, whether your phone is rooted or jailbroken, your firmware version, or even what colour it is.
So if I was Netflix or a bank or w/ever I'd only allow new registrations from "trusted" authenticators, which would incidentally only run on iOS and play-protected Android.
A closed-firmware USB key is really no better option.
Dear Yubico / Kensington / Google Titan, can I export my private key components to this other device, or even print them out on paper for storage?
No - you're just an untrustworthy user granted the fleeting privilege of using BigCorp devices. Now stop asking such questions or you might have your attestation revoked.
Not only are Yubikeys agnostic, they cleanly slip from host to guest and present exactly as you expect, without a technical fight. Passkeys? I assume they could do this in a few years..
People in Iran or China don't rely on Facebook and Instagram.
Before Facebook penetrated Europe in ~2010, each local market was dominated by copycat services like StudiVZ or Hyves, which then vanished within 2 years.
When the US floated the idea of banning TikTok, some people felt compelled to download Chinese social media apps.
If a country was to ban Facebook from operating in their country, people would be on an alternative 2 weeks after.
There are reasons why a Facebook ban would be hard to enact, but dependence on the product is not one of them
I think you’re not aware how deeply engrained WhatsApp and instagram are in Brasil.
I think dependence is definitely a problem because everyone would get mad at you for having to switch. Sure in the long run it wouldn’t be an issue but immediately you will lose some voters
reply