Graham's Number is notable for being one of the first, being an actual relevant bound in an unrelated proof (not just the output of a function), of teaching people the Up Arrow Notation.
To me, BB(10), TREE(3) etc. being so large is only minorly more impressive than GG(1) where I define I define GG(x) as "Graham's Number + X"
I don’t think such a bound in itself is interesting, unless the proof reveals some strategy to prove more. There are infinitely potential bounds to BB(n), why is this particular case BB(10) > G useful for understanding fast growing functions?
First, an upper bound would be more useful to understand BB. But second, out of all large numbers, does this one give any information? We’ll have to see. The fact in itself is useless.
To me there are the clear big 2 that actually seem to be innovating and pushing things forward. Meta almost earned a seat at the table with the open strategy then stagnated. Google obviously is the OG but at this point they seem in the same bucket as Amazon and Microsoft as "Big Enough To Matter and Influence Politics and Spend Billions To Hang Around". But they're nowhere near the same tier as OpenAI and Anthropic.
Oh and of course there's SpaceXAI which is Elon's personal money and reputation laundering corporate entity. Elon took over the "king of the reality distortion field" from Jobs as far as the rest of the industry is concerned, so he'll stay first in line for Jensen's chips and investor dollars until he's dead. But as a company what have they actually accomplished besides making the models with the sense of humour and judgement of 14 year old boys?
You can pick any 4 you like from your list. Heck you can even vote with your money on which would be in the top 4 (depending on how many align with your ideology). I am just speculating it will be 4 because of how all other sectors have at least 3 or 4 well-entrenched players. It seems to be by design. My guess is that it is driven by structure of the Military-Industrial Complex. That has at least 3 or 4 vendors in each sector, so as to not over-rely on any one company and create a monopoly in that specific industry.
Even before AI we used to say if you write code that you only barely understand, then it will be to complicated to debug. (and/or maintain)
Mochizuki was still one human and it required legions of other humans to unpack and untangle to confirm that it didn't lead to anywhere in particular.
AI is now capable of constructions so complex that no human or human team can unpack. And its ability to increase that complexity is growing while our human ability is stagnant.
meta-AI analysis cannot help. We (software professionals who use AI regularly) already know that if you run into a situation where a Fable/Astra-generated analysis reaches the limits of our comprehension/complexity due to their subjectivity, throwing more AI at the problem doesn't always converge.
There are many reasons to feel optimistic about AI, and ultimately its general ability to help science and mathematics.
I see no reason to feel optimistic about the future of mathematics and AI based on the current path of frontier labs, unless the misalignment Tao is writing about can be reconciled.
> AI is now capable of constructions so complex that no human or human team can unpack.
How can we possibly know this when we haven't even seriously started on the endeavor of actively reverse engineering these AI-generated proofs? That's a proper job for human mathematicians, because the AIs themselves are demonstrably clueless about what steps in a proof are genuinely interesting and load-bearing from a human POV. This is evidence of a limitation in AIs' capabilities, not of any kind of misaligned behavior. The fact that Tao actually uses that term in his complaint is deeply disappointing.
Not to mention, there are already (pre AI) machine-generated proofs that we've pretty much agreed not to try to explain fully, like the four-color theorem which ends up with brute-force verification of 600+ cases (down from close to 2,000 when first demonstrated)
> there are already (pre AI) machine-generated proofs that we've pretty much agreed not to try to explain fully, like the four-color theorem
Algorithmic verification is a very unsatisfying answer to the problem (e.g., surely it's not just dumb luck that every single case happen to have this exact property), but that's an entirely different issue than saying that no one follows logic of the proof method itself.
For the interested; the saying I believe you are referencing in regards to writing code / debugging is from Brian Kernighan, specifically:
Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?
The first major computer-assisted proof, of the four-color map theorem in 1976, was an example of this. It created a lot of controversy at the time. It used proof by exhaustion, i.e. essentially analyzing every possible relevant case, something that no human could do without the assistance of, at the time, a supercomputer.
In the last decade, we've seen rampant abuses and exploitation of open source projects and of maintainers. So much individual good will and generous efforts wasted because of corporate malfeasanse.
AND YET. If you "open source" a project, if you actually invest in BUILDING a community, you then become a PART Of that community - with the benefits and obligations that come with it.
You don't want the obligations? Fine. But then you don't really get to keep participating in the community. And if you act like "This is MY community, I get to do whatever I want", people are free to shrug, and say "maybe i'll go find a different community".
Which is exactly how I see people reacting to Zig right now. They're seeing that an active member of this community (which, as you said, keeps insisting "he owes them nothing because the community wouldn't exist without them") treats people he doesn't like, and they think "Oof i don't want that to one day be me."
Yeah I'm not defending the blog post as a great way to build the biggest bestest community. Clearly it isn't. But it's also pretty clear that's not the poster's priority. And that is defensible! They don't owe anyone perfection in community building or any other thing.
If they want to write a maybe slightly unhinged blog post excoriating someone they had an unpleasant time with... that's a choice! It's their choice. I don't think we get to demand a different one. We can agree or disagree with what they said, but we can't demand they say something else.
If you want that kind of power over someone you need to ensnare them in a binding contract. No such contract exists here.
> I'm not defending the blog post as a great way to build the biggest bestest community. Clearly it isn't.
This has been everyone's point the entire time. You agree with everyone here. No one is suggesting a binding contract exists, but a _social_ contract does exist -- and you concede that with the above recognition of how clearly this post violates it.
Putting aside that I don't agree with Bram (I've been using all the Claude versions he refers to and haven't experienced this), I do think it's interesting that there is no universally perceived golden sweet spot between "sycophantic" and "rude".
Many neurotypical people call neurodiverse people (software engineers) rude, while they think they're just being direct.
Many neurodiverse people call neurotypical people sycophantic, while they think they're just being polite and friendly.
It also happens across cultures (Eastern European vs. Western European; European vs. North American).
So I can easily imagine that when you have a software tool whose interface is language, but its user base is extremely wide across both cultural lines and neurodiversity spectrum, it's going to be basically impossible to nail a sweet spot.
You make it too friendly, and the nerds get mad. You make it too adverserial, and the normies call it rude.
I wonder what kind of communicator Bram Cohen is. Is he succeptible to this? From what I heard about his career, he's always been more of a solo programmer. Has he had to interact with other humans much giving feedback? Could it be that he asked the model/tweaked his prompts to ensure directness, and now he's interpreting that directness as rudeness?
> So I can easily imagine that when you have a software tool whose interface is language, but its user base is extremely wide across both cultural lines and neurodiversity spectrum, it's going to be basically impossible to nail a sweet spot.
> You make it too friendly, and the nerds get mad. You make it too adverserial, and the normies call it rude.
Easy: let the user set for himself how the model should be aligned on this axis (with some pre-defined example setups that the user can use or use as a base for an individual alignment).
like with most things, these things are both overrated and underrated.
are there performative jobs|tasks|employees|cultures? yes.
are most of the things that engineers think are performative and useless actually so? nope.
some examples:
* managers managing upward - feels useless - is actually the most impactful bang-for-buck for managers to give their teams space to operate without micromanaging
* sales and marketing. The best software in the world won't get known, bought, or used, without good sales/marketing. There is no meritocracy on quality. Almost no business succeeds through technical credibility alone.
* 1on1s. They may not add any value to you, but 1) you'd miss them when they're gone, 2) i don't know how else you expect managers to stay on top of employee concerns - just know "inately"? 3) they may matter A LOT for your teammates, and them being happy means your team will be happy
But first, in the phrase that you quoted, you do understand that in human society "trying to harm" someone is still a malicious act.
If I push a coworker at an office window, and it shatters and they fall to their death, there will certainly be some culpability to the building since the window "Should" have been able to hold the pressure.
But I will still be culpable to this.
Second, the threat of more harm is looming. Does the author know that this kind of prompt injection doesn't work anymore?
Either:
A) If they know, and like the principle of it, then every thread here debating their moral virtue is irrelevant. This is an empty protest that will be ignored by the AI model harnesses, and I simply don't see the value in it.
B) If they don't, because they're so hellbent on fighting AI that they're out of touch with the real capabilities of modern tools, then:
i) They are anti-intellectual and incurious and forming their beliefs on incomplete data, and therefore not credible. But more significantly:
ii) *They are a risk for escalation*. If the author realizes this kind of prompt injection can't work, who's to say they won't try to develop and inject more sophisticated attacks?
FAANG used to be the _dream_. Change the world. Work on groundbreaking tech. Solve harder problems than anywhere else. Get Paid incredibly well.
Then I guess a generation focused exclusively on that last part flooded the zone. I still believe that The Great Resignation of 2021 did more harm to our industry than COVID or any interest rate or VC changes.
It polluted the brains of most of the people in our industry from a missionary mindset to a mercenary, and it decimated big company's established cultural memory all to prop up a bunch of unicorns who will probably all slowly die over the next decade.
So now half of the people can't get a job, and the half that can are miserable. This was true BEFORE AI.
It's just a natural progression. When the company is a startup working on a new exciting tech it's chasing the dream and changing the world. When it's a behemoth employing 200k people, it's impossible for all of them to be chasing the dream. Probably like 90% of them would be doing extremely boring "keeping the lights on" tasks and ensuring this gargantuan machine does not go to pieces under it's own weight. It still can pay incredibly well, but it won't be exciting frontier work anymore. It just can't be, 200k people company can't move with the agility of 200 people company.
Its not like the CEOs unilaterally said "we want to suck now". It's the cumulative effect of many people optimizing for the promotion process within the company and then coaching their mentees to do the same.
They should be but its very hard. It's a culture and you can't exactly order a chain of command that grew into it and hold all the levers to just he different.
I'm not lionizing Sundar Pichai here, just saying the cumulative attitudes of 10k L7/L8 is really what sets the culture.
Every big tech company's embrace of AI is making all of their employees miserable.
Whereas if you're half-competent and at a startup, the AI is an incredible opportunity to try to leap ahead while the prices are subsidized (by the big tech behemoths fighting wth each other)
The reason is a complete inversion of Ownership and Agency.
For a decade of ZIRP, big tech convinced its employees that they're "changing the world", and what we did mattered. Sure the exhorbitant salaries and constantly rising stock value didn't hurt, but honestly other than the FIRE cultists, for most of us the difference between 200k/year and 800k/year didn't feel much day to day (other than the ability to buy a house or something, and feel safe with a retirement nest egg). No, most people were missionaries not mercanaries.
2021 was the first crack. The comps went crazy, half the industry turned over, and the ones who didn't felt a bitter sting where it became blatantly clear that all the new arrivals were just in it for the $$$, and the companies were willing to pay for the backfills but not to reward the loyalty of the missionaries.
Then came the yearly layoffs, chipping away further, and reminding every employee that they're at the mercy of a spreadsheet and the whims of people 3 levels above them in the org chart, in spite of the economic reality of their product, or their personal productivity.
And now we're here, and it's clear that all of the above is still relevant. The old-timers that hung around see that their personal output doesn't matter, their product's PnL doesn't matter. All that matters is 1) the company's AI strategy (and if they're not part of it, they're secondary), and 2) tokenmaxing.
How can anyone find joy in this environment unless they're purely in it for the comp?
I couldn't. I left my big tech job in December after 15 years, and have not been this happy at work since pre-COVID.
> the difference between 200k/year and 800k/year didn't feel much day to day (other than the ability to buy a house or something, and feel safe with a retirement nest egg)
I can’t believe I read this sentence, lol.
800k is the ability to buy a house and support a family on a single income. Do you see so many people lamenting the days when this was possible? So many memes about the lifestyle Homer Simpson could provide, and may modern families can’t? 800k makes it possible.
It’s a huge lifestyle upgrade, especially if your partner wants to do something artistic, academic, or otherwise less profitable.
While I mostly agree, $200k makes that possible too, if you play it right. For example, go remote, move to the countryside, let your spouse rear the kids or the dogs or whatever.
But yeah, "no difference between 200 and 800", while spelling out some MASSIVE differences is quite a statement.
How many do you think build that portfolio from $200k/yr? The point is an extra 600k, at least for several years, is life-changing when managed wisely. I could perhaps see the GP's point about "day to day" feeling, if you acclimate to the baseline of financial security that having that much money buys you. I'm only assuming though, having never had the opportunity to experience GP's claim about that kind of comp.
The idea that there is not much difference between 200K and 800K per year is absolute nonsense. I feel like its one of those things that are so stupid we are all dumber for having heard it. I don't know if its just one of those things where people on HN seem to virtue signal as if they are the most minimalists, live in 250 square foot tiny houses, live of radishes they grow in their microgrardens or whatever. This is like people that say, "eh, you don't need to make more than $95K."
>2021 was the first crack. The comps went crazy, half the industry turned over, and the ones who didn't felt a bitter sting where it became blatantly clear that all the new arrivals were just in it for the $$$, and the companies were willing to pay for the backfills but not to reward the loyalty of the missionaries.
Also SVB collapsed in late 2022, notice that AI hype started right after.
Sigh, if it actually collapsed, it would have been fine. Summers saved it depositors before it collapse. I still don't get how that was not a story on a par with 2008 ( or maybe it wasn't because the fallout was avoided ).
The Three Mile Island accident is also dramatically exaggerated in the public conscience with its severity and risk factor - solely because of the default fear of Nuclear.
Oil, Gas, Coal, and random chemical plants have had much more significant accidents even in the US, but never made a blip in the public's minds.
Aren't France and Canada the ones to learn from at this point with regards to safe nuclear operation?
Consider TREE(3) -> https://www.youtube.com/watch?v=3P6DWAwwViU
Graham's Number is notable for being one of the first, being an actual relevant bound in an unrelated proof (not just the output of a function), of teaching people the Up Arrow Notation.
To me, BB(10), TREE(3) etc. being so large is only minorly more impressive than GG(1) where I define I define GG(x) as "Graham's Number + X"