My extremely uncharitable view of of these kinds of people, who now infest the American AI scene, is that they got cyber bullied by domain experts for saying the most ridiculous crap on repeat, and have therefore decided to spend their life on eliminating domain expertise.
I wouldn't go that far. But as we've seen repeatedly, people who have enough qualms with how these labs operate just leave. And the people who stay behind and rise through the ranks are the ones with either very little conscience or some bizarre world view that rationalizes everything.
No, that has the opposite effect: it's very, very common to see LWers write about having a bad time in school in late adolescence and how later finding like minded people online was powerful community bonding.
Eliezer himself first blogged at 17 about how miserable school was for him which led to him dropping out/home schooling after 8th grade and discovering the Extropians listserv. Ziz of the Zizians is another (infamous) example.
We're talking about people that unironically think eugenics based on IQ is a good thing. I think it's very effective altruism to socially ostracize such individuals.
My view - there are a few outcomes
1. AI is never fully realized, but domain expertise is derated and society stagnates (bad ending for everyone, I think unlikely)
2. AI is never fully realized, but empowers people with domain expertise (Google wins, weird rationalist nerds go back to their LW cave and leave us alone, goodish ending)
3. AI becomes real and kills everyone (bad ending, but at least I don't have to read people claiming that the Harry Potter fanfic isn't niche)
4. AI becomes real and we live in the Culture universe (people like scam Altman get slap droned, most domain experts have the time of their life working with Minds on really interesting problems. Best ending)
The only outcome where we get swirlies is the first one, and we can take solace in the fact that all of humanity is getting swirlied with us.
That's a niche even among people who read fan fiction, which is a niche among people who read long texts online which is a niche among people who read the anglophone internet which is a niche among people who were on the internet when that type of "discourse" was popular which is a niche among people on the internet now.
You can (and probably will) quibble with any one of these but of course this is a niche. Thinking it isn't is commonly what people in a certain bubble end up believing, though.
Harry Potter has 600K works on AO3 and 852K on fanfiction.net (with plenty of overlap, granted). That's #1 on Fanfiction.net, and in the top 5 for AO3. AO3 is #53 globally, Fanfiction.net is #572, and Wattpad is #323 by Semrush. Given all that, calling HP fanfiction niche is pretty funny.
And no, not a member of any cult, just someone who enjoyed a send up of Harry Potter.
Fanfiction.net is a niche. None of my scifi loving, tech friends are aware it exists. And neither did I (or was barely aware or it, anyway), until you just mentioned it. I'm not young either, and I'm more online than the average (I know of Something Awful, Slashdot, 4chan, etc).
A while ago, a friend told me about Harry Potter's Rationality fanfic and he didn't know who Yudkowsky was, he just thought it was some random author doing a parody of HP. LessWrong is of course very niche, my friend didn't have the faintest idea of what LW was.
You people are so quick to correct me that you don't understand my argument: HPMOR is not niche in the world of fanfiction, and this doesn't require fanfiction to not be niche. You're also answering stats with anecdotes, so I think we are done here.
You didn't claim hpmor isn't niche in the fanfic community, you claimed it isn't niche on /the Internet/, and implied that basically anyone in a technical role would have heard of it. I'm sorry that you're upset to learn that very few people care about sophomoric LW thought experiments, but that's the reality of your existence.
> "niche"? It's literally one of the most read pieces of fanfiction of all time.
Where did I say it wasn't niche on the internet? Though you seem to be more interested in personal attacks (I'm not even a fucking EA/LW person, but go off lmao). Good day
>>> having read a few chapters of HPMoR, which I agree is standard for anyone who spends enough time on the Internet
>> You have to be quite unaware of your bubble if you think knowing about a niche Harry Potter fanfic is "standard" for people on the Internet.
> "niche"? It's literally one of the most read pieces of fanfiction of all time.
You jumped into a conversation about the ubiquity of something for people on the Internet. Either your claim is about that, in which case it's wrong, or it's not about that, in which case it's irrelevant.
I'm not sure what your argument is. I don't have anything against fanfiction, but a couple of sites with a few million users falls under my definition of niche. "Extreme" might be taking it a bit far, but it's a long distance from mainstream.
Literally over half the Earth's population doesn't read English. You need to touch grass. I work with extremely technical people in physics and AI and none of them have read any fanfic of any kind. Most of them have never seen a meme outside of me showing it to them.
You're so offended at the idea that your experiences are not generalizable that you're failing to read the thread. The OP that started this said "We all use the same Internet. I would find it strange for anyone who's seriously interested in AI to have never heard of Yudkowsky or Scott Alexander or the Rationalist movement."
Part of my comment addressed that. It's rather amusing that you completely skipped the first part, but I think that's par for the course for the Internet rationalist crowd.
Given that we have an embodied intelligence that is capable of being reasonably good at physics, it would be foolish to state categorically that AI can't do physics. I am however skeptical that LLMs can do physics. My experience is that it's really great at doing the stuff I can't be arsed to do, and is therefore very useful, but it has very poor "understanding" of physics
Can you elaborate on why you think the OP is incorrect? The hit piece on the Ypstilanti data center is quite an embarrassment for any honest news reporter.
There's complex politics around hyperscaler data centers, and I'm sympathetic to the complaints against (though in full disclosure I am biased towards building anything, having seen my country held back for years by people opposed to building anything). But the UMich supercomputing center isn't a hyperscaler data center, and it's the height of disingenuous, cynical and dishonest politics to claim that it is. It's an academic research supercomputer. And FYI the protestors have been harassing UMich professors and their families, so sorry. I'm not going to be polite and give them any benefit of doubt. I know the people they're harassing personally and professionally and they're wonderfully kind and intelligent.
If the supercomputing project dies because of this, yellow journalists like 404 media have scientific blood on their hands.
> But the UMich supercomputing center isn't a hyperscaler data center, and it's the height of disingenuous, cynical and dishonest politics to claim that it is.
It may not be a hyperscaler data center (I actually don't see that as the argument anyway) but it is a data center and that's part of the issue in today's landscape, irrespective of if you pick a particular side.
> It's an academic research supercomputer. And FYI the protestors have been harassing UMich professors and their families, so sorry.
Oh, I get it. I worked for a defense contractor for a number of years and every Thursday we had a small group of protestors that would harass us if we walked across the street for lunch. I engaged a few times to ask them if any of them had ever taken a commercial flight as the building they were protesting in front of built most all of the ATC systems the US operated at the time.
But, I'm still going to argue: it's not that simple. While I'm in support of the US continuing R&D through use of new HPC facilities the use cases are not cut and dry. Nuclear weapons are a zero sum game and while I think many would fully support continuued research into improving nuclear weapons systems and scenarios I don't think it's all that productive to continue to invest heavily in expanded nuclear weapons development. Especially having now seen too many administrations being handled by old men who don't have much to lose if they destroy the world.
But, to state "yellow journalists like 404 media have scientific blood on their hands" sounds fucking deranged to me.
Indeed that is true, which is the point of both OP(or at least my takeaway). Attaching a moral value to trade deficits or surpluses, as the current administration seems to be in favor of doing, is fairly nonsensical.
I can't speak for that guy, but I'm a physicist and work in clean energy... So it's not too hard! That said, I usually am closer to 10M on days I do heavy coding, so not nearly that bad.
Have you seen the average physicist code? I've crawled around in the muck of Fortran codes written in 1970 with unreadable 8 character variable names (the first of which encodes the type), with implicit typing, zero comments, computed go-tos, SAVE, the works. Tests? Never heard of it. Bugs everywhere. You would be far less sanguine about physicists coding if you looked into some of the "flagship" codes used today. And believe me, it isn't a "boomer bad" phenomenon. Every incoming student I've had in the last decade writes shit code too.
My (only somewhat facetious) opinion is that physicist access to programming languages should be controlled like doctors' access to opiates.
0. It's not about "looking pretty", it's about being nigh impossible to understand or reason about. There is a 1000 LoC subroutine that I'm convinced was ported from FORTRAN 1 unchanged that cannot be understood by anyone under the age of 50 that underpins one of the "flagship" physics codes in my field. It's a fucking nightmare every time I have to worry about that stupid thing. I remind you that this was 100% the product of a human brain (pre computer age, pre social media, and I'm told quite the genius too). Perhaps one should not place the human brain on too high a pedestal and deify it.
1. Why did you take from my OP that I tell codex "write a climate simulation code, make no mistakes" and go suntanning on a beach in the tropics for the rest of the semester?
2. Perhaps you have a different experience from me in writing HPC codes, but my experience is that > 90% of the code is boilerplate. I find GPT 5.6 can be prone to overengineering, but with a little steering and good judgement it generates very nice interfaces and high level code. I just have to think about the solver structure or metastructure.
3. Even with core numerics - pre-AI, it was a bunch of iteration going back and forth between code and optreports. Now codex will just do it. I suppose this may seem grim to you if you loved decorating every variable with !DIR$ ASSUME_ALIGNED, and manually batching array operations or whatever, but I didn't and I'm glad I no longer need to.
4. I'm now highly motivated to write tests, and AI makes it way easier to write the immense boilerplate around good tests (sorry not sorry, my {FUNDING_AGENCY} program manager doesn't give a flying fuck what my test coverage is, and my next grant won't depend on that in the slightest, so pre-AI I did the bare minimum. You can argue that the results will be worse, yadda yadda, but the incentive structure that {FUNDING_AGENCY} has in place don't promote good software standards, and my career never suffered for it)
5. I can generate docstrings with high accuracy (see the above)
I think it says more about the person reading it as an implicit insult than it does OP. I am, I think, a fairly competent scientist, and know many great scientists. I know very few great scientific communicators.
If 90+% of whatever society you are from was killed off by disease, and the people who came across the survivors treated them as little better than cattle and systematically destroyed any remaining trace of your society, how literate would history judge you to be?
Literate has a definition that requires the ability to read and write. The North American native population was, by definition, illiterate. They had no written word.
It doesn't mean they were stupid, they obviously picked it up quickly when they were exposed to it
reply