As a theorist in AI, the premise is funny but also really sad. And yet it is a very pervasive premise in the field.
OpenAI is not doing science. They are building a big shiny thing, showing it off, keeping it closed, and making money off of it. That is not part of the scientific process.
It is as if, every time SpaceX launched a rocket a little bit higher, every aerospace department ooh'd and aah'd and threw up their hands: "we don't have the resources to build a bigger rocket; how can we do science?"
Unfortunately, the research field of AI is diseased: it places way too much value on showing off big shiny things above progress in scientific understanding. But there is a ton of progress available to be made by small teams on small budgets. There is so much we don't understand about neural networks alone, even small ones.
What? OpenAI is absolutely conducting science amongst themselves. They are certainly engaging in the hypothesis -> test -> result scientific method, which is science in its purest form. Sharing the secret sauce is what is done or expected in academia, but that doesn’t mean that OpenAI isn’t conducting science behind their own walls. You’re conflating institutional academia with the scientific method.
If I go home and test the boiling point of water in my garage and record the result privately without publishing a paper, am I not conducting science? What nonsense.
By the "scientific process", I mean the way scientific knowledge as a whole progresses over time, i.e. the main undertaking of the target audience of the paper.
You're right that "OpenAI is not doing science" was hyperbolic, but I tried to clarify the aspect of science that I'm referring to.
Although this is a reasonable definition, it would mean many people who are referred to as scientists in practice would have to be considered non-scientists. Refusing to publish data and code is widespread in many academic fields, even in cases where academics sign documents promising to provide these things on demand.
For example, at minimum psychology, epidemiology and climatology have this problem. In climatology attempts to get the source code of models and data underlying papers has led to some of the biggest dramas in the field e.g. the Climategate hacker appears to have been motivated by the closedness of the field (similar to the llama leaker).
Given that society is clearly not ready to reclassify large parts of academic work as non-scientific, it seems like this definition cannot work even on its own terms.
"Science" is not solely the scientific method - it's the democratization of your methodology to enable the replication of the results, towards the goal of determining their soundness.
Yes, that is exactly the point. "Research" and "Scientific Research" are not inextricably linked - I work in an R&D department, and not all research we do is scientific. We do have scientific research, but it's handled very differently from the PoC type of work that occurs more broadly. We are forced to choose between handing out trade secrets in exchange for gaining more soundness in our theories, or keeping them sheltered and hoping our observations hold up. That doesn't mean our research can't inform our decisions towards profitability, only that it's not scientific.
Edit: I mean, c'mon - the final step to the scientific method is "communicate your results". I'm surprised this requires explaining.
Alchemists were doing research 600 years ago but they never shared their results with others and used arcane syntax to record those results which nobody else could understand. This didn't lead to much progress in the field of chemical knowledge, because it was just continual reinvention of the wheel.
The scientific (and then, industrial) revolution didn't begin until researchers started sharing their results in open literature form, or at conferences, or within bodies like Britain's Royal Institution. Dissemination of knowledge is absolutely part of science.
Similarly, a fundamental necessity in science is independent replication of results, and that's definitely not possible with closed secretive research done in the alchemical mode.
What if the science is disseminated within a closed group? For instance, suppose the classified research is shared with other people with a security clearance. That still wouldn't be science? Or suppose scientists at universities shared their research with scientists at other universities, but not the general public: that wouldn't be science either?
What happens when you have an authoritarian system in place controlling distribution of results, is that everyone in that system quickly learns that if they question the reliability or wisdom of the research, their career is over. So they go along and don't subject the results to the kind of scrutiny science relies on, because making the boss look bad or embarrassing the institution isn't an option in that situation.
There are many examples, such as the secretive public-private DOE FutureGen program aimed at "zero-emission coal-to-hydrogen plants" which wasted about ten billion dollars over twenty years with nothing to show for it (and whose data is still not available), the Trofim Lysenko plant breeding program in the Soviet Union which wrecked Soviet agriculture for decades (and falls into your latter category) and so on.
Another example could be the Challenger Space Shuttle disaster, although that's more an engineering failure than a science failure, but it had similar underlying causes, i.e. managers pushing engineers to go along with a flawed protocol.
That's an echo-chamber, especially if the closed group is consistently aligned in their goals so that they're predisposed to want the same things from the research. The entities would have to be arranged so that they're asserting checks-and-balances on one another.
Exactly. They are applying the scientific method, but they aren't contributing to science. it will be science when it's open and people can integrate the new knowledge in a broader set of knowledge.
Science is knowledge. The etymology of science is to split stuff and understand how it work. But from late 14c in the more specific sense of "collective human knowledge".
"Private" science is not science. It's some esoteric stuff.
I don't buy the argument that "public" science is the only real science. You can define it that way, but I don't see any reason to.
"Public" science is just a special case of "restricted" science, such that the set of people the knowledge is restricted to is equal to the set of all humans.
I think this might get philosophical quickly, but to really understand where you're at, you need to access all knowledge. Everybody is standing on the shoulder of giants. It's a collective effort that happens through socialization and long-standing institutions like universities.
An essential part of doing science is sharing results and waiting for others to challenge, expand, or falsify them. If you skip this step, you're not truly contributing to knowledge, although you may still create new knowledge in other ways... but not all new knowledge is necessarily scientific, it can be private, esoteric, commercial knowledge.
Take the example of Ida, a 47-million-year-old lemur fossil known as Darwinius masillae. https://en.wikipedia.org/wiki/Darwinius it was "discovered" and hailed as a transitional species bridging primates and mammals. Really a huge contribution to "science".
But here's the catch - Ida was actually found in 1983 and held by a private collector for 25 years, who didn't realize its significance. So we had no idea of this "discovery". It wasn't science, it was a good on a market.
OpenAI is the same. We might enjoy to toy with the technology. But I don't find it contributed to science, yet
I love to get philosophical, so I'm gonna throw this thought experiment here:
Let's say that there was a highly advanced civilization that existed way before the current civilized society. They performed scientific experiments and gained a lot of insight, but a catastrophical event erased them from the face of Earth together with all their knowledge.
Did that civilization engage in science?
If you think that they did, indeed, engage in science, then what is the difference between them and any other closed community that performs scientific experiments?
Or, if you think that they didn't, in fact, engage in science, then does that mean that science is only science if you and I, today, can gain access to the knowledge derived from the experiments? Therefore, science is defined by its utility to us?
> catastrophical event erased them from the face of Earth together with all their knowledge.
This thought experiment seems to introduce a huge variable which eliminates what I think might be a key factor in what many readers believe to be the practice of science.
> what is the difference between them and any other closed community that performs scientific experiments?
That difference here, and the key factor, would be intent. The catastrophic event eliminates any meaning behind what they accomplished or practiced in regards to why they practiced science in the first place.
Practicing it behind closed doors may appear less like science for science's sake and more like, say, an economic pursuit that coincidentally involves the scientific method.
That could be why the conversation appears so split at the moment. It's hard to claim science is about intent or a long-term purpose that, were it removed, it would bar a scientific activity from actually being Science proper. That's a very subjective thing.
> The catastrophic event eliminates any meaning behind what they accomplished or practiced in regards to why they practiced science in the first place.
But if they didn't suffer a catastrophic event, we would call their activities "science"? So, because they did suffer a catastrophic event, what they did is not science (or is not relevant, since you avoided answering the question directly)?
And if our current society was destroyed tomorrow in a catastrophic event along with all our science, and a new society arisen from the ashes that performed scientific experiments, would that mean that what we did wasn't science? Or that what the new society is doing isn't science?
I know I'm streching things here, but words (should?) have meanings in all possible scenarios, and thought experiments are used to decide meanings in such edge cases.
> Practicing it behind closed doors may appear less like science for science's sake and more like, say, an economic pursuit that coincidentally involves the scientific method.
Does "not doing science for science's sake" include only monetary motivation? Would it also mean that someone who primarily wants to achieve status and fame as a scientist also isn't doing real science? Or are they exempt from the rule, since they would most likely release their knowledge into the public (which brings us back to the "science is only science if the current society benefits from it")?
I believe the “advanced civilization VS closed-doors science” problem you propose is inherently faulty. The final step of the scientific method is “share your results”, and they did that. It cannot be accessed anymore, but they attempted to share it and it benefited their people at the time. However, closed-door science benefits no one.
Take, for instance, the Greco-Roman mix for concrete. They produced that scientifically, repeating experiments with their volcanic ash-filled cements to make the best they could, and shared those results across their territory. While that knowledge was lost for centuries, it was still science.
However, a cement company producing a better product and not sharing the production is not science. It’s an economic pursuit. Its goal is not to help everyone to produce a better product, but to produce a better product and hide the results. Even if it uses most of the scientific method, if it doesn’t share its result its not science.
And no, monetary stake isn’t the only reason someone may not be doing science properly. A major one we see time and time again is the desire for science-based renown.
For instance, Einstein is renowned for being one of the smartest men alive, who shaped a lot of our modern understanding of the universe. He may ultimately be wrong, but we have yet to prove that and in fact continue to prove he was decades ahead of his time.
Contrary to Einstein though is Edison, while he did do quite a bit of real science, he was less focused on actually advancing our knowledge and more focused on his own image. For instance, he invented the electric chair solely to try and slander Tesla’s AC, and tortured animals in public to try and breed fear.
Additionally, I will throw my hat in the ring and say most of modern academia doesn’t pursue science, as they push for ever-restricting holds on scientific papers in the pursuit of financial gain. While simultaneously depriving the actual researchers any of that gain, money which could surely help raise a lot of institutions into doing more effective research. Instead, they profit off of and restrict the free flow of knowledge to line their pockets, disabling millions of researchers worldwide from accessing it and imposing harsh prices on non-academics who wish to learn.
In the end, when it comes to what is and isn’t science, I will posit a question. If you never heard about it, and can’t reproduce it yourself/watch someone reproduce it, did it really happen? Or has “science” evolved from the pursuit of knowledge to a faith? If we blindly trust the people in power that they are right, and punish those who question their assertions, it can only end in destruction.
Similar to simply seeing animals and insects fly, humans intuited that they too could fly somehow. Seeing what's possible can open someone's mind in an instant. Would humans be capable of flight if other animals didn't fly already?
If science happens behind closed doors but we witness what it makes possible, that influence alone can inspire a lot more effort and discovery with or without the data that lead to the accomplishment.
I think that moment has been profoundly influential due to ChatGPT alone.
Whether or not it's a good thing that it's largely private is very difficult to say. I don't feel great about it, but I also don't think I know enough to have strong opinions yet.
Conversely: if you’re a depressed AI researcher: come on over to science, the water’s fine! We have centuries worth of open problems that are not going to be solved by ChatGPT, and need more smart technical people than we can hire.
What an opportunity! I get to spend most of my time writing grant proposals, compete with 30 other associate professors for 1 tenure track job, then when I get tired of that I can switch to a marginal wage industry job because AI researchers are the only scientists that get good compensation as a group, and the 28 other people who didn't get that tenure position are probably also trying to do the same thing (thus depressing wages)? Sign me up /s.
While true, there's more to academia that professors. There's a growing number of research scientists (particularly in software), as well as positions national labs (if you count that as pseudo-academia).
There's also lots of room in applying AI techniques to other areas of science (again, outside of professorships)
When you say "come over to science", where are you thinking? My impression is that industry -> academia is an almost impossible jump to make. Do you mean industry doing science?
I'm talking about working in a lab as a staff programmer. Most PIs I know, myself included, would absolutely kill to be able to hire experienced people from industry to write high-quality scientific code for them. The problem is that a) traditionally, it has been hard to get funding for this, though that's slowly starting to change, and b) even when the money is there, it's always going to be a fraction (1/2, often less) of what that person could make in industry.
To give you a counterpoint to what the scientists trying to lure you are giving you, being a "staff programmer" basically means you will be paid half what you should be to go work in an environment with no process and no engineering culture where you will be seen and treated as an inferior by everyone as you are not a "proper" researcher and academia is extremely status driven.
The problem is usually that science can't pay even a fraction of what you can make in industry right now, and meanwhile the good science jobs are often in high cost of living places.
I really think we need a UBI so that people who want to can afford to do science how many drug addicts less motivated to quit is a scientist less motivated to maximize clicks worth? It's gotta be more than zero.
> Unfortunately, the research field of AI is diseased: it places way too much value on showing off big shiny things
This applies pretty much to ..euh every field. The wow factor attract investors and money. The way the current economy is structured, it incentives these tactics since these individuals will be able to draw on significant funds and will have a comparative advantage even if they have a worse return ratio.
And then it'll pass and a new thing will kick again. Investors have no memory, and I am talking about institutional ones. The supposedly sophisticated folks.
Nothing sad about it. Keep theorizing and academizing. It's useful. It's not nearly as useful as tinkerers building usable non-theoretical things. It's no wonder that launching a rocket is more impressive and eye-catching than a guy shuffling papers. Don't dare knock down the doers, the risk-takers, the ones with skin in the game.
OpenAI provides me with a real, useful tool, something that directly improves mine and many others' lives. Your papers do not have that effect on my life.
I don't mean to knock down doers or risk-takers. On the contrary, I'm saying don't try to compete with the doers when you're not one.
> OpenAI provides me with a real, useful tool, something that directly improves mine and many others' lives. Your papers do not have that effect on my life.
Of course, OpenAI's work is built directly on top of publicly-funded research papers of the past 5, 10, 50 years.
Well, the core transformers/attention paper came from Google and the DNN revolution was also kicked off by Jeff Dean at Google, no? Yes, some of the very original neural net theory was laid in academia and maybe the original GAN paper came from it, don't recall, but AI academics were complaining about being unable to keep up and having their departments be raided even 7 or 8 years ago. It's not a new complaint. So this particular AI summer doesn't seem to owe much to academia.
The oddest thing about all these anti-academic takes is that OpenAI insiders and even some informed outsiders like John Carmack think of OpenAI (and google before that) as an extension of academia. The paper posted here is more akin to lower tier academics griping about R1 universities getting most of the funding (and using amoral tactics to get that funding). Seriously. OpenAI is just another R1 department (outsized corporate funding, together with an excellent devops and focussed PR team, explains all its shenanigans)
This is completely true from a personnel standpoint. But it's worth emphasizing the difference that they don't publish transparently their biggest findings, which makes them very different from e.g. Microsoft research.
But they are not academia. To pretend they are is disingenuous. Just because a group of people do research and write up their findings sometimes, does not make them academics.
Ooooh and here I was, an ignoramus thinking OpenAI was mostly trying to find a way to milk the generative models cash cow. Silly, cynical me, I should have known better.
OpenAI was created to trade (literally) on the "open" in their name. Any advancements in actual technology are likely secondary to the billion fucking dollar investment (which Altman has suggested might not be enough!). It's certain they aren't here to do "science", which IME involves showing your work to others.
OpenAI is not doing science. They are building a big shiny thing, showing it off, keeping it closed, and making money off of it. That is not part of the scientific process.
It is as if, every time SpaceX launched a rocket a little bit higher, every aerospace department ooh'd and aah'd and threw up their hands: "we don't have the resources to build a bigger rocket; how can we do science?"
Unfortunately, the research field of AI is diseased: it places way too much value on showing off big shiny things above progress in scientific understanding. But there is a ton of progress available to be made by small teams on small budgets. There is so much we don't understand about neural networks alone, even small ones.