Although I have to say, VentureBeat did much better than most media outlets I have seen writing about current research and what they write is not only accurate but also largely devoid of hype. Perhaps we actually managed to “keep the hype down” as we intended when writing this piece?
I will check in on this post now and then if you have questions and see if the first author is interested in joining when he wakes up as he really did all the legwork for this one.
Given that nearest-neighbor outperforms on closed book, is it reasonable to suspect the model is doing NN itself internally (which would explain the good performance on close duplicates?)
And if this is the case do you think training-time processing of data to attempt to move convert it to question/answer form data rather than raw QA would be a reasonable approach towards tackling this?
> Given that nearest-neighbor outperforms on closed book, is it reasonable to suspect the model is doing NN itself internally (which would explain the good performance on close duplicates?)
I think this is definitely the case for the BART model. It is essentially acting as a QA-pair memorizer over the training data, and at test time, it just matches the question onto those seen at training time. Note that the T5-11B+SSM closed-book model was able to do a little better on NQ, so very large models with task-specific pretraining objectives do seem to do something slightly more interesting than just NN, but still really struggle in some settings.
> And if this is the case do you think training-time processing of data to attempt to move convert it to question/answer form data rather than raw QA would be a reasonable approach towards tackling this?
Great question! Converting sentences into a series of QA pairs is something we're really interested in. The T5-11B+SSM model we evaluate in the paper uses a special "Salient span masking" pretraining objective that does this to some extent (only mask words at pretraining time that are likely to be "answers" to factual questions), so in essence the pretraining task becomes pretty standard cloze-question answering, and they find that leads to better downstream results (https://arxiv.org/abs/2002.08910)
> [BART] is essentially acting as a QA-pair memorizer over the training data, and at test time, it just matches the question onto those seen at training time.
Not super-surprising.
> The T5-11B+SSM model we evaluate in the paper uses a special "Salient span masking" pretraining objective that does this to some extent (only mask words at pretraining time that are likely to be "answers" to factual questions), so in essence the pretraining task becomes pretty standard cloze-question answering
This seems an obvious approach for cloze-type questions, but it seems non-obvious how to extent beyond this.
Are you aware of any work probing the differences in the representation using this style of masking vs a more normal language model objective? It would seem to me that this is the key to significant progress here (and of course one would speculate that a representation that works well for this would also work well for all kinds of KB-related tasks).
Thinking about this for a few minutes things like masking names, colors and numbers (the things that neural representations often confuse) and then asking questions based on them might be interesting. I wonder if bAbI could be extended for this?
Hi, first author here. Yes, we evaluated 3 retrieval-based models, DPR, RAG and FID - check out the paper for the numbers (https://arxiv.org/pdf/2008.02637.pdf)
I tend to think that lots of solutions could come from topics like those discussed in this book, with a lot of further development: http://probmods.org/
Understanding of causality is very likely an emergent property. While extremely important, it's unlikely that we have some hard-coded low level architecture of causal inference in brains. It probably will just arise as a necessity of grounded understanding of the world.
The conditions for causal inference being possible are pretty clear and have to do with the intentional modification of the local environment.
The intention to achieve some new environmental state, and your action to bring it about, is a dynamical activity that enables "deep" model building.
Causal inference is not going to be some "module" of the brain... it requires a body. When you place your hand on a hot surface, once, you immediately understand that it is hot. It does not require "induction" (as hume supposed). That is because our body identifies causes.
Its therefore pretty trivial to observe no NLP system understands language, or even can understand language, because it lacks this capacity to acquire language semantics via participation in environmental exploration. It has no body.
ie., you need to have experienced "on top", "green", etc. to know what "green leaves grow on top of trees" means. There is no meaning in the frequency co-incidence of symbols in text.
So no matter how much you are able to reproduce these patterns, they contain no content. The content is in the reader.
Your comment is interesting but seems to conflate two separate though related areas. The need for a body arises from the embodied cognition school of AI which suggests that intelligence is fundamentally embodied, hence the need for a robot equivalent to a body for truly understanding language.
However this does not necessarily have to be related to causality, and counterfactual statements about a causal model. The math behind counterfactuals and causality is actually well understood now (see any of Pearl's books). It does not actually require that a system be embodied, just that the system have some suitable (and correct) causal model of the world.
It would of course be amazing to have both in one system, but that is not required. An AI system that understood causality and language could be bootstrapped from causal models supplied by humans - or even other AIs :)
I'm conflating them because they are deeply connected.
Causal analysis can be performed, via Pearl, on datasets collected for causal analysis.
You still need some mechanism to collect the data, ie., the scientist. This requires solving the "relevance" (/framing) problem -- which, in my view, cannot be solved under a congitivist (/computational) theory of mind.
"Data" which is relevant to a causal hypothesis isn't selected via inference, the body "selects" it.
eg., when my hand is on a hot surface, it's temperature isnt "chosen as the relevant casual variable".
The body is the primary solution to the relevance problem. So you can't just "shove in causal math" into a computational system and expect it to grasp anything.
I also don't think bootstrapping will take you very far: causal models of, eg., dogs are very deep. ie., we understand their 2d, 3d, skeletal, behavioural, color, sound etc. "dimensions".
To say, "the dog was well behaved" requires an extraordinarily deep model of "dog".
The only way i see this being built is via play, ie., via hypothetical interaction with an environment -- as we do -- with bodies capable of discerning relevance.
I agree that causal inference certainly requires ability to interact with the environment. In do-calculus this follows basically from the definition of the "do" operator.
As other people already mentioned, the "do" operator does not need to be related to the physical human scale environment. Causal inference could be useful also in interacting with the internet or other virtual environments like games.
Additionally, even humans do not have intuitive understanding of physical reality outside of our evolutionary environment. Nobody is able to intuitively understand quantum mechanics or general theory of relativity. Our intuitions of causality can beak even in relatively mundane environment like low Earth orbit. For example, you can fire thrusters to push you towards an object and still miss that object. There are actual missions in LEO that failed do to this type of mistakes of elite test pilots employed by NASA.
Of course, you can use mathematical formalism to reason about unfamiliar environments, but no matter how much time you spend learning about multidimensional spaces you will not be able to imagine 4D space or a quantum wave. But nobody is claiming that we do not understand language like "spin of an electron" because we have never experienced what a spin of an electron is.
Now to get to my original point. My original point was that causal inference is not a basic building block of intelligence. Formalism like do-calculus or logic are too brittle to be a building block of intelligence. You need something that is robust to noise. Something that can consume a tensor of pixels and process it. Once you have a system that is able to deal with this type of inputs then you can strive to do something like do-calculus or logic on top of it. But my argument goes further. My guess is that ability to do formal reasoning will emerge from the need to carry out complex tasks. Nobody will intentionally program models to do logic or do-calculus. Ability to do it will arise in models as a combination of the necessity of solving complex tasks and clever training techniques as it did trough evolutionary process in humans.
PS.
> So no matter how much you are able to reproduce these patterns, they contain no content. The content is in the reader.
To me this type of assertions do not contain any content.
"The content is in the reader." I don't understand that at all. Maybe I'm just a Chinese room :) .
Approximately, the semantics of natural language are concepts. Concepts in this sense are deep bodily-inferential-causal models.
If I say, "No! The glass is under the desk" you can immediately: find the glass, reason about why you havent previously found it, ask for the glass to be filled, ask for the glass to be handed to you, report on whether you like where the glass is..
You could say, "Why is the glass on the floor!?" angrily.
And by doing so communicate an expectation about where glasses ordinarily are, express a desire-frustration, express a confusion over the intentions of others..
The information contained in "No! The glass is under the desk" is VAST.
It is vast because our understanding of the world is vast. Not because the sentence as a lot of letters. Not because the words in it occur in a certain frequency. Nothing about the sentence itself is vast. It is profoundly shallow.
Intelligence is this vastness. Intelligence is moving effortlessly from "why" to "how" to "when" to "what" across domains, across hypothetical/counterfactual scenarious, across intentions/expectations.
This vastness is laden in animal minds, even, essentially constitutive of animal minds. And it cannot be found in "data".
It is grown by the complex biophysical process of learning, which requires a deep (& playful) interaction with the environment. The environment grows your capacities as you play with it.
I disagree. There are other ways to observe and learn cause and effect than by physical interaction. We learn new causes and effect of many things each day by reading or watching. There are also most likely examples of humans who never had control over their physical world (through disability or whatnot) that came to understand cause and effect.
This sounds like a possible analog to someone who is intelligent but doesn't really understand math or science. This person might be able to BS their way through certain fields, but could never pass a calculus exam.
I'm always amazed that some people think performing statistical analysis (training neural networks) on text can lead to actual intelligence and knowledge about the world, if only we increase the number of model parameters by a few more orders of magnitude (GPT-3 and the likely strategy for GPT-4).
Of course, all this without any model of how likely it is that the knowledge is embedded in the text, such as trying to train simple models on descriptions of a simple world. I think it's very likely a model couldn't even learn simple arithmetic (addition, subtraction multiplication, division on the rational numbers, let's say) given all of human writing on arithmetic, nevermind being able to reason about the entire world.
It amazes me people think their brain works differently.
It's pretty close to believing in body/mind dualism, the only thing in neuroscience more outdated than Freud.
Your brain works within the same laws of physics as the outside world. We don't know how the brain works exactly. But once we understand it, it is unlikely to be qualitatively different from a neural network.
On the other side of the equation, the emergent behaviour of deep networks make them fundamentally different from the sort of statistics that came before. So even if the building blocks are old and boring, the larger system isn't.
> it is unlikely to be qualitatively different from a neural network
I'm sorry but this shows a profound misunderstanding of what a NN is, and what the brain is.
There are no "neural network"s. The NN algorithm is a method for optimizing the parameters of a piece-wise linear regression model.
These regression models have no homology to any brain structure and the process of producing them ("training") has no neurological analogue either. They can be produced with a variety of algorithms.
The phrase "neural network" is, as peddled by the media and poorly informed lecturers, a lie. It is neither neural nor a network. It's just gradient descent with more parameters.
A model of the brain would model, at least: neuroplasticity, biochemical signalling, activation frequency, etc.
There is nothing about a peice-wise linear regression model which does any of that. No matter how many circles and lines you draw. (NB. essentially any mathematical function can be drawn as a "neural network". The "network" is just a way of diagramming function application & dot-products).
Aside from all that, the body-brain system of animals is a physical process whose properties are not abstract. The reason we are intelligent is because we have bodies capable of causal analysis; and that capability is biophysical.
To put it another way, no algorithm which runs on a digital computer will turn it into gold. Not even one called, "the midas algorithm".
Causal. Imagine a video. Each frame is plotted in space, so instead of a sequence of 2d images, you have a cube. The z axis of the cube shows each frame of the video in turn. This is an equivalent representation to the representation that we are used to. We have just switched the time dimension for a physical dimension. What you are reverently calling causality, is only correlation along the z spacial dimension. In the same way an image can be compressed due to the similarities across it's 2 dimensions, so can a 3d representation of video be compressed along all 3 of it's axis. An appreciation of causality is only induction and induction is only a correlation, if a then probably b. Finding correlations in one of the dimensions you are more familiar with, like the pixels of an image is no different to finding it in time.
Within language the elements of time are encoded, just as they are within the video represented as a cube. "The boy threw the ball, the ball landed and then rolled."
Just because the time dimension is represented to the neural network within the input parameters of a single iteration of the neural network, does not make it any less able to understand correlations across time.
So this is to repeat hume and mistake causal analysis as a kind of induction or inference.
It isnt.
Our bodies are the primary site of "causal analysis". Eg., when I touch the hot surface of a stove I do not infer that temperature is a cause of my hand being hot.
Such an inference, as the basis for our models of the world, would be -- as you/hume/etc. say -- deeply insufficient.
The operation of the world on our bodies is already laden with causal information. A hand striking a face does not just create a "painful sense impression"... rather the body encodes it as caused by the object you also saw.
It is the action of the world upon the body that is the bedrock of our causal model building. Scientific/inferential processes sit on top of that to disambiguate between possible distal causes.
A machine, as a person, given "mere data" cannot hope to do much.
Suppose I place a pot on a stove and the water boils.
Now I feed into the machine what data?
Here, it gets this: all gravitational, electomagnetic, etc data within 1km; all geometric information about all objects within 1km (, and all of their properties, etc.)
Now, machine, what caused the pot to boil?
It has no clue. There are an infinite number of antecedent temporal events.
The problem isn't the mathematics of causal inference. The problem is relevance. That isn't an inferential problem.
>It has no clue. There are an infinite number of antecedent temporal events.
Right, determining the cause of the pot boiling given only local information about this one event is impossible. But that's not a good representation of learning from a real-world data set. A real training corpus might also have an example of a pot boiling in a campfire, a water heater heating up water using fire, someone touching an open flame and going "ouch!", fire burning down a house, etc. All these examples taken together can reasonably lead one to infer that fire causes things to become hot. This is the kinds of regularity found in real world datasets and a good learning algorithm will extract this regularity in the course of predicting the dataset.
No, I don't see your point. No one "prepared" common crawl to contain multiple instances of relevant examples for GPT-3 to learn from. It's just that a sufficiently large training corpus will naturally have this sort of regularity in it. My point is that a general learning algorithm trained on a sufficiently large and representative corpus will capture causal regularity without the sort of fine-tuning of the algorithm or the training data you are suggesting. You haven't given any reason to think otherwise.
identifying causes cannot be done statistically, as a fact of statistics
events A then B then C do not imply C is caused by B is caused by A
This problem is worse the more data you have, as with my example above of giving the machine /every/ event within a 1km radius of a pot boiling.
Identifying a cause is a dynamic experimental process. A machine can only accept a highly prepared dataset which has already been chosen because the associations are known to be causal.
But this problem isn't specific to machines. The fact that my hand hurts after touching an open flame stove doesn't entail the flame caused my pain. No amount of first-hand experience with flames can entail that flames cause pain. All we can do is increase the likelihood of this model. Experimental processes are in the same boat, except that the statistical power is greater. But our lived experiences with flames and pain and all other stimuli is a sort of low power ongoing experiment. But with enough of these poor experimental runs we can converge on an approximately true model. The same goes for a statistical learning algorithm and a large training corpus.
You are just pushing the solution to the problem of relevance (which isn't actually a real problem) down the stack, into the body. How does the body magically create this new kind of relation called cause? You are mistaking the image in your mind for reality. The image you see in your mind is the product of an extraordinarily large neural network that has done all the statistical analysis for you.
Very good points! I would add that we don't yet have a complete model of the computational properties of even a non-neural cell, even though we know that all cells exhibit simplistic perception and decision-making.
This is not to say that we should expect some non-turing weirdness or anything 'mystical' like that, but that we just haven't taken the time to analyze the exact mechanisms and to model cells at this level well enough to be able to say that they are similar to any particular computational structure. Thus, to claim that the brain in its entirety is similar to any particular computational structure such as an NN is either wrong or trivial - the only level at which 'similarity' can be claimed is that they are both useful in computation, and likely turing complete. Anything more specific is fanciful thinking.
As another poster wrote, the main problem is not the neural network itself, it is the training set, and very likely the training algorithm as well.
My claim is that working knowledge of the world is simply not encoded in human text. All human text (and speech) presupposes a certain model of the world that is learned from experiencing the world, likely based on a model that itself is partially pre-trained by evolutionary processes.
If a text corpus correlates with human knowledge (your "presupposes a certain model of the world"), then it necessarily encodes the portions of the knowledge it correlates with. That is, correlations drive the search of model parameters that explain the correlations. The stronger or more numerous the correlations, the more accurate a model can be computed. But the search space informed by this network of correlations is much more restricted than a naive brute force search, thus the correlations provide information about, i.e. encodes, the correct model.
Then I used the wrong words. My point is that human text is essentially encrypted using a particular model of the world as the encryption/decryption key. You can't derive the key from the cipher text.
For example, if I give you the text 'the car is red', there is no watly to imagine the meaning of this phrase (in any sense of the word 'meaning') unless you have some understanding of what cars are, what red is, and what it means for a car to be red (which is somewhat different from what it means for an apple to be red, for example).
It is interesting that with enough statistical analysis you can begin to decipher certain parts of the true message without knowing the cipher key. But this only works for those parts where you have a large corpus of text talking about them, and there are many common human needs or experiences that do not have such a corpus. It is also very easy to capture answer/response pairs in the corpus and repeat them, seeming to have high accuracy, but in fact failing to generalize more than meets the eye.
For example, with enough compute power, you could build a machine that, for any query, would find the most similar such query in all of its training data, and respond with the continuation of that query (which could be an answer or a follow-up question). This could be quite impressive in conversation, while doing nothing to elucidate anything about human intelligence, and being wholly unable to adapt to any novel situation.
Your encryption analogy doesn't really work either. Encrypted text encodes the key used to encrypt it. This is why it takes 2^N computational steps to recover the plain text, where N is the key size rather than where N is the text size. It's just that the computational cost of decrypting with sufficiently large keys is intractable.
But human languages have much more exposed regularity than ciphertext from good encryption algorithms, so the computational cost is much lower. The "key", i.e. world model, in case of human languages is plausibly much bigger, but since human languages have significant exposed regularity, it is still tractable to compute, with the trade off that you need much more text to do it.
What we need here, I think, is a GAN-like approach. Use a generator + a discriminator. The generator could be like GPT-3, but it needs another model to filter out the nonsense. We work the same way - come up with crazy ideas, then check them out to see if they hold ... 'I had an idea, but nah, it wouldn't work'. It's a process supported by different skills.
The issue isn't whether neural networks work, the issue is the dataset. Look again at what the parent wrote:
> I'm always amazed that some people think performing statistical analysis (training neural networks) on text can lead to actual intelligence and knowledge about the world
Emphasis mine. The human brain does not develop what we know as human intelligence by analyzing text.
I don't see why you're "amazed". Neural networks are _provably_ an universal approximator. As such, they can approximate _any_ function to an arbitrary degree of precision. It just might require an impractically large/complex neural net to do so, and you might not be able to train it yet, but it's _provably_ doable in the limit, if you remove size/training constraints.
I'm a practitioner in the field, and an AI skeptic because of it. I think perceptual tasks work reasonably well, NLP barely works for anything, and any sort of cognition doesn't work at all, as of August 2020. I will be the first to admit that we don't have real "AI" yet, and we won't have it in the foreseeable future. But to say that real intelligence is not possible with the current tool set is simply incorrect. It may be impractical. It may take 100+ years to figure out how to build nets this large and train them successfully before sun burns out, but it is _theoretically_ provably doable.
In a way it's kind of like fusion: you have a proof that it can work right above your head during the day. We just don't know how to do it at our "human" scale with our knowledge and capabilities. Our knowledge and capabilities, however, are not a constant - they improve exponentially over time, although most people don't perceive this process as exponential.
For clarity, Universal approximation doesn't mean ANNs can approximate just absolutely any function...
Depending on which proof you're looking at, ANNs can provably approximate any continuous function, or any Lebesgue integrable function, or any real vector-valued continuous function on a compact subset of R^n space [1]. That's certainly a very large class of functions, but it's not just any function.
Also, those proofs only work if you allow ANNs with arbitrary number of layers or layer width. Perhaps the required ANN size to approximate the AI / AGI function is so huge that it's beyond the number of particles on Earth. At what point is it beyond impractical? What if the ANN size (number of layers or neurons) is beyond the number of particles in the galaxy? Maybe it's just the wrong approach.
I haven't studied those proofs in detail, but as existence proofs, they just show that some ANN exists that can approximate a given function (of a certain class). There's no guarantee that there's a general algorithm that can learn the required ANN, is there? Maybe you just have to be really lucky to randomly initialize onto the correct approximating ANN...
Also, I'm not aware of any proof that human or bio NNs implement a Lebesgue integrable function, or etc. It's entirely possible bio NNs uses some weird (quantum?) physical process, and implements a function outside the scope of any ANN universal approximation theorem.
I'm just a little more cautious around those universal approximation theorems. I'm more or less in agreement with your comment otherwise!
> Perhaps the required ANN size to approximate the AI / AGI function is so huge that it's beyond the number of particles on Earth
Perhaps there's a teapot orbiting Mars. It's unlikely tho. I think a better hypothesis would be that to model a brain (if we knew how) one would need the same order of magnitude of neurons and connections. This is completely unachievable now, and even if it were technically possible nobody has a faintest clue about how to train such a thing. We do not understand _many_ things about how the brain learns (no backprop!), how it retains and retrieves memories, etc, etc. The best studied system is vision, and computer vision does actually work pretty well by now, achieving superhuman results on some tasks. Thus far the most complex brain we've been able to accurately and completely model is not even a brain at all - it's just a handful of neurons of a worm. I took an online course in neuroscience a few years ago hoping to gain some insight for my ANN work. The situation with our understanding of even primitive brains of i.e. rodents is, well, primitive. We also don't really have math to accurately model more than a handful of real, biological spiking neurons - PDEs get way, way too complex for anybody to handle. But one could argue we don't have to: ANNs effectively operate in the frequency domain instead, where magnitude of activation approximates spike train frequency, and timing effects are ignored.
Don't get me wrong, I'm pretty sure if we do achieve real AGI, it likely won't be with anything even remotely like the current tech and science stack. Energy efficiency and compute density is just not sufficient. But I'm also pretty confident that even with the current stack we will be able to, eventually, approximate intelligence closely enough to suit many practical uses that seem completely out of reach today, including even some limited degree of logical inference, and dare I say, cognition.
If only people (including myself) weren't so chickenshit and actually seriously worked on (and funded) this stuff, that could even happen in my lifetime. As things stand right now, people are afraid to touch anything AGI related due mostly to fear of ridicule.
In my comment above I am merely referring to the conjecture that _if_ we had ANNs that are large enough, and _if_ we knew how to structure and train them, there doesn't seem to be anything that would _fundamentally_ prevent them from supporting AGI. Those are very, very massive "ifs" indeed, without even a faintest glimmer of light at the end of the tunnel.
I completely agree with your conclusions starting from these ifs!
However, I would add that another interesting area of study that is essentially not being explored at all is exactly to try to understand more about human cognition at the computational level (whereas neuroscience is working at the biological level). Basically, it seems we have abandoned any work in AI that would try to model human learning or human understanding of language in a human-understandable way, and instead we are aiming for reproducing human behaviors for engineering purposes.
I'm not claiming that an NN can't approximate human or super human intelligence, I'm claiming that using human writing as the only training set is not going to get you there.
The text simply doesn't contain enough information to draw a model of the world. Human language only makes sense if you understand the world, not the other way around.
Essentially what we're doing is similar to a human trying to learn psychology by reading books in Mandarin on it, while knowing neither the writing system nor the language. Except that human languages share some universal grammar, and share more or less the same notions about the world.
The “completion” includes an answer to the problem but also provides a natural language summary (GPT3)
GPT3 has ~170billion parameters (GPT2 ~1.5B) & somehow is capable of many-digit arithmetic w/o fine-tuning or task-specific training. (Its training data was “messy” from Common Crawl like for search engines)
Microsoft’s Turing-NLG has ~17billion parameters & was successful only in 1 of every 2 attempts of 2-digit arithmetic and only successful less than 10% of the time for higher numbers of digits.
It does lead to intelligence and knowledge of the world. It leads to somewhat similar understanding of the world as our understanding of 4D, quantum mechanics or environment close to a black hole. Pretty bad one, mostly based on repeating whatever we have read about in popular science literature without any intuitive understanding. GPT-3 understanding of physical world is similarly bad. It's just simply a modality not available to the model.
But GPT-3 is able to do few-shots learning. If you show it couple of times how to do something it will try to repeat after you showing some rudimentary understanding of what you are trying to do. It's even able to learn to do simple analogies [0].
GPT-3 is able to do very, very simple arthritics like addition and subtractions with 2-3 numbers. Actually even a lot smaller models trained on arthritics are able to do impressive feats like symbolic integration and solving differential equations.
> Deep Learning for Symbolic Mathematics [1]: Neural networks have a reputation for being better at solving statistical or approximate problems than at performing calculations or working with symbolic data. In this paper, we show that they can be surprisingly good at more elaborated tasks in mathematics, such as symbolic integration and solving differential equations. We propose a syntax for representing mathematical problems, and methods for generating large datasets that can be used to train sequence-to-sequence models. We achieve results that outperform commercial Computer Algebra Systems such as Matlab or Mathematica.
What is true is that GPT-3 is not very good at following instructions. It would not be able to execute an algorithm. So, it would not learn arithmetic from instructions if you would not include explicit examples.
Yes. Specifically the way human brain interacts with environment, the same way any AGI will need to sense and react to an environment, the more real the better.
An interesting example would be if the NN learns something from a book, also fed it with the all the critique on that book ( the environment reaction ). So it can possibly learn from a human what is good or bad in that book, etc.
Then both AI and humans can speak the same language ( on terms.of "feeling"/sensing ) and eventually learn the proper context, so less parameters are needed.
> An interesting example would be if the NN learns something from a book, also fed it with the all the critique on that book ( the environment reaction ).
But isn't that exactly what has been done with GPT-3? AFAIK it has been trained with both facts (wikipedia and such) and web-crawled text content that contains both content and critique of content (book reviews, movie reviews etc.)?
It does lead to intelligence and knowledge of the world. It leads to somewhat similar understanding of the world as our understanding of 4D, quantum mechanics or environment close to a black hole. Pretty bad one [...] GPT-3 understanding of physical world is similarly bad.
No, GPT-3 has no "understanding" whatsoever - we at least have a conscious engagement with the topics you cite and a genuine understanding that we lack a directly relatable perspective.
Our understanding in many circumstances - like those you mention, may be severely constrained, but it exists as part of a sentience that there is no reason to believe GPT-3 or any other current AI tool - possesses.
Do we really understand? What's the mechanism for that?
My thinking is that we humans have some ability to do things by rote until we have a good statistical understanding, at which point we use the statistical understanding mostly, and fall back to rote where we have to be careful. But, mostly, we're stats machines too.
We obviously have a working model of the world that we draw facts from, that is partly built-in (trained by evolutionary processes) and partly refined with personal learning.
By contrast, GPT-3 only has a knowledge of text. To the extent you could say that it thinks, it thinks in terms of textual symbols that it has seen before - that is its world.
It is theoretically possible that if we found a way to encode experiences of the world of the world in a dataset and used the same training mechanisms and models that we used for GPT-3, we would get an AI with a decent model of the world. But it is magical thinking to imagine that we can get there by training a model on raw text.
It shouldn't be that hard to imagine. If the objective is to predict human text, at some point the best way to predict text is to capture the kinds of knowledge that humans know and use in text generation. Human knowledge is a strong prior for human text generation, and so it shouldn't be surprising that for a large model its parameters would end up in a portion of parameter space that encodes some human knowledge.
By this token, the best way to learn Mandarin would be to start looking at text in Mandarin and trying to assign it meaning. While it theoretically can work, it is vastly more difficult than trying to relate it to a previously learned model. In fact, it is so difficult that we have never successfully understood a lost human language this way (though with the caveat that we don't have the huge corpus of text that GPT-3 was trained on in any of the lost languages we tried it on).
No, I said the best way to learn to predict human text is to capture human knowledge. I did not say how best to capture human knowledge. Obviously doing gradient descent over a huge text corpus is massively inefficient for a human. Whether its the best way for a machine to learn it is another matter. After all, the amount of data that humans experience through their senses during formative years, as well as the "computational work" that went into evolving human brain architectures surely dwarfs GPT-3's training regime.
It is very true that GPT-XYZ is fundamentally limited, like all transformer models, but don't misattribute the root cause to the fact that it's just processing text.
The real limitation is that these are feedforward networks that just do perception without any processing of what they've perceived. You can try to hide that fact for a while by increasing the depth of the perception network, basically hard-coding some processing into the single pass, but you're still not capturing any of the "absolutely requires self-feedback" behavior that we care about that takes a human more than a split second to do (aka almost all actual thought).
A statistical model that made good use of what was coming in could absolutely learn (to take your example) math at every level based on nothing but text, transformers are just nowhere close to having that capability because of their design limitations.
But the fact that it 'lives' in a world of text is much more fundamental. There is simply not enough information in text to draw a working model of the world. We presuppose such a model in our words and in our communication, so fundamentally any algorithm that is solely trained on text can't learn a model of the world from it.
It seems they are on the right direction. But, the number of parameters hit the physical barrier of silicon-based device.
To make the neuron network be a strong AI, we may need the parameters in scales of 10^19~10^44 (uncertainty limit and Planck time) estimated from how many frames per second in real world.
I personally got a feeling that this is bound to how nature calculate the physical phenomenon. For simulation, there are lots of "for loop" in our code/machine. But, in real world, we could not observe any "for loop". It is fast and continuous in the macro scale.
In order to reach strong AI, it is time to focus on quantum computing.
> To make the neuron network be a strong AI, we may need the parameters in scales of 10^19~10^44 (uncertainty limit and Planck time) estimated from how many frames per second in real world.
There is absolutely no basis for claiming this. It might turn out that there are better algorithms for AI than neural nets. Do you even realize just how many parameters even 10^12 is???
For loops in AI are actually only there to implement continuous mathematical operations such as matrix multiplications in a single core. Modern AI chips and implementations use very little to no for loops.
Sorry, maybe it is a wrong analogy. What I want to emphasize is that there is no calculating time in nature. At least, we could not feel "lag" in our "Nature Server".
For the part of chips, did you mean that it runs the "for loop" as parallel in each micro-chip (or gate or whatever tiny structure it is)?
>Sorry, maybe it is a wrong analogy. What I want to emphasize is that there is no calculating time in nature. At least, we could not feel "lag" in our "Nature Server".
Yes, but the reality is that there is lag in our projection of what nature is and that we just ignore it. Different parts of our perception actually don't exist at the same instant.
>For the part of chips, did you mean that it runs the "for loop" as parallel in each micro-chip (or gate or whatever tiny structure it is)?
Pretty much, the for-loop is unfolded and ran in parallel.
That said the interesting thing about language AIs is that they are by design able to communicate, which leads to interesting results, even though it is increasingly obvious that they don't really have any 'awareness' as such.
The article doesn't give examples, but the paper does. Here are a few examples of question overlap, for context.
Test Question: who plays max voice in a goofy movie
Train Question: who does max voice in a goofy movie
Answer: Jason Marsden
Test Question: when will the 2018 oscar nominations be announced
Train Question: when are the oscar nominations for 2018 announced
Answer: January 23 2018
Test Question: who has scored more goals in the premier league
Train Question: most goals scored by a premier league player
Answer: Alan Shearer
Test Question: where are the cones in the eye located
Train Question: where are cone cells located in the eye
Answer: retina
Test Question: who led the conquest of the incas in south america
Train Question: conquistador who defeated the incan empire in peru
Answer: francisco pizarro
It makes sense that questions like these are especially susceptible to brute memorization. These are certainly bugs in the benchmark—and, in fact, about a third of questions in each dataset have this issue.
The paper also considers answer overlap; that is, when the answer to the question also occurs in the training set. This alone does not imply memorization, but it does open the door for some shortcuts. Some examples from the paper are:
Open Natural Questions
Duplicated: Phil Simms, Brian Johnson, 8, the Indians, the 1830s
Unique: Cloves, Matt Monro, 1,020 – 1,080 kg, Hermann Ebbinghaus, Matt Flinders
TriviaQA
Duplicated: David Bowie, Battle of camlann, Heligoland, Henry VII, Niagra Falls
Unique: Death in the afternoon, Clash of the Titans, ice-cream sundae, Camshaft, Cumberland
WebQuestions
Duplicated: Harvard, Alderaan, India, 2011, Zeus
Unique: Queen Victoria, Braslia, Paddington, Tom Corbett, Gary
It's less obvious how to treat answer overlap. In particular, removing answer overlap might bias the dataset towards harder questions. Some of the reduction in model scores is presumably because models have to work harder to understand the question, as merely looking for topical similarities will be ineffective, but it also removes questions whose answers are general enough to apply to many questions, like ‘8’, ‘the 1930s’, ‘Harvard’, ‘2011’, etc. This means that reductions in the score don't clearly say how much cheating happened. However, the success of the BERT-based Nearest Neighbor model, which retrieves the answer of the most semantically similar fine-tuning sample, scores similarly to BART, which seems much too high for comfort.
It should be noted that a lot of the discussions about causal knowledge and intelligence aren't too relevant to these questions, as they are largely tests of memory or retrieval. It is expected that these models answer the questions by searching their training data or their document index. The issue is that their training data isn't meant to contain copies of the test questions, just the information sufficient to answer them.
Thanks for the comment and for adding examples, and for your nuanced comments of the answer overlap split.
My position is that these datasets are still useful for QA, but what was lacking was an analysis of how easy/hard the questions in them were, and what kind of modelling was needed to do well. These overlap phenomena are less like "bugs" maybe, but more like poorly understood features.
We need models that can accurately recall QA pairs they have seen before, so being able to score well on "memorizable" QA pairs is still important to do well, but we also want models that can do more than that.
One single accuracy number on a leaderboard cannot capture all the behavioural information we need to properly understand the capabilities of these models.
That's useful for chatbots, where you have a big FAQ and people ask questions. That's exactly what RASA does, using TensorFlow. It's reasonably good at matching user questions with stored questions. But that's all the AI part does.
https://arxiv.org/abs/2008.02637
Although I have to say, VentureBeat did much better than most media outlets I have seen writing about current research and what they write is not only accurate but also largely devoid of hype. Perhaps we actually managed to “keep the hype down” as we intended when writing this piece?
I will check in on this post now and then if you have questions and see if the first author is interested in joining when he wakes up as he really did all the legwork for this one.