I have said it before and I will say it again - the problem is that humans are MORE than just pattern recognizers. We build models that we are trying to make logically consistent; while the function that provides pattern recognition doesn't have to be.
To see that better, consider what I call "simplified Chinese room". It's a variation on a traditional Chinese room, where inside the room, there is only a pattern recognizer, which basically will match the input to arbitrarily many inputs (but not all possible) it learned before and chooses the output for the best match.
Now imagine I want to train this "simplified Chinese room" on solving satisfiability problem. Because in that problem, an arbitrarily small change in the input (introducing a contradictory clause) can completely change the output. It is therefore impossible, I believe, to learn the concept of satisfiability by just using pattern recognition (storing and comparing, according to some metric, previously seen inputs and corresponding correct outputs). Instead, you need to build a mental model which is internally self-consistent with these example pairs.
<rant>That's where graph neural nets come into place. They can learn relations, scaling to a large number of objects. A traditional approach would have to learn all possible combinations, hitting the combinatorial explosion. Graph neural nets can solve problems such as shortest path, sorting and dynamic programming. I think in the future if we are to get closer to human level we need graphs as the intermediate representation. Graphs could represent the objects in an image/phrase and their relations, then answer about the attributes of an object, the relation between two objects or classify the graph itself. All simulators are evolving graphs as well, and code/automata could be represented and executed as a graph. The transformer could be considered an implicit graph where the adjacency matrix is computed from the nodes at each iteration. The closest to AGI in my view would be model based RL implemented with graphs.</>
But what bothers me is that the human animal is trying to create an artificial human mind, the most amazing piece of meat we have laying around.
If we could create the reasoning of a dog first, to then upscale to more complex logic, I'd be more confident research is going places (as in, we have biological evidence building blocks to understand first).
Deciding consistency is a ridiculously hard next bar to set though, lots of NP-hard problems have very useful human-crafted heuristics nowadays. Similarily AI-crafted heuristics should be more than possible.
I agree, but at the same time you're kind of missing the point. Adding more and more examples to the pattern recognizer is like adding more and more epicycles (for dealing with special cases) into the theory of celestial object movement, where a more general and simpler theory will do a better job.
But it's true we don't precisely know how to "find and represent the better theory", if it's at all possible (most likely not) and what the exact trade-off is.
Also note that my argument doesn't necessarily rely on computational difficulty of SAT - even if we limit to some "simple" subset of SAT instances, we might see that the pattern recognizer is unable to learn these problems correctly.
And you are wrong.
Consider a logic task of deciding !!!!!....!!!!!false with 100-200 !s. Can you solve it without iteratively applying pattern recognition? What if it is 1kk-2kk of !s?
I didn't claim that pattern recognition is useless, I claimed that by itself is not enough. And you can see that your model uses it in the calculation iteratively, that is, arbitrary number of times. It is not just giving out the result that is closest to the database of stored results.
Yet my argument exactly is that the logical abstraction you were talking about is a relatively simple pattern, and what is beyond it less important.
The author is right that what we do for NLP is not enough for reasoning. But it does not mean that we are too far from it.
Even in your variation of the room experiment, if the person in it could reply something along the lines of "here is what I think is a part of solution", which you simply had to feed back to eventually get the whole thing, would you claim it is a weak abstraction?
To see that better, consider what I call "simplified Chinese room". It's a variation on a traditional Chinese room, where inside the room, there is only a pattern recognizer, which basically will match the input to arbitrarily many inputs (but not all possible) it learned before and chooses the output for the best match.
Now imagine I want to train this "simplified Chinese room" on solving satisfiability problem. Because in that problem, an arbitrarily small change in the input (introducing a contradictory clause) can completely change the output. It is therefore impossible, I believe, to learn the concept of satisfiability by just using pattern recognition (storing and comparing, according to some metric, previously seen inputs and corresponding correct outputs). Instead, you need to build a mental model which is internally self-consistent with these example pairs.