> Is it not possible that general AI is a specialized AI applied over the field of specialized AI generation?
AI problems can be characterised as those where there's no clear path to a solution (otherwise we just call it "programming"); tackling them necessarily involves trial-and-error, backtracking, etc.
Since there are far too many possibilities to enumerate, solving such problems requires reasoning about the domain, e.g. finding representations which are smooth enough to allow gradient descent (or even exact derivatives); finding general patterns which will apply to unseen data; finding rules which facilitate long chains of deduction; etc.
The difficulty is that there's usually a tradeoff between the capability/expressiveness of a system, and how much it can be reasoned about. If we choose a domain powerful enough to represent "the field of specialised AI generation", for example turing machines or neural networks, methods like deduction, pattern-finding, gradient following, etc. get less and less applicable and we end up relying more on brute-force.
To me, this is where the AI breakthroughs are lurking. For example, discovering a representation for arbitrary programs which allows a meaningful form of gradient descent to be used, without degenerating into million-dimensional white noise; or to take deductive knowledge regarding one program and cheaply "patch" it to apply to another; and so on.
AI problems can be characterised as those where there's no clear path to a solution (otherwise we just call it "programming"); tackling them necessarily involves trial-and-error, backtracking, etc.
Since there are far too many possibilities to enumerate, solving such problems requires reasoning about the domain, e.g. finding representations which are smooth enough to allow gradient descent (or even exact derivatives); finding general patterns which will apply to unseen data; finding rules which facilitate long chains of deduction; etc.
The difficulty is that there's usually a tradeoff between the capability/expressiveness of a system, and how much it can be reasoned about. If we choose a domain powerful enough to represent "the field of specialised AI generation", for example turing machines or neural networks, methods like deduction, pattern-finding, gradient following, etc. get less and less applicable and we end up relying more on brute-force.
To me, this is where the AI breakthroughs are lurking. For example, discovering a representation for arbitrary programs which allows a meaningful form of gradient descent to be used, without degenerating into million-dimensional white noise; or to take deductive knowledge regarding one program and cheaply "patch" it to apply to another; and so on.