Egan is always dense. It's some mind bending physics/comp sci, but all cooked up in his brain so doesn't really apply to anything productive. I struggled with his books and his writing but toughened it out because I liked the concepts, but he's divisive.
An extremely layman answer is that most interesting innovation in parsing in relatively modern times has happened seems to be in the context of IDE's. I.e. incremental, high-performance parsing to support syntax highlighting, refactoring, etc. etc.
Actually the most important step of parsers (as even non-incremental, slow (or better: not fast) parsers are fast enough) is error recovery (error resilience) from syntax errors (mostly half written or half deleted code).
What is time consuming is e.g. type-checking. Semantic checking in general, like exhaustiveness checks of pattern matches, syntax checking is fast.
Parameters and training corpus size matter. Last year a new paper (Google "Chinchilla optimality") focused on compute-optimal LLMs found that we'd been under-training models -- i.e. you could wring more performance out of smaller models by training on more data. But (as far as I understand it - interested layman here) - for a given amount of data, model performance seems to scale more or less linearly with parameters.
Now, we could see another model architecture than the current reigning transformer architecture upend this (much work is ongoing on breaking the quadratic term in the transformer that computationally bounds its performance - an example is the Hyena paper that was published just the other day).
Biggest computer and most data wins is still the paradigm here.
Eric, want to echo what folks are saying here. I stumbled on your blog in high school (ca 2003?) and you (and Raymond Chen) fueled so much of my passion for compilers and API design respectively, which dictated both my school choice and at least some career choices later. You were highly influential from afar :)
Huge difference between a brand likey Disney able to pull off a streaming service and smaller producers. Smaller producers need demand, and for video demand = youtube.
For instance, let’s say I’m a comedian and have a thousand loyal followers. Why couldn’t I just pay $20/month for a PeerTube hosting service with my own brand. Then I can just tweet out the links.
Why would I need YouTube?
As another example, let’s say I’m a white supremacist, and I have a hundred followers. I can email a link out to them and never worry about censorship by YouTube.
So demand can be small, and it still works. In fact, since I now have more control over the experience, it can be much more tailored than youtube.
Caveat that I might be completely off, but if, say, 10 bn people tried once a nanosecond since the inception of the universe, you'd still have only 10^10 * 2^86 ≈ 2^33 * 2^86 = 2^119 attempts, that still only cover's 2^-9 (i.e. 1/512th) of the total addressable 2^128 space, i.e. still fairly unlikely that you'll have hit that specific number.
[/joke]