Maybe i'm not up to par, but I was under the impression that even ubuntu wasn't mature to upgrade between major versions. I never dabbled much but upgrade failure are not unheard of.
Ubuntu has offered distributions upgrades for years in the Software Updater. It just tells you, you click Install, it takes a while and a few reboots, and then you're updated. I've done it on servers, I've done it on desktops, works fine most of the time. However, I've only hopped one version at a time (16.04 -> 18.04, 18.04 -> 20.04), never multiple at once (16.04 -> 20.04).
Of course, sometimes failures do occur, but that happens whenever you do a major OS upgrade. I've had Windows fail on upgrade more than I've had Ubuntu fail on upgrade, but that's just my experience.
This in comparison to, well, Elementary OS saying scrap it all and reinstall. That wouldn't be so bad if they were, like, a technically-minded distribution - but for their primary market who hardly knows anything about updates, completely unacceptable.
I think the logic goes backward. Normal people are probably more at ease with backup + from scratch reinstall than update process + failure mitigation.
ps: I agree with your point about EOS trying to make things invisible for the user and not addressing this well though.
I've upgraded Debian based systems between versions for literally over two decades, and Fedora for half a decade. I can't recall one issue, excluding a few weird things that I managed to fix in Slink to Potato Debian, and that was two decades ago.
Ubuntu/Debian tests upgrades and they support it, the issues you will see are caused because in our circles people will install extra stuff outside from the main repos.
You most definitely can. do-release-upgrade handles all of it but you can change sources and do it the old school way. It's always preferable to reinstall using automation but if it's just a hobbyist server then there's no issue.
If you know what you're doing, the upgrade process is the same for any Debian-based distribution, and I've never had it fail in an unresolvable way. I assume the same is true for Elementary, though I've never used it.
Distros like Ubuntu make upgrade failures harder to work through because they go to some lengths to wrap the dist upgrade process in some software that reverts everything if anything goes wrong.
But if you just move all your apt sources forward to the next distro and read any specific error messages you get, you can generally work through them.
Distros other than Debian itself don't typically recommend that you do things this way, and instead have some extra software for going through the process. But the Debian way still works on them.
Did you stabilize 18.04 over a couple days to make sure you had caught all the reconfig required? It should be no different than 18.04 > 20.04 with that step I would think. I am currently at this phase ;).
Not always possible, of course. It's also understandable that you can't just jump through 4 years of updates in one go without some sort of work to catch up to current configs involved. I had to change 2 lines of config from 16.04 to 18.04 (due to my own tweaks to defaults), so it's been pretty pain free so far. And the few days as a precaution are on me, not Ubuntu. I should have updated years ago.