Someone in a thread here a few months back described webdevs and their constant obsession with changing tooling, frameworks, etc. as "mayflies debating politics" or something to that effect and it has stuck with me since. Do all these new frameworks really have some sort of inherent flaws that require rebuilding everything from the ground up every 5 years or less? Or is it just something inherent in the language/web (and/or the sorts of people who become webdevs) that causes this phenomenon?
Or when JS devs suddenly discover concepts like "serverside rendering" and reinvent PHP and ASP.NET.
I think it's mostly due to nobody solving "it" yet. If you look at the various frameworks, they approach the same problems in wildly different ways, even if the underlying ideas are similar.
Take Svelte vs. Angular vs NextJS for example - they all sort of do same thing (create web apps), but take very different approaches. People choose these various frameworks for reasons, including what style makes the most sense to them, which one is easier to use, which one might be more performant for their use, etc.
There's no clear "right" answer to which framework is best - in other words... the field has not yet been min-maxed.
Regarding server-side rendering - I'd assert today's SSR is not really the same thing as classical SSR. Today's SSR is almost always a hybrid with some sort of client-side payload backed by server-side stuff, giving you the best of both worlds.
People are going to downvote me for saying this, but because JavaScript's built-in batteries are so weak, everyone re-invents them all the time in slightly different ways.
- How do you build a Rust project? With cargo
- How about the Go project? With the Go tool
- How do you build a backend JavaScript project? - node, yarn (with different incompatible yarn versions), bun, and 5 other types of tools
I manage a 100+ engineer team, and any time an engineer complains about our Python stack being unwieldy, I propose they work on frontend tooling for a week.
JavaScript tooling makes Python tooling fragmentation look sane.
How can you possibly realistically manage a 100+ engineer team? 100+ direct reports?
Also, the Python landscape is just as bad as frontend! And I enjoy writing both quite a bit. Pip, pipenv, pyenv, poetry, conda, virtualenv, uv, ruff, black, pep8, etc, etc. They both need massive improvements.
> How can you possibly realistically manage a 100+ engineer team? 100+ direct reports?
They report to engineering managers who report to me.
> Also, the Python landscape is just as bad as frontend! And I enjoy writing both quite a bit. Pip, pipenv, pyenv, poetry, conda, virtualenv, uv, ruff, black, pep8, etc, etc. They both need massive improvements.
Yeah, it is bad, but a notch better.
Most things in Python land are 5-year-olds.
5-year-old React or Vue codebases (the two most popular frameworks) are not backward-compatible with the current release.
And we cannot stay on old versions as we have a government as a customer who requires SBOM, and SBOM declaring unmaintained dependencies creates even more trouble than upgrading frontends regularly!
ASP.NET supports JScript, I was doing "serverside rendering" with Javascript and ASP.NET about 7 years before nodejs was a thing. ASP.NET and PHP were reinventions of previous tech.
As for webdevs and tech-churn, there are a lot of people that want to make their mark on the industry in one way or another. And it's human nature to want to make a better mousetrap. Combine that with the ease of publishing to npm and you get a lot of people reinventing a lot of wheels. I wouldn't really say the previous tech is "flawed" but anyone can find a reason to dislike anything, and that's especially so for nerds.
I'm not much of an early adopter. I can create solutions with existing tech just fine. I'd maybe take a look at Deno once AWS has it as runtime option in Lambda, but that seems unlikely to happen.
> I'd maybe take a look at Deno once AWS has it as runtime option in Lambda, but that seems unlikely to happen.
Are you using lambda currently for node? If so can I ask (at a high level) what your stack/deploy process is? I'm interested to know what someone who doesn't identify as an early adopter is using for lambda.
I constantly waffle my decision to go with lambda (Typescript/nodeJS) for my company. In some ways it's amazing, in other it's blow-your-brains-out frustrating. I've benefited from my use of it overall I think but I often wonder if Express/NestJS/other would have been the better route (if not going all the way back to my roots with PHP).
I'm using SST 2 (which uses CDK, which uses CloudFormation) to deploy my functions (1 function per endpoint). Unfortunately SST v3 decided to drop CDK and move to terraform but the upgrade path is almost non-existent. Essentially "Just spin up new infra with v3 then turn off your v2 stuff". I understand their decision and it was probably the right decision, it still burned me pretty bad. Needless to say, this has soured me (I already moved from Serverless Framework to SST a couple years ago and made it through the v1 -> v2 transition) and I'm considering alternatives. Ideally ones that don't require a complete rewrite but due to the fact that I'm not using any kind of framework (only my homegrown code/scaffolding) it might not be that hard to replace the "leaves" (endpoints) of my stack while keeping most of the other code the same.
>I'm interested to know what someone who doesn't identify as an early adopter is using for lambda.
I was actually an early adopter of Lambda. At the time I didn't realize it was new, I was just looking for a way to host a new nodejs project in a scalable way. I did not want to spin up an EC2 instance for an Express based project and then have to manage load balancing and everything else that comes with it when the project needed to scale. I thought, why not make it scalable from the start? So I searched and found Lambda, which had launched only a few months earlier. It seemed great and exactly what I was looking for. I didn't realize it at the time that it was so new. I found it during Christmas vacation 2014 where I had a couple of weeks of "downtime" away from work to focus on my personal project and Lambda quickly won me over. Lambda was released in November 2014, so it was really new and I was unknowingly a very early adopter of it.
There was no "Serverless" project at the time. So I created my own toolchain that I still use to this day. It's very simple. The dev program I created simply watches a folder for file changes, and then .ZIPs up the files, sends them to S3, then a companion Lambda function does "npm install" within a Lambda instance, then .ZIPs up the entire package and then updates the Lambda function I'm working on - this is done so any dependencies that require compilation will compile for the Lambda environment and not my local dev machine environment. It just works. I've added support for Layers when those became a thing, but other than that it's very simple and has withstood the test of time. I did try "Serverless" last year for another project, and it was not to my liking at all. I don't like giving "Serverless" access to my AWS account, and lots of other things made it not a good fit for me. YMMV.
Yeah, there are some frustrating times with Lambda, but those are usually related to getting stuff running like a headless browser, or ffmpeg, or other heavier lifts to run inside a Lambda. But, I've got all that stuff working, and once it works, it just keeps working. I haven't touched some of the older Lambdas in years, some are still running on node v12.
The only thing I kind of dislike is decommissioning of older nodejs runtimes so they aren't available at all for new Lambdas anymore. It's not that big a deal until I run into a situation where updating an older Lambda requires me to update the runtime, and there would be breaking changes due to different runtimes. But, it really is best if my code is updated to run on the latest nodejs version, although when I sometimes fall into this situation I start to question my choice of using Lambda. My regret is always short lived though, because Lambda is great in so many other ways.
That's very interesting, thank you for typing it up!
> Express based project
Ahh, so you use a monolith in a lambda. I should have gone this route. I was concerned about package size (due to lambda limits) and so went the "one function per endpoint" route instead. That seemed to be what AWS was saying you should do but it really screws you in a number of ways, size is the only place where it really helps. Provisioned lambdas, for example, are so much better for a monolith.
Did you not have size issues putting all the code in 1 package? That was a real limit I hit when I was (naively) packaging all my code for every endpoint instead of just the handler and the code it needed. I'd be nervous of pushing up against the limit. I guess the escape hatch is moving to a docker container on lambda since that gives you 10GB.
> The only thing I kind of dislike is decommissioning of older nodejs runtimes so they aren't available at all for new Lambdas anymore.
Yep, I've been burned by this and forced to update.
One more question, how do you handle local development? Do you just run Express directly? That's one reason I've leaned into the serverless frameworks out there. My function-per-endpoint setup doesn't have any default scaffolding, running via lambda is the only way to run the code (or one of the project that tries to replicate lambda locally). SST has been the best on this front since you deploy your dev environment like normal (spin up real queues, etc) but the lambdas are just proxying the request to the code running locally on your computer. This has resulted in the least "gotchas" when it comes time to deploy (though arch differences are always a blast).
I abandoned Express when I moved to Lambda. Express was just handing the routing, which I didn't need anymore with Lambda, API Gateway handles that now. My front-end also calls most Lambdas directly using Cognito for auth.
The only time I have problems with Lambdas growing too large is when I try to include a really big payload that includes something like Chrome for headless screenshots, or FFMPEG, or other large binary programs. But for normal Lambda functions I don't have any size issues. Recently when trying to get headless Chrome working in Lambda, I had to boot up EC2 running AWS Linux, and create a .zip file there with library files that were missing in Lambda, then .zip up everything and upload to S3 and then import the .zip into Lambda. After this, I can modify my .js files locally and have my toolchain upload the changes to Lambda to update the Lambda. But this time the payload was too large for importing into a Lambda, and too big for a Lambda layer. The solution was saving the .zip file with Chrome in it to an S3 bucket, and when the Lambda machine starts up, it first downloads the .zip file to /tmp, unzips it, and now the Lambda has the Chrome binaries in the /tmp folder. This bypasses the size limits associated with Lambda. It's rare that I need to do this, and it's not fun, but there are always going to be some edge cases like this with Lambda and getting large architecture specific binaries running in Lambda.
I actually put many different functions into a single Lambda, and I call the Lambda with a specific action parameter to perform a specific function.
Local development... when I create a Lambda function I may add some functionality so I can also run it on the command line if I'm still early in developing a complex function. But most of the time I just use my self-built toolchain which watches my local code for changes, then creates/updates the Lambda in AWS, which takes about 2 to 5 seconds typically. Then I test the code as it runs inside AWS. If I didn't have internet access, then it's impossible to develop in this way, but that's almost never the case.
Well said to that person. I think about this a lot, and have wondered if it's the advent of code school? Which itself was a result of the insane FAANG hiring sprees + salaries (where was a result of 0% interest rates among other things).
A developer going through code school in the last ~decade was taught React, GraphQL, Redux, or whatever the cool framework was at that moment with the goal of getting hired (not learning how to build well).
I'm fortunate to have entered the market in the early 2000's juuuust before that wave started. I'm only now getting a few gray hairs in my beard and am glad I was taught to understand _all_ parts of a system. I don't know if that'd be the case if I started in the mid-2010's.
In one way or another they are all circling around a concept of "one runtime environment that is as close as possible to the browser JavaScript runtime"
So in a sense, yes, they need to converge on the concept of a singular runtime, something which is achieved by fiat with server-side rendering.
Or when JS devs suddenly discover concepts like "serverside rendering" and reinvent PHP and ASP.NET.