One rule that should be added here, but it might be a bit late:
* choose a language that supports async operations so that you aren't waiting for external webrequests or database calls to get back to local code.
A synchronous language can turn your big, beefy machine into a single concurrent transaction language and _that_, beyond any other pain, can really, really, really reduce your total throughput.
Pick languages that can do other things while they're waiting for IO.
Are there good articles on the benefits of async over something like PHP-FPM?
At $WORK, we run a lot of 1000s of php servers with 10-20 processes each using nginx. External calls we don’t need the result of immediately get kicked out to an external queue, but we haven’t seen any issues with DB calls blocking the whole server.
Long or slow DB connections are an issue for individual request time and DB load though (the biggest DBs we run handle 1000-2000 concurrent connections).
I think one of the forgotten blessings of PHP/CGI was that requests had to finish sometime so as to free the PHP process for the next request. This alone I feel ended up making decisions easy around when it felt right to offload stuff when slow requests started to affect index.php from rendering in a timely fashion but now that's often masked (rightly) by caches and other tricks.
Right now in a microservice model often you have to remind developers that it just isn't ok to have a ~1ms response in a basic case and a 1 second to infinitely blocking case during errors or slow remote services.
At the level of a programming language, the win from async isn't one of performance (at any given time you're operating at most one process per core, and you'd generally get better performance if it were always the same process for each core doing exactly the right thing at the right time, including choosing when to interact with your asynchronous network card), but rather one of developer ergonomics; it can be a lot easier to interact with async things (those network events) by making everything that calls into it async as well and relying on some framework to turn your async code into something that actually matches the machine's execution model.
I think it's relevant. Let's say you write an API that only needs to call another API that takes 1 second to respond. A golang or nodejs server would have no problem to serve thousands of parallel requests. You can't really justify having 5000 ruby threads, they are heavy and they will use too much memory.
This is not really a new thing but maybe it's not well known.
Ruby's had aysnc IO support for at at least 10 years I think.
Rails itself has been using Concurrent Ruby since at least Rails 5 but I think it had its own concurrency patterns even before that. e.g. ActiveRecord database drivers.
Jesse Storimer's Working with Ruby Threads book from 2013 is still a really good resource for concurrency in ruby.
It's very relevant if the external webservice calls take long enough.
Multithreading just increases your technical max concurrent request up to the size of your multithreading maximum.
Async lets the computer do _anything else_ while it's waiting for the request.
If you're handling thousands of concurrent requests, you need thousands of threads available ... or, 10s or hundreds of threads with async available (your mileage may vary).
Multithreaded python sorta works like async in web situations - the GIL still limits to one thread actually interpreting python code, but a thread can run while another one is blocking waiting for IO.
I've still run into issues where an external service took approx 1 second per call. We had 20 threads on each docker container running Python. We had a lot of web requests.
It regularly took down the multithreaded python instance.
The solution we went with was to put all the different web requests into 2 flask servers and then take the output of all those different web requests and make one big request tot the django instance that had all the business logic.
If memory serves, we only had 2 flask instances handling all the traffic that the old array of django servers.
Is async Django completed? I thought the initial async functionality wasn't 100% of Django being async, only specific parts (Like views?). Maybe I'm remembering wrongly though.
I've since moved on to a different company that does not use Python for its main services. Django async came out after I left. I was very excited to see it arrive, though, as I expected it would improve the areas of concern I had and reduce the need for the extra layer.
I still don't know why people are afraid of multiprocess python. Sure it's a bit more complicated, but not that much more. Use those processor cores. I use it all the day for doing things in parallel on testing hardware and deployments. Is it really that much harder for web?
Even then, it's a bit misleading; make it so concurrency, in general, is easy and efficient. Async is immaterial if you have green threads or similar (a la Go or Erlang). Point is, you shouldn't risk a model that ends up unnecessarily blocking/synchronizing execution.
Do you have any examples of this being used with Rails? I'm having a bit of trouble finding examples of both Socketry and Rails being used concurrently; and, I will admit, my Rails knowledge is a bit .... intermediate or less.
That's the earliest I could find off the top of my head - But the other comment responding to you found an even earlier one where they mention the "select" feature that was added sometime around 1998 (22 years ago). The twisted async networking library seems to have been released/started at least 13-14 years ago as well.
I think we all (myself included) kinda forget or underestimate how old Python really is.
I don't think I have even seen memory use be a benchmark for server frameworks. When is that ever actually a problem? Wouldn't that mean hundreds of thousands of in flight requests?
Python 3.7 and friends do have async support; and, Django has been recently gaining it, finally.
Prior to the new async stuff, though, Python was a horrible language if you had internal microservices bouncing requests off each other - the edge layer would wait a very long time for all those secondary requests to finish before it could respond.
OK, but please don't post unsubstantive/flamebait comments to HN, and certainly not personal attacks.
If you know more, the thing to do is to share some of what you know, so the rest of us can learn. If you don't have time or don't want to do that, that's fine of course, but then please don't post.
Edit: it looks like you've unfortunately been posting a lot of unsubstantive/flamebait comments. We ban that sort of account because we're trying for curious conversation here. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and sticking to the rules when posting, we'd be grateful.
* choose a language that supports async operations so that you aren't waiting for external webrequests or database calls to get back to local code.
A synchronous language can turn your big, beefy machine into a single concurrent transaction language and _that_, beyond any other pain, can really, really, really reduce your total throughput.
Pick languages that can do other things while they're waiting for IO.