As a sibling mentions, the multiprocessing support is basically support for spawning new processes. It's better than nothing, but it's not very good for the performance.
The real problem with Python multicore is that the main problem is solves is the one ctur is talking about, namely, "Oh crap, I used Python and it doesn't run fast enough for my needs... maybe I can run more processes?" Using a language that is already in the slowest class of languages and realistically 40-50x slower than other languages means that just to recover the performance you'd get from switching to a compiled language, you need ~50 perfectly parallel processes solving an embarrassingly parallel problem. If you can't do that... and that's a fairly common case, things that are a full, true 50x embarrassingly parallel aren't actually that common on real hardware because you'll get some sort of contention... then you can't even work your way up to the performance that you could have gotten by starting with Go or C# or some other reasonable language.
And that's ignoring the serialization overhead between the processes. If that starts costing you noticeable amounts the number of processes you need to recover goes up really fast, due to the way the math works.
Note the only thing about Python causing this effect is its performance; it is equally true of anything else that is the noticeably slower performance league. So, also, note that if you are using Python in one of the ways where it doesn't have this performance problem, such as NumPy with almost all your compute in native code, this analysis doesn't apply.
In the 1990s when Python was born, programming in Python was massively easier than programming in the static languages of the day. The landscape has shifted... Python is now only incrementally better than modern static languages in some dimensions, and I tend to agree with ctur, it just plain isn't better once you pass a certain size and the stereotypical problems with dynamically-typed code start hitting you harder and harder. There are now a lot of good statically-typed languages where you can start a new project, writing something just incrementally harder to write than Python, with the type inference, built-in associative types, tons more libraries, etc. all the advances of the past 25 years, and get static-typed performance. The gulf between "easy Python" and "pull your hair out static language" is not quite closed. Not quite. But it's a lot closer than it used to be; less "Grand Canyon" as it was in the 1990s and more "can I jump that creek? I mean, it's pretty close... I think I can jump it...".
The upshot is, if you're reaching for multiprocessing for performance reasons, not convenience reasons, you've very nearly already lost. There's a narrow window where it might still make a bit of sense, but you're getting perilously close to "You need to switch languages" just by reaching for it at all.
I agree with this, but when it comes to scaling, making things faster at the language level is only one option. And while, as you suggest, the Python/static_lang gulf has narrowed, I would guess the gulf between "what needs to be done in the language itself" vs. "what can be done by some external system/database/process" has also narrowed.
What I'm getting at is, let's say you have some system where you need highly performant real time processing of some data. Perhaps 10 years ago, you might have needed a complex multi-threaded java/c++ app to handle it. But perhaps now, you use, say, DynamoDB, kinesis, kafka, lambdas, or some other collection of services that you're basically gluing together with... python.
I'm not saying you're wrong. I'm just wondering, if you could truly factor in the costs in both developer time -- developing multi-threaded apps is usually pretty tricky after all -- and infrastructure costs and whatnot, where the boundary between "python is adequate" vs. "we really need go/java/c++" lies, and in which direction it's moving.
In retrospect, I guess it's sort of a silly question, since at the core, the services we'd be gluing together are no doubt written in faster languages. Perhaps a better way of framing it what % of problems can be adequately solved (cheaply) by just using language X, and how has that changed over time?
"I'm just wondering, if you could truly factor in the costs in both developer time -- developing multi-threaded apps is usually pretty tricky after all -- and infrastructure costs and whatnot, where the boundary between "python is adequate" vs. "we really need go/java/c++" lies, and in which direction it's moving."
"It depends.", of course. Despite the significant slow down of single-core performance improvements, 1 CPU is a lot of power in a lot of use cases, even if you divide it by 50. I do frequently find myself reminding some people who get a little too deeply into the cloud mindset and the believe that any non-trivial problem needs clusters of systems that you can still do an awful lot with one CPU. And I often wonder how many "clusters" are out there drinking down the watts doing work that if somebody would just spend a week or two optimizing their code to get the O(n^2.5) algorithm out of their system could be comfortably done on a mid-grade laptop... if somebody just realized you shouldn't need a "cluster" to do this task.
However, flipping your perspective around is probably more interesting... multi-threading is still not "easy", per se, but it is also way easier than it used to be. Threading hell is true and exists, but a non-trivial amount of it was down to the architecture being attempted. It's a bad idea to try to coordinate everything with piles of mutexes everywhere. If you program with more agent-like approaches to resource management, even if you don't do it 100% like Erlang forces you, multithreading gets much easier. What kind of things does it open up to for it to be much easier than it used to be?
I still use Python for certain tasks where it is suited. But I'd have a hard time going back to it for most of my programming. I've internalized the ability to say "and I need a server here with its own thread of control managing this resource, and I need to set up a worker pool there for this data processing task, and I can set up a recurring, independent process to check this other thing periodically without it interfering with anything else" whenever it is necessary to ever be able to go back to architecting systems without that capability. I'm not going back to cooperative scheduling unless basically forced. There's just so many places where you ought to have this capability available to you and it's actually easier to work multithreaded rather than do the work to try to thread a single execution context through all the things that shouldn't be that tightly coupled together.
It is generally underappreciated that having to have two bits of code share an execution context is a deep level of coupling, and languages like Python force all your code into one execution context.
Having multithreading easily available has its downsides, yes, but it also has its legitimate upsides for architecture.
Really I think this is only true for the data science ecosystem. In most other cases, modern static language ecosystems are at least as advanced as Python.
The real problem with Python multicore is that the main problem is solves is the one ctur is talking about, namely, "Oh crap, I used Python and it doesn't run fast enough for my needs... maybe I can run more processes?" Using a language that is already in the slowest class of languages and realistically 40-50x slower than other languages means that just to recover the performance you'd get from switching to a compiled language, you need ~50 perfectly parallel processes solving an embarrassingly parallel problem. If you can't do that... and that's a fairly common case, things that are a full, true 50x embarrassingly parallel aren't actually that common on real hardware because you'll get some sort of contention... then you can't even work your way up to the performance that you could have gotten by starting with Go or C# or some other reasonable language.
And that's ignoring the serialization overhead between the processes. If that starts costing you noticeable amounts the number of processes you need to recover goes up really fast, due to the way the math works.
Note the only thing about Python causing this effect is its performance; it is equally true of anything else that is the noticeably slower performance league. So, also, note that if you are using Python in one of the ways where it doesn't have this performance problem, such as NumPy with almost all your compute in native code, this analysis doesn't apply.
In the 1990s when Python was born, programming in Python was massively easier than programming in the static languages of the day. The landscape has shifted... Python is now only incrementally better than modern static languages in some dimensions, and I tend to agree with ctur, it just plain isn't better once you pass a certain size and the stereotypical problems with dynamically-typed code start hitting you harder and harder. There are now a lot of good statically-typed languages where you can start a new project, writing something just incrementally harder to write than Python, with the type inference, built-in associative types, tons more libraries, etc. all the advances of the past 25 years, and get static-typed performance. The gulf between "easy Python" and "pull your hair out static language" is not quite closed. Not quite. But it's a lot closer than it used to be; less "Grand Canyon" as it was in the 1990s and more "can I jump that creek? I mean, it's pretty close... I think I can jump it...".
The upshot is, if you're reaching for multiprocessing for performance reasons, not convenience reasons, you've very nearly already lost. There's a narrow window where it might still make a bit of sense, but you're getting perilously close to "You need to switch languages" just by reaching for it at all.