I quite liked this article, but both the article and the comments here mostly skip over my absolute favorite Julia feature: because Julia is fast end-to-end, you can code in the style that naturally matches your use case and mental models without sacrificing performance. Functional patterns and imperative patterns, vector-based or element-by-element, your types or built-in: they all work well and run quickly!
I do most of my programming in Julia and Python. Python libraries like numpy and pandas are fast and efficient—if you're staying "within the lines" of how the library is designed to work. And most of the time this is okay! But not irregularly I want to do array operations on arrays of user-defined types, or I want to walk through a dataframe row by row without paying a huge performance penalty, etc. And all the sudden the well-tuned Python ecosystem feels very restrictive.
In Julia, my workflow is roughly: 1) think about the problem. 2) Code an intuitive solution. 3) If necessary, tweak a little bit of code to improve performance by reducing allocations or type instability.
That's a lot less mental work than my Python/Matlab/Java workflow of 1) think about the problem. 2) Think about how the solution can be expressed in the paradigm the language supports performance with. 3) Write a solution in this particular paradigm. 4) Tune for performance, which may be awkward if the initial solution was not intuitive.
I believe the essence of what you are saying is that Julia is great for explorative programming, while Python et al are better suited for slightly more structured, more productionized programming.
I personally use Mathematica for this purpose and find it indispensable. I wonder if the PL designers should focus more on this class of languages, instead of features for production languages (like fancy type systems).
> Julia is great for explorative programming, while Python et al are better suited for slightly more structured, more productionized programming
In Julia, explorative programming is productionized programming, you don't have to switch to a different or restricted set of tools to make your program fast, it's the same code that you can profile on the fly. With types, you can patch your hotspots by adding a specific method to the generic function, and it will get invoked automatically instead of the more general case algorithm.
In those languages, I can actually take the theoretical practical hardware limits (memory BW, throughput, latency), and relatively easily achieve 99% of their utilization.
When people say “very fast” and “my cores are at 100%” they often mean “my program achieves 1% of the perf the hardware can deliver”.
Is Julia JIT deterministic / reproducible? It doesn't matter if you are writing the code to run yourself on your own machine, which is usually the case in research. But would you recommend Julia for a project has soft realtime requirements on the order of 10 ms and has to run in hundreds of machines scattered across the world, and you have to support personally?
Probably not in default configuration, but we have some people interested in this kind of thing. The compiler candy definitely be configured to make that work, but so far interest hadn't been sufficient to implement that. If you have a particular use case in mind, do let us know to see if we can help you out.
No, that is what people cannot wrap their head around. They always think there is a catch or a downside with Julia. But Julia is really to have a cake and it too.... ok you got to accept your first plot is slow ;-)
You have to accept that compilation is on the fly, so you cannot test the actually binary code your customers will run. There is an attempt at AOT in a library but it did not look stable last I checked (when Julia went 1.0).
If comment was made WRT alleged fit for demanding performance - JIT-ted languages usually can't compete and unpredictable performance could mean unpredictable failures.
Julia, by virtue of being compiled, is also probably better for production. But it’s much less well known and has less library support, which is a significant barrier to use.
How much of a speed difference is there for iterating over the rows of a data frame in Julia vs pandas itertuples with index false and name none? Any ballpark figure will do, I’m curious.
For a DataFrame with 1 column containing integers 0 to 99,999, in Python I'm seeing ~16ms to add up the column by iterating with itertuples (index false, name none - which I didn't know helped performance, so thanks!). Julia it's 8ms to add up when iterating with eachrow and accessing the column by name.
So ~2x faster in Julia? Though it's a sort of awkward comparison, since this type of iteration seems a little anti-idiomatic in Python/pandas and doesn't fully leverage type information in Julia.
I do most of my programming in Julia and Python. Python libraries like numpy and pandas are fast and efficient—if you're staying "within the lines" of how the library is designed to work. And most of the time this is okay! But not irregularly I want to do array operations on arrays of user-defined types, or I want to walk through a dataframe row by row without paying a huge performance penalty, etc. And all the sudden the well-tuned Python ecosystem feels very restrictive.
In Julia, my workflow is roughly: 1) think about the problem. 2) Code an intuitive solution. 3) If necessary, tweak a little bit of code to improve performance by reducing allocations or type instability.
That's a lot less mental work than my Python/Matlab/Java workflow of 1) think about the problem. 2) Think about how the solution can be expressed in the paradigm the language supports performance with. 3) Write a solution in this particular paradigm. 4) Tune for performance, which may be awkward if the initial solution was not intuitive.