Hacker Newsnew | past | comments | ask | show | jobs | submit | timholy's commentslogin

The point is, once you've gotten used to Julia you tend to write good code most of the time without even thinking about it. And that good code still "looks good," meaning it takes advantage of Julia's expressiveness and brevity. Understandably, newcomers make many more performance mistakes.

So there's often a huge difference between "unoptimized code" (something written by an experienced developer who's deliberately taking the easy way out) and "naive code" (something a newcomer might write). There can literally be orders-of-magnitude performance difference.

I agree that there isn't as much to learn about Python. But of course that's largely because of the gap in opportunities.


JuliaCon 2019: 500 attendees JuliaCon 2020: 20,000 attendees

Doesn't seem to be declining.


Was JuliaCon 2020 remote? That probably has more to do with it. I can’t possibly imagine there are 40X more active users in one year. The niche it serves isn’t particularly large, either.


Yes, it was completely online. The comparison doesn't make sense.

Also, next time it's IRL the numbers would seem to indicate a huge setback, so I wouldn't use those :D


I appreciate your well-balanced critique, thanks.

> Julia (at least by default) is constantly recompiling everything. This is a huge pain in a REPL style setup where you want to tweak one thing and see the changes, again and again. I know the Julia ecosystem is working on better caching etc to fix this problem but it’s a problem.

Maybe try Revise.jl? There are a few changes it can't handle, but you can do a lot of development without ever restarting. (Disclaimer: I'm its main author.)

> expertly crafted C usually beats Julia in performance

This isn't generically true, and there are now quite a few examples of the converse. I linked to it above as well, but check out the benchmarks in LoopVectorization's documentation (https://chriselrod.github.io/LoopVectorization.jl/latest/exa...) for examples of beating MKL, one of the most carefully-engineered libraries in existence.

I think an exciting area of growth for Julia will be exploiting the fact that Julia's compiler, written mostly in Julia, is more "morphable" than most and may develop its own plug-in architecture. This seems likely to provide performance opportunities that many fields seem hungry for.

> the Julia libraries are just not as mature as python’s

On balance I agree. While there are already many examples where Julia makes things easier than Python, as of today there are many more examples to the contrary. Julia's libraries are advancing rapidly, but I expect it will take a few more years of development until it's no longer so one-sided.


Check out LoopVectorization: https://github.com/chriselrod/LoopVectorization.jl

From its benchmarks (https://chriselrod.github.io/LoopVectorization.jl/latest/exa...), a 9-line naive matrix multiplication routine in Julia + LV slightly edges out Intel's MKL.


It looks like that @avx macro just tells Julia that it's okay to use the AVX instructions?

My specific question is, how do I tell Julia that I want to compute the popcount of the intersection of two byte strings of length 256 bytes?

A reference C code is at http://www.dalkescientific.com/writings/diary/archive/2020/1... in byte_intersect_256() and threshold_bin_tanimoto_search(); my blog posts shows the important parts - I link to the full definition for the C code.


@avx does a ton more than just use AVX instructions. It'll reorder and unroll loops when advantageous, swap out some functions for more vectorizable version of those functions and a few other tricks.

Julia uses avx instructions by default if your code is amenable to it.


Sorry, I see how what I wrote could be interpreted that way, and part of what I wrote was out of ignorance. While I didn't write it, I assumed the macro was doing some sort of equivalent to the template metaprogramming I've heard about in C++ to do similar things.

What I did't see was how I could use the AVX2 instructions myself.

Checking now, since Julia's count_ones() maps to the LLVM popcount instruction, and recent clang versions know how to optimize that fixed-length sequence in C even for AVX-512, the Julia equivalent to the code I wrote should have good performance.

There are a few optimizations (keeping one AVX register loaded with a constant byte string, and using prefetch instructions) which might be missing. I'll be talking with the conference participant who brought up Julia to work this out in more detail.

Thanks for the comment!


Ah I see, yeah I misunderstood you.

> What I did't see was how I could use the AVX2 instructions myself.

If you ever find yourself in a situation where you want manual control over vectorization, the package SIMD.jl [1] is pretty good for manual, handwritten vectorization. There's also VectorizationBase.jl [2] which LoopVectorization.jl uses. Which one of these two packages are most appropriate just kinda depends on what sort of interface you prefer.

[1] https://github.com/eschnett/SIMD.jl

[2] https://github.com/chriselrod/VectorizationBase.jl


Thanks for the pointers! The first looks like it gives a language for SIMD operations, but not all of the intrinsics. The second is for vectorization, which also doesn't include all of the intrinsics.

However! The second shows an example with Base.llvmcall, which lets me write LLVM assembly. That should let me do anything I want.

However however - the stuff I actually do doesn't need all that, and could likely be implemented with LoopVectorization.


If you're used to launching everything from your OS's command prompt, this is a killer. But if you get used to launching a REPL and running things from there, it's much better, because you only pay the startup latency once.


Some workflows require running scripts from the command line though


One reason for the choice may be encoded in the date of that post: April 2018. That was before the release of Julia 1.0. Were they making that choice today, I can't help but think the economics might be rather different.


For all those worried about whether Julia's indexing is good enough, let's broaden the conversation a bit. How good is your favorite language at, say, iterating along dimension m of an n-dimensional array, where m and n are not known by the programmer, and where you can't count on the array being strided? Trivial and extremely efficient in Julia: https://julialang.org/blog/2016/02/iteration.


Picking whatever you want is one line of code, and similar to the line you'd need to allocate any array. And your choice propagates to the downstream operations.

Instead of complaining in the abstract, check it out, you'll be impressed. https://julialang.org/blog/2017/04/offset-arrays


I already know about this but if you use someone else's code in a library for example in Julia the library will most likely only work with 1-indexed array, in Ada it will work with any base index.

Default matters!


It's straightforward to write library code in Julia that handles any type of index. If it doesn't, it is probably old code from before the time before offset arrays. Take a look at Tim's linked blog post.


It is straightforward to create a language where a user can specify any type of index (like Ada), if you need to import a library everywhere to have the correct behaviour then this library should be a part of the language instead.


As noted above, Julia does too. Heck, if you wanted to, in Julia with probably <100 lines of code you could create an array type that only allows prime numbers as indices. And subject to the constraints of the raw calculations needed, it would be as fast as it can be---there is no "extra" performance overhead associated with doing basically whatever you want.


If you compare Julia and Scipy at similar points in their evolution, Julia is far ahead of where Scipy was. And that's while the language itself is still evolving.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: