> I’m convinced there’s a contingent of devs who don’t like/grok abstraction.
I am one of those. I grok abstractions just fine (have commercially written idiomatically obtuse Scala and C#, some Haskell for fun, etc.), but I don't enjoy them.
I use them, of course (writing everything in raw asm is unproductive for most tasks), but rather than getting that warm fuzzy feeling most programmers seem to get when they finish writing a fancy clever abstraction and it works on the first try, I get it when I look at a piece of code I've written and realize there is nothing extraneous to take away, that it is efficient and readable in the sense of being explicit and clear, rather than hiding all the complexity away in order to look pretty or maximize more abstract concerns (reusability, DRY, etc.).
This mindset is a very good fit for writing compute-heavy numerical code, GPU stuff and lots of systems level code, not so much for being a cog in a large team on enterprise web backends, so I mostly write numerical code for physics simulations. You can write many other things this way and get very fast and bloatfree websites or anything else, but it doesn't work well in large teams or people using "industry best practices". It also makes me prefer C to Rust.
If that's true, how are they so much more reasonable in most developed countries with far greater government involvement still? Is the US government just uniquely bad at healthcare somehow? Why?
… Where on earth are you getting that? As a high earner in a European country, about 8-9% of my income goes on the health service (though that includes some non-healthcare stuff). And I’m an extreme outlier; multinational salary and equity, single, no kids. For a single childless person on the average wage it’s about 2.5%.
I don't know how other countries manage their health care systems, though I know that the British one is facing bankruptcy, and while health care was free in the Soviet Union patients had to pay for anesthetic for root canals, and bribery was the norm.
Here's a link to what's wrong with the American system:
This looks like a nice case study for when you're already using Rust for other reasons and just want to make a bit of numerical code go fast. However, as someone mostly writing C++ and Julia, this does not look promising at all - it's clear that the Julia implementation is both more elegant and faster, and it seems much easier to reproduce that result in C++ (which has no issues with compile time float constants, SIMD, GPU support, etc.) than Rust.
I've written very little Rust myself, but when I've tried, I've always come away with a similar impression that it's just not a good fit for performant numerical computing, with seemingly basic things (like proper SIMD support, const generics without weird restrictions, etc.) considered afterthoughts. For those more up to speed on Rust development, is this impression accurate, or have I missed something and should reconsider my view?
This sort of thing is where Julia really shines. https://github.com/miguelraz/StagedFilters.jl/blob/master/sr... is the julia code and it's only 65 lines and uses some fairly clean generated code to get optimal performance for all floating point types.
Regarding SIMD support, the only thing that is missing, is stable support for avx512, and some more exotic feature extensions for deep learning e.g. avx_vnni. Those are implemented and waiting to be included in the next stable versions.
Gpu support: this is still an issue b/c of not enough people working on it, but there projects trying to improve this:
see https://github.com/tracel-ai/cubecl .
Const generics: Yeah, there are a few annoying issues: it is limited to small set of types. For instance, you cant use const enum as a generic. Also, you cant use generic parameters in const operations on stable rust: see unstable feature generic_const_exprs.
My main reason for using rust in numerical computing:
- type system. Some find it weird. I find it explicit and easier to understand.
- cargo (nicer cross platform defaults, since I tend to develop both from windows and linux)
- unconditional code generation, with [target_feature(enable = "feature_list")]. This makes it so that I dont have to set different set of flags for each compilation unit when building. It is enough to put that on top of function making use of SIMD.
I agree that if you want to be fast/exploratory in developing algo and you can sacrifice a little bit of performance, Julia is a better choice.
Yeah, it so fast that AMD is not even able to catchup and that many of the extensions is available only on Intel CPUs.
As far as I could tell, it is only unstable in the sense of being fast and having many features. I dont see any breaking for my code using cpuid to detect avx512 features.
No, I mean things like when they removed BCD support, or when they removed simultaneous 16-bit ISA when they added 64-bit. Or the bit string instructions. Or moving the cmpxchg encoding across chip revisions.
While this is completely true, it is also true that OpenCL 1.2 is the one compute API that just works on every major platform and the drivers don't seem that unusably bad (though I'm not claiming experience of every platform here, just Nvidia on Windows/Linux and Apple Silicon on MacOS). Writing a limited dialect of C and sticking to 1.2 limitations is far from ideal, but it does at least work reliably. Sadly, that is more than can be said about most competitors.
The problem is that the drivers are merely OK. Presumably if you're using OpenCL you care about the performance (otherwise why would you??) and since that's the case, it's the best on no platforms, and there are alternatives for any set of platforms that do better.
I think OpenCL is sadly on its way out, and it's mostly Apple's fault (and Nvidia a little). Vulkan compute is much more interesting if you're looking to leverage iGPUs/mobile/other random CPUs.
If you're targetting workstations/server workloads only, it makes sense to restrict yourself to a subset of accelerator types and code for that (eg. Torch or JAX for GPUs, use highway for SIMD, etc.)
It depends on what you're doing. For writing FP32 number crunching code from scratch (meaning you don't care about something like Torch, or even cuBLAS/cuDNN), I haven't encountered cases where I couldn't match CUDA performance and if I did, I could always just use a bit of PTX assembly where absolutely necessary (which OpenCL lets you do, whereas Vulkan does not). This also gets me good performance on MacOS without rewriting the whole thing in Metal. There is no native FP16 support and there are other limitations that may matter to your usecase or be completely irrelevant.
I'm definitely not saying OpenCL is any sort of a reasonable default for cross platform GPGPU work. In truth, I don't think there is any reasonable "general" default for that sort of thing. Vulkan has its own issues (only works via a compatibility layer on MacOS, implementation quality varies widely, extension hell, boilerplate hell, some low level things are just impossible, etc.) and everything else is a higher level approach that can't work for everything by definition.
It's a pretty sad situation overall and every solution has severe tradeoffs. Personally, I just write CUDA when I can get away with it and try to stick to OpenCL otherwise, but everyone needs to make that choice for their own set of tradeoffs.
Yeah, TBH I'm kind of sad about where OpenCL ended up, because it "should have" been what CUDA was used for from 2011-2021. AlexNet, TF, Pytorch, etc. "should have" been written with OpenCL backends.
But the driver implementations inconsistency, version support issues, etc. meant people used CUDA instead.
I agree Vulkan has its own issues, and having written some MoltenVK stuff, you clearly know the quality-of-life pains in developping with it. That said, at least from the user side it works and performs well.
It is Intel, AMD and Google's fault for never supporting OpenCL as they should, and Khronos for pissing off Apple with how they took ownership of OpenCL.
Is there a way to directly use these developments to already write a reasonable subset of C/C++ for simpler usecases (basically doing some compute and showing the results on screen by just manipulating pixels in a buffer like you would with a fragment/pixel shader) in a way that's portable (across the three major desktop platforms, at least) without dealing with cumbersome non-portable APIs like OpenGL, OpenCL, DirectX, Metal or CUDA? This doesn't require anything close to full libc functionality (let alone anything like the STL), but would greatly improve the ergonomics for a lot of developers.
I'll describe what we've got, but fair warning that I don't know how the write pixels to the screen stuff works on GPUs. There are some instructions with weird names that I assume make sense in that context. Presumably one allocates memory and writes to it in some fashion.
LLVM libc is picking up capability over time, implemented similarly to the non-gpu architectures. The same tests run on x64 or the GPU, printing to stdout as they go. Hopefully standing up libc++ on top will work smoothly. It's encouraging that I sometimes struggle to remember whether it's currently running on the host or the GPU.
The datastructure that libc uses to have x64 call a function on amdgpu, or to have amdgpu call a function on x64, is mostly a blob of shared memory and careful atomic operations. That was originally general purpose and lived on a prototypey GitHub. Its currently specialised to libc. It should end up in an under-debate llvm/offload project which will make it easily reusable again.
This isn't quite decoupled from vendor stuff. The GPU driver needs to be running in the kernel somewhere. On nvptx, we make a couple of calls into libcuda to launch main(). On amdgpu, it's a couple of calls into libhsa. I did have an opencl loader implementation as well but that has probably rotted, intel seems to be on that stack but isn't in llvm upstream.
A few GPU projects have noticed that implementing a cuda layer and a spirv layer and a hsa or hip layer and whatever others is quite annoying. Possibly all GPU projects have noticed that. We may get an llvm/offload library that successfully abstracts over those which would let people allocate memory, launch kernels, use arbitrary libc stuff and so forth running against that library.
That's all from the compute perspective. It's possible I should look up what sending numbers over HDMI actually is. I believe the GPU is happy interleaving compute and graphics kernels and suspect they're very similar things in the implementation.
I’m cautiously optimistic for SYCL. The absurd level of abstraction is a bit alarming, but single source performance portability would be a godsend for library authors.
This is one area where I imagine C++ wannabe replacements like Rust having a very hard time taking over.
It took almost 20 years to move from GPU Assembly (DX 9 timeframe), shading languages, to regular C, C++, Fortran and Python JITs.
There are some efforts with Java, .NET, Julia, Haskell, Chapel, Futhark, however still trailing behind the big four.
Currently in terms of ecosystem, tooling and libraries, as far as I am aware, Rust is trailing those, and not yet being a presence on HPC/Graphics (Eurographics, SIGGRAPH) conferences.
This is one area where I imagine C++ wannabe replacements like Rust having a very hard time taking over.
I 100% agree. Although I have a keen interest in Rust I can’t see it offering any unique value to the GPGPU or HPC space. Meanwhile C++ is gaining all sorts of support for HPC. For instance the parallel stl algorithms, mdspan, std::simd, std::blas, executors (eventually), etc. Not to mention all of the development work happening outside of the ISO standard, e.g. CUDA/ROCm(HIP)/OpenACC/OpenCL/OpenMP/SYCL/Kokkos/RAJA and who knows what else.
C++ is going to be sitting tight in compute for a long time to come.
HPC researchers already employ some techniques to detect memory corruption, hardware flaws, floating point errors, and so on. Maybe Rust could meaningfully reduce memory errors, but if it comes at the cost of bounds checking (or any other meaningful runtime overhead) they will have absolutely zero interest.
If you’re willing to deal with 5 layers of C++ TMP, then a library like Kokkos will let you abstract over those APIs, or at least some of them. Eventually if or when SYCL is upstreamed in the llvm-project it’ll be possible to do it with clang directly.
Completely agree with all of that, but I would add that even if you never have to write any assembly at all, just the ability to read your own code in disassembled form is a superpower. It's not hard to learn the basics and, in addition to the ability to debug performance issues, it will forever inoculate you against the "magic compiler" delusion and keep your code somewhat grounded when you actually see what any new abstraction actually turns into.
I don't own a Tesla, but if I refused to buy any product with misleading marketing, I'm not sure I'd ever buy anything more sophisticated than a loaf of bread. Then again, I also don't use any driver assist features beyond parking sensors and don't expect to see any actual autopilot within my lifetime, so maybe I just haven't been as annoyed by Tesla's nonsense. Regardless, for a purchase most people regard as rather significant, it doesn't seem unreasonable to expect them to look at least marginally beyond a marketing misnomer.
Certainly not arguing that it's fair or just, but it's important to distinguish the mechanisms to have any hope of potentially improving the situation.
Can you elaborate on your experience? There's a lot of negative D opinions from people who've barely used it and, for obvious self-selection reasons, a lot of positive ones from long term true believers. It would be interesting to see a negative experience from someone who used it extensively nonetheless.
I can't even begin to imagine the kind of corporate dysfunction that leads to, first, at long last introducing the biggest change in the instruction set since the Haswell era into the consumer CPU line, slowly getting some actual adoption on that, and then suddenly dropping it altogether (for rather dubious reasons), but actually not really, so random motherboard vendors get to re-enable it behind your back.
This is crazy, both in terms of CPU design and just basic communication. Personally, I'll be waiting out this generation until I see AMD's response. Taking AVX-512 seriously is even harder than it already was.
Wouldn't it be possible for a process to ask to be pinned to a specific CPU core, or the OS to handle programs with extra tags indicating their ISA requirements, so that it could restrict the scheduling to those cores matching the requirements?
About a decade ago, as an intern helping research this, I developed an app to manually tag processes as "big core only" or "little core only" etc. This was for the QuickIA[0], which had a Xeon in one socket and an Atom in the other. It was fun to kick off a gigantic compression task and watch the "expected time to completion" progress bar change dramatically when you migrated the process between sockets.
You all would probably find the "QuickIA Software Support" section interesting, they go through a few issues you run into when you have incompatible micro architectures running in a single system. It also might help illuminate where Intel's head was at during the early research phase of creating a heterogeneous processor.
That's what people generally expected when the whole P-core vs E-core on different microarchitectures design was announced. Instead of going for this, Intel went out of their way to state that there would be no AVX-512 even on P-cores and that's that. Ignoring this obvious solution is part of why I'm so flabbergasted at the situation.
Then they can just say so and spin it as a PR win - "look how security conscious we are these days". Very publicly removing a formerly-headline feature with no explanation and then even failing at that is just mind boggling.
I decided to search for SMP, which stands for Symetric Multi Processing, to verify its definition. This acronym is used for hardware sporting several identical CPUs, which is what we've seen so far in the x86 world.
The opposite is called AMP, for Asymetric Multi Processing. The Wikipedia article on the subject describes several OS level strategies to handle that sort of hardware.
I am one of those. I grok abstractions just fine (have commercially written idiomatically obtuse Scala and C#, some Haskell for fun, etc.), but I don't enjoy them.
I use them, of course (writing everything in raw asm is unproductive for most tasks), but rather than getting that warm fuzzy feeling most programmers seem to get when they finish writing a fancy clever abstraction and it works on the first try, I get it when I look at a piece of code I've written and realize there is nothing extraneous to take away, that it is efficient and readable in the sense of being explicit and clear, rather than hiding all the complexity away in order to look pretty or maximize more abstract concerns (reusability, DRY, etc.).
This mindset is a very good fit for writing compute-heavy numerical code, GPU stuff and lots of systems level code, not so much for being a cog in a large team on enterprise web backends, so I mostly write numerical code for physics simulations. You can write many other things this way and get very fast and bloatfree websites or anything else, but it doesn't work well in large teams or people using "industry best practices". It also makes me prefer C to Rust.