Threads were historically expensive enough that “just spawn a thread” wasn’t a reasonable thing to do in many situations. Thread pools were sort of a last resort, and we ended up with control flow like objects (futures, await etc.) to multiplex concurrency without parallelism.
Go sort of asks why tho and just standardizes on go routines as a good abstraction over both concurrency and parallelism.
This is sorta true elsewhere too. Go rejects a lot of the machinery that OO languages seem to feel obliged to carry around - inheritance hierarchies, explicit interface implementation etc. For what it's worth, I don't write much go, and I don't think it's magical. I just like how clearly it revisited some basics.
Elsewhere on HA you’ll find my extended rant about how strange it is that we don't have a language that elegantly abstracts computation over threads, SIMD, GPUs etc. Compilers can do this sort of thing now, just not optimally.
> Elsewhere on HA you’ll find my extended rant about how strange it is that we don't have a language that elegantly abstracts computation over threads, SIMD, GPUs etc. Compilers can do this sort of thing now, just not optimally.
Autoparallelization has been a hot topic for literally decades, quite possibly longer than you've been alive.
The problem is that the techniques you need to do to write good SIMD code versus good GPU code versus good multithreaded code versus distributed computation are all different. Taking just memory concerns: a SIMD code needs you to carefully arrange memory so that every thread is accessing an adjacent memory location. GPU code likes locality, but you have large group sizes that can share all the local memory pretty cheaply, and loading from global memory to local memory is relatively expensive, so now you have to do a lot of tuned blocking. With multithreaded code, you now want to avoid sharing between different threads (which generally requires distributing loop iterations among threads very differently). And with a distributed platform, now you're primarily worrying about the overhead of communication of data between different nodes, and you're trying to minimize that.
Another axis: a GPU wants you to load a large batch of work and then start it - you can't be bouncing between CPU and GPU work all the time, but you can mix SIMD and non-SIMD instructions freely.
There is better than even odds I'm older than you, so I'd recommend you rethink using phrases like "possibly longer than you've been alive", it's not ... polite regardless of people's age.
The point (and I'd encourage you to find that thread to not retread ground) is that we absolutely can compile most computation heavy code for these different targets reasonably well - what we cannot garentee is that the resulting code is optimal given context. But gosh we can do so much - I’d encourage you to look into in profile guided, target aware, and autotuning optimization etc. (and then of course, there are LLM guided optimizations, but that's a whole other kettle of fish)
Different design goals, go ships with a runtime baked in, rust wanted an async design independent of runtime implementation that’d be usable in embedded contexts
In the nested function, the variable x is passed in edi, and the pointer to the nested stack frame is passed in r10. In the lambda, the variable x is passed in esi, and the 'this' pointer for the lambda is passed in rdi. The function-level ABIs end up being quite different.
I can write the signature of the generated function of a C++ lambda as a C function. I cannot write the signature of a generated GCC nested function as a C function.
If your argument is that ABI doesn't matter, then by all means, propose a patch to GCC to change the ABI and see if it gets accepted.
I can not write the signature of a function that requires a static chain (which exist in many languages) in C, because we have not added such a feature to the language. But we could. And we should, because we now need a (type-unsafe) extension (__builtin_call_with_static_chain) to invoke such functions.
I do not want to change the ABI for nested functions as it is a useful ABI and a cross-language standard. ABI obviously matters, but it is not a fundamental difference in implementation that makes nested functions fundamentally different to C++ lambdas. If your point is merely that a compiler translating nested functions to lambdas would need to adapt the ABI in this case, I agree. This is not difficult though. And this question is relevant only if one allows taking the address of a nested function, because as long as it is called only locally, the compiler can use whatever ABI it wants.
Okay, so you accept that nested functions have a different ABI than C++ lambdas. And it looks like you accept that neither ABI is going to change. So long as the two features have different ABIs, they cannot be compatible with one another.
> And this question is relevant only if one allows taking the address of a nested function
And that question is very relevant since taking the address of such functions (to pass to other functions, e.g., qsort) is one of the main use cases for their existence.
The ABI does not need to be compatible, because there is no way to call a C++ lambda directly from C.
If you take the address of a lambda function you get a pointer to an object of anonymous type, so you can not pass it to qsort, and qsort would also not know how to call this.
But if we added a feature similar to std::function_ref to C (i.e. a wide function pointer type), then such a type could be used to call both, nested functions and C++'s lambdas, and - in fact - many callable entities from other languages too. But for C++'s lambdas this would always involve a compiler generated thunk that adapts the ABI. This is also exactly what happens in C++ if you use std::function, because even in C++ you can not pass the address of lambda to a function without first erasing the type and creating the thunk.
Similar to how std::function_ref creates a thunk in C++ that calls the lambda so that it has a generic type-erased API that can be passed to non-templates, a conversion to a wide pointer would create a thunk that adapts the call from the nested function pointer ABI (that already exits for other languages also in LLVM whether we standardize the C feature or not) to whatever the lambda needs.
Showing things that are optimized by fully inlining into one function don't actually demonstrate equivalency, because you end up omitting anything that might actually evidence a difference in the semantics. And I get that, for your use cases, those differences might not matter. But as a compiler engineer, I can't say that only those use cases matter and therefore they're equivalent for all practical purposes.
It is also very unhelpful when you insist that this is "compatible" with other languages, where "compatible" actually means "compatible, if you put in a bunch of work in both languages to make something that makes them compatible, none of which I'm actually describing." Especially when there are competing proposals that do have compatibility in the sense of "I don't have to modify the C++ compiler to let it use this thing."
Equivalency does not mean that the everything has to be identical or even that the code has to be exactly identical for different implementations.
My point is that the implementation is structurally very similar: You synthesize a structure and put it on the stack and then pass a pointer to it around. It is so similar that you can certainly reuse your implementation of the lambda feature to implement this.
The wide pointer ABI question is also entirely orthogonal to other aspects, so we could decouple this discussion. The advantage of being compatible to other languages is because if we would use a common ABI that many other languages also use: Ada, Go, D, etc. In this case, no additional work has to be done for any of these languages. LLVM also supports this already.
The issue with C++ is that it does not use this common ABI and the closet thing it has even as a suitable API is std::function_ref. Where lambdas are not called locally, C++ already needs to create thunks anyway by going to some kind of these adaptors, so there is also no additional burden on the C++ side. One would simply have to implement this thunk in a slightly different way to adapt the calling convention.
I am not even sure that you need to do less adaption for other proposals, as many things the C++ semantics rely on do not exist in C (callable objects, templates), so you also need to adapt anyway at least in how you expose them in the language and in what other features you may need to make it work. In particular, JeanHeyds proposal does not even include the wide pointer part yet, which will also then be required at some point. The proposal also exposes far more features (different ways to capture), which makes it more work.
But I fully realize that the opposition for everything that looks different to C++ from the clang side comes from the perception that it is more work on your side. I can sympathize, but note that in GCC or other compiler that do not have a shared FE, we would essentially have to implement everything from scratch. So let's discuss this more if you want.
> The advantage of being compatible to other languages is because if we would use a common ABI that many other languages also use: Ada, Go, D, etc.
I have looked it up and I can already tell you that Go is not using the ABI you would be proposing. I cannot speak for the other languages.
> But I fully realize that the opposition for everything that looks different to C++ from the clang side comes from the perception that it is more work on your side.
That is not where the opposition comes from, and for as long as you continue to believe that, you will fail to understand the opposition at all.
"Closure calls follow the same conventions as static function and method calls, with one addition. Each architecture specifies a closure context pointer register and calls to closures store the address of the closure object in the closure context pointer register prior to the call."
In any case, the documented use of __builtin_call_with_static_chain in GCC and Clang is to be able to call closures of other languages, and for GCC Go is explicitly mentioned.
tl;dw: On older versions of Portal, setting mouse sensitivity to INF and using the strafe key to make mouse movement change position instead of look angle when the player is near a portal triggers a failsafe that sets the player coordinates to (0, 0, 0) if in the air or (0, 0, z) if on the ground. This can be used to speed up the completion of some maps for the in-bounds any% category.
ha! I was happily surprised to see this article posted because it felt like it was picking up where the technical discourse was before LLM's took over.
2022 explains that perfectly, albeit leaves me less happy.
To add on top this, the author wrote multiple blog posts on their setup [0, 1, 2]. The project itself is hosted at https://github.com/dbalsom/arduinoX86
> Only the small and unimpressive programs can be checked exhaustively.
Even if you assume that statement is true, there are techniques other than exhaustive checking/model checking. Proof assistants/theorem provers/etc. like Rocq/Isabelle/Lean are quite capable of formally verifying programs without needing to exhaustively explore the search space.
I'd question the accuracy of that statement in general as well; model checkers like CBMC/TLA+ are handy for proving properties about interesting systems. The latter, for example, sees use for verifying concurrent/distributed systems, which I think can be reasonably described as more than "small and unimpressive"
Is it "widespread"? The article you link is from 2022 and none of the discussion I saw on said article gave me the impression that it was particularly common issue back then, let alone now.
It is a mess, temporary lifetime extension is a mess in both Rust, C++ and Zig (despite Zig not having RAII unlike Rust and C++). Interestingly, Mojo might avoid some or all of that, by having some destructors be implicit, and some destructors be explicit, https://mojolang.org/docs/manual/lifecycle/death/ , requiring users to write the destructors manually. Thus, a lock in Mojo can be forced by the compiler to be explicit, and that prevents the Rust problem of https://fasterthanli.me/articles/a-rust-match-made-in-hell , since developers will have to explicitly destroy the lock in Mojo. C also does not have that temporary lifetime extension issue, since there is barely any temporary lifetime extension in C, apart possibly from compound literals, but compound literals might also have some issues and involved rules.
The edition system in Rust tripping LLMs up is not great either. That can happen because the same code in one edition of Rust can have very different behavior in another edition of Rust, like the same piece of Rust code having a deadlock in one edition and not in another.
Interesting, thanks for the links! I'm curious as to how many of the apparent issues are attributable to lifetime extension specifically as opposed to something else like async-related deadlocks as mentioned in the parent to the first linked comment (which I feel I've heard much more about), especially after the Rust 2024 changes.
That being said, after a bit more searching I found this 2023 blog from one of the Rust devs [0] which supports the "widespread" description at the time:
> One very common problem is deadlocks (or panics, for ref-cell) when mutex locks occur in a match scrutinee
so I think we can chalk this up to me being insufficiently well-read. I think it would be interesting to see to what extent the Rust 2024 changes alleviated the problem since it only changed if let, but I haven't found that information (yet).
There's also this related work [1], but I think the scope of that is rather larger.
> Interestingly, Mojo might avoid some or all of that, by having some destructors be implicit, and some destructors be explicit, [] , requiring users to write the destructors manually.
There's some relevant exploration being done in Rust that in principle could enable linear types [2], though obviously it remains to be seen to what extent this work will pan out.
> I think it would be interesting to see to what extent the Rust 2024 changes alleviated the problem since it only changed if let, but I haven't found that information (yet).
Regarding "super let", it looks interesting, but I am not sure about the details of it. This is the tracking GitHub issue https://github.com/rust-lang/rust/issues/139076 . There is also the challenge of backwards compatibility, which makes the feature harder to make good, I suspect.
> claims that some issues were made worse in practice in his experience by Rust edition 2024.
That begs the question of exactly what "some issues" encompasses. To me, the poster was clearly complaining about the increased use of a particular code pattern post-Rust-2024, but it's not clear to me that they were also claiming that there is a proportional increase in deadlocks post-Rust-2024. For instance, perhaps it's the case that people use that code pattern more because it doesn't deadlock; this may not be desirable from the commenter's perspective due to the multiple locks/unlocks, but that's a distinct issue from lifetime extension causing deadlocks.
> Regarding "super let", it looks interesting, but I am not sure about the details of it.
At least from a cursory skim it looks like the specifics are still being worked on, so it's hard to fault you for being unsure about the details. It is an experiment after all :P
Does it matter? The announcement is that they're funding a PhD project. I don't think it's that unusual to fund a project whose outcome you are interested in even if you don't have the expertise needed to carry it out yourself.
> Same people who keep the whole rust project going, a lot of those are volunteers aren't they?
Sure, but from my understanding the Rust project is generally "bottom-up" in that volunteers generally work on what they want to rather than submit their time into a pool for some kind of higher-level management to direct.
Do you mind elaborating on this? I don't understand what you're trying to get at.
reply