1. A python script to make sure you have the correct environment, and that kicks off cargo to compile the program used in step 2. ("rustbuild" proper.)
2. A Rust program that manages the full build (remember, Rust is bootstrapped, so there's three stages...)
3. That build kicks off a number of "cargo build"s for all the components and glues the final result together.
For any project written in Rust, pure Cargo should be enough.
I've always been a bit confused about Rust's need for so many build stages. The project I work on (crystal) only requires you to install a working build of the latest compiler release to build. (the makefile is pretty simple). As far as I know, rust can't bootstrap without downloading a rust release, so can you explain why rust has compiler stages.
1. This stage is a compiler built with the compiler from stage 0
2. This stage is a compiler built from stage 1.
If the process worked, then stage 1 and stage 2's output should be identical. You can skip stage 2 if you want to skip that verification, and that sounds like what Crystal does.
I always mess this up, but here's something from memory: I believe there's ABI things that motivate stage 2. The compiler built in stage 0 has historically had special ifdefs to deal with "features that don't exist in the old snapshot" (we aggressively adopt our own features in the compiler and stdlib, because that's a great way to test them). This implies it can potentially emit binaries with a different ABI from the compilers built in stage 1 and 2. So the compiler that's output from stage 1 can actually have a different ABI from the binaries it produces.
Normally this wouldn't matter, but Rust also supports compiler plugins written in Rust. These link directly to the compiler that compiles them. With the stage 1 compiler's quirky ABI, the plugins it compiles might not be able to interact with it.
The stage 2 compiler, on the other hand, uses the same set of ifdefs that the stage 1 compiler has. So it was built by a compiler with the same output-ABI as itself, and so plugins have the same ABI.
So:
* snapshot: Built with snapshot ABI, produces snapshot ABI (snapshots are old stage2 compilers)
* stage0-output: built with snapshot ABI, emits stage0 ABI.
* stage1-output: built with stage0 ABI, emits not(stage0) ABI.
* stage2-output: built with not(stage0) ABI, emits not(stage0) ABI.
Ah yes, crystal doesn't have compiler plugins, which simplifies things a bit. Say, if you specified that you had to be able to build rust master with the latest nightly release, could you use only 2 stages (latest nightly, built master)?
I'm currently working on crystal's CI infrastructure so that we can have nightly crystal builds (+ CI) for all architectures we support. Currently we release features incrementally such that the compiler can always be built with the latest release, and I'm wondering how much effect relaxing that constraint to the latest nighty would have on development speed.
Additionally, if one is building a compiler for other targets (both a compiler that runs on another target, and just a simple cross-compiler), the makefiles/rustbuild need to make sure the appropriate things get cross-compiled in the appropriate stages. Managing this was a source of a lot of complexity in the old makefiles.
Is it possible to bootstrap from a compiler compiled from C, or another language? Or is there another way to ensure there aren't any sneaky compiler backdoors?
https://github.com/rust-lang/rust/tree/master/src/bootstrap
It basically consists of:
1. A python script to make sure you have the correct environment, and that kicks off cargo to compile the program used in step 2. ("rustbuild" proper.)
2. A Rust program that manages the full build (remember, Rust is bootstrapped, so there's three stages...)
3. That build kicks off a number of "cargo build"s for all the components and glues the final result together.
For any project written in Rust, pure Cargo should be enough.