Rustdoc merged doctests (solved) issue on stable

The 1.85.1 Rust version just got released, and with it comes a fix for rustdoc merged doctests feature. You can take a look at the official blog post here.

As a reminder, rustdoc is the Rust tool used to generate documentation. We recently added a new feature called "the merged doctests feature" which allows to greatly reduce the time needed to run doctests (code examples in the documentation). I explained in details how it works here.

So now, let's first explained what went wrong.

Did someone say stable?

First: if you didn't notice anything when switching to the 2024 edition, it's normal: in case the merged doctests failed to compile, then rustdoc fallbacks to the original behaviour of compiling and running each doctest separately.

The merged doctests feature generates a big file containing all generated doctests making use of libtest. However, libtest is internal to rustc, so if you try to write:

Runextern crate test;

fn main() {}

You will get:

error[E0658]: use of unstable library feature `test`
 --> src/main.rs:1:1
  |
1 | extern crate test;
  | ^^^^^^^^^^^^^^^^^^

To be able to use it, we need to add #![feature(test)]... which requires nightly. So how come such an obvious issue got added, reviewed, tested and merged with no one noticing? Well, for multiple reasons:

1. Because rustdoc testsuite is using nightly.
2. Because it was a monstruous pull request, making it harder to spot such things when so many things were debated about.
3. Because we used crater which didn't return any big issue (because it runs in... nightly!).

So in short: it was one technical detail among many others. Sadly this one passed through.

Handling nightly on stable

So now, let's see how it was fixed. There are two requirements:

1. It needs to work on stable.
2. It should not enable users to use nightly features in doctests if they're not on nightly.

And the solution to make this possible is to put the nightly code in one crate, and the user's code in another. Let's use this code as example:

Run/// ```
/// let y = 'a';
/// ```
pub fn foo() {}

Now it'll generate in the "user code crate":

Run#![allow(unused)]
pub mod _doctest_0 {
    fn main()  {
        let y = 'a';
    }
    pub fn _main_fn() -> impl std::process::Termination { main() } 
}

And in the "nightly code crate":

Run#![allow(unused)]
#![allow(unused_extern_crates)]
#![allow(internal_features)]
#![feature(test)]
#![feature(rustc_attrs)]
extern crate test;
extern crate doctest_bundle_2024 as doctest_bundle;

mod _doctest_0 {
    pub const TEST: test::TestDescAndFn = test::TestDescAndFn::new_doctest(
        "foo.rs - foo (line 1)", false, "foo.rs", 1, false, false,
        test::StaticTestFn(
            || {
                if let Some(bin_path) = crate::_doctest_mod::doctest_path() {
                    test::assert_test_result(
                        crate::_doctest_mod::doctest_runner(bin_path, 0),
                    )
                } else {
                    test::assert_test_result(
                        doctest_bundle::_doctest_0::_main_fn(),
                    )
                }
            },
        ),
    );
}

The rest of the code is exactly the same as the one in the previous blog post.

So again, you might have noticed that we still have a "nightly code crate", meaning that the original problem is still there: we still need a nightly rustc to compile it. Well, here comes the dark trick!

⚠️ This is a trick performed by highly trained professional, please DO NOT try to reproduce at home! ⚠️

Joke aside: please never use it.

The "nightly code crate" is compiled with the RUSTC_BOOTSTRAP=1 environment variable, allowing the nightly checks to succeed and for us to successfully use any nightly code we need, making it possible for merged doctests to work with a stable rustc.

That's it for the explanations. Thanks a lot to @notriddle and the rustdoc team for the quick fix and their reviews.

You can see the fix in #137899.

Words of the end

Dark magic was carried out under the (careful?) attention of the cat: