Importing cube maps is hard

[janhohenheim]

How can Bevy's documentation be improved?

HDRI: first attempt

Imagine I'm an artist and want to use a skybox. I go to Polyhaven et al., download my HDRI:

I plug it into Bevy and...
Oh no! there's a weird runtime error about a thing called wgpu and texture bindings and cubemaps. Okay, I search the discord and I get linked to https://github.com/pcwalton/gltf-ibl-sampler-egui. Cool, let's download that. Oh no, there is no release. Help?

HDRI: second attempt

Fortunately, the artist can go to a programmer friend. Let's imagine I'm that programmer now. I clone the repo, run it and... it does not compile. Alright, all good, I can fix this. I patch it up, and have it running!

Let's convert our HDRI and look at the preview...

Eek! What happened there?! Maybe it looks better in Bevy...

Okay, it's not super dark, but it's weirdly super saturated!

I'm alright with using CLI tools, so I look at glTF IBL Sampler directly. I bet they have some more options to deal with this...

...looks like they don't? The documentation is fairly sparse, and playing around with the parameters that are documented doesn't give me anything useful either.

Okay, I have to give up on this HDRI business. Surely there is another way.

DIY cubemaps from PNGs

The Bevy example mentions PNGs. From some sleuthing on Discord, I can gather that I can stitch some PNGs together to a cubemap. Let's try that! First, I need some Bevy code to read the PNGs. Turns out we need some hacks. The example is nearly usable as-is, just some tweaking:

#[derive(Component)]
struct PlzConvertMe(Option<Skybox>);

fn load_skybox(
    mut asset_event: EventReader<AssetEvent<Image>>,
    asset_server: Res<AssetServer>,
    mut skybox: Query<(Entity, &mut PlzConvertMe)>,
    mut images: ResMut<Assets<Image>>,
    mut commands: Commands,
) {
    for event in asset_event.read() {
        let AssetEvent::LoadedWithDependencies { id } = event else {
            continue;
        };

        let Some(handle) = asset_server.get_id_handle(*id) else {
            return;
        };
        for (entity, mut skybox) in &mut skybox {
            let Some(skybox) = skybox.0.take() else {
                continue;
            };
            if skybox.image != handle {
                // Minor bug right here, the skybox is `None` now, but I'm too lazy to fix this 
                continue;
            }
            let image = images.get_mut(*id).unwrap();
            if image.texture_descriptor.array_layer_count() == 1 {
                image.reinterpret_stacked_2d_as_array(image.height() / image.width());
                image.texture_view_descriptor = Some(TextureViewDescriptor {
                    dimension: Some(TextureViewDimension::Cube),
                    ..default()
                });
            }
            commands
                .entity(entity)
                .insert(skybox)
                .remove::<PlzConvertMe>();
        }
    }
}

I don't think a Bevy beginner would be able to write this, but okay, I can deal with it. Now let's get our PNGs.
I found exrenvmap, so let's use that to convert from an EXR to PNGs:

exrenvmap my_cool_file.exr my_cool_file.png

There's a bunch of options for this CLI, but I don't really understand any of them.

Let's try it out in Bevy:

Failed to load asset 'golden_gate_hills_4k.png' with asset loader 'bevy_image::image_loader::ImageLoader': Could not load texture file: Error reading image file golden_gate_hills_4k.png: failed to load an image: Format error decoding Png: Invalid PNG signature., this is an error in `bevy_render`.

Okay, I guess not.
Let's google "Convert HDRI to cubemap"
I find this: https://hdri-to-cubemap-converter.vercel.app/
Uploading my file, and...

whew, no clue. Looking at the Bevy example, it looks like maybe option 2 is the right one?
Let's plug it into Bevy and...

thread 'Compute Task Pool (22)' panicked at /home/hhh/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/bevy_image-0.16.0/src/image.rs:894:20:
attempt to calculate the remainder with a divisor of zero
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

okay, maybe select another option when exporting? Let's see what "separate" does...

Okay, maybe I can convert these back into a KTX2 so I can remove my ugly system again? After some more Discord digging, I find a magic incantation:

ktx create --cubemap --format R8G8B8A8_SRGB --zstd 18 --assign-oetf srgb --assign-primaries bt709 --generate-mipmap px.png nx.png py.png ny.png pz.png nz.png k
tx2/cubemap_rgba8.ktx2

Apparently "--assign-oetf is deprecated and will be removed in the next release.". Good that it works for now.
Let's load this in Bevy again:

Okay, colors match, but now everything has a horrible resolution! I guess that website created low quality PNGs?

Let's google for alternatives.
I see https://matheowis.github.io/HDRI-to-CubeMap/, but that one generates a cross:

I don't really want to go into GIMP and manually create each subfile, especially since I don't really know which one is supposed to be px, nz, etc.
Next: https://github.com/ivarout/HdriToCubemap
Cross again.
Let's search Discord again... some more weird CLIs... people stitching PNGs together... error messages........

Okay, I give up, let's just use a clear color instead. Or the Ryfjallet cubemap of the Bevy examples, that one works, I guess.

[janhohenheim]

Related Bevy CLI feature request: TheBevyFlock/bevy_cli#6

[mate-h]

+1 on this issue, thanks for adding a detailed description of your though process and trial and error.
I went though this process of importing cubemaps from Polyhaven recently and I landed on using 4 different command line tools to end up with a 16bit precision floating point cubemap texture. I've attached the bash script I used in this PR:
#19076