Alternative API of eframe / epi

[coderedart]

Current API

epi has an App trait. this abstracts over both the windowing backend and rendering backend. eframe doesn't expose either of them directly, but rendering is now partially exposed via the PaintCallBacks now.

Limitations

1. egui gets a lot of issues asking it to expose things from winit. the NativeOptions struct grows by every version.
2. swapping backends is very hard at the moment. they will need to rewrite a lot of their code as there's no "uniformity" across winit or sdl implementations. most of the backends really don't use epi::App trait and when they use it, they need to limit themselves to a specific window + gfx api combination.

Alternative API

The primary goal is to separate the rendering from windowing functionalities completely.
Have a WindowBackend and GfxBackend traits like you an see at https://github.com/coderedart/etk/blob/master/crates/egui_backend/src/lib.rs .

1. we need to tell window backend in advance the configuration with which to create the window (opengl or vulkan and any additional settings)
2. we need some info from window for gfx like raw_window_handle (surface creation)
3. for opengl we need functions like swap_buffers or get_proc_address etc..

GfxApiConfig

this is given to window backend to set the right options before window creation

/// opengl or vulkan or directx etc.. gfx settings. 
pub enum GfxApiConfig {
    OpenGL { // opengl can have a bunch of settings that need to be specified at window creation. eg: srgb compatible framebuffer
        srgb: Option<bool>,
    },
    Vulkan {},// nothing to set, but backends like glfw or sdl need to know to NOT create a opengl context with window if we plan to use vulkan
}

WindowInfoForGfx

returned by Window backend after creation.

pub struct WindowInfoForGfx {
    pub gfx_api_config: GfxApiConfig, // settings used to create the window. 
    pub window_handle: RawWindowHandle, // window handle used by backend renderer. 
    pub opengl_context: Option<Box<dyn OpenGLWindowContext>>, // this is only available with opengl settings. see below trait
}

OpenGLWindowContext

some functions require by opengl renderers are usually provided by windowing backends and not available with opengl raw context itself. only for openGL usecase.

pub trait OpenGLWindowContext {
    fn swap_buffers(&mut self); // obvious
    fn make_context_current(&mut self); // to make gl context current. useful for multi threading maybe
    fn get_proc_address(&mut self, symbol: &str) -> *const core::ffi::c_void; // get opengl function pointers for glow or other GL renderers
}

with that, we have separated the data required by Windows and Renderers.

WindowBackend

pub trait WindowBackend: Sized {
    /// custom configuration type specific to each backend. could impl Default or just be unit type
    type Configuration;

    /// Create a new window backend. and return info needed for the GfxBackend creation and rendering
    fn new(config: Self::Configuration, gfx_api_config: GfxApiConfig) -> (Self, WindowInfoForGfx)
    where
        Self: Sized;
    /// This frame's events gather into rawinput and to be presented to egui's context
    fn take_raw_input(&mut self) -> RawInput;
    /// return Some(size) if there's been a framebuffer resize event recently. once we take it, it should
    /// return None until there's a fresh resize event.
    fn take_latest_size_update(&mut self) -> Option<[u32; 2]>;
    /// Run the event loop. different backends run it differently, so they all need to take care and
    /// call the Gfx or UserApp functions at the right time.
    fn run_event_loop<G: GfxBackend + 'static, U: UserApp<Self, G> + 'static>(
        self,
        gfx_backend: G,
        user_app: U,
    );
    fn get_live_physical_size_framebuffer(&mut self) -> [u32; 2];
}

GfxBackend

pub trait GfxBackend {
    /// similar to WindowBackendSettings. 
    type GfxBackendSettings;

    /// create a new GfxBackend using info from window backend and custom settings struct
    fn new(window_info_for_gfx: WindowInfoForGfx, settings: Self::GfxBackendSettings) -> Self;
    
    /// prepare (acquire) the surface / swapchain etc.. for rendering. 
    fn prepare_frame<W: WindowBackend>(
        &mut self,
        framebuffer_size_update: Option<[u32; 2]>, // if there's any resize event.
        window_backend: &mut W,
    );

// upload data to buffers or prepare bindgroups etc..
    fn prepare_render(&mut self, egui_gfx_output: EguiGfxOutput);// egui output is the meshes + texture delta + info like screen width/height, scale etc.. for clip rectangles and shader uniforms.
// do the actual rendering. called after `prepare_render`.
    fn render(&mut self);

    /// This is called at the end of the frame. after everything is drawn, you can now present
    /// the frame (swap buffers).
    fn present(&mut self);
}

UserApp

Finally, we have the trait that serves as the replacement for the current epi::App

// the app will get the window backend and gfx backend along with egui context. 
pub trait UserApp<W: WindowBackend, G: GfxBackend>: Sized {
    fn run(&mut self, egui_context: &egui::Context, window_backend: &mut W, gfx_backend: &mut G);
}

In the end, it all comes down to

type Gfx = egui_render_wgpu::WgpuBackend; // choose your rendering backend
type WB = egui_window_winit::WinitBackend; // choose your windowing backend

struct App {
    frame_count: usize,
}
// use whatever info from gfx or wb to create your app. like creating any custom pipelines, buffers, textures for your own rendering needs.
impl App {
    pub fn new(gfx_backend: &WgpuBackend) -> Self {
        Self { frame_count: 0 }
    }
}
// impl a single trait
impl UserApp<WB, WgpuBackend> for App {
    fn run(&mut self, egui_context: &egui::Context, _window_backend: &mut WB, _gfx_backend: &mut WgpuBackend) {
		// you can use wb or gfx to do whatever you want like custom rendering or moving window etc..
        Window::new("egui user window").show(egui_context, |ui| {
            ui.label(format!("frame number: {}", self.frame_count));
            self.frame_count += 1;
        });
    }
}

fn main() {

    let (window_backend, window_info_for_gfx) = WB::new(Default::default, GfxApiConfig::Vulkan {});// create default window with vulkan
    let gfx_backend = WgpuBackend::new(window_info_for_gfx, Default::default()); // create wgpu with vulkan and window's surface
    let app = App::new(&gfx_backend); // create app
    window_backend.run_event_loop(gfx_backend, app); // enter event loop
}

the rough event loop workflow happens like this. literally copy-pasted from glfw

let egui_context = egui::Context::default();
        while !self.window.should_close() {
            // gather events
            self.tick();
            // take egui input
            let input = self.take_raw_input();
            // take any frambuffer resize events
            let fb_size_update = self.take_latest_size_update();
            // prepare surface for drawing
            gfx_backend.prepare_frame(fb_size_update, &mut self);
            // begin egui with input
            egui_context.begin_frame(input);
            // run userapp gui function. let user do anything he wants with window or gfx backends
            user_app.run(&egui_context, &mut self, &mut gfx_backend);
            // end frame
            let output = egui_context.end_frame();
            // prepare egui render data for gfx backend
            let egui_gfx_output = EguiGfxOutput {
                meshes: egui_context.tessellate(output.shapes),
                textures_delta: output.textures_delta,
                screen_size_logical: [
                    self.size_physical_pixels[0] as f32 / self.scale[0],
                    self.size_physical_pixels[1] as f32 / self.scale[0],
                ],
                framebuffer_size_physical: self.size_physical_pixels,
                scale: self.scale[0],
            };
            // prepare rendering
            gfx_backend.prepare_render(egui_gfx_output);
            // render
            gfx_backend.render();
            // present the frame and loop back
            gfx_backend.present();
        }

Advantages

1. implementing a custom backend is very straight forward now. just a single trait.
2. users can mix and match different windowing and rendering backends. for example, we could offer a three_d backend or rend3 backend too and it would be really easy for users to use them instead of aiming for custom implmentations.
3. eframe is not limited by winit anymore. there's a lot of issues open right now which are due to winit, but if the user doesn't care about web, we can provide glfw as an alternative backend (feature gated).

[coderedart]

This is not really important, but thought it is better to document this idea somewhere rather than forget it. AFAIK, this doesn't even have to be officially part of egui if it feels unnecessary, it could be a separate project / repo.

[coderedart]

so, i made a proof of concept .
I was able to implement glow + wgpu for rendering backends and sdl2 + glfw + winit for windowing backends.
you can look at the examples to see how little difference there is in using different windowing or rendering backends.

this could be improved even more if we ignore opengl and only concentrate on newer API like vk, dx, metal.

eframe can just use winit and glow as backends and it won't even be a breaking change AFAIK.

[emilk]

I don't have the mental bandwidth to think about this too hard right now, but here is a short summary of my thoughts:

It would be really nice to be able to freely combine different windowing and rendering engines. I have been itching to try out both SDL2 and tao, for instance.

I am worried that it will be very difficult thing to find the right abstraction layer without building another windowing abstraction layer on top of winit, sdl2, tao, etc.

I also worry that if eframe supports SDL2, then https://crates.io/crates/egui_sdl2_gl would need to be moved into this repository (or we would have a cyclic dependency), which would just increase the maintenance burden and scope of eframe.

However, I never wanted to build a windowing framework, I only have done so in order to easily build portable egui applications. If someone else writes a better framework than eframe, I would switch in a heartbeat!

[coderedart]

Understood. I will keep this as a separate project and try to polish it a little more.