Audio API

[martinfouilleul]

Here's a tentative API for enumerating audio adapters and creating an audio device with an attached callback. This covers the happy path of a basic capture/playback scenario.

//------------------------------------------------------------
// Listing audio adapters
//------------------------------------------------------------

typedef struct oc_audio_adapter { u64 h; } oc_audio_adapter;

typedef enum oc_audio_adapter_kind
{
	OC_AUDIO_ADAPTER_INPUT = 0x01,
	OC_AUDIO_ADAPTER_OUTPUT = 0x02
} oc_audio_adapter_kind;

typedef struct oc_audio_adapter_info
{
	oc_audio_adapter adapter;
	oc_audio_adapter_kind kind;
	oc_str8 name;
	//...
} oc_audio_adapter_info;

typedef struct oc_audio_adapter_list_result
{
	u32 count;
	oc_audio_adapter_info* adapters;
} oc_audio_adapter_list_result;

oc_audio_adapter_list_result oc_audio_adapter_list(oc_arena* arena);
oc_audio_adapter_info oc_audio_adapter_default_input();
oc_audio_adapter_info oc_audio_adapter_default_output();

//------------------------------------------------------------
// Creating / Using audio devices
//------------------------------------------------------------
typedef void(oc_audio_device_callback*)(oc_audio_device device, 
                                                                        u32 frameCount,
                                                                        void* inputs,
                                                                        void* outputs);   

typedef struct oc_audio_endpoint_config
{
	oc_audio_adapter adapter;
	u32 channelCount;
	// format, etc... for now assume non interleaved linear f32
} oc_audio_endpoint_config;

typedef struct oc_audio_device_desc
{
	u32 sampleRate;
	oc_audio_endpoint_config input;
	oc_audio_endpoint_config output;
	
	oc_audio_device_callback callback;
	void* userData;
} oc_audio_device_desc;

typedef struct oc_audio_device { u64 h; } oc_audio_device;

oc_audio_device oc_audio_device_create(oc_audio_device_desc* desc);
void oc_audio_device_destroy(oc_audio_device device);
void oc_audio_device_start(oc_audio_device device);
void oc_audio_device_stop(oc_audio_device device);

oc_audio_status oc_audio_device_last_error(oc_audio_device device);

It should be a relatively thin abstraction over both CoreAudio and WASAPI (although it will require some buffering to support both input and output in the same callback). I can probably adapt some old CoreAudio code I wrote for an old project to draft an implementation in the platform layer, and we can work together with Skytrias on a WASAPI backend. We can then figure out more advanced features like device change notifications etc.

Since the API is designed around opaque handles it is straightforward to expose the API to the wasm guest as imported functions, but there are some difficulties to consider:

• On native the callback will be called from an audio thread. Currently the wasm interpreter is not multithreaded (and the wasm threading story is not very clear yet).

  • We could maybe create a second instance of the interpreter working on the same memory, although that's probably messed up if the module needs to touch globals/tables etc.
  • We could buffer audio an call the wasm callback on the main thread, but that will increase latency and we'll have to figure out how to interleave the timings for audio and graphics.
  • Ideally, we support calling wasm functions from multiple threads and don't have anything special to do.

• We should also avoid calling into the host from inside the callback, so the signature may need to be adjusted to include informations we would otherwise query from the device (e.g. the IO configuration and the user pointer).

• The callback will require additional buffer copies.