Virtual Instruments/Softsynth support

[toyoshim]

For long term goal, it would be important to find a way to give a name in a standardized way against each device on all operating systems.

E.g., once I create a song with an online DAW running on Chrome/Windows that is connected with a motif rack, I hope I can open the song correctly with the same online DAW running on Chrome/Mac that is connected with the same or another motif rack. But for now, it will be difficult because manufacturer, name, and so on depends on operating systems and browser implementation. So, I may need reconfiguration on device assignment against the song project on the DAW.

MIDI has several physical layer specs, e.g., the original DIN, USB, Bluetooth, WiFi, and so on. Some of them provides product specific unique ID. For example, On USB devices, VID/PID pairs can be a cross-platform persistent identifier. But it isn't trivial when we consider all kinds of physical layers.

[cwilso]

This sounds more like a standardization need for a VSTi-like spec that wraps WebMIDI and some kind of Web Audio streams together, than a specific ask for Web MIDI.

[joeberkovitz]

Agree this need not hold up WebMIDI for an initial rollout.

However I don't see that this issue is necessarily about a bigger MIDI/Audio spec. I think the fundamental ask here is that devices surfaced in WebMIDI expose some reasonable semantic identifiers. A cross-platform "durable discovery" mechanism can then be built on top of that.

[MidiHax]

Also agree with @joeberkovitz - let's not hold up for this.

I also agree with Joe's assessment of the ask. As the OP alluded to, there are many challenges. Particularly with DIN interfaces to external equipment where the only meaningful identifier might be a port number and there could be multiple devices attached to that port. Device assignment/management has always been an issue with existing shrinkwrap software, largely due to the shortcomings (device discovery) in the MIDI spec in my opinion, and it seems unlikely we can solve this problem within the context of WebMIDI.

[bome]

For me, this is a user interface issue. The WebMIDI app can store the name+manufacturer of the MIDI port(s) used. If on, restart, a port cannot be found, it should ask the user to identify the port to use instead of the stored name+manufacturer.

On a side note, the still pending HD Protocol will solve this by letting you query the device for its name, manufacturer and unique ID.

[cwilso]

A "cross-platform 'durable discovery' mechanism" sounds like exactly that I was saying - a way to refer to, load and utilize "instruments" across systems. That's great. It's kinda like VSTi, or more appropriately, Propellerheads' Rack Extensions.

[notator]

I think an API needs to be defined, that would be implemented by soft synth authors, and used by web app authors [1].
The API would contain things like the synth's metadata, whether or not it is a GM synth [2], how many sounds it supplies, whether or not it uses/can load VSTis, SoundFonts, etc.
I also think that compliant synths should be available from workers [3], and that we need to agree on the name(s) of the event(s) with which the threads communicate.

Here's a simple use case:
A surfer visits a web site. Presses a button, hears sound(s) or a piece of music.

The surfer should not have to know anything about the format of the sound(s) being played, and should not have to make any special settings on his/her computer. The web app author decides which sound(s) get played. They are not restricted to the sounds supplied by an OS-supplied GM synth.

Currently, audio files are handled by the <audio> tag. MIDI files would be handled by a soft-synth being accessed by the web app author.

The web app author needs to do things like:

• open() the synth -- tell it to allocate any resources it needs
• close() the synth -- tell it to release any resources it has allocated
• show the soft synth's metadata to the surfer.
• load a device selector with the soft synth's name (supplied by the synth)
• discover the soft synth's type: Does it simply provide a few special sounds of its own? Can it load sounds that the web app author provides? In which formats?
• send MIDI messages to the soft synth if it is in another thread
• easily replace the soft synth by a different one.
• use more than one soft synth
• etc?

[1] this is more or less what toyoshim proposed in #45
[2] see #110
[3] see #99