Virtual Instruments/Softsynth support

[toyoshim]

For long term goal, it would be important to find a way to give a name in a standardized way against each device on all operating systems.

E.g., once I create a song with an online DAW running on Chrome/Windows that is connected with a motif rack, I hope I can open the song correctly with the same online DAW running on Chrome/Mac that is connected with the same or another motif rack. But for now, it will be difficult because manufacturer, name, and so on depends on operating systems and browser implementation. So, I may need reconfiguration on device assignment against the song project on the DAW.

MIDI has several physical layer specs, e.g., the original DIN, USB, Bluetooth, WiFi, and so on. Some of them provides product specific unique ID. For example, On USB devices, VID/PID pairs can be a cross-platform persistent identifier. But it isn't trivial when we consider all kinds of physical layers.

[cwilso]

This sounds more like a standardization need for a VSTi-like spec that wraps WebMIDI and some kind of Web Audio streams together, than a specific ask for Web MIDI.

[joeberkovitz]

Agree this need not hold up WebMIDI for an initial rollout.

However I don't see that this issue is necessarily about a bigger MIDI/Audio spec. I think the fundamental ask here is that devices surfaced in WebMIDI expose some reasonable semantic identifiers. A cross-platform "durable discovery" mechanism can then be built on top of that.

[MidiHax]

Also agree with @joeberkovitz - let's not hold up for this.

I also agree with Joe's assessment of the ask. As the OP alluded to, there are many challenges. Particularly with DIN interfaces to external equipment where the only meaningful identifier might be a port number and there could be multiple devices attached to that port. Device assignment/management has always been an issue with existing shrinkwrap software, largely due to the shortcomings (device discovery) in the MIDI spec in my opinion, and it seems unlikely we can solve this problem within the context of WebMIDI.

[bome]

For me, this is a user interface issue. The WebMIDI app can store the name+manufacturer of the MIDI port(s) used. If on, restart, a port cannot be found, it should ask the user to identify the port to use instead of the stored name+manufacturer.

On a side note, the still pending HD Protocol will solve this by letting you query the device for its name, manufacturer and unique ID.

[cwilso]

A "cross-platform 'durable discovery' mechanism" sounds like exactly that I was saying - a way to refer to, load and utilize "instruments" across systems. That's great. It's kinda like VSTi, or more appropriately, Propellerheads' Rack Extensions.

[notator]

I think an API needs to be defined, that would be implemented by soft synth authors, and used by web app authors [1].
The API would contain things like the synth's metadata, whether or not it is a GM synth [2], how many sounds it supplies, whether or not it uses/can load VSTis, SoundFonts, etc.
I also think that compliant synths should be available from workers [3], and that we need to agree on the name(s) of the event(s) with which the threads communicate.

Here's a simple use case:
A surfer visits a web site. Presses a button, hears sound(s) or a piece of music.

The surfer should not have to know anything about the format of the sound(s) being played, and should not have to make any special settings on his/her computer. The web app author decides which sound(s) get played. They are not restricted to the sounds supplied by an OS-supplied GM synth.

Currently, audio files are handled by the <audio> tag. MIDI files would be handled by a soft-synth being accessed by the web app author.

The web app author needs to do things like:

• open() the synth -- tell it to allocate any resources it needs
• close() the synth -- tell it to release any resources it has allocated
• show the soft synth's metadata to the surfer.
• load a device selector with the soft synth's name (supplied by the synth)
• discover the soft synth's type: Does it simply provide a few special sounds of its own? Can it load sounds that the web app author provides? In which formats?
• send MIDI messages to the soft synth if it is in another thread
• easily replace the soft synth by a different one.
• use more than one soft synth
• etc?

[1] this is more or less what toyoshim proposed in #45
[2] see #110
[3] see #99

[notator]

This issue is marked "needs WG review". Can you say when that review will be done?
I'd like to move this forward, but don't want to tread on anyone's toes...

I'm getting a bit far ahead of the discussion here, but the WG might also like to consider the following:

1. I'd like to propose that the Web MIDI API WG set up a special GitHub project to develop an interface for Web MIDI Synths (see above and Web MIDI Synths #154). The intention should be to develop an interface that would eventually become part of the Web MIDI API. This sub-project would not need to concern all the participants in the main Web MIDI API project. (If, in Modify MIDIOptions to be able to specify software synths to be included #150, the browser manufacturers agree that they are only going to deal with hardware output devices, then they can ignore the sub-project altogether.)
2. IMO the sub-project should welcome the participation of both implementers and users of such an interface. Until now, users have been rather side-lined, as the Web MIDI API has been concentrating on the interface between browsers and hardware devices.
3. IMO the best way to develop the interface would be to create a working prototype Web MIDI Synth. That would make it much easier to find and resolve any issues. When the group is satisfied with its result, it can go back to the parent project to get approval there.

Hope that helps.
James

[joeberkovitz]

This issue is currently marked as V2, so although it needs review, it's placed in back of all the V1 issues that the WG deemed more urgent. I'll let others chime in but we did discuss the priority of this issue and it was clearly thought to be better left for later at the time (see the history of this issue).

[notator]

Well it took a bit of time, but I've now published a new repository on GitHub doing more or less what I suggested above:
https://github.com/notator/WebMIDISynthHost
The app itself can be tried out at
http://james-ingram-act-two.de/open-source/WebMIDISynthHost/host.html
Have fun! :-)