Add a <MIDIKeyboard /> component

[stfufane]

Hey @nick-thompson !

We had a talk with @JoshMarler about the possibility of having a native component in the react-juce package and your opinion on how to tackle this would be nice.

The options are :

• Have a new React Component in the react-juce package with its set of props and a dedicated C++ class to handle the mouse events and the MIDI messages. The rendering would be managed by the user using View components the way he wants.
• Have an example app that does all that only using what's already there to showcase the possibilities.
• A mix of both :)

The question is mainly if we need the component (we agree with Josh that we do), how much should it do and how much would be the user's responsibility.

What do you think?

[JoshMarler]

In on the fence in regards to whether we should offer something like MIDIKeyboard as a core react-juce component.

I think an example app which creates such a component using react-juce's core view types sounds great.

I wonder if it might be difficult to cover everyone's use-cases with a MIDIKeyboard component?

Be interested to hear what you think to this @nick-thompson and your thoughts around similar such components with a higher level audio specific focus.

[nick-thompson]

I definitely agree that this would make a very cool example, and an example that I think a lot of people would look to for implementing this kind of behavior. But, I think we should be careful about what becomes part of react-juce and what doesn't.

In particular, react-juce is a view layer library, and I think we should be very careful to leave it at that. That is, react-juce doesn't care if you're writing a midi app, an OSC app, or even a completely non-audio online poker app. We want to provide the right view layer abstraction to cover all these cases without imposing details on the end user. So if we introduce a MIDIKeyboard native component, we're now supporting a view layer component that's coupled to the idea of MIDI underneath, which I think is probably outside the bounds of what the library itself should provide.

That said, I think we're all aware that this project aims at the audio niche intentionally :). That's why we have a Slider component in the core library. But the thing about the slider component is that we don't care how you use it or what you link it to underneath: it's purely a view layer abstraction. And if we're careful about what a MIDIKeyboard component should be, we can probably end up somewhere similar.

That is, what's the difference between a MIDIKeyboard and a plain old notion of a Keyboard component? You can compose one from the other. For example, a <Keyboard> component is basically just a set of <View> components with a particular layout and some alternating background colors. Do we need to get anymore complicated than that?

<Keyboard {...styles.keyboard} numKeys={88} startKey={14} depressedKeys={[60, 64, 68]} />

With a component like that, we can start to see that the idea of a MIDIKeyboard is really just some state management on top of the Keyboard, and we can also see that that leaves the idea of the Keyboard as purely a view layer abstraction.

Then there's the idea of how to implement the MIDI. For that, really, I would use ReactApplicationRoot::dispatchEvent and on the javascript side I would EventBridge.addListener('midi', myCallback);. Right there you could do all your state management and then re-render your .

So far, I think everything we're discussing here makes for a great example project idea, and I'd love to see that project come to life.

Then, there's a follow up question about whether or not this notion of a Keyboard component becomes a core library component, while, in my opinion, leaving all of the midi event handling outside of the core library.

What do y'all think?

[stfufane]

That could be a long discussion 😃

I totally get your point, and it's a complicated one to decide. For me the main question is to know how much effort we expect from the developer who will use it and implement the feature.

Let's imagine there are 2 people working on the plugin, one on the DSP/processing part and the other on the interface, how much is he expected to dive into the C++ code to have the interface working? In this example, he would need to implement the MIDI callbacks in C++ for his keyboard and maybe it would be out of his scope (I understand that the React part is easily doable without a ready-to-use native component).

It seems hard indeed to decide what should be included and what should not. And for me even more cause I just started to follow and I don't have the project history and all the background philosophy.

But then I think of how Juce works with the modules approach. I'm going out of bounds here but I wonder if the same thing could exist for react-juce, with npm packages for extra components and C++ modules that you can include or not for certain needs. You'd have the core module and then you'd be able to create specific components for this kind of cases. It would also open the way for user-made modules that can be released on npm, without altering the core of the library. In my head it sounds better than having to copy examples when you want to use some "almost generic" component that fits your needs. Again it's just a thought from my naive point of view and there's probably a lot to discuss :)

I'm still up to write an example that implements a MIDI keyboard and see how it goes anyway :)

[olidacombe]

I think what frontend devs would want is an interface like web midi, so they can use whatever abstractions in npm they're used to. The backend devs ideally would have little more to do than conditionally (un)patch midi devices to the underlying provider in c++ as they become (un)available. If react-juce could implement webmidi, maybe the default behaviour would be to pass all midi devices through and / or provide one virtual device owned by ReactApplicationRoot, exposing whatever api makes sense to send/receive note messages and the like to/from javascript. Easier said than done I guess.