BLE HID composite devices

[benjaminaigner]

Dear all,

I've added the BLE HID composite device, where you can use either Mouse, Keyboard, Joystick or any combination of these libraries to have a variable HID device.

An example with all 3 of them is BLECircleKeysJoystick in the MouseBLE library.

[earlephilhower]

@benjaminaigner can you check that you've git commit and git pushed all the appropriate files? There are build errors related to the btstack config.h file throughout with the current push...

[benjaminaigner]

THX for the hint, I somehow overlooked the CI notice.
I had to add some preprocessor #ifdefs to avoid using the BT files from sdkoverride when NOT using BLE.

[earlephilhower]

Sorry for the delay in review/merge. Been occupied elsewhere recently but jut made a note to make GH bug me until I actually do review and merge this. 😆

[benjaminaigner]

No worries, I know it can be very time consuming managing a project.
I still need to order the hardware where I need this one for, so also no hurry :-)

[earlephilhower]

I just pulled this patch and did some basic sanity testing, Unfortunately, even the non-changed BLE teste (i.e. KeyboardBLE/Password) do not function and seem to hang when any HID communication happens (i.e. on BOOTSEL press). For sanity, I just reset to the git head and rebuilt and ran the BLE samples w/o any issue.

So, it looks like there's something on your system but not in this PR. Could you double check that nothing else is different? (you could simply pull a complete new copy of this repo, wget https://github.com/earlephilhower/arduino-pico/pull/1587.patch, patch -p1 < 1587.patch and do a recursive diff and/or build the example using that tree instead of your existing one).

[benjaminaigner]

Dear @earlephilhower

Interesting, I've tried it right now on a Notebook without any prior RP2040 core, using following steps:

mkdir -p ~/Arduino/hardware/pico
cd ~/Arduino/hardware/pico/
git clone https://github.com/benjaminaigner/arduino-pico rp2040
cd rp2040
git checkout BLEHID_Composite
git submodule update --init
cd pico-sdk
git submodule update --init
cd ../tools
python3 ./get.py

Opening Arduino, selected "RPi Pico W", changed to "IPv4 + Bluetooth" and uploaded
the MouseBLE/BLECircleKeysJoystick example.

Worked on:

• Android 12 (LineageOS 19)
• Debian Bookworm

What device do you use for testing?

[benjaminaigner]

Update: I've tried
MouseBLE/BLECircle.ino
and
KeyboardBLE/BLEKeyboardPassword.ino
examples as well, both of them work as expected.

One hint: you definitely need to re-pair your PicoW, otherwise the HID report map is cached and not updated.
In addition, the pairing keys get overwritten on uploading a new sketch.

[earlephilhower]

git clone https://github.com/benjaminaigner/arduino-pico rp2040

Therein lies the rub, maybe. Are you using any different submodules (different pico-sdk, different btstack)? That's your repo and not what will happen to this repo should it merge. I just did the following in a clean directory, moving my development git clone to a safe spot before beginning:

  542  git clone https://github.com/earlephilhower/arduino-pico.git ~/Arduino/hardware/pico/rp2040
  543  cd ~/Arduino/hardware/pico/rp2040
  544  git submodule update --init
  545  cd pico-sdk
  546  git submodule update --init
  547  cd ../tools
  548  python3 ./get.py
  549  git fetch origin pull/1587/head:pr1587
  550  git checkout  pr1587 
  551  git submodule update

This takes the PR here, as-is, and combines with the git master here. There were no submodules updated by that last git submodule update, FWIW, but I wanted to be 100% sure I had whatever was indicated in the PR.

I'm using Ubuntu 20.04 and a TP-Link USB BT adapter that runs fine w/master, and I did remove any pairing before reconnecting.

The behaviour I'm seeing under the Mouse/Circle.ino example is:

• I can pair properly in the Ubuntu control panel
• I see the "Press BOOTSEL to move the mouse in a circle" in the serial monitor.
• Pressing BOOTSEL prints "BARREL ROLL!!!" to the monitor, however no mouse movement happens.
• The Pico seems stuck at this point because additional presses of BOOTSEL have no effect (no printout, no motion).
• The Pico isn't hung, because it does respond to USB and I can reset and upload more code, but the same behavior is seen.

This hang happens even when there is no pairing present. With master it works properly when paired as many times as you want to hit BOOTSEL, and with no pairing it will just send the BLE messages into the ether and run fine, too.

Other examples give the same kind of behavior. No BLE HID messages received and no response to subsequent BOOTSEL presses.

[benjaminaigner]

Dear @earlephilhower

sry for the delay, other important stuff to do.

Unfortunately, I cannot reproduce this issue.
I've checked both repos (my fork & your master); there are no differences.

I'll need to check the problem with a Ubuntu machine, with Debian & Android it works out well.

[earlephilhower]

I don't believe this is related to the host OS at all. When I run this PR against master I get the hang in the Pico described, even when there is no BT pairing. In master today if there is no pairing the BT stack just silently drops things on the floor. But the main code continues running so I could press BOOTSEL and get multiple "Barrel Rolls!" or whatever printouts.

[earlephilhower]

I've just pulled and built this PR again, and then run against a Windows machine instead of my normal Ubuntu server. While Windows does show it as a BLE composite device (like Ubuntu), the new combined example doesn't actually seem to work or send anything at all when BOOTSEL is pressed.

The same failure is seen on the simple KeyboardBLE example. Pairs, but nothing at all is received by the Windows PC and the BT stack seems to lock up the loop, causing add'l BOOTSEL pressed to be ignored. This is the same as Ubuntu for me, unfortunately.

I think maybe you have a new library or missing override file that's not been checked in to the GH branch.

[benjaminaigner]

OK, I found 2 bugs, which might caused this error:

• The length of the sent HID report was too long, because in the callback, the report ID must be extracted and used to send the HID report to the BTStack, but without the ID the HID report is 1 Byte shorter.
• There was a wrong characteristics handle number in the Feature report, which was pointing to the joystick handle (0x2B) instead of the feature handle (0x2F)

I've tested it on my Windows machine and it works!

Please unpair the device, otherwise the old ATT db is still cached by Windows.

Nevertheless, interesting that it works on a Debian machine, but not on an Ubuntu.

[earlephilhower]

Thanks. I'm traveling this week, but will give the latest update a try when I get home this weekend.

[earlephilhower]

Success! This latest push does connect and run properly as far as I can see. The single-device and multi-device examples I've tried over BLE all seem to work fine.

Thx for digging into this!

[benjaminaigner]

Wonderful, I'm glad I found the issue.
THX for merging, see you next time (issue or much better a new PR :-))

[srrsparky]

I'm not sure if this is the place to say it, but...

MASSIVE THANK YOU to you both for this.

I've developed a bluetooth NFC reader and I have been struggling for ages with iPads not showing the onscreen keyboard when connected to a bluetooth keyboard. However, with this, I was able to present a "mouse" and a keyboard to the iPad.

Now that the iPad saw a mouse, I was able to turn on "Assistive Touch" on the ipad, and then turn on the "On Screen Keyboard".

This works like a charm with the NFC data being sent via the keyboard, but the user is still able to enter text with the onscreen keyboard.