Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ESP32-S3 - Arduino v3.1 - USBCDC Hangs and misses chunks of data for for large serial transmits #10836

Open
1 task done
TheCrypt0 opened this issue Jan 9, 2025 · 8 comments
Assignees
Labels
Status: Needs investigation We need to do some research before taking next steps on this issue

Comments

@TheCrypt0
Copy link

Board

ESP32-S3

Device Description

Custom ESP32-S3 keyboard but it should work on any ESP32-S3 devkit with USB connection.

Hardware Configuration

N.A.

Version

v3.1.0

IDE Name

Arduino IDE

Operating System

macOS

Flash frequency

80MHz

PSRAM enabled

no

Upload speed

921600

Description

I’m currently porting my project to the latest version of the Arduino Core but I'm experiencing data loss when using the USBCDC implementation with large chunks of data. It seems that the data is not reliably received and a buffer overflow occurs.

This seems to be related and exactly what I am experiencing: https://www.esp32.com/viewtopic.php?t=38858

Downgrading to 2.0.17 makes the code work again.

I suspect the issue might be related to how data is queued or buffered internally within the USBCDC implementation. If there are known limitations or configurations that could mitigate this issue, I’d really appreciate any guidance or suggestion.

I've provided an example that computes the SHA1 of the sent data to confirm integrity.

Sketch

#include <Arduino.h>
#include <ArduinoBearSSL.h>

#include <USB.h>
#include <USBCDC.h>

USBCDC         m_serial;
ESPUSB*        m_usb;

String m_buffer;

int m_last_message_ms;

void ingest_cdc_data(void)
{
  m_serial.println("GOT DATA");

  while (m_serial.available()) {
    m_buffer += m_serial.readString();
  }

  if (m_buffer.length() == 0) {
    return;
  }

  m_last_message_ms = millis();
}

void calculate_buffer_sha1()
{
  if(m_buffer == "")
    return;

  // calculate the sha1 of the buffer if it's been more than 1 second since the last message
  if (millis() - m_last_message_ms > 1000) 
  {
    SHA1.beginHash();
    SHA1.print(m_buffer);
    SHA1.endHash();

    print_hash();

    m_buffer = "";
  }
}

void print_hash()
{
  while (SHA1.available()) {
    byte b = SHA1.read();

    if (b < 16) {
      m_serial.print("0");
    }

    m_serial.print(b, HEX);
  }
  m_serial.println();
}

void
usb_event_callback(void*            arg,
                 esp_event_base_t event_base,
                 int32_t          event_id,
                 void*            event_data)
{
  if (event_base == ARDUINO_USB_CDC_EVENTS) {
    switch (event_id) {
      case ARDUINO_USB_CDC_RX_EVENT:
        ingest_cdc_data();
        break;
      case ARDUINO_USB_CDC_RX_OVERFLOW_EVENT:
        m_serial.printf("USB cdc overflow\n");
        break;

      default:
        break;
    }
  }
}

void
setup()
{
  // put your setup code here, to run once:
  m_buffer = "";
  m_last_message_ms = 0;

  USB.onEvent(usb_event_callback);
  m_serial.onEvent(usb_event_callback);

  m_serial.begin(115200);

  m_usb = &USB; // get the USB object

  m_usb->begin();
}

void
loop()
{
  // put your main code here, to run repeatedly:
  calculate_buffer_sha1();

  delay(1);
}

Debug Message

--- CORRECT OUTPUT WITH ARDUINO ESP32 VER 2.0.17 ---
GOT DATA
GOT DATA
GOT DATA
GOT DATA
GOT DATA
GOT DATA
GOT DATA
GOT DATA
GOT DATA
GOT DATA
GOT DATA
GOT DATA
GOT DATA
GOT DATA
128DA61AA85A72155E5229129D83E6044181FB0A

--- OUTPUT WITH ARDUINO ESP32 VER 3.1.0 ---
GOT DATA
GOT DATA
GOT DATA
(then it stops and doesn't accept any more data)

Other Steps to Reproduce

Open an Arduino Serial monitor and paste the following text into the Message box (No Line Ending, 115200 baud).

111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111

Expect the SHA1 being: 128DA61AA85A72155E5229129D83E6044181FB0A

I have checked existing issues, online documentation and the Troubleshooting Guide

  • I confirm I have checked existing issues, online documentation and Troubleshooting guide.
@TheCrypt0 TheCrypt0 added the Status: Awaiting triage Issue is waiting for triage label Jan 9, 2025
@TheCrypt0 TheCrypt0 changed the title ESP32-S3 -Arduino v3.1 - USBCDC Hangs and misses chunks of data for for large serial transmits ESP32-S3 - Arduino v3.1 - USBCDC Hangs and misses chunks of data for for large serial transmits Jan 9, 2025
@me-no-dev
Copy link
Member

@SuGlider can look into this, but could you try with the HWCDC instead? If Serial is all you need, HWCDC might do better

@TheCrypt0
Copy link
Author

@me-no-dev thanks for the reply!

Is there a way to use USBHIDKeyboard and the other HID peripherals along with HWCDC?

@SuGlider
Copy link
Collaborator

@me-no-dev thanks for the reply!

Is there a way to use USBHIDKeyboard and the other HID peripherals along with HWCDC?

Nop. HWCDC is exclusivelly a CDC/JTAG implementation in silicon. It can't work in any other way.

ESP32-S3 has USB OTG mode for a full USB Class mode software implementation.
Arduino uses TinyUSB implementation within its USB Class.

@TheCrypt0
Copy link
Author

Nop. HWCDC is exclusivelly a CDC/JTAG implementation in silicon. It can't work in any other way.

ESP32-S3 has USB OTG mode for a full USB Class mode software implementation. Arduino uses TinyUSB implementation within its USB Class.

@SuGlider

Yeah, that's what I was worried about, I have to use the USB OTG mode. At least we have full control via software on what happens.

Have you managed to replicate the issue with the instructions provided or is there something I can do meanwhile to help you guys with the process? I'll be happy to provide support if able.

@SuGlider
Copy link
Collaborator

@TheCrypt0 - Thanks for the support. I'll set up an environment for replicate and debug the issue.

@SuGlider SuGlider self-assigned this Jan 10, 2025
@SuGlider SuGlider added Status: Needs investigation We need to do some research before taking next steps on this issue and removed Status: Awaiting triage Issue is waiting for triage labels Jan 13, 2025
@EarlVadim
Copy link

In my project I use HWCDC, and sometimes it also misses packets. (((
There is some problem there.

@TheCrypt0
Copy link
Author

Hey @SuGlider apologies for bothering you again, have you been able to replicate the issue? It seems like it's affecting others as well.

Thanks!

@SuGlider
Copy link
Collaborator

Hey @SuGlider apologies for bothering you again, have you been able to replicate the issue? It seems like it's affecting others as well.

Thanks!

Hi, I'm on vacation right now and I shall return to work in the beginning of February. This issue is in top place for investigation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Needs investigation We need to do some research before taking next steps on this issue
Projects
None yet
Development

No branches or pull requests

4 participants