Data misread from serial adapter (serialport only)

[skmagiik]

Hello! I have an interesting issue that I am not sure how to resolve or debug.

Hardware: Macbook Pro M3 MAX
Architecture: aarch64 (apple silicon)
OS: Mac OS 14.4.1 (23E224)

I have 2 serial adapters. When talking to the same device adapter A works (an FTDI adapter), but adapter B (Prolific Technology adapter) does not. Same baud rate settings etc in the rust program using this crate. Example code below
If using tio (a command line utility for serial data) adapter A and B both work as expected.

Here is an example:
The blue output is adapter A (working FTDI adapter). The green output is adapter B (Prolific Technology adapter).
The data they are receiving should be the same or very similar.

What route should I take for debugging this further?

Device setup:

    let mut primary_device = serialport::new(args.device_primary, args.baudrate)
        .timeout(Duration::from_millis(1000))
        .open()
        .expect("Failed to open primary port");

    let _ = primary_device.write_data_terminal_ready(true);
    let _ = primary_device.set_baud_rate(args.baudrate);
    let _ = primary_device.set_data_bits(serialport::DataBits::Eight);
    let _ = primary_device.set_parity(serialport::Parity::None);

Read function:

fn read_serial_data(device: &mut Box<dyn SerialPort>, secondary_device: &mut Box<dyn SerialPort>, output_buffer: &mut Vec<u8>) -> usize{
    let mut serial_buf: Vec<u8> = vec![0; 256];
    let bytes_read = device.read(serial_buf.as_mut_slice()).unwrap_or_default();
    if bytes_read > 0 {
        // println!("read {} bytes", bytes_read);
        serial_buf[0..bytes_read].into_iter().for_each( |x| {
            output_buffer.push(*x);
            // print!("{}", *x as char);
        });
        // println!("");
        secondary_device.write_all(&serial_buf[0..bytes_read]).expect("Failed to send to secondary device");
    }
    bytes_read
}

Here is also usbdiagnostic output for the 2 adapters:

Full Speed device @ 14 (0x03220000): .............................................   Composite device: "USB Serial Converter"
    Port Information:   0x2018
           Not Captive
           External Device
           Connected
           Enabled
           On Thunderbolt
    Number Of Endpoints (includes EP0):   
        Total Endpoints for Configuration 1 (current):   3
    Device Descriptor   
        Descriptor Version Number:   0x0200
        Device Class:   0   (Composite)
        Device Subclass:   0
        Device Protocol:   0
        Device MaxPacketSize:   8
        Device VendorID/ProductID:   0x0403/0x6001   (unknown vendor)
        Device Version Number:   0x0600
        Number of Configurations:   1
        Manufacturer String:   1 "FTDI"
        Product String:   2 "USB Serial Converter"
        Serial Number String:   3 "FTBHOLDE"
    Configuration Descriptor (current config)   
        Length (and contents):   32
            Raw Descriptor (hex)    0000: 09 02 20 00 01 01 00 A0  16 09 04 00 00 02 FF FF  
            Raw Descriptor (hex)    0010: FF 02 07 05 81 02 40 00  00 07 05 02 02 40 00 00  
            Unknown Descriptor   0020: 
        Number of Interfaces:   1
        Configuration Value:   1
        Attributes:   0xA0 (bus-powered, remote wakeup)
        MaxPower:   44 mA
        Interface #0 - Vendor-specific ..............................................   "USB Serial Converter"
            Alternate Setting   0
            Number of Endpoints   2
            Interface Class:   255   (Vendor-specific)
            Interface Subclass;   255   (Vendor-specific)
            Interface Protocol:   255
            Endpoint 0x81 - Bulk Input   
                Address:   0x81  (IN)
                Attributes:   0x02  (Bulk)
                Max Packet Size:   64
                Polling Interval:   0 ms
            Endpoint 0x02 - Bulk Output   
                Address:   0x02  (OUT)
                Attributes:   0x02  (Bulk)
                Max Packet Size:   64
                Polling Interval:   0 ms
Full Speed device @ 16 (0x03240000): .............................................   Composite device: "USB-Serial Controller"
    Port Information:   0x2018
           Not Captive
           External Device
           Connected
           Enabled
           On Thunderbolt
    Number Of Endpoints (includes EP0):   
        Total Endpoints for Configuration 1 (current):   4
    Device Descriptor   
        Descriptor Version Number:   0x0200
        Device Class:   0   (Composite)
        Device Subclass:   0
        Device Protocol:   0
        Device MaxPacketSize:   64
        Device VendorID/ProductID:   0x067B/0x2303   (unknown vendor)
        Device Version Number:   0x0300
        Number of Configurations:   1
        Manufacturer String:   1 "Prolific Technology Inc."
        Product String:   2 "USB-Serial Controller"
        Serial Number String:   0 (none)
    Configuration Descriptor (current config)   
        Length (and contents):   39
            Raw Descriptor (hex)    0000: 09 02 27 00 01 01 00 A0  32 09 04 00 00 03 FF 00  
            Raw Descriptor (hex)    0010: 00 00 07 05 81 03 0A 00  01 07 05 02 02 40 00 00  
            Raw Descriptor (hex)    0020: 07 05 83 02 40 00 00 
        Number of Interfaces:   1
        Configuration Value:   1
        Attributes:   0xA0 (bus-powered, remote wakeup)
        MaxPower:   100 mA
        Interface #0 - Vendor-specific   
            Alternate Setting   0
            Number of Endpoints   3
            Interface Class:   255   (Vendor-specific)
            Interface Subclass;   0   (Vendor-specific)
            Interface Protocol:   0
            Endpoint 0x81 - Interrupt Input   
                Address:   0x81  (IN)
                Attributes:   0x03  (Interrupt)
                Max Packet Size:   10
                Polling Interval:   1 ms
            Endpoint 0x02 - Bulk Output   
                Address:   0x02  (OUT)
                Attributes:   0x02  (Bulk)
                Max Packet Size:   64
                Polling Interval:   0 ms
            Endpoint 0x83 - Bulk Input   
                Address:   0x83  (IN)
                Attributes:   0x02  (Bulk)
                Max Packet Size:   64
                Polling Interval:   0 ms

[sirhcel]

What route should I take for debugging this further?

Hi @skmagiik, as data you're receiving with the Prolific device looks just garbled and the setup seems to work at least with the FTDI adapter: are you using the same cable for connecting to the device? Not that we are dealing with a connection issue here.

Another issue could be baud rate setup. At which baud rate are you communicating with the external device? There might be an issue with setting some/non-standard baud rates with the Prolific adapter.

Do both adapters work when you are communicating between them in a cross-over configuration with our example hardware_check? Like for example:

$ cargo run --release --example hardware_check -- --loopback-port /dev/tty.usbserial-X /dev/tty.usbserial-Y

What does hardware_check say?

In case you have an oscilloscope at hand, you might look at the actual data on the wire between your device and the adapter.

[skmagiik]

Hey @sirhcel I will run that hardware check. I didn't know that existed. For the connection / baud rate etc:
The target device is the same. If i open it with tio or minicom with the same baudrate parameters I don't have any issue.
I'll report back with hardware check results soon.

[skmagiik]

Here is the result each time I run it:

RUST_BACKTRACE=full cargo run --release --example hardware_check -- --loopback-port /dev/tty.usbserial-3240 /dev/tty.usbserial-FTBHOLDE
    Finished `release` profile [optimized] target(s) in 0.02s
     Running `target/release/examples/hardware_check --loopback-port /dev/tty.usbserial-3240 /dev/tty.usbserial-FTBHOLDE`
Testing '/dev/tty.usbserial-FTBHOLDE':
Testing baud rates...
  9600: success
  38400: success
  115200: success
Testing non-standard baud rates...
  10000: success
  600000: success
  1800000: success
Testing data bits...
  Five: success
  Six: success
  Seven: success
  Eight: success
Testing flow control...
  Software: FAILED (flow control None does not match set flow control Software)
  Hardware: FAILED (flow control None does not match set flow control Hardware)
  None: success
Testing parity...
  Odd: success
  Even: success
  None: success
Testing stop bits...
  Two: success
  One: success
Testing bytes to read and write...
  SerialPort::bytes_to_write: success
  SerialPort::bytes_to_read: success
Test clearing software buffers...
  Input: success
  Output: success
  All: success
Testing data transmission...success
Testing '/dev/tty.usbserial-3240':
Testing baud rates...
  9600: success
  38400: success
  115200: success
Testing non-standard baud rates...
  10000: success
  600000: success
  1800000: success
Testing data bits...
  Five: success
  Six: success
  Seven: success
  Eight: success
Testing flow control...
  Software: FAILED (flow control None does not match set flow control Software)
  Hardware: FAILED (flow control None does not match set flow control Hardware)
  None: success
Testing parity...
  Odd: success
  Even: success
  None: success
Testing stop bits...
  Two: success
  One: success
Testing bytes to read and write...
  SerialPort::bytes_to_write: success
  SerialPort::bytes_to_read: success
Test clearing software buffers...
  Input: success
  Output: success
  All: success
Testing data transmission...success
Testing paired ports '/dev/tty.usbserial-FTBHOLDE' and '/dev/tty.usbserial-3240':
  Transmitting from /dev/tty.usbserial-FTBHOLDE to /dev/tty.usbserial-3240...
     At 2000000,8,n,1,noflow...
FAILED: Custom { kind: TimedOut, error: "Operation timed out" }
thread 'main' panicked at examples/hardware_check.rs:284:13:
assertion `left == right` failed: Received message does not match sent
  left: [
    0x0,
    0x0,
    0x0,
    0x0,
    0x0,
    0x0,
    0x0,
    0x0,
    0x0,
    0x0,
    0x0,
    0x0,
]
 right: [
    0x54,
    0x65,
    0x73,
    0x74,
    0x20,
    0x4d,
    0x65,
    0x73,
    0x73,
    0x61,
    0x67,
    0x65,
]
stack backtrace:
   0:        0x102eab400 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h01b2beffade888b2
   1:        0x102ec24cc - core::fmt::write::hbadb443a71b75f23
   2:        0x102ea9238 - std::io::Write::write_fmt::hc09d7755e3ead5f0
   3:        0x102eab258 - std::sys_common::backtrace::print::h28349e5c25acbac7
   4:        0x102eac588 - std::panicking::default_hook::{{closure}}::hd24b6196784d991e
   5:        0x102eac26c - std::panicking::default_hook::hfcec80a2720c8c73
   6:        0x102eace7c - std::panicking::rust_panic_with_hook::h84760468187ddc85
   7:        0x102eac868 - std::panicking::begin_panic_handler::{{closure}}::he666a5eb600a7203
   8:        0x102eab884 - std::sys_common::backtrace::__rust_end_short_backtrace::h592f44d2bf9f843f
   9:        0x102eac5e0 - _rust_begin_unwind
  10:        0x102ecc2f4 - core::panicking::panic_fmt::h98bbf7bdf4994454
  11:        0x102e466b4 - hardware_check::check_test_message::h1f6f45a25f320ef3
  12:        0x102e434a0 - hardware_check::main::h13a7110e26878ca3
  13:        0x102e47dd8 - std::sys_common::backtrace::__rust_begin_short_backtrace::h39af040a4a38b474
  14:        0x102e47df0 - std::rt::lang_start::{{closure}}::hb79ded03dedd80ed
  15:        0x102ea60f0 - std::rt::lang_start_internal::h39923ab4c3913741
  16:        0x102e46998 - _main

[skmagiik]

Running with devices in the reverse order gives similar stack trace results:

thread 'main' panicked at examples/hardware_check.rs:284:13:
assertion `left == right` failed: Received message does not match sent
  left: [
    0x0,
    0x0,
    0x0,
    0x0,
    0x0,
    0x0,
    0x0,
    0x0,
    0x0,
    0x0,
    0x0,
    0x0,
]
 right: [
    0x54,
    0x65,
    0x73,
    0x74,
    0x20,
    0x4d,
    0x65,
    0x73,
    0x73,
    0x61,
    0x67,
    0x65,
]
stack backtrace:
   0:        0x100e8b400 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h01b2beffade888b2
   1:        0x100ea24cc - core::fmt::write::hbadb443a71b75f23
   2:        0x100e89238 - std::io::Write::write_fmt::hc09d7755e3ead5f0
   3:        0x100e8b258 - std::sys_common::backtrace::print::h28349e5c25acbac7
   4:        0x100e8c588 - std::panicking::default_hook::{{closure}}::hd24b6196784d991e
   5:        0x100e8c26c - std::panicking::default_hook::hfcec80a2720c8c73
   6:        0x100e8ce7c - std::panicking::rust_panic_with_hook::h84760468187ddc85
   7:        0x100e8c868 - std::panicking::begin_panic_handler::{{closure}}::he666a5eb600a7203
   8:        0x100e8b884 - std::sys_common::backtrace::__rust_end_short_backtrace::h592f44d2bf9f843f
   9:        0x100e8c5e0 - _rust_begin_unwind
  10:        0x100eac2f4 - core::panicking::panic_fmt::h98bbf7bdf4994454
  11:        0x100e266b4 - hardware_check::check_test_message::h1f6f45a25f320ef3
  12:        0x100e2348c - hardware_check::main::h13a7110e26878ca3
  13:        0x100e27dd8 - std::sys_common::backtrace::__rust_begin_short_backtrace::h39af040a4a38b474
  14:        0x100e27df0 - std::rt::lang_start::{{closure}}::hb79ded03dedd80ed
  15:        0x100e860f0 - std::rt::lang_start_internal::h39923ab4c3913741
  16:        0x100e26998 - _main

[sirhcel]

This result does not look like what I would have expected in the first place: the received data is all zeroes where I would have expected some garbled data from what you reported in the description.

Well. As you told, that the Prolific adapter works in combination with Minicom - could you please give this combo a spin? May be the initialization sequence of Minicom makes a difference here and we can investigate this further. Like before, with both adapters connected in a cross-over fashion. We have two more examples for this scenario:

• receive_data to just print the received data to stdout. I've fixed this with Fix output from example receive_data #191, as I did not see any output on macOS in the first place. Please use the revision from the PR for testing. You could read the data you type into Minicom as shown below. Please cross-check that flow control is disabled and be aware that linebreaks will not work (as Minicom just sends them as \r).

  $ cargo run --example receive_data -- /dev/tty.usbserial-X 115200
  

• transmit just sends some string at a regular interval. To send some data to Minicom, you could use this as follows:

  $ cargo run --example transmit -- --string 'ping ' /dev/tty.usbserial-X 115200
  

Does this get you some data to be transmitted or received as expected with the Prolific adapter?

[skmagiik]

Alright, I was incorrect. Minicom is also having some issues reading the serial device:

Minicom shows the '.' char as a ? when transmitting from the known working to the Prolific adapter. Trying to transmit from either minicom or tio via the working adapter doesn't succeed with the receive_data.

However, tio (https://github.com/tio/tio) is not having any issues receiving

On the receive test, no matter which solution I tested we don't receive any data here:

[skmagiik]

For future reference. I modified the tio source code to print the termios struct and it did look significantly different than the termios struct that this library tried to write to the tty. Even replicating this struct exactly across to the serialport-rs library and running the example did not resolve. I'm not sure what else could be the culprit.

[sirhcel]

Thank you for pursuing the topic! This sounds strange indeed. Have you looked at ioctl::iossiospeed in this context as well? Just using the termios struct is likely not enough.

And by the way, thank you for the pointer to tio! This is a neat tool I did not know beforehand.

[skmagiik]

I have tried getting that to work, but I was getting an error result (-1) with an errno value of 14 Invalid file descriptor.
I tried submitting fd.0 as well as fd.into_raw() without success.

[sirhcel]

If you could publish this to a fork, I will have a look into it the next days. The fd should be valid and a small number. What fd do you actually get?

[skmagiik]

Looks like i'm getting fd.0 == 3

I'll create a fork for it :)

[skmagiik]

@sirhcel I did discover a solution, though I'm not sure if there is a better way to resolve. Please see my pull request:
Resolve Mac OS setting Prolific Technology adapter #194