Maximising I2C throughput

[ThinkTransit]

I have an I2C device that samples data at a rate of 8KHz, the I2C bus operates at 400KHz. The data size is 12 bytes + another 4 bytes for overhead and the initial write command, total of (16 bytes/128 bits). The theoretical max speed I could read data from the device would be 3,100 samples per second (400,000/128).

Using the following code I can read 8,000 samples in 4.9 seconds (1,600 samples per second). I'm reasonably happy with this result however I'm wondering if there are any easy gains to achieve a bit more bandwidth. Is there a bottleneck in my code where I append data to the buffer? Or maybe this is not an issue considering the clock speed of the ESP compared to the I2C bus?

Any thoughts?

def waveform_read(self):
    i = 0

    while i < 8000:
        self.i2c.writeto(ADDR, bytes([0xE5, 0x10]))
        data = self.i2c.readfrom(ADDR, 12)
        self.buffer.append(data)
        i += 1

[robert-hh]

You could try to vary the clock speed of I2C and compare the time for a transaction. Then you can tell which is fixed processing time and which data transfer time. Or you monitor the I2C bus directly. It may be that increasing the I2C clock speed is possible.

[ThinkTransit]

Thanks Rob that's a good idea I'll try that. I suppose I can also try removing the append to buffer to see if that increases speed.

[robert-hh]

Storing into a once pre-allocated array should be faster than appending to a list.

[robert-hh]

You could also pre-create a list containing 12 byte long bytearray objects, and then use i2c.readinto() for each element of the list. That avoids allocation memory during fetching data.

[dpgeorge]

You can time the I2C transactions by wrapping them in code like this:

t0 = time.ticks_us()
i2c...
dt = time.ticks_diff(time.ticks_us(), t0)
print(dt)

That has given good results for me in the past.

Also, I2C on ESP32 can be slow if you do lots of separate transactions. This was improved in 6bda80d. To take advantage of that you need to use i2c.readfrom_mem(...) which will do a back-to-back write-then-read (see also writeto_mem).

For efficient use of memory, see readfrom_into and readfrom_mem_into.

[ThinkTransit]

Thanks @robert-hh and @dpgeorge

I have put together some test cases and will leave the results here as it might help others.

Test 1 - base case
4,930.467ms

Test 2 - Discard data (don't append to buffer)
4,928.674ms

Test 3 - I2C read_from_mem
4,692.552ms

Test 4 - I2C read_from_mem_into with preallocated buffer
4659.853ms

Seems like appending to list doesn't impact run time
Biggest time saving is by using readfrom_into or readfrom_mem_into ~ 300ms faster or 5%

    def timed_function(f, *args, **kwargs):
        myname = str(f).split(' ')[1]

        def new_func(*args, **kwargs):
            t = time.ticks_us()
            print('Function { myname } starting')
            result = f(*args, **kwargs)
            delta = time.ticks_diff(time.ticks_us(), t)
            print('Function {} Time = {:6.3f}ms'.format(myname, delta / 1000))
            return result

        return new_func

    @timed_function
    # Base case
    def waveform_read(self):
        i = 0

        while i < 8000:
            self.i2c.writeto(ADDR, bytes([0xE5, 0x10]))
            data = self.i2c.readfrom(ADDR, 12)
            self.buffer.append(data)
            i += 1

    @timed_function
    # Read data from I2C but don't append to list
    def waveform_read_2(self):
        i = 0

        while i < 8000:
            self.i2c.writeto(ADDR, bytes([0xE5, 0x10]))
            data = self.i2c.readfrom(ADDR, 12)
            i += 1

    @timed_function
    # Read data using I2C read_from_mem
    def waveform_read_3(self):
        i = 0

        while i < 8000:
            data = self.i2c.readfrom_mem(ADDR, 58640, 12, addrsize=16)
            self.buffer.append(data)
            i += 1

    @timed_function
    # Read data directly into buffer using I2C read_from_mem_into
    def waveform_read_4(self):
        i = 0

        s = bytearray(12)

        while i < 8000:
            self.i2c.readfrom_mem_into(ADDR, 58640, s, addrsize=16)
            self.buffer.append(s)
            i += 1

[dpgeorge]

You can compute the expected best case time by t=num_bits/freq, which gives:

t = 8000 * 10 * (1 + 2 + 12) / 400000
t = 3

So that's 3 seconds, assuming 10 clock cycles per byte and a rate of 400kHz for the SCL.

[ThinkTransit]

Make sense, so 4.6 actual vs 3 theoretical isn't too bad I think. There are lots of other factors such as the delay in the IC side between the command and returned data that I can't change.