Parallel (buffered) readout of multiple I2C devices, python 3

Question

For experimental measurements of human balance, I would like to connect and read out multiple I2C sensors (gyro/mag/accelerometer, such as this one).

I have managed to hook them up (libftdi+python3.7) and can read from single registers.

However, I require the data from multiple devices and registers to be (approximately) synchronized or time-logged. How can I achieve this?

(Edit: a nice solution would be parallelized buffers of fixed length, one per I2C device/register, which are filled in the background and store upon a post-trigger.)

Background:

(1) I have not understood how ICL clocking of the I2C works. I know one can set a reading frequency, say 100Hz. I can read at arbitrary time points, and the sensor values seem to change, but maybe not all the time. How is the exact timing of the readouts controlled?

(2) Or is clocking only part of the solution? I have been looking for buffering solutions, and found bytearrays and io.BytesIO. But I don't know how to read only new values. Most guides refer to file storage, but I would rather not stream to a file (the data is not big). I have also "read a bit" (literally) into asynchronous execution, with asyncio. However, my attempt (below) is not ideal, because it would require large overhead to store precise time points of the measurements. And there are other limitations, see below.

(3) My problem is amplified because, by now, I can only read single registers from the I2C, i.e. I have to loop and read 9 times to get 9DOF data. I learned there are multiple solutions to access I2C via python, I'm using the FT232H breakout[*] which is supported by many of those. Some of them query multiple bytes at once, probably involving some buffer. However, the only way I got it to work was using libftdi and porting the I2C part from Adafruit_GPIO to python3. So I could not reproduce the parallel byte query yet. (how exactly) Is libftdi capable of such queries? Maybe this can be extended?

Starting Point:

My first intuition was to do something with multiprocessing and queues. However, when having multiple threads or processes, they would drift if I do not get the ICL syncing right (see 1). Also, Queues seem to be filled fifo, but if reading at sufficient rates I could work with last-in-first-out and a flush.

I then found out about asyncio and have made my first attempts. I have then tried to use a buffer-like connection based on io.BytesIO. But I've never learned how to really fill, read from and flush buffers, so my approach is probably clumsy.

Here's an emulation of the problem, creating random bytes and processing them.

import asyncio as AIO
import io as IO
import numpy as NP


# a sender object that keeps spitting out data
class Sender(IO.BytesIO):
    def __init__(self, label):
        super(Sender, self).__init__()
        self.label = label

    async def GetGoing(self):
        await self.PutStuffIn()

    async def PutStuffIn(self):
        while True:
            rnd = NP.random.bytes(1)
            print (self.label, 'sending', rnd)
            self.write(rnd)
            await AIO.sleep(0.01)



### a receiving object that collects data
class Receiver(object):
    def __init__(self, senders):
        self.streams = senders

    async def GetReceiving(self):
        await self.ReadStuffOut()

    async def ReadStuffOut(self):
        pointers = {stream.label: 0 for stream in self.streams}
        while True:
            await AIO.sleep(0.02)
            for stream in self.streams:
                data = stream.getvalue()
                print (stream.label, 'received', data[pointers[stream.label]:])
                pointers[stream.label] = stream.tell()


### main process
async def Main():
    # create multiple senders and a receiver
    senders = [Sender(1), Sender(2), Sender(3)]
    read = Receiver(senders)

    # execute sender and receiver in parallel
    await AIO.wait( [ \
        send.GetGoing() \
        for send in senders] \
        +[ read.GetReceiving() \
        ] )


### mission control
if __name__ == "__main__":
    loop = AIO.get_event_loop()
    loop.run_until_complete(Main())

Expectation:

As indicated above, this kind of works, but:

• The use of io.BytesIO.getvalue() seems to get the whole buffer all the time, and it is not emptied: I guess I have created a list, not a real buffer?

• With asyncio, execution is non-deterministic, so I cannot pre-allocate a buffer of fixed size. I'm using it only for its speed because I want to sample quicker than the I2C clock, so it might be the wrong track. On the other hand, it seems to be a good way to have multiple senders work quasi-parallel.

• (not shown) I would like to learn more about the deep structure of parallel I2C register access, and whether it can be used across devices.

Many thanks in advance for hints and guidance!

By the way, I'm almost illiterate to C, but (happy to learn) would also appreciate code examples in that language.

[*] I'm using the FT232H breakout, and not my odroid, because on top of what I presented there will be force plate data acquired through a DAQ. So that will have to be parallelized as well, which can happen via multiprocessing.

Answer 1

I2c protocol only supports serial data transfer from one device at a time. Typical multiplexers only shift the main interface to each isolated lane, so they can only support serial/sequential reads. In order to read multiple i2c devices in parallel each identical device should be on an physically isolated bus to the processor. Then each device can be read in parallel using multithreading, and finally all data can be aggregated. This would take about the same time as it would to only read from one sensor, but yield more data. To do this on pi maybe possible using extra software defined i2c busses and multithreaded code to read from them. Otherwise I don’t know of any hardware that performs such a function. It maybe possible to program an FPGA to do this at hardware level.