Use row protocol to support opentsdb

[fengjiachun]

What type of enhancement is this?

Refactor

What does the enhancement do?

In version 0.4, we support gRPC protocol based on row insert protocol.
We are currently using the row insert protocol to support both InfluxDB line protocol and Prometheus remote write.

We also provide tools to complete the conversion to row protocol.

Now, we are using column insert protocol to support OpenTSDB, This causes the overhead of having to convert to rows when actually writing (which can be avoided).

Implementation challenges

No response

[Lilit0x]

Hi @fengjiachun, I am interested in working on this refactor. Below is my understanding of the problem and how I might go about solving it.

As you have mentioned, it will be easier if OpenTSDB also supported the row insert protocol instead of column insert as in here.
Basically, I will want to convert the DataPoint type for an OpenTSDB request to a RowInsertRequests type to make it compatible with calling Instance.handle_row_inserts instead of handle_inserts as is defined here.
To do that, I will use the MultiTableData struct to construct rows from the tags, timestamps, values (all gotten from the DataPoint struct) and then finally to the desired RowInsertRequests type.

Thank you.

[fengjiachun]

@Lilit0x You are right 👍🏻

[fengjiachun]

Hi @Lilit0x , are you still working on this issue?

[Lilit0x]

Hi @fengjiachun .
Yes, I am, but I messed up the commits because I tried to squash my commits together since I had so many unnecessary commits fixing the tests. I had issues with the tests because I couldn't run them locally because it fails to compile every time. Below is a screenshot in case you might have a suggestion to fix it.
I don't know if it is because there's no more swap memory for it to compile or something else, I can't seem to figure it out.

[waynexia]

This might be an OOM. You can try to limit the compile job numbers by --build-jobs. E.g., cargo nextest run --build-jobs 4