Using google protobuf with fast-dds

[samehmohamed88]

Is there an already existing issue for this?

• I have searched the existing issues

Expected behavior

I am attempting to use Google Protobuf as the message definition and serialization. I have seen an example from Baidu's Apollo CyberRT where they use a kind of UnderlayMessage which is supposed to be a generic type. In the underlay type class they provide the overloads for the serialize and deserialize required by fast-dds.

They populate the underlay message's data string class member with the protobuf::SerializeToString which serializes the protobuf class to a binary represented as string.

The UnderlayMessage is then sent to the subscriber, who then uses protobuf::ParseFromString to recover the original message.

I was able to implement this successfully when my protobuf message contains string or int32, however using a float or double results in the payload data being empty when received by the subscriber.

           template<class M>
            bool Publisher::publish(M protobuf,
                                    bool waitForListener) {

              UnderlayMessage message_;
                
                protobuf.set_floatval1(1.5);
                msg.set_floatval2(0.5);
                
                if (listener_.firstConnected_ || !waitForListener || listener_.matched_ > 0) {
                    std::string str;
                    protobuf.SerializeToString(&str);
                    message_.data(str);
                    writer_->write(reinterpret_cast<void*>(&message_));
                    return true;
                }
                return false;
            }

The subscriber:

      void Subscriber::SubListener::on_data_available(
                    DataReader *reader) {
                SampleInfo info;

                std::this_thread::sleep_for(std::chrono::milliseconds(200));
                UnderlayMessage message_;
                proto::MyMessage msg;
                if (reader->take_next_sample(reinterpret_cast<void*>(&message_), &info) == ReturnCode_t::RETCODE_OK) {
                    if (info.instance_state == ALIVE_INSTANCE_STATE) {
                        samples_++;
                        msg.ParseFromString(message_.data());
                       // My Message float values are 0 and 0, instead of 1.5 and 0.5
                    }
                }

Using a debugger, I stepped through the desrialize method and the void* data that is passed in is empty, whereas when I am using an int32 or a string in my protobuf class, it received the correct values.

Current behavior

Described above

Steps to reproduce

Code supplied above

Fast DDS version/commit

Installed on Ubuntu 20.04 using apt package manager

Platform/Architecture

Ubuntu Focal 20.04 amd64

Transport layer

Shared Memory Transport (SHM)

Additional context

No response

XML configuration file

No response

Relevant log output

No response

Network traffic capture

No response

[richiware]

According to Protobuf documentation:

bool SerializeToString(string* output) const;: serializes the message and stores the bytes in the given string. Note that the bytes are binary, not text; we only use the string class as a convenient container.

If the returned std::string countains null bytes, then I suggest to read this answer #2786.

[richiware]

Instead of using a IDL string, you might use a IDL sequence.

[EduPonz]

We have been toying with the idea of providing a ready-to-use type support for things like protobuf (or any other application level serialization mechanism) but we never seem to have time for that. Actually, what we would do is for the data type to simple be a byte sequence (sequence<octet>) which the application would populate with a serialized buffer (as opposed to converting thing to a from strings, which has the problem of null termination characters).

[samehmohamed88]

I don't believe it's null terminated but I will check in the morning. However, why would it work when the serialized data is a string or integer, but not for double or float?

[samehmohamed88]

Using the IDE debugger, I was able to see the value of the serialized string as "\r\000\000\300?\025\000\000\000?" for a float value, where as for an integer it is "\b\001\020\005".

The problem is as you guys predicted. I'll examine using an std::vector<char> this afternoon/weekend, and report back here.