Include proto-files in jar

[ulejon]

Hi,

I'm trying out this plugin as a replacement over https://github.com/xolstice/protobuf-maven-plugin.
In the old plugin we could include the protobuf-files in the generated JAR. I'm guessing these configurations made that possible:

        <configuration>
          <writeDescriptorSet>true</writeDescriptorSet>
          <attachDescriptorSet>true</attachDescriptorSet>
          <descriptorSetFileName>fleet-domain.desc</descriptorSetFileName>
          <descriptorSetOutputDirectory>${project.build.outputDirectory}</descriptorSetOutputDirectory>
          <includeDependenciesInDescriptorSet>true</includeDependenciesInDescriptorSet>
        </configuration>

Is it possible to include the protobuf-files using this plugin?
I need them there to be able to reference/import them from libraries that are depending on our JAR

[ulejon]

Think I just found the answer myself! Adding this configuration did the job: <embedSourcesInClassOutputs>true</embedSourcesInClassOutputs>

[ulejon]

I'll show an example when I'm at the computer tomorrow!

[ulejon]

Here's a small demo I made.
I wonder if this piece of code could be more type safe with a descriptor definition, not having to use findFieldByName("status").
DemoMessage.getDescriptor().findFieldByName("status") (In the ProtoToJsonDemo class)

demo.proto

syntax = "proto3";

option java_multiple_files = true;

package se.ulejon.demo;

message DemoMessage {
  string name = 1;
  Status status = 2;
}

enum Status {
  ACTIVE = 0;
  ARCHIVED = 1;
}

ProtoToJsonDemo.java

package se.ulejon.demo;

import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;

import com.google.protobuf.Descriptors;
import com.google.protobuf.InvalidProtocolBufferException;
import com.google.protobuf.MessageOrBuilder;
import com.google.protobuf.util.JsonFormat;

public class ProtoToJsonDemo {

    private static final JsonFormat.Printer defaultPrinter = JsonFormat.printer().omittingInsignificantWhitespace();

    /**
     * Create a json representation omitting default values for unset fields
     */
    public static String toExactJson(MessageOrBuilder messageOrBuilder) {
        return toJson(messageOrBuilder, defaultPrinter);
    }

    /**
     * Create a json representation omitting default values for unset fields
     */
    public static String toJsonWithoutDefaultValues(MessageOrBuilder messageOrBuilder) {
        return toJson(messageOrBuilder, false);
    }

    /**
     * @param useAllDefaultValues if false, no default values will be included for unset fields. If true, all default values will be included
     */
    public static String toJson(MessageOrBuilder messageOrBuilder, boolean useAllDefaultValues) {
        if (useAllDefaultValues) {
            return toJson(messageOrBuilder, defaultPrinter.alwaysPrintFieldsWithNoPresence());
        }
        return toJson(messageOrBuilder, defaultPrinter);
    }

    /**
     * Create a json representation with default values defined in {@link ProtoToJsonDemo#allCustomDescriptors()}
     */
    public static String toJsonWithCustomDefaultValues(MessageOrBuilder messageOrBuilder) {
        return toJson(messageOrBuilder, allCustomDescriptors());
    }

    /**
     * Create a json representation with default values
     * @param defaultValueFields the fields to include default values for
     */
    public static String toJson(
        MessageOrBuilder messageOrBuilder,
        List<Descriptors.FieldDescriptor> defaultValueFields
    ) {
        if (defaultValueFields != null && !defaultValueFields.isEmpty()) {
            return toJson(
                messageOrBuilder,
                defaultPrinter.includingDefaultValueFields(new HashSet<>(defaultValueFields))
            );
        }
        return toJson(messageOrBuilder, defaultPrinter);
    }

    private static String toJson(MessageOrBuilder messageOrBuilder, JsonFormat.Printer jsonPrinter) {
        try {
            return jsonPrinter.print(messageOrBuilder);
        } catch (InvalidProtocolBufferException ignored) {}
        return "{}";
    }

    public static List<Descriptors.FieldDescriptor> allCustomDescriptors() {
        var fieldDescriptors = new ArrayList<>(demoDescriptors());
        //fieldDescriptors.addAll(... more descriptors ...);
        return fieldDescriptors;
    }

    public static List<Descriptors.FieldDescriptor> demoDescriptors() {
        return List.of(
            DemoMessage.getDescriptor().findFieldByName("status")
        );
    }
}

ProtoToJsonDemoTest.java

package se.ulejon.demo;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static se.ulejon.demo.ProtoToJsonDemo.allCustomDescriptors;

import java.util.List;

import org.junit.jupiter.api.Test;


class ProtoToJsonDemoTest {

    @Test
    void demoMessage_toJson() {
        var demoMessage = DemoMessage.newBuilder().build();
        assertEquals("{}", ProtoToJsonDemo.toExactJson(demoMessage));
        assertEquals("{}", ProtoToJsonDemo.toJsonWithoutDefaultValues(demoMessage));
        assertEquals("{}", ProtoToJsonDemo.toJson(demoMessage, false));
        assertEquals("{}", ProtoToJsonDemo.toJson(demoMessage, List.of()));
        assertEquals("{}", ProtoToJsonDemo.toJson(demoMessage, null));

        var expectedWithAllDefaults =
            """
            {"name":"","status":"ACTIVE"}
            """.strip();
        var expectedWithCustomDefaults =
            """
            {"status":"%s"}
            """.formatted(Status.forNumber(0).name()).strip();

        assertEquals(expectedWithAllDefaults, ProtoToJsonDemo.toJson(demoMessage, true));
        assertEquals(expectedWithCustomDefaults, ProtoToJsonDemo.toJsonWithCustomDefaultValues(demoMessage));
        assertEquals(expectedWithCustomDefaults, ProtoToJsonDemo.toJson(demoMessage, allCustomDescriptors()));
    }
}

[ascopes]

Ahh ok, I see.

Are you able to do this without desc files? The generated classes should contain reflective information unless you use lite sources.

[ulejon]

Yes, the above example using your plugin, thus without the descriptor file. And we had the exact same code using the other maven plugin. So I guess we never really used the descriptor file as intended (however that is).
I just found this post on SO: https://stackoverflow.com/questions/68667180/protobuf-java-create-a-descriptor-from-descriptorproto.
Maybe that could be a way of doing this more type safe.

[ascopes]

Cool, okay. Will have a look at the weekend to see if I can understand more about how they are used in this setting. My understanding was that they mostly acted as an archive of multiple protobuf sources. If I come up with anything, I'll throw some issues together for it.

Thanks for all the info, it is greatly appreciated. Also, I'm glad to hear the plugin is working for you!