protok

Kotlin code generator plugin for protoc.

This project is currently still in beta - please use at your own risk.

Usage

Compiler

You can use the compiler by downloading the latest version from the releases tab.

Once you have downloaded the compiler plugin, you should add the plugin to your PATH so that protoc can access it. For example, you could add the following to your .bash_profile:

export PATH=${PATH}:<path-to-install-location>/protoc-gen-kotlin/bin

Once that is done, you only need to add the --kotlin_out option to your protoc command, and the generator will create kotlin models in the specified location.

protoc --kotlin_out=path/to/kotlin/out input.proto

Runtime

The runtime is required for any implementations using the generated Kotlin models, as it contains the definitions for the required parent classes. please see the runtime module for more information. This will be uploaded to maven central in the future.

implementation ('jp.co.panpanini:protok-runtime:0.0.9')

Retrofit Converter

The retrofit-converter module is a Retrofit Converter factory, which can be used alongside Retrofit to marshal/unmarshal any requests/responses made through Retrofit. After adding the dependency in gradle:

implementation ('jp.co.panpanini:protok-retrofit-converter:0.0.30')

It is as simple as adding the following line to your Retrofit builder:

Retrofit.Builder()
    .addConverterFactory(ProtokConverterFactory.create())
    ...
    .build()

Generated models

Messages

Messages are implemented as data classes in Protok. Given the following input:

syntax = "proto3";

package api;

message Thing {
    string id = 1;
}

The following (truncated) Kotlin class will be generated:

// Code generated by protok protocol buffer plugin, do not edit.
// Source file: thing.proto
package api

data class Thing(
    @JvmField val id: String = "",
    val unknownFields: Map<Int, UnknownField> = emptyMap()
) : Message<Thing>, Serializable {

    constructor(id: String) : this(id, emptyMap())

    override fun protoMarshal(marshaller: Marshaller) = protoMarshalImpl(marshaller)

    fun encode(): ByteArray = protoMarshal()

    override fun protoUnmarshal(protoUnmarshal: Unmarshaller): Thing =
            Companion.protoUnmarshal(protoUnmarshal)

    fun newBuilder(): Builder = Builder()
        .id(id)
        .unknownFields(unknownFields)

    companion object : Message.Companion<Thing> {
        @JvmField
        val DEFAULT_ID: String = ""

        override fun protoUnmarshal(protoUnmarshal: Unmarshaller): Thing {
            ...
        }

        @JvmStatic
        fun decode(arr: ByteArray): Thing = protoUnmarshal(arr)
    }

    class Builder {
        var id: String = DEFAULT_ID

        var unknownFields: Map<Int, UnknownField> = emptyMap()

        fun id(id: String?): Builder {
            this.id = id ?: DEFAULT_ID
            return this
        }

        fun unknownFields(unknownFields: Map<Int, UnknownField>): Builder {
            this.unknownFields = unknownFields
            return this
        }

        fun build(): Thing = Thing(id, unknownFields)
    }
}

The important points of this class are:

Constructor

The constructor takes all of the proto defined fields as parameters, along with the unknownFields map for any fields that are not known to this version of the proto model. Note For proto3 syntax, all of these fields are Non-null, including nested message classes. There is also a secondary constructor which provides a default unknownFields value. This is for ease of use when creating an instance of the message in Java.

Companion Object

The companion object provides access to a few static methods, and default values for all fields in the message. The static methods provided are for unmarshalling a message from its wire format.

Marshal/Unmarshal functions

These functions are used for converting to/from the wire protocol buffer format. In general, if you are using Retrofit, you shouldn't need to worry about these functions, however if you want to serialize the message for one reason or another, you can use these functions. They are also duplicated under the function names encode() and decode().

Builder

To aid in creating a message from Java where named parameters are not available, there is a Builder class provided, which follows the builder pattern. All values inside the builder class are set to the default value for the field, and passing null to the builder function will reset that value back to the default value, so we are able to protect the non-nullability of message fields. There is also a newBuilder() function for a message, which will return a Builder instance populated with the values from the fields of the current instance.

Enum

Enum classes are also implemented as a data class. This is due to the proto3 requirement:

During deserialization, unrecognized enum values will be preserved in the message due to this, any when statements using generated enums should provide an else block to ensure all possible cases are covered. Lets look at the following input:

syntax = "proto3";

package api;

enum Language {
    PROTOBUF = 0;
    KOTLIN   = 1;
    JAVA     = 2;
    SWIFT    = 3;
    GO       = 4;
}

This will generate the following code:

// Code generated by protok protocol buffer plugin, do not edit.
// Source file: language.proto
package api

data class Language(override val value: Int, @JvmField val name: String) : Serializable,
        Message.Enum {
    override fun toString(): String = name
    companion object : Message.Enum.Companion<Language> {
        @JvmField
        val PROTOBUF: Language = Language(0, "PROTOBUF")

        @JvmField
        val KOTLIN: Language = Language(1, "KOTLIN")

        @JvmField
        val JAVA: Language = Language(2, "JAVA")

        @JvmField
        val SWIFT: Language = Language(3, "SWIFT")

        @JvmField
        val GO: Language = Language(4, "GO")

        @JvmStatic
        override fun fromValue(value: Int): Language = when(value) {
            0 -> PROTOBUF
            1 -> KOTLIN
            2 -> JAVA
            3 -> SWIFT
            4 -> GO
            else -> Language(value, "")
        }

        @JvmStatic
        fun fromName(name: String): Language = when(name) {
            "PROTOBUF" -> PROTOBUF
            "KOTLIN" -> KOTLIN
            "JAVA" -> JAVA
            "SWIFT" -> SWIFT
            "GO" -> GO
            else -> Language(-1, name)
        }
    }
}

The important points from this class are:

constructor

The constructor for these enums take two parameters - the value of the enum, and the name. The name is used in the toString() function to ensure the expected value for an enum is returned.

Companion Object

The companion object itself holds all (known) values for this Enum, and these can be referred to from a static context.

fromValue/fromName

There are two functions to help in getting the correct value for an enum when you only have the value or name, these are fromValue() and fromName() respectively. Note fromName() will set the value to -1 if it cannot find a known enum case. This is not efficient when converted to a wire representation of the enum (as enum values are represented as a varint), so it is advised not to use this option wherever possible.

How to contribute

Clone the repo & send a PR! 🎉 Code related to the code generator lives in the library module Code related to the runtime components is in the runtime module Code related to the Retrofit Converter is in the retrofit-converter module.

TODO

• custom services for RPC
• refactor code generated using only strings (use KotlinPoet correctly)
• ensure both proto2 and proto3 support
• tests
  • Retrofit-converter
  • test for both proto2 and proto3

Acknowledgments

This project is heavily influenced by pbandk. For a closer-to-finished solution, please take a look!