mscheong01 / krotodc

Protobuf Kotlin Dataclass, Converter & Custom Kotlin Coroutine gRPC Generator

krotoDC

krotoDC is a protoc plugin for generating Kotlin data classes and gRPC service/stub from a .proto input. This library makes it easy to work with Protocol Buffers and gRPC in your Kotlin projects.

check out my blog post that introduces krotoDC

Features

• Generates Kotlin data classes from .proto files

• Generates Converter extension methods between generated classes and GeneratedMessageV3 classes

• Generates gRPC service/stub based on the service definition in .proto files

• Supports Kotlin specific features like nullable fields and sealed oneof classes:

  see the generated code spec here

Installation

In your project's build.gradle.kts file, add the following dependencies:

dependencies {
    implementation("com.google.protobuf:protobuf-java:3.25.3")
    implementation("com.google.protobuf:protobuf-java-util:3.25.3")
    implementation("io.grpc:grpc-stub:1.61.1")
    implementation("io.grpc:grpc-kotlin-stub:1.4.1")
    implementation("io.github.mscheong01:krotoDC-core:1.1.1")
}

Usage

1. Configure the protobuf plugin in your build.gradle.kts file:

protobuf {
    protoc {
        artifact = "com.google.protobuf:protoc:3.25.3"
    }
    plugins {
        id("grpc") {
            artifact = "io.grpc:protoc-gen-grpc-java:1.61.1"
        }
        id("krotoDC") {
            artifact = "io.github.mscheong01:protoc-gen-krotoDC:1.1.1:jdk8@jar"
        }
    }
    generateProtoTasks {
        all().forEach {
            it.plugins {
                id("grpc")
                id("krotoDC")
            }
        }
    }
}

2. Create your .proto files with your message and service definitions.

3. Build your project, and the plugin will generate Kotlin data classes and gRPC service/stub based on the .proto files.

Snapshot Versions

krotoDC provides snapshot versions that are automatically released when changes are pushed to the main branch. The current snapshot version is the next minor version of the current release version with a -SNAPSHOT suffix. For example, if the current release is 1.2.3, the snapshot version will be 1.3.0-SNAPSHOT.

To use snapshot versions, add the maven snapshot repository to your build configuration

repositories {
    maven {
        url = uri("https://s01.oss.sonatype.org/content/repositories/snapshots/")
    }
}

Contributing

Contributions are welcome! Please see our contributing guidelines for more information.

License

This project is licensed under the Apache 2.0 License.

krotodc's Issues

Deserialize sealed classes generated by oneof

For example, suppose we have:

message TestWrapper {
  oneof data {
    string typeA = 1;
    string typeB = 2;
  }
}

Which generates:

@KrotoDC(forProto = TestWrapper::class)
public data class TestWrapper(
  public val `data`: Data? = null,
) {
  public sealed interface Data {
    public data class TypeA(
      public val typeA: String = "",
    ) : Data

    public data class TypeB(
      public val typeB: String = "",
    ) : Data
  }
}

By default, this serializes to:

"wrapper": {
  "data": {
    "typeA": "foobar"
  }
},

However it cannot be deserialized by Gson without specifying an InstanceCreator or TypeAdapter for that specific type.

Is there a way to handle serialization/deserialization of oneofs that I'm missing, or is this just something that's not yet supported by the KrotoDC code generation?

Windows support

Hi! The plugin seems to work fine on Linux, but on windows I've reached this problem:

// test.proto
syntax = "proto3";
message Test {}

:: run-krotoDC.cmd
java -jar protoc-gen-krotodc-1.0.0-jdk8.jar %*

Now run protoc --plugin=protoc-gen-krotoDC=run-krotoDC.cmd --krotoDC_out=. test.proto, and we get:

--krotoDC_out: protoc-gen-krotoDC: Plugin output is unparseable: \r\nC:\\Users\...>java -jar protoc-gen-krotoDC-1.0.0-jdk8.jar \r\n\020\001z\214\001\n\020/krotodc/Test.ktzxpackage krotodc\n\nimport io.github.mscheong01.krotodc.Krotodc\n\n@KrotoDC(forProto = Test.Test::class)\npublic class Test()\nz\225\004\n\037/krotodc/test/TestConverters.ktz\361\003package krotodc.test\n\nImport.....

To me it seems like a \n / \r\n issue. Any thoughts?

Enum argument can be null in Java, but exhaustive when contains no null branch

First, thanks so much for this project. It should be part of the Kotlin/gRPC distro.

The generated .toDataClass() methods are generating this compiler warning

Enum argument can be null in Java, but exhaustive when contains no null branch

for the when argument. Adding

null -> null

would get rid of the warning.

Again, thanks for your efforts with this project.

Problem when the name of a 'oneof' field matches the name of the message.

Simple Example:

message Content {
  oneof Content {
    string text = 1;
  }
}

is converted into

@KrotoDC(forProto = a.b.c.Content::class)
public data class Content(
    public val Content: Content? = null,
) {
    public sealed interface Content {
        public data class Text(
            public val text: String = "",
        ) : Content
    }
}

occurs kotlin compile error.

This could be a minor issue because typically, the name of the 'oneof' field is not the same as the message's field.
However, it may not be a forbidden action, so it requires support.

Generated defaultValue for TIMESTAMP is `LocalDateTime.MIN`

It causes an exception like below.

java.lang.IllegalArgumentException: Timestamp is not valid. See proto definition for valid values. Seconds (-31557014135596800) must be in range [-62,135,596,800, +253,402,300,799]. Nanos (0) must be in range [0, +999,999,999].
	at org.curioswitch.common.protobuf.json.ProtobufUtil.checkValid(ProtobufUtil.java:420)
	at org.curioswitch.common.protobuf.json.ProtobufUtil.formatTimestamp(ProtobufUtil.java:84)
	at org.curioswitch.common.protobuf.json.WellKnownTypeMarshaller$TimestampMarshaller.doWrite(WellKnownTypeMarshaller.java:279)
	at org.curioswitch.common.protobuf.json.WellKnownTypeMarshaller$TimestampMarshaller.doWrite(WellKnownTypeMarshaller.java:257)
	at org.curioswitch.common.protobuf.json.WellKnownTypeMarshaller$WrapperMarshaller.writeValue(WellKnownTypeMarshaller.java:64)
	at org.curioswitch.common.protobuf.json.SerializeSupport.printMessage(SerializeSupport.java:219)
	at org.curioswitch.common.protobuf.json.TypeSpecificMarshaller$ByteBuddy$diaKldbv.doWrite(Unknown Source)
	at org.curioswitch.common.protobuf.json.TypeSpecificMarshaller$ByteBuddy$diaKldbv.doWrite(Unknown Source)
	at org.curioswitch.common.protobuf.json.TypeSpecificMarshaller.writeValue(TypeSpecificMarshaller.java:88)

I think it should be something like com.google.protobuf.Timestamp.getDefaultInstance().toLocalDateTime().

Can I try to fix it?

Future vision and stability

What is the future vision of this library? What are the missing features that you are willing to add and any new features you are willing to add?

For stability, I think this library looks pretty good and I am thinking about using this in production environments, but there are several caveats,

1. I am unsure about stability and correctness of the api considering #13 and #19.
2. I am not sure about how well this will be maintained and whether this is gonna be used by any big project / company.
3. Conformance tests.

Those little thinks add some confidence to me. I really appreciate the work and enthusiasm and will be more than willing to help 👍 I think kotlin - protobuf compatability is something that definetly have a lot of room for improvement.

For example, can you think about reasons for using this project over https://github.com/streem/pbandk or https://github.com/open-toast/protokt?

Add option to make message type fields always nullable

In proto3, message type fields are always optional; adding the optional keyword does not change the compiled java code
i.e)

message Message {
    Field field = 1;
}
message Field {
    string name = 1;
}

is equal to

message Message {
    optional Field field = 1;
}
message Field {
    string name = 1;
}

However, krotoDC currently generates different code for these two cases

data class Message(
    val field: Field = Field()
)

data class Message(
    val field: Field? = null
)

Because of this, when a krotoDC message type field is not set, while message type fields with the optional keyword are "unset", fields without the keyword is set with an empty message (with all fields unset).
Strictly speaking, this contradicts the proto3 syntax's basic rule; that all message fields are optional. But it is true that this difference could be preferred for certain cases.
The best way to go would be adding an option for the krotoDC protoc plugin that when enabled, creates all message type fields as nullable. then, if this is preferred by the majority of users, we could adpot it as the default behavior

Tests not executed on `./gradlew test`

Reserved keywords are not escaped

When proto messages have field names that match Kotlin reserved keywords, error occurs in Converter where the field is used.
example proto: https://github.com/grpc-ecosystem/grpc-gateway/blob/23ec42db4030b50c1afa602f8329cd6555de944d/protoc-gen-openapiv2/options/openapiv2.proto#LL664C13-L664C13

Fix flaky WellKnownTypesTest junit tests

WellKnownTypesTest pass in local development environment, but fails in CI

Support generic serialization/deserialization methods for krotoDC dataclasses

reported at: #24 (comment) by @anjanbk

The currently provided serialization/deserialization extension methods, toJson and fromJson are hard to use inside generic serialization/deserialization classes.
Although workaround through reflection is possible(something like this), official support through krotoDC would be better.

[discussion] compatible with two cases

There were some cases I encountered in wild, I will illustrate using the following code as an example.

message Persons {
  Person person = 1;
  Person person_2nd = 2;
  oneof TagCode {
    string tag = 3;
    string tag_2nd = 4;
  }
}

oneof type

In the provided code, if the type name of "oneof" is capitalized (which is technically not recommended and may be considered illegal), the "krotoDC" fails to generate the code correctly. However, if we compare it with the code generated by the Protoc Java plugin for the "oneof" type, we observe that making the first character lowercase produces the same result as the Java plugin.

special field name like xx_2nd

In the provided code, if the field name follows the pattern "_{digit}{first_alphabet}{other_alphabets}", the "protoc" Java plugin converts the first alphabet of "xx_2nd" to uppercase, resulting in "xx2Nd". However, in the current implementation of the "krotoDC" code, the field name is converted to "xx2nd", which is not compatible with the original Java code.

The draft PR serves as a reference, and it's important to note that the implementation is subject to change.