tomboyo / lily

Java 17 native HTTP client code generator for OpenAPI v3

Lily OAS-To-Java Compiler

Lily is a compiler that consumes OpenAPI (Swagger) specifications and produces Java source code. It is intended to be an alternative to OpenAPI Generator, which at time of writing is the only option, and substantially more complete than Lily.

Usage

Please be aware that Lily is in the early stages of development and may not be suitable for production use-cases. Use Lily at your own risk.

That said, beta testing is the best way to contribute to this project.

Java API

Generated source code looks like this:

List<Pet> exampleHappyPath() {
  var api = Api.newBuilder()
      .uri("https://example.com/")
      .build();

  try {
    var response = api.petsOperations()
        .listPets()
        .query(query -> query.limit(50))
        .sendSync();

    return switch (response) {
      case ListPets200 ok -> ok.body().value();
      case ListPetsDefault other -> throw new RuntimeException(other.body().message());
    };
  } catch (IOException | InterruptedException e) {
    // The java.net.http layer encountered an exception.
    throw new RuntimeException("Unable to complete Pets API request", e);
  }
}

Here are some of the Lily features we just saw:

• If an operation has the 'pets' tag, then we can access it via the petsOperations() operation group. Every operation is also part of the everyOperation() group, and operations without tags are also members of the everyUntaggedOperation() group. These groups are intended to help us explore the API using IDE type-ahead/auto-complete hints.

• Responses form a sealed interface. If we have the pattern-matching for switch expressions feature enabled, we can create an exhaustive switch expression to handle all possible responses, including undocumented and unexpected ones. Otherwise, we can use pattern-matching in an if-else ladder, or even access the status code via response.httpResponse().statusCode() (the native java.net.http API).

In the real world, OpenAPI specifications have errors in them that could prevent a generated API from successfully making requests. Rather than wait for service owners to update their specifications or try to fix them in a local copy ourselves, we can use Lily’s API to do as much as possible, then dip down into the underlying java.net.http API for full customization and control:

var operation = api.petsOperations()
        .listPets()
        .query(query -> query.limit(50));
var request = HttpRequest.newBuilder(operation.httpRequest(), (k, v) -> true)
        .header("x-some-undocumented-header", "foo;bar;baz")
        .build();

// If the API has correctly documented responses, lily will help us deserialize
// the response and we can handle it like before.
var response = operation.sendSync(request);

// Otherwise, we can use the httpClient to get an HttpRequest of an InputStream
// and deserialize it however we see fit, including not at all.
var response = api.httpClient().send(request, BodyHandlers.ofInputStream());

Here’s what we just saw:

• We can use the operation to customize an HttpRequest, then use the java.net.http API to copy-and-modify that request. We can use Lily for everything that is documented by the OpenAPI specification correctly, but then arbitrarily modify the request with the native API. This lets us accommodate nearly any specification error, and even flaws in Lily.

• We can then ask the operation to send the customized request, which will return a response that lazily deserializes the response body to the documented type. If we know the documented type is wrong, we can instead send the request with the native API and deserialize the InputStream however necessary, or not at all.

In other words, Lily is designed to facilitate HTTP interactions whenever possible, but fall back gracefully to the native java.net.http API in the presence of specification errors. Notably, all of these workarounds are forwards-compatible: Once the service owners update their OpenAPI specification to correct whatever errors were present, all of our code continues working. We can go back and update the code to use the generated API at our own pace.

Maven Dependency

To generate sources from an OAS document in your maven project, and the following maven build plugin and dependencies:

<build>
    <plugins>
        <plugin>
            <groupId>io.github.tomboyo.lily</groupId>
            <artifactId>lily-compiler-maven-plugin</artifactId>
            <version>${lilyVersion}</version>
            <configuration>
                <!-- Any URI to an OAS document, be it https:// or file://. -->
                <uri>https://raw.githubusercontent.com/OAI/OpenAPI-Specification/main/examples/v3.0/petstore.yaml</uri>

                <!-- Uncomment to customize the default generated sources directory. -->
                <!-- <outputDir>target/generated-sources</outputDir> -->

                <basePackage>com.exmaple.my.api</basePackage>
            </configuration>
            <executions>
                <execution>
                    <goals>
                        <goal>compile-client</goal>
                    </goals>
                    <phase>generate-sources</phase>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

 <dependencyManagement>
    <dependencies>
        <!-- BEGIN generated code dependency management -->
        <dependency>
            <groupId>com.fasterxml.jackson</groupId>
            <artifactId>jackson-bom</artifactId>
            <version>2.13.0</version>
            <scope>import</scope>
            <type>pom</type>
        </dependency>
        <!-- END generated code dependency management -->
    </dependencies>
</dependencyManagement>

<dependencies>
     <!-- BEGIN Generated code dependencies -->
    <dependency>
        <groupId>io.github.tomboyo.lily</groupId>
        <artifactId>lily-http</artifactId>
        <version>${lilyVersion}</version>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
    </dependency>
    <dependency>
        <!-- ZonedDatetime support -->
        <groupId>com.fasterxml.jackson.datatype</groupId>
        <artifactId>jackson-datatype-jsr310</artifactId>
    </dependency>
    <!-- END Generated code dependencies -->
</dependencies>

The generated source code relies on jackson and the lily-http library at runtime, which is why these dependencies are necessary.

These configurations can be stand-alone or embedded in a larger project.

Goals

1. Generate java source code directly from an OAS document within a java build pipeline (e.g. integrated with Maven or Gradle).

2. Support OAS v3.

3. Target Java 17+, with special attention paid to upcoming language features.

4. Help end-users work around incorrect or incomplete schema specifications so that they can make progress while awaiting upstream fixes.

5. Expose a high-level API to guide the user through API interactions.

6. Ensure that whenever possible, generated source code is compatible with user code between API specification revisions. In other words: "If I update to the latest API specification, and there are not breaking changes to the API, then Lily’s generated source code doesn’t break my application."

7. Support all OpenAPI features, including unusual things like matrix-style requests.

Non-Goals

1. Do not (yet) support other languages than Java. It’s not clear that a Java-oriented AST will cleanly translate to another language target.

2. Do not support too many options. Options become confusing to maintain — prefer opinionated code that works for most people who are doing sensible things.

Design

Lily is a layered API with "high-level" layers that orchestrate full requests using generated code and "low-level" layers that help the developer implement requests from scratch if necessary.

High-level layers always allow the developer to move into lower levels. This allows the developer to use the convenient high-level API as much as possible, then resort to the lower-level API (which could be the java.net.http API itself) only as necessary to work around missing features or undocumented API parameters.

Lily should make simple things easy, and complex things possible.

Quick Tour

Lily is composed of four modules in the modules directory:

• example compiles the v3.0 petstore YAML as an example. Check out the generated-sources directory after a build to see what Lily generates, and the test directory to see example usage of the generated code.

• lily-compiler-maven-plugin is a teensy-weensy Maven plugin that reads configuration from the pom and hands it off to the compiler project. This is what the user adds to their projects to compile code.

• lily-compiler is responsible for reading an OAS document, translating it to an intermediary AST (abstract syntax tree), rendering the AST as source code, and finally saving source code to disc.

• lily-http defines classes to help create and receive HTTP requests, including RFC6570 encoders, deser implementations, and the UriTemplate. This is a dependency of generated source code and may also be used directly by users to work around Lily or OAS shortcomings.

lily's Issues

Support #/components/parameters

Lily must read the `#/components/parameters' section of an OAS document and create source code from its elements.

Notes
Lily already evaluates anonymous query parameters. It currently skips any parameters which are not anonymous (i.e. it evaluates parameters with schemas and skips $refs). Part of the scope of this ticket is to ensure $refs are no longer merely ignored.

Add path parameter builders to operations

As a user, I can set path parameters using the Operations API so that I do not have to manually encode them and add them to the http request.

var httpRequest = Api.newBuilder.build()
  .allOperations()
  .getPetById()
  .id("1234")
  .include(List.of(NAME, AGE, BREED))
  .request();

myClient().send(request);

Generate responses

Given an ApiResponses specification like the following:

paths:
  /pets:
    get:
      operationId: listPets
      ...
      responses:
        '200':
          headers:
            x-next:
              schema:
                type: string
          content:
            application/json:    
              schema:
                $ref: "#/components/schemas/Pets"
                # Note: the usual schemas are allowed here, not just $refs.
        default:
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"

Generate a the following objects to represent the response:

• com.example.ListPetsOperation (this is already implemented and just used for context)
  • com.example.listpetsoperation/
    • ListPetsResponse :: ListPets200 | ListPetsDefault
    • ListPets200
    • ListPetsDefault
  • listpetsresponse/
    • ListPets200Content
    • ListPets200Headers
    • ListPetsDefaultContent
    • ListPetsDefaultHeaders

The ListPetsResponse, its members, and subordinate types need unique names because they are very likely to be used in a file alongside each other (e.g. a Dao) and it's the best option to prevent name conflicts. To keep the verbosity trade-off low, we'll omit the "Response" part of the name for response members and subordinate content/headers types, since (a) Response isn't necessary to differentiate their names and (b) the meaning of these types is still obvious based on the context.

ListPetsResponse is a sealed interface whose members include ListPets200 and ListPetsDefault. They are defined something like this:

public sealed interface ListPetsResponse permits ListPets200, ListPetsDefault {}

public record ListPets200Headers(...) {}
public record ListPets200Content(...) {}
public record ListPets200(ListPets200Headers headers, ListPets200Content) implements ListPetsResponse<> {}

// ... and so on

The *Headers types hold all header schemas named by the specification, and also a Map<String, String> to access the raw results (in case any headers are not named). The *Content types are the response bodies themselves.

The Response object should expose a getter for the original HttpResponse allowing access to any arbitrary information through the native API. While we could also expose the headers as a Map via the *Headers, there's not clearly a name we could use for the getter which won't hypothetically collide with a poorly-named header on the part of the OAS specification (We could prefix or suffix the name of all fields in *Headers which are declared in the schema to prevent any possible collision, but that's needlessly verbose compared to exposing the native response itself.)

Superficially, the interface looks like it should expose generic H headers() and C content() functions, but the underlying types will never have anything in common, so it's a stretch to understand how a generic function over these responses could work. Any such function would have to resort to using Map<String, String> instead (to manipulate headers, at least).

Prevent OneOf anonymous type name collisions

Given the following component schema, Lily will try to render multiple classes with the same name. Lily should, if possible, prevent such a name collision from happening.

FooStringEmailAlias:
  type: string
Foo:
  oneOf:
    - type: string
      format: email

Bar1:
  type: object
 Bar:
  oneOf:
    - type: object

In the Foo case, our naming strategy for anonymous alias types collides with a root-level component. In the Bar case, our naming strategy for anonymous objects collides with the root level component.

Contributing: Automatically test, enforce formatting for new PRs

As a contributor, when I open an PR in the Lily project, my contributions are automatically verified via mvn verify and mvn spotless:check. If my tests fail or my formatting is inconsistent, my PR will not be merged.

Requirements:

• All test cases in a PR must pass or else the PR may not be merged.
• Tests are run after each modification to a PR.
• A PR must contain formatted code according to mvn spotless:check or else the PR may not be merged.
• Formatting is checked after each modification to a PR.

Style=form, explode=true parameter encoding support (Default form and cookie parameter encoding support)

As a developer writing HTTP queries by hand, I can use a Lily API to encode parameters in style=form, explode=true style. For example:

var request = HttpRequest
    .newBuilder(URI.create("example.com/" + Encodings.formExplode(new MyQuery().text("Foo").number(3))))
    .header("Cookie", "myCookieName=" + Encodings.formExplode(new MyCooke().animal("cat").number(5))
    .build();
client.send(request, discarding());

where the query would encode as ?text=Foo&number=3, and the cookie as ?animal=cat&number=5 (so, myCookieName=?animal=cat&number=5).

Note that style=form, explode=true is the default style and explode for query and cookie parameters.

See RFC6570 for a definition of form-style query expansion.
See OAS3 Issue for why form query expansion is a strange default for cookies. Note that it is only applied to the value of cookies, not to the keys, which must be prepended manually. Most likely, the common user will simply set primitive values on their cookies.

Requirements:

• Support style=form, explode=true parameter encoding via a Encodings.formExplode(Object) API.

Evaluate path parameter schema

As a user, I want Lily to automatically generate classes for all PathItem parameters and Operation parameters so that I do not have to hand-write them myself.

Design

Parameters at the PathItem level ("inherited parameters") cannot be safely, repeatably, and uniquely identified by their path, so we will copy PathItem parameters down into their Operations. Operations will define one type per object-typed parameter schema in a subordinate package structure (if an operation's FQN is p.GetPetById, then it's parameters shall be generated in the p.getpetbyid package). Note that OperationIDs are globally unique (sort of [1]). This should generate uniquely qualified type names that closely align to the source OAS, though the user will need to use FQNs for like-named and inherited parameters. We've chosen to accept this as unavoidable without adding unnecessary complexity to Lily itself [2].

1. Operation IDs are, regrettably, case sensitive. This shouldn't be a real issue -- the OAS spec seems to discourage this -- but as a workaround, a lexical sort followed by suffixing a number should be stable between OAS definitions and eliminate duplicate package names without adding confusion.
2. We could use static classes or type name suffixes, but neither of these are "free" for the user experience or for Lily development. Ideally, Java one day adds import aliases and this problem resolves itself.

Style=simple explode=false parameter encoding support (Default path & header encoding support)

As a developer writing HTTP queries by hand, I can use a Lily API to encode parameters in style=simple, explode=false style. For example, I might encode a simple path and header parameters (without explosion) as follows:

var request = HttpRequest
    .newBuilder(URI.create("example.com/" + Encodings.simple(new MyPathParam().number(1).text(Foo))))
    .header("my-header", Encodings.simple(List.of("a", "b", "c"))
    .build();
client.send(request, discarding());

where the path parameter encodes to number,1,text,Foo and the header to a,b,c.

Note that style=simple, explode=false is the default for path and header parameters.

Requirements:

• Support style=simple, explode=false parameter encoding via a Encodings.simple(Object) API.

Parameters with equal "name" but different "in" cause compile errors

Bug:
Two parameters of an operation may share the same name but appear in different locations. If, for example, two such parameters are in the path and query, then Lily will try to generate an Operation with two like-named variables and setters, which fails as "variable foo is already defined."

If both parameters are object schemas, then the second parameter overwrites the first parameter's class definition because they share a fully-qualified name.

ToDo:

• Two parameters with the same "name" but different "in" locations is supported

Configure deployment action(s)

As a repository owner, I can perform a release which automatically runs the full test suite, assembles artifacts, and deploys them to maven central.

Fluently send requests

As a user, I can use Lily's generated api to execute http requests so that I do not need to access the underlying client or http request directly when not necessary:

var response = api
  .allOperations()
  .getPetById()
  .petId("123")
  .sendSync(); // <-- this
  //.sendAsync() // <-- or this

The native HTTP response must be accessible so that the user can access undocumented headers/cookies/bodies and work around shortcomings in Lily or other flaws in the OAS.

The response should facilitate access to documented response bodies, since deserializing a response is one of the more common things a user will need to do.

Header deserialization is deferred to #58

Document maven artifacts

Lily is published in maven central. Add documentation to the README which tells new users how to add artifacts to their projects.

Handle duplicate OneOf elements

If an OAS document erroneously lists the same element twice in a OneOf definition, we should ignore duplicate entries and render the OneOf successfully.

MyType:
  oneOf:
    - $ref: Foo
    - $ref: Foo #ignore me
    - type: string
    - type: string #ignore me too

Implement API to extract response headers (and cookies)

As a user, I can use the generated API to conveniently access documented headers and cookies according to their documented type:

// get the ms backoff from the http response, if any
Optional<Long> retryAfter = response.headers().retryAfter();

// get the cookie payload. E.g. `cookie: foo=1; bar=2` as an object.
MyCookie cookie = response.cookie();
cookie.foo() // 1

Questions:

• Does OAS use Lists to differentiate repeatable / non-repeatable headers?
• Is there a "required" equivalent for possible response headers?
• Can a declared header be missing?

Stable, unique names for anonymous OneOf schemas

Given a OneOf schema with anonymous (in-line) schemas in it like the following,

Foo:
  oneOf:
    - type: object
      properties:
        a: ...
    - type: object
      properties:
        b: ...
    

we generate objects named Foo1 and Foo2 in the same package as Foo (per sealed interface rules about packages -v- Java 9 modules). If the order of those two elements changes in a subsequent release of the OAS, then existing code will break: Lily will still generate objects named Foo1 and Foo2, but Foo1 will have property b, not a.

Consider implementing some kind of hash-like function which produces no collisions and is not dependent on element order so that we can create unique names for anonymous schemas that work between releases of the OAS and do not cause name collisions.

Generate AllOperations class for all operations

As a user, I can access any operation builder from the Api via a single method so that I can ignore tags when the API is simple, or when the API's use of tags is confusing.

Api.newBuilder().build()
  .allOperations()
  .getPetById();

Support #/components/responses

Lily must read the #/components/responses section of the OAS and generate re-usable response objects from it.

We are already able to generate responses from anonymous definitions in operations, which created sealed interfaces (MyResponse permits MyResponse200, MyResponse404, ...). However, that code doesn't account for $refs when creating the interfaces (that work is deferred to this ticket). Because responses are generated as a sealed interface, Lily needs to make $referenced responses from /components/responses members of each API response that uses them. So, for example, if GetFoo and BetBar operations rely on a shared Generic500Response schema, then Generic500Response's class must implement GetFooResponse and GetBarResponse.

Support #/components/requestBodies schema

Add UriTemplate support to API and operations classes

As a user, I can get a convenient URI template for each operation which helps me create the complete URI for any operation, like so:

var uri = Api.newBuilder()
  .uri("https://example.com:8080/")
  .build()
  .allOperations()
  .getPetById() // The GetPetById operation builder
  .uriTemplate()
  .put("id", "1234")
  .build(); // returns a URI instance of "https://example.com:8080/pets/1234"

// Use the templated URI to create an HTTP request. Send it using an HttpClient.
var request = HttpRequest.newBuilder(operation.request(), (k, v) -> true)
  .GET()
  .uri(uri)
  .build();

Non-requirements:

• The operation builder does not need to support http methods yet.

Where does this go next? Future iterations will add operation builders per parameter so that the user flow simplifies to ... .getPetById().id("1234").request();. That hypothetical workflow automatically uses a UriTemplate to create an HttpRequest object.

Project operation setters to avoid name clashes

The operation setters are currently derived from OAS parameter names, but this leaves us vulnerable to the small edge case where a parameter is named the same as a generated method like sendSync. To avoid conflicts, we should change the name of parameters to set* so that all OAS provided names occupy a "namespace" that our own methods can't conflict with.

This will change the API, so increment the major version.

Add type alias support for generated responses

Responses with primitive schemas ($ref, array, string, integer, and so on) must be generated to "type aliases" just like we do for #component schemas, since response types are members of sealed interfaces and we can't just add java.util.String to any old sealed interface.

Encoding::formExplode should not prepend '?'

The Encoding::formExplode encoder should not prepend the ? symbol, as this should be inserted by the client code building a URL. The ? prevents this encoder from being used in other contexts.

Generate application/json response schema to classes

Generate AST for response objects.

If a response object is a $ref to a component, ignore it (it will be generated along with all other components).

If a response object is instead an inline schema, generate AST for it. It should be named based on the response code and operation ID, since there is no meaningful alternative for in-line schemas. For example:

com.example.myoperationid.Response200
com.example.myoperationid.Response404
com.example.myoperationid.ResponseDefault

Improve & complete URL manipulation support via URITemplate

As a user, I can easily overwrite components of a URL, overwrite path parameters, and add and remove query parameters so that I can manually create request URLs whenever Lily or the OAS is missing necessary support.

The URITemplate needs to evolve into a fluent API to make adding and overwriting parts of a URL convenient. We don't want to force the user to URI.create(uriTemplate.toURI().toString() + "?foo=bar") or perform even more invasive string manipulation when they could do things like uriTemplate.setQueryParameter("foo", "bar") or uriTemplate.setPath("/pets/or/something") instead. Encoding should be performed automatically so the user doesn't forget a '?' and break their program.

The updated API might look like this:

// https://example.com/pets/{petId = 1234}
var uriTemplate = ...;

// Assume the following statements do not build on one another; each time, the uriTemplate begins as it is above this line. The text {petId = 1234} represents a parameter in the URL which is bound to a variable, and would render as just 1234.

uriTemplate.setPathParameter("petId", 789); // https://example.com/pets/{petId = 789} (null is invalid argument)
uriTemplate.setHost("github.com") // https://github.com/pets/{petId = 1234}
uriTemplate.setScheme("ftp")  // https://example.com/pets/{petId = 1234}
uriTemplate.setPort(8080) // // https://example.com:8080/pets/{petId = 1234}
uriTemplate.setPath("foo/bar/baz") // // https://example.com/foo/bar/baz (no petId anymore!)
uriTemplate.clearQuery() // remove the query string entirely
uriTemplate.setQuery(Map.of("include", List.of("a", "b'))) // // https://example.com/pets/{petId = 1234}?include=a&include=b (order of query parameters may be different)
uriTemplate.setQueryParameter("foo", 1234) // // https://example.com/pets/{petId = 1234}?foo=1234 (can override current parameter value) (null can unset the parameter)
uriTemplate.removeQueryParameter("foo") // would remove foo=1234 if it were part of the URL, and do nothing otherwise. Same as setQueryParameter("foo", null).

We should not support a way to set a raw query string ("?foo=bar&...") because it conflicts with setQueryParameter(String, Object): there is no one format that clients will use to encode array and object parameters, so we don't have an obvious way to decode their parameters to a Map internally. As a result, setQueryParameter would have to override their query, do nothing, or raise an exception. This doesn't seem like a valuable API given the client has more convenient ways to set parameters one at a time.

We wont support setTemplate(String) method that accepts the scheme://host:port/path/with/{parameters}/ URI template, even without a query parameter, because decoding even that promises to be tricky to do correctly. It would be a slightly more convenient API, but setting each part individually is simple and works.

The proposed API will let the user combine URL templating with builders that automatically encode parameters (such as to form explode for the query string).

Now, what about if the user needs to override encoding? For example, path parameters might use the matrix style instead of the default simple style, and not all OAS specifications are good about indicating their preference (and Lily only supports simple at time of writing). We could let the user set a custom encoder (such as an encodings.Encoding function reference), and we could also let the user set a "raw" (already encoded by them) value that that template should not try to encode. Those APIs might look like this:

uriTemplate.setRawPathParameter("color", ";color=blue,black,brown"); // https://example.com/;color=blue,black,brown
uriTemplate
  .setPathParameter("petId", List.of("blue", "black", "brown"))
  .setPathEncoder(Encoding::matrix) // https://example.com/;color=blue,black,brown

Query parameters do not, as far as I understand, ever get encoded differently than style=form, explode=true (?foo=bar&...). It's legal for clients to use ; instead of &, but that's only a requirement for servers to handle. That said, there is no standard for encoding complex data like lists. These are all valid strategies:

• ?array[]=foo&array[]=bar
• ?array=foo&array=bar
• ?array=foo,bar
  At some point, we could parameterize the form-explode encoder to allow different array and object encoding strategies, which would let the user evolve from using setRawQueryParameter to using a customized encoder.

Remove LinkedHashSet

We use LinkedHashSet throughout the AST to ensure set iteration order is consistent between runs. This is a relic of older test strategies that are on the way out and which relied upon consistent encounter order to pass.

In cases where the name of an anonymous type is dependent upon its position in a list, the ordinal comes from the underlying OAS list and is therefore consistent between runs regardless of Set implementation used thereafter.

Remove LinkedHashSet in favor of Set, which is easier to work with.

Implement OneOf schema generation

As a Lily user, when I generate a schema with a type that relies on the oneOf keyword, I expect Lily to create a class for that type.

Notes
A oneOf can contain anonymous schema as well as $refs to named schema.

Remove JacksonBodyHandler

This is an ill-advised implementation, and it's substantially better to use BodyHandlers.ofInputStream() followed by objectMapper.readValue(inputStream, clazz), since this decouples deserialization failures from the overall response. In other words, a 200 response can still be used even if the body is unreadable, and the user can handle deser errors within normal response handling flow.

Also, it's worth noting that when using ofInputStream, the HttpClient is able to return the response object a little more quickly, since it doesn't have to wait for all response content. The user can start consuming the response as soon as the status code and headers are available, before the remaining content is.

Handle identifiers in camel, snake, and kebab case.

As an OAS author, I can write identifiers (like operation IDs or parameter names) in kebab case ("foo-bar-baz"), camel case ("fooBarBaz"), and snake case ("foo_bar") and expect Lily-generated source code to convert those identifiers to well-formed class names ("FooBarBaz"), function names ("fooBarBaz"), and field names ("fooBarBaz") as appropriate.

(Currently, our names only support camel case.)

Note that things like parameter names may need to serialize to a different string representation. The object { foo_bar: 123 } would deserialize to an object with field fooBar, but needs to serialize back to snake case.

Non Goals

I don't think we should try to handle adversarial input like "foo-barBaz" or "fooBaRbAz." At that point, we'd give the user other means to map adversarial inputs to clean strings.

Notes

We should introduce an abstraction that allows us to defer name formatting decisions (like lower case and upper case) unambiguously and completely to the AstToJava class. It should allow the ICG layer (and tests) to completely ignore capitalization or other formatting decisions.

Some names have packages, others do not. For example, field names never have or need a package, whereas ast references always do. We may be able to support both use-cases with the same abstraction, or we can use different abstractions based on what the code needs.

Something like this, perhaps:

var fqn = Fqn.withPackage("com.example", "foo.bar").withName("ShowPetByIdOperation");
fqn.fullyQualified().upperCamelCase(); // com.example.foo.bar.ShowPetByIdOperation
fqn.shortName().lowerCamelCase(); // showPetByIdOperation

assertThat(fqn, equalTo(Fqn.withPackage("com.example.foo", "bar").withName("ShowPetByIdOperation"));

var shortOnly = Fqn.withName("ShowPetByIdOperation");
fqn.fullyQualified() // exception
fqn.shortName().upperCamelCase(); // ShowPetByIdOperation
fqn.shortName().lowerCamelCase(); // showPetByIdOperation

Add path parameter support to Operation builders

As a user of the Lily generated API, I can use builder methods to set path parameters so that I do not have to manually encode and interpolate those parameters into URIs myself.

Instead of this:

var template = api.petsOperations().getPetById().request();
var request = HttpRequest.newBuilder(request, (k, v) -> true)
    .uri("http://localhost:8080/pets/" + Encoding.simple(petId))
    .build();

I can do this:

var request = api.petsOperations().getPetById()
    .setId(petId)
    .request();

Notes

The user must be able to set a default or base URL when instantiating the API for the first time, such as Api.newBuilder().uri("http://localhost:8080/"). The API can pass this base URL to Operations so that they can resolve subpaths against it, like so:

builder.uri(baseUrl.resolve("pets/" + Encodings.simple(petId)));

Parameters are defined by Path Items as well as by Operations. Parameters are unique by name and location. Parameters defined at the Path Item label may be overridden by parameters at the operation level. Parameter schema may be in-line or use $refs.

Support header parameters

As a user, I can configure header parameters using the API:

var response = myOperation
  .headers(headers -> headers.foo("foo").bar("bar"))
  .sendSync()

Address 'other' and 'all' tag groups conflicts with OAS tags of same name

Expected: If I generate code for an OpenAPI document which uses the string "other" or "all" as tag names, then operations which do not have those tags are not part of the otherOperations or allOperatons groups in the API.

Actual: Untagged operations are always added to the otherOperations group, and all operations are added to the allOperations group, because these tag names are used by default respectively to organize operations without tags and to organize all operations together.

Notes
We could make the 'other' and 'all' strings configurable, but it may be better to make 'other' and 'all' special cases which do not evaluate to tagged operations groups the normal way. For example, if untagged operations are exposed via an operationsWithoutTags() API, then there does not exist a tag name which will map to that same string and collide.

Use ByteBuffer instead of Byte[] for binary types

Prefer ByteBuffer to Byte[] for the superior and generally more desirable API of ByteBuffer.

This has the additional ramification that we no longer represent array types in the AST, so we can remove a small amount of special-case handling around them and hopefully refactor the AST as well.

Graceful failure - Candlepin API

As a user, I want Lily to generate API support for as much of an API as possible and simply skip past components of the schema that are malformed or not supported. This ensures I get as much benefit from Lily as possible while waiting for schema or Lily improvements.

This will be scoped to the Candlepin API specification.

support #/components/pathItems

Add validation to Encodings functions

Validate inputs to Encodings function to ensure that we do not silently create an erroneous encoding for bad input, forcing the user to track down a subtle bug in our code.

• formExplode must only accept objects and maps since it only meaningfully encodes key-value pairs.
• formExplode must refuse to encode nested objects, since this is not part of the formExplode specification.
  • formExplode may encode nested lists.
• simple must refuse to encode nested objects, since this is not part of the specification
  • simple may encode top-level objects
• simple must refuse to encode nested lists, since this is not part of the specification
  • simple may encode top-level lists

Generate sealed interfaces for enumerated response schema bodies

When I deserialize a response which takes the form of one of the enumerated response schemas, I can use a switch expression to match the specific response so that I do not have to manually check the status code and deserialize the response myself.

This:

var response = api.allOperations()
  .getFoo()
  .send(); // returns a Lily response object (not defined at time of writing)

// using preview switch expression patterns (j17 preview feature):
var result = switch (response.body()) {
   GetFoo200 ok -> ... ;
   GetFoo201 ok -> ... ;
   GetFoo404 error -> ... ;
}

// Using if-else pattern matching:
if (response.body() instanceof GetFoo200 ok) {
  // handle happy path
} else if (response.body() instanceof GetFoo201 ok) {
  // handle accepted response
} else if (response.body() instanceof GetFoo404 error) {
  // handle missing record
}

Not this:

var response = api.allOperations()
  .getFoo()
  .send();

if (response.statusCode() == 200) {
  var body = response.getBody(GetFoo200.class);
  // ...
} else if (response.statusCode() == 201) {
  // ...
} else if (response.statusCode() == 404) {
  // ...
} else {
  // ...
}

Using a sealed interface has the advantage that if I use pattern matching switch expressions (now in preview mode or in a future release), the compiler can exhaustively check that I've handled each enumerated response type. My code is more likely to be correct as a result, and my IDE will be able to suggest missing switch arms.

Without switch expressions, it's a convenience for deserializing the response object to the correct class. It's not likely to affect code quality but may positively affect the user experience.

Note that above, the interface of the response object (from #send()) is just an example.

Also, some responses may use component schemas (named schemas used throughout the document for request and response schemas). So, a response type might end up being the union of types like Pet | GetPetById404 | GetPetById201 where some responses are named by their response code (these are essentially "anonymous" responses; responses without a name.)

Adopt code formatter that understands java17 multi-line strings

Configure the current code formatter or adopt a new one such that our Java 17 multi-line strings are formatted neatly. Currently, they get randomly indented.

If a new formatter is adopted, then we must update the pipeline actions that verify the repository is consistently formatted.

Builders for all models

Request body objects can contain optional fields, and will acquire optional fields over time as an API evolves. Developers who have written code against an earlier version of an API should not have to "update" that code to explicitly set a new and optional field to null when they update to a newer version of the API.

The only solution I am aware of in Java is to create builders:

MyRequestBody.newBuilder()
  .foo("foo!")
  .build();

The user might construct response objects or any other type in their test suite (or maybe even in server code, if they use this to generate their model), and we wouldn't want that to break with a version upgrade, either. So we pretty much ought to add builders everywhere.

Migrate generated component test coverage to lily-compiler

Potential name collision between components and operations (and others)

Actual:
Suppose a schema contained a component called MyOperation, and an operation with ID My. Then Lily would generate base.package.MyOperation from the component as well as base.package.MyOperation from the operation, causing undefined behavior.

Expected:
All types should be given distinct FQNs in all scenarios. For example, Lily could generate base.package.component.schema.MyOperation, and a base.package.operation.MyOperation, though there are many other solutions.

Notes
Components are generated to the base package, so they can collide with Api, operation groups (including EveryOperation and EveryUntaggedOperation), and anything else also generated directly to the root.

Expose underlying HttpClient

As a user, I can retrieve the HttpClient from the api so that I can resort to low-level customization at any time without modifying my code structure (such as by adding constructor parameters or public static singletons).

var client = api.client();

This enables "one line" requests with custom body handlers or lightly-modified requests:

api.client().send(
  api.allOperations().someOperation().httpRequest(),
  myBodyHandler);

Reduce maven logging in CI

Reduce maven logging output down to just warnings, errors, and phase/goal output if possible. At the very least, prevent maven from logging download progress for every dependency.

Generic error resolution schema

There are a variety of circumstances we may or may not foresee in which we can't generate code, such as when the specification causes name conflicts. As a user, I do not want to modify the OAS locally because I'd be responsible for complex three-way merges with the source of truth, but if I could write "rules" in a file that override the OAS in order to address specification flaws, that would let me work around issues without compromising the stability/repeatability of my builds.

Consider how such a framework could work, and what categories of problems it could solve. Can Lily emit exceptions that describe the issue and instruct the User how to fix them?

This might replace #86, #90, and #91.

Gracefully handle duplicate schema elements

As a user, when I try to compile an OAS document that has duplicate elements due to author error, Lily should warn me about the error (and tell me which generated types are impacted), but continue generating classes so that I can work with as much of the OAS that is well-formed as possible rather than be completely blocked.

For example:

openapi: 3.0.2
  components:
    schemas:
      Foo: ...
      Foo: ... # A duplicate component

This should also be the case when Lily decides to name an anonymous type in such a way that it collides with another type. Ideally this is logged at a more severe level than usual since it's a design flaw on our part, if possible.

Add query parameter support to operation APIs

As a user, I can use the fluent builder operation API to set parameters which appear in the query string of requests.

api
  .getPetByIdOperation()
  .id(5)
  .include("name", "age")
  .otherParam("foo")
  .uriTemplate()
  .toUri();
  // https://example.com/pets/5?include=name&include=age&otherParam=foo

Challenges

The first query parameter which is set must be encoded using Encoders.form, whereas any others must be encoded using formContinuation. This could become tricky when the user needs to access the UriTemplate, but that workflow is intentionally coupled to implementation details, and isn't a long-term strategy, so it should be tolerable, at least for now.

support #/components/headers

Allow separate packages for model and api classes

Allow the user to configure separate package names for model and api types, like com.example.api and com.example.model, so that it is easier to explore generated source code.

Add HTTP method to templated HttpRequests

As a user, when I retrieve an HttpRequest from the Lily fluent API, that request is already parameterized with the http request method from the OpenAPI specification so that I do not have to set it myself every time.

Instead of this:

var template = api.allOperations().someGetRequest().httpRequest();
var request = HttpRequest.newBuilder(template, (x, y) -> true).GET().build();
var response = client.send(request, myBodyHandler);

I can do this:

var response = client.send(
  api.allOperations().someGetRequest().httpRequest(),
  myBodyHandler);

Support body parameters

As a user, I can configure request bodies using the API:

var response = myOperation
  .body(new MyRequestBody(...))
  .sendSync();

Fluently create HttpRequests

As a user, I can use Lily to create parameterized HttpRequests so that I can send them with a HttpClient manually, such as when I need to override some Request parameters first, or when I need to provide my own BodyHandler for unsupported response media types.

or, equivalently:

var gzipPayload = api.client().send(
  api.allOperations()
      .getCompressedBlobById()
      .id(1234)
      .httpRequest(),
  BodyHandlers.ofByteArray());

Note that the API does not provide "helper" methods like sendWithCustomBodyHandler(...) because this only serves as an at-best-incomplete proxy to the underlying API. Instead, we simply yield control back to the user by giving them all the objects we have created so far. The user can therefore do anything that they could otherwise do with a request and client pair using the native API. This is the most flexible option for users and requires the least maintenance from us.

Requirements:

• Operations expose a httpRequest method that returns an HttpRequest.
• The http request is already parameterized with a URL.
• The request need not be parameterized in any other way yet.