io7m-com / cedarbridge Goto Github PK

View Code? Open in Web Editor NEW

0.0 3.0 0.0 3.84 MB

Cedarbridge message protocol specification language

Home Page: https://www.io7m.com/software/cedarbridge

License: ISC License

Java 99.06% CSS 0.84% Shell 0.02% Smalltalk 0.08%

message-protocol specification-language cedarbridge

cedarbridge's People

Contributors

Watchers

cedarbridge's Issues

Namespace CSS files

Currently, the XHTML says:

<link href="reset.css" rel="stylesheet" type="text/css"/>
<link href="document.css" rel="stylesheet" type="text/css"/>

It should say something more like:

<link href="cedarbridge-reset.css" rel="stylesheet" type="text/css"/>
<link href="cedarbridge-document.css" rel="stylesheet" type="text/css"/>
<link href="cedarbridge-custom.css" rel="stylesheet" type="text/css"/>

Where cedarbridge-custom is only present if a custom CSS option is specified.

The compiler is written in a manner that should make it easy to migrate to sealed classes when those get into an LTS JDK (likely JDK 17). Upgrade all pseudo-algebraic types in the compiler to sealed classes, and upgrade the code generator to generate sealed classes.

Variant case fields aren't scoped correctly

[documentation Ic1PermissionScoped "A scoped permission."]
[variant Ic1PermissionScoped
  [documentation Global "A globally scoped permission."]
  [case Global
    [documentation permission "The permission."]
    [field permission Ic1Permission]
  ]
  [documentation Projectwide "A project-scoped permission."]
  [case Projectwide
    [documentation project "The project ID."]
    [field project cb:IntegerUnsigned64]
    [documentation permission "The permission."]
    [field permission Ic1Permission]
  ]
  [documentation Ticketwide "A ticket-scoped permission."]
  [case Ticketwide
    [documentation ticket "The ticket ID."]
    [field ticket Ic1TicketID]
    [documentation permission "The permission."]
    [field permission Ic1Permission]
  ]
]

ERROR: file:///home/rm/git/com.github/io7m/icatiro/com.io7m.icatiro.protocol.tickets.cb/src/main/resources/com/io7m/icatiro/protocol/tickets/cb/Tickets.cbs:97:12: Name "permission" has already been defined.
  Specification: https://www.io7m.com/software/cedarbridge/specification/index.xhtml#id_1cf38aec-7544-4b5e-a5ac-bb01567ffe77
    [field permission Ic1Permission]]
           ^
  See file:///home/rm/git/com.github/io7m/icatiro/com.io7m.icatiro.protocol.tickets.cb/src/main/resources/com/io7m/icatiro/protocol/tickets/cb/Tickets.cbs:90:12:
    [field permission Ic1Permission]]
           ^
ERROR: file:///home/rm/git/com.github/io7m/icatiro/com.io7m.icatiro.protocol.tickets.cb/src/main/resources/com/io7m/icatiro/protocol/tickets/cb/Tickets.cbs:104:12: Name "permission" has already been defined.
  Specification: https://www.io7m.com/software/cedarbridge/specification/index.xhtml#id_1cf38aec-7544-4b5e-a5ac-bb01567ffe77
    [field permission Ic1Permission]]
           ^
  See file:///home/rm/git/com.github/io7m/icatiro/com.io7m.icatiro.protocol.tickets.cb/src/main/resources/com/io7m/icatiro/protocol/tickets/cb/Tickets.cbs:90:12:
    [field permission Ic1Permission]]

Add a standard URI type

Similar to the UUID type.

Better context info for runtime serialization errors

Recently, the code was switched to allow the context to raise exceptions rather than generated code raising them itself. The context has all kinds of information (a complete stack of identifiers that lead to the current location, and the current byte offset), but none of that information is included in any raised exceptions.

Empty records are rendered incorrectly in bridgedoc

An empty record yields documentation like this:

<h4>Fields</h4>
 <table class="cbFieldsTable">
    <thead>
       <tr>
          <th>Name</th>
          <th>Type</th>
          <th>Description</th>
       </tr>
    </thead>
    <tbody/>
 </table>

It should yield a simple paragraph that says "there are no fields" the same way variant cases are rendered.

Documentation annotations

Add a documentation expression that targets an element for documentation:

(record T
  [field x IntegerUnsigned32]
  [documentation x "This is documentation for the field x"]
  [field y IntegerUnsigned32])

(documentation T "This is documentation for the type T.")

The specification doesn't say how protocols are encoded

It describes how records and variants are encoded, but not the versions of protocols.

Add standard time and duration types

The standard Duration type can be represented as:

[record Duration
  [field seconds cb:Unsigned64]
  [field nanos   cb:Unsigned32]
]

The nanos field is actually defined as never exceeding 999999999.

Type applications with no arguments should not be accepted

An expression such as this:

[IntegerUnsigned32]

... shouldn't be accepted. The actual expression should be rejected as a type error, as it's a function application applied to a type constructor that takes no arguments.

Migrate to quarrel

Replace claypot and jcommander with quarrel.

Generated variant serializer doesn't serialize index values

[variant IdA1AdminPermission
  [documentation AdminBan "A permission that allows for banning admins."]
  [case AdminBan]
  [documentation AdminCreate "A permission that allows for creating admins."]
  [case AdminCreate]
  [documentation AdminDelete "A permission that allows for deleting admins."]
  [case AdminDelete]
  [documentation AdminWrite "A permission that allows updating admins."]
  [case AdminWrite]
  [documentation AdminWriteSelf "A permission that allows an admin to update itself."]
  [case AdminWriteSelf]
  [documentation AdminRead "A permission that allows reading admins."]
  [case AdminRead]
  [documentation AuditRead "A permission that allows reading the audit log."]
  [case AuditRead]
  [documentation UserDelete "A permission that allows deleting users."]
  [case UserDelete]
  [documentation UserCreate "A permission that allows creating users."]
  [case UserCreate]
  [documentation UserWrite "A permission that allows updating users."]
  [case UserWrite]
  [documentation UserRead "A permission that allows reading users."]
  [case UserRead]
  [documentation UserBan "A permission that allows for banning users."]
  [case UserBan]
]

This yields a serialize method that looks like this:

  @Override
  public void serialize(final CBSerializationContextType context, final IdA1AdminPermission value)
      throws IOException {
    if (value instanceof IdA1AdminPermission.AdminBan) {
      serializeAdminBan(context, (IdA1AdminPermission.AdminBan) value);
      return;
    }
    if (value instanceof IdA1AdminPermission.AdminCreate) {
      serializeAdminCreate(context, (IdA1AdminPermission.AdminCreate) value);
      return;
    }
    if (value instanceof IdA1AdminPermission.AdminDelete) {
      serializeAdminDelete(context, (IdA1AdminPermission.AdminDelete) value);
      return;
    }
    if (value instanceof IdA1AdminPermission.AdminWrite) {
      serializeAdminWrite(context, (IdA1AdminPermission.AdminWrite) value);
      return;
    }
    if (value instanceof IdA1AdminPermission.AdminWriteSelf) {
      serializeAdminWriteSelf(context, (IdA1AdminPermission.AdminWriteSelf) value);
      return;
    }
    if (value instanceof IdA1AdminPermission.AdminRead) {
      serializeAdminRead(context, (IdA1AdminPermission.AdminRead) value);
      return;
    }
    if (value instanceof IdA1AdminPermission.AuditRead) {
      serializeAuditRead(context, (IdA1AdminPermission.AuditRead) value);
      return;
    }
    if (value instanceof IdA1AdminPermission.UserDelete) {
      serializeUserDelete(context, (IdA1AdminPermission.UserDelete) value);
      return;
    }
    if (value instanceof IdA1AdminPermission.UserCreate) {
      serializeUserCreate(context, (IdA1AdminPermission.UserCreate) value);
      return;
    }
    if (value instanceof IdA1AdminPermission.UserWrite) {
      serializeUserWrite(context, (IdA1AdminPermission.UserWrite) value);
      return;
    }
    if (value instanceof IdA1AdminPermission.UserRead) {
      serializeUserRead(context, (IdA1AdminPermission.UserRead) value);
      return;
    }
    if (value instanceof IdA1AdminPermission.UserBan) {
      serializeUserBan(context, (IdA1AdminPermission.UserBan) value);
      return;
    }
    throw new IllegalArgumentException(String.format("Unrecognized variant case class: %s",value.getClass()));
  }

Note the lack of calls to context.writeVariantIndex().

Expression sources off-by-one

Creating a record with an invalid field yields this error:

ERROR: file:///home/rm/git/com.github/io7m/idstore/com.io7m.idstore.protocol.user.cb1/src/main/resources/com/io7m/idstore/protocol/user/cb1/User1.cbs:196:3: Unrecognized record member declaration.
  [field requestId IdU1UUID]]
  ^
  Object:        Record member
  Specification: https://www.io7m.com/software/cedarbridge/specification/index.xhtml#id_b43940c3-038f-4330-971f-ac76d56d5fad
  Expected:
    (parameter <type-parameter-name>)
  | (field <field-name> <type-expression>)
  | (documentation <name> <quoted-string>)

The problem is the [field requestId IdU1UUID]] line; although the line and column numbers are correct in the error message, the actual line displayed is the line after the line where the actual error occurred. Somewhere, an off-by-one error has snuck in and the expression source is being given the wrong value.

Consider an encoding profile system

Currently, CCBE is the only defined encoding for Cedarbridge. Although it is an efficient encoding in that it transfers absolutely no type information whatsoever, it does have a bit of waste in the sense that string and sequence lengths are always 32-bit integers, and variant indices are also always 32-bit integers. The utter simplicity is a desirable property and should be preserved and left as the default. However, this doesn't mean that it shouldn't be possible to introduce new alternative encodings if two peers can communicate this encoding out-of-band.

For many protocols, 16-bit integers might be enough for strings and sequences, and 8-bit integers might be enough for variant indices.

However, the Cedarbridge compiler has no way to know which encoding is going to be used and so it can't perform checks that might be useful. For example, if there's a hypothetical mini8 encoding that encodes all lengths and indices as 8-bit integers, then the compiler should check that variants and protocols don't contain more than 255 cases/messages. This could be achieved with an encoding declaration:

[encoding mini8]

... Where the default if no encoding is specified is:

[encoding canonical]

Document container protocol

Document the container protocol for version negotiation.

Add a boolean type

[variant Boolean
  [case True]
  [case False]]

Add a Maven plugin

Using com.io7m.cedarbridge.cmdline in combination with the exec-maven-plugin brings in the possibility of classpath pollution, and doesn't play nicely if there are XML dependencies present (for example, xercesImpl conflicts with the bridgedoc generator).

Make UUIDs a standard type

Currently, every project using Cedarbridge has ended up with a type like this:

[documentation CAI1UUID "A 128-bit UUID value."]
[record CAI1UUID
  [documentation msb "The most significant bits of the UUID."]
  [field msb cb:IntegerUnsigned64]
  [documentation lsb "The least significant bits of the UUID."]
  [field lsb cb:IntegerUnsigned64]
]

Limit variants and protocols to 255 cases and 255 types

This has a number of benefits:

Variant indices can be encoded into a single unsigned byte. This provides a lot of space improvements when serializing sequences of variant types without increasing decoding or encoding complexity.

Code generation for protocol versions currently passes the serializers for all types in a version into the constructor of the protocol version serializer. This means there's a limit of 255 possible serializers that can be passed in before hitting the Java method parameter limit.

Because cedarbridge supports variants, limiting protocols to 255 messages wouldn't harm expressivity; if someone really wants to define a protocol with 255 message types, they can group those messages into an infinite number of enclosing variants and have as many protocol messages as they want - they just can't all be at the "top level" of the protocol.

Minor version statement is too strong

If a language statement names an unrecognized language minor version, parsing of the compilation unit MUST be aborted, and all subsequent statements in the compilation unit ignored.

The point of the minor version is so that compatible changes can be introduced. The above statement should be reworded such that an unrecognized minor version is allowed to produce a warning, but otherwise processing can continue.

Reproducible Builds issue with sealed interfaces bytecode

reference: https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/content/com/io7m/cedarbridge/README.md

public sealed interface CBTypeExpressionType extends Formattable source code gives unpredictable order in bytecode

├── com/io7m/cedarbridge/schema/compiled/CBTypeExpressionType.class
│ ├── procyon -ec {}
│ │ @@ -1,9 +1,9 @@
│ │  
│ │  package com.io7m.cedarbridge.schema.compiled;
│ │  
│ │  import java.util.Formattable;
│ │  
│ │ -public sealed interface CBTypeExpressionType extends Formattable permits CBTypeExpressionType$CBTypeExprParameterType, CBTypeExpressionType$CBTypeExprNamedType, CBTypeExpressionType$CBTypeExprApplicationType
│ │ +public sealed interface CBTypeExpressionType extends Formattable permits CBTypeExpressionType$CBTypeExprNamedType, CBTypeExpressionType$CBTypeExprParameterType, CBTypeExpressionType$CBTypeExprApplicationType
│ │  {
│ │      boolean contains(final CBTypeParameterType p0);
│ │  }

I'm reporting here because your project is the first one I see that is affected, but I suppose you can' solve anything at your level.
The root cause is probably in the JDK compiler: I'll need to open a JDK issue

Enforce that versions in a protocol can be described by a half-open range

Versions must be contiguous, with no gaps.

Document canonical binary encoding

Document the canonical binary encoding for serialized values.

Incorrect type parameters generated

The current idstore codebase gets incorrect type parameters (no parameters!) generated for the following Page type:

[record IdA1Page
  [parameter T]
  [field items [cb:List T]]
  [field pageIndex cb:IntegerUnsigned32]
  [field pageCount cb:IntegerUnsigned32]
  [field pageFirstOffset cb:IntegerUnsigned64]]

Make protocol UUIDs more visible

They're not exposed in documentation.

Might also be worth allowing them to be manually specified for protocols.

Security policy

The maximum length of byte arrays and strings should be limited, because there's no sensible thing for a decoder to do other than read the maximum length, allocate a buffer of that length, and then read data into it. It only takes one stream that contains a large integer to allocate gigabytes of heap.

Generate protocol UUIDs

We can infer protocol UUIDs from the fully qualified name of a protocol, and this UUID can be used in the container protocol. There should ideally also be a way to specify the UUIDs manually.

Implement serializers for protocols

The interface types have been completed, but the code to actually read/write protocols doesn't exist yet.

Implement serialization context

The serialization context is an implementation of the serializer context API. This should be built on top of bssio.

Names and types in field tables should be monospace in bridgedoc

Currently, they're rendered in the default font.

Upgrade to records

The code generator and much of the compiler code was written to make migration to Java record classes easy when records make it into an LTS JDK (likely JDK 17 at this point). Do the migration!

Redesign how protocols are declared

Right now, creating a new version of a protocol involves re-declaring all of the messages again:

[record A]
[record B]
[record C]
[record D]

[protocol D
  [version 1 A B C]
  [version 2 A B C D]
  [version 3 B C D]
]

When the number of messages is large, this is unwieldy. It might be better to introduce the concept of protocol expressions that are evaluated to yield a final protocol. Nothing needs to change with regards to type checking, code generation, or otherwise.

[record A]
[record B]
[record C]
[record D]

[protocol D
  [version 1
    [introduce A]
    [introduce B]
    [introduce C]
  ]
  [version 2
    [introduce D]
  ]
  [version 3
    [remove A]
  ]
]

Limit and deduplicate type references to 255 types

Currently, code generation fails if a type references more than 255 types. Worse, type references are not deduplicated - a record type with 255 integer-typed fields results in code generation that refers to 255 integer serializers.

Type references should be deduplicated for code generation, and limits should be put in place in the compiler to reject type declarations that reference more than 255 deduplicated types.

Misleading parse error

The attached file yields the following error:

ERROR: Admin1.cbs:218:1: Invalid S expression.
  Specification: https://www.io7m.com/software/cedarbridge/specification/index.xhtml#id_d1cd60f0-8880-4754-964f-5b0489947c59
]

Admin1.txt

Nested protocols

Because protocols are limited to 255 messages, it would be helpful if protocols could be nested (strawman syntax):

[protocol P
  [version 1
    [types-added A B C]]]

[protocol Q
  [version 1
    [types-added D E F]]]

[protocol R
  [version 1
    [protocol P 1]
    [protocol Q 1]]]

Validator boilerplate generation

Currently, writing validation code to convert from versioned Cedarbridge types to application types involves a ton of boilerplate. Being able to pattern match will reduce some of this, but some of it will remain. The language knows all of the types that are going to need to be converted, so it could generate some empty classes filled with static methods that need to be filled in.

Javadoc doesn't validate

The generator generates code that doesn't pass JavaDoc validation:

/**
 * A serializer for color values.
 */

public final class IdColorSerializer
  extends StdSerializer<IdColor>
{
  /**
   * A serializer for color values.
   *
   * @inheritDoc
   */

  public IdColorSerializer()
  {
    this(null);
  }


  /**
   * A serializer for color values.
   *
   * @inheritDoc
   *
   * @param t The serialized class
   */

  public IdColorSerializer(
    final Class<IdColor> t)
  {
    super(t);
  }

Generating /home/build/git/com.github/io7m/idstore/com.io7m.idstore.documentation/target/documentation/apidocs/com/io7m/idstore/colors/IdColorSerializer.html...
/home/build/git/com.github/io7m/idstore/com.io7m.idstore.documentation/target/javadoc-sources/com/io7m/idstore/colors/IdColorSerializer.java:36: error: incorrect use of inline tag
   * @inheritDoc
     ^
/home/build/git/com.github/io7m/idstore/com.io7m.idstore.documentation/target/javadoc-sources/com/io7m/idstore/colors/IdColorSerializer.java:48: error: incorrect use of inline tag
   * @inheritDoc
     ^

io7m-com / cedarbridge Goto Github PK

cedarbridge's People

Contributors

Watchers

cedarbridge's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs