io7m-com / cedarbridge Goto Github PK
View Code? Open in Web Editor NEWCedarbridge message protocol specification language
Home Page: https://www.io7m.com/software/cedarbridge
License: ISC License
Cedarbridge message protocol specification language
Home Page: https://www.io7m.com/software/cedarbridge
License: ISC License
Currently, the XHTML says:
<link href="reset.css" rel="stylesheet" type="text/css"/>
<link href="document.css" rel="stylesheet" type="text/css"/>
It should say something more like:
<link href="cedarbridge-reset.css" rel="stylesheet" type="text/css"/>
<link href="cedarbridge-document.css" rel="stylesheet" type="text/css"/>
<link href="cedarbridge-custom.css" rel="stylesheet" type="text/css"/>
Where cedarbridge-custom
is only present if a custom CSS option is specified.
The compiler is written in a manner that should make it easy to migrate to sealed classes when those get into an LTS JDK (likely JDK 17). Upgrade all pseudo-algebraic types in the compiler to sealed classes, and upgrade the code generator to generate sealed classes.
[documentation Ic1PermissionScoped "A scoped permission."]
[variant Ic1PermissionScoped
[documentation Global "A globally scoped permission."]
[case Global
[documentation permission "The permission."]
[field permission Ic1Permission]
]
[documentation Projectwide "A project-scoped permission."]
[case Projectwide
[documentation project "The project ID."]
[field project cb:IntegerUnsigned64]
[documentation permission "The permission."]
[field permission Ic1Permission]
]
[documentation Ticketwide "A ticket-scoped permission."]
[case Ticketwide
[documentation ticket "The ticket ID."]
[field ticket Ic1TicketID]
[documentation permission "The permission."]
[field permission Ic1Permission]
]
]
ERROR: file:///home/rm/git/com.github/io7m/icatiro/com.io7m.icatiro.protocol.tickets.cb/src/main/resources/com/io7m/icatiro/protocol/tickets/cb/Tickets.cbs:97:12: Name "permission" has already been defined.
Specification: https://www.io7m.com/software/cedarbridge/specification/index.xhtml#id_1cf38aec-7544-4b5e-a5ac-bb01567ffe77
[field permission Ic1Permission]]
^
See file:///home/rm/git/com.github/io7m/icatiro/com.io7m.icatiro.protocol.tickets.cb/src/main/resources/com/io7m/icatiro/protocol/tickets/cb/Tickets.cbs:90:12:
[field permission Ic1Permission]]
^
ERROR: file:///home/rm/git/com.github/io7m/icatiro/com.io7m.icatiro.protocol.tickets.cb/src/main/resources/com/io7m/icatiro/protocol/tickets/cb/Tickets.cbs:104:12: Name "permission" has already been defined.
Specification: https://www.io7m.com/software/cedarbridge/specification/index.xhtml#id_1cf38aec-7544-4b5e-a5ac-bb01567ffe77
[field permission Ic1Permission]]
^
See file:///home/rm/git/com.github/io7m/icatiro/com.io7m.icatiro.protocol.tickets.cb/src/main/resources/com/io7m/icatiro/protocol/tickets/cb/Tickets.cbs:90:12:
[field permission Ic1Permission]]
Similar to the UUID type.
Recently, the code was switched to allow the context to raise exceptions rather than generated code raising them itself. The context has all kinds of information (a complete stack of identifiers that lead to the current location, and the current byte offset), but none of that information is included in any raised exceptions.
An empty record yields documentation like this:
<h4>Fields</h4>
<table class="cbFieldsTable">
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Description</th>
</tr>
</thead>
<tbody/>
</table>
It should yield a simple paragraph that says "there are no fields" the same way variant cases are rendered.
Add a documentation
expression that targets an element for documentation:
(record T
[field x IntegerUnsigned32]
[documentation x "This is documentation for the field x"]
[field y IntegerUnsigned32])
(documentation T "This is documentation for the type T.")
It describes how records and variants are encoded, but not the versions of protocols.
The standard Duration
type can be represented as:
[record Duration
[field seconds cb:Unsigned64]
[field nanos cb:Unsigned32]
]
The nanos
field is actually defined as never exceeding 999999999
.
An expression such as this:
[IntegerUnsigned32]
... shouldn't be accepted. The actual expression should be rejected as a type error, as it's a function application applied to a type constructor that takes no arguments.
Replace claypot and jcommander with quarrel.
[variant IdA1AdminPermission
[documentation AdminBan "A permission that allows for banning admins."]
[case AdminBan]
[documentation AdminCreate "A permission that allows for creating admins."]
[case AdminCreate]
[documentation AdminDelete "A permission that allows for deleting admins."]
[case AdminDelete]
[documentation AdminWrite "A permission that allows updating admins."]
[case AdminWrite]
[documentation AdminWriteSelf "A permission that allows an admin to update itself."]
[case AdminWriteSelf]
[documentation AdminRead "A permission that allows reading admins."]
[case AdminRead]
[documentation AuditRead "A permission that allows reading the audit log."]
[case AuditRead]
[documentation UserDelete "A permission that allows deleting users."]
[case UserDelete]
[documentation UserCreate "A permission that allows creating users."]
[case UserCreate]
[documentation UserWrite "A permission that allows updating users."]
[case UserWrite]
[documentation UserRead "A permission that allows reading users."]
[case UserRead]
[documentation UserBan "A permission that allows for banning users."]
[case UserBan]
]
This yields a serialize method that looks like this:
@Override
public void serialize(final CBSerializationContextType context, final IdA1AdminPermission value)
throws IOException {
if (value instanceof IdA1AdminPermission.AdminBan) {
serializeAdminBan(context, (IdA1AdminPermission.AdminBan) value);
return;
}
if (value instanceof IdA1AdminPermission.AdminCreate) {
serializeAdminCreate(context, (IdA1AdminPermission.AdminCreate) value);
return;
}
if (value instanceof IdA1AdminPermission.AdminDelete) {
serializeAdminDelete(context, (IdA1AdminPermission.AdminDelete) value);
return;
}
if (value instanceof IdA1AdminPermission.AdminWrite) {
serializeAdminWrite(context, (IdA1AdminPermission.AdminWrite) value);
return;
}
if (value instanceof IdA1AdminPermission.AdminWriteSelf) {
serializeAdminWriteSelf(context, (IdA1AdminPermission.AdminWriteSelf) value);
return;
}
if (value instanceof IdA1AdminPermission.AdminRead) {
serializeAdminRead(context, (IdA1AdminPermission.AdminRead) value);
return;
}
if (value instanceof IdA1AdminPermission.AuditRead) {
serializeAuditRead(context, (IdA1AdminPermission.AuditRead) value);
return;
}
if (value instanceof IdA1AdminPermission.UserDelete) {
serializeUserDelete(context, (IdA1AdminPermission.UserDelete) value);
return;
}
if (value instanceof IdA1AdminPermission.UserCreate) {
serializeUserCreate(context, (IdA1AdminPermission.UserCreate) value);
return;
}
if (value instanceof IdA1AdminPermission.UserWrite) {
serializeUserWrite(context, (IdA1AdminPermission.UserWrite) value);
return;
}
if (value instanceof IdA1AdminPermission.UserRead) {
serializeUserRead(context, (IdA1AdminPermission.UserRead) value);
return;
}
if (value instanceof IdA1AdminPermission.UserBan) {
serializeUserBan(context, (IdA1AdminPermission.UserBan) value);
return;
}
throw new IllegalArgumentException(String.format("Unrecognized variant case class: %s",value.getClass()));
}
Note the lack of calls to context.writeVariantIndex()
.
Creating a record with an invalid field yields this error:
ERROR: file:///home/rm/git/com.github/io7m/idstore/com.io7m.idstore.protocol.user.cb1/src/main/resources/com/io7m/idstore/protocol/user/cb1/User1.cbs:196:3: Unrecognized record member declaration.
[field requestId IdU1UUID]]
^
Object: Record member
Specification: https://www.io7m.com/software/cedarbridge/specification/index.xhtml#id_b43940c3-038f-4330-971f-ac76d56d5fad
Expected:
(parameter <type-parameter-name>)
| (field <field-name> <type-expression>)
| (documentation <name> <quoted-string>)
The problem is the [field requestId IdU1UUID]]
line; although the line and column numbers are correct in the error message, the actual line displayed is the line after the line where the actual error occurred. Somewhere, an off-by-one error has snuck in and the expression source is being given the wrong value.
Currently, CCBE is the only defined encoding for Cedarbridge. Although it is an efficient encoding in that it transfers absolutely no type information whatsoever, it does have a bit of waste in the sense that string and sequence lengths are always 32-bit integers, and variant indices are also always 32-bit integers. The utter simplicity is a desirable property and should be preserved and left as the default. However, this doesn't mean that it shouldn't be possible to introduce new alternative encodings if two peers can communicate this encoding out-of-band.
For many protocols, 16-bit integers might be enough for strings and sequences, and 8-bit integers might be enough for variant indices.
However, the Cedarbridge compiler has no way to know which encoding is going to be used and so it can't perform checks that might be useful. For example, if there's a hypothetical mini8
encoding that encodes all lengths and indices as 8-bit integers, then the compiler should check that variants and protocols don't contain more than 255
cases/messages. This could be achieved with an encoding
declaration:
[encoding mini8]
... Where the default if no encoding is specified is:
[encoding canonical]
Document the container protocol for version negotiation.
[variant Boolean
[case True]
[case False]]
Using com.io7m.cedarbridge.cmdline
in combination with the exec-maven-plugin
brings in the possibility of classpath pollution, and doesn't play nicely if there are XML dependencies present (for example, xercesImpl
conflicts with the bridgedoc
generator).
Currently, every project using Cedarbridge has ended up with a type like this:
[documentation CAI1UUID "A 128-bit UUID value."]
[record CAI1UUID
[documentation msb "The most significant bits of the UUID."]
[field msb cb:IntegerUnsigned64]
[documentation lsb "The least significant bits of the UUID."]
[field lsb cb:IntegerUnsigned64]
]
This has a number of benefits:
Variant indices can be encoded into a single unsigned byte. This provides a lot of space improvements when serializing sequences of variant types without increasing decoding or encoding complexity.
Code generation for protocol versions currently passes the serializers for all types in a version into the constructor of the protocol version serializer. This means there's a limit of 255 possible serializers that can be passed in before hitting the Java method parameter limit.
Because cedarbridge
supports variants, limiting protocols to 255 messages wouldn't harm expressivity; if someone really wants to define a protocol with 255 message types, they can group those messages into an infinite number of enclosing variants and have as many protocol messages as they want - they just can't all be at the "top level" of the protocol.
If a language statement names an unrecognized language minor version, parsing of the compilation unit MUST be aborted, and all subsequent statements in the compilation unit ignored.
The point of the minor version is so that compatible changes can be introduced. The above statement should be reworded such that an unrecognized minor version is allowed to produce a warning, but otherwise processing can continue.
public sealed interface CBTypeExpressionType extends Formattable
source code gives unpredictable order in bytecode
├── com/io7m/cedarbridge/schema/compiled/CBTypeExpressionType.class
│ ├── procyon -ec {}
│ │ @@ -1,9 +1,9 @@
│ │
│ │ package com.io7m.cedarbridge.schema.compiled;
│ │
│ │ import java.util.Formattable;
│ │
│ │ -public sealed interface CBTypeExpressionType extends Formattable permits CBTypeExpressionType$CBTypeExprParameterType, CBTypeExpressionType$CBTypeExprNamedType, CBTypeExpressionType$CBTypeExprApplicationType
│ │ +public sealed interface CBTypeExpressionType extends Formattable permits CBTypeExpressionType$CBTypeExprNamedType, CBTypeExpressionType$CBTypeExprParameterType, CBTypeExpressionType$CBTypeExprApplicationType
│ │ {
│ │ boolean contains(final CBTypeParameterType p0);
│ │ }
I'm reporting here because your project is the first one I see that is affected, but I suppose you can' solve anything at your level.
The root cause is probably in the JDK compiler: I'll need to open a JDK issue
Versions must be contiguous, with no gaps.
Document the canonical binary encoding for serialized values.
The current idstore
codebase gets incorrect type parameters (no parameters!) generated for the following Page type:
[record IdA1Page
[parameter T]
[field items [cb:List T]]
[field pageIndex cb:IntegerUnsigned32]
[field pageCount cb:IntegerUnsigned32]
[field pageFirstOffset cb:IntegerUnsigned64]]
They're not exposed in documentation.
Might also be worth allowing them to be manually specified for protocols.
The maximum length of byte arrays and strings should be limited, because there's no sensible thing for a decoder to do other than read the maximum length, allocate a buffer of that length, and then read data into it. It only takes one stream that contains a large integer to allocate gigabytes of heap.
We can infer protocol UUIDs from the fully qualified name of a protocol, and this UUID can be used in the container protocol. There should ideally also be a way to specify the UUIDs manually.
The interface types have been completed, but the code to actually read/write protocols doesn't exist yet.
The serialization context is an implementation of the serializer context API. This should be built on top of bssio
.
Currently, they're rendered in the default font.
The code generator and much of the compiler code was written to make migration to Java record classes easy when records make it into an LTS JDK (likely JDK 17 at this point). Do the migration!
Right now, creating a new version of a protocol involves re-declaring all of the messages again:
[record A]
[record B]
[record C]
[record D]
[protocol D
[version 1 A B C]
[version 2 A B C D]
[version 3 B C D]
]
When the number of messages is large, this is unwieldy. It might be better to introduce the concept of protocol expressions that are evaluated to yield a final protocol. Nothing needs to change with regards to type checking, code generation, or otherwise.
[record A]
[record B]
[record C]
[record D]
[protocol D
[version 1
[introduce A]
[introduce B]
[introduce C]
]
[version 2
[introduce D]
]
[version 3
[remove A]
]
]
Currently, code generation fails if a type references more than 255 types. Worse, type references are not deduplicated - a record
type with 255 integer-typed fields results in code generation that refers to 255 integer serializers.
Type references should be deduplicated for code generation, and limits should be put in place in the compiler to reject type declarations that reference more than 255 deduplicated types.
The attached file yields the following error:
ERROR: Admin1.cbs:218:1: Invalid S expression.
Specification: https://www.io7m.com/software/cedarbridge/specification/index.xhtml#id_d1cd60f0-8880-4754-964f-5b0489947c59
]
Because protocols are limited to 255 messages, it would be helpful if protocols could be nested (strawman syntax):
[protocol P
[version 1
[types-added A B C]]]
[protocol Q
[version 1
[types-added D E F]]]
[protocol R
[version 1
[protocol P 1]
[protocol Q 1]]]
Currently, writing validation code to convert from versioned Cedarbridge types to application types involves a ton of boilerplate. Being able to pattern match will reduce some of this, but some of it will remain. The language knows all of the types that are going to need to be converted, so it could generate some empty classes filled with static methods that need to be filled in.
The generator generates code that doesn't pass JavaDoc validation:
/**
* A serializer for color values.
*/
public final class IdColorSerializer
extends StdSerializer<IdColor>
{
/**
* A serializer for color values.
*
* @inheritDoc
*/
public IdColorSerializer()
{
this(null);
}
/**
* A serializer for color values.
*
* @inheritDoc
*
* @param t The serialized class
*/
public IdColorSerializer(
final Class<IdColor> t)
{
super(t);
}
Generating /home/build/git/com.github/io7m/idstore/com.io7m.idstore.documentation/target/documentation/apidocs/com/io7m/idstore/colors/IdColorSerializer.html...
/home/build/git/com.github/io7m/idstore/com.io7m.idstore.documentation/target/javadoc-sources/com/io7m/idstore/colors/IdColorSerializer.java:36: error: incorrect use of inline tag
* @inheritDoc
^
/home/build/git/com.github/io7m/idstore/com.io7m.idstore.documentation/target/javadoc-sources/com/io7m/idstore/colors/IdColorSerializer.java:48: error: incorrect use of inline tag
* @inheritDoc
^
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.