ossuminc / riddl

A compiler for the DDD-based design language RIDDL

RIDDL

Full Documentation

Is here: https://riddl.tech

Introduction

RIDDL, the Reactive Interface to Domain Definition Language, is a specification lanugage and tooling to help capture requirements and specify designs for the class of systems that can be designed with DDD and utilize a distributed, reactive cloud native architecture.

RIDDL allows subject matter experts, business analysts, and system architects to work at a higher level of abstraction than they would if they were coding directly in a programming language. RIDDL aims to relieve developers of the burden of maintaining infrastructural code through evolution of the system abstractions.

Important Documentation Sections

• Introduction - answer the important initial questions
• Concepts - provides a conceptual understanding of RIDDL before diving into the details
• Guides - four guides for different kinds of RIDDL audiences.
• riddlc - various ways to obtain the RIDDL compiler

Install With brew

TBD

Quickly Building On Your Computer

To use riddlc locally and be able to update it with new changes, use this approach:

• git clone https://github.com/ossuminc/riddl.git
• Change the directory to that cloned repository
• Put the full path to riddl/riddlc/target/universal/stage/bin directory in your PATH variable
• Build the entire package with sbt compile
• Chane directory to the riddlc sub-project
• Run sbt stage to build the program into theriddlc/target/universal/stage/bin directory
• To update, run git pull from the riddl cloned repository and rerun the sbt stage command in the riddlc sub-project to rebuild.

This allows you to both make local changes and pull in changes from others to keep your local copy of riddlc up to date.

Usage

To get the most recent options, run riddlc help. That command will give you the syntax for doing various things with the riddl compiler (riddlc)

Version / Info

The riddlc compiler has two commands, info and version that just print out information about the build, and the version number, respectively, and then exit.

Contributing

Contributions are very welcome!

If you see an issue that you'd like to see fixed or want us to consider a change, the best way to make it happen is to help by submitting a pull request implementing it. We welcome contributions from all, even if you are unfamiliar with RIDDL. We will endeavor to guide you through the process once you've submitted your PR.

Please refer to the CONTRIBUTING.md file for more details about the workflow and general hints on preparing your pull request. You can also ask for clarifications or guidance on GitHub issues directly.

The RIDDL family of repositories is owned by Ossum, Inc., and they require the use of a CLA (Contributor License Agreement). You can sign at that link or be prompted to do so when you submit your first Pull Request.

IntelliJ File Type & Color Scheme Support

The language directory contains two files that will improve the visual appeal of RIDDL source code in your IntelliJ IDEA. To load them, follow the instructions below:

File Type Support

This will provide support for the .riddl file type which you should use for your RIDDL files. This is provided in intellij-idea-riddl-file-type-settings.zip

• To load into IntelliJ:
  • Select File -> Manage IDE Settings -> Import Settings... from IntelliJ menu
  • Navigate to your cloned repository in the language folder
  • Select intellij-idea-riddl-file-type-settings.zip

Color Scheme Support

Differentiating between keywords, readability words, definitions, punctuation, types, etc. is supported through the use of color scheme settings in this file: intellij-idea-riddl-colour-scheme-settings.jar. Note that using this colour scheme requires overriding other themes as only 1 scheme can be used in IDEA at a time. You can switch themes in the Settings under Editor --> Color Schemes

• To load into IntelliJ:
  • Load the IntelliJ Settings dialog (File -> Settings or Settings gear on top right)
  • Navigate to the Editor -> Color Scheme settings
  • Click the Show Scheme Actions gear icon to the right of the scheme selector
  • Choose Import Scheme... from the pop up menu
  • Navigate to your cloned repository in the language folder
  • Select intellij-idea-riddl-colour-scheme-settings.jar

riddl's Issues

Fix linkage between Hugo pages and RIDDL source

GeekDoc’s “Edit” button isn’t getting the right URL to allow editing. The Hugo Translator needs to set the source URL in each generated file.

SAGAs need to be able to coordinate function calls

Currently a SAGA expects to be able to coordinate a complex transaction via async messaging to entities (exclusively). SAGAs may also need to coordinate a complex transaction via synchronous calls to external services.

Host riddl at riddl.tech

This project has graduated from "nascent" to merely "early". In the spirit of Linus Torvalds "commit early, commit often", this task asks for https://riddl.tech/ to be hosted on Google Cloud as a single VM that:

• • every hour:
    *checks out riddl to a directory.

• cleans, packages and runs the test suite

• if successful, starts "hugo -D server" from the examples/target/translator/ReactiveBBQ.riddl

• makes ReactiveBBQ site (see above) available at riddl.yoppworks.com/examples/rbbq

• hosts the API docs generated from the last build at https://riddl.yoppworks.com/apidoc

• hosts the riddl hugo docs at https://riddl.yoppworks.com/doc

This can probably be done most easily by constructing a container that has nginx, git, sbt, scala compiler and a few other things in it.

Left Nav is colliding with main body content in RIDDL hugo output

!Screen Shot 2022-02-18 at 11.39.40 AM.png|thumbnail!

Names that are longer than the space allocated to the left nav simply bleed over into the main body content. See attachment.

Make TODO a language feature?

Perhaps somewhere (with each page maybe?) in the HUGO output files we need to have a TODO section that would list tasks that need work. These could be included based on a TODO marker in the RIDDL file, or maybe even with a ??? definition. (multiple ways to mark a TODO to include in the output)

RIDDL Documentation - Missing Backoffice and Corporate content

BackOffice and Corporate RIDDL code examples are missing from the examples pages. The RIDDL files exist for the content, they are just not linked properly.

┆Issue is synchronized with this Jira Story by Unito

Add Ritter's Blog Ideas To Practitioners Best Practices

This blog:
https://rafaelritter.medium.com/ddd-model-integrity-patterns-36d0e8ac9561
has some good points that should be mentioned in practitioners documentation as best practices, etc.

Remove Scala package named "ossum" from all code

Riddl is its own thing and not a subpart of ossum. When we get to ossum, it will be used, but isn’t logically contained within ossum. Since RIDDL is open source, it needs to be not tied to the idea of Ossum.

This also means the GitLab project should not be buried inside the GitLab Ossum group. It is correct on GitHub (Yoppworks/riddl)

Allow a .md file to be incorporated in an "explained by" section

Sometimes RIDDL authors want to put a long explanation of a RIDDL definition into its own .md file. We need to support that option.

AST Should Represent Comments Too

Comments are currently ignored by the scala whitespace parser by fastparse. We actually want to keep the comments, as an array of strings, so they may be put out by reformat or otherwise processed by any translator.

We need to:

• Add Comment(loc: Location, isSingleLine: Boolean, lines: Seq[String]) to AST
• Change the white space parser to capture both single line (//) and multi-line (/* */) comments
• Have these things ignored by most translators
• Have ReformatTranslator regurgitate them.

Create Chroma Lexer In Go For Riddl

This will allow Hugo documentation to highlight riddl source code when used between triple backticks sections, like this:

```riddl
domain foo is { ??? }
```

┆Issue is synchronized with this Jira Story by Unito

Implement a SeeAlso feature in RIDDL

When creating a RIDDL specification it may be useful to link a user to another part of the specification. This could be done with a See Also declaration.

Definitional Hierarchy->Domains->Bounded Contexts->Entities->Consumption needs example

I expected to see an example of a Command or a Query. Instead, it seems there is an example of an Entity definition with an "options" modifier

┆Issue is synchronized with this Jira Story by Unito

Finish ReformatTranslator

• Rename: FormatTranslator → ReformatTranslator
• Base it on a Folding function, not Folding Trait (avoid inheritance)
• Make “Include” be honored and implement both sides of “single-file” option

Autocompletion

Autocompletion works when typing in a reference of any kind. Since RIDDL prefixes such references with a keyword (type, entity, etc.), only the matching identifiers for that kind of definition will be shown. Upon completion, the shortest identifier path to the selected identifier will be inserted.

https://gitlab.com/yoppworks/ossum/riddl-idea-plugin/-/blob/main/img/autocompletion.gif

┆Issue is synchronized with this Jira Story by Unito

Validation Can Be Done In Parallel

Bounded Contexts are very isolated from other constructs. To speed up validation, AST.Context instances could be processed in parallel during validation without harm. The message list will need to become thread safe but there should be little other impact than that.

Detection of URLs and identifier paths in string literals

When URLS and Path Identifiers are used in the model's documentation they should be converted into URLs in the generated documentation to the corresponding definition or site's location.

Create RIDDL installer for Windows

Install RIDDL binaries to a user's machine and configure environment variables to get the user operating with riddl quickly.

Create RIDDL installer for Linux

Install RIDDL binaries to a user's machine and configure environment variables to get the user operating with riddl quickly.

Modify Adaptors->Non-DDD External Systems to fit the Focuser Pattern

I believe that this section is describing a situation in which we are employing the Focuser Pattern.

Invite users to participate in defining details of a RIDDL model

I am going through the detail of adding definitions for each type, entity, context, domain, command/event/query/result, etc. in a RIDDL file. It made me think, this would be a tremendous opportunity to get users from the business and others involved in the definition of the domain model. It would offload a lot of work from the RIDDL author, and it would serve to get buy in from the true owners of the model.

This is certainly a down the road OSSUM feature. But I think we may need to give some thought on how we could simplify the RIDDL syntax to accommodate this. For example, you declare a new domain as potentially the very first thing in the RIDDL file, however, because of syntax you have to scroll to the end (of a potentially large file) of the file in order to reach a point where you can actually define it. One thing that might ease things a bit on this point is extending the IntelliJ plugin to allow you to collapse blocks of code? Allowing us to include definition blocks outside of the actual RIDDL file (which we have discussed previously) may also help.

A nifty integration for this would be if the validator noticed that something was missing a definition, rather than throwing an error create a JIRA (plug in task management system here) ticket to capture the work. It would be fantastic if the (future) OSSUM feature would allow a conversation (comments/replies) around defined by sections.

It would be handy to include all these defined by blocks in a glossary output - maybe as part of the Hugo site.

Anyway, capturing a thought. Maybe we can discuss further in the future.

Quick Documentation

This action causes the editor to pop up the RIDDL provided documentation for the selected identifier.

https://gitlab.com/yoppworks/ossum/riddl-idea-plugin/-/blob/main/img/quickdoc.png

┆Issue is synchronized with this Jira Story by Unito

RIDDL init project {project name}

create JIRA Project
create Git Repo
Initialize project structure (Riddl, src, test, docs, etc.)
boilerplate RIDDL source file

Create RIDDL installer for MacOS

Install RIDDL binaries to a user's machine and configure environment variables to get the user operating with riddl quickly.

Glossary Language Component

Should we add a glossary language component where we can declare a term and provide a definition? This would be useful in establishing the Ubiquitous Language within a bounded context or domain. If, elsewhere in the spec (maybe a described as block) if a glossary term is used and annotated the output could generate a link to that definition.

A glossary term must be defined within a hierarchy. A term defined within a bounded context or domain has no relevance or meaning outside of the scope where it was defined. If you are in a subdomain the glossary terms of the parent domain still apply - unless expressly overridden. Other relationships (child, sibling) could explicitly reference a definition using that relationship, under the condition that the term is always referenced and cannot be altered unless done where the term was defined.

As an architect, I need information organized by messages, so I can comprehend an event driven system

This is a request to create a new section in the hugo translator's output that organizes information focusing on message types, how they are used, where they are defined, and where in the hierarchy they exist.

Message types are the four DDD types: command, event, query, and result. The documentation for every leaf subdomain should have four pages added, one for each kind of message type. Those pages should list all the messages of their type that are USED by the subdomain (all contexts) but none that are merely accessible by the subdomain. Ideally, that wouldn't happen, but in practice, it will.

Each page contains a list of relevant messages. Each row in that list starts with the name of the message and then a set of tabs:

• Type - shows the fields of the message, with their brief descriptions if any
• Defined In - shows the subdomain, context, entity, or other definition in which the message is defined, along with that definition's brief description
• Used In - shows the places from which this message is used with full path identification, and each path component is a link to the documentation for that definition.
• Description - Full description of the message as defined in its "described by" type which could be fairly involved with diagrams, etc.

#include markdown files/snippets

We should be able to author markdown files - or snippets of markdown as a standalone file and #include that file into the RIDDL source. It will make authoring, commenting, and managing those files easier.

┆Issue is synchronized with this Jira Story by Unito

Error Highlights

Text entered into RIDDL files that is not syntactically or referentially correct will be highlighted with squiggly red underlines as is common for programming languages.

┆Issue is synchronized with this Jira Story by Unito

Support traverals or indexing of values of types with cardinality

If you have a type like:

type FooRef = reference to entity Foo
type Bar = { refs: FooRef+ }

then you may want to iterate over those references in an action, say, to tell each one of them something. This is worth modeling in RIDDL.

Find Usages

This action, when invoked upon any identifier, looks for all usages of that identifier in the project (not just the current root domain). Usages include identical occurrences of the identifier in strings and comments.

┆Issue is synchronized with this Jira Story by Unito

Generate Context Maps From A Domain

The documentation generated by riddlc should create context map diagrams for each domain showing both the contained contexts and the ones they interact with

Clean up formatting for Glossary of terms

The primary function of the glossary of terms is to have a quick reference to terms and their definitions. In the current presentation, the type and path columns take up so much real estate that it crowds the definition and makes it hard to read.

Here are a couple of ideas for how we might do this (these are not requirements, but rather ideas to consider)

1. assign an icon for each of the types (Context, Entity, Function, etc.), move the icon to the first column, and drop the Type text-based column.
2. move the path under the term in a smaller font; or, maybe make it a popup when you hover over the term. Both could work, but we aren't sure how to pull that off in MD format...

Revise AST to use Long instead of Location

The use of Location objects in every AST construct is time-consuming and wastes memory allocations. A significant saving could be had if we:

• Replace "Location" with just a Long (offset in file)
• Calculate Location only when its needed (method on RiddleNode)
• Create nodes for input sources (also helps with reflection and

Unfortunately, this is a long, unforgiving, touches everything kind of change at this point, especially in test cases, but worth it in the long run. Without this, every successful parse is penalized for the utility in generating error messages that won't be utilized.

┆Issue is synchronized with this Jira Story by Unito

Handler section is implemented twice in Definitional Hierarchy->Domains->Bounded Contexts->Entities

The first instance has content. The second does not.

links to Restaurant, Back Office, and Corporate in the RBBQ example do not work (404)

The RIDDL files exist. In the restaurant domain there are several bounded contexts defined that need to somehow be represented in the documentation.

┆Issue is synchronized with this Jira Story by Unito

Update documentation to reflect handler syntax and options

What was in the docs and what was accepted by the validator were vastly different. Too many errors to document here.

┆Issue is synchronized with this Jira Story by Unito

Add content of "RIDDL Initial Thoughts" Google doc to Documentation

James has a document on Google Docs that should be factored into the documentation

GOTO Prev/Next Definition

This action helps when navigating the hierarchy of definitions. It performs a depth first traversal of containers and definitions but not into the details of a definition.

https://gitlab.com/yoppworks/ossum/riddl-idea-plugin/-/blob/main/img/gotoprevnext.gif

┆Issue is synchronized with this Jira Story by Unito

Documentation & Examples

Any tasks having to do with the doc and examples sections of RIDDL

┆Issue is synchronized with this Jira Epic by Unito

Language->Interactions needs more content

How do these UML Sequence Diagrams get generated. We know that the answer lies in Hugo/Mermaid, but the documentation will need to help the user understand how to accomplish this.

┆Issue is synchronized with this Jira Story by Unito

Update RIDDL documentation to reflect new syntax when declaring message types

Syntax should be:
type DoIt is command {}
type DidIt is event {}

┆Issue is synchronized with this Jira Story by Unito

Fix failing GitLab pipelines

For example:

https://yoppworks.slack.com/archives/CSN8D062K/p1643682989256299

which looks like this:

!Screen Shot 2022-02-01 at 2.51.44 PM.PNG|width=672,height=221!

and has this in the pipeline log:

Checking out d9d9daaa as main...

Skipping Git submodules setup
Executing "step_script" stage of the job script
Using docker image sha256:d456c516012bd9badbc8c66c8d5261fc9c3f81d5f637eedaa43b574d43f71dd0 for yoppworks/gitlab-runner-sbt-scala:latest with digest yoppworks/gitlab-runner-sbt-scala@sha256:9801590af7653a9acd9c4ce734d7c5266bb486642c33b0f79a20aed5ab2fd1af ...
Runtime platform                                    arch=amd64 os=linux pid=6 revision=4b9e985a version=14.4.0
FATAL: Command sh not found.                       
Cleaning up project directory and file based variables
ERROR: Job failed: exit code 1

┆Issue is synchronized with this Jira Bug by Unito
┆Attachments: Screen Shot 2022-02-01 at 2.51.44 PM.PNG

Add A riddlc "shell" command

Add a "shell" command that allows riddlc to be used interactively so that commands and options can be typed in to a prompt and executed that way.

Implement BAST file read & write

Create a binary file format, BAST, to store pre-compiled RIDDL specifications that can be imported into other models.

dot notation in navigation hierarchy is misleading

For example, if you navigate to Language->Definitional Hierarchy-> Root it appears that Definitional Hierarchy is indented at the level of Interactions before it and Root below it. But there is a pipe in front of it to show navigation context. Styling, however, makes it appear that it is one level down in the hierarchy. Also, the same is true of Root.

┆Issue is synchronized with this Jira Story by Unito

Remove Interaction From AST And All Uses

Interaction was meant to capture an sequence diagram as an interaction between system components as an example of processing, but they are not definitional of even typical processing and therefore more a documentation tool than a system specification.

Sequence diagrams should be generated from definitions made in RIDDL, not the other way around.

Consequently, Interactions should not be part of the language.

Create two documentation tracks - one for RIDDL authors and another for RIDDL developers

┆Issue is synchronized with this Jira Story by Unito

Make navigation pane/portal independently scrollable from main body panel.

The left navigation pane currently scrolls together with the main body of the topic you are currently viewing. For example if you navigate to Language -> Conventions you will find that the conventions content is larger than your current screen. If you want to maybe navigate to another topic you cannot scroll to that topic in the navigation pane without scrolling the whole page, losing context of what you were just reading. These two things should be independently scrollable.

┆Issue is synchronized with this Jira Story by Unito

Need to establish a convention for naming of types, attributes, fields, and so on in RIDDL files.

I am getting a lot of name conflict warnings with my RIDDL file. Here is the thing, however, I only see it as a conflict within the RIDDL file, but not in actual implementation because the definitions are context sensitive. Here is an example:

[warning] Style: projectx.riddl(249:53): Field 'retailerId' overloads:
Field 'retailerId' at projectx.riddl(66:21),
Field 'retailerId' at projectx.riddl(283:21),
Field 'retailerId' at projectx.riddl(240:21),
Field 'retailerId' at projectx.riddl(326:21),
Field 'retailerId' at projectx.riddl(253:67),
Field 'retailerId' at projectx.riddl(106:21),
Field 'retailerId' at projectx.riddl(257:21)

Each of these is used in either defining a state for another entity, or is an argument for a command/event. It would get noisy and tedious to define each of these uniquely, for example orderBaseRetailerId so that the name was unique across the entire file. This method of naming is a bit redundant because retailerId is used in the definition of the OrderBase state.

Not sure if I am making sense here, but happy to talk it through when it is convenient to do so.

Keep an mutable ancestors stack while folding for symbol context lookup

This change implies revision of the FormatTranslator and Validator to use the simpler foldLeft definition and without using a Folding trait subclass. This makes traversal more flexible and functional.

┆Issue is synchronized with this Jira Story by Unito