just-dandi / dandi

🌻A modular DI, MVC, and Model binding/validation framework for NodeJS and TypeScript or ES6

@dandi

Dandi is a modular DI and MVC application framework designed to make it easier to write RESTful APIs, console applications, and other services for NodeJS with TypeScript. It is split into modules to allow developers to use only the features they require.

Get Started

Use Cases

• MVC-style Application Servers
• CLI Applications
• AWS Lambda functions

Features

Dependency Injection

• Modeled after Angular's dependency injection system
• Inject dependencies into class constructors or methods
• 3rd party dependencies can be configured to be injected with Providers

HTTP Pipeline

• Standalone pipeline for handling and responding to HTTP requests
• Automatic input content negotiation based on request content-type header
• Automatic output content negotiation based on request accept header
• Create and configure interdependent middleware handlers (HttpPipelinePreparer) and they will automatically be invoked in the correct order
• Add global transformers to control response data structure

MVC

• Modeled after ASP.NET Core MVC
• Builds on functionality of @dandi/http-pipeline to add routing and other MVC features
• Web framework agnostic - Built for Express 4, but can be used with other frameworks if desired
• Decorator-based route configuration
• Automatic path parameter, query parameter, and body model mapping and validation
• Support for automatically generating HA JSON output

Model Building and Validation

• Robust set of decorators for defining models and validation metadata
• Automatically construct model class instances from JSON objects or POJOs
• Convert from objects using a different property key casing (e.g. snake_case to camelCase)

AWS Lambda

• Wraps @dandi/http-pipeline for use with handline AWS Lambda requests
• Use the same framework, models, and add-ons (HttpPipelinePreparer, HttpPipelineResultTransformer) between your Lambda functions and MVC API

Misc

• Disposable interface and utilities for managing disposable resources
• Uses Luxon as a replacement for Date objects
• Models can be reused between backend NodeJS and frontend TypeScript/JavaScript applications
• Uuid class based on the uuid library for working with and comparing UUIDs.

Core Modules

• @dandi/cache - Caching functionality
• @dandi/common 🕸 - Common types and utilities
• @dandi/config 🕸 - Configuration services
• @dandi/core 🕸 - Dependency Injection
• @dandi/core/logging 🕸 - Core logging and configuration
• @dandi/core-node - Additional DI utilities specific to NodeJS
• @dandi/data 🕸 - Base types and utilities for working with data services
• @dandi/hal - 🕸 - Model decorators, basic types and utilities for supporting HAL
• @dandi/hal-model-builder - 🕸 - Model building utilities supporting HAL embedded resources
• @dandi/http - Types, utilities and DI tokens for basic HTTP concepts (headers, mime types, etc)
• @dandi/http-model - Types, decorators, and utilities for using model building and validation in an HTTP context
• @dandi/http-pipeline - A pipeline for handling and responding to HTTP requests
• @dandi/logging - 🕸 - Additional utilities for logging and logging configuration
• @dandi/model 🕸 - Decorators for describing models, for use with @dandi/model-builder
• @dandi/model-builder 🕸 - Utilities for dynamically constructing and validating models
• @dandi/mvc - MVC decorators and base utilities (not specific to Express)
• @dandi/mvc-hal - Supports rendering HAL JSON from existing @dandi/mvc controllers
• @dandi/mvc-view - Use @dandi/mvc with your favorite templating engine

3rd Party Integration Modules

• @dandi-contrib/aws-lambda Helpers for using @dandi/core and @dandi/http-pipeline with AWS Lambda
• @dandi-contrib/config-aws-ssm - @dandi/config client for AWS SSM Parameter Store
• @dandi-contrib/data-pg - @dandi/data client implementations for Postgres
• @dandi-contrib/mvc-auth-firebase - Google Firebase Authorization service implementation for @dandi/mvc
• @dandi-contrib/mvc-express - Express-specific @dandi/mvc service implementations
• @dandi-contrib/mvc-view-ejs - EJS implementation for @dandi/mvc-view
• @dandi-contrib/mvc-view-pug - Pug implementation for @dandi/mvc-view

🕸 - web browser compatible/no NodeJS-specific dependencies

Examples

Simple Express REST API - An implementation of a very simple REST API using @dandi

Dev Setup

To set up this project as a local repository:

• Install Yarn Classic if you don't have it installed
• After cloning the repostory, run yarn setup - this will install the dependencies for all the individual packages, as well as the Dandi builder
• You can now run yarn build to build all packages, or yarn test to run all tests

dandi's Issues

fix broken unit tests

• Disposable in @dandi/core
• PgDbTransactionClient in @dandi/data-pg

automatically generate json schema from models

• create @dandi/model-schema package
• use model metadata to generate JSON schema
• create service to provide access to model schemas

Make ConsoleLogListener more customizable

• Allow configurations for each log level, plus default config
• function to format entire log entry, or
  • granular configuration
    • timestamp format - function or Luxon format name/string
    • context tag format - function, tokenized string, prefix/suffix
    • args - prefix/suffix
• Allow filtering by log level
• Write tests

Extract transaction logic from PgDbTransactionClient into helper utilities in @dandi/data

• allow overrides for actions required for beginning, committing, and rolling back transaction
• remaining behavior is SQL implementation agnostic

Use model schema with fastify

dependent on #23, #36

• use model schema with fastify's JSON serializer

improve error handling when using views

Currently, the framework does differentiate between pipeline results with errors and those without - it will attempt to render the response regardless. This is fine for data responses, as the appropriate error information will be rendered in the format requested by the client, however, it doesn't work as well when using views, since it will attempt to render the requested view, which will often fail due to missing required data.

• add configuration option for default error view
• add configuration option for per-status code error views
• update MvcViewRenderer to check for errors in the HttpPipelineResult, and use error rendering if errors are present
• render a per-status code error view if one is configured for the error
• use the default error view if there is no status-code-specific view configured
• render a plain text representation of the error if there are no error views configured
• show the RequestError.internalMessage (if defined) only for non-production environments
• show the stack trace only for non-production environments
• Also include innerError data

load type data at runtime

Angular uses a customized wrapper around tsc to extract type data and allow it to be used at runtime. Dandi would benefit from the same functionality.

• allow Container to infer injected types from type data
• make the injection token parameter of @Inject optional

Implement OPTIONS/CORS handler

• options handler - check against available HTTP methods
• CORS - host whitelist
• CORS - header whitelist
• CORS - method whitelist
• CORS - host match function
• Finalize cross-origin request detection
• CorsHandler - also consider CORS request headers (e.g. access-control-request-headers)
• Only include CORS-enabled sibling methods for allowMethods
  • Only include sibling methods whose CORS configuration would allow a request, given the headers, etc
• MVC - Make sure route-level cors config overrides pipeline config
  • Allow headers should always include default allowed headers
• do not emit CORS headers on non-cross-origin requests
• make sure it works with aws-lambda (may need to add providers)
• tests!

fix Disposable.useAsync not actually taking a Promise<Disposable> for the source object

• Disposable.useAsync accepts a promise for the source object and resolves it before invoking the usage function

@dandi/hal - better solution for defining interdependent resources

The existing implementation using of @ListRelation and @Relation does not work when two resources have interdependent relations (e.g. a parent with an array of children, and the children with a relation to the parent).

A solution would be to allow offloading the decoration of relations to a class defined in a separate file. The @Relations(forModel) decorator links the relation data defined in the "relations" class to the specified model.

Example:

// parent.resource.ts
export class ParentResource {
  // decorating without referencing ChildResource acts as a placeholder, and
  // prevents a circular reference at runtime
  @ListRelation() 
  public children: ChildResource[];
}

// parent.resource.relations.ts
@Relations(ParentResource)
export class ParentResourceRelations {
  // fills in the placeholder with the correct class reference
  @ListRelation(ChildResource)
  children: ChildResource[];
}

// child.resource.ts
export class ChildResource {
  // decorating without referencing ParentResource acts as a placeholder, and
  // prevents a circular reference at runtime
  @Relation()
  public parent: ParentResource;
}

// child.resource.relations.ts
@Relations(ChildResource)
export class ChildResourceRelations {
  // fills in the placeholder with the correct class reference
  @Relation(ParentResource)
  public parent: ParentResource;
}

Questions

• Doesn't this still has circular dependencies?
  Technically yes, but since the circularity is only referenced for typing (e.g. we aren't referencing the circularly referenced types as classes/values), and not at runtime, it still works.

• How do the "relations" files get referenced naturally? As it is, they'll need to get explicitly imported somewhere just for their side effects, which is not ideal.

Option A:

parent.resource.model.ts // export class ParentResource
parent.resource.relations.ts // export class ResourceResourceRelations
parent.resource.ts // export * from './parent.resource.model'; export * from './parent.resource.relations'

Caveat: "relations" must import directly from the "model" file, all other references must import from "resource" file

add validation translation layer for angular

Add a validator for Angular that maps an input/ngModel binding to @dandi/model validation information.

new package: @dandi/ng-forms

finish logging documentation

• ConsoleLogListener formatting
• ConsoleLogListener high water mark spacing feature
• Logger API
• PrettyColorsLogging
• using LoggerFixture for testing Logger usage

@dandi/mvc - throw an error and respond with 400 status when missing path param values

Remove @dandi/core dependency on fs-extra

fs-extra is only used by FileSystemScanner, but prevents @dandi/core and anything dependent on it from being used in the browser.

• Create new @dandi/core-node package
• Move FileSystemScanner into @dandi/core-node, remove fs-extra dependency

update CI script to compile example projects

It'd be good to have an automated check that changes to the library don't break the examples

migrate integration modules to @dandi-contrib org

• migrate all 3rd party integration packages
• contrib packages are independently versioned
• follows same naming convention (e.g. @dandi/mvc-express -> @dandi-contrib/mvc-express)

@dandi/aws-lambda - support add SNS events

• add SNS event transformer
• add SNS event responder
• add SNS module

return a `406 Not Acceptable` response if a renderer matching the content negotiation headers is not available, and there is no default renderer

add @dandi/aws-lambda package with helpers for using @dandi/di-core with AWS Lambda

• Support API Gateway events
• Complete unit test coverage

formalize logging

• adds rxjs
• LogStream is an Observable of log items
• Logger becomes an implementation that broadcasts to LogStream
• LogListeners subscribe to LogStream
• Remove NoopLogger
• ConsoleLogger becomes ConsoleLogListener

_

add fastify mvc implementation

add eslint/tslint

add more granular control over content negotiation

• add @Accept() decorator, which can take one or more MIME type strings as input
• @Accept() can be applied at controller or method level
• With no @Accept(), any configured renderer will be used
• With @Accept(), content type rendering will be limited to the route's specified content types

Depends on #24

rename core packages

• rename @dandi/core to @dandi/common
• rename @dandi/di-core to @dandi-core

add view template rendering

• add @dandi/mvc-view
• add ViewEngine interface and injection token
• add ViewResult class
• add RenderOptions interface
• add injectable view(path: string, options: RenderOptions) => ViewResult method
• add ViewEngine implementation for pug in @dandi-contrib/mvc-view-pug
• add ViewEngine implementation for ejs @dandi-contrib/mvc-view-ejs

@dandi/data-pg - test coverage

Add test coverage for the following items:

• PgDbQueryableBase
• PgDbClient
• PgDbPool
• PgDbPoolClient
• PgDbTransactionClient
• PgDbConfig

integrate typedoc

@dandi/mvc - add "disableRequired" option to @RequestBody to support PATCH endpoints

• add options parameter including disableRequired to @RequestBody() decorator
• when disableRequired is set, don't fail on missing required properties

add auto-documenting for MVC APIs

add @dandi/mvc-autodocs package

• add view renderer that will respond to text/html requests

add example app

@dandi/core - make it easier to debug invalid registration targets

Since "invalid registration target" can mean different things, including a bad/missing default value or configuration, it can be very difficult to track down the actual problem, requiring debugging in @dandi/core itself, add tooling to make it easier to narrow down the source of the bad target.

• include the reason the target is invalid in the error message (e.g. target was null/undefined or not one of the accepted registration objects)
• expand and standardize on "module" - the array of types provided as defaults by several packages. Add a standard helper for creating one, include a Symbol property that identifies its source and what it is
• when available, include path of module(s) - including nested modules - in the error messaging

// @dandi/common
export class DandiModule extends Array<any> {
  
  public static readonly package = Symbol.for('@dandi/common#DandiModule.package');
  public static readonly name = Symbol.for('@dandi/common#DandiModule.Name');

  constructor(package: string, name: string, ...providers: any[]) {
    super(providers);
    this[DandiModulePackage] = package;
    this[DandiModuleName] = name;
  }
  providers[DandiModulePackage] = package;
  providers[DandiModuleName] = name;
}

// @dandi/some-pkg
export const SomePkgModule = dandiModule(SomePkgService, AnotherPkgService];

@dandi/aws-lambda - support DynamoDB Stream events

• add DynamoDB Stream event transformer
• add DynamoDB Stream event responder
• add DynamoDB Stream module

add documentation for data-pg

• configuration/connection
• transactions
• SELECT expansion for models

update model package to use runtime type data

dependent on #34

features TBD

test coverage: @dandi/mvc

Add test coverage for the following items:

• DecoratorRouteGenerator
• mergeAuthorization in authorization.metadata.ts
• authorizedDecorator
• DefaultRouteInitializer
• requestBodyProvider in request.body.decorator.ts
• getCorsConfig

Refactor model validation

The current validation scheme actually does two jobs:

• "reconstituting" a model from a "lesser" state
• "lesser" meaning plain JSON, a database representation, etc
• converts JS primitives into Dandi types (e.g. Uuid, Url, etc)
• converts complex objects into classes as determined by decorator

The goals of this change are:

• decouple the "reconstitution" step from the validation
• support for alternate discovery of property values (e.g. getting embedded values from HAL resources from the _embedded property)

This most likely means moving the "reconstitution" logic into @dandi/model, which will not be a problem after #12. Alternatively, use a new package (e.g. @dandi/model-builder)

formalize output formatting/rendering

• ControllerResults - generally data only (remove JsonResult)
• Renderers - can be registered at the application-level (json/xml/etc), or at the route-level (views - html).
• Switch output formatting based on Accept header
• Default to JSON when Accept is ambiguous
• Default to text/html if a view is defined
• allow returning text/plain

request body content negotiation

similar to rendering content negotiation:

• readable body formats can be configured globally (json/form encoded/plain text/multipart/etc)
• body formats are mapped to parsers that can create an object from them
• body validation happens independently after the pipeline receives the request body parsing result

Add support for receiving application/octet-stream data

• Determine what object type is most appropriate to return (e.g. Buffer, etc)
• Add an HttpBodyParser implementation that supports receiving the application/octet-stream MIME type
• Ensure that the new parser works with files transmitted using multipart/form-data requests

add generalized request error handling

depends on #24

• add @ErrorHandler method decorator
• decorated method is invoked when an error is caught in DefaultRouteExecutor

test coverage: @dandi/model-validation

Add test coverage for the following items:

• DecoratorModelValidator
• primitive type validators
• UuidTypeValidator

Move data/key mapping out of @dandi/data and @dandi/data-pg

We should not be doing any modification of the data coming out of data clients from within the data client libraries. This is causing problems with JSON blob columns that use UUID style keys, since the data mapper is trying to convert them to camelCase.

• building on #13, add an option to transform the data before constructing the target model
• use existing @Json decorator to optionally ignore transforms for JSON fields

make token arguments of mvc decorators optional

• @QueryParam
• @PathParam
• @RequestBody

dependent on #34

add telemetry

• add observable for broadcasting telemetry events

@dandi/core

• app startup/bootstrap events
• app errors

@dandi/mvc

• routing startup events
• request lifecycle

@dandi/mvc-hal

• embedding events

fix @QueryParam throwing MissingPathParamError

• @QueryParam should be treated as optional

add "kitchen sink" MVC example

• mvc with express, models, model builder, hal, etc
• postgres db (run postgres in docker container)
• firebase auth
• minimal angular frontend
• pseduo-fake config implementation

Add @RequestHeader decorator

Similar to @RequestBody(), @QueryParam(), and @PathParam(), @RequestHeader() will allow a header value to be injected into a method or class constructor.

• Accept HttpHeader value as an argument
• Inject parsed header value from HttpRequestHeaders

Add @Provides class and method decorator

• exposes the specified providers to the resolver when resolving dependencies of decorated class constructor or method
• can define providers statically or with a function

Separate Resolver logic from Container

• create a Resolver implementation based on existing logic in Container
• refactor resolve and resolveInContext to inject/injectInContext, with additional injectSingle and injectMulti variants
• rename Container -> DandiApplication
• DandiApplication will retain the startup/bootstrapping logic