deepkit / deepkit-framework

A new full-featured and high-performance TypeScript framework

Deepkit

Deepkit is an open-source platform and cross-platform desktop application to execute, track, and debug modern machine learning experiments. With integrated server management tool to execute experiments on any Linux server with a simple click or cli command.

Download - Example Python

Features

• Experiment execution using uniform experiment definition
• Execute experiments locally or in the cloud, seamlessly
• Pipeline support to increase execution time and reduce hardware costs
• Layer debugger for Keras2 and Pytorch
• Automatic source code versioning
• Experiment artifacts (aka output: weights, snapshots, etc)
• Experiment execution via Docker container with auto-build
• Experiment comparison: Side by side, file diff, metrics diff
• Server job scheduling with resource limitation and assignment (and NVIDIA Docker GPU support)
• Real-time experiment tracking: Log metrics, text, images, numpy arrays, and more
• Python SDK
• Project management: Issue tracker, Kanban board, and notes
• Real-time desktop (and web) app with analytical tools to monitor and organize experiments
• Optional centralized server for teams

Development

Deepkit is written entirely in Typescript and consists of following main packages:

• packages/cli: command line interface tools
• packages/deepkit: The GUI, desktop and web app interface
• packages/server: The server component that stores all the data using the filesystem and mongodb
• packages/website: The website as SSR Angular 9 SPA app
• packages/core: Common models, functions, services
• packages/core-node: Common models, functions, services for node environment
• packages/electron: The electron bootstrapper for the GUI

Following open-source packages have been created for Deepkit that wouldn't exist otherwise and were absolutely necessary:

• Marshal.ts - The fastest universal Typescript data serializer and validator
• Glut.ts - A reactive real-time server framework with distributed entity/file abstraction and syncing based on ReactJS
• angular-desktop-ui - A GUI library for native-like GUI widget based on Angular 9+

Development app

cd deepkit
npm run bootstrap

cd deepkit
npm-local-development

cd deepkit
make angular-watch

cd deepkit
server-watch

# Open in browser the angular dist
# e.g. file:///Users/marc/bude/deepkit/packages/deepkit/dist/deepkit/index.html
# or electron
cd deepkit
make electron-build
make electron

Development website

Watcher

cd packages/website
ng run website:server:production --watch

Server

cd packages/website
npm run serve:ssr

deepkit-framework's Issues

How to do array of enums?

Hey I can't quite figure out the syntax, could I please get some help?

Type mismatch error with arrays

Using the example code provided, I'm getting the following error:

decorators.js:1749 Uncaught Error: SimpleModel::tags type mismatch. Given String[], but declared is undefined. Please use the correct type in @f.type(T).
at eval (decorators.js:1749)
at fn (decorators.js:1421)

I have modified my tsconfig.json to enable experimentalDecorators and emitDecoratorMetadata

{
"compilerOptions": {
"experimentalDecorators": true,
"emitDecoratorMetadata": true,
...
}
}

I have installed the reflect-metadata package as well and importing it with the example code.

Removing any array decorators, it runs successfully, but the array properties are not populated obviously.

Example code:

import {
f,
plainToClass,
uuid,
} from '@marcj/marshal';
import 'reflect-metadata';

class SubModel {
@f label: string;
}

export enum Plan {
DEFAULT,
PRO,
ENTERPRISE,
}

class SimpleModel {
@f.primary().uuid()
id: string = uuid();

@f.array(String)
tags: string[] = [];

@f.optional() //binary
picture?: ArrayBuffer;

@f
type: number = 0;

@f.enum(Plan, /*allowLabelsAsValue=*/ true)
plan: Plan = Plan.DEFAULT;

@f
created: Date = new Date;

@f.array(SubModel)
children: SubModel[] = [];

@f.map(SubModel)
childrenMap: {[key: string]: SubModel} = {};

constructor(
    @f.index().asName('name') //asName is required for minimized code
    public name: string
) {}

}

//data comes usually from files or http request
const instance = plainToClass(SimpleModel, {
name: 'myName',
tags: ['foo', 'bar'],
plan: 'PRO',
created: 'Sat Oct 13 2018 14:17:35 GMT+0200',
children: [{label: 'foo'}],
childrenMap: {'foo': {label: 'foo'}},
});
console.log(instance);

classToPlain does not serialize functions

import {
    f, classToPlain
} from '@marcj/marshal';
import "reflect-metadata"

class Config {
    @f.type(String)
    name = "Name of config"

    @f.type(String)
    nameByFunction(): string {
        return "Name of config by function"
    }

}

const arg0 = classToPlain(Config, new Config());
console.log('arg0', JSON.stringify(arg0))

Console log shows only name but not nameByFunction in serialized object:

arg0 {"name":"Name of config"}

Issue with IE11/targeting es5

When running ng serve on a project using @marcj/marshal, I receive the following error in IE11:

Which points to:

I am also targeting es5 in my tsconfig.

Get class name of nested properties for enum

Hi,

I don't know if this is the right place to ask for questions.

Is there a way to get the name of the enum?

class SubModel {
    @f label: string;
    @f classIsArray:boolean = true
}

export enum PlanClass {
    DEFAULT = "DEFAULT",
    PRO = "PRO",
    ENTERPRISE = "ENTERPRISE",
}

class SimpleModel {
    @f.enum(PlanClass, /*allowLabelsAsValue=*/ true)
    plan: PlanClass;

    @f.enum(PlanClass, /*allowLabelsAsValue=*/ true).asArray()
    plans: PlanClass[]

    @f.array(SubModel)
    children: SubModel[] = [];
}
//this works - returns "SubModel"
console.log("class name for children",getClassSchema(SimpleModel).getProperty('children').getResolvedClassSchema().classType.name)

//this doesn't
console.log("enum class name",getClassSchema(SimpleModel).getProperty('plan').getResolvedClassSchema.name)
console.log("enum class name",getClassSchema(SimpleModel).getProperty('plans').getResolvedClassSchema.name)

Thanks in advance

Custom Validator not throwing any errors

Hi,

I tried this code but it doesn't return any error.

class MyCustomValidator implements PropertyValidator {
    validate<T>(value: any): PropertyValidatorError {
        console.log("value.length",value.length,value)
        if (value.length > 10) {
            return new PropertyValidatorError('too_long', 'Too long :()');
        }
    };
}
class SimpleModel {
    @f.primary().uuid()
    id: string = uuid();
    
    @f.validator(MyCustomValidator)
    @f.array(String)
    tags: string[]

    @f.validator(MyCustomValidator)
    tag: string
}

const instance = validatedPlainToClass(SimpleModel, {
    name: 'myName',
    tags: ['foowererererefdrwerer', 'bar'],
    tag: 'foowererererefdrwerer',
});
console.log(instance)

Am I doing something wrong?

Validate doesn't work in Angular 9?

Hi Marc,

I created a new project using Angular 9 and added the code for validation I found in the documentation. The test fails as no error is returned. I am wondering what am I doing wrong here? Am I missing something?

I understand that this may have something to do with Angular.

Repo with the code: https://github.com/ericjeker/marshal-validation

Steps:

ng new marshal-validation
npm install @marcj/marshal reflect-metadata
# create the page.spec.ts file
ng test

page.spec.ts

import {f, validate, ValidationError} from '@marcj/marshal';

class Page {
    @f
    name: string;

    @f
    age: number;
}

describe('Page', () => {

  it('should fail', () => {

  	const errors = validate(Page, {name: 'peter'});
	expect(errors.length).toBe(1);
	expect(errors[0]).toBeInstanceOf(ValidationError);
	expect(errors[0].path).toBe('age');
	expect(errors[0].message).toBe('Required value is undefined');

  });

  it('should not fail', () => {

  	const errors = validate(Page, {name: 'gabriel', age: 42});
	expect(errors.length).toBe(0);

  });

});

[Question] Dependency on typeorm ?

Your package is really interesting to me in its relations to mongo.

One thing confuses me is its relations to typeorm as an intermediary wrapper (or not) for mongo connection and entity handling.

• Is Typeorm a required dependency for other than the createConnection part ?
• Do I still need to annotate my classes with the Entity decorator, or any other decorator from Typeorm, and also still extend from the BaseEntity ?

I see that you go straight for the mongo db instance instead of using the typeorm query manager for your queries.

Why I ask you this ?

We are building an app using typeorm for Mongo, and it has a lot of quirks and inconsistencies as it was built as an afterthought into Typeorm.
We're considering reimplementing it, and I just stubbled upon your package, which seems to almost perfectly fit the bill.

Side note

I know it's bikeshedding, but your inconsistent use of yoda notation confuses me as well 😄 . Is there a reason other than habit from another language ?

Validator not working for array fields

Hello,

I'm having some problem with validator not being invoked.

Here's a test case:

import {
  f,
  PropertyValidatorError,
  PropertyValidator,
  validatedPlainToClass,
  validate,
  plainToClass,
} from '@marcj/marshal';

import { expect } from 'chai';

class NotEmptyArrayValidator implements PropertyValidator {
  validate<T>(value: []): PropertyValidatorError | void {
    console.log('INSIDE VALIDATOR');
    if (value === null || value.length === 0) {
      return new PropertyValidatorError('notEmpty', 'notEmpty');
    }
  }
}

enum Role {
  ADMIN = 'ADMIN',
}

class TestEntity {
  @(f
    .enum(Role, true)
    .asArray()
    .validator(NotEmptyArrayValidator))
  roles: Role[] = [];
}

describe('Empty Array Validator', () => {
  it('should throw error when validatedPlainToClass', async () => {
    const testEntity = validatedPlainToClass(TestEntity, {});
    expect.fail('should have thrown exception');
  });
  it('should return error when validate', function() {
    const testEntity = plainToClass(TestEntity, {});
    const validationResult = validate(TestEntity, testEntity);
    expect(validationResult.length).to.be.equals(1);
  });
});

This test case fail in both "expect"

Am I doing something wrong?

(De)Serialize external class

How to deserialize from json and recreate a class instance from an external class definition?

Let's consider an external Timestamp class that I don't control:
With 2 constructor args: s and n.

export class Timestamp {
   constructor(s: number, n: number);

   static now(): Timestamp;
   static fromDate(date: Date): Timestamp;
   static fromMillis(milliseconds: number): Timestamp;
   
   private readonly seconds: number;
   private readonly nanoseconds: number;
   
   toDate(): Date;
   toMillis(): number;
   isEqual(other: Timestamp): boolean;
   valueOf(): string;
}

So far I have something along the lines of:

   const schema = createClassSchema(Timestamp);
   schema.addProperty("seconds", f.type(Number));
   schema.addProperty("nanoseconds", f.type(Number));

   const json = localStorage.getItem("timestamp");
   const obj = JSON.parse(json);
   const objWithTimestamp = plainToClass(ObjWithTimestamp, obj);

I've seen in the doc that you can decorate method props but I didn't see how to do it for external classes programatically (without decorators)

I've only seen an internal method like and tried

   const s = schema.getOrCreateMethodProperties("s");
   const n = schema.getOrCreateMethodProperties("n");

But then I couldn't figurate how to indicate Marshal ts to use the seconds, nanoseconds and map s,n param names as arguments.

How can I achieve that? If it's possible, could you hint me in the right direction ?

Related to #36

Feature snake_case to camelCase conversion option

I went through docs i couldn't find anything that let me convert snake_case to CamelCase using plainToClass
In many cases json returning from API's are mostly snake_case but my classes contain CamelCase properties. it would be nice to have something like below.
Just a thought can be implement in @f decorator as well to be able to covert in both plainToClass and classToPlain.

e.g const post = plainToClass(Post, postJson, {snakeToCamel: true});

Custom convertors for properties

Hi,
I have certain properties in class which should be an integer.
So is it possible to write custom converters that would convert a value to Integer during plainToClass?

Say for example:

class MyCustomConvertor implements PropertyValidator {
    validate<T>(value: any): PropertyValidatorError {
          return Math.trunc(value)
    };
}
class SimpleModel {
@f.validator(MyCustomConvertor)
    type: number
}

const instance = validatedPlainToClass(SimpleModel, {id:1.1});
console.log(instance) //instance.id will be 1

Status of discriminator support

Thanks for all of your work on this library, I've been searching for a good solution to this problem, and this is very close to all I need. The major feature it's missing for me is "discriminator support" as listed in the TODO section of the README, and so I wondered if there's any work in progress or a timeline/effort estimate for this feature?

Alternatively, is there a plausible workaround that we can implement on our side? I had a look at implementing a custom serialization target, but that didn't strike me as the right tool - although I could easily be missing something.

Thanks!

Error: Invalid ENUM

Hi,
Executing this example https://github.com/marcj/marshal.ts#example-entity
gives an error
Error: Invalid ENUM given in property plan: PRO, valid: 0,1,2
at eval (eval at createXToClassFunction

Firestore - expected plain | mongo

I'm trying to build a classToFirestore and firestoreToClass mapping, however the call for deleteExcludedPropertiesFor only takes "mongo" or "plain" types, and this constraint extends to exclude. So I'm not sure how best to wrangle this, other than to drop the deleteExcludedPropertiesFor, which would mean all the excluded fields would be retained.

TypeError: Cannot add property boolean, object is not extensible

When validating a frozen object, that has a missing prop, get the following error:

TypeError: Cannot add property boolean, object is not extensible

Can see an example here:

https://github.com/moltar/typescript-runtime-type-benchmarks/blob/5990bd77506b6778ba45a4a924ebb8fb83934afe/test/abstract.test.ts#L65

Serialize Class from an external module

What would be the best way to serialize a Class imported from an external module 'library'. BabylonJS for instance.

Would it be possible to create a custom function that creates a new Class instance on deserialization.

[Suggestion / Request] Add validation error code mechanism, and or I18n

In many cases, the client needs to "understand" the error, and not only use the error string as is.

Use cases: show specific ui for specific errors.
i18n the errors on client side.

Custom Property Validator Classes fail in minimized (not uglified) webpack builds

Hi Marc,

Found another nice one. When using Webpack with Terser (mangling disabled), custom validator classes will throw the following error:

Class constructor  cannot be invoked without 'new'

Cause: the Terser plugin compacts those classes into class{..., which causes the isClass check in core.ts to provide a false negative, since that particular function checks for class with an extra trailing space. That trailing space is missing in minified webpack builds.

I'll test a fix and submit a PR to this repo and estdlib.ts shortly.

Feature: Option to keep fields that is not marked with decorators

class-transformer is keeping unknown fields by default and it has an option to disable this behavior. Is it possible to implement something similar?
https://github.com/typestack/class-transformer#enforcing-type-safe-instance

Unable to build any code/example due to reflect-metadata

Hello,

I was hoping to get a little bit of help.

I keep getting this error no matter what I try;
That's on any decorator, the minute I remove any f decorator, everything is fine.

../node_modules/@marcj/marshal/src/decorators.ts:1265
const isMethod = propertyOrMethodName && Reflect.hasMetadata('design:returntype', target, propertyOrMethodName) && !isNumber(parameterIndexOrDescriptor);

I cannot work out what is the problem with my environment. I tried first in a new NestJS app and I had no issues, but even in the most basic Typescript app that I have I get this error.

Even the examples don't work.

import { f, plainToClass, uuid } from '@marcj/marshal';

class SubModel {
    @f label: string = '';
}

export enum Plan {
    DEFAULT,
    PRO,
    ENTERPRISE
}

class SimpleModel {
    @(f.primary().uuid())
    id: string = uuid();

    @f.array(String)
    tags: string[] = [];

    @f.optional() //binary
    picture?: ArrayBuffer;

    @f
    type: number = 0;

    @f.enum(Plan, /*allowLabelsAsValue=*/ true)
    plan: Plan = Plan.DEFAULT;

    @f
    created: Date = new Date();

    @f.array(SubModel)
    children: SubModel[] = [];

    @f.map(SubModel)
    childrenMap: { [key: string]: SubModel } = {};

    constructor(
        @(f.index().asName('name')) //asName is required for minimized code
        public name: string
    ) {}
}

//data comes usually from files or http request
const instance = plainToClass(SimpleModel, {
    name: 'myName',
    tags: ['foo', 'bar'],
    plan: 'PRO',
    created: 'Sat Oct 13 2018 14:17:35 GMT+0200',
    children: [{ label: 'foo' }],
    childrenMap: { foo: { label: 'foo' } }
});
console.log(instance);

package.json

{
  "name": "proton-server",
  "version": "0.0.1",
  "description": "",
  "author": "",
  "private": true,
  "license": "UNLICENSED",
  "scripts": {
    "start": "node -r ts-node/register ./src/server.ts",
    "dev": "nodemon ",
    "build": "tsc",
    "script": "cd src/scripts && ts-node"
  },
  "dependencies": {
    "@marcj/marshal": "^2.1.4"
  },
  "peerDependencies": {
    "reflect-metadata": "^0.1.12"
  },
  "devDependencies": {
    "reflect-metadata": "^0.1.13",
    "@types/node": "^14.0.5",
    "nodemon": "^2.0.3",
    "ts-node": "^8.9.1",
    "typescript": "^3.8.3"
  }
}

tsconfig.json

{
    "compilerOptions": {
        "target": "es2019",
        "module": "commonjs",
        "declaration": true,
        "removeComments": true,
        "emitDecoratorMetadata": true,
        "experimentalDecorators": true,
        "sourceMap": true,
        "outDir": "./dist",
        "baseUrl": "./",
        "incremental": true,
        "esModuleInterop": true,
        "lib": ["es2020"],
        "resolveJsonModule": true
    },
    "exclude": ["node_modules", "dist"]
}

Question about nested refs

Hi Marc,

In the example below, is there a way with the current decorators to make owner: Data.owners.a in the nested objects a reference to the instance in the top level owners object?

Currently when I classToPlain() --> plainToClass() it creates a new instance of Owner each time (each with the same data as owner.a)
i.e. currently...
owners.a !== collections.c.owner
owners.a !== collections.c.items[0].owner

but I need ===

Data
|
|-owners {
    a: Owner
    b: Owner
  }
|
|-collections {
    c: Collection
    d: Collection
        |
        |- owner = Data.owners.a
        |- items [
             0: Item
             |
             |-owner = Data.owners.a
           ]
  }

This is simplified version of the code...

class Data {
  @f.map(Owner) public owners;
  @f.map(Collection) public collections;

  constructor() {}

  // in some init method
  this.owners = {
    a: new Owner;
    b: new Owner;
  }
  this.collections = {
    c: new Collection(this.owner.a);
    d: new Collection(this.owner.b); 
  }
}

class Collection {
  @f.type(Owner) public owner; // *toClass creates new copy of owners.a *
  @f.array(Item) public items;

  constructor(owner) {
    this.owner = owner;
    this.items = [];
  }
  // in some init method
  this.items.push(new Item(this.owner))
}

class Item {
  @f.type(Owner) public owner; // *toClass creates new copy of owners.a *

  constructor(owner) {
    this.owner = owner;
  }
}

Manged to get @ParentReference() with @f.forward working nicely with a parent-child elsewhere in the code, but this is proving a bit more tricky!

Many thanks

Feature: type guarded validation method

Would it be possible to add a type guarded validation method?

A method would either return a validated and typed class (or plain object), or throw if invalid.

Usage:

const res = guard(DataType, data);

// and `res` is now typed

Can’t use with Moment.js

In the following code, the first example fails at runtime, per the comment, while the rest fail to compile:

import moment = require("moment");
import "reflect-metadata";
import { Field } from "@marcj/marshal";

class Foo {
    // Runtime error:
    // Foo::bar type mismatch. Given undefined, but declared is Object or undefined. When you don't declare a type in TypeScript, you need to pass a type in @Field(), e.g. @Field(String).
    // If you don't have a type, use @FieldAny(). If you reference a class with circular dependency, use @Field(forwardRef(() => MyType)).
    @Field()
    bar: moment.Moment;

    @Field(moment)
    baz: moment.Moment;

    @Field(moment.Moment)
    quux: moment.Moment;

    @Field(moment.fn)
    quuz: moment.Moment;
}

I understand that Moment isn’t a class, per se, but then how can I work with instances of it?

marshal-mongo query with $and not generated correctly

This is related to @marcj/[email protected]
While the original repository is no longer online and this is the successor I place it here.

Using the filter() method results in a wrong query when using $and:

original filter: { tags: { '$all': [ 'one', 'two' ] } }
processed filter: { tags: { '$all': [ [], [] ] } }

Here's example code
I use convertPlainQueryToMongo directly here but this will also happen with filter() (which calls convertPlainQueryToMongo!?)

    class Test {
        @f.array(String)
        public tags: string[] = [];
    }

    let tags = ['one', 'two'];

    let filter = {
        tags: { $all: tags }
    };

    console.log('filter plain', filter);

    const fieldNames = {};
    const m = convertPlainQueryToMongo(Test, filter, fieldNames);
    console.log('filter mongo', m);


    // filter plain (expected): { tags: { '$all': [ 'one', 'two' ] } }
    // filter mongo (wrong):    { tags: { '$all': [ [], [] ] } }

I tracked down the Problem in convertQueryToMongo()

                    else if (j === '$in' || j === '$nin' || j === '$all') {
                        fieldNamesMap[targetI] = true;
                        fieldValue[j] = queryValue.map(v => converter(classType, targetI, v));

                        // using this here generates the right query - no idea if this is bad
                        // fieldValue[j] = converter(classType, targetI, queryValue);

I guess this will not be fixed in [email protected], so I have a few questions:

• am I doing something wrong?
• can I circumvent the problem?
• can I use marshal and marshall-mongo from super-hornet already or is it too early?

Support for @Transform

In class-transformer, we have been using the @Transform() annotation to patch jsonb objects when we parse them from Postgres to facilitate migrations, for example adding default values for new fields. Is there anything similar in marshal.ts or deepkit? Thanks :)

Custom validator not working as expected

I would expect this new minLength validation to work along with the enum and asArray but it seems like the .validator is ignored.

function minLength(length= 0) {
    return class implements PropertyValidator {
        validate<T>(value: any[] | "" = "", propertySchema: PropertyCompilerSchema): PropertyValidatorError {
            if(value.length < length){
                return new PropertyValidatorError('Array too short', `Was expecting array length of ${length}`);
            }
        };
    }
}
class Job{
   @f.enum(EmploymentTypes)
        .asArray()
        .validator(minLength(1)) // will ignore this line if validated
    private employmentTypes: EmploymentTypes[];
}

Only When i omit the .enum .asArray the .validator runs.

class Job{
   @f.any().validator(minLength(1)) // will run this line now
    private employmentTypes: EmploymentTypes[];
}

Marshal question - complex objects

Hi,

I'm using the marshal package (which is awesome btw) and I have this case, the issue I'm running into is that when deserializing I'm not getting the child objects as class instances but as anonymous types, so when I try to get for example: simpleModel.foo.x
I'm getting an error because foo and bar are plain data, so at runtime, I see: simpleModel.props.foo.props.x

class BaseClass {
    @f.any() 
    protected props: T;

    constructor(props: T) {
        this.props = props;
    }
}

interface SimpleModelProps {
  foo: SubModel;
  bar: SubModel;
}

interface SubModelProps {
  x: number;
  y: Date;
}

class SubModel {
    constructor(props: SubModelProps) {
        this.props = props;
    }

    public get x(): number {
        return this.props.x;
    }

    public get y(): Date {
       return this.props.y;
    }
}

class SimpleModel extends BaseClass {    

     public get foo(): SubModel {
         return this.props.foo;
     }

     public get bar(): SubModel {
         return this.props.bar;
     }

    constructor(props: SimpleModelProps) {
        super(props);
    }
}

Any idea of what it's wrong with my code?

Thanks in advance!

Support for marshal.ts

We were using marshal.ts before, what is the plan going forward with this project, is it still something supported? Is there documentation for marshal.ts still available?

Thanks

looking for help/contributors?

Are you looking for help/contributors on this lib?

Problem when use validatedPlainToClass with getter in object structure

Hi Marc, how are you ?

We are using this awesome lib!
I have a problem, when we have a class that contains getters, and this getters had in your function some algorithms that use other properties that can be undefined, the process "validatedPlainToClass" fails with the error message:

"Sum constructor is not callable without values. Make sure not to depend on constructor arguments. This is necessary for default value checking."

The code that we are using is:

class MonthYearClass {
        @f
        transDate!: Date;

        @f.type(Number)
        get month(): number {
          return this.transDate.getMonth();
        }

        @f.type(Number)
        get year(): number {
          return this.transDate.getFullYear();
        }
      }

      const jun = new Date(2020, 7, 1);

      const testObj = validatedPlainToClass(MonthYearClass, {
        numberOne: 50,
        numberTwo: 123,
        transDate: jun,
      });

      console.log(testObj instanceof MonthYearClass);
      console.log(testObj.month === jun.getMonth());
      console.log(testObj.year === jun.getFullYear());

Any suggestions about how can I solve this issue ?

Thanks a lot

Compare performance with jackson-js?

class-transformer looks dead. Maybe you could compare with
https://github.com/pichillilorenzo/jackson-js
This lib also supports class polymorphism, would be nice to see performance difference

plainToClass for generic type

Hello! I want to ask if is possible use plainToClass with a generic .

Example:

 public postHTTP<T>(url: string): T {
    const jsonResponse = axios.get(url);

    const response: T = plainToClass(T, jsonResponse);
    return response;
}


public test(): TestResponse {
    const data: TestResponse = this.postHTTP<TestResponse>('myhttpurl');

    console.log(data.Field1);
    console.log(data.Field2);
}

Actually the error is:
'T' only refers to a type, but is being used as a value here.

I try to use your function getClassTypeFromInstance on the generic object but the error is:

Element implicitly has an 'any' type because expression of type '"constructor"' can't be used to index type 'unknown'.
Property 'constructor' does not exist on type 'unknown'.

Cheers

No OSS license

There is no OSS license listed on github.

NPM has the License field as "MIT" but github has no matching information. Code posted on github cannot be assumed to be OSS without an explicit license, and NPM has no evidence of the license text or author's license intent.

Cannot access 'class' before initialization

Hi,

Given the following scenario:

class Parent {
  @f
  name: string;

  @f.array(Child)
  children: Child[];
}

class Child {
  @f
  name: string;

  @f.type(Parent)
  parent: Parent;
}

How do you get around the issue of using a class before its declared?
I'm using this with angular 8
Thanks

example of partialPlainToMongo

Hello,

I am trying to understand how to use the partials, especially in the context of Mongo. Could you provide a quick example of making a partial update of a document in Mongo? There must be something simple that I missing but I am struggling. I got plainToClass to work just fine, they return class instance, but partialPlainToClass returns an object not an instance of a class. And I am failing to understand how to make use of it to update specific fields in a Mongo document.

Thank you

npm

Can we publish this package on npm? I created a PR here to tidy-up the package.json a bit.

Update Nest Package Version

I'm pretty sure I you mention that you no longer use Nest, but I was wondering if we could get a peerDep change on the nest specific package to use >5.0.0 instead of ^5.0.0. I've been using it myself and have had very few problems on v7, other than the install warnings about peer dependency version mismatch. It's something ignorable, just wondering if we could get the update.

Custom converter and validators for a type

This is like a continuation from my question in #12 (comment).

Looking at the source code I figured out a way to to add custom converter compilers to specifics types:

https://gist.github.com/leandro-manifesto/cac78114030756383462b6c7b641aaf2#file-register-ts

It works by overring the default plain -> class and class -> plain converter compilers and the default class checker compiler, but I get the feeling that it can break at any moment if internals change.

Would be nice if a feature like this was provided by Marshal itself.

MessagePack

Do you plan to implement a messagepack package? Do you have a "template" or a recommendation for a package, where i could start to implement messagepack?

Best Regards
Uzlopak

Using decorated classes inside another module

Hello, thanks for your investment in marshal.ts, we really appreciate it.

I was running into an issue when I have a multi-module project.

We have basically packages

• server
• client
• domain

In domain we have shared classes using marshal decoration, and then we serialize and deserialize in client and server. Now it seems like the decorations are not picked up if we try to use marshal from client or server, only when we call marshal from the domain package.

I had to set up wrappers like these in the domain package in order to make it work

export function validatedPlainToClass<T>(classType: ClassType<T>, data: object, parents?: any[]): T {
  return marshal.validatedPlainToClass(classType, data, parents)
}

export function classToPlain<T>(
  classType: ClassType<T>,
  target: T,
  options?: {
    excludeReferences?: boolean
  }
): any {
  return marshal.classToPlain(classType, target, options)
}

It seems to be the same issue as they have in class-transformer

typestack/class-transformer#337

I was thinking maybe you knew why this is an issue, it seems very strange to me!

Firestore mapper - expected (plain | mongo) in argument?

I'm trying to build a classToFirestore and firestoreToClass mapping, however the call for deleteExcludedPropertiesFor only takes "mongo" or "plain" target, and this constraint extends to exclude. So I'm not sure how best to wrangle this, other than to drop the deleteExcludedPropertiesFor, which would mean all the excluded fields would be retained.

Or forking the whole project?

Advice would be really appreciated!

ReferenceError: Can't find variable: global ./node_modules/buffer/index.

I am not sure if this is a bug or maybe I missed something on the install.

I am testing this as a replacement to typestack/class-transformer. When using marshal.ts in an Ionic 5 Angular application I am getting.

Error: Uncaught (in promise): ReferenceError: Can't find variable: global ./node_modules/buffer/index.

const serviceData = await this.dataService.getUserOptions();
const results = plainToClass(UserOptions, serviceData);
console.log(results);

If I swap the plainToClass back to typestack/class-transformer I don't get this error.

   Ionic CLI                     : 6.3.0 (/usr/local/lib/node_modules/@ionic/cli)
   Ionic Framework               : @ionic/angular 5.0.5
   @angular-devkit/build-angular : 0.803.25
   @angular-devkit/schematics    : 8.3.25
   @angular/cli                  : 8.3.25
   @ionic/angular-toolkit        : 2.2.0

Possibly related to this? angular/angular-cli#9827 (comment)

Enums are broken in webpack builds > 2.1.3

Hey there,

I recently upgraded to one of the last versions you published (2.1.5). On the face of it, in a classic tsc build, everything works fine. However, when I run a webpack build (e.g. for my serverless deploys), my classes annotated with a classic enum throw the following error on startup:

Error: GJOriginConfig::logo type mismatch. Given Object, but declared is Object or undefined. The actual type is an Object, but you specified a Class in @f.type(T).
Please declare a type or use @f.map(Object) for 'logo: {[k: string]: Object}'.

Reverted to 2.1.3, and all is well. I annotate them as you usually would with marshal.ts, nothing fancy going on.

I'm woefully short on time but will try to set up a repo with a repro for this, unless you already have an idea of what might be going on. Please let me know!

More strict handling of Date object

  @Field(Date)
  public end_time: Date;

When i'm passing "start_time": "",
I get no validation error, but null. same for when passing blah string.
When passing number.
Is there a way to make it accept only JSONed date objects?

Marshal.ts JIT crash on validation of keys with `:`, `-` or `#`

Minimal repro

// index.ts
import 'reflect-metadata';
import { validate, f } from '@marcj/marshal';

class Test {
  constructor() {}
  @f public 'some:prop'?: string;
}

const data = {
  'some:prop': 'hello',
};

const validated = validate(Test, data);

will result in

jit code
        return function(_data, _path, _errors) {
            _path = _path ? _path + '.' : '';
            _errors = _errors || [];

        var value = _data.some:prop;
        if (undefined !== value && null !== value) {

            //property some:prop, string
            if ('string' !== typeof value) _errors.push(new ValidationError(_path + 'some:prop', "invalid_string", "No string given"));


        } else {
            _errors.push(new ValidationError(_path + 'some:prop', 'required', 'Required value is undefined or null'));
        }

            return _errors;
        }


/Users/hilkeheremans/projects/thinman/repro/node_modules/@marcj/marshal/src/jit-validation.ts:218
        const compiled = new Function(...context.keys(), functionCode);
                                                         ^
SyntaxError: Unexpected token ':'
    at new Function (<anonymous>)
    at Object.jitValidate (/Users/hilkeheremans/projects/thinman/repro/node_modules/@marcj/marshal/src/jit-validation.ts:218:58)
    at Object.validate (/Users/hilkeheremans/projects/thinman/repro/node_modules/@marcj/marshal/src/validation.ts:97:12)
    at Object.<anonymous> (/Users/hilkeheremans/projects/thinman/repro/src/index.ts:13:19)
    at Module._compile (internal/modules/cjs/loader.js:1158:30)
    at Module.m._compile (/usr/local/Cellar/nvm/0.34.0/versions/node/v12.9.1/lib/node_modules/ts-node/src/index.ts:858:23)
    at Module._extensions..js (internal/modules/cjs/loader.js:1178:10)
    at Object.require.extensions.<computed> [as .ts] (/usr/local/Cellar/nvm/0.34.0/versions/node/v12.9.1/lib/node_modules/ts-node/src/index.ts:861:12)
    at Module.load (internal/modules/cjs/loader.js:1002:32)
    at Function.Module._load (internal/modules/cjs/loader.js:901:14)

var value = _data.some:prop; is probably to blame here, aka https://github.com/marcj/marshal.ts/blob/master/packages/core/src/jit-validation.ts#L192 ?

JIT compiled validators should be much much faster

In the intro you perform a comparison against class-transformer which is probably at the lowest end of validators in terms of performance...

You should actually compare to other JIT libraries and not to composition library.

ts-json-validator (avj) or quartet

They both have a huge advantage over marshal.ts

Since the top 3 libraries are JIT, you can see the gap.

Note that JIT libraries do excel at benchmarks, less at realtime world scenarios since JIT libraries LOVE it when the same validator function is running over and over, getting inlined all the way.

---

BTW, I believe that if you can shorten the amount of function calls done from calling to perform validation and until returning a result you will get better benchmark results.

Cross field validation

First of all I want to congratulate to this awesome lib! The integration was easy due to the documentation 👍
I started to implement some custom validation for specific fields, but I have a use case where the validation logic depends on the data of other fields. Is it somehow possible to access these other fields to implement cross field validation?

Dependency on `typeorm`

I know there's a similar issue to this (or perhaps it's pretty much the same), but for my sanity, I'm wondering why typeorm is a dependency. typeorm + mongodb has so many issues. I want to plug this into my https://github.com/j/type-mongodb library (ironically, when I started this mapper, my first idea was to do JIT as well, but didn't to keep it simple for now). WIth this package, I'm going to switch it over to using Marshal.ts. With that said, it'd be nice to not be able to pull in typeorm when using the mongo mapper utility functions. Maybe marshal-mongo should just be the core of mongo logic and can have a marshal-typeorm package utilizing it.

Filtering properties with PartialClassToPlain

Hi Marc,

Thanks for this awesome lib!
I have a feature request, I Thought I could use partialPlainToClass with @f.exclude but doesn't really cover my need.

I would like to filter all nested params based on some filter name. Example:

class Book {
  @f.filter('myFilter')
  title:string

  @f
  isbn: string
}

class Reader {
  @f.filter(['myFilter', 'myAgeFilter'])
  name:string

  @f.filter(['myAgeFilter'])
  age: int
  
  @f.array(Book).filter("myFilter")
  books: Book[]
}

const reader:Reader =  plainToClass(Reader, {
  name: 'rafael',
  age: 34,
  books: [ 
     { title: "Lean Startup", isbn: "xxxx-yyyy-zzzz" },
     { title: "Traction", isbn: "zzzz-xxxx-yyyy" },
  ]
});

I need to convert this reader back into a plain object, but only containing some of its properties (and nested properties), based on the filter name. I would expect something like this:

const filteredReader = partialClassToPlain(Reader, reader, 'myFilter');
{ 
  name: 'rafael',
  books: [ 
    {title: 'Lean Startup'}, 
    {title: 'Traction'}
  ] 
}
const filteredReader2 = partialClassToPlain(Reader, reader, 'myAgeFilter');
{ 
  name: 'rafael',
  age: 34
}

There is a way to achieve this with the current api? If there's not, does this make sense to you?

I Want to use this to send json through an API, only sending the pertinent fields to each URL.