GithubHelp home page GithubHelp logo

simple-runtypes's Introduction

npm version unit-tests npm-publish

Preface

I said I want SIMPLE runtypes. Just functions that validate and return data. Combine them into complex types and TypeScript knows their structure. That's how runtypes work.

Install

# npm
npm install simple-runtypes

# yarn
yarn add simple-runtypes

Example

  1. Define the Runtype:
import * as st from 'simple-runtypes'

const userRuntype = st.record({
  id: st.integer(),
  name: st.string(),
  email: st.optional(st.string()),
})

now, ReturnType<typeof userRuntype> is equivalent to

interface {
  id: number
  name: string
  email?: string
}
  1. Use the runtype to validate untrusted data
userRuntype({ id: 1, name: 'matt' })
// => {id: 1, name: 'matt'}

userRuntype({ id: 1, name: 'matt', isAdmin: true })
// throws an st.RuntypeError: "invalid field 'isAdmin' in data"

Invoke a runtype with use to get a plain value back instead of throwing errors:

st.use(userRuntype, { id: 1, name: 'matt' })
// => {ok: true, result: {id: 1, name: 'matt'}}

st.use(userRuntype, { id: 1, name: 'matt', isAdmin: true })
// => {ok: false, error: FAIL}

st.getFormattedError(FAIL)
// => 'invalid keys in record: ["isAdmin"] at `<value>` in `{"id":1,"name": "matt", ... }`'

Not throwing errors is way more efficient and less obscure.

Throwing errors and catching them outside is more convenient:

try {
  ... // code that uses runtypes
} catch (e) {
  if (st.isRuntypeError(e)) {
    console.error(getFormattedError(e))

    return
  }

  throw e
}

Why?

Why should I use this over the plethora of other runtype validation libraries available?

  1. Strict: by default safe against __proto__ injection attacks and unwanted properties
  2. Fast: check the benchmark
  3. Friendly: no use of eval, and a small footprint with no dependencies
  4. Flexible: optionally modify the data while it's being checked - trim strings, convert numbers, parse dates

Benchmarks

@moltar has done a great job comparing existing runtime type-checking libraries in moltar/typescript-runtime-type-benchmarks.

@pongo has benchmarked simple-runtypes against io-ts in pongo/benchmark-simple-runtypes.

Documentation

Intro

A Runtype is a function that:

  1. receives an unknown value
  2. returns that value or a copy if all validations pass
  3. throws a RuntypeError when validation fails or returns ValidationResult when passed to use
interface Runtype<T> {
  (v: unknown) => T
}

Runtypes are constructed by calling factory functions. For instance, string creates and returns a string runtype. Check the factory functions documentation for more details.

Usage Examples

Nesting

Collection runtypes such as record, array, and tuple take runtypes as their parameters:

const nestedRuntype = st.record({
  name: st.string(),
  items: st.array(st.record({ id: st.integer, label: st.string() })),
})

nestedRuntype({
  name: 'foo',
  items: [{ id: 3, label: 'bar' }],
}) // => returns the same data

Strict Property Checks

When using record, any properties which are not defined in the runtype will cause the runtype to fail:

const strict = st.record({ name: st.string() })

strict({ name: 'foo', other: 123 })
// => RuntypeError: Unknown attribute 'other'

Using record will keep you safe from any __proto__ injection or overriding attempts.

Ignore Individual Properties

To ignore individual properties, use ignore, unknown or any:

const strict = st.record({ name: st.string(), other: st.ignore() })

strict({ name: 'foo', other: 123 })
// => {name: foo, other: undefined}

Optional Properties

Use the optional runtype to create optional properties:

const strict = st.record({
  color: st.optional(st.string()),
  width: st.optional(st.number()),
})

Non-strict Property Checks

Use nonStrict to only validate known properties and remove everything else:

const nonStrictRecord = st.nonStrict(st.record({ name: st.string() }))

nonStrictRecord({ name: 'foo', other: 123, bar: [] })
// => {name: foo}

Discriminating Unions

simple-runtypes supports Discriminating Unions via the union runtype.

The example found in the TypeScript Handbook translated to simple-runtypes:

const networkLoadingState = st.record({
  state: st.literal('loading'),
})

const networkFailedState = st.record({
  state: st.literal('failed'),
  code: st.number(),
})

const networkSuccessState = st.record({
  state: st.literal('success'),
  response: st.record({
    title: st.string(),
    duration: st.number(),
    summary: st.string(),
  }),
})

const networdStateRuntype = st.union(
  networkLoadingState,
  networkFailedState,
  networkSuccessState,
)

type NetworkState = ReturnType<typeof networkStateRuntype>

Finding the runtype to validate a specific discriminating union with is done efficiently with a Map.

Custom Runtypes

Write your own runtypes as plain functions, e.g. if you want to turn a string into a BigInt:

const bigIntStringRuntype = st.string({ match: /^-?[0-9]+n$/ })

const bigIntRuntype = st.runtype((v) => {
  const stringCheck = st.use(bigIntStringRuntype, v)

  if (!stringCheck.ok) {
    return stringCheck.error
  }

  return BigInt(stringCheck.result.slice(0, -1))
})

bigIntRuntype('123n') // => 123n
bigIntRuntype('2.2') // => error: "expected string to match ..."

Reference

Basic runtypes that match JavaScript/TypeScript types:

Meta runtypes:

Objects and Array Runtypes:

Combinators:

Shortcuts:

Roadmap / Todos

  • size - a meta-runtype that imposes a size limit on types, maybe via convert-to-json and .length on the value passed to it
  • rename stringLiteralUnion to literals or literalUnion and make it work on all types that literal accepts
  • rename record to object: #69
  • improve docs:
    • preface: what is a runtype and why is it useful
    • why: explain or link to example that shows "strict by default"
    • show that simple-runtypes is feature complete because it can
      1. express all TypeScript types
      2. is extendable with custom runtypes (add documentation)
    • add small frontend and backend example projects that show how to use simple-runtypes in production
  • test all types with tsd
  • add more combinators: partial, required, get, ...
  • separate Runtype and InternalRuntype and type runtype internals (see this comment)

current tasks (metadata) notes

  • check that intersection & union tests do properly test the distribution stuff
  • make getMetadata public
  • maybe make metadata typed and include all options so that you can walk the tree to create testdata orjson-schemas from types
  • maybe add a serialize function to each runtype too? to use instead of JSON.stringify and to provide a full-service library?
  • maybe make any a forbidden type of a runtype

simple-runtypes's People

Contributors

dependabot[bot] avatar grebaldi avatar hoeck avatar mandx avatar pabra avatar remnantkevin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

simple-runtypes's Issues

Add lazy or some way to create recursive types

zod etc have a lazy combinator that allows creating recursive validators.

For example, given this TS type:

type Node = {
  label: string;
  children: Node[];
};

There is no way to define this with simple-runtypes

optional runtype isn't really optional on records

Your first example in the readme actually isn't correct (anymore?).
The email key isn't really optional in a way that it can be absent too. See this codesandbox.

For the runtype check it seems to be fine, if the email key is absent. But in the resulting typescript type (ReturnType<typeof userRuntype>) the key isn't marked optional.

I would expect this Type:

type User = {
    id: number;
    name: string;
    email?: string | undefined;
}

but actually it's this:

type User = {
    id: number;
    name: string;
    email: string | undefined;
}

Question: What is `internalRuntype` for?

Just browsing through the code, I noticed the internalRuntype thing. I haven't looked to deeply, but my initial concern was that the library might be using mechanisms that weren't available to normal users like me. Is that the case?

(I'm looking at various TS validation libraries. I initially just picked Zod but that was a mistake! I'm optimistic about simple-runtypes because it's strict by default and allows you to transform+validate together.)

rename `record` to `object`

Rename record to object because that seems to be the standard name for "interface" runtypes.

Drop the existing object runtype completely or rename it to plainObject or javascriptObject.
Keep record an alias for object for a few versions.

`trim` string option causes optional fields to be returned with `undefined` values

See the following test cases:

(1) Without trim option - optional fields are correctly not returned as they are not defined in the input.

// Returns { ok: true, result: {} } ✅
st.use(st.partial(st.record({ name: st.string({ trim: false }), other: st.string() })), {})

(2) With trim option - all optional fields incorrectly returned with undefined values.

// Returns { ok: true, result: { name: undefined, other: undefined } } ❌
st.use(st.partial(st.record({ name: st.string({ trim: true }), other: st.string() })), {})

Many libraries will treat the two objects differently which can cause downstream issues.

`sloppyRecord` in `array` doesn't remove extra properties

In the following example I would have expected the age property (eventInput.attendees[0].age) to be removed, and so not be present in event.result, because it is not specified in the attendees array of SloppyEvent (it is an extra property). However, the age property is present in the result.

const SloppyEvent = st.sloppyRecord({
  title: st.string(),
  attendees: st.array(st.sloppyRecord({ name: st.string() }))
})
const eventInput = { title: "Superbowl", attendees: [{ name: "Joe", age: 5 }] };
const event = st.use(SloppyEvent, eventInput);
assert.deepEqual(event, { ok: true, result: eventInput }); // ✗ 'attendees[0].age' property not removed

See online reproduction here (on replit.com).

I'm new to simple-runtypes so please ignore if I've misunderstood how the above should work.

Performance and throwing errors

Hello. You've created an good library, but there is a performance problem.

Look at this benchmark. We validate two cases: correct data and incorrect data. io-ts in both cases has approximately the same performance. But simple-runtypes has a significant performance decrease on incorrect data.

In my opinion, this is because you are throwing errors. The standard error should collect a stacktrace — it's slow. Throwing is also slow.


What can we do?

1. Throw away Error and use stackless error objects

import { inherits } from 'util';

export class RuntypeError {
  readonly name: string
  readonly path?: (string | number)[]
  readonly value?: any

  // eslint-disable-next-line @typescript-eslint/explicit-module-boundary-types
  constructor(message: string, value?: any, path?: (string | number)[]) {
    this.name = 'RuntypeError'
    this.path = path
    this.value = value
  }
}

inherits(RuntypeError, Error); // new RuntypeError() instanceof Error === true

2. Stop throwing errors

You can, for example, return a ValidationResult:

type ValidationResult<T> = 
  | { ok: true; value: T }
  | { ok: false; error: RuntypeError };

RuntypeUsageError: boolean literals inside a union

I believe I've found a bug adding boolean literals inside a union, ex.:

import * as st from 'simple-runtypes';
const d = st.union(st.record({ d: st.literal(false) }), st.record({ d: st.literal(true) }));

At runtime I get the following error in logs

257 | 
258 | var RuntypeUsageError = /*#__PURE__*/function (_Error2) {
259 |   _inheritsLoose(RuntypeUsageError, _Error2);
260 | 
261 |   function RuntypeUsageError() {
262 |     return _Error2.apply(this, arguments) || this;
               ^
error: broken record type definition, function (v, failOrThrow) {
Complete error
257 | 
258 | var RuntypeUsageError = /*#__PURE__*/function (_Error2) {
259 |   _inheritsLoose(RuntypeUsageError, _Error2);
260 | 
261 |   function RuntypeUsageError() {
262 |     return _Error2.apply(this, arguments) || this;
               ^
error: broken record type definition, function (v, failOrThrow) {
    if (typeof v !== "object" || Array.isArray(v) || v === null)
      return createFail(failOrThrow, "expected an object", v);
    var o = v;
    var res = isPure ? o : {};
    for (var i = 0;i < typemapKeys.length; i++) {
      var k = typemapKeys[i];
      var t = typemapValues[i];
      var value = t(o[k], failSymbol);
      if (isFail(value)) {
        if (!(k in o))
          return createFail(failOrThrow, "missing key in record: " + debugValue(k));
        return propagateFail(failOrThrow, value, v, k);
      }
      if (!isPure)
        res[k] = value;
    }
    if (!sloppy) {
      var unknownKeys = [];
      for (var _k in o)
        if (!Object.prototype.hasOwnProperty.call(typemap, _k))
          unknownKeys.push(_k);
      if (unknownKeys.length)
        return createFail(failOrThrow, "invalid keys in record: " + debugValue(unknownKeys), v);
    }
    return res;
  }[d] must be a string or number, not false
      at new RuntypeUsageError (/home/runner/GainsboroDismalDiscussion/node_modules/.pnpm/[email protected]/node_modules/simple-runtypes/dist/simple-runtypes.esm.js:262:11)
      at /home/runner/GainsboroDismalDiscussion/node_modules/.pnpm/[email protected]/node_modules/simple-runtypes/dist/simple-runtypes.esm.js:1128:12
      at internalDiscriminatedUnion (/home/runner/GainsboroDismalDiscussion/node_modules/.pnpm/[email protected]/node_modules/simple-runtypes/dist/simple-runtypes.esm.js:1119:2)
      at /home/runner/GainsboroDismalDiscussion/index.ts:11:10

Is this project live?

Looks like this project is exactly what i need. But i cannot understand project status correctly. I see an issues with merged PRs, but it was not released for months.

Omit does not work on intersections

[Node] ~/backend/node_modules/simple-runtypes/src/omit.ts:15
[Node]     throw new RuntypeUsageError(`expected a record runtype`)
[Node]           ^
[Node] Error: expected a record runtype
[Node]     at Object.omit (~/backend/node_modules/simple-runtypes/src/omit.ts:15:11)

nonStrict modifier

Change the existing sloppyRecord runtype functionality into a nonStrict combinator that works on record runtypes, similar to how pick works atm.

Before:

import * as st from 'simple-runtypes'

const item = st.sloppyRecord({
  id: st.number(),
  label: st.string(),
})

After:

import * as st from 'simple-runtypes'

const item = st.nonStrict(
    st.record({
        id: st.number(),
        label: st.string(),
    }),
)

Reasons:

  • "sloppy" has a bad connotation that looks weird when spread out all over runtype definitions
  • using a modifier function allows for separating the record definition from the way its parsed
  • it is still possible to mix nonStrict with strict objects in a single runtype

Publish modern ESNext/ES2022 code

Right now the code is compiled to ES5 which is a lot more verbose than needed.

Can the esm.js file at least be compiled to ES6 if not esnext?

Incorrect error message for invalid keys in a `record`

The error message that is shown when a record runtype finds invalid keys is incorrect.

As far as I can see this bug was introduced in v7.0.0, specifically in commit 14bac166c.

Below are examples showing the incorrect error message in the current version (v7.1.2). See reproduction on replit.

import * as st from "simple-runtypes"

const recordRT = st.record({
  a: st.string()
})

console.log(st.use(recordRT, { a: "ay" }))
// => { ok: true, result: { a: 'ay' } }

const result1 = st.use(recordRT, { a: "ay", b: "bee" })
console.log(result1)
// => {
//      ok: false,
//      error: {
//        reason: 'invalid keys in record [{"a":"ay","b":"bee"}]',
//        path: [],
//        value: { a: 'ay', b: 'bee' },
//        [Symbol(SimpleRuntypesFail)]: true
//      }
//    }
console.log(st.getFormattedError(result1.error)) // (A)
// => invalid keys in record [{"a":"ay","b":"bee"}] at `<value>` for `{"a":"ay","b":"bee"}`


const nestedRecordRT = st.record({ 
  a: st.string(),
  b: st.record({ 
    c: st.string()
  })
})

console.log(st.use(nestedRecordRT, { a: "ay", b: { c: "cee" } }))
// => { ok: true, result: { a: 'ay', b: { c: 'cee' } } }

const result2 = st.use(nestedRecordRT, { a: "ay", b: { c: "cee", d: "dee" } })
console.log(result2)
// => {
//      ok: false,
//      error: {
//        reason: 'invalid keys in record [{"c":"cee","d":"dee"}]',
//        path: [ 'b' ],
//        value: { a: 'ay', b: [Object] },
//        [Symbol(SimpleRuntypesFail)]: true
//      }
//    }
console.log(st.getFormattedError(result2.error)) // (B)
// => invalid keys in record [{"c":"cee","d":"dee"}] at `<value>.b` for `{"c":"cee","d":"dee"}`

const result3 = st.use(nestedRecordRT, { a: "ay", b: { c: "cee" }, e: "eee" })
console.log(result3)
// => {
//      ok: false,
//      error: {
//        reason: 'invalid keys in record [{"a":"ay","b":{"c":"cee"},"e":"eee"}]',
//        path: [],
//        value: { a: 'ay', b: [Object], e: 'eee' },
//        [Symbol(SimpleRuntypesFail)]: true
//      }
//    }
console.log(st.getFormattedError(result3.error)) // (C)
// => invalid keys in record [{"a":"ay","b":{"c":"cee"},"e":"eee"}] at `<value>` for `{"a":"ay","b":{"c":"cee"},"e":"eee"}`

The error messages should read:

(A) invalid keys in record ["b"] at `<value>` for `{"a":"ay","b":"bee"}`
(B) invalid keys in record ["d"] at `<value>.b` for `{"c":"cee","d":"dee"}`
(C) invalid keys in record ["e"] at `<value>` for `{"a":"ay","b":{"c":"cee"},"e":"eee"}`

Chain custom runtimes with other runtypes

How can I use custom runtimes (that transform a value into another type) with other runtypes?

For example, if I wanted to verify an optional BigInt type:

st.optional(bigIntRuntype()) // 🛑

This would fail because bigIntRuntype() returns bigint whereas st.optional() expects Runtype<unknown> as a parameter.

With the built-in runtypes, it works. Example:

st.optional(stringAsInteger()) // ✅

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.