openknowledge-archive / datapackage-validate-js

[DEPRECATED] Replacement: https://github.com/frictionlessdata/datapackage-js

datapackage-validate

Validate Data Package datapackage.json files.

Installation

npm install datapackage-validate

Usage

Following assume you've imported the module as follows:

var validator = require('datapackage-validate');

validate

Validate the provided object or string as a Data Package.

validator.validate(raw, schema)

• raw: datapackage.json string or object to validate (note method will take care of parsing the string and checking it is valid JSON if it is not parsed already)
• schema: A schema to validate raw against, or, a string that is the unique identifier for a schema in the Data Package Registry, which is used to retrieve the matching schema

Note the method is asynchronous and returns a Promise (implemented with bluebird). The Promise resolves with following structure:

{
  valid: true | false,
  errors: [
    {
      // every error has a message
      message: 'Invalid JSON: ...'
      // JSON errors come from json-lint and will also have
      line: 
    },
    {
      message: 'Array is too short (0), minimum 1',
      // schema errors come from schema validator and include additiona
      // path in input JSON
      dataPath: '/resources',
      // path in schema
      schemaPath: '/properties/resources/minItems',
    },
    ...
  ]
  warnings: [
    {
      message: 'No title field'
    },
    ...
  ]
};

Changelog

• v0.3.0:
  • Adds support for any Profile in the Data Package Registry
  • API is now async using bluebird Promises
  • Removes hardcoded schemas
  • Removes validateUrl
• v0.2.0: #1, #2, #4 (jsonlint, much better schema validation using official schemas)
• v0.1.0: first working release

datapackage-validate-js's Issues

Switch from JSV to another schema validation library

JSV does not support draft v4 of the spec (the latest one) which we need.

Review of JS JSON validation libraries here: http://cosmicrealms.com/blog/2014/02/07/benchmark-of-node-dot-js-json-validation-modules-part-2/

Shortlist:

• tv4 - https://github.com/geraintluff/tv4
• jayschema - https://github.com/natesilva/jayschema

jaySchema has Slightly cleaner API but is slower and tv4 output is slightly nicer (good stuff in both but tv4 output is nicer)

Output

tv4 output

{ errors: 
   [ { message: 'Array is too short (0), minimum 1',
       code: 400,
       dataPath: '/resources',
       schemaPath: '/properties/resources/minItems',
       subErrors: null,
       stack: 'Error: Array is too short (0), minimum 1\n  .... 
    } ],
  missing: [],
  valid: false }

jaySchema output

[ { instanceContext: '#/resources',
    resolutionScope: 'anon-schema://c91ec96e0499fb50b671e6a7a575ef3bbe467607/#/properties/resources',
    constraintName: 'minItems',
    constraintValue: 1,
    testedValue: 0,
    kind: 'ArrayValidationError' } ]

Publish v0.3 to NPM

Done, and ready to publish:

• Changes merged: #7
• Docs updated: https://github.com/okfn/datapackage-validate

Datapackage viewer should provide all the validator info from Datapackage validator

From @danfowler on August 21, 2015 8:24

For example, when trying to view this datapackage.json, I get the following error:

There was an error.

datapackage.json is invalid JSON. Details: Unexpected string

Much richer error info can be found in the validator (including the offending line number), but the viewer doesn't display it, nor does it link to the validator result.

Copied from original issue: frictionlessdata/frictionlessdata.io#193

Removal of validateUrl - logic for it

@pwalsh why did we remove validateUrl?

I ask as it lead to breakage upstream in data.okfn.org - not the end of the world - but just wondered on motivation :-)

Use the schemas at https://github.com/dataprotocols/schemas

Use schema repo as submodule https://github.com/dataprotocols/schemas and load schemas from there.

Use jsonlint to give more useful json errors

Default node json parser has very uninformative json errors (basically: "there's an error").

One can do much better using https://github.com/codenothing/jsonlint

datapackage-validate-js is not strict enough

Hello,

https://github.com/datasets/gdp/tree/4149e0b5612719d6a1332edf868cda634b49781a is said to be an invalid datapackage
see datasets/gdp#5

but

http://data.okfn.org/tools/validate?url=https%3A%2F%2Fgithub.com%2Fdatasets%2Fgdp%2F

shows it as valid.

validate is not strict enough

Kind regards

Refactor to use registry, and to take custom schemas

@rgrp /cc @gvidon

I'd like to suggest we make the following changes to datapackage-validate:

• Remove schemas submodule
• Instead, get our "official" schemas via the new datapackage-registry module
• Don't read from a filepath, but read a schema object. For schemas in our registry, could pass a key which is the unique id of the schema

Goals

• Package works seamlessly in browser and server environments (doesn't have to read a file from path)
• Package always stays up-to-date with changes to the core Data Package schemas
• Package has instant access to any new schemas in our registry
• Package can be used with any other custom Data Package schemas that a user may have

Cons

• The only con is see is that datapackage-registry gets the "official" schemas via an API call over the web. I think it is not really a problem though - if the user does not want to rely on having a connection, that user could simply keep local copies of the schemas as part of their codebase, as we are adding the ability to load any schema

Pros

• Because of the API call to the schemas - the official schemas are always up-to-date, and new schemas are seamlessly integrated :)

check files exist at specified paths

Check that files listed in resources exist at the specified paths