openreferral / deprecated-hsds-1.1-validator Goto Github PK

Right now the validator API returns a 400 Bad Request whenever a request is made with a file that doesn't meet validation requirements. This was pretty confusing because 400s are meant to be returned for malformed syntax, not semantic issues or otherwise syntactically valid requests. This error code led me to believe there was something malformed about my file or HTTP request, and it took about an hour of debugging and head-scratching for us to realize that the request and the CSV themselves were well-formed but that our CSV just didn't pass the validation logic.

Notably, when using curl you probably won't see the 400 status code response unless you're using the verbose flag, and you will see the validation message; however, when using Postman or in our case Ruby HTTP clients, only the error is surfaced, not the validation message in the body.

I'd advocate for updating the validation API to return a different error status code (e.g. 422 - semantically bad request), or to return a 200 with the validation message in the body. I'm suggesting a 200 because the API request itself is actually valid: the endpoint is performing the function we expect, which is checking the received entity against some logic, and the response includes information that we've asked for (the validation message). Therefore as a client, everything is OK, I've just now gotten info that I have work to do.

Thoughts?

http://localhost:1400/validate/datapackage?uri=https://raw.githubusercontent.com/openreferral/sample-data/master/simple/datapackage.json

The connection was reset

If I host the package myself, it validates fine. I suspect this has something to do with GitHub's headers.

Error log:

$ docker logs openreferral-validator

> [email protected] start /app
> node src/app.js

info: Server launched [0.0.0.0:8004]
/app/node_modules/csv-parse/lib/index.js:369
      })(this))) || (l - i <= this.options.delimiter.length && this.options.delimiter.substr(0, l - i) === remainingBuffer) || (l - i <= this.options.escape.length && this.options.escape.substr(0, l - i) === remainingBuffer)) {
                                                     ^

TypeError: Cannot read property 'length' of null
    at Parser.__write (/app/node_modules/csv-parse/lib/index.js:369:54)
    at Parser._transform (/app/node_modules/csv-parse/lib/index.js:207:14)
    at Parser.Transform._read (_stream_transform.js:186:10)
    at Parser.Transform._write (_stream_transform.js:174:12)
    at doWrite (_stream_writable.js:396:12)
    at writeOrBuffer (_stream_writable.js:382:5)
    at Parser.Writable.write (_stream_writable.js:290:11)
    at IncomingMessage.ondata (_stream_readable.js:639:20)
    at emitOne (events.js:121:20)
    at IncomingMessage.emit (events.js:211:7)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start: `node src/app.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-07-15T15_39_36_668Z-debug.log

@spilio would it be hard to change the name to hsds_validator?