blueoakjs / blueoak-server Goto Github PK

Ziggy (from Randstad) had built a service which he called 'auth'.
This name clearly collided with the built-in auth service in sprout server.
Here's the error:

info: Starting sprout-server v1.0.1 in development mode
info: Clustering is disabled
job-search is still initializing. Check that the callback is being called
debug: express-monitor - Enabled express monitor.
debug: csrf - Enabled CSRF protection.
debug: body-parser - Enabled body parser for type json.
debug: cors - Enabled CORS.
TypeError: undefined is not a function
  at C:\MyMobile\DEV\Recruiter\server\node_modules\sprout-server\handlers\_routes.js:60:31
  at Array.forEach (native)
  at registerDeclarativeRoutes (C:\MyMobile\DEV\Recruiter\server\node_modules\sprout-server\handlers\_routes.js:18:20)
  at exports.init (C:\MyMobile\DEV\Recruiter\server\node_modules\sprout-server\handlers\_routes.js:6:5)
  at initConsumer (C:\MyMobile\DEV\Recruiter\server\node_modules\sprout-server\lib\loader.js:269:37)
  at C:\MyMobile\DEV\Recruiter\server\node_modules\sprout-server\node_modules\async\lib\async.js:162:20
  at iterate (C:\MyMobile\DEV\Recruiter\server\node_modules\sprout-server\node_modules\async\lib\async.js:256:13)
  at Object.async.forEachOfSeries.async.eachOfSeries (C:\MyMobile\DEV\Recruiter\server\node_modules\sprout-server\node_modules\async\lib\async.js:27
5:9)
  at Object.async.forEachSeries.async.eachSeries (C:\MyMobile\DEV\Recruiter\server\node_modules\sprout-server\node_modules\async\lib\async.js:211:22
)
  at Object.initConsumers (C:\MyMobile\DEV\Recruiter\server\node_modules\sprout-server\lib\loader.js:240:19)
Startup failed [TypeError: undefined is not a function]

sprout server should give a better error message when there is a name collision between services (especially from user services to built-in services)

When a NPM module is declared as a handler in the config JSON, that module's dependencies fail to load:

Error: Error loading dependency meticulous for handlers.sprout-wcm-handler: Cannot find module 'meticulous'
    at /Users/Work/Documents/sfi-website/sfi-esb/node_modules/sprout-server/lib/loader.js:76:31
    at Array.forEach (native)
    at fetchUnmetDependencies (/Users/Work/Documents/sfi-website/sfi-esb/node_modules/sprout-server/lib/loader.js:71:18)
    at Object.init (/Users/Work/Documents/sfi-website/sfi-esb/node_modules/sprout-server/lib/loader.js:269:17)
    at initServices (/Users/Work/Documents/sfi-website/sfi-esb/node_modules/sprout-server/index.js:249:19)
    at /Users/Work/Documents/sfi-website/sfi-esb/node_modules/sprout-server/index.js:60:24
    at /Users/Work/Documents/sfi-website/sfi-esb/node_modules/sprout-server/index.js:250:9
    at /Users/Work/Documents/sfi-website/sfi-esb/node_modules/sprout-server/lib/loader.js:327:17
    at /Users/Work/Documents/sfi-website/sfi-esb/node_modules/async/lib/async.js:157:25
    at /Users/Work/Documents/sfi-website/sfi-esb/node_modules/sprout-server/lib/loader.js:324:21

This issue originates in loader:registerConsumerModule() (line 397), which adds the handler's dependencies to a map where the key is prefixed with "handlers.". This same key is then used in loader:fetchUnmetDependencies() and passed through to subRequire:loadFromParent(), which fails to find the parent module because of said prefix.

When supplying configuration in a test using injectCore, the configuration is overridden by the default configuration included in BlueOak Server, for example:

var testUtility = require('blueoak-server').testUtility();
var coreServices = ['config', 'logger'];

var testSpecificConfig = {
  app: {
    key: 'value'
  }
};

testUtility.injectCore(coreServices, testSpecifcConfig, function (initializedModules) {
  var config = initializedModules.config;
  var logger = intializedModules.logger;

  var appConfig = config.get('app');

  // appConfig is {} instead of { key: 'value' }
});

Files that would possibly be affected by a fix for this:

https://github.com/BlueOakJS/blueoak-server/blob/master/testlib/util.js#L92
https://github.com/BlueOakJS/blueoak-server/blob/master/services/config.js#L18

In config.js, the fs.readFile block overrides any config passed into injectCore

In a few tests we need to see what the config would look like if there server were launched, but we don't necessarily want to go through the process of starting a server. Would be nice if there was an api exposed through the server itself that would return the config service.

Randstat requested a way to get stats about the system.

I'm adding in a stats service, which will expose a getStats method that will query all the services for stats.

Services that want to expose stats can export a stats(callback) function.

the Swagger spec provides support of inheritance using the discriminator property

BlueOak Server does not process these models to validate against the actual type of the provided model.
Ideally it does.

Need to allow additional middleware to be plugged into swagger-based endpoints

Add a version check during startup to ensure a "supported" node.js version is being used.

The recommended min version will be the latest LTS security release, currently 4.2.3, or an equivalent 5.x version (5.1.1)

If the check fails, a warning message will be displayed recommended a minimum LTS version.

this appears to be a regression from v2.1.2

When using the built-in swagger service the values returned from getPrettySpecs() and getSimpleSpecs() appear to be identical.
The should no be.

The "pretty" specs should be a "bundled" (all $refs are maintained, but only reference within the document) version of the each spec.

I am going to investigate.

There are cases where BlueOak and/or external BlueOak packages will introduce custom fields to Swagger, add data to the Express-based request object, and so on. In those cases, it'd be nice to have a standard namespace to reduce the risk of collision and to spot something BlueOak-specific at a glance.

For example:

• In middleware/handlers: req.bos.user instead of req.user
• In Swagger, x-bos-security instead of x-security

When using the internal cache, there's currently no way to change the default TTL.

calling something like logger.info("foo", null) causes this error:

TypeError: Cannot read property 'pid' of null
at Object.module.exports.(anonymous function) as info

Thinking about letting the swagger validators return an Error rather than returning a response themselves. Then Express error middleware can handle the errors and formatting them however they'd like.

First glance at the config wiki looks to me like there is a lot missing. I think there is information about the config scattered across the other parts of the wiki that would also be good to have in here. There may also be parts of the config that are missing from the wiki entirely. Need to investigate further.

Investigate whether it would make sense to include an RPC-based library so that ps-app can be a foundation for both REST and RPC endpoints.

This would help with a migration from Worklight.

there are several ways to solve this problem ...

here it is: after turning on response model validation for my project I've found that some responses are acceptably invalid and I'd like to keep having validation done for other API methods without filling up my logs with known and acceptable errors

some potential solutions (I'm sure there are more):

• provide a whitelist and/or blacklist config for excluding (or opting-in) API methods
• provide an error matcher config for exclusion (so that you could turn off only matching invalid responses, not everything

Have a logout endpoint that revokes refresh tokens

i need a tool that can mock the data for swagger api spec, do anybody knows how? maybe a third-part middleware or handler?

Currently using a period in a handler name will cause a problem because periods are used internally to delineate module prefixes, such as handlers be "handler." or middleware being "middleware.".

It's unlikely a module will have a period in the filename, so this will be low priority.

We also need to decide how periods would be handled in the dependency injection, since JS variables cannot have periods.

For example, I've got:

"properties": {
                "date": {
                    "type": "string",
                    "format": "date-time"
                }
}

The current implementation considers {date: 'a-string'} to be valid even though it's not a RFC3339 date-time string. Implementations using sprout-server could do this level of validation on their own, but perhaps this should happen here instead?

These formats aren't part of the JSON Schema spec, so tv4 isn't attempting to validate them.

yaml is great, but the import > work > download cycle in the web editor isn't that fun.
if I could have yaml files in my swagger folder and edit them with my local editor, that would be a nice improvement

e.g.:
if I add a file express.json to the config directory, with this content:

{
  "port": "5350"
}

I expect that the value for port will supersede any value set in an express object in either the default.json or production.json files.

This doesn't happen right now.

FYI:
Writing a config/express.json file with the contents above is a simple way to control the port to use during a CI deployment step, especially when multiple sprout server instances may be started on the same machine.

Swagger supports a "format" for data types that lets you specify more specific types for validation.
https://github.com/swagger-api/swagger-spec/blob/master/versions/2.0.md

Already added date and date-time via #18, but would like to eventually have complete support for the spec.

To avoid any confusion about what mode the server is in, display the config mode as part of startup.

Bug in code, pull request to follow.

Need a way to configure and enable CSRF protection out of the box.

For string types, swagger allows custom formats to be specified, e.g. email, uuid. We should have a convention for allowing validators for these formats to be created.

Currently, BOS's built in Multer support only supports disk storage. It would be nice to also implement memory storage support.

in the situation where an access token is expired and there is no refresh token, this section of google-oauth.js causes a 404:
getToken(tokenData, function (err, authData) {
if (err) {
return next(err);
}

Have already identified bug and will submit pull request.

I was getting an error about an invalid issuer, and it seems like sprout was passing an 'issuer' field to jwt instead of a 'iss' field.

Changing it to iss fixes the problem, but I want to investigate if this was always broken or if there was a recent change in the JWT library that caused it.

I could not get this auth code passed thru. It keeps returning string.length error.

function setAuthCodeOnReq(req, res, next) {
if (req.query.code) {
req.code = req.query.code;
ilog.debug('Found auth code %s on query param', req.code);
return next();
}

// Added this part and it worked for us
if (req.body.code) {
    req.code = req.body.code;
    ilog.debug('Found auth code %s on query param', req.code);
    return next();
}

getRawBody(req, function(err, string) {
    if (string.toString().length > 0) {
        req.code = string.toString();
        ilog.debug('Found auth code %s in body', req.code);
    }
    next();
});

}

When reviewing a server log, especially if it has been restarted several times, timestamps for the different restarts and debug/error logging are important.

Timestamps should proceed the existing logger messages, say using the format used by gulp.

(Having a stack trace on a restart condition would be great too, but that's probably something else.)

In order to unit test an individual service, it would be nice to be able to automatically instantiate it and inject any real or mock dependencies into it.

Per @gconsidine , would like a way to encrypt an entire JSON file rather than having to use the encrypt tool to encrypt each value individually.

On thought is to encode values with a special prefix that can be used to determine which fields to encrypt.

In the example below, the tool would just encrypt field2 but leave field1 untouched.

{
"field1": "blah blah blah",
"field2": "{pre}some plain text="
}

Likewise, the decrypt tool should be able to decrypt an entire file and replace the encrypted values with {pre} values.

Add a middleware service for enabling CORS.

I was playing with the blueoak-server 2.4.3 + swagger definitions this morning and ran into a problem related to defining a formData parameter.

consumes:
- application/x-www-form-urlencoded
parameters:
- name: test2
in: formData
description: test param
required: true
type: string

Error on console:
debug: -swagger - Wiring up route post /other2 to other.other2Set
TypeError: Cannot read property 'fields' of undefined
at registerRoute (/Users/jefflee/project/node_modules/blueoak-server/handlers/_swagger.js:247:42)

When I look at the _swagger.js file, the problem is it's trying to use _upload.fields inside a containsFormData() block; however, _upload is only set in a containsMultiPartFormData() block.

provide some config to have response models validated against the swagger response model based on a config value

It'd be nice for there to be some hint as to why the server just stopped to help me know where to look.

So, in the swagger spec, there's /paths (a "Paths object") which then contains a number of "Path Item objects".

It'd be nice if I could indicate a filename in the Path Item object for where I want the operationId to be found.

In swagger-tools this is provided by a "x-swagger-router-controller" key on the Path Item object, e.g.

...
"/tasks": {
            "x-swagger-router-controller": "Tasks",
            "get": {
                "summary": "Retrieve Tasks",
                "parameters": [],
                "responses": {
...

I'm not proposing we use that name (maybe "x-handler" or something like that?), but just showing an example of the implementation.

This enhancement would allow me to break up my handlers instead of having one file for the entire swagger spec (which could be huge, as it is in MM).

Sometimes I would like to have detailed logging that I can quickly enable for debugging issues. The auth and loader utils both have their own loggers that I can enable by modifying the source, but I'd like to be able to do it quickly from the CLI as needed.

I feel like existing log levels aren't adequate because debug to a sprout-server developer has different meaning than debug to an app developer.

Add the ability to specify auth providers for individual swagger-based routes.

If you don't provide this optional query parameter, the swagger validor
Here's the stack:

TypeError: str.split is not a function
    at parseArray (/tmp/my-project/node_modules/blueoak-server/lib/swaggerUtil.js:74:16)
    at Object.castValueFromString [as cast] (/tmp/my-project/node_modules/blueoak-server/lib/swaggerUtil.js:48:17)
    at setDefaultQueryParams (/tmp/my-project/node_modules/blueoak-server/handlers/_swagger.js:231:52)
    at /tmp/my-project/node_modules/blueoak-server/handlers/_swagger.js:197:13
    at validateParameters (/tmp/my-project/node_modules/blueoak-server/handlers/_swagger.js:345:12)
    at /tmp/my-project/node_modules/blueoak-server/handlers/_swagger.js:193:9
    at Layer.handle [as handle_request] (/tmp/my-project/node_modules/express/lib/router/layer.js:95:5)
    at next (/tmp/my-project/node_modules/express/lib/router/route.js:131:13)
    at Route.dispatch (/tmp/my-project/node_modules/express/lib/router/route.js:112:3)
    at Layer.handle [as handle_request] (/tmp/my-project/node_modules/express/lib/router/layer.js:95:5)
    at /tmp/my-project/node_modules/express/lib/router/index.js:277:22
    at Function.process_params (/tmp/my-project/node_modules/express/lib/router/index.js:330:12)
    at next (/tmp/my-project/node_modules/express/lib/router/index.js:271:10)
    at /tmp/my-project/node_modules/i18next/lib/i18next.js:119:13
    at /tmp/my-project/node_modules/i18next-client/i18next.js:992:21
    at /tmp/my-project/node_modules/i18next/lib/i18nextWrapper.js:317:29
    at /tmp/my-project/node_modules/i18next/lib/i18nextWrapper.js:360:33
    at ReadFileContext.callback (/tmp/my-project/node_modules/i18next/lib/filesync.js:19:17)
    at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:303:13)

Here's an example query definition:

name: types
in: query
required: false
type: array
items:
  type: string
  enum:
    - ANY
    - Type_A
    - Type_R
    - Type_S
collectionFormat: csv
default:
  - ANY
description: |
  A comma-separated list of types to search against , or the value `ANY` for all.

Used the config example in examples/custom-auth, but used "google-oauth" as the provider. Looks like there's an implementation that respects config.auth in handlers/_routes.js but not one in handlers/_swagger.js

Will not try to fix this myself :)

Is there is a way through which we can directly launch the swagger UI on the browser and check whether the API is working or not.
In addition, do we have an authentication module available to verify the authenticity of the request.
Rest the project seems a great one.. looking forward.

I've got a zip code query parameter called zip, but when a bad zip code is sent, it the message doesn't indicate it in the "field" parameter.

Here's the message:

{
  "message": "Error validating query parameter zip",
  "type": "ValidationError",
  "status": 422,
  "validation_errors": [
    {
      "message": "String does not match pattern: ^(\\d{5})(?:-(\\d{4}))?$",
      "field": ""
    }
  ]
}

Here's the query parameter swagger:

name: zip
in: query
required: false
type: string
pattern: ^(\d{5})(?:-(\d{4}))?$
description: |
  A string representing the USPS zip code for which records should be returned.

I expect the respond to be as-is with this update: "field": "zip".