GithubHelp home page GithubHelp logo

feathersjs-ecosystem / feathers-knex Goto Github PK

View Code? Open in Web Editor NEW
112.0 9.0 59.0 1.73 MB

Service adapters for KnexJS a query builder for PostgreSQL, MySQL, MariaDB, Oracle and SQLite3

License: MIT License

JavaScript 98.49% TypeScript 1.51%
feathersjs feathers-service-adapter knexjs

feathers-knex's Introduction

Important: This module has been moved to @feathersjs/knex and is developed in feathersjs/feathers

feathers-knex

CI Download Status

A database adapter for KnexJS, an SQL query builder for Postgres, MSSQL, MySQL, MariaDB, SQLite3, and Oracle.

npm install --save mysql knex feathers-knex

Important: feathers-knex implements the Feathers Common database adapter API and querying syntax.

Note: You also need to install the database driver for the DB you want to use.

API

service(options)

Returns a new service instance initialized with the given options.

const knex = require('knex');
const service = require('feathers-knex');

const db = knex({
  client: 'sqlite3',
  connection: {
    filename: './db.sqlite'
  }
});

// Create the schema
db.schema.createTable('messages', table => {
  table.increments('id');
  table.string('text');
});

app.use('/messages', service({
  Model: db,
  name: 'messages'
}));
app.use('/messages', service({ Model, name, id, events, paginate }));

Options:

  • Model (required) - The KnexJS database instance
  • name (required) - The name of the table
  • schema (optional) - The name of the schema table prefix (example: schema.table)
  • id (optional, default: 'id') - The name of the id field property.
  • events (optional) - A list of custom service events sent by this service
  • paginate (optional) - A pagination object containing a default and max page size
  • multi (optional) - Allow create with arrays and update and remove with id null to change multiple items. Can be true for all methods or an array of allowed methods (e.g. [ 'remove', 'create' ])
  • whitelist (optional) - A list of additional query parameters to allow (e..g [ '$regex', '$geoNear' ]). Default is the supported operators

adapter.createQuery(query)

Returns a KnexJS query with the common filter criteria (without pagination) applied.

params.knex

When making a service method call, params can contain an knex property which allows to modify the options used to run the KnexJS query. See customizing the query for an example.

Example

Here's a complete example of a Feathers server with a messages SQLite service. We are using the Knex schema builder and SQLite as the database.

$ npm install @feathersjs/feathers @feathersjs/errors @feathersjs/express @feathersjs/socketio feathers-knex knex sqlite3

In app.js:

const feathers = require('@feathersjs/feathers');
const express = require('@feathersjs/express');
const socketio = require('@feathersjs/socketio');

const service = require('feathers-knex');
const knex = require('knex');

const db = knex({
  client: 'sqlite3',
  connection: {
    filename: './db.sqlite'
  }
});

// Create a feathers instance.
const app = express(feathers());
// Turn on JSON parser for REST services
app.use(express.json());
// Turn on URL-encoded parser for REST services
app.use(express.urlencoded({ extended: true }));
// Enable REST services
app.configure(express.rest());
// Enable Socket.io services
app.configure(socketio());
// Create Knex Feathers service with a default page size of 2 items
// and a maximum size of 4
app.use('/messages', service({
  Model: db,
  name: 'messages',
  paginate: {
    default: 2,
    max: 4
  }
}))
app.use(express.errorHandler());

// Clean up our data. This is optional and is here
// because of our integration tests
db.schema.dropTableIfExists('messages').then(() => {
  console.log('Dropped messages table');

  // Initialize your table
  return db.schema.createTable('messages', table => {
    console.log('Creating messages table');
    table.increments('id');
    table.string('text');
  });
}).then(() => {
  // Create a dummy Message
  app.service('messages').create({
    text: 'Message created on server'
  }).then(message => console.log('Created message', message));
});

// Start the server.
const port = 3030;

app.listen(port, () => {
  console.log(`Feathers server listening on port ${port}`);
});

Run the example with node app and go to localhost:3030/messages.

Querying

In addition to the common querying mechanism, this adapter also supports:

$and

Find all records that match all of the given criteria. The following query retrieves all messages that have foo and bar attributes as true.

app.service('messages').find({
  query: {
    $and: [
      {foo: true},
      {bar: true}
    ]
  }
});

Through the REST API:

/messages?$and[][foo]=true&$and[][bar]=true

$like

Find all records where the value matches the given string pattern. The following query retrieves all messages that start with Hello:

app.service('messages').find({
  query: {
    text: {
      $like: 'Hello%'
    }
  }
});

Through the REST API:

/messages?text[$like]=Hello%

$notlike

The opposite of $like; resulting in an SQL condition similar to this: WHERE some_field NOT LIKE 'X'

app.service('messages').find({
  query: {
    text: {
      $notlike: '%bar'
    }
  }
});

Through the REST API:

/messages?text[$notlike]=%bar

$ilike

For PostgreSQL only, the keywork $ilike can be used instead of $like to make the match case insensitive. The following query retrieves all messages that start with hello (case insensitive):

app.service('messages').find({
  query: {
    text: {
      $ilike: 'hello%'
    }
  }
});

Through the REST API:

/messages?text[$ilike]=hello%

Transaction Support

The Knex adapter comes with three hooks that allows to run service method calls in a transaction. They can be used as application wide (app.hooks.js) hooks or per service like this:

// A common hooks file
const { hooks } = require('feathers-knex');

const { transaction } = hooks;

module.exports = {
  before: {
    all: [ transaction.start() ],
    find: [],
    get: [],
    create: [],
    update: [],
    patch: [],
    remove: []
  },

  after: {
    all: [ transaction.end() ],
    find: [],
    get: [],
    create: [],
    update: [],
    patch: [],
    remove: []
  },

  error: {
    all: [ transaction.rollback() ],
    find: [],
    get: [],
    create: [],
    update: [],
    patch: [],
    remove: []
  }
};

To use the transactions feature, you must ensure that the three hooks (start, end and rollback) are being used.

At the start of any request, a new transaction will be started. All the changes made during the request to the services that are using the feathers-knex will use the transaction. At the end of the request, if sucessful, the changes will be commited. If an error occurs, the changes will be forfeit, all the creates, patches, updates and deletes are not going to be commited.

The object that contains transaction is stored in the params.transaction of each request.

Important: If you call another Knex service within a hook and want to share the transaction you will have to pass context.params.transaction in the parameters of the service call.

Customizing the query

In a find call, params.knex can be passed a KnexJS query (without pagination) to customize the find results.

Combined with .createQuery({ query: {...} }), which returns a new KnexJS query with the common filter criteria applied, this can be used to create more complex queries. The best way to customize the query is in a before hook for find.

app.service('messages').hooks({
  before: {
    find(context) {
      const query = context.service.createQuery(context.params);

      // do something with query here
      query.orderBy('name', 'desc');

      context.params.knex = query;
      return context;
    }
  }
});

Configuring migrations

For using knex's migration CLI, we need to make the configuration available by the CLI. We can do that by providing a knexfile.js (OR knexfile.ts when using TypeScript) in the root folder with the following contents:

knexfile.js

const app = require('./src/app')
module.exports = app.get('postgres')

OR

knexfile.ts

import app from './src/app';
module.exports = app.get('postgres');

You will need to replace the postgres part with the adapter you are using. You will also need to add a migrations key to your feathersjs config under your database adapter. Optionally, add a seeds key if you will be using seeds.

// src/config/default.json
...
  "postgres": {
    "client": "pg",
    "connection": "postgres://user:password@localhost:5432/database",
    "migrations": {
      "tableName": "knex_migrations"
    },
    "seeds": {
      "directory": "../src/seeds"
    }
  }

Then, by running: knex migrate:make create-users, a migrations directory will be created, with the new migration.

Error handling

As of version 4.0.0 feathers-knex only throws Feathers Errors with the message. On the server, the original error can be retrieved through a secure symbol via error[require('feathers-knex').ERROR]

const { ERROR } = require('feathers-knex');

try {
  await knexService.doSomething();
} catch(error) {
  // error is a FeathersError with just the message
  // Safely retrieve the Knex error
  const knexError = error[ERROR];
}

Waiting for transactions to complete

Sometimes it can be important to know when the transaction has been completed (committed or rolled back). For example, we might want to wait for transaction to complete before we send out any realtime events. This can be done by awaiting on the transaction.committed promise which will always resolve to either true in case the transaction has been committed, or false in case the transaction has been rejected.

app.service('messages').publish((data, context) => {
  const { transaction } = context.params

  if (transaction) {
    const success = await transaction.committed
    if (!success) {
      return []
    }
  }

  return app.channel(`rooms/${data.roomId}`)
})

This also works with nested service calls and nested transactions. For example, if a service calls transaction.start() and passes the transaction param to a nested service call, which also calls transaction.start() in it's own hooks, they will share the top most committed promise that will resolve once all of the transactions have succesfully committed.

License

Copyright (c) 2021

Licensed under the MIT license.

feathers-knex's People

Contributors

ahdinosaur avatar arlair avatar arryan avatar bertho-zero avatar betarabbit avatar corymsmith avatar daffl avatar ekryski avatar green3g avatar greenkeeper[bot] avatar greenkeeperio-bot avatar jayalfredprufrock avatar jerfowler avatar jhuizingh avatar kc-dot-io avatar kidkarolis avatar lucas-portela avatar lvivier avatar markneub avatar marshallswain avatar mcchrish avatar nilsboy avatar nuc avatar omeid avatar oumar24sh avatar paflopes avatar runningskull avatar sylvainlap avatar vasilcenko avatar vonagam avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

feathers-knex's Issues

Error handling question.

This is more a question than an issue with this module. Whats the best approach for mapping the errors returned from feathers-knex to make it more user friendly to display to the client side? Here is an example, if you have a column:

table.string('col_1').unique();

If you attempt to post to the table and add a value that already exists to col_1 , it returns this to the client:

{
"name": "BadRequest",
"message": "insert into `col_1` (`text`) values ('add_this') - Duplicate entry 'add_this' for key 'table_text_unique'",
"code": 400,
"className": "bad-request",
"errors": {}
}

How do you display a user friendly error message based on the returned object above?

I believe knex returns the specific database driver error and that the error-handler used within feathers-knex is intercepting the error and sending this error object. Knex returns something similar to (for mysql):

code: 'ER_DUP_ENTRY',
errno: 1062,
sqlState: '23000',
sqlMessage: 'Duplicate entry \'value\' for key \'table_col_1_unique\'' }

Why not send the error.code/errno to the front end and display appropriate message accordingly?

Looking through the error-handler.js code. Within the case statement, where the feathersError is assigned, what if you pass in an additional errors object.

        const errorsObject = { code: error.code, errno: error.errno };
        feathersError = new errors.BadRequest(error, { errors: errorsObject });

This results in the response to the client to look like below. You get the additional errors object from the specific database driver you are using:

	"name": "BadRequest",
	"message": "insert into `task` (`another_col`, `text`) values ('afssxxsaaxafmA', 'ff') - Duplicate entry 'afssxxsaaxafmA' for key 'task_another_col_unique'",
	"code": 400,
	"className": "bad-request",
	"data": {},
	"errors": {
		"code": "ER_DUP_ENTRY",
		"errno": 1062
	}
}

Please let me know if I am missing something. Thanks in advance.

$or filter doesn't group together conditions

This is a bit of a security issue. (See feathersjs/feathers#404)

I was able to fix this by altering a few lines in the knexify() method to use knex's grouping capabilities. My fork is all screwed up right now so I can't submit a PR at the moment, but I wanted to go ahead and post the fix here:

// line 69
if (method) {
    if (key === '$or') {
        return query.where(function(){
            value.forEach(condition => this[method].call(this, condition));
        });
     }
     return query[method].call(query, column, value);
 }

result.total type is string in postgres

Steps to reproduce

Create a feathers-knex service using postgres and run the find method.

myFeathersKnexService.find({ query: { [field]: value } })
  .then(result => {
    console.log(result.total);
  });

Expected behavior

Returns an integer (e.g. 0)

Actual behavior

Returns a string (e.g. '0')

System configuration

Module versions:
feathers-knex: 2.7.0
knex: 0.13.0
pg: 7.0.1

NodeJS version: v6.11.1

Operating System: Alpine Linux 3.6

Service call sometimes returns data from wrong table when called internally

Steps to reproduce

  • create a knex db service
  • add a find before hook that uses hook.params.knex with any custom query:
hook => {
    hook.params.knex = hook.service.createQuery(hook.params.query);
}
  • at find after hook add a populate common hook including something from other service

Expected behavior

  • it should find the data from other service and include it

Actual behavior

  • it includes the record in this service again in itself

System configuration

Newest versions of feathers

Reason

https://github.com/feathersjs/feathers-knex/blob/master/src/index.js#L132
var q = params.knex || this.createQuery(params);

this reuses a previously executed query, although it has been set in the "parent" service fetching the parent record. As params.knex is already set, the params of the called "child" service are ignored and the parent query is run again.

How to solve this?

README example does not work

var feathers = require('feathers');
var hooks = require('feathers-hooks');
var knex = require('feathers-knex');

// Initialize a MongoDB service with the users collection on a local MongoDB instance
var app = feathers()
  .configure(hooks())
  .use('/users', knex('users', {
    dialect: 'sqlite3',
    connection: {
      filename: './data.db'
    }
  }));

app.lookup('users').before({
  create: function(hook, next) {
    hook.data.createdAt = new Date();
    next();
  },

  update: function(hook, next) {
    hook.data.updatedAt = new Date();
    next();
  }
});

app.listen(8080);

Output:

TypeError: app.lookup is not a function
    at Object.<anonymous> (/Users/dan.kaplun/euclidsnap/server.js:15:5)
    at Module._compile (module.js:425:26)
    at Object.Module._extensions..js (module.js:432:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:311:12)
    at Function.Module.runMain (module.js:457:10)
    at startup (node.js:136:18)
    at node.js:972:3

nested tables on join

Hi,
Using feathers-knex (postgresql), I am trying to nest my tables on a join. Here is an example on how to achieve this with node-postgres.

brianc/node-postgres#733

SELECT to_json(a.*) AS table1, to_json(b.*) AS table2 FROM some_table a LEFT JOIN another_table b ON a.id = b.id;
With this all rows will be structured as such:

{ table1: { ... },
table2: { ... } }

However, using:
const query = this.createQuery({ query: hook.params.query });
query.select(to_json(a.*) AS table1)

is giving me an error because it assumes that the table name is β€œto_json(a”. Is there a way to generate raw queries with feathers-knex?

Social Authentication SQL parse error with Knex

Hello,

I try to setup Feather Authentication with the Google provider.
I encounter many difficulties as there is no enough documentation with feathers-knex. But for this one I think something goes wrong :

I want to get authenticated by Google by going to this route: /auth/google.

I encounter this error:

knex:client acquired connection from pool: __knexUid2 +0ms
knex:query update `users` set `avatar` = ?, `createdAt` = ?, `email` = ?, `github` = ?, `githubId` = ?, `google` = ?, `googleId` = ?, `password` = ?, `updatedAt` = ?, `username` = ? where `id` = ? +1ms
knex:client releasing connection to pool: __knexUid2 +3ms
feathers:rest Error in REST handler: `update `users` set `avatar` = NULL, `createdAt` = NULL, `email` = NULL, `github` = NULL, `githubId` = NULL, `google` = '{my google data}', `googleId` = '10154235...152695121', `password` = NULL, `updatedAt` = NULL, `username` = NULL where `id` = 1 - ER_PARSE_ERROR: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''[object Object]', '[object Object]', `objectType` = 'person', `id` = '104385465' at line 1` +0ms
express:router <anonymous>  : /auth/google/callback?state=true&code=4/lCleOw9x2Z3uBu4SpsDUCfUVDcpCFl9zO4Ix3SApbZ4 +1ms
feathers-authentication:middleware An authentication error occurred. +2ms { Error: ER_PARSE_ERROR: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''[object Object]', '[object Object]', `objectType` = 'person', `id` = '104385465' at line 1
  at Query.Sequence._packetToError (/home/knguyen/web/reveal-your-heroes/node_modules/mysql/lib/protocol/sequences/Sequence.js:51:14)
  at Query.ErrorPacket (/home/knguyen/web/reveal-your-heroes/node_modules/mysql/lib/protocol/sequences/Query.js:83:18)
  at Protocol._parsePacket (/home/knguyen/web/reveal-your-heroes/node_modules/mysql/lib/protocol/Protocol.js:280:23)
  at Parser.write (/home/knguyen/web/reveal-your-heroes/node_modules/mysql/lib/protocol/Parser.js:74:12)
  at Protocol.write (/home/knguyen/web/reveal-your-heroes/node_modules/mysql/lib/protocol/Protocol.js:39:16)
  at Socket.<anonymous> (/home/knguyen/web/reveal-your-heroes/node_modules/mysql/lib/Connection.js:109:28)
  at emitOne (events.js:96:13)
  at Socket.emit (events.js:188:7)
  at readableAddChunk (_stream_readable.js:176:18)
  at Socket.Readable.push (_stream_readable.js:134:10)
  at TCP.onread (net.js:543:20)
  --------------------
  at Protocol._enqueue (/home/knguyen/web/reveal-your-heroes/node_modules/mysql/lib/protocol/Protocol.js:141:48)
  at Connection.query (/home/knguyen/web/reveal-your-heroes/node_modules/mysql/lib/Connection.js:214:25)
  at /home/knguyen/web/reveal-your-heroes/node_modules/knex/lib/dialects/mysql/index.js:124:18
  at Promise._execute (/home/knguyen/web/reveal-your-heroes/node_modules/bluebird/js/release/debuggability.js:299:9)
  at Promise._resolveFromExecutor (/home/knguyen/web/reveal-your-heroes/node_modules/bluebird/js/release/promise.js:481:18)
  at new Promise (/home/knguyen/web/reveal-your-heroes/node_modules/bluebird/js/release/promise.js:77:14)
  at Client._query (/home/knguyen/web/reveal-your-heroes/node_modules/knex/lib/dialects/mysql/index.js:118:12)
  at Client.query (/home/knguyen/web/reveal-your-heroes/node_modules/knex/lib/client.js:187:24)
  at Runner.<anonymous> (/home/knguyen/web/reveal-your-heroes/node_modules/knex/lib/runner.js:129:36)
  at Runner.tryCatcher (/home/knguyen/web/reveal-your-heroes/node_modules/bluebird/js/release/util.js:16:23)
  at Runner.query (/home/knguyen/web/reveal-your-heroes/node_modules/bluebird/js/release/method.js:15:34)
  at /home/knguyen/web/reveal-your-heroes/node_modules/knex/lib/runner.js:55:21
  at tryCatcher (/home/knguyen/web/reveal-your-heroes/node_modules/bluebird/js/release/util.js:16:23)
  at /home/knguyen/web/reveal-your-heroes/node_modules/bluebird/js/release/using.js:185:26
  at tryCatcher (/home/knguyen/web/reveal-your-heroes/node_modules/bluebird/js/release/util.js:16:23)
  at Promise._settlePromiseFromHandler (/home/knguyen/web/reveal-your-heroes/node_modules/bluebird/js/release/promise.js:510:31)
code: 'ER_PARSE_ERROR',
errno: 1064,
sqlState: '42000',
index: 0 }

{ my google data } is something like :

{
    "kind": "plus#person",
    "etag": "\\"
    xw0en60W6 - NurXn4VBU - CMjSPEw / eQLrqUnFu635 - 6 o1feH - f1c49Ho\\ "",
    "occupation": "xxx",
    "skills": "xxx",
    "gender": "male",
    "urls": [{
        "value": "http://gplus.to/xxx",
        "type": "otherProfile",
        "label": "Kevin Nguyen"
    }, {
        "value": "http://www.youtube.com/user/xxx",
        "type": "otherProfile",
        "label": "Kevin Nguyen"
    }, {
        "value": "http://twitter.com/xxx",
        "type": "otherProfile",
        "label": "kevinnth"
    }],
    "objectType": "person",
    "id": "111xxx",
    "displayName": "Kevin Nguyen",
    "name": {
        "familyName": "Nguyen",
        "givenName": "Kevin"
    },
    "braggingRights": "xxxx",
    "url": "https://plus.google.com/+xxx",
    "image": {
        "url": "https://xxx",
        "isDefault": false
    },
    "organizations": [{
        "name": "xxx",
        "type": "school",
        "startDate": "xxx",
        "primary": true
    }, {
        "name": "xxx",
        "title": "xxx xxxx",
        "type": "school",
        "startDate": "xxx",
        "endDate": "xxx",
        "primary": false
    }, {
        "name": "xxx.fr",
        "title": "xxx xxxx",
        "type": "work",
        "startDate": "xxx",
        "endDate": "xxx",
        "primary": false
    }],
    "isPlusUser": true,
    "language": "fr",
    "circledByCount": 180,
    "verified": false,
    "cover": {
        "layout": "banner",
        "coverPhoto": {
            "url": "https://xxx",
            "height": 530,
            "width": 940
        },
        "coverInfo": {
            "topImageOffset": 0,
            "leftImageOffset": 0
        }
    },
    "accessToken": "ya29.xxx-0ZGJXYm4_SLv3ZA5xxxTv5xxyQ1ZzQg"
}

So, this request should work fine but does not work:

update `users` set `avatar` = NULL, `createdAt` = NULL, `email` = NULL, `github` = NULL, `githubId` = NULL, `google` = '{ my google data }', `googleId` = '104385465479140482121', `password` = NULL, `updatedAt` = NULL, `username` = NULL where `id` = 1
Error: ER_PARSE_ERROR: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''[object Object]', '[object Object]', `objectType` = 'person', `id` = '104385465' at line 1

I execute this in with mysql and the update action is executed : 1 row affected. (Query took 0.0040 seconds.)

Here's my model (should be documented, isn't it?):

knex.schema.createTable('users', function (table) {
      table.increments('id').primary()
      table.string('username')
      table.string('email').index().unique()
      table.string('password')
      table.string('avatar')
      table.string('googleId') // I had to generate a feathers app with sequelize in order to get its model...
      table.json('google') // when attempt to authenticate, there is an insert into this column
      table.string('githubId')
      table.json('github')
      table.datetime('createdAt')
      table.datetime('updatedAt')
    })

The only thing weird in the { my google data } is the etag property, I don't know what it is...

Thanks for help.

Support conditional queries

Conditionals we should support:

  • $in
  • $or
  • $nin
  • $not

By default all queries are and queries so that isn't required.

I don't think we need to support complex conditional queries, at least not off the bat, but simple ones should work the same as when using feathers-mongo and feathers-mongoose.

Let's take a todo list example and say I want to get todos that were created on or after a certain date and only if they have a "complete" or "incomplete" status. When sending a GET request to /todos, the non-encoded url params might look like this:

{
  date: {
    $gte: "2015-10-25T18:13:12.843Z"
  },
  $or: [
    { "status": "complete" },
    { "status": "incomplete" }
  ]
}

Properties not passed to Feathers are getting set to null

Related: #55

Steps to reproduce

There are fields I exclude from my client side application to not allow editing them. These fields are not passed to the endpoint and are being set to null by feathers-knex. This is what I would consider an unexpected behavior.

I would like to change this so that fields that are not passed into the rest endpoint are kept as is.

https://github.com/feathersjs-ecosystem/feathers-knex/blob/master/lib/index.js#L288

      for (var key of Object.keys(oldData)) {
        if (data[key] === undefined) {
          newObject[key] = oldData[key] || null; // <--note reusing old data 
        } else {
          newObject[key] = data[key];
        }
      }

Expected behavior

Keys that are not passed to feathers should not be changed.

Actual behavior

These keys are set to null.

Thoughts??

Module versions (especially the part that's not working):
[email protected]

NodeJS version:
8.9.4

Operating System:
Windows 7

Browser Version:
Chrome 64

Module Loader:
Steal.js

Support comparators, like $gte, $gt, etc.

You should be able to support comparators like with feathers-mongo and feathers-mongoose.

Let's take a todo list example and say I want to get todos that were created on or after a certain date. When sending a GET request to /todos, the non-encoded url params might look like this:

{
  date: {
    $gte: "2015-10-25T18:13:12.843Z"
  }
}

ER_DUP_KEYNAME: Duplicate key name

Hi there,

I'm trying to use Feathers-Knex with Mysql.

I've run this cmd "feathers generate authentication"

After, i run the server, all it's good but if i restart it again i have this message :

Error updating users table { Error: ER_DUP_KEYNAME: Duplicate key name 'users_email_unique'
    at Query.Sequence._packetToError (/Users/jonathanroze/Documents/server/node_modules/mysql/lib/protocol/sequences/Sequence.js:52:14)
    at Query.ErrorPacket (/Users/jonathanroze/Documents/server/node_modules/mysql/lib/protocol/sequences/Query.js:77:18)
    at Protocol._parsePacket (/Users/jonathanroze/Documents/server/node_modules/mysql/lib/protocol/Protocol.js:280:23)
    at Parser.write (/Users/jonathanroze/Documents/server/node_modules/mysql/lib/protocol/Parser.js:75:12)
    at Protocol.write (/Users/jonathanroze/Documents/server/node_modules/mysql/lib/protocol/Protocol.js:39:16)
    at Socket.<anonymous> (/Users/jonathanroze/Documents/server/node_modules/mysql/lib/Connection.js:103:28)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at Socket.Readable.push (_stream_readable.js:134:10)
    at TCP.onread (net.js:548:20)
    --------------------
    at Protocol._enqueue (/Users/jonathanroze/Documents/server/node_modules/mysql/lib/protocol/Protocol.js:141:48)
    at Connection.query (/Users/jonathanroze/Documents/server/node_modules/mysql/lib/Connection.js:208:25)
    at /Users/jonathanroze/Documents/server/node_modules/knex/lib/dialects/mysql/index.js:152:18
    at Promise._execute (/Users/jonathanroze/Documents/server/node_modules/bluebird/js/release/debuggability.js:300:9)
    at Promise._resolveFromExecutor (/Users/jonathanroze/Documents/server/node_modules/bluebird/js/release/promise.js:483:18)
    at new Promise (/Users/jonathanroze/Documents/server/node_modules/bluebird/js/release/promise.js:79:10)
    at Client_MySQL._query (/Users/jonathanroze/server/node_modules/knex/lib/dialects/mysql/index.js:146:12)
    at Client_MySQL.query (/Users/jonathanroze/Roadtrip/server/node_modules/knex/lib/client.js:197:17)
    at Runner.<anonymous> (/Users/jonathanroze/Documents/server/node_modules/knex/lib/runner.js:146:36)
    at Runner.tryCatcher (/Users/jonathanroze/Documents/server/node_modules/bluebird/js/release/util.js:16:23)
    at Runner.query (/Users/jonathanroze/Documents/server/node_modules/bluebird/js/release/method.js:15:34)
    at Runner.<anonymous> (/Users/jonathanroze/Documents/server/node_modules/knex/lib/runner.js:193:19)
    at Runner.tryCatcher (/Users/jonathanroze/Documents/server/node_modules/bluebird/js/release/util.js:16:23)
    at Object.gotValue (/Users/jonathanroze/Documents/server/node_modules/bluebird/js/release/reduce.js:157:18)
    at Object.gotAccum (/Users/jonathanroze/Documents/server/node_modules/bluebird/js/release/reduce.js:144:25)
    at Object.tryCatcher (/Users/jonathanroze/Documents/server/node_modules/bluebird/js/release/util.js:16:23)
  code: 'ER_DUP_KEYNAME',
  errno: 1061,
  sqlState: '42000',
  index: 0 }

Do you have any idea?

Support population one level deep

It's pretty reasonable, especially when using population inside of the server, to allow populating (ie. joining) directly related tables.

Let's take an example of getting all the blog posts for a user but you also want the comments.

When a GET request is sent to /posts, the unencoded query string syntax should probably be:

{
  userId: 1,
  $populate: ['comments']
}

Which would result in a find on the posts table with a join on comments where posts.id = comments.postId.

Soliciting feedback, as you can also accomplish this with hooks, so maybe it should be left up to the user to implement.

Pagination broken in SQL-Server

Feathers-knex doesn't add a "sort" parameter when using Offset. This results in an SQL error on sql server. I realize this is sql dialect specific, but there is a simple solution that shouldn't really have any impact on other dialects.

The solution is add a default sort parameter to the idProp if a sort is not set:

https://github.com/feathersjs-ecosystem/feathers-knex/blob/master/lib/index.js#L144

    // Handle $skip
    if (filters.$skip) {
      q.offset(filters.$skip);

      // provide default sorting if its not set
      if(!filters.$sort){
        q.orderBy(this.id, 'asc');
      }
    }

Would this be something you'd accept a pr for?

Steps to reproduce

Pass parameters to a service using Sql-server:

$limit:10
$skip:10

Expected behavior

The server should return the correct rows.

Actual behavior

The server returns an error message from SQL:

select [plan_index].* from [plan_index] offset @p0 rows fetch next @p1 rows only - Invalid usage of the option next in the FETCH statement.

System configuration

Tell us about the applicable parts of your setup.

Module versions (especially the part that's not working):
[email protected]

NodeJS version:
8.9.4

Operating System:
Windows Server 2012 / SQL Server 2017

Browser Version:
Chrome

Exposing the query builder

I'm trying to figure out how to access the query builder object instead of the resolved promise containing row data that's returned by .find().

I'm using this lib vs. my own 'raw' knex connection to map feathers' query operators, and its handling of pagination, etc. At the same time, I often need more granular access to what's returned in a typical .find -- left joins, mapping hasMany relationships, scoping SQL to a transaction, etc.

It'd be super handy to have a generic query method that could return the QB instead of the raw results... is there a best practice for doing that?

$or bug

Hi,

I'm using the last version of feathers-knex.

Here is my hook.params.query:

{
    $or: [
        { firstname: { $like: "dace" },
        { lastname: { $like: "dace" },
    ],
    company: 1
}

And here is how feathers-knex transforms it:

"select "users".* from "users" where "company" = 1 or ("firstname" like 'dace') or ("lastname" like 'dace') limit 10"

As you can see, the or after the company = 1 should be a AND, not a OR.

Support for deeply nested/complex queries

We might draw the line here and just say this is too complex to handle. But essentially a query that won't work right now is something like this:

{
   query: {
      $or: [
         { name: 'Doug' },
         { 
            age: {
               $gte: 18,
               $not: 25
            }
         }
      ]
  }
};

Service methods are not executed in transaction

Steps to reproduce

  1. Make any service method call (get, find, create, update, delete)
  2. Throw an error

Expected behavior

  1. Begin transaction when starting a service method.
  2. If error is thrown transaction should be rolled back.

Actual behavior

When error is thrown, data are commited to database.

Warning message output from hook.js

Version 2.9.0

I'm using transaction.start()/end() hooks in my project. I saw a warning message printed on console in debug:
(node:3889) Warning: .then() only accepts functions but was passed: [object Object]

It seems that there is a typo in function end()/rollback() in hook.js:

      return trx.commit()
        .then(() => debug('finished transaction %s with success', id))
        .then(hook);

When I removed .then(hook), the warning message disappeared.

Support querying null value

Steps to reproduce

I want to filter records where the column value is MySQL NULL type.

service('myservice').find({ query: { column: null } })

Expected behavior

Return all records where value is MySQL NULL type ( query.whereNull(column) )

Actual behavior

Returns nothing because knexify function is doing query.where(column, 'null')

Expose Knex lib

So that end users don't need to require it separately we should just expose the underlying knex lib. Just in case people need access to it.

Wrong total if params.knex is set

If params.knex is set, params.query is bypassed, and the knex query is executed insted. However, for the count query (ie total), the params.query is taken into account (instead of params.knex), and results in a wrong total.

I think this is inconsistant, shouldn't params.knex, if set, should be used to compute the count query?

Example code problems

Hi, I'm not very good programming with JS but I managed to use this extension to FeathersJS (it rocks!). In the process I've encountered some problems:

  1. The example code doesn't work because the object referenced in the createTable sentence is "people", not "todos": people.knex.schema.createTable()
  2. I'm using Postgresql. In this case I have to finish the block with a promise .return() to get it work. This is my code block:
// This drops and creates table every time
todos.knex.schema.dropTableIfExists('todos').then(function(rows){
  return todos.knex.schema.createTable('todos', function(table) {
    table.increments('id');
    table.string('text');
    table.boolean('complete');
  })}).return();

How to setup relations between tables ?

Hi guys,

I'm testing an app with Knex right now, and having a bit of a problem when preparing my schemas. Since app.configure doesn't wait to be finished to go on the next one, i'm getting "relation does not exist" errors half of the time when loading with my empty database.

I'm using the app structure from the generator for postgres (which is loaded with sequelize by default). So far, i got this :

// services/index.js

const knex = require( 'knex' );
const authentication = require( './authentication' );
const user = require( './user' );
const client = require( './client' );

module.exports = function () {
    const app = this;

    const knex_connection = knex( {
        client: 'pg',
        connection: app.get( 'postgres' )
    } );
    app.set( 'knex', knex_connection );

    app.configure( authentication );
    app.configure( user );
    app.configure( client );

}
// services/user/index.js

const service = require( 'feathers-knex' );
const user = require( './user-model' );
const hooks = require( './hooks' );
const auth = require( '../../filters/auth' );

module.exports = function () {
    const app = this;

    return user( app.get( 'knex' ) ).then( () => {

        const options = {
            Model: app.get( 'knex' ),
            name: 'users',
            paginate: {
                default: 1,
                max: 10
            }
        };

        // Initialize our service with any options it requires
        app.use( '/users', service( options ) );

        const userService = app.service( '/users' );
        userService.before( hooks.before );
        userService.after( hooks.after );
        userService.filter( [ auth.isAuthenticated(), auth.isOwner() ] );

    } );

};
// services/user/user-model.js

module.exports = function ( knex ) {

    const schema = function() {
         return knex.schema.createTable( 'users', table => {

            table.increments( 'id' ).primary();
            table.string( 'email' ).notNullable().unique();
            table.string( 'password' ).notNullable();
            table.specificType( 'roles', 'varchar(255)[]' ).notNullable();
            table.timestamp( 'created_at' ).notNullable().defaultTo( knex.raw( 'now()' ) );
            table.timestamp( 'updated_at' ).notNullable().defaultTo( knex.raw( 'now()' ) );

        } );
    }

    return knex.schema.hasTable( 'users' ).then( exists => !exists && schema() );

};

Is it possible to make app.configure wait or i am doing it the wrong way ?

Thanks.

Example in README does not work

> var knex = require('feathers-knex');
undefined
> var todos = knex('todos', {
...   dialect: 'sqlite3',
...   connection: {
.....     filename: './data.db'
.....   }
... });
TypeError: knex is not a function
    at repl:1:13
    at REPLServer.defaultEval (repl.js:164:27)
    at bound (domain.js:280:14)
    at REPLServer.runBound [as eval] (domain.js:293:12)
    at REPLServer.<anonymous> (repl.js:393:12)
    at emitOne (events.js:82:20)
    at REPLServer.emit (events.js:169:7)
    at REPLServer.Interface._onLine (readline.js:210:10)
    at REPLServer.Interface._line (readline.js:549:8)
    at REPLServer.Interface._ttyWrite (readline.js:826:14)
    at ReadStream.onkeypress (readline.js:105:10)
    at emitTwo (events.js:87:13)
    at ReadStream.emit (events.js:172:7)
    at emitKeys (readline.js:1250:14)
    at next (native)
    at ReadStream.onData (readline.js:918:36)

Initialization should conform with other adapters

All other db adapters support these options:

  • Model (required) - The knex object scoped to a table
  • id (default: id) [optional] - The name of the id property
  • paginate [optional] - A pagination object containing a default and max page size (see below)

So it should look like this:

var knex = require('knex');
var knexService = require('feathers-knex');

app.use('/todos', knexService({
  Model: knex('todos')
}));

Need a way to exclude automatically generated columns from the update method.

It's fairly common to use automatically generated columns, i.e. datetimes like created_at and updated_at which knex has good support for. The problem is the knex service won't let you update tables like this, since it automatically includes NULL for those columns. I suppose you could argue PATCH should be used in this case, but I propose another way:

A simple extra option like nonUpdateableColumns that would default to an array of [this.id] could be used to prevent the overwriting of these columns during an update, more of a generalization of what is already being done to the "id" column. A hook really wouldn't work, since you'd have to apply the removal to the GET method, however in just about every other case you would want GET to return those columns. I think building support for this use case into the service makes since. If a maintainer agrees and wants to offer up an option name, I'm happy to create a pull request.

An in-range update of mocha is breaking the build 🚨

Version 3.3.0 of mocha just got published.

Branch Build failing 🚨
Dependency mocha
Current Version 3.2.0
Type devDependency

This version is covered by your current version range and after updating it in your project the build failed.

As mocha is β€œonly” a devDependency of this project it might not break production or downstream projects, but β€œonly” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this πŸ’ͺ

Status Details - ❌ **continuous-integration/travis-ci/push** The Travis CI build failed [Details](https://travis-ci.org/feathersjs/feathers-knex/builds/225215139)

Release Notes coverave

Thanks to all our contributors, maintainers, sponsors, and users! ❀️

As highlights:

  • We've got coverage now!
  • Testing is looking less flaky \o/.
  • No more nitpicking about "mocha.js" build on PRs.

πŸŽ‰ Enhancements

  • #2659: Adds support for loading reporter from an absolute or relative path (@sul4bh)
  • #2769: Support --inspect-brk on command-line (@igwejk)

πŸ› Fixes

  • #2662: Replace unicode chars w/ hex codes in HTML reporter (@rotemdan)

πŸ” Coverage

πŸ”© Other

Commits

The new version differs by 89 commits0.

  • fb1687e :ship: Release v3.3.0
  • 1943e02 Add Changelog for v3.3.0
  • 861e968 Refactor literal play-icon hex code to a var
  • 1d3c5bc Fix typo in karma.conf.js
  • 9bd9389 Fix spec paths in test HTML files
  • 0a93024 Adds tests for loading reporters w/ relative/absolute paths (#2773)
  • 73929ad Comment special treatment of '+' in URL query parsing
  • e2c9514 Merge pull request #2769 from igwejk/support_inspect_break_in_opts
  • 038c636 Support --inspect-brk on command-line
  • b4ebabd Merge pull request #2727 from lamby/reproducible-build
  • 882347b Please make the build reproducible.
  • a2fc76c Merge pull request #2703 from seppevs/cover_utils_some_fn_with_tests
  • ed61cd0 cover .some() function in utils.js with tests
  • f42cbf4 Merge pull request #2701 from craigtaub/landingSpec
  • 6065242 use stubbed symbol

There are 89 commits in total.

See the full diff

Not sure how things should work exactly?

There is a collection of frequently asked questions and of course you may always ask my humans.


Your Greenkeeper Bot 🌴

Ability to return value from original insert object

I just ran into a big issue that is largely a result of how knex works, issue here: knex/knex#95

Basically Knex inserts for MySQL can only return auto-increment primary keys. I'm generating my own UUIDs for some of my tables which causes the "create" service method to fail, since it is unable to retrieve the newly created user using "get" because the primary key is not available.

The easiest way to fix this would be to provide the ability to return a property from the original object being inserted. I'm happy to submit a pull request for this, but I'm not sure of the best way to expose this functionality. We can either add a flag option that indicates the "id" option specified should be pulled from the original object, or use a convention like if the "id" begins with a "." then the value should be retrieved from the original object instead of the result from knex. Thoughts?

Update selected fields error

Steps to reproduce

When you call an update method without all the table columns values.

service.update(data.id, {password: hook.data.password.toUpperCase()})
        .then(response => resolve(hook))
        .catch(error => reject(error));

Expected behavior

It should update only the password field.

Actual behavior

It sets other fields to NULL.

System configuration

Using oracledb.

createQuery on empty table throws an error

Steps to reproduce

Changing the knex query (via createQuery()) in a before hook returns an error when there are 0 rows in my postgres DB:

TypeError: Cannot read property 'total' of undefined
    at countQuery.then.count (/path/to/node_modules/feathers-knex/lib/index.js:184:48)

As soon as there are rows, everything works fine.

My before hook that does a count / join:

const { mapKeys } = require('lodash');

module.exports = (options = {}) => { // eslint-disable-line no-unused-vars
  return async context => {
    // PREVENT: "column reference "[column name]" is ambiguous"
    const sort = mapKeys(context.params.query.$sort, (order, field) => `plannings.${field}`);
    context.params.query.$sort = sort;
    
    const query = context.service.createQuery({ query: context.params.query });
    query
      .count('plannings.id as planninRuleCount')
      .leftJoin('planningRules as plr', 'plr.planningId', 'plannings.id')
      .groupBy('plannings.id');
    
    if (context.id) // Scope the query if it's a single GET call
      query.where('plannings.id', context.id);
    
    context.params.knex = query;
    return context;
  };
};

PostgreSQL Error Codes

Is there a way to detect the dabase in this file: feathers-knex/blob/master/src/error-handler.js ?

I found the error codes for postgress and would like to map them, but looking in the error object there is nothing to help to identify the database.

https://www.postgresql.org/docs/9.2/static/errcodes-appendix.html

And also how should I implement the errors handlers?
Like I was getting and error about a unique_violation but in the client the error was showing the SQL used to insert.
Should it be something like this?:
new errors.FeathersError(error.detail, error.code, 'unique_violation', error );

Support $search

This is a proposed special attribute that allows you to fuzzy match a property. Possibly even multiple properties and/or nested documents.

Suggested syntax:

name: {
  $search: ['alice', 'Alice', 'bo', /$bob/i]
}

Following similar syntax to our other special query filters, this would allow you to filter by a singular value, multiple values (treated like an or) and/or regular expressions directly.

For knex we'd have to manually construct OR queries so it would get a little messy because in order to do LIKE queries the syntax is:

knex('users').where('name', 'like', '%Test%');

knexfile.js for migrations

I'm trying to configure knex migrations in my feathersjs app, I need to create a knexfile with the configurations for knex migrations to work, but I don't want to have my database configuration in two places (feathersjs config and knexfile), does anybody know how handle database configuration for both?

Datetype for mysql

When you try to pass JS date like new Date().toString(), or new Date().toISOString() to datetime field it won't you need to create new compatible with mysql date string new Date().toISOString().slice(0, 19).replace('T', ' '), or you need to re-config your sql service what I don't want to do. It would be useful if this module will do it for you.

Primary key is not "id"

I tried to declare the user table as:

table.increments('user_id').primary();
table.string('username');

but when I do a HTTP GET call to the server /user/1
I got the error

{
  "name": "BadRequest",
  "message": "select `user`.* from `user` where `id` = '1' - ER_BAD_FIELD_ERROR: Unknown column 'id' in 'where clause'",
  "code": 400,
  "className": "bad-request",
  "errors": {}
}

Expected result

the user record would be shown accordingly.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.