ncb000gt / node-es

NodeJS module for ElasticSearch.

node-es

This is a Node.js module for the elasticsearch REST API.

NOTE: node-es v0.6 and newer work with ElasticSearch 5 and up. For older versions of ElasticSearch, prior versions of node-es should be used.

Install

npm install es

Elasticsearch Version Compatibility

When working with Elasticsearch v7.x and up, use v0.8.x (latest npm install es). When using previous versions of Elasticsearch, please use v0.7.4 of this module (npm install [email protected]).

Usage

var
  elasticsearch = require('es'),
  config = {
    _index : 'kittehs'
  },
  es = elasticsearch(config);

es.search({
    query : {
      field : {
        animal : 'kitteh'
      }
    }
  }, function (err, data) {
    // work with data here
    // response data is according to ElasticSearch spec
  });

API

Unless otherwise stated, all callback signatures are function (err, data), with data being the parsed JSON response from elasticsearch.

createClient

Calling elasticsearch.createClient(config) is the same as elasticsearch(config).

var
  elasticsearch = require('es'),
  es = elasticsearch.createClient(config);

config._index

When initializing the library, you may choose to specify an index to work with at the start to save from having to supply this information in the options for each operation request:

var config = {
  _index : 'pet'
};

Additionally, if working with multiple indexes, you may specify them as arrays:

var config = {
  _indices : ['pet', 'family'],
};

Note: When index or indices are supplied via operation options, those settings will take precedent over the base configuration for the library:

var
  elasticsearch = require('es'),
  config = {
    _index : 'kitteh'
  },
  es = elasticsearch.createClient(config);

es.indices.exist({ _index : 'canine' }, function (err, data) {
  // will result in a HEAD request to /canine instead of /kitteh
});

config.server

If omitted from configuration, the server settings default to the following:

var config = {
  // optional - when not supplied, defaults to the following:
  server : {
    host : 'localhost',
    port : 9200
  }
};

Anything specified within the server element of config is passed directly through to each HTTP/HTTPS request. You may configure additional options for connecting to Elasticsearch:

var config = {
  server : {
    agent : false,
    auth : 'user:pass',
    host : 'localhost',
    port : 9243,
    rejectUnauthorized : false,
    secure : true // toggles between https and http
  }
};

cluster support and failover

Elasticsearch is pretty much rad at clustering. If you want to specify multiple servers to failover to, you may do so by either supplying an array as the value for the property host, hosts, hostname or hostnames:

var elasticsearch = require('es');
var config = {
  _index : 'bawss',
  server : {
    hosts : ['es1.myhost.com', 'es2.myhost.com', 'es3.myhost.com']
    secure : true
  }
};

var es = elasticsearch(config);

If you run on different ports for each server, use the hostnames property:

var elasticsearch = require('es');
var config = {
  _index : 'bawss',
  server : {
    hostnames : ['localhost:9200', 'localhost:9201', 'localhost:9202']
  }
};

var es = elasticsearch(config);

operation timeout

The default timeout for any operation against Elasticsearch is set at 30 seconds. You can override this value by specifying a timeout property in the options for the operation:

var options = {
  timeout : 60000 // 60 seconds
};

es.bulk(options, commands, function (err, data) {
  // teh datas
});

request event

An event named request with a signature of function (options) { } is emitted for each API call.

var elasticsearch = require('es');

var config = {
  _index : 'bawss',
  server : {
    hosts : ['localhost:9200', 'localhost:9201', 'localhost:9202']
  }
};

var es = elasticsearch(config);

es.request.on('request', function (options) {
  console.log('request initiated');
  console.log(options);
});

es.count(function (err, results) {
  // event results in request options being logged to console...
});

options for any operation

For each ES operation, options may be specified as the first argument to the function. In most cases, these are entirely optional, but when supplied, the values specified will take precedent over the config values passed to the library constructor. Additionally, if there are extra option keys supplied beyond what is required for the operation, they are mapped directly to the querystring.

var options = {
  _index : 'bawss',
  refresh : true
};

var doc = {
  field1 : 'test value'
};

es.index(options, doc, function (err, data) {
  // this will result in a POST with path /bawss/man?refresh=true
});

Core

For more specifics and details regarding the core API for ElasticSearch, please refer to the documentation at http://www.elasticsearch.org/guide/reference/api/.

Bulk

Please Note: The default timeout is set at 30 seconds... if you are performing a large bulk insert you may need to increase this limit by specifying a higher value for timeout in the options parameter.

This method doesn't take into account the underlying config that was used when instantiating the client. It requires index to be specified via the commands array or via the options parameter. Conflict will occur if one specifies a different index in the options than what is specified via the commands parameter.

At a high level, when performing a bulk update, you must supply an array with an action object followed by the object that the action will use during execution. In the following example, the first item in the array specifies the action is index and the second item represents the data to index:

[
  { index : { _index : 'dieties' } },
  { name : 'hamish', breed : 'manx', color : 'tortoise' }
]

In this example, two index actions will be performed on the 'dieties' index in ElasticSearch:

[
  { index : { _index : 'dieties' } },
  { name : 'dugald', breed : 'siamese', color : 'white' },
  { index : { _index : 'dieties' } },
  { name : 'keelin', breed : 'domestic long-hair', color : 'russian blue' }
]

For more information regarding bulk, please see the ElasticSearch documentation at http://www.elasticsearch.org/guide/reference/api/bulk/

es.bulk(options, commands, callback)

var
  elasticsearch = require('es'),
  es = elasticsearch();

var commands = [
  { index : { _index : 'dieties' } },
  { name : 'hamish', breed : 'manx', color : 'tortoise' },
  { index : { _index : 'dieties' } },
  { name : 'dugald', breed : 'siamese', color : 'white' },
  { index : { _index : 'dieties' } },
  { name : 'keelin', breed : 'domestic long-hair', color : 'russian blue' }
];

es.bulk(options, commands, function (err, data) {
  // teh datas
});

Bulk Index

This is not a core action for ElasticSearch, but is a convenience method added to this ElasticSearch client to make bulk indexing more straight forward. Simply supply an array of documents you wish to bulk index in ElasticSearch and the method will take of the details for you.

es.bulkIndex(options, documents, callback)

var
  elasticsearch = require('es'),
  es = elasticsearch();

var documents = [
  { name : 'hamish', breed : 'manx', color : 'tortoise' },
  { name : 'dugald', breed : 'siamese', color : 'white' },
  { name : 'keelin', breed : 'domestic long-hair', color : 'russian blue' }
];

var options = {
  _index : 'dieties'
}

es.bulkIndex(options, documents, function (err, data) {
  // teh datas
});

Count

es.count(options, query, callback)

var
  elasticsearch = require('es');
  es = elasticsearch();

es.count(function (err, data) {
  // teh datas
});

// count docs in a specific index
var options = {
  _index : 'bawss'
}

es.count(options, function (err, data) {
  // counted... like a bawss
});

// count docs for a specific query
var query = {
  query : {
    term : {
      breed : 'manx'
    }
  }
};

es.count(options, query, function (err, data) {
  // teh count of teh manx kittehs
});

Delete

Requires _index be specified either via lib config (as shown below) or via options when calling the operation.

es.delete(options, callback)

var
  elasticsearch = require('es'),
  es = elasticsearch();

core.delete({ _id : 'mbQZc_XhQDWmNCQX5KwPeA' }, function (err, data) {
  // teh datas
});

Delete By Query

Requires _index be specified either via lib config (as shown below) or via options when calling the operation.

es.deleteByQuery(options, query, callback)

var
  elasticsearch = require('es'),
  es = elasticsearch({ _index : 'kitteh' });

var query = {
  query : {
    field : { breed : 'siamese' }
  }
};

es.deleteByQuery(query, function (err, data) {
  // teh datas
});

Exists

Requires _index be specified either via lib config or via options when calling the operation.

es.exists(options, callback)

var
  elasticsearch = require('es'),
  es = elasticsearch();

es.exists({ _index : 'kitteh' }, function (err, data) {
  // teh datas
});

Explain

Requires _index be specified either via lib config or via options when calling the operation. Also requires _id, but this must be specified via options.

es.explain(options, query, callback)

Get

Requires _index be specified either via lib config or via options when calling the operation. Also requires _id, but this must be specified via options.

es.get(options, callback)

Index

Requires _index be specified either via lib config or via options when calling the operation.

es.index(options, doc, callback)

More Like This

Requires _index be specified either via lib config or via options when calling the operation. Also requires _id, but this must be specified via options.

es.moreLikeThis(options, callback)

Multi Get

If _index is supplied via options (or lib config), the will applied to the doc that is transmitted for the operation.

es.multiGet(options, docs, callback)

Multi Search

es.multiSearch(options, queries, callback)

Search

Requires _index be specified either via lib config or via options when calling the operation.

es.search(options, query, callback)

Scroll

Requires scroll be specified via options when calling the method. The following code snippet shows how to perform a search with a subsequent scroll.

var
  elasticsearch = require('es'),
  config = {
    _index : 'kittehs'
  },
  es = elasticsearch(config);

// first search
es.search({
    scroll : '10m'
  }, {
    query : {
      match_all : {}
    }
  }, function (err, data) {
    // next, perform the scroll with the _scroll_id value returned
    es.scroll({ scroll : '10m' }, data['_scroll_id'], callback);
  });

Suggest

es.suggest(options, query, callback)

Update

Requires _index be specified either via lib config or via options when calling the operation. Also requires _id, but this must be specified via options.

es.update(options, doc, callback)

Validate

Requires _index be specified either via lib config or via options when calling the operation.

es.validate(options, query, callback)

Indices

All operations here interact with the indices segment of the Elasticsearch API.

Alias

es.indices.alias(options, data, callback)

Aliases

Requires alias, but this must be specified via options.

es.indices.aliases(options, callback)

Analyze

es.indices.analyze(options, data, callback)

Clear Cache

es.indices.clearCache(options, callback)

Close Index

Requires _index be specified either via lib config or via options when calling the operation.

es.indices.closeIndex(options, callback)

Create Index

Requires _index be specified either via lib config or via options when calling the operation.

es.indices.createIndex(options, data, callback)

Create Template

Requires name, but this must be specified via options.

es.indices.createTemplate(options, template, callback)

Delete Alias

Requires _index and _alias be specified either via lib config or via options when calling the operation.

es.indices.deleteAlias(opitons, callback)

Delete Index

Requires _index be specified either via lib config or via options when calling the operation.

es.indices.deleteIndex(options, callback)

Delete Mapping

Requires _index be specified either via lib config or via options when calling the operation.

es.indices.deleteMapping(options, callback)

Delete Template

Requires name, but this must be specified via options.

es.indices.deleteTemplate(options, callback)

Exists

Requires _index be specified either via lib config or via options when calling the operation.

es.indices.exists(options, callback)

Flush

es.indices.flush(options, callback)

Mappings

es.indices.mappings(options, callback)

Open Index

Requires _index be specified either via lib config or via options when calling the operation.

es.indices.openIndex(options, callback)

Put Mapping

Requires _index be specified either via lib config or via options when calling the operation.

es.indices.putMapping(options, mapping, callback)

Refresh

es.indices.refresh(options, callback)

Segments

es.indices.segments(options, callback)

Settings

Requires _index be specified either via lib config or via options when calling the operation.

es.indices.settings(options, callback)

Snapshot

es.indices.snapshot(options, callback)

Stats

es.indices.stats(options, callback)

Templates

Requires name, but this must be specified via options.

es.indices.templates(options, callback)

Update Settings

es.indices.updateSettings(options, settings, callback)

Cluster

All operations here interact with the Cluster portion of the Elasticsearch API.

Delete River

Requires name, but this must be specified via options.

es.cluster.deleteRiver(options, callback)

Field Stats

Requires field or fields, but this must be specified via options.

es.cluster.fieldStats(options, callback)

Health

es.cluster.health(options, callback)

Hot Threads

es.cluster.hotThreads(options, callback)

Nodes Info

es.cluster.nodesInfo(options, callback)

Nodes Stats

es.cluster.nodesStats(options, callback)

Put River

Requires name, but this must be specified via options.

es.cluster.putRiver(options, meta, callback)

Reroute

es.cluster.reroute(options, commands, callback)

Rivers

Requires name, but this must be specified via options.

es.cluster.rivers(options, callback)

Settings

es.cluster.settings(options, callback)

Shutdown

es.cluster.shutdown(options, callback)

State

es.cluster.state(options, callback)

Update Settings

es.cluster.updateSettings(options, updates, callback)

Testing

Code coverage data generated from npm test is located in ./lib-cov and is not included in the git repo.

npm install
npm test

To run code coverage and generate local report at ./reports/coverage.html:

npm run-script coverage

Requirements

• Node.js
• elasticsearch
• The need for search

License

MIT

node-es's Issues

need some example

need some example :D

Give cpsubrian commit access

In the spirit of http://felixge.de/2013/03/11/the-pull-request-hack.html, I think we should give @cpsubrian commit access. Any objections?

New percolator support

Are there plans to support the new percolator api? (.percolate)

http://www.elasticsearch.org/blog/percolator-redesign-blog-post/

Only half the queries get executed (exactly half)

Hello, I am sending a bulk set of queries via the multiSearch() method. But the response does not match what I ask for.

When I do 46 queries, I get back 23. When I do 76, I get back 38. When I do 64, I get 32.

More specifically, I pass in an array with say 76 query objects in it, and I get back an array of 38 properly formatted results. There are no errors anywhere I can see.

I have tried sending with explicitly setting the index and without, and each time I get the same result.

Any idea why this would be happening?

usage info is wrong

The current info says
" var elasticsearch = require('elasticsearch');
var es = elasticsearch({index: 'kitteh'});
"
first the name is 'node-elastisearch'
but even after that, the last line has still an error
says elasticsearch is not a function

In the trace log all query are empty

Hello !

this is my code

`
const esClient = new es.Client({
host: 'http://localhost:9200',
log: {
type: 'file',
level: 'trace'
}
});

return esClient.search({index : 'notices'},
{'query' : {
'bool':{
'must':[
{'match':{'title.normalized':jsonLine.titre.normalized}},
{'match':{'doi.normalized':jsonLine.doi.normalized}}
]
}
}
})`

and in the tracelog the queries are empty ...
Do you have a hint about what I did wrong ?

Functional Tests

I'm working on functional tests for my memcached request handler. This is uncovering some broken methods in the core api.

I'd like to add functional tests here (that hit an actual elasticsearch server). They'll end up looking something like this: https://github.com/cpsubrian/elasticsearch-memcached/blob/master/test/core.js

I don't want to step on the current tests at all. Do you guys have any preference on how I add these? Technically I should be able to get them working in travis. Not sure how coveralls works though.

update docs

it is require('elasticsearch') not require('es')

Can't have _id=0

utils.pathAppend doesn't work if resource === 0:

exports.pathAppend = function (resource) {
        'use strict';

        if (resource) {
                return '/' + resource;
        }

        return '';
};

This means operations on documents with _id=0 will instead operate on the entire type. For example, this:

es.delete({_index: 'foo', _type: 'bar', _id: 0}, ...)

ignores the _id and issues DELETE /foo/bar.

EventEmitter Listeners when a lot of concurrent query

When a lot of concurrent query and async queries are done at the same time, Node raise en error :

(node) warning: possible EventEmitter memory leak detected. 11 listeners added. Use emitter.setMaxListeners() to increase limit.
Trace
    at Socket.EventEmitter.addListener (events.js:160:15)
    at Socket.Readable.on (_stream_readable.js:689:33)
    at ClientRequest.<anonymous> (E:\Web\node\node_modules\elasticsearch\lib\request.js:150:12)
    at ClientRequest.EventEmitter.emit (events.js:117:20)
    at http.js:1758:9
    at process._tickCallback (node.js:415:13)

To have this error, i have like 1000 queries in 1 sec, so I guess I have 11 concurrent queries.

Is it possible to adjust the setMaxListeners with the number of concurrent queries ?

Ability to log http requests

It would be nice to have the final http requests showing up in the node console.

Creating a river

Hi,

I am trying to create river. That's what I have done. But it complaints that name is required.

var options = {_index : 'roles_index', _type : 'roles'};
var meta = {name: "couchdb", "couchdb" : { "host" : "localhost", "port" : 5984, "db" : "roles", "filter" : null }};

        riverClient.cluster.putRiver(options, meta, function(err, data){
            console.log(err);
            callback(null);
        });

I would really appreciate your help.

Thanks.

deleteByQuery

Unless I'm going crazy, I don't think client.deleteByQuery will work as expected. In the code we find:

// documentation indicates DELETE method...
// sending POST data via DELETE not typical, using POST instead
return req.post(options, query, callback);

The problem is, now this request doesn't have any indication that it should delete anything. The path just has _query. I will try to put together a test-case that proves (or disproves) my assumption.

Either make http_compat a module in npm or submodule.

Best handle the package/dependency management.

getaddrinfo ENOTFOUND

I cant create a new index .. show this error getaddrinfo ENOTFOUND

var options = {
server: {
hosts:settings.ES_URI
}
};

var conn;
conn = elasticsearch(options);

conn.indices.createIndex({_index:settings.ES_INDEX},{},function (err, data){
if(err){
console.log('Cannot create index')
console.log(err.stack);
console.log(data);
} else {
console.log('Created index')
}
});

Travis Build Error

> [email protected] pretest /home/travis/build/ncb000gt/node-es
> jshint *.js ./lib/*.js ./test/*.js ; jscoverage ./lib ./lib-cov
/home/travis/build/ncb000gt/node-es/node_modules/jscoverage/bin/jscoverage:50
      exclude.forEach(function (v) {
              ^
TypeError: Cannot call method 'forEach' of undefined
    at /home/travis/build/ncb000gt/node-es/node_modules/jscoverage/bin/jscoverage:50:15
    at /home/travis/build/ncb000gt/node-es/node_modules/jscoverage/node_modules/xfs/lib/async.js:222:9
    at Object.oncomplete (fs.js:297:15)
npm ERR! weird error 1
npm ERR! not ok code 0
The command "npm test" exited with 1.
Done. Your build exited with 1.

Setting options dynamically

This example from the README which sets the index dynamically does not appear to work. It always returns undefined. Am I doing something wrong? See comments...

var hapi = require('hapi');
var server = new hapi.Server();

var elasticsearch = require('es'),
    config = {
      _index : 'dogs',
      _type : 'dog'
    },
    es = elasticsearch(config);


server.connection({ port: 3300, host: 'localhost' });

server.route({
    method: 'GET',
    path: '/',
    handler: function (request, reply) {

        // this works
        es.count(function (err, data) {
            console.log(data);
        });

        // setting options dynamically does not
        var options = {
            _index : 'cats',
            _type : 'cat'
        }
        es.count(options, function (err, data) {
            console.log(data);
        });

    }
});

server.start(function (err) {
    if (err) {
        throw err;
    }
  console.log('hapi server started');
});

Promises

Thoughts on moving towards a promise-based API?

Version not passed?

I am trying to use the versioning feature and therefore I am passing the doc version with the update query:

var elasticsearch = new ESClient();
elasticsearch.request.on('request', function (options) {
    console.log(options.path);
});
elasticsearch.update({
        _index: 'my_index',
        _type: 'my_type',
        _id: 'lJ5P4EFBN',
        version: 5
    }, {
        doc: {
            name: 'new-name-should-not-work'
        }
    },
    function(err, result) {
        console.dir(err);
    }
);

The logged path is: /my_index/my_type/lJ5P4EFBN/_update?version=5

Since the version number of the doc is higher than 5 this should raise an error, but the update is working. When I use curl requests I get the version conflict errors as expected:

curl -XPUT localhost:9200/my_index/my_type/lJ5P4EFBN?version=5 -d '{
    "name" : "new-name"
}'
OR
curl -XPOST localhost:9200/my_index/my_type/lJ5P4EFBN?version=5 -d '{
    "name" : "new-name"
}'

Versionning

Is node-elasticsearch handling in any way versionning? Right now I can't seem to find a mention of this in the README file.

If not, it might be a cool addition.

New suggest API

There's recently added _suggest REST endpoint which should be implemented: http://www.elasticsearch.org/guide/reference/api/search/term-suggest/

Memcached Protocol

First, please consider this more of a conversation starter than an actual 'issue'.

Elastic Search features an optional memcached transport that supports a subset of the REST api.

In some initial, ROUGH, benchmarks the memcached transport handles significantly more throughput than the REST api. I put together a quick bench comparing the two at https://github.com/cpsubrian/modeler-elasticsearch/blob/noclient/bin/bench.sh .

The benchmark uses a wrapper I threw together that uses the bulk of node-elasticsearch's api and just replaces lib/request.js with a memcache version. Ultimately that driver still has to fall-back to REST for HEAD requests (and possibly others that we'll find if we really dive in). You can find it here: https://github.com/cpsubrian/elasticsearch-memcached

Results are:

Please be patient.
{ http_parser: '1.0',
  node: '0.10.15',
  v8: '3.14.5.9',
  ares: '1.9.0-DEV',
  uv: '0.10.13',
  zlib: '1.2.3',
  modules: '11',
  openssl: '1.0.1e' }
Scores: (bigger is better)

elasticsearch-memcached
Raw:
 > 5.228771228771229
 > 4.932067932067932
 > 5.014985014985015
 > 5.1688311688311686
Average (mean) 5.086163836163836

elasticsearch
Raw:
 > 1.4155844155844155
 > 1.4475524475524475
 > 1.4195804195804196
 > 1.4175824175824177
Average (mean) 1.425074925074925

Winner: elasticsearch-memcached
Compared with next highest (elasticsearch), it's:
71.98% faster
3.57 times as fast
0.55 order(s) of magnitude faster
QUITE A BIT FASTER

So clearly this is at least worth pursuing further.

I think the first steps are implementing a couple changes to your module:

• Allow an alternative request handler to be passed in createClient(options). This will allow elasticsearch-memcached to just implement the request handler, rather than have to 'copy' createClient() also. It would also open up other possible request handlers that use alternative http libraries (or whatever else people think up).
• Possibly some light refactoring to move processing the actual request options into the request handler.
  • For example, the memcached protocol needs to add ?source={serialized data} to the path for _search requests (searches need to run through get() rather than set()). This would be easier if req.formatParameters() was called inside the request handler.

Anyhow, really just hoping to get your thoughts. I don't want to spin my wheels if you're not interested in these kinds of changes. Thanks!

support thrift api

Support for different search types

Support for different search types (e.g. .../_search?search_type=count) would be nice.

See:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-search-type.html

Query plugins

There's plugin that we use: https://github.com/spinscale/elasticsearch-suggest-plugin
What would be the best way to extend node-elasticsearch functionality to support queries to plugins? Other plugins use different query styles, e.g. /_view/param1/param2 or /_index/_type/__suggest like with suggest plugin. Maybe export "raw" function to construct query as user wants? I.e. function that accepts URI resource: es.query("/myindex/mytype/__suggest").

I get only one field back

I only get the last field('name') from server
If I change the order, I still get only the last one.

elastic_client.search({
        _index : 'sanger',
        _type  : 'products',
        fields: [
            'tags',
            'stores_available_in',
            'product_id',
            'locale',
            'name'
        ]
    }, {
        query : qryObj
    }, function (err, data) {
        callback(data['hits'].hits);
    });

Could you add more documentation on how to use Bulk?

everything else seems pretty straightforward.

thanks!

Question about schema mapping error with es 1.2,2

I created a gist for the mappings I am sending through node-es, with a returned error:
MapperParsingException[Root type mapping not empty after parsing!

The gist is at
https://gist.github.com/KnowledgeGarden/b965b7e78f19f9be9025

I'm wondering if anyone recognizes an error in my mapping. This bug seems to crop up since 1.2

Thanks!

update() method and '' for script parameter

The update() method throws an error if the script parameter is the blank string, '', as it falsely detects this as "not set" in the following code:

if (!doc.script && !doc.doc) {
            return callback(new Error('script or doc is required for update operation'));
        }

This should probably be if (doc.script == null .. ) as having a script with '' is perfectly valid with elasticsearch UPDATE requests, like I ended up having to do, where I have to use ' ' instead of '':

/**
 * Does a mongoDB style $setOnInsert & $set combo upsert
 * (see http://docs.mongodb.org/manual/reference/operator/update/setOnInsert/)
 *
 * @param {es} The elasticsearch client instance
 * @param {options} Options for elasticsearch update() call
 * @param {insert} Document to be inserted
 * @param {update} Document to be updated
 * @param {callback} Callback: function(err, insertResult, updateResult)
 */
function elasticsearchUpsertEx(es, options, insert, update, callback)
{
    // First do the INSERT ONLY portion
    es.update(options,
        {
            'script': ' ', // required because elasticsearch.update() is weird
            'params': {},
            'upsert': insert
        },
        function(err, insertResult)
        {
            if (err)
            {
                if (callback) { callback(err, insertResult, null); return; }
                else throw err;
            }

            // Then do the UPSERT (insert & update) portion
            es.update(options,
                {
                    'doc': update,
                    'doc_as_upsert': true
                },
                function(err, updateResult)
                {
                    if (err && !callback) throw err;
                    if (callback) callback(err, insertResult, updateResult);
                }
            );
        }
    );
}

Mapping index not possible

In ES, one can setup the mapping with various index-related settings and multiple types, all in one request. This is not possible with node-es.

Eg:
curl -XPUT "http://localhost:9200/myindex" -d mapping.json

mapping.json:

{
    settings: {... analyzers, mapping type ...},
    mappings: {
        collection1: {... properties, routing ...},
        collection2: {...},
        collection3: {...}
    },
}

node-es forces a type to be specified everytime an index is specified with indices.putMapping, which reduces the generic ES api to a limited set. Is there a workaround?

Scrolling search results

Hi,

Is there any way to scroll search results (http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-scroll.html or http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-search-type.html#scan)?

I was able to get a scroll_id by using the following options when performing a search:

var options = {
    _index: "myindex",
    search_type: "scan",
    scroll: "10m"
};

This gives me a scroll_id in the data returned by the search, but I have not found any way to use it to actually retrieve the results. How can I do this?

Thank you for your help in advance!

Best regards,
Thomas

I get back "too many" hits

I create several documents which look something like these:
{'locator':'foo', 'instanceOf':'bar'}
{'locator':'bla': 'instanceOf':'bah'}

I've used several kinds of query strings, one of which looks like this:
{"term":{"instanceOf":"bar"}}
another like this:
{"fields":["instanceOf"],"term":{"instanceOf":"bar"}}
another like this:
{"fields":["instanceOf"],"query":{"term":{"instanceOf":"bar"}}}

All queries return all nodes, and don't focus on the lone node requested.
What am I missing?

Count using a query

Looking at the source code it appears there is no way to count with a query? Or am I doing something wrong? Trying to use this:

var options = {
    _index : 'cats',
    _type : 'cat',
    body: {
      "query": {
        "geo_bounding_box": { 
          "location": {
            "top_left": {
              "lat": 43,
              "lon": -72
            },
            "bottom_right": {
              "lat": 40,
              "lon": -74
            }
          }
        }
      }
    }
}
this.es.count(options,function (err, data) {
    console.log(data);
});

To run this:

GET cats/_count
{
  "query": {
    "geo_bounding_box": { 
      "location": {
        "top_left": {
          "lat": 43,
          "lon": -72
        },
        "bottom_right": {
          "lat": 40,
          "lon": -74
        }
      }
    }
  }
}

How about a changelog?

For me a changelog is a checklist I go through before updating any dependency. It's very tricky when there isn't any available. Could you add one? I would really appreciate it a ton.

Remove sys references.

Sys doesn't exist in 0.4.x so those need to be removed or conditionally changed.

Removed is better.