elastic / app-search-node

Elastic App Search Official Node.js Client

⚠️ This client is deprecated ⚠️

As of Enterprise Search version 8.3.2, we are directing users to the new Enterprise Search Node Client and deprecating this client.

Our development effort on this project will be limited to bug fixes. All future enhancements will be focused on the Enterprise Search Node Client.

Thank you! - Elastic

A first-party Node.JS client for building excellent, relevant search experiences with Elastic App Search.

Contents

• Getting started
• Versioning
• Usage
• Running tests
• FAQ
• Contribute
• License

---

Getting started 🐣

To install this package, run:

npm install @elastic/app-search-node

Versioning

This client is versioned and released alongside App Search.

To guarantee compatibility, use the most recent version of this library within the major version of the corresponding App Search implementation.

For example, for App Search 7.3, use 7.3 of this library or above, but not 8.0.

If you are using the SaaS version available on swiftype.com of App Search, you should use the version 7.5.x of the client.

Usage

Setup: Configuring the client and authentication

Using this client assumes that you have already an instance of Elastic App Search up and running.

The client is configured using the baseUrlFn and apiKey parameters.

const apiKey = 'private-mu75psc5egt9ppzuycnc2mc3'
const baseUrlFn = () => 'http://localhost:3002/api/as/v1/'
const client = new AppSearchClient(undefined, apiKey, baseUrlFn)

Note:

The [apiKey] authenticates requests to the API. You can use any key type with the client, however each has a different scope. For more information on keys, check out the documentation.

Swiftype.com App Search users:

When using the SaaS version available on swiftype.com of App Search, you can configure the client using your hostIdentifier instead of the baseUrlFn parameter. The hostIdentifier can be found within the Credentials menu.

const AppSearchClient = require('@elastic/app-search-node')
const hostIdentifier = 'host-c5s2mj'
const apiKey = 'private-mu75psc5egt9ppzuycnc2mc3'
const client = new AppSearchClient(hostIdentifier, apiKey)

API Methods

Indexing: Creating or Replacing Documents

const engineName = 'favorite-videos'
const documents = [
  {
    id: 'INscMGmhmX4',
    url: 'https://www.youtube.com/watch?v=INscMGmhmX4',
    title: 'The Original Grumpy Cat',
    body: 'A wonderful video of a magnificent cat.'
  },
  {
    id: 'JNDFojsd02',
    url: 'https://www.youtube.com/watch?v=dQw4w9WgXcQ',
    title: 'Another Grumpy Cat',
    body: 'A great video of another cool cat.'
  }
]

client
  .indexDocuments(engineName, documents)
  .then(response => console.log(response))
  .catch(error => console.log(error))

Note that this API will not throw on an indexing error. Errors are inlined in the response body per document:

[
  { "id": "park_rocky-mountain", "errors": [] },
  {
    "id": "park_saguaro",
    "errors": ["Invalid field value: Value 'foo' cannot be parsed as a float"]
  }
]

Indexing: Updating Documents (Partial Updates)

const engineName = 'favorite-videos'
const documents = [
  {
    id: 'INscMGmhmX4',
    title: 'Updated title'
  }
]

client
  .updateDocuments(engineName, documents)
  .then(response => console.log(response))
  .catch(error => console.log(error))

Retrieving Documents

const engineName = 'favorite-videos'
const documentIds = ['INscMGmhmX4', 'JNDFojsd02']

client
  .getDocuments(engineName, documentIds)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Listing Documents

const engineName = 'favorite-videos'

// Without paging
client
  .listDocuments(engineName)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

// With paging
client
  .listDocuments(engineName, { page: { size: 10, current: 1 } })
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Destroying Documents

const engineName = 'favorite-videos'
const documentIds = ['INscMGmhmX4', 'JNDFojsd02']

client
  .destroyDocuments(engineName, documentIds)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Listing Engines

client
  .listEngines({ page: { size: 10, current: 1 } })
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Retrieving Engines

const engineName = 'favorite-videos'

client
  .getEngine(engineName)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Creating Engines

const engineName = 'favorite-videos'

client
  .createEngine(engineName, { language: 'en' })
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Destroying Engines

const engineName = 'favorite-videos'

client
  .destroyEngine(engineName)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Searching

const engineName = 'favorite-videos'
const query = 'cat'
const searchFields = { title: {} }
const resultFields = { title: { raw: {} } }
const options = { search_fields: searchFields, result_fields: resultFields }

client
  .search(engineName, query, options)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Multi-Search

const engineName = 'favorite-videos'
const searches = [
  { query: 'cat', options: {
      search_fields: { title: {} },
      result_fields: { title: { raw: {} } }
  } },
  { query: 'grumpy', options: {} }
]

client
  .multiSearch(engineName, searches)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Query Suggestion

const engineName = 'favorite-videos'
const options = {
  size: 3,
  types: {
    documents: {
      fields: ['title']
    }
  }
}

client
  .querySuggestion(engineName, 'cat', options)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Listing Curations

const engineName = 'favorite-videos'

client
  .listCurations(engineName)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

// Pagination details are optional
const paginationDetails = {
        page: {
          current: 2,
          size: 10
        }
      }

client
  .listCurations(engineName, paginationDetails)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Retrieving Curations

const engineName = 'favorite-videos'
const curationId = 'cur-7438290'

client
  .getCuration(engineName, curationId)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Creating Curations

const engineName = 'favorite-videos'
const newCuration = {
  queries: ['cat blop'],
  promoted: ['Jdas78932'],
  hidden: ['INscMGmhmX4', 'JNDFojsd02']
}

client
  .createCuration(engineName, newCuration)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Updating Curations

const engineName = 'favorite-videos'
const curationId = 'cur-7438290'
// "queries" is required, either "promoted" or "hidden" is required.
// Values sent for all fields will overwrite existing values.
const newDetails = {
  queries: ['cat blop'],
  promoted: ['Jdas78932', 'JFayf782']
}

client
  .updateCuration(engineName, curationId, newDetails)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Deleting Curations

const engineName = 'favorite-videos'
const curationId = 'cur-7438290'

client
  .destroyCuration(engineName, curationId)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Retrieving Schemas

const engineName = 'favorite-videos'

client
  .getSchema(engineName)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Updating Schemas

const engineName = 'favorite-videos'
const schema = {
  views: 'number',
  created_at: 'date'
}

client
  .updateSchema(engineName, schema)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Create a Signed Search Key

Creating a search key that will only return the title field.

const publicSearchKey = 'search-xxxxxxxxxxxxxxxxxxxxxxxx'
// This name must match the name of the key above from your App Search dashboard
const publicSearchKeyName = 'search-key'
const enforcedOptions = {
  result_fields: { title: { raw: {} } },
  filters: { world_heritage_site: 'true' }
}

// Optional. See https://github.com/auth0/node-jsonwebtoken#usage for all options
const signOptions = {
  expiresIn: '5 minutes'
}

const signedSearchKey = AppSearchClient.createSignedSearchKey(
  publicSearchKey,
  publicSearchKeyName,
  enforcedOptions,
  signOptions
)

const baseUrlFn = () => 'http://localhost:3002/api/as/v1/'
const client = new AppSearchClient(undefined, signedSearchKey, baseUrlFn)

client.search('sample-engine', 'everglade')

Create a Meta Engine

const engineName = 'my-meta-engine'

client
  .createMetaEngine(engineName, ['source-engine-1', 'source-engine-2'])
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Add a Source Engine to a Meta Engine

const engineName = 'my-meta-engine'

client
  .addMetaEngineSources(engineName, ['source-engine-3'])
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Remove a Source Engine from a Meta Engine

const engineName = 'my-meta-engine'

client
  .deleteMetaEngineSources(engineName, ['source-engine-3'])
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

Creating Engines

const engineName = 'my-meta-engine'

client
  .createEngine(engineName, {
    type: 'meta',
    source_engines: ['source-engine-1', 'source-engine-2']
  })
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))

For App Search APIs not available in this client

We try to keep this client up to date with all of the available API endpoints available from App Search.

There are a few APIs that may not be available yet. For those APIs, please use the low-level client to connect to hit any App Search endpoint.

const engineName = 'favorite-videos'
const options = {
  query: 'cats'
}

const Client = require('@elastic/app-search-node/lib/client')
const client = new Client('private-mu75psc5egt9ppzuycnc2mc3', 'http://localhost:3002/api/as/v1/')
client.post(`engines/${encodeURIComponent(engineName)}/search`, options).then(console.log)

Running tests

npm test

The specs in this project use node-replay to capture fixtures.

New fixtures should be captured from a running instance of App Search.

To capture new fixtures, run a command like the following:

nvm use
HOST_IDENTIFIER=host-c5s2mj API_KEY=private-b94wtaoaym2ovdk5dohj3hrz REPLAY=record npm run test -- -g 'should create a meta engine'

To break that down a little...

• HOST_IDENTIFIER - Use this to override the fake value used in tests with an actual valid value for your App Search instance to record from
• API_KEY - Use this to override the fake value used in tests with an actual valid value for your App Search instance to record from
• REPLAY=record - Tells replay to record a new response if one doesn't already exist
• npm run test - Run the tests
• -- -g 'should create a meta engine' - Limit the tests to ONLY run the new test you've created, 'should create a meta engine' for example

This will create a new fixture, make sure you manually edit that fixture to replace the host identifier and api key recorded in that fixture with the values the tests use.

You'll also need to make sure that fixture is located in the correctly named directory under fixtures according to the host that was used.

You'll know if something is not right because this will error when you run npm run test with an error like:

Error: POST https://host-c5s2mj.api.swiftype.com:443/api/as/v1/engines refused: not recording and no network access

FAQ 🔮

Where do I report issues with the client?

If something is not working as expected, please open an issue.

Where can I learn more about App Search?

Your best bet is to read the documentation.

Where else can I go to get help?

You can checkout the Elastic App Search community discuss forums.

Contribute 🚀

We welcome contributors to the project. Before you begin, a couple notes...

• Prior to opening a pull request, please create an issue to discuss the scope of your proposal.
• Please write simple code and concise documentation, when appropriate.

License 📗

Apache 2.0 © Elastic

Thank you to all the contributors!

app-search-node's Issues

Add method update partial index

I see in the documentation there is update partial but in @elastic/app-search-node there is no this method.

version: 7.6.0

https://swiftype.com/documentation/app-search/api/documents#partial

appSearch.js

/**
   * Partial update a batch of documents.
   *
   * @param {String} engineName unique Engine name
   * @param {Array<Object>} documents Array of document objects to be updated.
   * @returns {Promise<Array<Object>>} a Promise that returns a result {Object} when resolved, otherwise throws an Error.
   */
  updateDocuments(engineName, documents) {
    return this.client.patch(`engines/${encodeURIComponent(engineName)}/documents`, documents)
  }

client.js

  patch(path, params) {
    return this._jsonRequest('PATCH', path, params)
  }

Fix jsonwebtoken <=8.5.1 security issue

Hi,

Can you please fix the security issue of jsonwebtoken package by updating it to v9.0.0. There is no break change AFAIK.

# npm audit report

jsonwebtoken  <=8.5.1
Severity: high
jsonwebtoken unrestricted key type could lead to legacy keys usage  - https://github.com/advisories/GHSA-8cf7-32gw-wr33
jsonwebtoken has insecure input validation in jwt.verify function - https://github.com/advisories/GHSA-27h2-hvpr-p74q
jsonwebtoken's insecure implementation of key retrieval function could lead to Forgeable Public/Private Tokens from RSA to HMAC - https://github.com/advisories/GHSA-hjrf-2m68-5959
jsonwebtoken vulnerable to signature validation bypass due to insecure default algorithm in jwt.verify() - https://github.com/advisories/GHSA-qwph-4952-7xr6
No fix available
node_modules/jsonwebtoken
  @elastic/app-search-node  *
  Depends on vulnerable versions of jsonwebtoken
  node_modules/@elastic/app-search-node

2 vulnerabilities (1 moderate, 1 high)

Thank you.

Is there a way to return a calculated distance with App Search?

Elasticsearch allows script_fields but I don't think that extends to app search. I want to show the distance between two users without returning the exact coordinates to the client side, is this possible?

{
    "query": "",
    "filters": {
        "location": {
            "center": "a, b",
            "distance": 1000,
            "unit": "mi"
        }
    }

My current code filters by location but does not return the distance in the response.

missing click api

https://www.elastic.co/guide/en/app-search/current/clickthrough.html

Allow configuring JWT signature options for signed keys

Currently, the options are hard coded as { algorithm: SIGNED_SEARCH_TOKEN_JWT_ALGORITHM }. So if you wanted to pass in an option such as expiresIn, you can not.

PR incoming

Feature Requests: Add support for retries and custom http agent

It would be nice if retries could be configured and a sensible default set (sometimes I hit a bad gateway)

Also it would be nice if there was an ability to set a custom http agent, but one with with a sensible max connections and keep alive timeout so that connections can be reused server side. Currently every request is made over a brand new connection and incurs expensive handshake costs.

Inconsistently Designed Endpoints

Here are two endpoints from AppSearch.js. indexDocument properly handles errors returned from ElasticSearch API. None of the other endpoints do this, including indexDocuments below. Instead, the user has to manually check the response for errors and handle the errors manually.

This inconsistent behavior had me going in loops trying to figure out why documents were not being indexed, since they were failing in elastic search but the api client was letting them fail silently.

  /**
   * Index a document.
   *
   * @param {String} engineName unique Engine name
   * @param {Object} document document object to be indexed.
   * @returns {Promise<Object>} a Promise that returns a result {Object} when resolved, otherwise throws an Error.
   */
  indexDocument(engineName, document) {
    return this.indexDocuments(engineName, [document])
      .then((processedDocumentResults) => {
        return new Promise((resolve, reject) => {

          const processedDocumentResult = processedDocumentResults[0]
          const errors = processedDocumentResult['errors']
          if (errors.length) {
            reject(new Error(errors.join('; ')))
          }
          delete processedDocumentResult['errors']
          resolve(processedDocumentResult)
        })
      })
  }

  /**
   * Index a batch of documents.
   *
   * @param {String} engineName unique Engine name
   * @param {Array<Object>} documents Array of document objects to be indexed.
   * @returns {Promise<Array<Object>>} a Promise that returns a result {Object} when resolved, otherwise throws an Error.
   */
  indexDocuments(engineName, documents) {
    return this.client.post(`engines/${encodeURIComponent(engineName)}/documents`, documents)
  }

More details on the parameters for the search function

Hi, I was looking at the documentation for the search function and was looking for more details on the parameters that should be passed in:

const query = 'cat'
const searchFields = { title: {} }
const resultFields = { title: { raw: {} } }
const options = { search_fields: searchFields, result_fields: resultFields }

client
  .search(engineName, query, options)
  .then(response => console.log(response))
  .catch(error => console.log(error.errorMessages))```

I have documents in my index with a number of fields including: category, title, location, name
I'm looking to execute a query on the name field, given specified values for category, title and location. 

Any help would be greatly appreaciated.

Syntax for match/term queries

Hi, is it possible to perform match queries using this library?

{
  "query": {
    "match": {
      "full_text": "Blue Tea"
    }
  }
}

OR

{
  "query": {
    "term": {
      "product_type": {
        "value": "Blue Tea",
      }
    }
  }
}

I'd like to get 2 separate results for 'Blue Tea' and 'Red Tea' for example, and I'm not sure it's possible with the example in the documentation.

Thanks!

Corrects redundant variable nomenclature

In the file "test/test.js" in the describe #multiSearch function the nomenclature "res" is used repeatedly, which causes confusion in the indentation, due to the declaration in the upper scope.

A simple and quick solution would be to implement the following code:

CURRENT:

.then((resp) => { assert.deepEqual(resp.map(res => res.results.map(res => res.title.raw)), [[

SUGGESTION:

.then((response) => { assert.deepEqual(response.map(resp => resp.results.map(res => res.title.raw)), [[

Or we can even rename the variables in more detail within each scope.

No support to handle timeouts

We currently are having issues where Swiftype occasionally times out. This was opened as an issue on the previous repo but doesn't seem to have ever been implemented. swiftype/swiftype-app-search-node#32

It would be ideal if we could pass a timeout parameter to request.js

Lates NPM version does not match repository

When installing the latest version from npm 7.6.0, it's missing the latest functions like listDocuments.

If copying the /lib folder from the repo they work.

[ 'self signed certificate in certificate chain' ]

Please help how to sort out this error without buying ssl certificate

Regards,
SITARAM LAB

Client doesn't work with intermediate SSL certs

This is caused by request/request#3106.

Typescript support

It would be great to have out of the box typescript support for this and other libraries.

How to filter for documents where field is null?

I would like to query records and filter for those where a certain field is null. How can I accomplish that?

Example body to /search endpoint

{
    "query": "blah",
    "filters": {
        "all": [
            {
                "user_id": "abcd123"
            },
            {
                "org_id": null
            }
        ]
    }
}

Thank you!

method "destroyDocuments" fails

• app-serach-node version: 7.9.0
• System: ubuntu 20.04.1
• node version: v14.5.0

when calling:
client.destroyDocuments(engine, documentIds)

I got the following error:

.../node_modules/@elastic/app-search-node/lib/client.js:24
  if (responseBody.errors) {
                   ^

TypeError: Cannot read property 'errors' of undefined

I then tried fixing the problem by adding a test for undefined in client.js before line 24 like this:
if (!responseBody) return []

It seemed to work but I got the following error:

Error: Http method DELETE is not supported by this URL
    at Request._callback (.../node_modules/@elastic/app-search-node/lib/client.js:96:26)
    at Request.self.callback .../node_modules/request/request.js:185:22)
    at Request.emit (events.js:314:20)
    at Request.<anonymous> .../node_modules/request/request.js:1154:10)
    at Request.emit (events.js:314:20)
    at IncomingMessage.<anonymous> (.../node_modules/request/request.js:1076:12)
    at Object.onceWrapper (events.js:420:28)
    at IncomingMessage.emit (events.js:326:22)
    at endReadableNT (_stream_readable.js:1226:12)
    at processTicksAndRejections (internal/process/task_queues.js:80:21) {
  errorMessages: []

Using Postman to delete the documents worked without issue!

Need an example of using a signed search key

https://github.com/elastic/app-search-python#create-a-signed-search-key

https://swiftype.com/documentation/app-search/authentication#signed

This is the example used in the documentation. It needs to be updated:

const SwiftypeAppSearchClient = require('swiftype-app-search-node')

const readOnlyApiKey = 'private-xxxxxxxxxxxxxxxxxxxx'
const apiKeyName = 'private-key'
const enforcedOptions = {
  result_fields: { title: { raw: {} } },
  filters: { world_heritage_site: 'true' }
}

const signedSearchKey = SwiftypeAppSearchClient.createSignedSearchKey(readOnlyApiKey, apiKeyName, enforcedOptions)

const client = new SwiftypeAppSearchClient('host-2376rb', signedSearchKey)
client.search('national-parks-demo', 'everglade')

Add support for schemas

Currently, there is no support for the schema API (https://www.elastic.co/guide/en/app-search/current/schema.html)

Is this something that would be accepted if a PR was made for adding support?

Update the overview page docs page

Update constructor in 8.0 to remove deprecated "host" parameter

User-Agent missing in Appsearch

How I can put "User-Agent" header data into Appsearch.
I try to edit code in file "client.js" as follows.

  _jsonRequest(method, path, params) {
    return this._wrap({
      method: method,
      url: `${this.baseUrl}${path}`,
      json: params,
      auth: {
        bearer: this.apiKey
      },
      headers: {
        'X-Swiftype-Client': this.clientName,
        'X-Swiftype-Client-Version': this.clientVersion,
        'Content-Type': 'application/json',
        'User-Agent': `${this.clientName} - ${this.clientVersion}`,  // Edit this line
      }
    })
  }

It works but has another solution?

Can't connect to my Elastic App Search Engine

Hi there,

I can't seem to find the right URL to use to connect to my engine:

const AppSearchClient = require('@elastic/app-search-node')
const apiKey = 'private-key'
const baseUrlFn = () => 'http://localhost:3002/api/as/v1/engines/example-engine/documents'
const client = new AppSearchClient(undefined, apiKey, baseUrlFn)
const engineName = 'example-engine'

client.indexDocuments(engineName, productSet).then(response => console.log(response)).catch(error => console.log(error))

I get this error:

{
  errno: -111,
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 3002,
  errorMessages: [ 'connect ECONNREFUSED 127.0.0.1:3002' ]
}

If I use the API endpoint from my App Search dashboard:

const baseUrlFn = () => 'https://instance-id.ent-search.us-east-1.aws.cloud.es.io'

I get this error:

{
  errno: -3008,
  code: 'ENOTFOUND',
  syscall: 'getaddrinfo',
  hostname: 'instance-id.ent-search.us-east-1.aws.cloud.es.ioengines',
  errorMessages: [
    'getaddrinfo ENOTFOUND instance-id.ent-search.us-east-1.aws.cloud.es.ioengines'
  ]
}

I'm not sure where or how to construct the proper baseUrlFn

Please advise, thank you!!

Indentation with EsLint and Prettier

Is implementing a standard indentation for the project used Eslint and Prettier seen as interesting?

No option for updating field using string

I was trying to update a document with help of a script but didn't find a way through which we can do this.

The same queries in kibana work as expected. But whenever I add a script in app-search API it just adds another field named "script" and adds the script in a stringified format as its value.

Please add search_settings endpoint to api client

Feature request: We have many environments, and we want to be able to easily create engines and tweak their settings programmatically on various environments without needing to log in. I know I can use the low level api directly but this would be a lot more consistent. Thanks!