apostrophecms / apostrophe-elasticsearch Goto Github PK

Apply rules similar to those used by the built-in search engine, so that textual schema fields are seen unless they ask not to be. Don't let Lucene eat everything, there could be security concerns.

it can take a long time!

The project that i support is currently running apostrophe 2.111.3. I was trying to integrate apostrophe-elasticsearch version 2.1.3. I installed elasticsearch via a docker image that i was running locally following the instructions below
https://www.elastic.co/guide/en/elasticsearch/reference/7.9/docker.html. The current project as a large amount of content, especially apostrophe-blogs. I used elasticsearch version 7.9.1. After installing apostrophe-elasticsearch and running the node app apostrophe-elasticsearch:reindex --verbose command
I received the following error:
StatusCodeError: [mapper_parsing_exception] Root mapping definition has unsupported parameters: [aposDoc : {properties={tagsESExact={type=keyword}, lowSearchTextESExact={type=keyword}, titleESExact={type=keyword}, typeESExact={type=keyword}, highSearchText={type=text}, slugESExact={type=keyword}, title={type=text}, type={type=text}, tags={type=text}, path={type=text}, highSearchTextESExact={type=keyword}, lowSearchText={type=text}, slug={type=text}, pathESExact={type=keyword}}}]

as a workaround i navigated to the node_modules/apostrophe-elasticsearch/index.js, and updated lines 178 -190 with the following code:

return self.client.indices.create({ index: self.getLocaleIndex(locale), body: { settings: self.getLocaleSettings(locale), mappings: { // aposDoc: { properties: properties // } } } }, callback);

The reindex was able to complete, but it was not successful the docker logs, showed the following error:
{"type": "server", "timestamp": "2020-09-17T21:25:19,354Z", "level": "INFO", "component": "o.e.a.b.TransportShardBulkAction", "cluster.name": "docker-cluster", "node.name": "39e14168f092", "message": "[s-g-cmsaposdocsendraft][0] mapping update rejected by primary", "cluster.uuid": "Ilv6dTTpSWq9rJIhNAa4-g", "node.id": "J9B_vr18TIaahyOvVbgu9Q" , "stacktrace": ["java.lang.IllegalArgumentException: mapper [titleESExact] cannot be changed from type [keyword] to [text]",

This appears to be an issue with the titleESExact and the type property, i have tried commenting out the title in optional fields, but then highSearchTextESExact threw the same error where it cannot be changed from type [keyword] to [text].

The apostrophe-elasticsearch still works with elasticsearch version 6.8.0. This just needs to address the breaking changes from elasticsearch 7 and above.

To Reproduce

Step by step instructions to reproduce the behavior:

1. Pull the latest version of elasticsearch, 7.9.1, from docker, by following the instructions in the link above
2. Run the docker container and ensure it's status is running.
3. Add apostrophe-elasticsearch to an existing apostrophe cms project, preferably the latest version.
4. Run node app apostrophe-elasticsearch:reindex --verbose

Expected behavior

Running the apostrophe-elasticsearch:reindex actually reindexes the content allows for searching content on the cms with data stored in mongodb instance.

Describe the bug

The apostrophe-elasticsearch reindex is not reindexing content from mongodb. The mapping types needs to be addressed to work with elasticsearch versions 7 and above.

Version of Node.js:
We are running node 14.2.0
mongo:4.2.3

For both familiarity and safety (full lucene can DOS the server if used maliciously).

Hi, first of all, thanks a ton for making this, it's saved me a bunch of time.

I've created an analyzer for autocomplete that needs to be added to the fields when they're initialized. I saw the fields setting i.e fieds: ['tags'] etc but what I need to do is add the anaylzer to each of these fields, below the type: 'text' in the object.

I saw the function responsible in your module here:

self.createIndexes = function(callback) {
      const locales = self.getLocales();
      return async.eachSeries(locales, function(locale, callback) {
        const properties = {};
        _.each(self.fields, function(field) {
          properties[field] = {
            type: 'text'
          };
          properties[field + 'ESExact'] = {
            type: 'keyword'
          };
        });
        return self.client.indices.create({
          index: self.getLocaleIndex(locale),
          body: {
            settings: self.getLocaleSettings(locale),
            mappings: {
              aposDoc: {
                properties: properties
              }
            }
          }
        }, callback);
      }, callback);
    };

and what I've done for testing purposes is add the anylzer to the module and it appeared to work.

 _.each(self.fields, function(field) {
          properties[field] = {
            type: 'text',
            analyzer: 'autocomplete'
          };
          properties[field + 'ESExact'] = {
            type: 'keyword'
          };
        });

I'm new to Elastichsearch and this module and I was wondering if I'm doing this right way? Or is there a way to do this that I'm missing. Obviously I don't want to edit the node_module directly and just did it for testing purposes.

If a query is, for instance, restricted by workflowLocale or tag or type (or all of the above), ES should build queries smart enough to winnow things down at least that much so that we don't have to troll through 50 times as much stuff before we get to stuff that satisfies the mongo criteria.