trying to index attributes which are not meant to be indexed?

ryders

Hi folks, We use `include Elasticsearch::Model` in order to index a rails model into an elasticsearch index (indice) and I've come across a problem which I'm not sure should be raised as a bug. Before I do so, I'd like to confirm my understanding, as well as validate that my troubleshooting is correct. ## Seamless integration with models When `include Elasticsearch::Model` is used in a model, the `as_indexed_json` method is called in order to automatically send/index a model into elasticsearch when the model `.save` method is called. The model `settings do... mappings do...` allow to define WHICH attributes are actually indexed/how they are analysed, etc. ### Problem •1 Although of a lesser importance, but I seem to have confirmed that if an index/indice is not created before hand, the `.save` method will automatically create the index, ignoring the settings defined (namely `dynamic: 'false'` that I'm using) and as a result ES automatically indexes all model attributes. Manually calling the `model.__elasticsearch__.create_index!` first works around this. This also only seems to be a problem when a model is created without the index present. if I use `model.import` without the index present, settings are observed. **Or am I crazy?** ## Configuration I'm trying to implement; 1. dynamic: 'false'; 2. _source: false; ### Problem •2 The main problem I have here is that when I do `model.save` and only a single attribute was changed, the gem is 'smart enough' to only send the attribute changed to the existing document in ES, which in turns throws up a tantrum because it has no clue about that attribute (thanks to my config#2 -- source:false, WITH source, it doesn't index, but doesn't whinge). The gem however is ignoring the settings configuration in this case, not unlike problem •1. When calling `model.__elasticsearch__.index_document`: the whole document is sent (`-d {"id":"c479a116-d78c-4957-8591-c791415d486f"}`) ``` curl -X PUT -H 'User-Agent: elasticsearch-ruby/8.16.0; elastic-transport-ruby/8.3.5; RUBY_VERSION: 3.3.4; linux aarch64, x-elastic-client-meta: es=8.16.0,rb=3.3.4,t=8.3.5,fd=1.10.3,ty=1.4.1, Content-Type: application/json' 'http://localhost:9200/companies/_doc/c479a116-d78c-4957-8591-c791415d486f?pretty' -d '{"id":"c479a116-d78c-4957-8591-c791415d486f","legal_name":...... ``` When calling `model.save` with only the `.packaging` attribute changed: a partial document is sent (`-d '{"doc":{"packaging":`) ``` curl -X POST -H 'User-Agent: elasticsearch-ruby/8.16.0; elastic-transport-ruby/8.3.5; RUBY_VERSION: 3.3.4; linux aarch64, x-elastic-client-meta: es=8.16.0,rb=3.3.4,t=8.3.5,fd=1.10.3,ty=1.4.1, Content-Type: application/json' 'http://localhost:9200/companies/_update/c479a116-d78c-4957-8591-c791415d486f?pretty' -d '{"doc":{"packaging":[{"id":"056efb26-ad3a-4e43-a7b9-242dba99ed1b"...... ``` **Is my understanding correct?** And if so, can I get the gem to observe the model settings before sending? or always send the whole document?.

ryders

Ok, doing more investigation I seem to have answered Problem •1; I was not crazy 😅 and just discovered this is an elastic behaviour to [auto create index](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-creation) when missing, a setting activated by default. Nothing to do with the gem. [As documented](https://github.com/elastic/elasticsearch-rails/tree/main/elasticsearch-model#index-configuration). When turning this off, the model creation (when the underlying index doesn't exist) correctly returns a `index_not_found_exception`, with the genius message 😍 `no such index [companies] and [action.auto_create_index] is [false]`. Now onto the Problem •2. ~~I have confirmed that with/without `_source` enabled, the server 'never' likes receiving attributes which are not indexed/mapped. For all intents and purposes, this renders the [documented](https://github.com/elastic/elasticsearch-rails/tree/main/elasticsearch-model#index-configuration) `mappings dynamic: 'false' do` unusable given any attribute is sent to ES when changed/updated?~~ With my elastic knowledge deepening, I've come to realise that using `_source: {enabled: false}`, partial updates are **never** going to work given, the document is essentially 'reconstructed from _source' before being indexed. So now.. I can't seem to find a setting or config to always send the whole `as_indexed_json` each time... Any guidance/help in confirming my thinking and pointing me in the right direction would be appreciated 👍