Can someone point me out where the data is stored? about manticoresearch

[barryhunter]

Have you actually looked at the source code? Seems like this change is going to require a LOT of changes in the code, and quite deep understanding.

Whats the end goal here? Seems a lot of work, for little benefit. Can't see a key/value store, being an ideal mechanism for storing an Inverted keyword Index.

[tomatolog]

In case you need just data-storage you might contact us at our site contact tab.
In case you just want to write new storage you might look how strings attributes got managed and write something similar.

[unisqu]

In case you need just data-storage you might contact us at our site contact tab.
I would like to explore making sphinx search 's data storage tied with other database / nosql backend system for distributed storage purposes.

In case you just want to write new storage you might look how strings attributes got managed and write something similar.
Please also point me to the right direction to this issue.

@barryhunter I would like to split the storage across multiple resilient storage backend. Not leveldb but other nosql / sql solutions.

[barryhunter]

Could you just put the index files on a distributed file system? Avoid having to redo the storage layer.

The main complication is the .spl lock files. each searchd process wants to create its own spl file, so if all searchd servers point to the same folder it doesnt work. I've used a complicated structure using symlinks, but it's messy (particully around index rotation)

[unisqu]

@barryhunter distributed filesystem adds another layer of complexity to the scaling process.
I would like to scale it through manipulation of the storage layer. Please just point me to a direction for this and i'll try to port one nosql ontop of it. Please please give me an idea how and which files to look at for the porting of what i need.

i will also be a contributor to this project (in other areas if needed)

[barryhunter]

This seems a good place to start:
https://github.com/manticoresoftware/manticore/search?l=C%2B%2B&q=fopen&type=&utf8=%E2%9C%93
or maybe
https://github.com/manticoresoftware/manticore/search?l=C%2B%2B&q=open&type=&utf8=%E2%9C%93

Leads to
https://github.com/manticoresoftware/manticore/search?utf8=%E2%9C%93&q=OpenFile&type=

src/sphinxint.h also looks to have a number of the function definitions of interest

Not trying to be awkward, but seems you need to do this 'digging' as you need to understand the code to be able to modify it.

[tomatolog]

as I said you might look for implementation of SPH_ATTR_STRING or SPH_ATTR_UINT32SET and try to implement similar attribute type.

You might start your investigation from sphinx.cpp file.

[unisqu]

ok thanks. hope that's good enough. I will work on reading them.