Freeze while inserting huge texts about manticoresearch

[tomatolog]

could it be that the huge requests uses all the memory at the box that and box goes swap as it can not handle request at the script along at the daemon network buffer and at the RT index?

[sanikolaev]

@PavelShilin89 pls try to reproduce the issue

[PavelShilin89]

@scruel I can reproduce this bug on manticoresearch/manticore:dev-6.3.1-9bd0267 and version manticoresearch/manticore:dev-6.2.12-dc5144d, but I see in the log...

[Fri Jun 14 09:20:37.234 2024] [1515] watchdog: main process 1516 killed cleanly with SIGKILL, shutting down 

So the process is terminated forcibly, by something from outside. It would be great if you could send me your searchd.log.

[sanikolaev]

I can't reproduce it when there's nothing external killing the searchd:

root@9219fb8b629c:/# python3 test.py
...
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊| 999/1000 [6:16:46<00:22, 22.30s/it]docs = completed in 0.00 seconds.
data = completed in 0.93 seconds.
bulk() completed in 21.76 seconds.
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1000/1000 [6:17:08<00:00, 22.63s/it]
root@9219fb8b629c:/#

root@9219fb8b629c:/# mysql -P9306 -h0 -e "select count(*) from testx"
+----------+
| count(*) |
+----------+
|  1000000 |
+----------+

root@9219fb8b629c:/# du -sh /var/lib/manticore/
52G     /var/lib/manticore/

root@9219fb8b629c:/# ps aux|grep searchd
mantico+    5752  0.0  0.0  56008  1260 ?        S    02:54   0:00 /usr/bin/searchd --config /etc/manticoresearch/manticore.conf
mantico+    5753  108  0.8 16714880 1185308 ?    Sl   02:54 516:51 /usr/bin/searchd --config /etc/manticoresearch/manticore.conf

Version info:

root@9219fb8b629c:/# searchd -v
Manticore 6.3.1 9bd02672d@24061008 dev (columnar 2.3.1 89ed74a@24060612) (secondary 2.3.1 89ed74a@24060612) (knn 2.3.1 89ed74a@24060612)

[scruel]

This issue should be resolved in v6.3.0, will reopen it if not.