Translation speed is to slow on HDD about libretranslate

[PJ-Finlay]

What do you mean by "cache database indexes in the RAM"? The majority of the time translation takes is performing the translation itself (longer for longer translations). CTranslate2 seems to be a pretty well optimized inference engine and we currently split sentences and then batch translate each one on its own core.

It's possible that preloading the models from disk into memory could speed things up but I'd want to see some evidence of a meaningful speed improvement before adding code complexity. If you're interested in looking into it I'd be curious to hear what you find.

@guillaumekln

Now I understand, the issue mostly related to https://github.com/argosopentech/argos-translate engine as the LibreTranslate is web application based on that project. So maybe we need to close that question here or the engine is different from original version?

[pierotofy]

Translation speed is dependent on argos-translate.

[guillaumekln]

Disk speed only impacts the time it takes to load the model, or when the process is swapping on disk due to a lack of memory. Which one is it here?

[PJ-Finlay]

I meant loading the model from disk to RAM to run inference on it. Argos Translate currently keeps models in memory once they've been used but that's to prevent memory leaks not for performance:

• https://forum.qt.io/topic/120341/how-to-debug-a-hard-to-find-memory-leak-in-pyqt/16
• argosopentech/argos-translate@26802a3