The engine caches the data in the translation about opennmt

[guillaumekln]

Hi,

During translation, the memory only increases if the batch size increases or the sentence length increases.

[sdlmw]

Hi @guillaumekln

Can I turn off the caching mechanism? We have a lot of request connections. Frequent cases of low memory.


Now I'm in a situation where the video card memory consumes much more than memory.

[guillaumekln]

Did you already set this environment variable?

THC_CACHING_ALLOCATOR=0

[sdlmw]

Hi @guillaumekln ,

I shouldn't have set it up. Excuse me, how should I do it?

[guillaumekln]

Try:

THC_CACHING_ALLOCATOR=0 th translate.lua <your options>

[sdlmw]

It doesn't look like it has any effect. @guillaumekln