Complex beam search about tensorrt-llm

[byshiue]

"diverse beam search" is supported in TensorRT-LLM, you can enable it through this parameter beam_search_diversity_rate.

I remember "group beam search" is part of "diverse beam search", and we can decide how many groups we want to use. That's what TensorRT-LLM does not support now. The number of group in TensorRT-LLM is fixed when beam_search_diversity_rate is set by non-default value. We will consider how to extend it in the future.

[haramjo]

Oh, sorry, I meant to say "constraint beam search" and "group beam search".

[byshiue]

We don't support constraint beam search, but we support group beam search now.

[ChristianPala]

Hi folks!
Are there updates on constraint beam search?
It's something we would like to integrate.