Issue while compiling Mistral 7B 0.2 Instruct about transformers-neuronx

[awsilya]

@josete89 this is a known issue with the fix that should be available in the next release. Also, Mistral 0.1 should not be affected, you could try it in the meantime.

[jimburtoft]

@josete89 It looks like it is similar to this issue:
#71

You can test it by downloading locally and editing the sliding_window value in config.json to 4096 before you compile.

[jimburtoft]

Changing the sliding_window worked. See https://huggingface.co/aws-neuron/Mistral-7B-Instruct-v0.2-seqlen-2048-bs-1-cores-2

[aws-donkrets]

jimburtoft. Great to hear that worked for you. If there is no other issue can we close this ticket?

[mrnikwaws]

Since we haven't heard back for two weeks, closing this ticket