LLM seems to ignore the grammar I'm setting for it

#404 · open · 0 comments

View on GitHub ↗

HossamBasion

I'm trying to force my LLM to only output 5 to 15 words Arabic output using the attached file (It's a gbnf file but I uploaded it as a .txt file so I can attach it here) [WordCount.txt](https://github.com/user-attachments/files/27178973/WordCount.txt) And if it helps, this is my system prompt: "I will give you a topic. Respond with one short sentence in Egyptian Arabic language. The sentence must be between five words and fifteen words ONLY. Just produce a simple, everyday Egyptian sentence that mentions the topic naturally. Return only the sentence and nothing else. Under no circumstance should you ever respond in more than fifteen words . And your response must always be more than five words. This is the most important thing. And always make sure to respond in Egyptian Arabic Language Remember a word is defined as letters surrounded by whitespace" It does generate Arabic output correctly. But no matter what I do, it keeps outputting all kinds of word counts including a lot of two word responses and sometimes even one word responses. This is an example of something it generated: السجاد جميل (translates to "the carpets are beautiful" but it's only two words) I don't even use the SetGrammar function. I just use the loadGrammar button to load the grammar document I attached here. ### Steps to reproduce 1. Load any LLM gguf and keep all default settings the same. 2. Use the System prompt and grammar file I posted. 3. Create a for loop and run the llm for let's say 20 times: response = await Allam.Chat(topic, null, null, false); 4. Observe how many sentences ignore the grammar file word count requirement. ### LLMUnity version 6000.0.50 ### Operating System Windows

Comments