Prompt gets added as first message in messages array and looses its attention about ai

[lgrammel]

I think switching the system prompt half-way through a conversation is an edge case. You already addressed it through your last system prompt (which could also be an injected user message btw.).

Given that the recommend default is to have the system prompt first, and your situation is an edge case with a workaround, I'll close this issue.

[themataleao]

Thanks for the quick response @lgrammel

Yeah I have checked the ai/rsc package and apparently, you are injecting the system prompt as a user message:

    case 'prompt': {
      languageModelMessages.push({
        role: 'user',
        content: [{ type: 'text', text: prompt.prompt }],
      });
      break;
    }

Do you know why it is recommended to have the system prompt first? As written it seems to me to lose attention if the message array gets too long. It could even get cropped if we go above the max_tokens for an API request.

Closing is fine for me.

[lgrammel]

a) no we are not, this is the prompt and not the system message
b) it's recommend to have the system prompt first because that's how the language models are trained. they get lots of examples where the system prompt (inside special tokens) comes at the beginning.

that being said, i understand your issue and ran into it myself. adding a system or user message as the final message can help (i've done this as well), but it is up to you to add it, and it can lead to issues with e.g. tool calls and tool results.

[themataleao]

Okay, thanks for the response and the feedback.

I will try my best to bend it towards my needs then :) I haven't tried with tools calling yet.