What about Serverless Functions? about ai

[jaredpalmer]

It works perfectly! Be mindful of costs depending on your provider. Edge functions might be significantly cheaper (and faster).

[NickHeiner]

I think it would be helpful to add an example to the docs. 😄

I'm fumbling around with it myself, but I'm not able to get something working:

export default async function handler(req: VercelRequest, res: VercelResponse) {
  // Ask OpenAI for a streaming completion given the prompt
  const response = await openai.createCompletion({
    model: 'text-davinci-003',
    stream: true,
    prompt: 'List 20 dog names',
  });
 
  // Convert the response into a friendly text-stream
  const stream = OpenAIStream(response);

  streamToResponse(stream, res);
}

When I run this, I get a short response:

curl --no-buffer http://localhost:3000/api/demo
1. Spot
2. Milo
3. Max
4.⏎ 

Based on what I see in the playground, I think the full response is longer, and I'm doing the streaming wrong, such that it's only returning the first chunk.

Maybe --no-buffer is not what I should pass to curl? (Removing that flag doesn't change the behavior I see.) When I visit that endpoint in the browser, I get the same result.

This is the branch in which I'm seeing this, in case it's helpful.

[NickHeiner]

Actually, never mind – I figured it out. I think the issue is just that I wasn't setting max_tokens in my code, but the Playground defaulted it to 200.