GithubHelp home page GithubHelp logo

st3w4r / openai-partial-stream Goto Github PK

View Code? Open in Web Editor NEW
65.0 5.0 2.0 735 KB

Turn a stream of token into a parsable JSON object as soon as possible. Enable Streaming UI for AI app based on LLM.

Home Page: https://partial.stream/

License: MIT License

TypeScript 44.04% Makefile 0.91% HTML 54.89% JavaScript 0.16%
chatgpt chatgpt-api function-calling gpt-3 gpt-4 json-parser json-parsing llm openai openai-api

openai-partial-stream's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

openai-partial-stream's Issues

Does JsonCloser work for nested objects?

I have an entity/schema that has nested objects. Even with StreamObjectKeyValueTokens, my streaming behaves more like StreamObjectKeyValue whereby one field only becomes available when its value is complete. I suspect it's because JsonCloser doesn't work with a entity/schema with nested objects.

Azure OpenAI: TypeError: Cannot read properties of undefined (reading 'delta')

I am using Azure OpenAI and trying to use this library for streaming the json object. Getting this error

error: TypeError: Cannot read properties of undefined (reading 'delta')
      at OpenAiHandler.process (/proj/backend/openai-partial-stream/packages/openai-partial-stream/dist/index.js:205:32)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async Entity.genParse (/proj/backend/openai-partial-stream/packages/openai-partial-stream/dist/index.js:270:22)
      at async /proj/backend/index.js:389:30

I tried to fix it like this but did not work
Screenshot 2024-06-08 at 19 35 58

NextJS 13 TS/Zod dependency issue

First off thank you for contributing towards building this. I think this is super useful in building streaming applications with function calling.

I am trying to use this in React with NextJS 13. I am also using the Vercel AI SDK's useCompletion hook on the client side that then calls an API route in the api directiory to stream the responses. I get this error with this setup:

./node_modules/typescript/lib/typescript.js
Critical dependency: the request of a dependency is an expression

Import trace for requested module:
./node_modules/typescript/lib/typescript.js
./node_modules/zod-to-ts/dist/index.js
./node_modules/openai-partial-stream/dist/chunk-66SDEIDM.mjs
./node_modules/openai-partial-stream/dist/index.mjs
./src/app/api/completion/route.ts
./node_modules/next/dist/build/webpack/loaders/next-app-loader.js?name=app%2Fapi%2Fcompletion%2Froute&page=%2Fapi%2Fcompletion%2Froute&appPaths=&pagePath=private-next-app-dir%2Fapi%2Fcompletion%2Froute.ts&appDir=%2FUsers%2Fakashdeepdeb%2FDesktop%2Ftopshelf%2Fsrc%2Fapp&pageExtensions=tsx&pageExtensions=ts&pageExtensions=jsx&pageExtensions=js&rootDir=%2FUsers%2Fakashdeepdeb%2FDesktop%2Ftopshelf&isDev=true&tsconfigPath=tsconfig.json&basePath=&assetPrefix=&nextConfigOutput=&preferredRegion=&middlewareConfig=e30%3D!./src/app/api/completion/route.ts?__next_edge_ssr_entry__
<w> [webpack.cache.Pack

Any ideas if this is related to a zod to ts version issue?

NodeJS + Express

Thank you so much for the work on this! It's an awesome contribution. On running the Color example in an endpoint I get status of "COMPLETED", when the output didn't finish streaming yet.

Screenshot 2023-11-12 at 10 06 59

Example with postcodes not working

Hey! Unfortunately I'm having a hard time getting this to run correctly.

I'm using copy-pasted postcodes example from readme.

Unfortunately, it only detects the last entity:

image

image

Expected output based on ReadMe:

{ index: 0, status: 'COMPLETED', data: { name: 'Los Angeles', postcode: '90001', population: 3971883 }, entity: 'postcodes' }
{ index: 1, status: 'COMPLETED', data: { name: 'San Francisco', postcode: '94102', population: 883305 }, entity: 'postcodes' }
{ index: 2, status: 'COMPLETED', data: { name: 'San Diego', postcode: '92101', population: 1425976 }, entity: 'postcodes'}

If I iterate over the entityStream instead, it looks like this:

image

image

Is this some known issue, or did I miss anything? Thanks for your help!

NextJS 13

Hello, thank you for the package, i am trying to implement the color example into a next JS app, but i don't understand how to,

here is my api/colors :

 import { z } from "zod";
import { Configuration, OpenAIApi } from 'openai-edge';
import { OpenAIStream, StreamingTextResponse } from 'ai';

import { OpenAiHandler, StreamMode, Entity } from "openai-partial-stream";

// Set the runtime to edge for best performance
export const runtime = 'edge';



const config = new Configuration({
    apiKey: process.env.OPENAI_API_KEY,
  });

  const openai = new OpenAIApi(config);


async function callGenerateColors(
    mode = StreamMode.StreamObjectKeyValueTokens,
) {
    // Call OpenAI API, with function calling
    // Function calling: https://openai.com/blog/function-calling-and-other-api-updates
    const stream = await openai.createChatCompletion({
        messages: [
            {
                role: "user",
                content:
                    "Give me a palette of 2 gorgeous color with the hex code, name and a description.",
            },
        ],
        model: "gpt-3.5-turbo", // OR "gpt-4"
        stream: true, // ENABLE STREAMING
        temperature: 1.3,
        functions: [
            {
                name: "give_colors",
                description: "Give a list of color",
                parameters: {
                    type: "object",
                    properties: {
                        colors: {
                            type: "array",
                            items: {
                                type: "object",
                                properties: {
                                    hex: {
                                        type: "string",
                                        description:
                                            "The hexadecimal code of the color",
                                    },
                                    name: {
                                        type: "string",
                                        description: "The color name",
                                    },
                                    description: {
                                        type: "string",
                                        description:
                                            "The description of the color",
                                    },
                                },
                            },
                        },
                    },
                },
            },
        ],
        function_call: { name: "give_colors" },
    });

    // Handle the stream from OpenAI client
    const openAiHandler = new OpenAiHandler(mode);
    // Parse the stream to valid JSON
    const entityStream = openAiHandler.process(stream);

    return entityStream;
}


export  async function POST(
  req: Request
) {
    // Instantiate OpenAI client

        // // Select the mode of the stream parser
 const mode = StreamMode.StreamObject; // ONE-BY-ONE
        // const colorEntityStream = await callGenerateColors(mode);

    return   callGenerateColors(mode);   
}

and my page:


import React, { useState, useEffect } from 'react';

import { useCompletion } from 'ai/react';


interface Color {
  hex: string;
  name: string;
  description: string;
}

const IndexPage = () => {
  const [colors, setColors] = useState<Color[]>([]);
  const [loading, setLoading] = useState<boolean>(true);

  const {
    completion,
    input,
    stop,
    isLoading,
    handleInputChange,
    handleSubmit,
  } = useCompletion({
    api: '/api/colors',
  });

  if (isLoading) return <p>Loading...</p>;

  return (
    <div>
      <h1>Generated Colors</h1>

      {completion}
      <ul>
        {colors.map((color, index) => (
          <li key={index}>
            <strong>{color.name}</strong> ({color.hex}): {color.description}
          </li>
        ))}
      </ul>
    </div>
  );
};

export default IndexPage;

do you have any idea about how i could integrate your package into nextJS 13? thank you!

Generate TypeScript declaration maps

I'd really like to add support for declaration maps so that in my VS Code I can use Go-to-Definition to navigate to the .ts files instead of .mts or .mjs files.

What I've tried to do:

  1. Add "declaration": true and "declarationMap": true to packages/openai-partial-stream/tsconfig.json.
  2. Run npm run build.
  3. Still, no .map files were generated.

Since the npm run build uses tsup, I suspect it's related to egoist/tsup#488 and apparently this remains unsupported. I'd love to hear your thoughts. I'm happy to help.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.