patrickmaub / function-call-extender

README.md

Overview

This project showcases a novel approach to generate extensive text outputs from OpenAI models, such as GPT-3.5-turbo-16k, GPT-4, and GPT-4-32k, by leveraging function calls to bypass the models' tendencies to cap their token count. The application demonstrates how to reliably produce long-form content, significantly exceeding the typical output length.

Table of Contents

• Project Structure
• Installation
• Usage
  • Running the Application
  • Interacting with the Application
• How It Works
  • Main Application
  • Function Call Builder
  • AI Interaction
• License

Project Structure

1. main.py: User interface for interacting with OpenAI models.
2. function_call_builder.py: Constructs function calls and interacts with OpenAI API.
3. ai.py: Contains functions for token counting and chat completion.

Installation

Ensure Python 3.6+ is installed, then run:

pip install openai tiktoken

Usage

Running the Application

python main.py

Interacting with the Application

• Premade Example: Run examples from a JSON file, examples.json

• Input Your Details: Manually input details to generate a function call and receive a response.

How It Works

Main Application

main.py guides the user through the process, interacting with FunctionCallBuilder to generate function calls.

Function Call Builder

function_call_builder.py dynamically constructs function call definitions and handles the invocation of these calls, managing extensive text outputs.

AI Interaction

ai.py contains functions for token counting and chat completion, crucial for interacting with OpenAI API and managing token limits.

License

This project is open source and freely available for use.

function-call-extender's Issues

thoughts on the 4096 limits coming in

Hey, interesting repo. Unfortunately looks like openai's 3.5-turbo-1106 model limits output to 4096 tokens. As does the 4-turbo model.
Claude has also changed every model's output to 4096 tokens.

OpenaAI's 0613 models will be deprecated in July this year. Azure as well. Soon we will have no major provider (of not OSS LLMs) providing >4096 generation tokens capabilities. IMO this is going to hit a lot of different use cases hard.

Have you had any thoughts on this?