stefanbratanov / jvm-openai

A minimalistic OpenAI API client for the JVM, written in Java 🤖

jvm-openai

A minimalistic unofficial OpenAI API client for the JVM, written in Java. The only dependency used is Jackson for JSON parsing.

Add dependency

Java 17+ is a prerequisite

Gradle

implementation("io.github.stefanbratanov:jvm-openai:${version}")

Maven

<dependency>
    <groupId>io.github.stefanbratanov</groupId>
    <artifactId>jvm-openai</artifactId>
    <version>${version}</version>
</dependency>

Minimal example

OpenAI openAI = OpenAI.newBuilder(System.getenv("OPENAI_API_KEY")).build();

ChatClient chatClient = openAI.chatClient();
CreateChatCompletionRequest createChatCompletionRequest = CreateChatCompletionRequest.newBuilder()
    .model(OpenAIModel.GPT_3_5_TURBO)
    .message(ChatMessage.userMessage("Who won the world series in 2020?"))
    .build();
ChatCompletion chatCompletion = chatClient.createChatCompletion(createChatCompletionRequest);

Supported APIs

NOTE: Legacy APIs are not supported

API	Status
Audio	✔️
Chat	✔️
Embeddings	✔️
Fine-tuning	✔️
Batch	✔️
Files	✔️
Images	✔️
Models	✔️
Moderations	✔️

Beta APIs

API	Status
Assistants	✔️
Threads	✔️
Messages	✔️
Runs	✔️
Run Steps	✔️
Vector Stores	✔️
Vector Store Files	✔️
Vector Store File Batches	✔️

More examples

• Configure an organization and project

OpenAI openAI = OpenAI.newBuilder(System.getenv("OPENAI_API_KEY"))
    .organization("org-zweLLamVlP6c5n66zY334ivs")
    .project(System.getenv("PROJECT_ID"))        
    .build();

• Configure a custom base url

OpenAI openAI = OpenAI.newBuilder(System.getenv("OPENAI_API_KEY"))
    .baseUrl("https://api.foobar.com/v1/")     
    .build();

• Configure a custom Java's HttpClient

HttpClient httpClient = HttpClient.newBuilder()
    .connectTimeout(Duration.ofSeconds(20))
    .executor(Executors.newFixedThreadPool(3))
    .proxy(ProxySelector.of(new InetSocketAddress("proxy.example.com", 80)))
    .build();
OpenAI openAI = OpenAI.newBuilder(System.getenv("OPENAI_API_KEY"))
    .httpClient(httpClient)
    .build();

• Configure a timeout for all requests

OpenAI openAI = OpenAI.newBuilder(System.getenv("OPENAI_API_KEY"))
    .requestTimeout(Duration.ofSeconds(10))
    .build();

• Create chat completion async

ChatClient chatClient = openAI.chatClient();
CreateChatCompletionRequest request = CreateChatCompletionRequest.newBuilder()
    .model(OpenAIModel.GPT_3_5_TURBO)
    .message(ChatMessage.userMessage("Who won the world series in 2020?"))
    .build();
CompletableFuture<ChatCompletion> chatCompletion = chatClient.createChatCompletionAsync(request);
chatCompletion.thenAccept(System.out::println);

• Streaming

ChatClient chatClient = openAI.chatClient();
CreateChatCompletionRequest request = CreateChatCompletionRequest.newBuilder()
    .message(ChatMessage.userMessage("Who won the world series in 2020?"))
    .stream(true)
    .build();
// with java.util.stream.Stream
chatClient.streamChatCompletion(request).forEach(System.out::println);
// with subscriber
chatClient.streamChatCompletion(request, new ChatCompletionStreamSubscriber() {
    @Override
    public void onChunk(ChatCompletionChunk chunk) {
        System.out.println(chunk);
    }

    @Override
    public void onException(Throwable ex) {
        // ...
    }
    
    @Override
    public void onComplete() {
        // ...
    }
});

• Create image

ImagesClient imagesClient = openAI.imagesClient();
CreateImageRequest createImageRequest = CreateImageRequest.newBuilder()
    .model(OpenAIModel.DALL_E_3)
    .prompt("A cute baby sea otter")
    .build();
Images images = imagesClient.createImage(createImageRequest);

• Create speech

AudioClient audioClient = openAI.audioClient();
SpeechRequest request = SpeechRequest.newBuilder()
    .model(OpenAIModel.TTS_1)
    .input("The quick brown fox jumped over the lazy dog.")
    .voice(Voice.ALLOY)
    .build();
Path output = Paths.get("/tmp/speech.mp3");
audioClient.createSpeech(request, output);

• Create translation

AudioClient audioClient = openAI.audioClient();
TranslationRequest request = TranslationRequest.newBuilder()
    .model(OpenAIModel.WHISPER_1)
    .file(Paths.get("/tmp/german.m4a"))
    .build();
String translatedText = audioClient.createTranslation(request);

• List models

ModelsClient modelsClient = openAI.modelsClient();
List<Model> models = modelsClient.listModels();

• Classifiy if text violates OpenAI's Content Policy

ModerationsClient moderationsClient = openAI.moderationsClient();
ModerationRequest request = ModerationRequest.newBuilder()
    .input("I want to bake a cake.")
    .build();
Moderation moderation = moderationsClient.createModeration(request);
boolean violence = moderation.results().get(0).categories().violence();

• Create and execute a batch

// Upload JSONL file containing requests for the batch
FilesClient filesClient = openAI.filesClient();
UploadFileRequest uploadInputFileRequest = UploadFileRequest.newBuilder()
    .file(Paths.get("/tmp/batch-requests.jsonl"))
    .purpose(Purpose.BATCH)
    .build();
File inputFile = filesClient.uploadFile(uploadInputFileRequest);

BatchClient batchClient = openAI.batchClient();
CreateBatchRequest request = CreateBatchRequest.newBuilder()
    .inputFileId(inputFile.id())
    .endpoint("/v1/chat/completions")
    .completionWindow("24h")
    .build();
Batch batch = batchClient.createBatch(request);
// check status of the batch
Batch retrievedBatch = batchClient.retrieveBatch(batch.id());
System.out.println(retrievedBatch.status());      

• Build AI Assistant

AssistantsClient assistantsClient = openAI.assistantsClient();
ThreadsClient threadsClient = openAI.threadsClient();
MessagesClient messagesClient = openAI.messagesClient();
RunsClient runsClient = openAI.runsClient();

// Step 1: Create an Assistant
CreateAssistantRequest createAssistantRequest = CreateAssistantRequest.newBuilder()
    .name("Math Tutor")
    .model(OpenAIModel.GPT_3_5_TURBO_1106)
    .instructions("You are a personal math tutor. Write and run code to answer math questions.")
    .tool(Tool.codeInterpreterTool())
    .build();
Assistant assistant = assistantsClient.createAssistant(createAssistantRequest);

// Step 2: Create a Thread
CreateThreadRequest createThreadRequest = CreateThreadRequest.newBuilder().build();
Thread thread = threadsClient.createThread(createThreadRequest);

// Step 3: Add a Message to a Thread
CreateMessageRequest createMessageRequest = CreateMessageRequest.newBuilder()
    .role(Role.USER)
    .content("I need to solve the equation `3x + 11 = 14`. Can you help me?")
    .build();
ThreadMessage message = messagesClient.createMessage(thread.id(), createMessageRequest);

// Step 4: Run the Assistant
CreateRunRequest createRunRequest = CreateRunRequest.newBuilder()
    .assistantId(assistant.id())
    .instructions("Please address the user as Jane Doe. The user has a premium account.")
    .build();
ThreadRun run = runsClient.createRun(thread.id(), createRunRequest);

// Step 5: Check the Run status
ThreadRun retrievedRun = runsClient.retrieveRun(thread.id(), run.id());
String status = retrievedRun.status();

// Step 6: Display the Assistant's Response
MessagesClient.PaginatedThreadMessages paginatedMessages = messagesClient.listMessages(thread.id(), PaginationQueryParameters.none(), Optional.empty());
List<ThreadMessage> messages = paginatedMessages.data();

• Build AI Assistant with File Search Enabled

AssistantsClient assistantsClient = openAI.assistantsClient();
ThreadsClient threadsClient = openAI.threadsClient();
MessagesClient messagesClient = openAI.messagesClient();
RunsClient runsClient = openAI.runsClient();
VectorStoresClient vectorStoresClient = openAI.vectorStoresClient();
FilesClient filesClient = openAI.filesClient();
VectorStoreFileBatchesClient vectorStoreFileBatchesClient = openAI.vectorStoreFileBatchesClient();

// Step 1: Create a new Assistant with File Search Enabled
CreateAssistantRequest createAssistantRequest = CreateAssistantRequest.newBuilder()
    .name("Financial Analyst Assistant")
    .model(OpenAIModel.GPT_4_TURBO)
    .instructions("You are an expert financial analyst. Use you knowledge base to answer questions about audited financial statements.")
    .tool(Tool.fileSearchTool())
    .build();
Assistant assistant = assistantsClient.createAssistant(createAssistantRequest);

// Step 2: Upload files and add them to a Vector Store
CreateVectorStoreRequest createVectorStoreRequest = CreateVectorStoreRequest.newBuilder()
    .name("Financial Statements")
    .build();
VectorStore vectorStore = vectorStoresClient.createVectorStore(createVectorStoreRequest);
UploadFileRequest uploadFileRequest1 = UploadFileRequest.newBuilder()
    .file(Paths.get("edgar/goog-10k.pdf"))
    .purpose(Purpose.ASSISTANTS)
    .build();
File file1 = filesClient.uploadFile(uploadFileRequest1);
UploadFileRequest uploadFileRequest2 = UploadFileRequest.newBuilder()
    .file(Paths.get("edgar/brka-10k.txt"))
    .purpose(Purpose.ASSISTANTS)
    .build();
File file2 = filesClient.uploadFile(uploadFileRequest2);
CreateVectorStoreFileBatchRequest createVectorStoreFileBatchRequest = CreateVectorStoreFileBatchRequest.newBuilder()
    .fileIds(List.of(file1.id(), file2.id()))
    .build();
VectorStoreFileBatch batch = vectorStoreFileBatchesClient.createVectorStoreFileBatch(vectorStore.id(), createVectorStoreFileBatchRequest);
// need to query the status of the file batch for completion
vectorStoreFileBatchesClient.retrieveVectorStoreFileBatch(vectorStore.id(), batch.id());

// Step 3: Update the assistant to use the new Vector Store
ModifyAssistantRequest modifyAssistantRequest = ModifyAssistantRequest.newBuilder()
    .toolResources(ToolResources.fileSearchToolResources(vectorStore.id()))
    .build();
assistantsClient.modifyAssistant(assistant.id(), modifyAssistantRequest);

// Step 4: Create a thread
CreateThreadRequest.Message message = CreateThreadRequest.Message.newBuilder()
    .role(Role.USER)
    .content("How many shares of AAPL were outstanding at the end of of October 2023?")
    .build();
CreateThreadRequest createThreadRequest = CreateThreadRequest.newBuilder()
    .message(message)
    .build();
Thread thread = threadsClient.createThread(createThreadRequest);

// Step 5: Create a run and check the output
CreateRunRequest createRunRequest = CreateRunRequest.newBuilder()
    .assistantId(assistant.id())
    .instructions("Please address the user as Jane Doe. The user has a premium account.")
    .build();
ThreadRun run = runsClient.createRun(thread.id(), createRunRequest);
// check the run status
ThreadRun retrievedRun = runsClient.retrieveRun(thread.id(), run.id());
String status = retrievedRun.status();
// display the Assistant's Response
MessagesClient.PaginatedThreadMessages paginatedMessages = messagesClient.listMessages(thread.id(), PaginationQueryParameters.none(), Optional.empty());
List<ThreadMessage> messages = paginatedMessages.data();

• Create a run and stream the result of executing the run (Assistants Streaming)

RunsClient runsClient = openAI.runsClient();
CreateRunRequest createRunRequest = CreateRunRequest.newBuilder()
    .assistantId(assistant.id())
    .instructions("Please address the user as Jane Doe. The user has a premium account.")
    .stream(true)   
    .build();
// with java.util.stream.Stream
runsClient.createRunAndStream(thread.id(), createRunRequest).forEach(assistantStreamEvent -> {
    System.out.println(assistantStreamEvent.event());
    System.out.println(assistantStreamEvent.data());
});
// with subscriber
runsClient.createRunAndStream(thread.id(), createRunRequest, new AssistantStreamEventSubscriber() {
    @Override
    public void onThread(String event, Thread thread) {
        // ...
    }

    @Override
    public void onThreadRun(String event, ThreadRun threadRun) {
        // ...
    }

    @Override
    public void onThreadRunStep(String event, ThreadRunStep threadRunStep) {
        // ...
    }

    @Override
    public void onThreadRunStepDelta(String event, ThreadRunStepDelta threadRunStepDelta) {
        // ...
    }

    @Override
    public void onThreadMessage(String event, ThreadMessage threadMessage) {
        // ...
    }

    @Override
    public void onThreadMessageDelta(String event, ThreadMessageDelta threadMessageDelta) {
        // ...
    }

    @Override
    public void onUnknownEvent(String event, String data) {
        // ...
    }

    @Override
    public void onException(Throwable ex) {
        // ...
    }

    @Override
    public void onComplete() {
        // ...
    }    
});
// "createThreadAndRunAndStream" and "submitToolOutputsAndStream" methods are also available

jvm-openai's Issues

Return Json

When submitting a single quote message to ChatCompletionRequest, the system returns a json string which I want.
When I submit a messges() array (empty system, assistant) with a quote, I get a 400 error indicating Chat Exception [400 - message: 'messages' must contain the word 'json' in some form, to use 'response_format' of type 'json_object'., type: When I supply a 'return json' in any and all of the inputs (system,assistant,prompt) I get this message.

Can not parse ChatCompletionChunk when using streamChatCompletion

I got this error.

Can not set final java.lang.String field io.github.stefanbratanov.jvm.openai.ChatCompletionChunk$Choice.finishReason to null value (through reference chain: io.github.stefanbratanov.jvm.openai.ChatCompletionChunk["choices"]->java.util.ArrayList[0])

Debug into the code

  <T> T deserializeResponse(byte[] response, Class<T> responseClass) {
    try {
      return objectMapper.readValue(response, responseClass);
    } catch (IOException ex) {
      throw new UncheckedIOException(ex);
    }
  }

I see the response is valid.

{
  "id": "chatcmpl-xyz",
  "object": "chat.completion.chunk",
  "created": 1711604346,
  "model": "gpt-4-0613",
  "system_fingerprint": null,
  "choices": [
    {
      "index": 0,
      "delta": {
        "role": "assistant",
        "content": ""
      },
      "logprobs": null,
      "finish_reason": null
    }
  ]
}

Ability to call self hosted OpenaiCompatible endpoints

Just a thought, many companies are hosting their own versions of "Chat" where the url and model name changes. Would be helpful to have that feature to change them.Sometimes same url but different params.

ToolResources visibility

The static helper methods in ToolResources class is not visible from other packages and this is causing compilation issues when trying to run provided sample code as is. if there is not a special reason not to make them public, changing them would be helpful.

streamChatCompletion: java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0

The call to the OpenAI API was mostly successful, but the library returned a strange error. Below is my code.

new ChatCompletionStreamSubscriber() {
                      private final StringBuilder joinedContentBuilder = new StringBuilder();
                      private Usage usage;

                      @Override
                      public void onChunk(ChatCompletionChunk chunk) {
                          List<ChatCompletionChunk.Choice> choices = chunk.choices();

                          String content = choices.get(0).delta().content();
                          if (content != null) {
                                joinedContentBuilder.append(content);
                          }

                          if (chunk.usage() != null) {
                              usage = chunk.usage();
                          }
                      }

                      @Override
                      public void onException(Throwable ex) {
                          onError.accept(ex);
                      }

                      @Override
                      public void onComplete() {
                          String joinedContent = joinedContentBuilder.toString();

                          var generationResponse = GenerationResponse.builder()
                              .response(joinedContent)
                              .promptTokens(usage.promptTokens())
                              .completionTokens(usage.completionTokens())
                              .totalTokens(usage.totalTokens())
                              .build();

                          onSuccess.accept(generationResponse);
                      }
                  });

Class namings

io.github.stefanbratanov.jvm.openai.Thread class, sharing the same name java.lang.Thread, which is imported by default, causes some issues when using the library. Each time prefixing it with the full package name become a bit tedious. Maybe changing the class names that could potentially cause conflicts could help?

This is also applicable for some other classes like File, Constants etc. But since these classes are not included automatically, it is not that big problem.

thank you

Function parameters are serialized with JSON escaping, making it difficult to emit the write JSON

It's expected that properties values would not be serialized with escaping.

Example:

`
public class SerializeTest {

public static void main(String[] args)
        throws ClassNotFoundException, InvocationTargetException, IllegalAccessException, JsonProcessingException,
        NoSuchMethodException {
    final List<Tool> tools = new LinkedList<>();
    tools.add(Tool.functionTool(
            FunctionTool.Function.newBuilder().name("getFriends").description("Returns the friends of the person").parameters(
                            Map.of("type", "object",
                                    "properties", "{\"person_name\":{\"type\":\"string\", \"description\":\"the persons name, in lower case\"}}",
                                    "required", "[\"person_name\"]"))
                    .build()));
    final Method method =
            Class.forName("io.github.stefanbratanov.jvm.openai.ObjectMapperSingleton").getDeclaredMethod("getInstance");
    method.setAccessible(true);
    final ObjectMapper objectMapper = (ObjectMapper) method.invoke(null);
    System.out.println(objectMapper.writeValueAsString(tools));
    //[{"function":{"name":"getFriends","description":"Returns the friends of the person","parameters":{"properties":"{\"person_name\":{\"type\":\"string\", \"description\":\"the persons name, in lower case\"}}","type":"object","required":"[\"person_name\"]"}},"type":"function"}]
    //pretty print:
    /*
[
    {
        "function":
        {
            "name": "getFriends",
            "description": "Returns the friends of the person",
            "parameters":
            {
                "properties": "{\"person_name\":{\"type\":\"string\", \"description\":\"the persons name, in lower case\"}}",
                "type": "object",
                "required": "[\"person_name\"]"
            }
        },
        "type": "function"
    }
]
     */
}
}

`

Same with required[] really

ChatMessage.systemMessage or Any Other ChatMessage.*Message

@StefanBratanov great work on this project!

Are these ChatMessage.userMessage, etc.., constructor helpers you use in your examples meant to be package-private only?

I cannot call them outside the package: io.github.stefanbratanov.jvm.openai.

JSON Serialization Issue

I have below code:

AssistantsClient ac = openAI.assistantsClient( ) ;
var aclist = ac.listAssistants( PaginationQueryParameters.none( ) ) ;
String jsonString = ObjectMapperSingleton.getInstance( ).writeValueAsString( aclist ) ;

This jsonString is not valid JSON. The "tools" array is as follows:
"tools":[{"type":"file_search","type":"file_search"}]

Notice the duplicate type fields. I managed to get rid of the duplicate by modifiying the current annotation:
@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type")
to
@JsonTypeInfo( use = JsonTypeInfo.Id.NAME, property = "type", include = JsonTypeInfo.As.EXISTING_PROPERTY )

merging this change (and maybe updating other classes for similar issues) could be helpful.

using in Android project

I am trying to use jvm-openai on an Android project but I get this error while compiling:

import io.github.stefanbratanov.jvm.openai.OpenAI;
^
bad class file: C:\Users\ilker.gradle\caches\modules-2\files-2.1\io.github.stefanbratanov\jvm-openai\0.9.1\d96fe26bdb0f8aa8e23a27e0e88ae776c30cd95\jvm-openai-0.9.1.jar(/io/github/stefanbratanov/jvm/openai/OpenAI.class)
class file has wrong version 61.0, should be 55.0
Please remove or make sure it appears in the correct subdirectory of the classpath.

I am not a gradle expert but a quick search shows that there is a Java JDK sompatibility issue. I could not figure out the solution
Any ideas ? Or root cause of the issue ?

Expose the HttpResponse for requests

Hi,

I've tried a few openAi sdks and this one shows the most promise.

My current use case is using it inside a lambda function, and this is ideal. And already live

However when the api errors for whatever reason it returns the error as a string.

It would be really helpful to have the response (or a mapped object) to check the status code is OK before carrying on with the function

HttpRequest Timeout

In the ChatClient (or any client) it would be nice to have a timeout for the HttpRequest:

  private HttpRequest createPostRequest(CreateChatCompletionRequest request, Long requestTimeout) {
    return newHttpRequestBuilder(
            Constants.CONTENT_TYPE_HEADER,
            Constants.JSON_MEDIA_TYPE,
            Constants.ACCEPT_HEADER,
            Constants.JSON_MEDIA_TYPE)
            .timeout(Duration.ofMillis(requestTimeout))
        .uri(endpoint)
        .POST(createBodyPublisher(request))
        .build();
  }

Support for non-OpenAI Providers

A number of vendors support the OpenAI api. It appears this is doable with your implementation as there is support for a URL 'connectTo' parameter. Will this SDK support use with alternate vendors that implement the OpenAi api faithfully?

adding some constants

not a big deal, but adding some constants/enums to be used in the api could be helpful. file upload purpose, message role are the ones i came accross.

reference:

UploadFileRequest uploadInputFileRequest = UploadFileRequest.newBuilder()
.file(Paths.get("/tmp/batch-requests.jsonl"))
.purpose("batch") // here
.build();

Message message = Message.newBuilder()
.role("user") // here
.content("How many shares of AAPL were outstanding at the end of of October 2023?")
.build();