I can't seem to figure out the incompatibility between dependencies when downloading with poetry as recommended by the readme.md ,the problem is that astra-assistants and streaming-assistants both depend on different versions of httpx

Because no versions of astra-assistants match >2.0.16,<3.0.0
and astra-assistants (2.0.16) depends on httpx (>=0.27.0,<0.28.0), astra-assistants (>=2.0.16,<3.0.0) requires httpx (>=0.27.0,<0.28.0).
And because streaming-assistants (0.18.7rc0) depends on httpx (>=0.26.0,<0.27.0), astra-assistants (>=2.0.16,<3.0.0) is incompatible with streaming-assistants (0.18.7rc0).
So, because astra-assistants-api depends on both streaming-assistants (0.18.7-pre) and astra-assistants (^2.0.16), version solving failed.

Since uvicorn now does multi-proc well by itself

WIP

Description

When attempting to create a file using the OpenAI Client with custom options set via with_options, a PermissionDeniedError is thrown indicating a missing authentication token in the header.

Steps to Reproduce

1. Initialize the OpenAI Client and patch it.
2. Attempt to create a file using the following code:

   file_id = self.client.with_options(
       timeout=80 * 1000,  # Set a custom timeout
   ).files.create(file=f, purpose="assistants").id

3. Observe the PermissionDeniedError with a 403 status code indicating a missing 'astra-api-token'.

Expected Behavior

The file creation should proceed without any permission errors, utilizing the provided options including the custom timeout.

Actual Behavior

The operation fails with the following error:

PermissionDeniedError: Error code: 403 - {'detail': 'Must pass an astradb token in the astra-api-token header'}

Additional Information

• Removing the with_options method call and directly using files.create() without any options works correctly, suggesting an issue with how headers or tokens are managed when with_options is used.
• OpenAI version: 1.21.2
• Python version: 3.10.13
• Operating System: macOS 14.1.1

WIP

for both streaming and non streaming

Should work but add tests.

https://github.com/openai/openai-python/blob/main/examples/

deepseek api key mention in https://github.com/datastax/astra-assistants-api/tree/main/client but not supported

here is the .env file
Authentication
Provide api keys for third party LLMs via environment variables. We support LLM completions through litellm and support litellm environmental variables and models.

Rename the .env.bkp to .env and fill in the appropriate values for the LLMs you want to use.

#!/bin/bash

AstraDB -> https://astra.datastax.com/ --> tokens --> administrator user --> generate

export ASTRA_DB_APPLICATION_TOKEN=""

OpenAI Models - https://platform.openai.com/api-keys --> create new secret key

export OPENAI_API_KEY=""

Groq Models - https://console.groq.com/keys

export GROQ_API_KEY=""

Anthropic claude models - https://console.anthropic.com/settings/keys

export ANTHROPIC_API_KEY=""

Gemini models -> https://makersuite.google.com/app/apikey

export GEMINI_API_KEY=""

Perplexity models -> https://www.perplexity.ai/settings/api --> generate

export PERPLEXITYAI_API_KEY=""

Cohere models -> https://dashboard.cohere.com/api-keys

export COHERE_API_KEY=""

Bedrock models -> https://docs.aws.amazon.com/bedrock/latest/userguide/setting-up.html

export AWS_REGION_NAME=""
export AWS_ACCESS_KEY_ID=""
export AWS_SECRET_ACCESS_KEY=""

vertexai models https://console.cloud.google.com/vertex-ai

export GOOGLE_JSON_PATH=""
export GOOGLE_PROJECT_ID=""

AI21 models

export AI21_API_KEY=""

Aleph Alpha models

export ALEPHALPHA_API_KEY=""

Anyscale models

export ANYSCALE_API_KEY=""

Azure models

export AZURE_API_KEY=""
export AZURE_API_BASE=""
export AZURE_API_VERSION=""
export AZURE_AD_TOKEN=""
export AZURE_API_TYPE=""

Baseten models

export BASETEN_API_KEY=""

Cloudflare Workers models

export CLOUDFLARE_API_KEY=""
export CLOUDFLARE_ACCOUNT_ID=""

DeepInfra models

export DEEPINFRA_API_KEY=""

DeepSeek models

export DEEPSEEK_API_KEY=""

Fireworks AI models

export FIREWORKS_AI_API_KEY=""

Hugging Face models

export HUGGINGFACE_API_KEY=""
export HUGGINGFACE_API_BASE=""

Mistral models

export MISTRAL_API_KEY=""

NLP Cloud models

export NLP_CLOUD_API_KEY=""

OpenAI models

export OPENAI_API_KEY=""
export OPENAI_ORGANIZATION=""
export OPENAI_API_BASE=""

OpenRouter models

export OPENROUTER_API_KEY=""
export OR_SITE_URL=""
export OR_APP_NAME=""

PaLM models

export PALM_API_KEY=""

Replicate models

export REPLICATE_API_KEY=""

TogetherAI models

export TOGETHERAI_API_KEY=""

Vertex AI models

export VERTEXAI_PROJECT=""
export VERTEXAI_LOCATION=""
export GOOGLE_APPLICATION_CREDENTIALS=""

Voyage models

export VOYAGE_API_KEY=""

WatsonX models

export WATSONX_URL=""
export WATSONX_APIKEY=""
export WATSONX_TOKEN=""
export WATSONX_PROJECT_ID=""
export WATSONX_DEPLOYMENT_SPACE_ID=""

XInference models

export XINFERENCE_API_BASE=""
export XINFERENCE_API_KEY=

Description

When using the Astra Assistants API, there is an issue where the execution stops if the assistant outputs a message before using a tool. In the OpenAI Assistants API, the agent can output a message and then use a tool subsequently. However, with Astra Assistants, if the agent outputs any message, the execution stops and prevents further tool usage. The execution works correctly if the agent uses tools before outputting any message.

Steps to Reproduce

1. Create an assistant using the Astra Assistants API.
2. Instruct the assistant to output a message and then use a tool: "You must say 'I am using test tool' and then use test tool in this exact order in a single message. Do not use test tool before you said you are using it."
3. Observe that the execution stops after the message is outputted, and the tool is not used.

Expected Behavior

The assistant should be able to output a message and then use a tool seamlessly, as it works in the OpenAI Assistants API.

Actual Behavior

The execution stops after the assistant outputs a message, preventing any subsequent tool usage.

Example

from openai import OpenAI
from astra_assistants import patch
from agency_swarm import Agent, Agency, set_openai_client
from dotenv import load_dotenv
from agency_swarm.tools import BaseTool

load_dotenv('../.env')

client = patch(OpenAI())

set_openai_client(client)

class TestTool(BaseTool):
    """
    A simple tool that prints 'test'.
    """

    def run(self):
        """
        This method prints the word 'test'.
        """
        print("test")
        return "Test has been printed."

ceo = Agent(name="Test Agent",
            description="Test Agent for testing purposes",
            instructions="You must say 'I am using test tool' and then use test tool in the same message.",
            # model="perplexity/llama-3-8b-instruct",
            # model="anthropic/claude-3-haiku-20240307",
            # model="groq/mixtral-8x7b-32768",
            model="gpt-4o",
            # files_folder="./examples/python/agency-swarm/files",
            temperature=0,
            tools=[TestTool])

agency = Agency([ceo])
 
completion = agency.get_completion("Please say 'I am using test tool' and ONLY THEN use test tool.")
print(completion)

Environment

• Agency Swarm version: 0.2.5
• Astra Assistants version: 2.0.5

Additional Information

• The issue does not occur if tools are used before outputting any message.

github.com/openai/openai-openapi/commit/3bbc26bd740ecd9c4d76af02adb981e914ebac05

openai.InternalServerError: Error code: 500 - {'message': "Error: litellm.BadRequestError: AnthropicException - litellm.BadRequestError: AnthropicException - Invalid first message. Should always start with 'role'='user' for Anthropic. System prompt is sent separately for Anthropic. set 'litellm.modify_params = True' or 'litellm_settings:modify_params = True' on proxy, to insert a placeholder user message - '.' as the first message, "}

Will require sdk support

"c", "cpp", "css", "html", "java", "js", "json", "md", "php", "py", "rb", "ts", "xml"

Description

When adding tools to an assistant, the API throws an error if the assistant does not utilize them. In OpenAI, assistants can choose whether to use tools or not. This issue originates from the threads.py create_run method.

Steps to Reproduce

1. Create an assistant with any custom tool. (function)
2. Ask the assistant to say "hi."
3. Observe the error: "Could not extract function arguments from LLM response, may have not been properly formatted. Consider retrying or use a different model."

Expected Behavior

In OpenAI, tools can be attached to an assistant, allowing it to decide whether or not to use them. This autonomy in tool utilization is crucial for any agentic system.

Actual Behavior

The operation fails with the error:
ValueError("Could not extract function arguments from LLM response, may have not been properly formatted. Consider retrying or use a different model.")

Additional Information

• OpenAI version: 1.20.0
• Python version: 3.10.13
• Streaming assistants version: 0.16.0
• Operating System: macOS 14.1.1

in streaming and non streaming

Adding an example of using agency-swarm with astra-assistants and claude-haiku

poetry run python examples/python/agency-swarm/basic.py 

File retrieval is working well.

ollama_chat is recommended by litellm over ollama because it uses a different api under the hood. Ideally we can get both working.

ollama_chat completions currently fail with:

openai.InternalServerError: Error code: 500 - {'message': 'Error: litellm.APIConnectionError: CIMultiDict takes either dict or list of (key, value) pairs'}

which seems to be due to a problem adding headers to the request.

If I create thread with await this.openai.beta.threads.create() then await this.openai.beta.threads.retrieve(threadId) fails with

ERROR - Unexpected error: 1 validation errors:
{'type': 'missing', 'loc': ('response', 'tool_resources'), 'msg': 'Field required', 'input': ThreadObject(id='18f14d98-25c2-11ef-825c-0242ac110003', object='thread', created_at=1717870104000, metadata={})}

Why do you need any validations here that breaks API?

We've been testing running assistants with ollama. This is of interest for some folks using agency-swarm with assistants but may be interesting to others as well (assistants-api with local models).

Currently this endpoint seems to stream even though the stream argument is not set to true:

response = client.chat.completions.create(
    model="ollama/phi3",
    messages=[{"role": "user", "content": "respond in 20 words who are you"}]
)

sample response:

ChatCompletion(id='chatcmpl-b6dc85b5-ecb4-4c20-bc74-7006aee33f15', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='assistant are', role='assistant', function_call=None, tool_calls=None))], created=1719329075, model='ollama/dolphin-llama3', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=3, prompt_tokens=10, total_tokens=13))

Related discord chat here:
https://discord.com/channels/1245465949679915008/1245478014737977344/1255181994317451346

OpenAI Assistants uses prefix-alphanumeric ID's and some libraries (like agency-swarm) depend on this https://github.com/VRSEN/agency-swarm/blob/94f79ca4d6bec95a866ffbb4b04e15210ec05e6e/agency_swarm/agents/agent.py#L345

https://github.com/sweepai/sweep/blob/main/notebooks/chunking.ipynb

https://platform.openai.com/docs/assistants/whats-new

For users that want to do their own chunking and embedding but still want to leverage assistants

Does function calling work with other models such as gemini?

Is it possible to change the model of the assistant at the run?

run = await self.beta.threads.runs.create(
            thread_id=thread.id,
            assistant_id=assistant_id
            model="gemini/gemini-pro"
        )

I already have assistants in the playground, do I need to create them since you are possibly storing it in your database?

Do you use my data?

What is stored on AstraDB? (edited)