cyclenerd / google-cloud-gcp-openai-api

🌴 Drop-in replacement REST API for Vertex AI (PaLM 2, Codey, Gemini) that is compatible with the OpenAI API specifications

Hey everyone, Nils here! 👋

I'm primarily an Ops guy with a toolbox of Bash scripts and Perl utilities, hoping they might be useful for your projects too. Pull requests are always welcome!

You can also find me on LinkedIn if you want to connect.

---

google-cloud-gcp-openai-api's Issues

(Allow unauthenticated invocations) cannot be set.

After May 3rd, deploying this container as a public API (allowing unauthenticated calls) in Google Cloud Run will be disabled by default. Changing this setting requires an email address with domain management permissions, which I do not have. Is it possible to achieve public, unauthenticated access using methods like intranet penetration?

[Translation text]

Consider adding support for safety_settings?

Should we consider adding support for safety_settings so that we can disable moderation to some extent?

Refer: langchain-ai/langchain#15344

request: Please Support Claude 3 Opus model

as title.
Can Support Claude 3 Opus model?

Reference:
https://console.cloud.google.com/vertex-ai/publishers/anthropic/model-garden/claude-3-opus

Anthropic on Vertex AI:
https://python.langchain.com/v0.2/docs/integrations/llms/google_vertex_ai_palm/#anthropic-on-vertex-ai

Thank you

Feature request: Please Update Documentation with the following for Windows Users

While deploying your program locally. I was simply converting the exports

by changing export to set.
Completely forgetting (despite a lot of years), to remove the double-quotes.
once I did that it worked like a charm.
linux:
export DEBUG="True"
export OPENAI_API_KEY="sk-XYZ"
export MODEL_NAME="gemini-pro"

Win:
set DEBUG=True
set OPENAI_API_KEY=sk-XYZ
set MODEL_NAME=gemini-pro

I do want to thank you, this will be solving an issue with using autogen, that I was tired of messing with, and I am so grateful you wrote this api! Thank you!

Error "404 - /v1/models was not found on this server." after Colab deployment

I deployed this via the Colab feature:

• created a project in Google Cloud Console
• set domain restricted sharing to "allow" (in this order)
• clicked "open in colab"
• deployed all steps using the play button, my openAI key was in the form sk-123-only13letters
• went to the frontend URL, in my case https://chatbot-ui-XXX-ew.a.run.app/
• only a wheel is spinning there, sometime I see the message "Error fetching models.
  Make sure your OpenAI API key is set in the bottom left of the sidebar.
  If you completed this step, OpenAI may be experiencing issues. Error 500"
• in the Cloud Run logs, I find an Error message "POST500666 B135 msChrome 121 https://chatbot-ui-35XXXa-ew.a.run.app/api/models "
• when I expand this, I find

{
httpRequest: {
latency: "0.135149560s"
protocol: "HTTP/1.1"
referer: "https://chatbot-ui-35XXX-ew.a.run.app/"
remoteIp: "2a02:3100:5867:8701:a596:638d:4670:c449"
requestMethod: "POST"
requestSize: "862"
requestUrl: "https://chatbot-ui-35XXX.a.run.app/api/models"
responseSize: "666"
serverIp: "2001:4860:4802:36::35"
status: 500
userAgent: "Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"
}
insertId: "65a158cd0006fcb5d44be846"
labels: {
instanceId: "0087599d427185aebb313689bb408e0c88da7eaf33934605a2ff3204ad1649412c336d290dadc8458e2f8ef07655d8a216fa860c3d3d88a53cd44f259ff3202f6a9c"
}
logName: "projects/thorstenstaerk-nils2/logs/run.googleapis.com%2Frequests"
receiveTimestamp: "2024-01-12T15:20:45.618651319Z"
resource: {
labels: {
service_name: "chatbot-ui"
project_id: "thorstenstaerk-nils2"
revision_name: "chatbot-ui-00001-gfs"
configuration_name: "chatbot-ui"
location: "europe-west1"
}
type: "cloud_run_revision"
}
severity: "ERROR"
spanId: "13873342680540356080"
timestamp: "2024-01-12T15:20:45.320472Z"
trace: "projects/thorstenstaerk-nils2/traces/b4419a8dd29302b4eb53be48e42bac76"
}

When I look at the next messages in the log, they are not marked as Error messages, but I find one that I regard as relevant:

The requested URL /v1/models was not found on this server. That’s all we know.