hpatel95
# Legacy OpenAI_Chat: temperature=0.7 causes 400 with models that only support default ## Description When using **Vanna legacy** `OpenAI_Chat` (e.g. with Azure OpenAI), some models do not support the `temperature` parameter and return **400 Bad Request** if any value other than the API default is sent. **Example error:** ``` openai.BadRequestError: Error code: 400 - {'error': {'message': "Unsupported value: 'temperature' does not support 0.7 with this model. Only the default (1) value is supported.", 'type': 'invalid_request_error', 'param': 'temperature', 'code': 'unsupported_value'}} ``` **Observed with:** - Azure OpenAI deployment **gpt-5-nano** - Likely same class of models as in [#855](https://github.com/vanna-ai/vanna/issues/855) (e.g. o3-mini) where temperature is not supported **Root cause:** In `vanna/legacy/openai/openai_chat.py`, `self.temperature` defaults to `0.7` and is always passed to `client.chat.completions.create(..., temperature=self.temperature)`. There is no way to omit the parameter for models that only accept the API default. ## Workaround we used We override via `config` when building our Vanna instance: ```python OpenAI_Chat.__init__( self, client=AzureOpenAI(...), config={ "model": AZURE_DEPLOYMENT, "temperature": 1, # required for gpt-5-nano; only 1 is supported **config, }, ) ``` This works but is a workaround: we had to hardcode `1` for this model. Other models might require the parameter to be **omitted** entirely rather than set to `1`. ## Suggested fix (align with #855) Make the temperature parameter **optional** in the legacy OpenAI chat client: - If `config["temperature"]` is `None` (or not set and a new default of `None` is used), **do not** include `temperature` in the request to the OpenAI API. - If `config["temperature"]` is a number, pass it as today. That way: - Models that only support the default get no `temperature` argument. - Models that support tuning can still get `temperature` via config. Same idea as [Make Temperature parameter optional #855](https://github.com/vanna-ai/vanna/issues/855), but specifically for the **legacy** `openai_chat` path (e.g. `vanna/legacy/openai/openai_chat.py`), where the call is synchronous and always sends `temperature=self.temperature`. ## Environment - **Vanna:** 2.x with legacy API (`vanna.legacy.openai.openai_chat`, `vanna.legacy.chromadb`, `vanna.legacy.flask`) - **Backend:** Azure OpenAI - **Model/deployment:** gpt-5-nano (Azure) - **Python:** 3.12 ## References - [#855](https://github.com/vanna-ai/vanna/issues/855) – Make Temperature parameter optional (same idea for newer models) - [#262](https://github.com/vanna-ai/vanna/issues/262) – Allow setting temperature via config (we use this for the workaround)