Regression in support of customized "role" in OpenAI compatible API (v.0.4.2) about vllm

[simon-mo]

Merged.

[DarkLight1337]

This is likely the result of #4355, which made ChatCompletionRequest.messages more strict to avoid unrecognized attributes. Would this interface be sufficient for defining custom roles?

class CustomChatCompletionContentPartParam(TypedDict, total=False):
    __pydantic_config__ = ConfigDict(extra="allow")  # type: ignore

    type: Required[str]
    """The type of the content part."""


ChatCompletionContentPartParam = Union[
    openai.types.chat.ChatCompletionContentPartParam,
    CustomChatCompletionContentPartParam]


class CustomChatCompletionMessageParam(TypedDict, total=False):
    """Enables custom roles in the Chat Completion API."""
    role: Required[str]
    """The role of the message's author."""

    content: Union[str, List[ChatCompletionContentPartParam]]
    """The contents of the message."""

    name: str
    """An optional name for the participant.

    Provides the model information to differentiate between participants of the
    same role.
    """


ChatCompletionMessageParam = Union[
    openai.types.chat.ChatCompletionMessageParam,
    CustomChatCompletionMessageParam]


class ChatCompletionRequest(OpenAIBaseModel):
    messages: List[ChatCompletionMessageParam]
    ... # The rest is the same as OpenAI API

[Tostino]

Thank you for the PR @DarkLight1337. Was wondering why my data pipeline stopped working when I just upgraded vLLM.

[tanliboy]

Thank you, @simon-mo, @DarkLight1337 and @Tostino !
Is the PR ready to merge and incorporate into the upcoming release?