[feature request] Support different serving servers about kserve

[issue-label-bot]

Issue-Label Bot is automatically applying the label feature_request to this issue, with a confidence of 0.94. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: dashboard, app homepage and code for this bot.

[gaocegege]

/kind feature

[ellistarn]

Thanks for the feature! Given our data model (high level "Tensorflow" spec), do you think it would make most sense to swap different Tensorflow technologies with annotations?

Regarding:
"Maybe we also should investigate how to support co-serving multiple models in one serving CRD.".

I think this is an anti-pattern in KFServing. A KFServing Service is a "unit" of model serving. The idea of hosting multiple models in a single model server is extremely interesting and something I've been working on in my share time. I've been thinking we should control this via implementation, not interface, and allow users to specify an annotation like "enable-multitenancy".

I'll make a separate issue for this since I think it's a huge topic.

[ellistarn]

This is either implemented or open in separate issues
/close

[k8s-ci-robot]

@ellis-bigelow: Closing this issue.

In response to this:

This is either implemented or open in separate issues
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.