Huggingface Spaces about uniformer

[Andy1621]

@AK391 I have created a demo for image classification followed your code (https://huggingface.co/spaces/Andy1621/uniformer_image_demo). I will try to add video demo later ~~

[Andy1621]

Thanks for your interesting suggestions! Since these days are the Chinese spring festival, I will finish a web demo when free.

[AK391]

@Andy1621 Thanks, I opened a PR #9 to add a link/badge to the demo in the readme. Also for outputs types label can be used here https://gradio.app/docs/#o_label

[Andy1621]

@AK391 I have create a video demo and merge the PR! Thanks for your interesting suggestion!

[Andy1621]

@AK391 How can I set the font size of gradio.outputs.Label? It seems that the default font size is too large to show the prediction😂

[AK391]

@Andy1621 thanks video demo look great, you can try using custom css https://gradio.app/docs/#interface for the font size

[AK391]

@Andy1621 also I created a org for https://huggingface.co/Sense-X SenseTime X-Lab (similar to github https://github.com/Sense-X), you can move all your spaces+models+datasets here similar to how you do on github, just send a invite request and I can accept it

[Andy1621]

@AK391 Look great!
(1) Is there any good practice for uploading models? I find swin create one model for each model (https://huggingface.co/microsoft/swin-small-patch4-window7-224). And there are README.md, config.json, preprocessor_config.json, and pytorch_model.bin.
(2) Besides, how can I upload my model code? I find they can simply import their model like from transformers import SwinForImageClassification.
(2) How can I upload multi models fast? It seems that I should upload one model at one time.

[Andy1621]

@AK391 BTW, I have sent a request to the organization you created. Would you please accept it and give me the corresponding permission to verify the organization?

[AK391]

@Andy1621 Thanks, approved request to adding to organization. Regarding adding models, there is a step-by-step guide for adding your model to the Hugging Face Hub. Once models are added they can be used in spaces in two ways, one like here https://huggingface.co/spaces/akhaliq/JoJoGAN by using

from huggingface_hub import hf_hub_download

and if the inference api is setup they can be added using

gr.Interface.load()

like here https://huggingface.co/spaces/akhaliq/xm_transformer_600m

[Andy1621]

@AK391 Thanks for your patience. I will try to add some models today.

[Andy1621]

@AK391 I have updated some popular models and demo in hugging. More models will be uploaded if needed.

[AK391]

@Andy1621 Great thanks, also once a model is added to the hub it can be easily turned in a Gradio space using, for example

import gradio as gr
gr.Interface.Load("huggingface/Sense-X/uniformer_image").launch()

see

https://gradio.app/docs/#load

[Andy1621]

@AK391 Thanks! But how does it work? It seems that such load api does not create an Interface (including fn, inputs and outputs). And the code you use does not initialize a model like model=uniformer_small().

import gradio as gr
gr.Interface.Load("huggingface/Sense-X/uniformer_image").launch()

gradio.Interface(
  self, fn, inputs=None, outputs=None, examples=None, examples_per_page=10, 
  live=False, layout="unaligned", interpretation=None, num_shap=2.0, theme=None, 
  title=None, description=None, article=None, thumbnail=None, css=None, 
  allow_screenshot=True, allow_flagging=None, flagging_options=None, flagging_dir="flagged"
)

Gradio interfaces are created by constructing a Interface object with a locally-defined function,
or with Interface.load() with the path to a repo or by Interface.from_pipeline() with a Transformers Pipeline.

Should I change the code for repo? It seems confused to use such load api.

---

One more question, since I have not pull UniFormer to the transformers repo, I can not use the model api of transformers like swin:

from transformers import AutoFeatureExtractor, SwinForImageClassification
feature_extractor = AutoFeatureExtractor.from_pretrained("microsoft/swin-tiny-patch4-window7-224")
model = SwinForImageClassification.from_pretrained("microsoft/swin-tiny-patch4-window7-224")

thus I suggest others used as in model card: create model => load model by hf_hub_download => model inference.
And I upload multi models in one Model, which is not like deit and swin that created by Hugging Face team. They simply upload one model for one Model (see https://huggingface.co/models?search=swin).

[AK391]

@Andy1621 for the load function see https://github.com/gradio-app/gradio/blob/90c28eae4612c17c9f9dc94ac30ef63eb036818d/gradio/interface.py#L65

[Andy1621]

@AK391 Thanks!

def get_huggingface_interface(model_name, api_key, alias):
    model_url = "https://huggingface.co/{}".format(model_name)
    api_url = "https://api-inference.huggingface.co/models/{}".format(model_name)
    print("Fetching model from: {}".format(model_url))


    if api_key is not None:
        headers = {"Authorization": f"Bearer {api_key}"}
    else:
        headers = {}


    # Checking if model exists, and if so, it gets the pipeline
    response = requests.request("GET", api_url, headers=headers)
    assert response.status_code == 200, "Invalid model name or src"
    p = response.json().get("pipeline_tag")

The followed dict is the response of https://huggingface.co/microsoft/swin-tiny-patch4-window7-224

{
  'id': 'microsoft/swin-tiny-patch4-window7-224', 
  'modelId': 'microsoft/swin-tiny-patch4-window7-224', 
  'private': False, 
  'pipeline_tag': 'image-classification', 
  'sha': '896e18f760e16594e63d8bbe01ada6142bb0ef8e', 
  'lastModified': '2022-01-28T13:12:55.000Z', 
  'tags': ['pytorch', 'swin', 'image-classification', 'dataset:imagenet-1k', 'arxiv:2103.14030', 'transformers', 'license:apache-2.0', 'vision'], 
  'siblings': [{'rfilename': '.gitattributes'}, {'rfilename': 'README.md'}, {'rfilename': 'config.json'}, {'rfilename': 'preprocessor_config.json'}, {'rfilename': 'pytorch_model.bin'}], 
  'downloads': 1165, 
  'library_name': 'transformers', 
  'widgetData': [{'src': 'https://huggingface.co/datasets/mishig/sample_images/resolve/main/tiger.jpg', 'example_title': 'Tiger'}, {'src': 'https://huggingface.co/datasets/mishig/sample_images/resolve/main/teapot.jpg', 'example_title': 'Teapot'}, {'src': 'https://huggingface.co/datasets/mishig/sample_images/resolve/main/palace.jpg', 'example_title': 'Palace'}], 
  'likes': 0, 
  'model-index': None, 
  'config': {'architectures': ['SwinForImageClassification'], 'model_type': 'swin'}, 
  'cardData': {'license': 'apache-2.0', 'tags': ['vision', 'image-classification'], 'datasets': ['imagenet-1k'], 
  'widget': [{'src': 'https://huggingface.co/datasets/mishig/sample_images/resolve/main/tiger.jpg', 'example_title': 'Tiger'}, {'src': 'https://huggingface.co/datasets/mishig/sample_images/resolve/main/teapot.jpg', 'example_title': 'Teapot'}, {'src': 'https://huggingface.co/datasets/mishig/sample_images/resolve/main/palace.jpg', 'example_title': 'Palace'}]}
}

As what I guessed, it seems that such api load() only works for those models in transformers repo (required special config and pipeline_tag). And such api does not support video_classification.
Now I simply upload multi models in one Model and used it as space to share models, which is not consistent with other Model. Their model codes are merged in transformers, so they can used the apis.

[AK391]

@Andy1621 you can see the currently supported inference api pipelines gradio has here: https://github.com/gradio-app/gradio/blob/c9298b38021323918037a5a39914a787e8517f60/gradio/external.py#L49, video classification is currently not support through huggingface or gradio but you can try image classification, see current pipelines here for huggingface model hub https://github.com/huggingface/huggingface_hub/blob/df012f1489cb50f15e0908598d7e355f1f31b52f/js/src/lib/interfaces/Types.ts

[Andy1621]

As there is no more activity, I am closing the issue, don't hesitate to reopen it if necessary.