Image Preprocess in Clip about transformers

[lesjie-wen]

Thank you very much @amyeroberts, l change my code to resize images bigger than the target size and then crop to target size. And it works well.

[amyeroberts]

Hi @lesjie-wen, thanks for opening this issue!

The CLIPImageProcessor defines a series of steps which will prepare raw images in the format expected by the CLIP model. It follows the preprocessing steps done for the original model.

The first resize will resize according to processor.size. You can modify the output size by passing size={"shortest_edge: x"} or size={"height": h, "width": w} when calling the image processor e.g. processor(images, size={"height": h, "width": w}).

The reason the image is (324, 324) when output, is because this is the crop size specified in the example provided:

image = self.processor.preprocess(image, return_tensors='pt', do_normalize=False, do_rescale=False, do_center_crop=True, crop_size={'height':324,'width':324})['pixel_values']

If you want a size that matches the target size you can either:

• Change crop_size when calling the processor to match your needs i.e crop_size is the target size. This is the most common behaviour when processing images.

image = self.processor.preprocess(image, return_tensors='pt', do_normalize=False, do_rescale=False, do_center_crop=True, crop_size={'height': target_height, 'width': target_width})['pixel_values']

• Call processor.resize on the output images

from transformers.image_processing_utils import BatchFeature

images = self.processor.preprocess(image, do_normalize=False, do_rescale=False, do_center_crop=True, crop_size={'height': target_height, 'width': target_width})['pixel_values']
images = [self.processor.resize(image, size={"height": target_height, "width": target_width}) for image in images]
images = BatchFeature({"pixel_values": images}, tensor_type="pt")