TypeError: normalize() argument 2 must be str, not numpy.ndarray about transformer-ocr

[him4318]

Sorry closed it by mistake you can re-open it.

[him4318]

Hi,
Can you check your data loader, this is with it only instead of text it is getting arrays.

[corbinan]

Hi thanks, what should I change?

[him4318]

You can try to put print statement in the get_item function of data loader.

    def __getitem__(self, i):
        img = self.dataset[self.split]['dt'][i]
        
        #making image compatible with resnet
        img = np.repeat(img[..., np.newaxis],3, -1)    
        img = pp.normalization(img)
        
        if self.transform is not None:
            img = self.transform(img)

        ###Here
        ###if it's array then change it accordingly in the code.
        print(self.dataset[self.split]['gt'][i])


        y_train = self.tokenizer.encode(self.dataset[self.split]['gt'][i]) 
        
        #padding till max length
        y_train = np.pad(y_train, (0, self.tokenizer.maxlen - len(y_train)))

        gt = torch.Tensor(y_train)

        return img, gt          

    def __len__(self):
      return self.size

[corbinan]

Did that, it has printed an array of numbers

How do I fix this in the data loader?

[him4318]

So it seems there is a problem with hdf5 itself, while creating it you have saved the integer values instead of text.

[corbinan]

I had run it like this:

python3 main.py --source=bentham --transform

[corbinan]

Ran it again and this time I got this:

normalize() argument 2 must be str, not numpy.bytes_

From printing like you asked me to above, I get this: