Issue with the .hdf5 file about transformer-ocr

[him4318]

Hi,
Can you check if the image is able to load as NumPy array or not?
One more thing can you give the path of the images just as a simple string not with the symbol b as it converts it to bytes and checks if it solves your problem.

[sbuitragoh]

Hi,

I'll check right now the path. For some reason it saves in that way.
How can I load any image from the hdf5 file?

Edit: Apparently, the path naming thing is in the transformation to the .hdf5 file. The dataset is normal.

[sbuitragoh]

cnt.

I checked every part but can't find why there's that b in the string of the .hdf5 file.
In the dataset it's like this:
{'train': {'dt': ['./data/lines/rm_149_075_001.png', './data/lines/rm_149_075_005.png', ...},
also
type(ds.dataset['train']['dt'])
<class 'list'> and
type(ds.dataset['train']['dt'][0])
<class 'str'>
But if I look inside the file, the string is like this:
hf[i]['dt'][0]
b'./data/lines/rm_149_075_001.png'

Sadly, still can't find a way to change it.

[him4318]

In the Data Generator class can you look in the __getitem__ function and check img = self.dataset[self.split]['dt'][i] this line, as this is the line where image is being read from the hdf5 file. you can print the shape of the array and see if it is properly loaded or not.

[sbuitragoh]

I guess it's not loaded correctly since img has class bytes. As I read, the hdf5 file should contain the numpy array of the image but I only have the string (who mysteriously transforms into bytes). Is it to crazy if I load the image using opencv, changing the bytes to string?

[him4318]

There is no need to create hdf5 file if it is creating issue for you. You can directly read an image using open CV and perform same pre-processing on it and send it to generator. For pre-processing step you can look at Single_inference_image function in engine.py to get some idea.

[sbuitragoh]

It seems is the hdf5 file. I'll be handling all of it directly with opencv.
Thanks.