Issue with Training - Generator error about triq

[junyongyou]

I think you probably need to check your data (images). You don't need to convert the format to object.

[ngun7]

During inference, image size is being reduced to (512,384) with maximum_positional_encoding size of 193. Here, I want to train a model with bigger images so that I don't have to downsample during prediction. During splitting, I resized koniq_normal & koniq_small images to (1024,768) and set maximum positional encoding to 769. Do you think this might be the issue?

triq/src/databases/random_split_imageset.py

Line 16 in 5a0a797

image = Image.open(os.path.join(original_image_folder, image_file)).resize((512, 384))

[junyongyou]

Hi, I don't think that matters. I meant that you need to check your data. Because the model has already been trained for 170 steps without any problems, and then the generator throws an error of data conversion. I would guess something wrong with an image data at the 171st step. So you can try to only run the generator on the images, and see if you can get the correct data at all steps.

[ngun7]

Hello @junyongyou, thanks for the reply. I checked the data(even tried with a single dataset koniq_normal) and everything seems fine. More than generator error, script stops with "val_loss" key error. This happens all the time after running for few steps in an epoch. Attaching screenshot:

[junyongyou]

Hi, I have just downloaded the code and tried to train for a couple of epochs, and everything was fine. I really don't know what your problem is from. My current TF version 2.5.1. Maybe check your TF?