AttributeError: 'Image' object has no attribute 'shape' about vila

[ZackBradshaw]

I keep getting this bug. Is the training expecting already converted tensor data?

[ZackBradshaw]

this will train for a moment up to 13% then throws these errors

[Lyken17]

Can u attach a full log here?

[ZackBradshaw]

Traceback (most recent call last):
  File "/home/user/.lib/.lib/VILA/llava/train/train_mem.py", line 36, in <module>
    train()
  File "/home/user/.lib/.lib/VILA/llava/train/train.py", line 436, in train
    trainer.train(resume_from_checkpoint=resume_from_checkpoint)
  File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1537, in train
    return inner_training_loop(
  File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1821, in _inner_training_loop
    for step, inputs in enumerate(epoch_iterator):
  File "/usr/local/lib/python3.10/dist-packages/accelerate/data_loader.py", line 452, in __iter__
    current_batch = next(dataloader_iter)
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py", line 633, in __next__
    data = self._next_data()
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py", line 1345, in _next_data
    return self._process_data(data)
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py", line 1371, in _process_data
    data.reraise()
  File "/usr/local/lib/python3.10/dist-packages/torch/_utils.py", line 644, in reraise      
    raise exception
AttributeError: Caught AttributeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop
    data = fetcher.fetch(index)
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/_utils/fetch.py", line 51, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataset.py", line 243, in __getitem__
    return self.datasets[dataset_idx][sample_idx]
  File "/home/swarms/.lib/.lib/VILA/llava/data/dataset.py", line 837, in __getitem__        
    if len(image.shape) == 4:
AttributeError: 'Image' object has no attribute 'shape'

I'm able to push it along further by adding logic to just convert this to tensor but ultimately this just leads to more issues further on in the training.

[Lyken17]

my suggestion would be

1. disable the training logic and simpley iterate through the dataset
2. check every data and make sure the return value if tensor

This looks like some of your images are grayscale or corrupted.

[ZackBradshaw]

Will this work work with other tools such as xtuner I cannot for the life of me figure this out