Do not perform reverse update weights. about ultralytics

[glenn-jocher]

@cainiao123s hello,

Thank you for providing detailed information about your issue. It seems you are encountering variability in loss values across epochs even though the model parameters are not being updated. This can happen due to several reasons, even with a fixed seed:

1. Data Loading: If there's any randomness in the way data is loaded or augmented during training, this could lead to variations in the loss. Since you've disabled parameter updates, the model itself should not be changing, but the input data might be.

2. Floating Point Precision: Operations on CPUs and GPUs can have slight differences in floating point precision, which might cause minor variations in computed loss.

3. Environment Factors: Other factors such as underlying library updates, OS-level updates, or hardware-specific optimizations might also influence results.

To further investigate, you might want to ensure that all data loading and augmentation steps are deterministic. Also, check if any external libraries or functions you use in the data preprocessing or loss computation might introduce randomness.

If the issue persists, providing more details about the data handling and any preprocessing steps might help in diagnosing the problem more effectively.

[cainiao123s]

After a long time of effort, I found that it is very likely that the data will be processed for each epoch. Do you operate like this?
Because I modified the category of a training label file, I am now using the file coco8.yaml, and the new data obtained is as follows:

loss_items: tensor([0.5801, 2.4663, 1.3199])	loss_items: tensor([2.1151, 5.2178, 2.1823])	loss_items: tensor([0.5253, 1.9911, 1.2469])
loss_items: tensor([1.8331, 4.7669, 2.0235])	loss_items: tensor([0.7512, 4.4382, 0.9191])	loss_items: tensor([2.5066, 3.5849, 2.0813])
loss_items: tensor([0.9504, 4.8735, 1.2347])	loss_items: tensor([0.3981, 2.1305, 1.1685])	loss_items: tensor([0.7035, 5.8424, 0.9036])
loss_items: tensor([2.0359, 4.1217, 1.7355])	loss_items: tensor([2.2144, 3.5352, 2.1413])	loss_items: tensor([1.7366, 4.7452, 1.9406])

The data above indicates that the first data in the first column, the third data in the second column, and the first data in the third column have changed, which suggests that some operations should be performed on the dataset for each epoch, and not every data loading process for each epoch is the same.

[glenn-jocher]

Hello @cainiao123s,

Thank you for sharing your observations. Yes, in YOLOv8, data processing can indeed vary for each epoch. This variation is typically due to the data augmentation techniques applied during training to enhance the model's ability to generalize. These techniques might include random transformations like scaling, cropping, flipping, or color adjustments, which can lead to different loss values even for the same image across different epochs.

The changes you're observing in the loss items across epochs are expected behavior when such data augmentations are active. If you need consistent data input for each epoch to test specific behaviors or modifications, you might consider disabling these augmentations temporarily.

Let us know if you have further questions or need more details on how to proceed! 😊