Regenerating thumbnails doesn't relink all of them inside the database about immich

[mertalev]

Thanks for the detailed issue!

After migrating from WebP to Jpeg I noticed that all of the Jpeg files were left over inside the thumbnails folder

This is an oversight. It used to be that there was only one possible path, so re-running thumbnail generation could only overwrite the existing thumbnail. But now we need to clean up more explicitly to avoid having leftover images.

I think Live Previews do have a thumbnail of their own which means extra storage used. Why is that?

This was a bug that was recently fixed in #8891. It’s because we mostly treat the motion part as a regular video in jobs.

the ACTUAL bug is, sure the job regenerated their thumbnails into WebP format, but it forgot to update the database, it seems, because in my Offline Paths section now I have 1048 missing assets and faces

I think it’s the opposite. They show up as untracked because the paths in the DB were updated, so there’s no longer any reference to them. We should change the job to clean up the old thumbnail.

[NyaomiDEV]

I think it’s the opposite. They show up as untracked because the paths in the DB were updated, so there’s no longer any reference to them. We should change the job to clean up the old thumbnail.

At first there were untracked paths (a ton of Jpeg thumbnails indeed), however after my brutal attempt at cleanup I have offline paths!

Again, if I take a look at the actual paths, the offline path ends in .jpeg, but there is a .webp counterpart available.

[NyaomiDEV]

What I think happened is that the job runs for all thumbnails including live previews, but since the live previews are "invisible" the thumbnail reference doesn't get updated in the database (it's supposed to not even exist by now). Since I have an Immich instance that existed before that MR you pointed me at, probably it's an issue for upgraded instances only, as they still have live previews with a generated thumbnail and a reference to that.

However this should mean that, in new instances, there'd be a lot of untracked files in the case someone also stores live previews. I think that may also happen if a live preview video is uploaded before the image counterpart and the two are still not linked (on a fast server perhaps?) but that's only speculation.

[mertalev]

I think that may also happen if a live preview video is uploaded before the image counterpart and the two are still not linked (on a fast server perhaps?) but that's only speculation.

Oh, you're right: the motion part can still end up going through the full pipeline before it's linked. I'm not sure how often this would happen, but it's a bit unfortunate that it can. I guess the best we could do there is to clean it up during linking after it wastes time processing them.