Comments (22)
I think the manual deletion of images is probably the source of the issues here: we don't expect these files to be touched by anything but the server.
Yeah, but also they show up as missing, so why is the server not regenerating them? The thumbnail cache should be safe to delete, for all the reasons imaginable. It's not actually important data.
As it is, closing the issue without knowing why specifically this happens is a bit counterproductive. If at least we know exactly why that happens, someone could make a MR working around or fixing the main issue.
from immich.
I think we moved this to the face-detection job, correct? @mertalev
from immich.
The facial recognition job just queues person thumbnail generation. The thumbnail generation itself is the same as always and queued the same way here when clicking All or Missing. All person thumbnails are generated in the thumbnail generation queue.
@NyaomiDEV Can you elaborate on why you think it isn't regenerating person thumbnails? Just deleting the files won't re-generate them when running Missing, if this is what you tested. That queues assets based on data in the database, not by checking each asset. All would re-generate everything, so that should indeed re-generate a deleted thumbnail.
from immich.
@mertalev In my testing I pressed "all" and it didn't regenerate any of them. Since I deleted the old thumbnails manually, if there was at least one thumbnail to be generated I would've catched that, but there was nothing at all. In the night time (EU here) I scheduled a facial recognition job to redo everything from scratch, though I should still check the results.
Okay so I checked and re-running facial detection and recognition for all files actually mitigates this issue; point is I was lucky to never have used the people feature so I didn't assign any names or merged anyone yet. I can't imagine the stress of doing this when you've already tracked that sort of stuff.
from immich.
OH! Yeah... people's thumbnails are still a Jpeg.
from immich.
I don't think we mention anywhere that facial thumbnails can be generated with different extensions. In the release note, it was about the preview (small) and the thumbnail (large) for an asset
from immich.
I don't think we mention anywhere that facial thumbnails can be generated with different extensions. In the release note, it was about the preview (small) and the thumbnail (large) for an asset
True that, however this means that I shouldn't have deleted all the Jpegs... that's my fault. However, I'd love to ask for this feature for the sake of consolidation.
This still doesn't mean that the bug isn't there, as they didn't get regenerated when I ran the thumbnails job.
from immich.
All not regenerating the thumbnails is weird. A few things come to mind:
-
We don't handle the preview size being different from when the faces were detected (see #8317). If you changed it then at least some of the jobs would fail because of invalid bounding boxes.
-
Some of the person thumbnails might have been based on the motion part of a live photo. Person thumbnails are cropped from preview images. Since you deleted the existing previews and thumbnail generation now skips these assets, jobs to generate person thumbnails will fail in this case since they're trying to crop images that don't exist.
Could you check the logs in the microservices container to see if there are any errors during thumbnail generation?
Also yes, the format change should ideally apply to person thumbnails as well. It can just use the thumbnail setting for this.
from immich.
Could you check the logs in the microservices container to see if there are any errors during thumbnail generation?
Sadly I had to bring down and up the server again, and in doing so I missed a bunch of logs, as I am not keeping them on disk.
The only thing I remember about the microservices container is that it sometimes timed out, so I brought down the number of concurrent tasks, thinking Postgres couldn't probably handle that traffic. I rebuilt the thumbnail cache since, and still, no thumbnails; also, if that was the case I would get at least some thumbnails, whereas all my other ones would've been spotty.
Of course, I could just re-delete all the Jpegs again and try it out yet again, but I'd kinda want to not reschedule another face recognition task later tonight.
from immich.
That's fine! I'll see if I can reproduce this. If the person thumbnails do get regenerated under normal circumstances, then it means the jobs are failing in your case.
from immich.
Just wanted to chime in saying that I have this very same issue for a few weeks now, I just didn't get around to creating an issue.
Had to delete thumbnails when my SSD filled up and services stopped working, and ever since all person thumnails are gone. Thumbnails can be "restored" by selecting a new featured photo, but that would mean doing that for hundreds of persons.
Since I deleted all thumbnails, I also definitely ran the "All" job and not just "Missing".
Sadly I've made heavy use of the persons-feature, and a lot of work would be lost by re-running face recognition for all assets.
Let me know if I can help in any way!
from immich.
I tested and the person thumbnails really are queued and generated when running thumbnail generation on all assets. Setting the logging to verbose and running All shows Cropping face for person
log entries after some time. I think the manual deletion of images is probably the source of the issues here: we don't expect these files to be touched by anything but the server.
from immich.
If you say thumbnail generation is working correctly and the database is simply in a bad state after the external deletion, maybe there should be a way to repair the missing assets somehow? So that afterwards the thumbnails can be re-generated normally :)
from immich.
why is the server not regenerating them
The jobs are most likely failing for the reasons above, which we're already aware of. Sharing the full verbose logs after running All would be helpful to see if the issue is distinct from those.
from immich.
I'd like to avoid doing a full rescan until #8760 is fixed, otherwise I might end up with a full SSD again. But once that is the case I'd be happy to provide logs!
from immich.
That should be fixed via #9126, but you're right that it hasn't made it into a release yet.
from immich.
Not fixed in v1.103.1, I regenerated ALL thumbnails and still doesn't have people faces showing up :(
from immich.
After digging through the log, there's only one message that could correspond to this issue :
[JobService] Unable to run job handler (thumbnailGeneration/queue-generate-thumbnails): QueryFailedError: insert or update on table "person" violates foreign key constraint "FK_2bbabe31656b6778c6b87b61023"
[JobService] QueryFailedError: insert or update on table "person" violates foreign key constraint "FK_2bbabe31656b6778c6b87b61023"
Should I open a new issue?
from immich.
That's an interesting log. Encountering that error would prevent any later people from getting queued for thumbnail generation.
from immich.
Using the ghcr.io/immich-app/immich-server:pr-9195
image for immich-microservices
and re-running should fix that issue. Would you be able to try it?
from immich.
That's an interesting log. Encountering that error would prevent any
laterpeople from getting queued for thumbnail generation.
Exactly! I've seen this error only ONE time in the log but ~ hundred people don't have a thumbnail...
Using the
ghcr.io/immich-app/immich-server:pr-9195
image forimmich-microservices
and re-running should fix that issue. Would you be able to try it?
Yes I'll try, thank you! Do I have to regenerate all thumbnails or only missing ones would do it?
from immich.
Yes I'll try, thank you! Do I have to regenerate all thumbnails or only missing ones would do it?
I think it'd have to be all thumbnails for the person thumbnails to get regenerated.
from immich.
Related Issues (20)
- External Library - Delete Library, also deletes all files on the storage HOT 5
- [IOS] Upload video above 1gb, never finish even when 100% file transfered HOT 15
- [IOS] Upload video above 1gb, never finish even when 100% file transferred BUT file 360mb works ok. HOT 3
- Modify the section of the document about external libraries
- timeline view - shortcut Shift-a not working to move assets to archive
- Playing motion photo pauses music
- Photos are not updated in the Mobile version HOT 5
- Fujifilm color space from RAW (RAF file) is incorrect (preview images and thumbnail) HOT 2
- Cann't set language in Immich web HOT 1
- Username used for SMTP auth is not valid email address
- Ignore Certificate Error issue on registering a new account
- Number of media uploaded HOT 1
- BackUp Again all items after reinstalling Android App HOT 6
- Notification popups are covering import progress HOT 1
- [JobService] Unable to run job handler (thumbnailGeneration/generate-preview): Error: Processed image is too large for the WebP format HOT 3
- Immich defaults to software encoding for remaining videos after a hardware accelerated encode fails. HOT 2
- Encoding fails with opus audio and hardware acceleration enabled HOT 1
- 360 panorama in HEIF/HEIC format doesn't render
- Duplicate files, remainder is 0, but files are still being uploaded HOT 1
- WebSocket Connect Error, version status offline
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from immich.