Plex DupeFinder is a python script that finds duplicate versions of media (TV episodes and movies) in your Plex Library and tells Plex to remove the lowest rated files/versions (based on user-specified scoring) to leave behind a single file/version.
Duplicates can be either in bulk (automatic) or on-by-one (interactively).
Click to enlarge.
-
Python 3.6+.
-
Required Python modules (see below).
Note: Steps below are for Debian-based distros (other operating systems will require tweaking to the steps).
-
Install Python 3 and PIP
sudo apt install python3 python3-pip -
Clone the Plex DupeFinder repo.
sudo git clone https://github.com/l3uddz/plex_dupefinder /opt/plex_dupefinder -
Find your user & group.
id -
Fix permissions of the Plex DupeFinder folder (replace
user/groupwith yours).sudo chown -R user:group /opt/plex_dupefinder -
Go into the Plex DupeFinder folder.
cd /opt/plex_dupefinder -
Install the required python modules.
sudo python3 -m pip install -r requirements.txt -
Create a shortcut for Plex DupeFinder.
sudo ln -s /opt/plex_dupefinder/plex_dupefinder.py /usr/local/bin/plex_dupefinder -
Generate a
config.jsonfile.plex_dupefinder -
Fill in Plex URL and credentials at the prompt to generated a Plex Access Token (optional).
Dumping default config to: /opt/plex_dupefinder/config.json Plex Server URL: http://localhost:32400 Plex Username: your_plex_username Plex Password: your_plex_password Auto Delete duplicates? [y/n]: n Please edit the default configuration before running again! -
Configure the
config.jsonfile.nano config.json
{
"AUDIO_CODEC_SCORES": {
"Unknown": 0,
"aac": 1000,
"ac3": 1000,
"dca": 2000,
"dca-ma": 4000,
"eac3": 1250,
"flac": 2500,
"mp2": 500,
"mp3": 1000,
"pcm": 2500,
"truehd": 4500,
"wmapro": 200
},
"AUTO_DELETE": false,
"FIND_DUPLICATE_FILEPATHS_ONLY": false,
"FILENAME_SCORES": {
"*.avi": -1000,
"*.ts": -1000,
"*.vob": -5000,
"*1080p*BluRay*": 15000,
"*720p*BluRay*": 10000,
"*HDTV*": -1000,
"*PROPER*": 1500,
"*REPACK*": 1500,
"*Remux*": 20000,
"*WEB*CasStudio*": 5000,
"*WEB*KINGS*": 5000,
"*WEB*NTB*": 5000,
"*WEB*QOQ*": 5000,
"*WEB*SiGMA*": 5000,
"*WEB*TBS*": -1000,
"*WEB*TROLLHD*": 2500,
"*WEB*VISUM*": 5000,
"*dvd*": -1000
},
"PLEX_LIBRARIES": [
"Movies",
"TV"
],
"PLEX_SERVER": "https://plex.your-server.com",
"PLEX_TOKEN": "",
"SCORE_FILESIZE": true,
"SKIP_LIST": [],
"VIDEO_CODEC_SCORES": {
"Unknown": 0,
"h264": 10000,
"h265": 5000,
"hevc": 5000,
"mpeg1video": 250,
"mpeg2video": 250,
"mpeg4": 500,
"msmpeg4": 100,
"msmpeg4v2": 100,
"msmpeg4v3": 100,
"vc1": 3000,
"vp9": 1000,
"wmv2": 250,
"wmv3": 250
},
"VIDEO_RESOLUTION_SCORES": {
"1080": 10000,
"480": 3000,
"4k": 20000,
"720": 5000,
"Unknown": 0,
"sd": 1000
}
}The scoring is based on: non-configurable and configurable parameters.
-
Non-configurable parameters are: bitrate, duration, height, width, and audio channel.
-
Configurable parameters are: audio codec scores, video codec scores, video resolution scores, filename scores, and file sizes (can only be toggled on or off).
-
Note: bitrate, duration, height, width, audio channel, audio and video codecs, video resolutions (e.g. SD, 480p, 720p, 1080p, 4K, etc), and file sizes are all taken from the metadata Plex retrieves during media analysis.
-
You can set
AUDIO_CODEC_SCORESto your preference. -
The default settings should be sufficient for most.
-
Under
AUTO_DELETE, set your desired option.-
"AUTO_DELETE": true,- Plex DupeFinder will run in automatic mode. -
"AUTO_DELETE": false,- Plex DupeFinder will run in interactive mode. (Default)-
Options:
-
Skip (i.e. keep both):
0 -
Choose the best one (and delete the rest):
b -
Select the item to keep (and delete the rest):
#(i.e.1,2,3, etc).
-
-
-
-
Finds duplicates that only share the same file path.
"FIND_DUPLICATE_FILEPATHS_ONLY": false,
-
This option has a very limited use case, i.e. in instances where Plex may have glitched and created multiple duplicates of the same media item.
-
If using this setting, we recommend using UnionFS-Fuse that can generate whiteout files (
*_HIDDEN~) to prevent the deletion of the actual file on the system. The_HIDDEN~files can then be removed afterwards or even during the dupe cleanup (e.g.watch -n 5 rm -rf /mnt/local/.unionfs-fuse/*). -
The default settings should be sufficient for most.
-
You can set
FILENAME_SCORESto your preference. -
The default settings should be sufficient for most.
-
Go to Plex and get all the names of your Plex Libraries you want to find duplicates in.
-
Example Library:
-
-
Under
PLEX_LIBRARIES, list your Plex Libraries exactly as they are named in your Plex.-
Format:
"PLEX_LIBRARIES": [ "LIBRARY_NAME_1", "LIBRARY_NAME_2" ],
or
"PLEX_LIBRARIES": ["LIBRARY_NAME_1", "LIBRARY_NAME_2"],
-
Example:
"PLEX_LIBRARIES": [ "Movies", "TV" ],
-
-
Your Plex server's URL.
-
This can be any format (e.g. http://localhost:32400, https://plex.domain.ltd).
-
Obtain a Plex Access Token:
-
Fill in the Plex URL and Plex login credentials, at the prompt, on first run. This only occurs when there is no
config.jsonpresent.or
-
Visit https://support.plex.tv/hc/en-us/articles/204059436-Finding-an-authentication-token-X-Plex-Token
-
-
Add the Plex Access Token to
"PLEX_TOKEN"so that it now appears as"PLEX_TOKEN": "abcd1234",.- Note: Make sure it is within the quotes (
") and there is a comma (,) after it.
- Note: Make sure it is within the quotes (
-
"SCORE_FILESIZE": truewill add more points to the overall score based on the actual file size. -
The default settings should be sufficient for most.
-
Note: In some situations (e.g. a bad encode resulting in a large size), this may be something you want to turn it off (i.e.
false).
-
In Auto Delete mode, any file paths matching the patterns (i.e folders), listed in
SKIP_LIST, will be ignored. -
Example:
"SKIP_LIST": ["/Movies4K/"]
-
The default settings should be sufficient for most.
-
You can set
VIDEO_CODEC_SCORESto your preference. -
The default settings should be sufficient for most.
-
You can set
VIDEO_RESOLUTION_SCORESto your preference. -
The default settings should be sufficient for most.
You will need to make sure that Allow media deletion is enabled in Plex.
-
In Plex, click the Settings icon -> Server -> Library.
-
Set the following:
- Allow media deletion:
enabled
- Allow media deletion:
-
Click SAVE CHANGES.
Simply run the script/command:
plex_dupefinder
If you find this project helpful, feel free to make a small donation to the developer:
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent