GithubHelp home page GithubHelp logo

rickdb / absolute-series-scanner Goto Github PK

View Code? Open in Web Editor NEW

This project forked from zeroqi/absolute-series-scanner

0.0 2.0 0.0 829 KB

Seasons, absolute mode, Subfolders...

Python 99.70% PowerShell 0.30%

absolute-series-scanner's Introduction

Absolute Series Scanner

The Plex Scanner will make the video files showing in Plex, if not showing in Plex, it is a scanner issue. A Plex series scanner choose the following from the folders and file names:

  • Serie name
  • Season number
  • Episode number
  • Episode title (not filled by plex default series scanner, until the metadata agent refreshes it)
  • Episode year

The Plex metadata agent will find metadata (Serie Title, summary year, episode title, summary, posters, fanart, tags, ... ) for files showing in Plex. Anything missing there while the file shows up in Plex is an Agent issue, refer to the Agent readme here: https://github.com/ZeroQI/Hama.bundle/blob/master/README.md

Which Metadata/Title source to select?

  • Anime: AniDB.net, Hama use an offline title database from them ("main title" is the best, or romaji "x-jat". "En" titles have hoorrors like "bombshells from the sky" for "Asobi ni Iku yo!" serie). AniDB use small posters, no background. Hama use ScudLee's xml mapping files to crosss reference the anidb id to the tvdb series
  • TV Series: TheTVDB.com or TVrage or TheMovieDB (yep support series now), no db site will store (DVD) boxset specific files (nor sport or porn for tvdb). TVDB has high resolution posters, background images, screenshots, and episodes summaries, all lacking from AniDB.net, but they do not carry porn series so no metadata for this type. TheTVDB uses seasons which can be practical for long anime.
  • Movies: TheMovieDB.org, naming convention: "Movie Title (Year).ext"

File Naming Conventions

This scanner supports absolute and season numbering, but here are two references for guidelines

Grouping folder

  • If you used to use "Grouping folder / Show Name / Season 1 / Show Name e01.ext" convention from the root, it will now be skipped. You can just add it as additionnal root folder in the library: "D:/Anime/Dragon Ball/" for "D:/Anime/Dragon Ball/[2] Dragon Ball Z" folder for example...

Season folders

  • Seasons folders can have serie name afterwards ("Zero no tsukaima / Season 1 Zero no tsukaima")
  • Files in "Extras" folders will be ignored.
  • Allow grouping in Ark xxxxx folders transparently with seasons folders inside, or within a season folder
  • Specials go in "Specials" or "Season 0" folders.
    • Single seasons series will follow anidb specials numbering (unless specific tvdb guid forced).
    • Multiple seasons series will follow tvdb specials numbering
    • You can use Anidb numbering for specials (OP1a, NCOP, etc...) or explicitely label them as follow (s00e101, etc...).
    • Include all files not recognised as Season 0 episode 501+

Files

File naming convention Template / Folder Exemple
Splitting folders: 0-9 0-9, A,...,Z folder. Add EACH as folder. Do not use the parent folder
Episode Name Pattern: Season %S/%SN s%0Se%0E Season 2/Show Name s02e03.ext
Multi-Episode style: Extend Season 2/Show Name s02e03-05.ext
Multi-part episodes: cdX, discX, diskX, dvdX, partX, ptX Season 2/Show Name s02e03 - pt1.ext
Multi-Media Version: Movie Name (year) - 1080p.ext Movie Name (year) - 1080p.ext
Specials scrapped: Specials, Season 0 s00e01/OP1/Ed3a/NCOP/S01/S1.ext
Other non scrapped: Extras Extras/Show Name xxxx.ext
BD rips /path/to/series-library/Series Name Season 2 Series.Name.Disc1.S02.E01-E12/BDMV/STREAM
Movie files in Series libraries (since this is a Series Scanner) are supported if:
  • Files are in a folder with the same name or with a single file inside it
  • Files are numbered (01|ep 01|s01e01)
  • Filename contain " - Complete Movie"
Specials

Hama is a anidb (single season) & tvdb (multiple seasons) agent so either naming convention is fine. It will detect either successfully but you can convert one convention to the other whicle displaying by forcing ids (further down)

Let's use One piece special "Heart of Gold":

TVDB seasons

Anidb (single) season

####### Anidb type special numbering is detailed below:

Type Internal letter Episode number
Specials S Episodes 001-100
OPs C Episodes 101-150
EDs C Episodes 151-200
Trailers T Episodes 201-300
OPs/EDs P Episodes 301-400
Others O Episodes 401-500
unmapped Episodes 501-600
Local media assets

It is supported but through "local media assets" agent, add it and and put it before HAMA in the priority order.
https://support.plex.tv/hc/en-us/articles/200220717-Local-Media-Assets-TV-Shows

Data type Source Comment
fanart art/backdrop/background/fanart-1.ext -1 can be ommited (same level as Video TS)
Serie poster Serie folder: Show name-1/folder/poster/show.ext (jpg, jpeg, png, tbn)
Season poster Season folder: Season01a.ext (jpg, jpeg, png, tbn)
Banner banner/banner-1.jpg
Theme song theme.mp3
Subtitles file name.ext (srt, smi, ssa, ass)
Plexignore files .plexignore

Movie libraries can have "Extra" in a specifically named folders or with the following at the end of the filename (hyphen important, no space afterwards):

  • "Behind The Scenes" folder or "-behindthescenes" at the end of the filename
  • "Deleted Scenes" folder or "-deleted" at the end of the filename
  • "Featurettes" folder or "-featurette" at the end of the filename
  • "Interviews" folder or "-interview" at the end of the filename
  • "Scenes" folder or "-scene" at the end of the filename
  • "Shorts" folder or "-short" at the end of the filename
  • "Trailers" folder or "-trailer" at the end of the filename
Note: "Extras" folder is skipped by the absolute series scanner, put unsorted files in there, it won't show up in Plex

Forcing the movie/series ID

Hama supports the following guid_type:

  • anidb for AniDB.net (and and the behaviour changing mode anidb2)
  • tvdb for TheTVDB.com (and the behaviour changing modes: tvdb2, tvdb3, tvdb4)
  • [deprecated] tmdb For TheMovieDB.net (and the serie part of TheMovieDB: tsdb)
  • [deprecated] imdb For the International Movie DataBase (ids starts with "tt...")

You can specify the guid to use the following way:

  • In guid_type.id inside serie folder with the id in it (ex: tvdb.id file with tvdbid "114801" without double quotes in it)
  • In Serie folder name by adding " [guid_type-id_number]" at the end
  • In custom search serie name by adding " [guid_type-id_number]" at the end
guid_type Real file numbering Seasons numbering Episodes numbering Use case (example)
anidb Absolute 1 AniDb Force the anidb serie id. Serie will follow anidb episode numbering convention including specials
  • Sword Art online [anidb-8692]
anidb2 Absolute TVDB TVDB Map Anidb entries to the tvdb at the right season and ep number to show one Plex serie entry only (need to be within a single tvdb entry)
  • Sword Art online [anidb2-8692]
  • Sword Art online II [anidb2-10376]
  • Sword Art Online Extra Edition [anidb2-10022]
  • Gekijouban Sword Art Online: Ordinal Scale [anidb2-11681]
tvdb Season TVDB TVDB Force the tvdbid, serie will follow tvdb episode numbering convention including specials
  • Sword Art Online [tvdb-259640]
  • Season 1 - Sword Art Online [1-25]/ep ##.ext with ## from 1 to 25
  • Season 2 - Alfheim & Gun Gale Online [1-25]/ep ##.ext ## from 1 to 25
tvdb2 Absolute TVDB TVDB for absolute numbering displayed virtually as tvdb numbering, episode number resets to 1 each season, for series like Sword art Online(1-50, will be split into Season 1 [1-25] and Season 2 [1-25])
  • Sword Art Online [tvdb2-259640]/Ep ##.ext with ## from 1 to 50
tvdb3 Absolute TVDB Absolute For absolute numbering episodes displayed virtually using tvdb season numbering for long running series like One piece (1-700+, will be split into seasons while keeping the absolute episode number intact without having to create seasons in the real folder
  • Metantei Conan [tvdb3-72454]
tvdb4 Absolute, random season Abs/Custom/Arc db Absolute For absolute numbering episodes displayed using series arc as season for long running series with arcs like Dragon Ball Kai, or separated anidb series considered as half seasons by thetvdb (like 'Seraph of the end' numbered 1-24 splitted into 2 seasons). Will take the arc definitions from tvdb4.mapping.xml and posters from tvdb4.posters.xml unless the absolute numbered episodes were placed in season folders already
  • One Piece [tvdb4-81797]
tvdb5 TVDB Absolute Absolute Chronological order (specifically for Star Wars: The Clone Wars) will remove seasons present and use the chronological order to re-sort the episodes. First ep is s02e15 from memory...
  • Star Wars: The Clone Wars [tvdb5-83268]
Advanced modes

For when you have episodes of a series in SEPARATE parent folders but want them to show as a single series in Plex:

  • " [anidb2-xxxxx]" will find the season & eposide offset defiend in the ScudLee file and add into Plex with it's corresponding TVDB series/season/episode numbers
  • " [tvdb/2/3/4-xxxxx-sY]" episode numbers found in the files are left alone and added to season Y
  • " [tvdb/2/3/4-xxxxx-eZ]" episode numbers found in the files are adjusted (epNum+Z-1)
  • " [tvdb/2/3/4-xxxxx-sYeZ]" episode numbers found in the files are adjusted (epNum+Z-1) and added to season Y, Z is the offset for the episodes in season Y for when we want it to start mid tvdb season
  • !!IMPORTANT NOTES!!
    • When defining your modes on your folders:
      • If you don't use the same mode or compatible modes for all separate folders for a series, you will run into issues.
        • "anidb2", "tvdb" & "tvdb2" will work together
      • You might have to manually merge Plex series together if "anidb2"/"tvdb2" or "tvdb"/"tvdb2" are both used.
      • "anidb2"/"tvdb" should automatically merge (but Plex is not perfect so you still might have to manually merge)
      • "tvdb3" will not work correctly with any other modes so all folders of a series will have to have this mode
      • "tvdb4" will not work correctly with any other modes so all folders of a series will have to have this mode
Examples:
== Example 1 ==
- "Bakuman      [anidb2-7251]" = "Bakuman      [tvdb-193811-s1]" = "Bakuman [tvdb-193811]"
- "Bakuman 2011 [anidb2-8150]" = "Bakuman 2011 [tvdb-193811-s2]"
- "Bakuman 2012 [anidb2-8836]" = "Bakuman 2012 [tvdb-193811-s3]"

== Example 2 ==
- "Sailor Moon Crystal [tvdb2-275039]"
- "Sailor Moon Crystal Season 3 [anidb2-11665]" = "[tvdb-275039-s3]" | "[tvdb2-275039-s3]" (depending if you keep absolute numbered eps in seasons)
  
== Example 3 ==
  "Bleach                                    [tvdb3-74796]"
  "Bleach movie 1 Memories in the Rain       [tvdb3-74796-s0e3]"
  "Bleach movie 2 The Diamond Dust Rebellion [tvdb3-74796-s0e4]"
  
== Example 4 == tvdb4 Custom selected Arcs as seasons (as tvdb use them as half seasons for black lagoon for example)
  The arc definition to split into seasons the absolute numbering is done using the following order:
    - Seasons folders manually created by the user with absolute numbered episodes inside (seasons already mapped manually)
    - in a local "tvdb4.mapping" file inside serie folder with the following format lines, one per arc/season:
      \|\|\|\(optional)
    - without doing anything using the online arc database [github tvdb4.mapping.xml](https://github.com/ZeroQI/Absolute-Series-Scanner/blob/master/tvdb4.mapping.xml)
    
    <anime tvdbid="79604" name="Black Lagoon">
      01|001|012|The First Barrage
      02|013|024|The Second Barrage
      03|025|029|Roberta's Blood Trail
    </anime>
    <anime tvdbid="289906" name="Seraph of the End">
      01|001|012|Vampire Reign
      02|013|024|Battle in Nagoya
    </anime>

  "Black Lagoon                         [tvdb4-79604]"
  "Black Lagoon - The Second Barrage    [tvdb4-79604-s2]"
  "Black Lagoon - Roberta`s Blood Trail [tvdb4-79604-s3]"
  
  "Seraph of the End - Vampire Reign    [tvdb4-79604]"
  "Seraph of the End - Battle in Nagoya [tvdb4-79604-s2]"

Install

  • Download https://github.com/ZeroQI/Absolute-Series-Scanner/blob/master/Scanners/Series/Absolute%20Series%20Scanner.py
  • Save into [...]/Plex/Library/Application Support/Plex Media Server/Scanners/Series/Absolute Series Scanner.py Note:
  • "Scanners" and "Series" folder are not created by default and will need creating.
  • "Scanners" folder will be at the same level as "Plug-in Support" folder (in other words the same parent directory)
  • "Absolute Series Scanner.py" resides in Series folder, do not create an additional folder not listed like "absolute-series-scanner-master"

Linux install script example


mkdir -p '/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Scanners/Series'
wget -O '/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Scanners/Series/Absolute Series Scanner.py' https://raw.githubusercontent.com/ZeroQI/Absolute-Series-Scanner/master/Scanners/Series/Absolute%20Series%20Scanner.py
chown -R plex:plex '/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Scanners'
chmod 775 -R '/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Scanners'

Logs

Absolute series Scanner uses a pre-made list of folders to try to locate Plex Logs folder. If custom logs are not present, then either you created a library using default Plex scanner and not my "Absolute Series Scanner" or you have an unknown Logs folder location and will need to forward me the path to add in the source code... If the scanner crash, you will get either no files (library creation) or no change (library already created) and will need to attach the Plex log "Plex Media Scanner.log"

List of logs files:

  • Plex Media Scanner (custom ASS) - Library_Name.log contain all folders and files processed in a readable fashion, perfect for troubleshooting scanner issues.
  • Plex Media Scanner (custom ASS) - Library_Name - filelist Root_Folder_name.log contain all files in the root folder, so i can re-create your library with zero size files. I use a batch file to recreate a user's library after converting to utf-8 with notepad
  • Plex Media Scanner.log - Standard Plex Scanner Log, contain crash error in case of a bug in the scanner code

List of configuration files:

Troubleshooting:

  • Update the scanner to the latest (on windows, powershell script in same folder as the scanner can be used to update)
  • If the scanner not listed in scanner list
    • Make sure you did create a "Series" library, as a "Movie" library will not show a "Series" scanner like ASS
    • Make sure it is saved in [...]/Plex/Library/Application Support/Plex Media Server/Scanners/Series/Absolute Series Scanner.py
    • Check scanner file size and open the scanner file to check if it does have html/xml tags.
  • Recreate a Series library to the same folder, using Hama agent, this way all logs will start afresh
  • if no files are showing, the scanner is crashing, probably a code error, check Plex Media Scanner.log for crash errors
  • If Episodes are missing, wrong seasons or episode numbers, or serie(s) missing, check all "(custom ASS)" logs
  • If library doesn't add new content then most likelly the scanner is crashing (after scanner update) and will revert any changes to the library so nothing changes...

Include the following logs (in any case, specify if file not present):

  • [...]/Plex Media Server/Logs/Plex Media Scanner.log (scanner crash info)
  • [...]/Plex Media Server/Logs/Plex Media Scanner (custom ASS).log (episodes info)
  • [...]/Plex Media Server/Logs/Plex Media Scanner (custom ASS) filelist.log (library file list)

And post in:

Known issues:


Nov 16, 2016 18:48:53.594 [0x7f48c2324800] DEBUG - Adding subdirectory for scanner: /home/plex/things/anime/Ah! My Goddess 2
Nov 16, 2016 18:48:53.597 [0x7f48c2324800] ERROR - No module in VideoFiles
Nov 16, 2016 18:48:53.597 [0x7f48c2324800] ERROR - Error scanning directory .
Nov 16, 2016 18:48:53.597 [0x7f48c2324800] ERROR - No module in Absolute Series Scanner
Nov 16, 2016 18:48:53.598 [0x7f48c2324800] ERROR - We got an error scanning in /home/plex/things/anime

You bloody downloaded the web page and not the actual py file:


Jul 23, 2016 12:55:54.558 [5288] ERROR - Error scanning directory .
Jul 23, 2016 12:55:54.574 [5288] ERROR - No module in Absolute Series Scanner
Jul 23, 2016 12:55:54.574 [5288] ERROR - Error in Python: Looking up module:
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Plex Media Server\Scanners\Series\Absolute Series Scanner.py", line 8, in 
from lxml import etree # fromstring
ImportError: DLL load failed: The specified module could not be found.

On windows install https://www.microsoft.com/en-us/download/details.aspx?id=48145


2016-06-29 23:30:09,104 (30c) : CRITICAL (core:574) - Exception while loading code (most recent call last):
File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-a17e99e\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\core.py", line 563, in load_code
self.init_code = self.loader.load(self.init_path, elevated, use_xpython = Framework.constants.flags.use_xpython in self.sandbox.flags)
File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-a17e99e\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\code\loader.py", line 47, in load
code = self.compile(str(source), str(uni(filename)), elevated)
File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-a17e99e\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\code\loader.py", line 52, in compile
return RestrictedPython.compile_restricted(source, name, 'exec', elevated=elevated)
File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-a17e99e\Framework.bundle\Contents\Resources\Platforms\Shared\Libraries\RestrictedPython\RCompile.py", line 115, in compile_restricted
gen.compile()
File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-a17e99e\Framework.bundle\Contents\Resources\Platforms\Shared\Libraries\RestrictedPython\RCompile.py", line 68, in compile
tree = self._get_tree()
File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-a17e99e\Framework.bundle\Contents\Resources\Platforms\Shared\Libraries\RestrictedPython\RCompile.py", line 59, in _get_tree
tree = self.parse()
File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-a17e99e\Framework.bundle\Contents\Resources\Platforms\Shared\Libraries\RestrictedPython\RCompile.py", line 56, in parse
return niceParse(self.source, self.filename, self.mode)
File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-a17e99e\Framework.bundle\Contents\Resources\Platforms\Shared\Libraries\RestrictedPython\RCompile.py", line 38, in niceParse
compile(source, filename, mode)
TypeError: compile() expected string without null bytes

Rights issues on Windows 2008 R2, solved by changing rights for:

  • ...\Users\Administrator\AppData\Local\Plex Media Server\Plug-Ins
  • ...\Users\Administrator\AppData\Local\Plex Media Server\Scanners

On linux, permissions issues could prevent the scanner execution.

Task list

  • Support Media stubs .Disc files ? http://kodi.wiki/view/Media_stubs
  • Shall i write a Movie scanner using the same code? The Plex default movie scanner does an good job i believe ?

Reference: Link to Markdown or https://help.github.com/articles/basic-writing-and-formatting-syntax/

absolute-series-scanner's People

Contributors

circulon avatar deusxanime avatar endofline369 avatar erliz avatar frosty5689 avatar isairz avatar sven-7 avatar violetvivirand avatar xxshiina avatar zeroqi avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.