stokkes / plex_rcs

Plex rclone cache scanner

Plex rclone cache scanner (plex_rcs)

A small little program that will monitor an rclone log file waiting for notices of file cache expiration. Upon receiving a notice, a local Plex scan of that folder will be triggered and new media will appear in Plex almost instantly.

This is useful for people who run Plex Media Server on a different server than Sonarr/Radarr/etc. It's possible to have media appear in your Plex server within 5-10 minutes of downloading using this program, along with an rclone --cache-tmp-wait-time 5m.

Requirements

1. Python 3+
2. Your rclone cache mount must include --log-level INFO
3. Your rclone cache mount must include --syslog OR --log-file /path/to/file.log

Installation

1. Clone this repo: git clone https://github.com/stokkes/plex_rcs.git
2. Install the requirements: sudo pip3 install -r requirements.txt

Configuration

1. Copy the config.yml.default to config.yml
2. Edit config.yml to include your X-Plex-Token, set your media_root setting and any other settings. See below for more information on the media_root setting.

Running plex_rcs

There are two ways I recommend you run this. Using screen or using the included plex_rcs.service systemd service (coming soon).

Using screen

Execute the program using screen:

/usr/bin/screen -dmS plexrcs /path/to/plex_rcs/plex_rcs.py

To view the console: screen -r plexrcs

Using systemd

Coming soon

1. Edit the included plex_rcs.service file and change the path to where the plex_rcs.py file is located
2. Copy the systemd file to /etc/systemd/service: sudo cp plex_rcs.service /etc/systemd/service
3. Reload systemd: sudo systemctl daemon-reload
4. Enable the service [auto-starts on boot]: sudo systemctl enable plex_rcs
5. Start the service: sudo systemctl start plex_rcs

More info on media_root configuration setting

This setting may be tricky to figure out at the first glance, but it is critical to get plex_rcs working properly.

The value of this setting is the folder inside your docker container (if using docker) that contains all your media. Typically, this would be in /media. If not using docker, this will likely be the path to your rclone cache remote mount, i.e.: /mnt/media

However, at this time there is 1 requirement:

The root of your rclone cache remote (i.e.: gdrive-cache:) must contain all your media in sub-folders, so that the remote and the folder that is mounted inside your docker container/on your system both contain the same sub-folders.

How to test:

1. rclone lsd cache: and
2. docker exec -ti plex ls /media (where /media is where your media is located inside your docker container)
3. If not using docker, ls /path/to/rclone/cache/mount

If the result of these two folders yield the same sub-folders, then plex_rcs will work correctly.

What won't work

1. rclone lsd cache: shows many different folders, not just media sub-folders
2. You've mounted the cache: remote to /mnt/media using something like rclone mount cache:Media /mnt/media
3. docker exec -ti plex ls /media shows only your media sub-folders

I hope to build some logic to help figure this out, but don't hold your breath.

Testing

You can test plex_rcs if you use the built in /var/log/syslog monitoring by executing the following command (replace tvshows and the series/episode by your values):

logger "Apr 21 07:20:51 plex rclone[21009]: tvshows/Survivor/Season 20/Survivor - S20E01 - Episode.mkv: received cache expiry notification"

If you're monitoring the plex_rcs console, you should see activity:

Starting to monitor /var/log/syslog with pattern for rclone                                                             
Match found: tvshows/Survivor/Season 20/Survivor - S20E01 - Episode.mkv
Processing section 1, folder: /media/tvshows/Survivor/Season 20
GUI: Scanning Survivor/Season 20

TODO

• Support Plexdrive (analyis required)
• Smarter logic to detect rclone cache root/docker media root
• Logging to file

plex_rcs's Issues

PSA - Latest Plex update changes library paths

Just keeping people aware......Plex changes some paths around. I have updated to fix that here:
https://github.com/zhdenny/plex_rcs

permission issues

plexrcs is having issues while running as non plex user:

Starting to monitor /var/log/syslog with pattern for rclone cache
Detected new file: tv/Billions/Season 01/Billions - S01E01 - Pilot.mkv

Processing section 1, folder: /home/omanko/plex/mount/tv/Billions/Season 01
terminate called after throwing an instance of 'std::runtime_error'
  what():  Codecs: Initialize: 'boost::filesystem::create_directory: Permission denied: "/var/lib/plexmediaserver/Plex Media Server/Codecs/4b51335-1293-linux-ubuntu-x86_64"'

Permissions:

plex@THERMALTAKE:~/Library/Application Support/Plex Media Server$ ls -la
total 48
drwxr-xr-x 10 plex plex 4096 May  4 02:16 .
drwxr-xr-x  3 plex plex 4096 Apr  8 14:08 ..
drwxr-xr-x  5 plex plex 4096 May  4 09:19 Cache
drwxr-xr-x  3 plex plex 4096 May  4 01:53 Codecs
drwxr-xr-x  4 plex plex 4096 May  4 01:53 Crash Reports
drwxr-xr-x  3 plex plex 4096 May  4 02:16 Logs
drwxr-xr-x  3 plex plex 4096 Apr  8 14:21 Media
drwxr-xr-x  4 plex plex 4096 Apr 10 19:17 Metadata
-rw-r--r--  1 plex plex    3 May  4 02:15 plexmediaserver.pid
drwxr-xr-x  3 plex plex 4096 Apr 30 05:19 Plug-ins
drwxr-xr-x  7 plex plex 4096 Apr  8 14:08 Plug-in Support
-rw-------  1 plex plex 1498 May  4 02:16 Preferences.xml

Tried logging in as plex but can't screen. Also tried:

plex@THERMALTAKE:~$ script /dev/null
Script started, file is /dev/null
plex@THERMALTAKE:~$ export LD_LIBRARY_PATH="/usr/lib/plexmediaserver" 
plex@THERMALTAKE:~$ /usr/bin/screen -dmS plexrcs /home/omanko/plex/tools/plex_rcs/plex_rcs.py
plex@THERMALTAKE:~$ screen -r plexrcs
There is no screen to be resumed matching plexrcs.
plex@THERMALTAKE:~$ 

Scanned directory 'None' not found in Plex library

I'm not sure what I'm doing wrong, but im running this on centos 7 outside a docker.

When I look at the printout of plex_rcs I see the following error.

Scanned directory 'None' not found in Plex library
Detected new file: 2018/06/02 19:23:39 INFO : tv/How to Get Away with Murder/Season 04/How to Get Away with Murder - S04E14 - The Day Before He Died.mkv: received cache expiry notification

The rclone cache is mount to /media

with subfolders
tv
movies

What can i do to fix the None directory issue.

Crypt Compatible?

Does this work with rclone encrypted mounts?

Create Docker Container

Would be useful for organizing different programs and maintaining updates

Work with vfs cache?

Just curious.....I see this talk about rclone mount cache. Does this work with vfs?

rclone mount
--max-read-ahead 1024k
--allow-other
--dir-cache-time 72h
--buffer-size 64M
--vfs-read-chunk-size 128M
--vfs-read-chunk-size-limit=1G
--uid 99 --gid 100 --default-permissions
--log-level INFO --syslog
tdrive: /mnt/disks/tdrive

Wrong configuration?

Hello,

unfortunately the tool does not work for me. I installed it as follows:

rclone mount Plexcache: /mnt/plexcache --allow-other --cache-db-purge --p oll-interval 1m --log-level DEBUG --log-file /root/plex_rcs/file.log --daemon

plex_rcs:
host: localhost
port: 32400
token: https://support.plex.tv/articles/204059436-finding-an-authentication-token-x-plex-token/
backend: vfs
docker: false
container: plex
media_root: /mnt/plexcache
env:
LD_LIBRARY_PATH: "/usr/lib/plexmediaserver"
PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR: "/var/lib/plexmediaserver"

Configuration with VFS

Would you like to know if the project has been abandoned? I can not configure with VFS and Cache. Settings are as follows:

[Unit]
Description=Mount and cache Google drive to /home/plexuser/mnt/Gdrive
After=syslog.target local-fs.target network.target
[Service]
Environment=RCLONEHOME=/home/plexuser/.config/rclone
Environment=MOUNTTO=/home/plexuser/mnt/Gdrive
Type=simple
User=root
ExecStartPre=/bin/mkdir -p ${MOUNTTO}
ExecStart=/usr/bin/rclone mount \
  --rc \
  --syslog \
  --log-level INFO \
  --fuse-flag sync_read \
  --umask 022 \
  --allow-non-empty \
  --allow-other \
  --tpslimit 10 \
  --tpslimit-burst 10 \
  --dir-cache-time=1m \
  --buffer-size=64M \
  --attr-timeout=1s \
  --vfs-read-chunk-size=5M \
  --vfs-read-chunk-size-limit=2G \
  --vfs-cache-max-age=5m \
  --vfs-cache-mode=writes \
  --config ${RCLONEHOME}/rclone.conf \
  GdriveC: ${MOUNTTO}
ExecStop=/bin/fusermount -u -z ${MOUNTTO}
ExecStop=/bin/rmdir ${MOUNTTO}
Restart=always
[Install]
WantedBy=multi-user.target

Is there something wrong?

Plex Media Server crashes when new file is detected

When plex_rcs detects a new file, my PMS crashed (not running in Docker). Any thoughts? :-)

Starting to monitor /var/log/syslog with pattern for rclone cache
Detected new file: tv/Derek/Derek - S01E03 (1080p BluRay).mkv

Processing section 1, folder: /mnt/media/tv/Derek
terminate called after throwing an instance of 'soci::soci_error'
  what():  sqlite3_statement_backend::prepare: no such table: library_sections for SQL: select library_sections.id as 'library_sections_id', library_sections.library_id as 'library_sections_library_id', library_sections.name as 'library_sections_name', library_sections.name_sort as 'library_sections_name_sort', library_sections.section_type as 'library_sections_section_type', library_sections.language as 'library_sections_language', library_sections.agent as 'library_sections_agent', library_sections.scanner as 'library_sections_scanner', library_sections.user_thumb_url as 'library_sections_user_thumb_url', library_sections.user_art_url as 'library_sections_user_art_url', library_sections.user_theme_music_url as 'library_sections_user_theme_music_url', library_sections.created_at as 'library_sections_created_at', library_sections.updated_at as 'library_sections_updated_at', library_sections.scanned_at as 'library_sections_scanned_at', library_sections.user_fields as 'library_sections_user_fields', library_sections.query_xml as 'library_sections_query_xml', library_sections.query_type as 'library_sections_query_type', library_sections.uuid as 'library_sections_uuid', library_sections.changed_at as 'library_sections_changed_at' from library_sections where id=?
****** PLEX MEDIA SCANNER CRASHED, CRASH REPORT WRITTEN: /var/lib/plexmediaserver/Plex Media Server/Crash Reports/1.13.9.5439-7303bc002/PLEX MEDIA SCANNER/2773c6a1-d3ef-c59d-59f9437f-75f9dd61.dmp
Error in command line:the argument for option '--serverUuid' should follow immediately after the equal sign

Crash Uploader options (all are required):
  --directory arg       Directory to scan for crash reports
  --serverUuid arg      UUID of the server that crashed
  --userId arg          User that owns this product
  --platform arg        Platform string
  --url arg             URL to upload to
  --help                show help message
  --version arg         Version of the product

Traceback (most recent call last):
  File "/opt/plex_rcs/plex_rcs.py", line 152, in <module>
    tailf(lf)
  File "/opt/plex_rcs/plex_rcs.py", line 76, in tailf
    for line in tail("-Fn0", logfile, _iter=True):
  File "/usr/local/lib/python3.6/dist-packages/sh.py", line 857, in next
    chunk = self.process._pipe_queue.get(True, 0.001)
  File "/usr/lib/python3.6/queue.py", line 173, in get
    self.not_empty.wait(remaining)
  File "/usr/lib/python3.6/threading.py", line 299, in wait
    gotit = waiter.acquire(True, timeout)