Scheduled update makes PMS inaccessible about plexupdate

[Proplex]

I also have the same issue -- I have a feeling it might be related to the plemediaserver service not stopping. If I try to run sudo service plexmediaserver stop it doesn't halt.

[kevindd992002]

I don't understand why it's only happening on my NUC though. My two boxes have the same Debian version. And when I do a systemctl stop plexmediaserver.service, the service does halt.

[kevindd992002]

@mrworf are you still maintaining this script? please help.

[mrworf]

It definitely sounds like a permissions issue. Can you provide a ps aux before and after the cron job on the failing machine? Also please share your cron entry just-in-case something's off

[kevindd992002]

How do I do the before and after the cron job? There was a PMS update today (at around 6AM). So If I run ps aux now, is it the before or the after? Please be gentle on me as I'm a Linux noob.

root@nuc:~# cat /etc/plexupdate.cron.conf
CONF='/etc/plexupdate.conf'
SCRIPT='/opt/plexupdate/plexupdate.sh'
LOGGING='true'

root@nuc:~# cat /etc/cron.daily/plexupdate
#!/bin/bash
#
# Script to be placed in one of
#   /etc/cron.daily
#   /etc/cron.weekly
#
# or called directly via /etc/crontab
#
# Do NOT rename it so it has a dot "." in the name, this will cause
# ubuntu (and perhaps other distros) to ignore it.
#
# CONF    is used to point out a configuration file (optional)
# SCRIPT  points out where to find plexupdate.sh
# LOGGING if true, logs all output to syslog daemon facility
#         (typically /var/log/daemon.log or /var/log/syslog)
#

if [ ! -f /etc/plexupdate.cron.conf ]; then
        echo "ERROR: You have not configured /etc/plexupdate.cron.conf" >&2
        exit 255
else
        source /etc/plexupdate.cron.conf
fi

if [ -z "${SCRIPT}" -o ! -f "${SCRIPT}" ]; then
        echo "ERROR: Cannot find plexupdate.sh (tried ${SCRIPT})" >&2
        exit 255
elif [ ${EUID} -eq 0 ]; then
        UNSAFE_FILES=$(find -L "$(dirname "${SCRIPT}")" -perm /002 -or -not -uid 0 -or -not -gid 0)
        if [ ! -z "${UNSAFE_FILES}" ]; then
                echo "ERROR: Permissions on some files are too lax for running as root. Files must be owned by root:root and not world-writeable." >&2
                echo "Unsafe files found:" >&2
                echo "${UNSAFE_FILES}" >&2
                exit 255
        fi
fi

if [ ! -z "$CONF" ]; then
        # We have a config file, prefix it with parameter
        if [ ! -f "${CONF}" ]; then
                echo "ERROR: Cannot find config file (tried ${CONF})" >&2
                exit 255
        fi
        CONF="--config=${CONF}"
fi

LOGFILE=$(mktemp /tmp/plexupdate.cron.XXXX)

RET=0
if $LOGGING; then
        "${SCRIPT}" "${CONF}" 2>&1 | tee ${LOGFILE} | logger -t plexupdate -p daemon.info
        RET="${PIPESTATUS[0]}"
else
        "${SCRIPT}" "${CONF}" >${LOGFILE} 2>&1
        RET=$?
fi

if [ $RET -ne 0 ] ; then
        # Make sure user gets an email about this (when not success or if user has specified when)
        cat ${LOGFILE} >&2

        # Output will produce a cron email, so we can reset the exit status
        if [ $RET -eq 10 ]; then
                RET=0
        fi
fi

rm "${LOGFILE}" 2>/dev/null
exit $RET

[mrworf]

Well, you can manually run /etc/cron.daily/plexupdate as root which should replicate the behavior of cron. If you do that now, will plex seize to respond or it only happens when an actual update happened? (should do nothing when no update is available).

You can also, for kick and giggles, add a cronjob which runs systemctl stop plexmediaserver followed by systemctl start plexmediaserver ... It should result in the same issue, ie, no working plex server. This is a good test to do, because if this doesn't break plex, then we have to look for new ideas. If this also breaks plex then it's something to do with how plex is started.

To create a new cronjob, simply put something like the following in /etc/cron.hourly

#!/bin/bash

echo "plexupdate testing"
systemctl stop plexmediaserver
systemctl start plexmediaserver

You may need to reload cron ( service cron reload) if it doesn't run it within 1h10m. Let me know what happens

[mrworf]

I forgot, make sure your script is executable, ie chmod +x plextesting (assuming it's called plextesting do not add .sh since cron will not run it)

[kevindd992002]

Ok, so I added FORCE='yes' in my plexupdate.conf file so that it forces an update when I try to run manually. Like I said above, running /opt/plexupdate/plexupdate.sh --config /etc/plexupdate.conf does not break plex.

I tried running /etc/cron.daily/plexupdate and it also didn't break it. For the ps aux results, see following:

Before running cronjob:

root@nuc:~# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0 173248 10960 ?        Ss   Jul19   0:16 /sbin/init
root         2  0.0  0.0      0     0 ?        S    Jul19   0:02 [kthreadd]
root         3  0.0  0.0      0     0 ?        I<   Jul19   0:00 [rcu_gp]
root         4  0.0  0.0      0     0 ?        I<   Jul19   0:00 [rcu_par_gp]
root         6  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/0:0H-events_highpri]
root         9  0.0  0.0      0     0 ?        I<   Jul19   0:00 [mm_percpu_wq]
root        10  0.0  0.0      0     0 ?        S    Jul19   0:00 [rcu_tasks_rude_]
root        11  0.0  0.0      0     0 ?        S    Jul19   0:00 [rcu_tasks_trace]
root        12  0.0  0.0      0     0 ?        S    Jul19   0:02 [ksoftirqd/0]
root        13  0.0  0.0      0     0 ?        I    Jul19   6:50 [rcu_sched]
root        14  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/0]
root        15  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/0]
root        16  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/1]
root        17  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/1]
root        18  0.0  0.0      0     0 ?        S    Jul19   0:01 [ksoftirqd/1]
root        20  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/1:0H-events_highpri]
root        21  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/2]
root        22  0.0  0.0      0     0 ?        S    Jul19   0:06 [migration/2]
root        23  0.0  0.0      0     0 ?        S    Jul19   0:00 [ksoftirqd/2]
root        25  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/2:0H-kblockd]
root        26  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/3]
root        27  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/3]
root        28  0.0  0.0      0     0 ?        S    Jul19   0:00 [ksoftirqd/3]
root        30  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/3:0H-events_highpri]
root        31  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/4]
root        32  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/4]
root        33  0.0  0.0      0     0 ?        S    Jul19   0:00 [ksoftirqd/4]
root        35  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/4:0H-events_highpri]
root        36  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/5]
root        37  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/5]
root        38  0.0  0.0      0     0 ?        S    Jul19   0:01 [ksoftirqd/5]
root        40  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/5:0H-kblockd]
root        41  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/6]
root        42  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/6]
root        43  0.0  0.0      0     0 ?        S    Jul19   0:00 [ksoftirqd/6]
root        45  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/6:0H-events_highpri]
root        46  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/7]
root        47  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/7]
root        48  0.0  0.0      0     0 ?        S    Jul19   0:00 [ksoftirqd/7]
root        50  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/7:0H-events_highpri]
root        51  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/8]
root        52  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/8]
root        53  0.0  0.0      0     0 ?        S    Jul19   0:00 [ksoftirqd/8]
root        55  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/8:0H-events_highpri]
root        56  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/9]
root        57  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/9]
root        58  0.0  0.0      0     0 ?        S    Jul19   0:00 [ksoftirqd/9]
root        60  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/9:0H-events_highpri]
root        61  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/10]
root        62  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/10]
root        63  0.0  0.0      0     0 ?        S    Jul19   0:01 [ksoftirqd/10]
root        65  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/10:0H-kblockd]
root        66  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/11]
root        67  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/11]
root        68  0.0  0.0      0     0 ?        S    Jul19   1:23 [ksoftirqd/11]
root        70  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/11:0H-events_highpri]
root        83  0.0  0.0      0     0 ?        S    Jul19   0:00 [kdevtmpfs]
root        84  0.0  0.0      0     0 ?        I<   Jul19   0:00 [netns]
root        85  0.0  0.0      0     0 ?        S    Jul19   0:00 [kauditd]
root        86  0.0  0.0      0     0 ?        S    Jul19   0:01 [khungtaskd]
root        87  0.0  0.0      0     0 ?        S    Jul19   0:00 [oom_reaper]
root        88  0.0  0.0      0     0 ?        I<   Jul19   0:00 [writeback]
root        89  0.0  0.0      0     0 ?        S    Jul19   4:31 [kcompactd0]
root        90  0.0  0.0      0     0 ?        SN   Jul19   0:00 [ksmd]
root        91  0.0  0.0      0     0 ?        SN   Jul19   0:15 [khugepaged]
root       111  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kintegrityd]
root       112  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kblockd]
root       113  0.0  0.0      0     0 ?        I<   Jul19   0:00 [blkcg_punt_bio]
root       115  0.0  0.0      0     0 ?        I<   Jul19   0:00 [edac-poller]
root       116  0.0  0.0      0     0 ?        I<   Jul19   0:00 [devfreq_wq]
root       117  0.0  0.0      0     0 ?        I<   Jul19   0:02 [kworker/6:1H-kblockd]
root       126  0.0  0.0      0     0 ?        S    Jul19   1:52 [kswapd0]
root       127  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kthrotld]
root       128  0.0  0.0      0     0 ?        S    Jul19   0:00 [irq/122-pciehp]
root       129  0.0  0.0      0     0 ?        S    Jul19   0:00 [irq/126-pciehp]
root       130  0.0  0.0      0     0 ?        I<   Jul19   0:00 [acpi_thermal_pm]
root       135  0.0  0.0      0     0 ?        I<   Jul19   0:00 [ipv6_addrconf]
root       144  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kstrp]
root       147  0.0  0.0      0     0 ?        I<   Jul19   0:00 [zswap-shrink]
root       170  0.0  0.0      0     0 ?        I<   Jul19   0:02 [kworker/11:1H-kblockd]
root       211  0.0  0.0      0     0 ?        I<   Jul19   0:00 [sdhci]
root       212  0.0  0.0      0     0 ?        S    Jul19   0:00 [irq/145-mmc0]
root       214  0.0  0.0      0     0 ?        I<   Jul19   0:00 [nvme-wq]
root       215  0.0  0.0      0     0 ?        I<   Jul19   0:00 [nvme-reset-wq]
root       216  0.0  0.0      0     0 ?        I<   Jul19   0:00 [nvme-delete-wq]
root       217  0.0  0.0      0     0 ?        I<   Jul19   0:00 [ata_sff]
root       218  0.0  0.0      0     0 ?        S    Jul19   0:00 [scsi_eh_0]
root       219  0.0  0.0      0     0 ?        I<   Jul19   0:00 [scsi_tmf_0]
root       220  0.0  0.0      0     0 ?        S    Jul19   0:00 [scsi_eh_1]
root       221  0.0  0.0      0     0 ?        I<   Jul19   0:00 [scsi_tmf_1]
root       222  0.0  0.0      0     0 ?        S    Jul19   0:00 [scsi_eh_2]
root       223  0.0  0.0      0     0 ?        I<   Jul19   0:00 [scsi_tmf_2]
root       225  0.0  0.0      0     0 ?        I<   Jul19   0:02 [kworker/4:1H-kblockd]
root       226  0.0  0.0      0     0 ?        I<   Jul19   0:04 [kworker/5:1H-kblockd]
root       228  0.0  0.0      0     0 ?        I<   Jul19   0:02 [kworker/3:1H-kblockd]
root       231  0.0  0.0      0     0 ?        I<   Jul19   0:00 [md]
root       241  0.0  0.0      0     0 ?        I<   Jul19   0:00 [raid5wq]
root       274  0.0  0.0      0     0 ?        I<   Jul19   0:02 [kworker/2:1H-kblockd]
root       277  0.0  0.0      0     0 ?        S    Jul19   0:00 [jbd2/nvme0n1p2-]
root       278  0.0  0.0      0     0 ?        I<   Jul19   0:00 [ext4-rsv-conver]
root       284  0.0  0.0      0     0 ?        I<   Jul19   0:02 [kworker/1:1H-kblockd]
root       311  0.0  0.0      0     0 ?        I<   Jul19   0:01 [kworker/10:1H-kblockd]
root       313  0.0  0.0      0     0 ?        I<   Jul19   0:01 [kworker/0:1H-kblockd]
root       315  0.0  0.0      0     0 ?        I<   Jul19   0:01 [kworker/8:1H-kblockd]
root       321  0.0  0.0      0     0 ?        I<   Jul19   0:01 [kworker/9:1H-kblockd]
root       325  0.0  0.0      0     0 ?        I<   Jul19   0:02 [kworker/7:1H-kblockd]
root       342  0.0  0.0  29768 13804 ?        Ss   Jul19   0:04 /lib/systemd/systemd-journald
root       350  0.0  0.0  22500  4692 ?        Ss   Jul19   0:02 /lib/systemd/systemd-udevd
root       351  0.0  0.0      0     0 ?        I<   Jul19   0:00 [rpciod]
root       352  0.0  0.0      0     0 ?        I<   Jul19   0:00 [xprtiod]
root       354  0.0  0.0   7692   220 ?        Ss   Jul19   0:00 /usr/sbin/blkmapd
root       394  0.0  0.0      0     0 ?        I<   Jul19   0:00 [tpm_dev_wq]
root       435  0.0  0.0      0     0 ?        S    Jul19   0:00 [watchdogd]
root       437  0.0  0.0      0     0 ?        S    Jul19   0:00 [irq/162-mei_me]
root       445  0.0  0.0      0     0 ?        I<   Jul19   0:00 [cryptd]
root       473  0.0  0.0      0     0 ?        S    Jul19   0:05 [jbd2/sda1-8]
root       474  0.0  0.0      0     0 ?        I<   Jul19   0:00 [ext4-rsv-conver]
root       477  0.0  0.0      0     0 ?        S    Jul19   0:35 [jbd2/nvme0n1p3-]
root       478  0.0  0.0      0     0 ?        I<   Jul19   0:00 [ext4-rsv-conver]
root       480  0.0  0.0      0     0 ?        S    Jul19   0:00 [jbd2/nvme0n1p5-]
root       481  0.0  0.0      0     0 ?        I<   Jul19   0:00 [ext4-rsv-conver]
root       484  0.0  0.0      0     0 ?        S    Jul19   0:40 [jbd2/nvme0n1p6-]
root       485  0.0  0.0      0     0 ?        I<   Jul19   0:00 [ext4-rsv-conver]
root       518  0.0  0.0   9084   172 ?        Ss   Jul19   0:00 /usr/sbin/rpc.idmapd
_rpc       565  0.0  0.0   6828  3456 ?        Ss   Jul19   0:02 /sbin/rpcbind -f -w
root       577  0.0  0.0   9492  5292 ?        Ss   Jul19   0:00 /sbin/dhclient -4 -v -i -pf /run/dhclient.eno1.p
root       595  0.0  0.0  19524  6540 ?        Ss   Jul19   0:02 /lib/systemd/systemd-logind
message+   596  0.0  0.0   9984  4784 ?        Ss   Jul19   0:02 /usr/bin/dbus-daemon --system --address=systemd:
root       597  0.0  0.0  19772  4364 ?        Ss   Jul19   0:07 /sbin/wpa_supplicant -u -s -O /run/wpa_supplican
root       598  0.0  0.0 225828  4100 ?        Ssl  Jul19   0:00 /usr/sbin/rsyslogd -n -iNONE
root       600  0.0  0.0  12288  5280 ?        Ss   Jul19   0:00 /usr/sbin/smartd -n
root       601  0.0  0.0 235848  6276 ?        Ssl  Jul19   0:00 /usr/lib/policykit-1/polkitd --no-debug
root       603  0.0  0.0 332796 16584 ?        Ssl  Jul19   0:28 /usr/sbin/NetworkManager --no-daemon
root       604  0.0  0.0 399944 11568 ?        Ssl  Jul19   1:09 /usr/lib/udisks2/udisksd
root       614  0.0  0.0 318952  7928 ?        Ssl  Jul19   0:00 /usr/sbin/ModemManager
root       615  0.0  0.0      0     0 ?        S    Jul19   0:00 [card0-crtc0]
root       616  0.0  0.0      0     0 ?        S    Jul19   0:00 [card0-crtc1]
root       617  0.0  0.0      0     0 ?        S    Jul19   0:00 [card0-crtc2]
root       667  1.1  0.1 1933428 40936 ?       Ssl  Jul19 195:11 /usr/bin/containerd
root       670  0.0  0.0  31764  4432 ?        Ss   Jul19   0:00 /usr/sbin/rpc.mountd --manage-gids
root       672  0.0  0.0  15856  6416 ?        Ss   Jul19   0:00 /usr/sbin/sshd -D
root       675  0.0  0.0      0     0 ?        S    Jul19   0:00 [lockd]
minidlna   696  0.0  0.0 247280  7624 ?        Ssl  Jul19   4:52 /usr/sbin/minidlnad -f /etc/minidlna.conf -P /ru
root       707  0.0  0.0  32340 12052 ?        Ss   Jul19   0:46 /usr/sbin/nmbd --foreground --no-process-group
root       712  0.0  0.2 2131284 83736 ?       Ssl  Jul19  10:25 /usr/bin/dockerd -H fd:// --containerd=/run/cont
root       713  0.0  0.0   6648  1856 ?        Ss   Jul19   0:00 /bin/bash /opt/RoonServer/start.sh
root       727  0.0  0.1 423628 41248 ?        Sl   Jul19   3:13 /opt/RoonServer/RoonMono/bin/RoonServer --debug
root       828  5.0  7.7 8734160 2538712 ?     SLl  Jul19 865:35 /opt/RoonServer/RoonMono/bin/RoonAppliance --deb
root       834  0.0  0.0   3696   968 ?        S    Jul19   0:00 /opt/RoonServer/Server/processreaper 828
root       962  0.0  0.0 111972 11648 ?        Sl   Jul19   2:26 /usr/bin/containerd-shim-runc-v2 -namespace moby
root       995  0.0  0.0   1596   820 ?        Ss   Jul19   0:00 /bin/sh /entry.sh
root      1014  0.0  0.0 113444  8372 ?        Sl   Jul19   2:17 /usr/bin/containerd-shim-runc-v2 -namespace moby
root      1042  0.0  0.0 721748 25256 ?        Ssl  Jul19   2:02 /watchtower
root      1148  0.0  0.0 111972  9916 ?        Sl   Jul19   2:21 /usr/bin/containerd-shim-runc-v2 -namespace moby
root      1177  0.0  0.0      0     0 ?        S    Jul19   0:00 [nfsd]
root      1179  0.0  0.0      0     0 ?        S    Jul19   0:00 [nfsd]
root      1180  0.0  0.0      0     0 ?        S    Jul19   0:00 [nfsd]
root      1181  0.0  0.0      0     0 ?        S    Jul19   0:00 [nfsd]
root      1182  0.0  0.0      0     0 ?        S    Jul19   0:00 [nfsd]
root      1183  0.0  0.0      0     0 ?        S    Jul19   0:00 [nfsd]
root      1184  0.0  0.0      0     0 ?        S    Jul19   0:00 [nfsd]
root      1185  0.0  0.0      0     0 ?        S    Jul19   0:00 [nfsd]
root      1219  0.0  0.0   3656  2260 ?        Ss   Jul19   0:00 /usr/sbin/smartd --debug
root      1298  0.0  0.0   1640   900 ?        S    Jul19   0:03 /usr/sbin/crond -f
root      1737  0.0  0.2 1285640 82284 ?       Sl   Jul19  10:10 /opt/RoonServer/RoonMono/bin/RAATServer --debug
root      1918  0.0  0.0  50428 20176 ?        Ss   Jul19   0:04 /usr/sbin/smbd --foreground --no-process-group
root      1920  0.0  0.0  46676  5908 ?        S    Jul19   0:02 /usr/sbin/smbd --foreground --no-process-group
root      1921  0.0  0.0  46684  5944 ?        S    Jul19   0:02 /usr/sbin/smbd --foreground --no-process-group
root      1922  0.0  0.0  50412  5696 ?        S    Jul19   0:02 /usr/sbin/smbd --foreground --no-process-group
systemd+  2005  0.0  0.0  92868  5432 ?        Ssl  Jul19   0:02 /lib/systemd/systemd-timesyncd
root      2040  0.0  0.0      0     0 ?        I<   Jul19   0:00 [nfsiod]
root      2044  0.0  0.0      0     0 ?        S    Jul19   0:00 [NFSv4 callback]
root      2046  0.0  0.0 1413636 30352 ?       Ssl  Jul19   0:24 /usr/sbin/libvirtd
root      2049  0.0  0.0   8508  2668 ?        Ss   Jul19   0:02 /usr/sbin/cron -f
root      2053  0.0  0.0   5616  1528 tty1     Ss+  Jul19   0:00 /sbin/agetty -o -p -- \u --noclear tty1 linux
Debian-+  2332  0.0  0.0  22720  4508 ?        Ss   Jul19   0:00 /usr/sbin/exim4 -bd -q30m
root      5814  0.0  0.0      0     0 ?        I    Jul30   0:02 [kworker/3:1-mm_percpu_wq]
root      8973  0.0  0.0      0     0 ?        I<   Jul23   0:00 [cifsiod]
root      8974  0.0  0.0      0     0 ?        I<   Jul23   0:00 [smb3decryptd]
root      8975  0.0  0.0      0     0 ?        I<   Jul23   0:00 [cifsfileinfoput]
root      8976  0.0  0.0      0     0 ?        I<   Jul23   0:00 [cifsoplockd]
root      8977  0.0  0.0      0     0 ?        I<   Jul23   0:00 [cifs-dfscache]
root      8981  0.0  0.0      0     0 ?        S    Jul23   0:08 [cifsd]
root      9908  0.0  0.0      0     0 ?        I    Jul30   0:03 [kworker/6:1-events]
root      9965  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/4:0-events]
root     11253  0.0  0.0  50864 14348 ?        S    Jul30   0:00 /usr/sbin/smbd --foreground --no-process-group
root     11424  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/10:0-events_power_efficient]
root     12216  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/3:0]
root     14537  0.0  0.0      0     0 ?        I<   Jul30   0:02 [kworker/u25:0-xprtiod]
root     15057  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/11:2-events]
root     15151  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/8:0-mm_percpu_wq]
root     15376  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/9:0-mm_percpu_wq]
root     15723  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/2:1-events]
root     15783  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/5:2-mm_percpu_wq]
root     17409  0.0  0.0      0     0 ?        I    Jul30   0:02 [kworker/11:1-events]
root     17743  0.0  0.0 475104 11264 ?        Sl   Jul21   0:01 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.
root     17749  0.0  0.0 548836 13552 ?        Sl   Jul21   0:01 /usr/bin/docker-proxy -proto tcp -host-ip :: -ho
root     17756  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/2:2-mm_percpu_wq]
root     17764  0.0  0.0 113124  8688 ?        Sl   Jul21   1:59 /usr/bin/containerd-shim-runc-v2 -namespace moby
root     17785  0.0  0.0 144776 22276 ?        Ssl  Jul21   6:16 /portainer
root     17904  0.0  0.0 548836 11476 ?        Sl   Jul21   0:00 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.
root     17911  0.0  0.0 548836 13504 ?        Sl   Jul21   0:00 /usr/bin/docker-proxy -proto tcp -host-ip :: -ho
root     17926  0.0  0.0 113380 12180 ?        Sl   Jul21   2:01 /usr/bin/containerd-shim-runc-v2 -namespace moby
root     17947  0.0  0.0    204     4 ?        Ss   Jul21   0:00 s6-svscan -t0 /var/run/s6/services
root     17951  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/1:1-events]
root     18019  0.0  0.0    204     4 ?        S    Jul21   0:00 s6-supervise s6-fdholderd
root     18255  0.0  0.0    204     4 ?        S    Jul21   0:00 s6-supervise tautulli
dockeru+ 18258  0.4  0.2 120400 76900 ?        Ssl  Jul21  59:32 python3 /app/tautulli/Tautulli.py --datadir /con
root     18579  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/9:2-pm]
root     19461  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/7:1-mm_percpu_wq]
root     19593  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/5:0-ipv6_addrconf]
root     19924  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/4:1-mm_percpu_wq]
root     21220  0.0  0.0  16640  7780 ?        Ss   Jul30   0:00 sshd: kevindd992002 [priv]
kevindd+ 21223  0.0  0.0  21288  9264 ?        Ss   Jul30   0:00 /lib/systemd/systemd --user
kevindd+ 21224  0.0  0.0 174496  4420 ?        S    Jul30   0:00 (sd-pam)
kevindd+ 21238  0.0  0.0  16924  6016 ?        S    Jul30   0:00 sshd: kevindd992002@pts/0
kevindd+ 21240  0.0  0.0   7656  4404 pts/0    Ss   Jul30   0:00 -bash
root     21243  0.0  0.0  10428  3920 pts/0    S    Jul30   0:00 sudo su -
root     21244  0.0  0.0   9772  3292 pts/0    S    Jul30   0:00 su -
root     21245  0.0  0.0   8640  5388 pts/0    S    Jul30   0:00 -bash
root     21277  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/1:0-events]
root     21449  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/0:1-events]
root     21820  0.0  0.0      0     0 ?        I    00:00   0:00 [kworker/8:1-mm_percpu_wq]
root     21829  0.0  0.0      0     0 ?        I    00:00   0:00 [kworker/7:0-pm]
root     21842  0.0  0.0      0     0 ?        I    00:01   0:00 [kworker/10:2-events_power_efficient]
root     21992  0.0  0.0      0     0 ?        I    00:09   0:00 [kworker/0:2-events]
root     22062  0.0  0.0      0     0 ?        I    00:12   0:00 [kworker/u24:3-rpciod]
root     22178  0.0  0.0      0     0 ?        I<   Jul29   0:19 [kworker/u25:1-xprtiod]
root     22197  0.0  0.0      0     0 ?        I    00:20   0:00 [kworker/6:2-events]
root     22321  0.0  0.0      0     0 ?        I    00:28   0:00 [kworker/u24:1-events_unbound]
root     22400  0.0  0.0      0     0 ?        I    00:32   0:00 [kworker/7:2-events]
root     22409  0.0  0.0      0     0 ?        I    00:33   0:00 [kworker/u24:0-events_unbound]
root     23165  0.0  0.0      0     0 ?        I    00:34   0:00 [kworker/3:2]
root     23202  0.0  0.0      0     0 ?        I    00:34   0:00 [kworker/0:0-events]
root     23203  0.0  0.0      0     0 ?        I    00:34   0:00 [kworker/11:0-events]
root     23285  0.0  0.0      0     0 ?        I    00:34   0:00 [kworker/9:1-pm]
root     23286  0.0  0.0      0     0 ?        I    00:34   0:00 [kworker/9:3-events]
root     23287  0.0  0.0      0     0 ?        I    00:34   0:00 [kworker/9:4-pm]
root     23288  0.0  0.0      0     0 ?        I    00:34   0:00 [kworker/9:5-events]
root     23289  0.0  0.0      0     0 ?        I    00:34   0:00 [kworker/9:6-events]
root     23290  0.0  0.0      0     0 ?        I    00:34   0:00 [kworker/9:7]
root     23318  0.0  0.0      0     0 ?        I    00:34   0:00 [kworker/10:1-events_power_efficient]
root     23365  0.0  0.0      0     0 ?        I    00:35   0:00 [kworker/5:1-mm_percpu_wq]
root     23975  0.0  0.0      0     0 ?        I    00:35   0:00 [kworker/u24:2-ext4-rsv-conversion]
plex     24046  9.4  0.1  99768 61668 ?        Ssl  00:35   0:14 /usr/lib/plexmediaserver/Plex Media Server
plex     24086  1.4  0.1  55056 42404 ?        SNl  00:35   0:02 Plex Plug-in [com.plexapp.system] /usr/lib/plexm
root     24087  0.0  0.0      0     0 ?        I    00:35   0:00 [kworker/8:2]
plex     24124  0.2  0.0  35408 12440 ?        Sl   00:35   0:00 /usr/lib/plexmediaserver/Plex Tuner Service /usr
plex     24166  0.9  0.1  54324 42380 ?        Sl   00:35   0:01 Plex Plug-in [com.plexapp.agents.imdb] /usr/lib/
root     24171  0.0  0.0      0     0 ?        I    00:35   0:00 [kworker/7:3-pm]
root     24172  0.0  0.0      0     0 ?        I    00:35   0:00 [kworker/7:4-pm]
root     24173  0.0  0.0      0     0 ?        I    00:35   0:00 [kworker/7:5-mm_percpu_wq]
root     24174  0.0  0.0      0     0 ?        I    00:35   0:00 [kworker/7:6-events]
root     24175  0.0  0.0      0     0 ?        I    00:35   0:00 [kworker/7:7]
root     24219  0.0  0.0      0     0 ?        I    00:36   0:00 [kworker/4:2-mm_percpu_wq]
root     24275  0.0  0.0  10636  3108 pts/0    R+   00:38   0:00 ps aux

After running cronjob:

root@nuc:~# /etc/cron.daily/plexupdate
Retrieving list of available distributions
Downloading release "plexmediaserver_1.23.6.4881-e2e58f321_amd64.deb"
File downloaded
(Reading database ... 104347 files and directories currently installed.)
Preparing to unpack .../plexmediaserver_1.23.6.4881-e2e58f321_amd64.deb ...
PlexMediaServer install: Pre-installation Validation.
PlexMediaServer install: Pre-installation Validation complete.
Unpacking plexmediaserver (1.23.6.4881-e2e58f321) over (1.23.6.4881-e2e58f321) ...
Setting up plexmediaserver (1.23.6.4881-e2e58f321) ...
PlexMediaServer install: PlexMediaServer-1.23.6.4881-e2e58f321 - Installation starting.
PlexMediaServer install:
PlexMediaServer install: Now installing based on:
PlexMediaServer install:   Installation Type:   Update
PlexMediaServer install:   Process Control:     systemd
PlexMediaServer install:   Plex User:           plex
PlexMediaServer install:   Plex Group:          plex
PlexMediaServer install:   Video Group:         render
PlexMediaServer install:   Metadata Dir:        /home/plex/Library/Application Support
PlexMediaServer install:   Temp Directory:      /home/plex/Transcode
PlexMediaServer install:   Lang Encoding:       en_US.UTF-8
PlexMediaServer install:   Config file used:    /etc/systemd/system/plexmediaserver.service.d/override.conf
PlexMediaServer install:   Intel i915 Hardware: Found
PlexMediaServer install:   Nvidia GPU card:     Not Found
PlexMediaServer install:
PlexMediaServer install:   OpenCL:              Installed
PlexMediaServer install:   Intel Gmmlib:        Installed
PlexMediaServer install:   Intel IGC Core:      Installed
PlexMediaServer install:   Intel IGC OpenCL:    Installed
PlexMediaServer install:   Intel OpenCL:        Installed
PlexMediaServer install:   Intel OCLoc:         Installed
PlexMediaServer install:   Intel Zero GPU:      Installed
PlexMediaServer install:
PlexMediaServer install: Completing final configuration.
PlexMediaServer install: Starting Plex Media Server.
PlexMediaServer install: PlexMediaServer-1.23.6.4881-e2e58f321 - Installation successful.  Errors: 0, Warnings: 0
Processing triggers for mime-support (3.62) ...
Deleted "plexmediaserver_1.23.6.4881-e2e58f321_amd64.deb"
root@nuc:~# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0 173248 10964 ?        Ss   Jul19   0:16 /sbin/init
root         2  0.0  0.0      0     0 ?        S    Jul19   0:02 [kthreadd]
root         3  0.0  0.0      0     0 ?        I<   Jul19   0:00 [rcu_gp]
root         4  0.0  0.0      0     0 ?        I<   Jul19   0:00 [rcu_par_gp]
root         6  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/0:0H-events_highpri]
root         9  0.0  0.0      0     0 ?        I<   Jul19   0:00 [mm_percpu_wq]
root        10  0.0  0.0      0     0 ?        S    Jul19   0:00 [rcu_tasks_rude_]
root        11  0.0  0.0      0     0 ?        S    Jul19   0:00 [rcu_tasks_trace]
root        12  0.0  0.0      0     0 ?        S    Jul19   0:02 [ksoftirqd/0]
root        13  0.0  0.0      0     0 ?        I    Jul19   6:50 [rcu_sched]
root        14  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/0]
root        15  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/0]
root        16  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/1]
root        17  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/1]
root        18  0.0  0.0      0     0 ?        S    Jul19   0:01 [ksoftirqd/1]
root        20  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/1:0H-events_highpri]
root        21  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/2]
root        22  0.0  0.0      0     0 ?        S    Jul19   0:06 [migration/2]
root        23  0.0  0.0      0     0 ?        S    Jul19   0:00 [ksoftirqd/2]
root        25  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/2:0H-kblockd]
root        26  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/3]
root        27  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/3]
root        28  0.0  0.0      0     0 ?        S    Jul19   0:00 [ksoftirqd/3]
root        30  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/3:0H-events_highpri]
root        31  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/4]
root        32  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/4]
root        33  0.0  0.0      0     0 ?        S    Jul19   0:00 [ksoftirqd/4]
root        35  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/4:0H-events_highpri]
root        36  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/5]
root        37  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/5]
root        38  0.0  0.0      0     0 ?        S    Jul19   0:01 [ksoftirqd/5]
root        40  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/5:0H-kblockd]
root        41  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/6]
root        42  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/6]
root        43  0.0  0.0      0     0 ?        S    Jul19   0:00 [ksoftirqd/6]
root        45  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/6:0H-events_highpri]
root        46  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/7]
root        47  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/7]
root        48  0.0  0.0      0     0 ?        S    Jul19   0:00 [ksoftirqd/7]
root        50  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/7:0H-events_highpri]
root        51  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/8]
root        52  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/8]
root        53  0.0  0.0      0     0 ?        S    Jul19   0:00 [ksoftirqd/8]
root        55  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/8:0H-events_highpri]
root        56  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/9]
root        57  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/9]
root        58  0.0  0.0      0     0 ?        S    Jul19   0:00 [ksoftirqd/9]
root        60  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/9:0H-events_highpri]
root        61  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/10]
root        62  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/10]
root        63  0.0  0.0      0     0 ?        S    Jul19   0:01 [ksoftirqd/10]
root        65  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/10:0H-kblockd]
root        66  0.0  0.0      0     0 ?        S    Jul19   0:00 [cpuhp/11]
root        67  0.0  0.0      0     0 ?        S    Jul19   0:07 [migration/11]
root        68  0.0  0.0      0     0 ?        S    Jul19   1:23 [ksoftirqd/11]
root        70  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kworker/11:0H-events_highpri]
root        83  0.0  0.0      0     0 ?        S    Jul19   0:00 [kdevtmpfs]
root        84  0.0  0.0      0     0 ?        I<   Jul19   0:00 [netns]
root        85  0.0  0.0      0     0 ?        S    Jul19   0:00 [kauditd]
root        86  0.0  0.0      0     0 ?        S    Jul19   0:01 [khungtaskd]
root        87  0.0  0.0      0     0 ?        S    Jul19   0:00 [oom_reaper]
root        88  0.0  0.0      0     0 ?        I<   Jul19   0:00 [writeback]
root        89  0.0  0.0      0     0 ?        S    Jul19   4:31 [kcompactd0]
root        90  0.0  0.0      0     0 ?        SN   Jul19   0:00 [ksmd]
root        91  0.0  0.0      0     0 ?        SN   Jul19   0:15 [khugepaged]
root       111  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kintegrityd]
root       112  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kblockd]
root       113  0.0  0.0      0     0 ?        I<   Jul19   0:00 [blkcg_punt_bio]
root       115  0.0  0.0      0     0 ?        I<   Jul19   0:00 [edac-poller]
root       116  0.0  0.0      0     0 ?        I<   Jul19   0:00 [devfreq_wq]
root       117  0.0  0.0      0     0 ?        I<   Jul19   0:02 [kworker/6:1H-kblockd]
root       126  0.0  0.0      0     0 ?        S    Jul19   1:52 [kswapd0]
root       127  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kthrotld]
root       128  0.0  0.0      0     0 ?        S    Jul19   0:00 [irq/122-pciehp]
root       129  0.0  0.0      0     0 ?        S    Jul19   0:00 [irq/126-pciehp]
root       130  0.0  0.0      0     0 ?        I<   Jul19   0:00 [acpi_thermal_pm]
root       135  0.0  0.0      0     0 ?        I<   Jul19   0:00 [ipv6_addrconf]
root       144  0.0  0.0      0     0 ?        I<   Jul19   0:00 [kstrp]
root       147  0.0  0.0      0     0 ?        I<   Jul19   0:00 [zswap-shrink]
root       170  0.0  0.0      0     0 ?        I<   Jul19   0:02 [kworker/11:1H-kblockd]
root       211  0.0  0.0      0     0 ?        I<   Jul19   0:00 [sdhci]
root       212  0.0  0.0      0     0 ?        S    Jul19   0:00 [irq/145-mmc0]
root       214  0.0  0.0      0     0 ?        I<   Jul19   0:00 [nvme-wq]
root       215  0.0  0.0      0     0 ?        I<   Jul19   0:00 [nvme-reset-wq]
root       216  0.0  0.0      0     0 ?        I<   Jul19   0:00 [nvme-delete-wq]
root       217  0.0  0.0      0     0 ?        I<   Jul19   0:00 [ata_sff]
root       218  0.0  0.0      0     0 ?        S    Jul19   0:00 [scsi_eh_0]
root       219  0.0  0.0      0     0 ?        I<   Jul19   0:00 [scsi_tmf_0]
root       220  0.0  0.0      0     0 ?        S    Jul19   0:00 [scsi_eh_1]
root       221  0.0  0.0      0     0 ?        I<   Jul19   0:00 [scsi_tmf_1]
root       222  0.0  0.0      0     0 ?        S    Jul19   0:00 [scsi_eh_2]
root       223  0.0  0.0      0     0 ?        I<   Jul19   0:00 [scsi_tmf_2]
root       225  0.0  0.0      0     0 ?        I<   Jul19   0:02 [kworker/4:1H-kblockd]
root       226  0.0  0.0      0     0 ?        I<   Jul19   0:04 [kworker/5:1H-kblockd]
root       228  0.0  0.0      0     0 ?        I<   Jul19   0:02 [kworker/3:1H-kblockd]
root       231  0.0  0.0      0     0 ?        I<   Jul19   0:00 [md]
root       241  0.0  0.0      0     0 ?        I<   Jul19   0:00 [raid5wq]
root       274  0.0  0.0      0     0 ?        I<   Jul19   0:02 [kworker/2:1H-kblockd]
root       277  0.0  0.0      0     0 ?        S    Jul19   0:00 [jbd2/nvme0n1p2-]
root       278  0.0  0.0      0     0 ?        I<   Jul19   0:00 [ext4-rsv-conver]
root       284  0.0  0.0      0     0 ?        I<   Jul19   0:02 [kworker/1:1H-kblockd]
root       311  0.0  0.0      0     0 ?        I<   Jul19   0:01 [kworker/10:1H-kblockd]
root       313  0.0  0.0      0     0 ?        I<   Jul19   0:01 [kworker/0:1H-kblockd]
root       315  0.0  0.0      0     0 ?        I<   Jul19   0:01 [kworker/8:1H-kblockd]
root       321  0.0  0.0      0     0 ?        I<   Jul19   0:01 [kworker/9:1H-kblockd]
root       325  0.0  0.0      0     0 ?        I<   Jul19   0:02 [kworker/7:1H-kblockd]
root       342  0.0  0.0  37960 14068 ?        Ss   Jul19   0:04 /lib/systemd/systemd-journald
root       350  0.0  0.0  22500  4704 ?        Ss   Jul19   0:02 /lib/systemd/systemd-udevd
root       351  0.0  0.0      0     0 ?        I<   Jul19   0:00 [rpciod]
root       352  0.0  0.0      0     0 ?        I<   Jul19   0:00 [xprtiod]
root       354  0.0  0.0   7692   220 ?        Ss   Jul19   0:00 /usr/sbin/blkmapd
root       394  0.0  0.0      0     0 ?        I<   Jul19   0:00 [tpm_dev_wq]
root       435  0.0  0.0      0     0 ?        S    Jul19   0:00 [watchdogd]
root       437  0.0  0.0      0     0 ?        S    Jul19   0:00 [irq/162-mei_me]
root       445  0.0  0.0      0     0 ?        I<   Jul19   0:00 [cryptd]
root       473  0.0  0.0      0     0 ?        S    Jul19   0:05 [jbd2/sda1-8]
root       474  0.0  0.0      0     0 ?        I<   Jul19   0:00 [ext4-rsv-conver]
root       477  0.0  0.0      0     0 ?        S    Jul19   0:35 [jbd2/nvme0n1p3-]
root       478  0.0  0.0      0     0 ?        I<   Jul19   0:00 [ext4-rsv-conver]
root       480  0.0  0.0      0     0 ?        S    Jul19   0:00 [jbd2/nvme0n1p5-]
root       481  0.0  0.0      0     0 ?        I<   Jul19   0:00 [ext4-rsv-conver]
root       484  0.0  0.0      0     0 ?        S    Jul19   0:40 [jbd2/nvme0n1p6-]
root       485  0.0  0.0      0     0 ?        I<   Jul19   0:00 [ext4-rsv-conver]
root       518  0.0  0.0   9084   172 ?        Ss   Jul19   0:00 /usr/sbin/rpc.idmapd
_rpc       565  0.0  0.0   6828  3456 ?        Ss   Jul19   0:02 /sbin/rpcbind -f -w
root       577  0.0  0.0   9492  5292 ?        Ss   Jul19   0:00 /sbin/dhclient -4 -v -i -pf /run/dhclient.eno1.p
root       595  0.0  0.0  19524  6540 ?        Ss   Jul19   0:02 /lib/systemd/systemd-logind
message+   596  0.0  0.0   9984  4784 ?        Ss   Jul19   0:02 /usr/bin/dbus-daemon --system --address=systemd:
root       597  0.0  0.0  19772  4364 ?        Ss   Jul19   0:07 /sbin/wpa_supplicant -u -s -O /run/wpa_supplican
root       598  0.0  0.0 225828  4100 ?        Ssl  Jul19   0:00 /usr/sbin/rsyslogd -n -iNONE
root       600  0.0  0.0  12288  5280 ?        Ss   Jul19   0:00 /usr/sbin/smartd -n
root       601  0.0  0.0 235848  6276 ?        Ssl  Jul19   0:00 /usr/lib/policykit-1/polkitd --no-debug
root       603  0.0  0.0 332796 16584 ?        Ssl  Jul19   0:28 /usr/sbin/NetworkManager --no-daemon
root       604  0.0  0.0 399944 11596 ?        Ssl  Jul19   1:09 /usr/lib/udisks2/udisksd
root       614  0.0  0.0 318952  7928 ?        Ssl  Jul19   0:00 /usr/sbin/ModemManager
root       615  0.0  0.0      0     0 ?        S    Jul19   0:00 [card0-crtc0]
root       616  0.0  0.0      0     0 ?        S    Jul19   0:00 [card0-crtc1]
root       617  0.0  0.0      0     0 ?        S    Jul19   0:00 [card0-crtc2]
root       667  1.1  0.1 1933428 40808 ?       Ssl  Jul19 195:13 /usr/bin/containerd
root       670  0.0  0.0  31764  4432 ?        Ss   Jul19   0:00 /usr/sbin/rpc.mountd --manage-gids
root       672  0.0  0.0  15856  6416 ?        Ss   Jul19   0:00 /usr/sbin/sshd -D
root       675  0.0  0.0      0     0 ?        S    Jul19   0:00 [lockd]
minidlna   696  0.0  0.0 247280  7624 ?        Ssl  Jul19   4:52 /usr/sbin/minidlnad -f /etc/minidlna.conf -P /ru
root       707  0.0  0.0  32340 12052 ?        Ss   Jul19   0:46 /usr/sbin/nmbd --foreground --no-process-group
root       712  0.0  0.2 2131284 83632 ?       Ssl  Jul19  10:25 /usr/bin/dockerd -H fd:// --containerd=/run/cont
root       713  0.0  0.0   6648  1856 ?        Ss   Jul19   0:00 /bin/bash /opt/RoonServer/start.sh
root       727  0.0  0.1 423628 41248 ?        Sl   Jul19   3:13 /opt/RoonServer/RoonMono/bin/RoonServer --debug
root       828  5.0  7.7 8734176 2536468 ?     SLl  Jul19 865:44 /opt/RoonServer/RoonMono/bin/RoonAppliance --deb
root       834  0.0  0.0   3696   968 ?        S    Jul19   0:00 /opt/RoonServer/Server/processreaper 828
root       962  0.0  0.0 111972 11648 ?        Sl   Jul19   2:26 /usr/bin/containerd-shim-runc-v2 -namespace moby
root       995  0.0  0.0   1596   820 ?        Ss   Jul19   0:00 /bin/sh /entry.sh
root      1014  0.0  0.0 113444  8372 ?        Sl   Jul19   2:17 /usr/bin/containerd-shim-runc-v2 -namespace moby
root      1042  0.0  0.0 721748 25248 ?        Ssl  Jul19   2:02 /watchtower
root      1148  0.0  0.0 111972  9916 ?        Sl   Jul19   2:21 /usr/bin/containerd-shim-runc-v2 -namespace moby
root      1177  0.0  0.0      0     0 ?        S    Jul19   0:00 [nfsd]
root      1179  0.0  0.0      0     0 ?        S    Jul19   0:00 [nfsd]
root      1180  0.0  0.0      0     0 ?        S    Jul19   0:00 [nfsd]
root      1181  0.0  0.0      0     0 ?        S    Jul19   0:00 [nfsd]
root      1182  0.0  0.0      0     0 ?        S    Jul19   0:00 [nfsd]
root      1183  0.0  0.0      0     0 ?        S    Jul19   0:00 [nfsd]
root      1184  0.0  0.0      0     0 ?        S    Jul19   0:00 [nfsd]
root      1185  0.0  0.0      0     0 ?        S    Jul19   0:00 [nfsd]
root      1219  0.0  0.0   3656  2260 ?        Ss   Jul19   0:00 /usr/sbin/smartd --debug
root      1298  0.0  0.0   1640   900 ?        S    Jul19   0:03 /usr/sbin/crond -f
root      1737  0.0  0.2 1285640 82284 ?       Sl   Jul19  10:10 /opt/RoonServer/RoonMono/bin/RAATServer --debug
root      1918  0.0  0.0  50428 20176 ?        Ss   Jul19   0:04 /usr/sbin/smbd --foreground --no-process-group
root      1920  0.0  0.0  46676  5908 ?        S    Jul19   0:02 /usr/sbin/smbd --foreground --no-process-group
root      1921  0.0  0.0  46684  5944 ?        S    Jul19   0:02 /usr/sbin/smbd --foreground --no-process-group
root      1922  0.0  0.0  50412  5696 ?        S    Jul19   0:02 /usr/sbin/smbd --foreground --no-process-group
systemd+  2005  0.0  0.0  92868  5432 ?        Ssl  Jul19   0:02 /lib/systemd/systemd-timesyncd
root      2040  0.0  0.0      0     0 ?        I<   Jul19   0:00 [nfsiod]
root      2044  0.0  0.0      0     0 ?        S    Jul19   0:00 [NFSv4 callback]
root      2046  0.0  0.0 1413636 30340 ?       Ssl  Jul19   0:30 /usr/sbin/libvirtd
root      2049  0.0  0.0   8508  2668 ?        Ss   Jul19   0:02 /usr/sbin/cron -f
root      2053  0.0  0.0   5616  1528 tty1     Ss+  Jul19   0:00 /sbin/agetty -o -p -- \u --noclear tty1 linux
Debian-+  2332  0.0  0.0  22720  4508 ?        Ss   Jul19   0:00 /usr/sbin/exim4 -bd -q30m
root      5814  0.0  0.0      0     0 ?        I    Jul30   0:02 [kworker/3:1-mm_percpu_wq]
root      8973  0.0  0.0      0     0 ?        I<   Jul23   0:00 [cifsiod]
root      8974  0.0  0.0      0     0 ?        I<   Jul23   0:00 [smb3decryptd]
root      8975  0.0  0.0      0     0 ?        I<   Jul23   0:00 [cifsfileinfoput]
root      8976  0.0  0.0      0     0 ?        I<   Jul23   0:00 [cifsoplockd]
root      8977  0.0  0.0      0     0 ?        I<   Jul23   0:00 [cifs-dfscache]
root      8981  0.0  0.0      0     0 ?        S    Jul23   0:08 [cifsd]
root      9908  0.0  0.0      0     0 ?        I    Jul30   0:03 [kworker/6:1-events]
root     11253  0.0  0.0  50864 14348 ?        S    Jul30   0:00 /usr/sbin/smbd --foreground --no-process-group
root     14537  0.0  0.0      0     0 ?        I<   Jul30   0:02 [kworker/u25:0-xprtiod]
root     15151  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/8:0-mm_percpu_wq]
root     15723  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/2:1-events]
root     15783  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/5:2-mm_percpu_wq]
root     17409  0.0  0.0      0     0 ?        I    Jul30   0:02 [kworker/11:1-events]
root     17743  0.0  0.0 475104 11264 ?        Sl   Jul21   0:01 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.
root     17749  0.0  0.0 548836 13552 ?        Sl   Jul21   0:01 /usr/bin/docker-proxy -proto tcp -host-ip :: -ho
root     17756  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/2:2-mm_percpu_wq]
root     17764  0.0  0.0 113124  8688 ?        Sl   Jul21   1:59 /usr/bin/containerd-shim-runc-v2 -namespace moby
root     17785  0.0  0.0 144776 22276 ?        Ssl  Jul21   6:16 /portainer
root     17904  0.0  0.0 548836 11476 ?        Sl   Jul21   0:00 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.
root     17911  0.0  0.0 548836 13504 ?        Sl   Jul21   0:00 /usr/bin/docker-proxy -proto tcp -host-ip :: -ho
root     17926  0.0  0.0 113380 12180 ?        Sl   Jul21   2:01 /usr/bin/containerd-shim-runc-v2 -namespace moby
root     17947  0.0  0.0    204     4 ?        Ss   Jul21   0:00 s6-svscan -t0 /var/run/s6/services
root     17951  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/1:1-events]
root     18019  0.0  0.0    204     4 ?        S    Jul21   0:00 s6-supervise s6-fdholderd
root     18255  0.0  0.0    204     4 ?        S    Jul21   0:00 s6-supervise tautulli
dockeru+ 18258  0.4  0.2 120400 76904 ?        Ssl  Jul21  59:33 python3 /app/tautulli/Tautulli.py --datadir /con
root     19461  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/7:1-pm]
root     19924  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/4:1-mm_percpu_wq]
root     21220  0.0  0.0  16640  7780 ?        Ss   Jul30   0:00 sshd: kevindd992002 [priv]
kevindd+ 21223  0.0  0.0  21288  9264 ?        Ss   Jul30   0:00 /lib/systemd/systemd --user
kevindd+ 21224  0.0  0.0 174496  4420 ?        S    Jul30   0:00 (sd-pam)
kevindd+ 21238  0.0  0.0  16924  6016 ?        S    Jul30   0:00 sshd: kevindd992002@pts/0
kevindd+ 21240  0.0  0.0   7656  4404 pts/0    Ss   Jul30   0:00 -bash
root     21243  0.0  0.0  10428  3792 pts/0    S    Jul30   0:00 sudo su -
root     21244  0.0  0.0   9772  3280 pts/0    S    Jul30   0:00 su -
root     21245  0.0  0.0   8640  5388 pts/0    S    Jul30   0:00 -bash
root     21277  0.0  0.0      0     0 ?        I    Jul30   0:00 [kworker/1:0-mm_percpu_wq]
root     21829  0.0  0.0      0     0 ?        I    00:00   0:00 [kworker/7:0-pm]
root     21842  0.0  0.0      0     0 ?        I    00:01   0:00 [kworker/10:2-events_power_efficient]
root     21992  0.0  0.0      0     0 ?        I    00:09   0:00 [kworker/0:2-events]
root     22062  0.0  0.0      0     0 ?        I    00:12   0:00 [kworker/u24:3-events_unbound]
root     22178  0.0  0.0      0     0 ?        I<   Jul29   0:19 [kworker/u25:1-xprtiod]
root     22197  0.0  0.0      0     0 ?        I    00:20   0:00 [kworker/6:2-events_freezable]
root     22321  0.0  0.0      0     0 ?        I    00:28   0:00 [kworker/u24:1-ext4-rsv-conversion]
root     22400  0.0  0.0      0     0 ?        I    00:32   0:00 [kworker/7:2-pm]
root     22409  0.0  0.0      0     0 ?        I    00:33   0:00 [kworker/u24:0-ext4-rsv-conversion]
root     23165  0.0  0.0      0     0 ?        I    00:34   0:00 [kworker/3:2]
root     23202  0.0  0.0      0     0 ?        I    00:34   0:00 [kworker/0:0-events]
root     23203  0.0  0.0      0     0 ?        I    00:34   0:00 [kworker/11:0-mm_percpu_wq]
root     23285  0.0  0.0      0     0 ?        I    00:34   0:00 [kworker/9:1-pm]
root     23288  0.0  0.0      0     0 ?        I    00:34   0:00 [kworker/9:5-mm_percpu_wq]
root     23318  0.0  0.0      0     0 ?        I    00:34   0:00 [kworker/10:1-pm]
root     23365  0.0  0.0      0     0 ?        I    00:35   0:00 [kworker/5:1-mm_percpu_wq]
root     23975  0.0  0.0      0     0 ?        I    00:35   0:00 [kworker/u24:2-events_unbound]
root     24087  0.0  0.0      0     0 ?        I    00:35   0:00 [kworker/8:2-mm_percpu_wq]
root     24171  0.0  0.0      0     0 ?        I    00:35   0:00 [kworker/7:3-pm]
root     24172  0.0  0.0      0     0 ?        I    00:35   0:00 [kworker/7:4-events]
root     24173  0.0  0.0      0     0 ?        I    00:35   0:00 [kworker/7:5-mm_percpu_wq]
root     24175  0.0  0.0      0     0 ?        I    00:35   0:00 [kworker/7:7-events]
root     24219  0.0  0.0      0     0 ?        I    00:36   0:00 [kworker/4:2-mm_percpu_wq]
root     25014  0.0  0.0      0     0 ?        I    00:39   0:00 [kworker/0:1-events]
root     25165  0.0  0.0      0     0 ?        I    00:40   0:00 [kworker/10:0-pm]
plex     25893  5.0  0.1  97264 56548 ?        Ssl  00:42   0:00 /usr/lib/plexmediaserver/Plex Media Server
plex     25933  7.5  0.1  55436 42712 ?        SNl  00:42   0:01 Plex Plug-in [com.plexapp.system] /usr/lib/plexm
root     25934  0.0  0.0      0     0 ?        I    00:42   0:00 [kworker/8:1]
plex     25971  0.2  0.0  35408 12644 ?        Sl   00:43   0:00 /usr/lib/plexmediaserver/Plex Tuner Service /usr
plex     26015  7.2  0.1  54544 42420 ?        Sl   00:43   0:01 Plex Plug-in [com.plexapp.agents.imdb] /usr/lib/
root     26016  0.0  0.0      0     0 ?        I    00:43   0:00 [kworker/10:3-events_power_efficient]
root     26017  0.0  0.0      0     0 ?        I    00:43   0:00 [kworker/10:4-pm]
root     26018  0.0  0.0      0     0 ?        I    00:43   0:00 [kworker/10:5-events_power_efficient]
root     26019  0.0  0.0      0     0 ?        I    00:43   0:00 [kworker/10:6-events]
root     26020  0.0  0.0      0     0 ?        I    00:43   0:00 [kworker/10:7-events_power_efficient]
root     26061  0.0  0.0  10636  3076 pts/0    R+   00:43   0:00 ps aux

For the hourly cronjob that you want me to test, is it also fine if I have FORCE='yes' in my conf file? Can I run it like the daily cronjob to test? Or should I wait for 1h10m for it to run?

[mrworf]

the hourly job was just to see if we were able to replicate the issue without plexupdate which we were. The problem is now figuring out why. The odd thing here is that cron is running as root and it's able to restart plex and plex is running as a new process, yet it's not accessible.

It's worth looking at the plexmediaserver log to see what it shows on start. If you haven't yet restarted plex, can you locate the logs, zip it and attach to this ticket so we can have a look.

Also, you can edit the /etc/crontab to make the test job to run much sooner than 1h10m for additional tests (cron.hourly was just a easier method but for subsequent testing it will be a pain).

[kevindd992002]

I haven't run the hourly job yet. So I don't know if it breaks plex or not. Do you want me to wait for that 1h10m interval for the hourly job to run or can I just force it by running /etc/cron.hourly/plextesting?

[mrworf]

I misunderstood you last message.

However, you won't be replicating the issue if you force it since you'll be running as root and not cron. If you want to run this faster, edit /etc/crontab and have it run the plextest script at a much earlier time.

We need to see if stop/start via cron is the root cause of this issue. Running the script manually will not replicate the path and we'd be comparing apples to oranges. With script I'm referring to the small script which only stops and then starts plexmediaserver again.

[kevindd992002]

Ok, gotcha. Hold on, let me use /etc/crontab instead and post back.

[kevindd992002]

So I added an entry in crontab -e instead so that the script is ran every 5 minutes:

*/5 * * * * /tmp/plextesting

Waited for the job to run which it did:

root@nuc:~# tail -f /var/log/syslog
Jul 31 01:02:43 nuc crontab[26676]: (root) BEGIN EDIT (root)
Jul 31 01:02:51 nuc crontab[26676]: (root) END EDIT (root)
Jul 31 01:03:05 nuc crontab[26691]: (root) BEGIN EDIT (root)
Jul 31 01:04:44 nuc Plex Media Server[26424]: Generated new chapter thumbnails for Indiana Jones and the Kingdom of the Crystal Skull.
Jul 31 01:06:31 nuc crontab[26691]: (root) REPLACE (root)
Jul 31 01:06:31 nuc crontab[26691]: (root) END EDIT (root)
Jul 31 01:06:36 nuc crontab[26770]: (root) BEGIN EDIT (root)
Jul 31 01:06:49 nuc crontab[26770]: (root) END EDIT (root)
Jul 31 01:07:57 nuc crontab[26810]: (root) BEGIN EDIT (root)
Jul 31 01:08:04 nuc crontab[26810]: (root) END EDIT (root)
Jul 31 01:10:01 nuc CRON[26885]: (root) CMD (/tmp/plextesting)
Jul 31 01:10:01 nuc systemd[1]: Stopping Plex Media Server...
Jul 31 01:10:01 nuc systemd[1]: plexmediaserver.service: Succeeded.
Jul 31 01:10:01 nuc systemd[1]: Stopped Plex Media Server.
Jul 31 01:10:01 nuc systemd[1]: Starting Plex Media Server...
Jul 31 01:10:01 nuc systemd[1]: Started Plex Media Server.

BUT it did not break plex

[mrworf]

Okay, then the next thing is to track down where plex stores its logs and next time it breaks, gather those logs and share here, since it seems to be something with plex itself.
Also, keep in mind that usually a plex server update can take a few minutes until PMS is responding after update.

[kevindd992002]

I'm familiar where the plex logs are but just to confirm, do you want the logs while it's in a hung state? And yes, the plexupdate usually happens early in the morning at around 6AM and I notice plex in this NUC being down hours after that update.

[mrworf]

Nah, it's good to grab the logs before you restart it to get it going again. Should hopefully have clues as to why it's stuck. You might even have some logs now with clues, because I think it keeps a couple of revisions before it erases the logs. As long as you know what time it happened you should be able to figure out which logfiles are relevant.

[kevindd992002]

Ok, I was actually able to reproduce the issue. This is what I put in crontab to intentionally break plex:

*/5 * * * * /etc/cron.daily/plexupdate

I tried */5 * * * * /opt/plexupdate/plexupdate.sh --config /etc/plexupdate.conf earlier and it DID NOT break plex.

So at this point, I think it's a combination of the cron permissions while using the cronwrapper (/opt/plexupdate/extras/cronwrapper)

Here are the relevant plex logs.

Plex Media Server.1.log
Plex Media Server.log

And here are the syslog entries relevant to the update that broke plex:

Jul 31 01:25:02 nuc plexupdate: Retrieving list of available distributions
Jul 31 01:25:03 nuc plexupdate: Downloading release "plexmediaserver_1.23.6.4881-e2e58f321_amd64.deb"
Jul 31 01:25:06 nuc plexupdate: File downloaded
Jul 31 01:25:07 nuc plexupdate: (Reading database ... 104347 files and directories currently installed.)
Jul 31 01:25:07 nuc plexupdate: Preparing to unpack .../plexmediaserver_1.23.6.4881-e2e58f321_amd64.deb ...
Jul 31 01:25:07 nuc systemd[1]: Stopping Plex Media Server...
Jul 31 01:25:07 nuc systemd[1]: plexmediaserver.service: Succeeded.
Jul 31 01:25:07 nuc systemd[1]: Stopped Plex Media Server.
Jul 31 01:25:07 nuc plexupdate: PlexMediaServer install: Pre-installation Validation.
Jul 31 01:25:07 nuc root: PlexMediaServer install: Pre-installation Validation.
Jul 31 01:25:07 nuc plexupdate: PlexMediaServer install: Pre-installation Validation complete.
Jul 31 01:25:07 nuc logger[28754]: PlexMediaServer install: Pre-installation Validation complete.
Jul 31 01:25:07 nuc plexupdate: Unpacking plexmediaserver (1.23.6.4881-e2e58f321) over (1.23.6.4881-e2e58f321) ...
Jul 31 01:25:12 nuc plexupdate: Setting up plexmediaserver (1.23.6.4881-e2e58f321) ...
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install: PlexMediaServer-1.23.6.4881-e2e58f321 - Installation starting.
Jul 31 01:25:12 nuc logger[28939]: PlexMediaServer install: PlexMediaServer-1.23.6.4881-e2e58f321 - Installation starting.
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install: Now installing based on:
Jul 31 01:25:12 nuc logger[28941]: PlexMediaServer install: Now installing based on:
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Installation Type:   Update
Jul 31 01:25:12 nuc logger[28948]: PlexMediaServer install:   Installation Type:   Update
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Process Control:     systemd
Jul 31 01:25:12 nuc logger[28950]: PlexMediaServer install:   Process Control:     systemd
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Plex User:           plex
Jul 31 01:25:12 nuc logger[28952]: PlexMediaServer install:   Plex User:           plex
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Plex Group:          plex
Jul 31 01:25:12 nuc logger[28954]: PlexMediaServer install:   Plex Group:          plex
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Video Group:         render
Jul 31 01:25:12 nuc logger[28956]: PlexMediaServer install:   Video Group:         render
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Metadata Dir:        /home/plex/Library/Application Support
Jul 31 01:25:12 nuc logger[28958]: PlexMediaServer install:   Metadata Dir:        /home/plex/Library/Application Support
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Temp Directory:      /home/plex/Transcode
Jul 31 01:25:12 nuc logger[28960]: PlexMediaServer install:   Temp Directory:      /home/plex/Transcode
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Lang Encoding:       en_US.UTF-8
Jul 31 01:25:12 nuc logger[28962]: PlexMediaServer install:   Lang Encoding:       en_US.UTF-8
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Config file used:    /etc/systemd/system/plexmediaserver.service.d/override.conf
Jul 31 01:25:12 nuc logger[28974]: PlexMediaServer install:   Config file used:    /etc/systemd/system/plexmediaserver.service.d/override.conf
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Intel i915 Hardware: Found
Jul 31 01:25:12 nuc logger[28976]: PlexMediaServer install:   Intel i915 Hardware: Found
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Nvidia GPU card:     Not Found
Jul 31 01:25:12 nuc logger[28978]: PlexMediaServer install:   Nvidia GPU card:     Not Found
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:
Jul 31 01:25:12 nuc logger[28980]: PlexMediaServer install:
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   OpenCL:              Installed
Jul 31 01:25:12 nuc logger[28982]: PlexMediaServer install:   OpenCL:              Installed
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Intel Gmmlib:        Installed
Jul 31 01:25:12 nuc logger[28984]: PlexMediaServer install:   Intel Gmmlib:        Installed
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Intel IGC Core:      Installed
Jul 31 01:25:12 nuc logger[28986]: PlexMediaServer install:   Intel IGC Core:      Installed
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Intel IGC OpenCL:    Installed
Jul 31 01:25:12 nuc logger[28988]: PlexMediaServer install:   Intel IGC OpenCL:    Installed
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Intel OpenCL:        Installed
Jul 31 01:25:12 nuc logger[28990]: PlexMediaServer install:   Intel OpenCL:        Installed
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Intel OCLoc:         Installed
Jul 31 01:25:12 nuc logger[28992]: PlexMediaServer install:   Intel OCLoc:         Installed
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:   Intel Zero GPU:      Installed
Jul 31 01:25:12 nuc logger[28994]: PlexMediaServer install:   Intel Zero GPU:      Installed
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install:
Jul 31 01:25:12 nuc logger[28996]: PlexMediaServer install:
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install: Completing final configuration.
Jul 31 01:25:12 nuc logger[28998]: PlexMediaServer install: Completing final configuration.
Jul 31 01:25:12 nuc systemd[1]: Reloading.
Jul 31 01:25:12 nuc systemd-udevd[29033]: Using default interface naming scheme 'v240'.
Jul 31 01:25:12 nuc systemd[1]: /lib/systemd/system/smbd.service:9: PIDFile= references path below legacy directory /var/run/, updating /var/run/samba/smbd.pid → /run/samba/smbd.pid; please update the unit file accordingly.
Jul 31 01:25:12 nuc systemd[1]: /lib/systemd/system/nmbd.service:9: PIDFile= references path below legacy directory /var/run/, updating /var/run/samba/nmbd.pid → /run/samba/nmbd.pid; please update the unit file accordingly.
Jul 31 01:25:12 nuc systemd[1]: /lib/systemd/system/rpc-statd.service:13: PIDFile= references path below legacy directory /var/run/, updating /var/run/rpc.statd.pid → /run/rpc.statd.pid; please update the unit file accordingly.
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install: Starting Plex Media Server.
Jul 31 01:25:12 nuc logger[29127]: PlexMediaServer install: Starting Plex Media Server.
Jul 31 01:25:12 nuc systemd[1]: Starting Plex Media Server...
Jul 31 01:25:12 nuc systemd[1]: Started Plex Media Server.
Jul 31 01:25:12 nuc plexupdate: PlexMediaServer install: PlexMediaServer-1.23.6.4881-e2e58f321 - Installation successful.  Errors: 0, Warnings: 0
Jul 31 01:25:12 nuc logger[29145]: PlexMediaServer install: PlexMediaServer-1.23.6.4881-e2e58f321 - Installation successful.  Errors: 0, Warnings: 0
Jul 31 01:25:12 nuc plexupdate: Processing triggers for mime-support (3.62) ...
Jul 31 01:25:12 nuc plexupdate: Deleted "plexmediaserver_1.23.6.4881-e2e58f321_amd64.deb"
Jul 31 01:25:14 nuc ModemManager[614]: <info>  [base-manager] couldn't check support for device '/sys/devices/pci0000:00/0000:00:1f.6': not supported by any plugin

[mrworf]

Odd, shouldn't make a difference, but debian can be more finicky than ubuntu.

What if you run cronwrapper from crontab, does that break it too? Just to rule out it isn't because it's running from the cron.daily folder.

[kevindd992002]

Having */5 * * * * /opt/plexupdate/extras/cronwrapper in crontab DOES NOT break plex. So it's only when it's the cron.daily folder that the issue happens.

[mrworf]

What does your crontab file look like, also what's the permission on cronwrapper in cron.daily on the work vs non-working machine?

[kevindd992002]

Crontab only has */5 * * * * /opt/plexupdate/extras/cronwrapper in it, nothing else.

Working machine:

root@epsilon:~# ls -la /etc/cron.daily/plexupdate
lrwxrwxrwx 1 root root 34 Apr 23 02:26 /etc/cron.daily/plexupdate -> /opt/plexupdate/extras/cronwrapper

Non-working machine:

root@nuc:~# ls -la /etc/cron.daily/plexupdate
lrwxrwxrwx 1 root root 34 May 10 09:30 /etc/cron.daily/plexupdate -> /opt/plexupdate/extras/cronwrapper

So yeah, they have the same permissions.

[mrworf]

Really? Your /etc/crontab only has that? Did you delete the rest? Typically a crontab looks more like this

ha@development:~$ cat /etc/crontab 
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# m h dom mon dow user	command
17 *	* * *	root    cd / && run-parts --report /etc/cron.hourly
25 6	* * *	root	test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
47 6	* * 7	root	test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
52 6	1 * *	root	test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )

since it's responsible for triggering all the /etc/cron.* jobs :)

[mrworf]

are these systems running the same distro and are they on the same branch of said distro?

[kevindd992002]

Ahhh, sorry! Like I said, Linux noob :) I was talking about the contents of when I do crontab -e. Here are the contents of my /etc/crontab:

root@nuc:~# cat /etc/crontab
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# Example of job definition:
# .---------------- minute (0 - 59)
# |  .------------- hour (0 - 23)
# |  |  .---------- day of month (1 - 31)
# |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
# |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# |  |  |  |  |
# *  *  *  *  * user-name command to be executed
17 *    * * *   root    cd / && run-parts --report /etc/cron.hourly
25 6    * * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
47 6    * * 7   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
52 6    1 * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
#

Yes, these systems run the same distro and from the same branch.

[mrworf]

No worries :) what's the permission of /opt/plexupdate/extras/cronwrapper on both systems?

Also can you run git show -s in the /opt/plexupdate of each machine? This will tell us what version is running

[kevindd992002]

Working system:

root@epsilon:~# ls -la /opt/plexupdate/extras/cronwrapper
-rwxr-xr-x 1 root root 1881 Apr 23 02:16 /opt/plexupdate/extras/cronwrapper
root@epsilon:~# cd /opt/plex
plex_dupefinder/ plexupdate/
root@epsilon:~# cd /opt/plexupdate
root@epsilon:/opt/plexupdate# git show -s
commit 9a3019223b669b20064d9640cbf62439e95371ee (HEAD -> master, origin/master, origin/HEAD)
Author: Simone Karin Lehmann <[email protected]>
Date:   Wed Jun 9 08:59:04 2021 +0200

    fixed comment about naming scheme on Raspberry Pi

Non-working machine:

root@nuc:~# ls -la /opt/plexupdate/extras/cronwrapper
-rwxr-xr-x 1 root root 1881 May 10 09:28 /opt/plexupdate/extras/cronwrapper
root@nuc:~# cd /opt/plexupdate/
root@nuc:/opt/plexupdate# git show -s
commit 9a3019223b669b20064d9640cbf62439e95371ee (HEAD -> master, origin/master, origin/HEAD)
Author: Simone Karin Lehmann <[email protected]>
Date:   Wed Jun 9 08:59:04 2021 +0200

    fixed comment about naming scheme on Raspberry Pi

[mrworf]

Seems to me like this is more of a debian issue than a specific plexupdate issue. I know that sounds like a cop out :-/ but without a system to work with locally it will be really difficult to diagnose this issue.

I'd make sure you've run apt update on both system (which I don't doubt you have already) and also compare uname -a and lsb_release -a on both just in-case something is different. But if these are identical, I'd post on a debian forum for some guidance. Maybe there's something else interfering with it.

At least it sounds like you've found a workaround that works in the meantime

[kevindd992002]

I understand. What is the purpose of using the cronwrapper anyway? I'll try posting on the debian forums but it would be hard to explain this to them. I expect push back saying that it's the script's fault or something but I guess it's worth trying.

And yes, when I do an apt update && apt upgrade, I do it for both.

Working machine

root@epsilon:~# uname -a
Linux epsilon 5.10.0-0.bpo.7-amd64 #1 SMP Debian 5.10.40-1~bpo10+1 (2021-06-04) x86_64 GNU/Linux
root@epsilon:~# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 10 (buster)
Release:        10
Codename:       buster

Non-working machine

root@nuc:~# uname -a
Linux nuc 5.10.0-0.bpo.7-amd64 #1 SMP Debian 5.10.40-1~bpo10+1 (2021-06-04) x86_64 GNU/Linux
root@nuc:~# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 10 (buster)
Release:        10
Codename:       buster

[mrworf]

It's to allow the plexupdate.sh to live elsewhere and also to handle things like if you want to get an email with the log from a failed run, etc. It also adds some sanity checking for the script you're invoking so it isn't allowed to be edited by non-root people (security).

[kevindd992002]

I see. Do you think reinstalling everything from scratch is worth a try before I resort to posting on debian forums? If so, is there a way to cleanly uninstall your script?

[mrworf]

You could try re-running the installer. It places all under /opt except for a few .conf files, but rerunning it will overwrite those anyway so it should make no difference.

[kevindd992002]

Ok. I'll try. Are scripts in the cron.daily folder ran as root? That seems to be what's in crontab. And crontab entries are ran as cron?

[mrworf]

All the cron.daily stuff is run as root, the crontab specifies the user to run things as

[kevindd992002]

Is /etc/crontab the same as crontab -e? From what I see, they're different files but I don't know how they interact. I never put a user when I put something in crontab -e.

[mrworf]

crontab -e edits your own crontab file which is located under /var/spool/cron/crontabs and then they run as the user's permission. Only /etc/crontab can define who the program should run as

[kevindd992002]

Ahh I see. And since I'm doing that under the root user, then it should be running as root too. This problem is really odd :)

[mrworf]

indeed 😸

[ChuckPa]

I've been thinking about this one.

When developing the new packaging, I found one issue on NVMe SSD NUCs which didn't exist on HDDs or, even SATA-3 SSD NUCs for that matter.

Issue specifically is sockets and the network adapter.

The issue is timing.

1. The commanded stop (signal 15) has had time to flush the com.plexapp.plugins.library.db-wal and shm files back into the DB
2. The sockets themselves have had time to actually close in the kernel after program close()
3. During startup, PMS would launch so quickly, the adapter and route table were not ready.

Both resulted in being unable to open AF_INET sockets for PMS.

NVMe SSD NUCs (i7 class & higher like my NUC8-i7-HVK) using xfs can stop, write out all the data, install the package, and spin back around for a restart BEFORE the kernel is ready to open the sockets again.

During startup, we trip over the WantedBy=multi-user.target launching PMS too fast (which Debian is KNOWN to be too aggressive at network mounts). PMS gets started before the adapter is ready and before the network mounts are ready. Result = PMS failure.

Restarting PMS (which has forced a delay) avoids the problem.

Some of the users have an override now, which I probably can implement globally if there's adequate need,

I run with this as a matter of course because upstream doesn't seem to care.

[chuck@lizum ~.502]$ cat /etc/systemd/system/chuck-nfs.service 
#  Chuck's fix for broken Ubuntu NFS mounts

[Unit]
Description=Mount NFS partitions after system boot
After=network-online.target

[Install]
WantedBy=multi-user.target

[Service]
Type=oneshot
ExecStart=/bin/sh -c 'sleep 15 ; mount -a ; exit 0'
User=root
Group=root
[chuck@lizum ~.503]$ 

How about a "sleep 5" in the process ? ( I had to inject sleep statements on QNAP because my TVS-1282-i7 with SSD is too fast )

[kevindd992002]

Thank you for chiming in as requested, Chuck. So for that sleep workaround, would it need to be implemented in mrworf's plexupdate.sh or cronwrapper script?

@mrworf Chuck is familiar with my issue. I should've mentioned that the non-working NUC machine has an nVME SSD while the working one has a SATA3 SSD.

[kevindd992002]

@mrworf False alarm. I tried running cronwrapper in crontab again and it did break plex. Not sure how I tested yesterday when I said it didn't break plex. So that's not a workaround. There's indeed something going on with cronwrapper on this machine. I hope we can test out @ChuckPa's theory.

[demonbane]

If this is being caused by the systemd issue that @ChuckPa described then there's nothing plexupdate can do to fix it directly unfortunately. We just download the deb and install it, so any changes would have to be made upstream, or you'd have to do a local workaround as described.

@ChuckPa, for your workaround above, I'd suggest using systemd requires to manage this instead. systemd uses systemd-fstab-generator to generate .mount configs for each mount point that can then be referenced in other unit files. So I use an override for plexmediaserver that just Requires my network mount point to ensure that the NFS mount is accessible before PMS starts.

[ChuckPa]

@demonbane

Isn't it a bit inappropriate for me to mandate other dependencies? I'm just the app here. I'm not the system administrator. I am atomic and isolated.

When used normally, system restart (which has the inherent delays between kernel start and PMS start now) everything works.
When typing via command line, everything works.

It's only in this rapid fire Down-Install-Start where it seems vulnerable -- which is discussed here about being distro-dependent.

I'm not in favor of over-engineering something when a simple sleep 5 grants the needed grace time. The update is occurring automatically, probably at night, when everyone is sleeping? Will anyone care if it takes 5 seconds longer for PMS to be relaunched?

[demonbane]

Unfortunately, the Down-Install-Start process is all being done by the deb itself, so we have no control over it. The postinst file in the deb is what manages the restart (look at lines 663-665 in the current package), so there's no way for us to interrupt it.

The closest you could come in that case would be to write your own cron script that waits until the installation is complete and then manually restart PMS just in case. If you don't care about any of the logging or other security things that cronwrapper does, you could replace it with something as simple as:

#!/usr/bin/env bash
/opt/plexupdate/plexupdate.sh
sleep 5
systemctl restart plexmediaserver

@ChuckPa my comment about the mounts was specifically for the workaround you're using on your system, not regarding this issue specifically. I'm happy to clarify more about how it works and why I think it's a good solution elsewhere if you're interested, I just don't want to confuse this thread with that.

[ChuckPa]

Thanks. sorry about misreading that.
For me, a simple sleep works. I bounce through distros and VMs at a fever pace. What I have makes it easy to "drop a tarball" , add a bunch of lines to /etc/fstab, and magically everything is configured for this environment (which includes all that Plex needs)

[demonbane]

@ChuckPa Since I was already thinking about it I decided to write up a wiki guide about how to use this setup that also has some background on how it works.

[kevindd992002]

Does starting PMS before the NFS mounts really that important? I had the same issue before and PMS starts properly even if the NFS mount failed. My fix to do that was this:

root@nuc:~# cat /etc/systemd/system/mnt-data.mount
[Unit]
Description=Mount Synology data NFS Share
Requires=network-online.service nss-lookup.target
After=network-online.service nss-lookup.target

[Mount]
What=synology.home.arpa:/volume1/data
Where=/mnt/data
Type=nfs
Options=vers=4,noauto
# Uncomment the below if your server is real slow
# TimeoutSec=10

[Install]
WantedBy=multi-user.target

But I don't that has a direct correlation with the issue at hand here, right? When the NFS mount is inaccessible, PMS just shows that all media is inaccessible. And you can still restart the PMS service either way.

@demonbane , I do care about the cronwrapper "features". @ChuckPa so are we in agreement to put that extra sleep line in the PMS code or something?

[demonbane]

Correct, NFS mounts aren't related at all to this particular issue.

Unfortunately, unless you're going to unpack and repack each deb release of PMS manually, you can't add the extra sleep line on that end. Since you do want to use the other cronwrapper features, the next step is to copy cronwrapper into /etc/cron.daily/ (and remove the linked version that the installer creates for you).

In theory, we want cronwrapper to be linked so that if we make any changes you'll get them next time the script runs, but in practice, that script hasn't been updated since 2017, so you should be fine with a copy.

I'd recommend adding your code in at line 57 of cronwrapper (right between the two if/thens). You can then try something as simple as:

sleep 10
systemctl restart plexmediaserver

[kevindd992002]

Instead of copying cronwrapper, can't I just edit cronwrapper directly and keep the linked file that the installer puts in /etc/cron.daily/?

@ChuckPa Also, if the problem is because of how fast an nVME SSD is, why are there no issues when running plexupdate manually (without using cronwrapper)? If the issue is timing, then I would expect that even with a manual update using the deb file, it will also make PMS inaccessible since the restart of PMS is being triggered by the deb package reinstallation itself, no?

[demonbane]

If you edit the original it will prevent the auto-update for plexupdate from working (the feature that checks for new versions of plexupdate itself).

[kevindd992002]

I see. Ok, that workaround seems to have worked. Initially, I tried just adding the service restart and it did not work. Adding the sleep solved it. At this point, I think it's a deb package issue that @ChuckPa should take a look at. I agree that the sleep needs to be incorporated in the package itself.

[ChuckPa]

Question:

Should I put a sleep in the package at removal or installation?
Should I make everyone wait 5 seconds?
Is that fair to those not using this script?

This is your automation. Respectfully submit that you need manage how it interacts with the host.

Solution would be to put the sleep in THIS script and forget it.

[kevindd992002]

@ChuckPa ok, now you're making more sense. Sorry, I will let the devs of this script and you communicate as I'm obviously a Linux noob and my opinion is of less importance as to where the sleep is really needed.

[ChuckPa]

You folks have communicated the issue.
It's been shown that a sleep between stop and restart is needed (highly host performance dependent).
In all regards, how the script devs wish it resolved is their decision.

I'm stepping out of this thread. I can be reached in the forums if needed further.

[demonbane]

We don't handle the stop and restart unfortunately. That's done entirely in the deb package's postinst which we have no control over.

For the time being, I think you'll just have to stick with the workaround of modifying the cronwrapper for yourself. If more people start running into this in the future it will have to be taken up with upstream so they can troubleshoot further. (While adding a sleep to the postinst would fix it in this case, this seems like a clear case of a race condition in the interactions between Plex and systemd and will end up needing to be addressed there.)

[kevindd992002]

When you say the deb package's postinst, you mean the plex package installer itself, right? Chuck is one of the Plex devs.

[demonbane]

Yes, the package that's distributed directly by the Plex team. To Chuck's point, however, I agree that adding a sleep into the postinst isn't the correct way to fix this because it will impact everyone. I believe this is a race condition that is at least partly (if not wholly) involving systemd itself, but so far we have a sample size of 1 unfortunately.

If this is indeed something that's specific to next-gen NVMe drives I expect we'll see a lot more reports of things like this not just for Plex but for lots of other things as well at some point in the future. You just happen to be one of the first to be running into it.

[kevindd992002]

Fair point. I'll stick with the workaround for now and hope that nothing negative happens to my PMS in the long run when the update process is done like this.

[demonbane]

I can't imagine there being any ill effects. PMS is pretty resilient about restarts, and this one is a graceful restart so I'd be surprised if you run into any problems. Closing this issue.

[kevindd992002]

Correct, NFS mounts aren't related at all to this particular issue.

Unfortunately, unless you're going to unpack and repack each deb release of PMS manually, you can't add the extra sleep line on that end. Since you do want to use the other cronwrapper features, the next step is to copy cronwrapper into /etc/cron.daily/ (and remove the linked version that the installer creates for you).

In theory, we want cronwrapper to be linked so that if we make any changes you'll get them next time the script runs, but in practice, that script hasn't been updated since 2017, so you should be fine with a copy.

I'd recommend adding your code in at line 57 of cronwrapper (right between the two if/thens). You can then try something as simple as:

sleep 10
systemctl restart plexmediaserver

With this workaround, will plexupdate still update itself?

[demonbane]

Yes it will. The cronwrapper won't, but there's very little chance of it being updated anytime soon. So just copy cronwrapper, edit the copy, and you should be good to go.

[kevindd992002]

Yes it will. The cronwrapper won't, but there's very little chance of it being updated anytime soon. So just copy cronwrapper, edit the copy, and you should be good to go.

I see. Yup, that's what I did since the time we were discussing this and I just wondered if the plexupdate.sh script gets updated as it was recently just updated in the repository.