reenignearcher / web2tv

web2tv

This project contains python scripts which load and parse tv guide information and return the information in xml format (http://wiki.xmltv.org/index.php/XMLTVFormat) (https://github.com/XMLTV/xmltv/blob/master/xmltv.dtd). You can then use the xml in various other programs such as xTeVe (https://github.com/xteve-project/xTeVe) or other iptv clients that allow epg in xml format.

Additionally there are scripts which generate m3u lists as well as some helper scripts.

---

EPG Generation

• plex.tv (plex.py)
• pluto.tv (plutotv.py)

---

M3U Generation

• NextPVR (nextpvr.py)
• plex.tv (plex.py)
• pluto.tv (plutotv.py)
• m3u (convert m3u for use with Streamlink)

---

Helper Scripts

• Plex DVR (update_plexDVR.py)

All scripts were tested using Python 3.8

m3u_modder

description="Python script to convert m3u for use with streamlink."

'-i', '--inFile', type=str, nargs=1, required=False, default=['streamlink.m3u'], help='Full input file filepath. Full file path can be specified. If only file name is specified then file will be used from the current working directory if it exists.'

'-o', '--outFile', type=str, nargs=1, required=False, default=['streamlink.m3u'], help='Full destination filepath. Full file path can be specified. If only file name is specified then file will be placed in the current working directory.'

'-p', '--protocol', type=str, nargs=1, required=False, default=['httpstream://'], help='Stream url protocol.'

nextpvr

description="Python script to convert pluto tv channels into m3u format."

-f', '--file', type=str, nargs=1, required=False, default=['nextpvr.m3u'], help='Full destination filepath. Default is nextpvr.m3u. Full file path can be specified. If only file name is specified then file will be placed in the current working directory.'

'-p', '--prefix', type=str, nargs=1, required=False, default=[''], help='Channel name prefix.'

'-s', '--startNumber', type=int, nargs=1, required=False, default=[1], help='Start numbering here. For example 9000. If -k, --keepNumber is used then channel 2 would become channel 9002, otherwise the first channel number found would be 9000, second channel found would be 9001, etc.'

'-k', '--keepNumber', action='store_true', required=False, help='Keep existing number scheme. Script will add existing number to start number. Recommended start number ends with a 0.'

'-i', '--ip', type=str, nargs=1, required=False, default=['127.0.0.1'], help='IP Address of NextPVR server. Default is 127.0.0.1'

'--port', type=int, nargs=1, required=False, default=[8866], help='Port number of NextPVR server. Default is 8866.'

'--pin', type=str, nargs=1, required=False, default=['0000'], help='Pin used to access NextPVR api. Default is 0000.'

'--streamlink', action='store_true', required=False, help='Generate the stream urls for use with Streamlink.'

plex

description="Python script to convert plex livetv guide into xml/m3u format."

'-t', '--token', type=str, nargs=1, required=True, help='Token is required. Follow Plex instructions for finding the token. https://support.plex.tv/articles/204059436-finding-an-authentication-token-x-plex-token/#toc-0')

'-d', '--days', type=int, nargs=1, required=False, default=[7], help='Days of info to collect. Max if 21.'

'-p', '--pastdays', type=int, nargs=1, required=False, default=[0], help='Days in past of info to collect. Max is 1.'

'-l', '--language', type=str, nargs=1, required=False, default=['en'], help='Plex language... Get from url same as token.'

#xml arguments

'-x', '--xmlFile', type=str, nargs=1, required=False, default=['plex2.xml'], help='Full destination filepath for xml. Full file path can be specified. If only file name is specified then file will be placed in the current working directory.'

'--xml', action='store_true', required=False, help='Generate the xml file.'

#m3u arguments

'-m', '--m3uFile', type=str, nargs=1, required=False, default=['plex2.m3u'], help='Full destination filepath for m3u. Full file path can be specified. If only file name is specified then file will be placed in the current working directory.'

'--prefix', type=str, nargs=1, required=False, default=[''], help='Channel name prefix.'

'-s', '--startNumber', type=int, nargs=1, required=False, default=[1], help='Start numbering here. For example 9000. If -k, --keepNumber is used then channel 2 would become channel 9002, otherwise the first channel number found would be 9000, second channel found would be 9001, etc.'

'-k', '--keepNumber', action='store_true', required=False, help='Keep existing number scheme. Script will add existing number to start number. Recommended start number ends with a 0.'

'--m3u', action='store_true', required=False, help='Generate the m3u file.'

'--streamlink', action='store_true', required=False, help='Generate the stream urls for use with Streamlink.'

plutotv

description="Python script to convert pluto tv guide into xml/m3u format."

'-e', '--epgHours', type=int, nargs=1, required=False, default=[10], help='Hours of EPG to collect. Pluto.TV only provides a few hours of EPG. Max allowed is 12.'

#xml arguments

'-x', '--xmlFile', type=str, nargs=1, required=False, default=['plutotv.xml'], help='Full destination filepath. Full file path can be specified. If only file name is specified then file will be placed in the current working directory.'

'--xml', action='store_true', required=False, help='Generate the xml file.'

#m3u arguments

'-m', '--m3uFile', type=str, nargs=1, required=False, default=['plutotv.m3u'], help='Full destination filepath. Full file path can be specified. If only file name is specified then file will be placed in the current working directory.'

'-p', '--prefix', type=str, nargs=1, required=False, default=[''], help='Channel name prefix.'

'-s', '--startNumber', type=int, nargs=1, required=False, default=[1], help='Start numbering here. For example 9000. If -k, --keepNumber is used then channel 2 would become channel 9002, otherwise the first channel number found would be 9000, second channel found would be 9001, etc.'

'-k', '--keepNumber', action='store_true', required=False, help='Keep existing number scheme. Script will add existing number to start number. Recommended start number ends with a 0.'

'--m3u', action='store_true', required=False, help='Generate the m3u file.'

'--streamlink', action='store_true', required=False, help='Generate the stream urls for use with Streamlink.'

update_plexDVR

description="Python script to refresh Plex DVR guide(s)."

'-u', '--uri', type=str, nargs=1, required=False, default=['http://127.0.0.1:32400'], help='Uri to access plex.'

'-t', '--token', type=str, nargs=1, required=True, default=[''], help='Plex server token'

web2tv's Issues

m3u_modder.py does not work

Describe the Bug

my settings look like this

however, these settings only work for the plutotv.py script.

the m3u_modder.py does not create a playable m3u for converting ersatz tv m3us to be played in streamlink through xteve to emby or plex. only the plutotv.py script is working that way.

Code Snippets

C:\cygwin64\home\user\xteve2>python C:\cygwin64\home\user\xteve2\web2tv-main\m3u_modder.py -i http://localhost:8409/iptv/channels.m3u -o C:\cygwin64\home\user\xteve2\streamlink.m3u

Expected Behavior

No response

Additional Context

No response

Operating System and Version

windows 10

Python Version

Python 3.7.9

Feature request: get posters from TheMovieDB for xml epg

https://www.themoviedb.org/documentation/api

plex.py not working

Describe the Bug

plex.py started today not working
attached is error log from run
plexrunlog.txt

Code Snippets

''' Traceback (most recent call last):
File "/storage/emulated/0/IPTV/web2tv/plex.py", line 3, in
import requests_cache
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/requests_cache/init.py", line 43, in
from .backends import *
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/requests_cache/backends/init.py", line 8, in
from .base import BaseCache, BaseStorage
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/requests_cache/backends/base.py", line 18, in
from ..serializers import init_serializer
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/requests_cache/serializers/init.py", line 6, in
from .preconf import (
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/requests_cache/serializers/preconf.py", line 26, in
base_stage = CattrStage() #: Base stage for all serializer pipelines
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/requests_cache/serializers/cattrs.py", line 32, in init
self.converter = init_converter(factory)
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/requests_cache/serializers/cattrs.py", line 67, in init_converter
converter.register_structure_hook(
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/cattr/converters.py", line 263, in register_structure_hook
self._structure_func.register_cls_list([(cl, func)])
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/cattr/dispatch.py", line 57, in register_cls_list
self._single_dispatch.register(cls, handler)
File "/data/data/com.termux/files/usr/lib/python3.10/functools.py", line 856, in register
raise TypeError(
TypeError: Invalid first argument to register(). ForwardRef('CachedResponse') is not a class.
'''

Expected Behavior

No response

Additional Context

No response

Operating System and Version

termux android

Python Version

3.10.2

the mod m3u doesnt convert anything to a playable format for streamlink

Describe the Bug

i have converted an m3u but it does not play in streamlink

here is the command
C:\cygwin64\home\user\xteve2>python C:\cygwin64\home\user\xteve2\web2tv-main\m3u_modder.py -i http://localhost:8409/iptv/channels.m3u -o C:\cygwin64\home\user\xteve2\streamlink.m3u

this is the output

#EXTM3U url-tvg="http://localhost:8409/iptv/xmltv.xml" x-tvg-url="http://localhost:8409/iptv/xmltv.xml"

#EXTINF:0 tvg-id="68.etv" channel-id="vP98rrCWT0O1gnFUt7IXRQ" channel-number="68" CUID="vP98rrCWT0O1gnFUt7IXRQ" tvg-chno="68" tvg-name="24/7 christmas favorites 3" tvg-logo="http://localhost:8409/iptv/logos/2CC550640213402C609F44B333CA521E.jpg" group-title="ErsatzTV" tvc-stream-vcodec="h264" tvc-stream-acodec="aac", 24/7 christmas favorites 3

httpstream://http://localhost:8409/iptv/channel/68.ts

Operating System and Version

windows 10

Python Version

Python 3.7.9

plex2xml improvement: get guide as json

Import plex guide data as json. Will make it much easier to parse and more consistent.

invalid syntax with path

Describe the Bug

invalid syntax in path

Code Snippets

>>> m3u_modder.py -i C:\cygwin64\home\user\PlutoIPTV\playlist.m3u8 -o C:\cygwin64\home\user\xteve2\PlutoXteveStreamlink.m3u
  File "<stdin>", line 1
    m3u_modder.py -i C:\cygwin64\home\user\PlutoIPTV\playlist.m3u8 -o C:\cygwin64\home\user\xteve2\PlutoXteveStreamlink.m3u
                     ^
SyntaxError: invalid syntax

Expected Behavior

No response

Additional Context

No response

Operating System and Version

windows 10

Python Version

Python 3.7.9

issue with getting playlist

Hello
im getting this error what should i do to fix ?

root@Ubuntu-1804-bionic-64-minimal ~/web2tv-main # python plutotv.py
hours: 10
offset: -0000
timezone: -0000
xmlFile: plutotv.xml
m3uFile: plutotv.m3u
prefix:
startNumber: 1
keepNumber: False
streamlink: False
xml: False
m3u: False
Loading Grid for PlutoTV
url: https://service-channels.clusters.pluto.tv/v1/guide?start=2021-02-17T23:00:00-00:00&stop=2021-02-18T09:30:00-00:00
Traceback (most recent call last):
File "plutotv.py", line 219, in
'channelName': fix(grid['channels'][x]['name']), #name
File "plutotv.py", line 37, in fix
text = str(html.escape(text, quote=False).encode('ascii', 'xmlcharrefreplace'))[2:-1] #https://stackoverflow.com/a/1061702/11214013
AttributeError: 'module' object has no attribute 'escape'

Trying to startup "ubuntu-in-termux"

I tried starting it but it's.Showing error (unknown architecture)

Move common functions to separate python file

Move common functions to separate python file

feature to use blank protocol

Describe the feature

my m3u already has http:// prepended but your modm3u script is forcefully adding another prefix so it looks like this

httpstream://http://localhost:8409/iptv/channel/73.ts

Desired solution

my url output should just look like this

http://localhost:8409/iptv/channel/73.ts

Additional Context

No response

Add zap2it

Example url: https://tvlistings.zap2it.com/api/grid?lineupId=USA-DITV-DEFAULT&timespan=6&headendId=DITV&country=USA&timezone=&device=-&postalCode=28278&isOverride=true&time=1609171200&pref=16%2C256&userId=-&aid=gapzap&languagecode=en

This could be of use... https://github.com/edit4ever/script.module.zap2epg

Also this: https://web.archive.org/web/20200426004001/http://zap2xml.awardspace.info/

pluto tv 80s rewind channel freezing

Describe the Bug

only the pluto tv 80s Rewind channel is freezing for me every time it goes to commercial. this is the reason i switched from the old pluto tv script to use your pluto tv script to be used with xteve and streamlink.
would you happen to know what i can try? the other channels make it through the commercials fine though.

Code Snippets

No response

Expected Behavior

the 80s channel should transition through the commercial and then continue playing the show

Additional Context

it seems to happen exactly before a commercial starts, is when it freezes forever. i never make it to the pluto tv screen saver commercial. but the channel plays fine up until the commercial.

also if i stop the channel and start it again, the channel plays fine up until the commercial again, then freezes again.

Operating System and Version

windows 10, regual android 9

Python Version

Python 3.7.9

http-user-agent=xteve could not be found.

Describe the Bug

when i run the mod m3u in xteve through emby, xteve shows this error and will not start the stream

Code Snippets

VLC log:                error: The specified stream(s) ':http-user-agent=xteve' could not be found.

Expected Behavior

No response

Additional Context

No response

Operating System and Version

windows 10

Python Version

Python 3.7.9

getting error running plex.py when creating xml

getting error running plex.py when creating xml (using branch not main)

creating m3u is fine

below is top and bottom of run upto error received I XXXX my token

$ python3 plex.py -t=XXXXXXXXXXXXXXXXXX -d=1 -x="plex2us.xml" --xml

Loading Grid for PlexTV, day: 1/1
url[1]: https://epg.provider.plex.tv/grid?type=1%2C4&sort=beginsAt&endsAt%3E=1640962800&beginsAt%3C=1641049200&X-Plex-Token=XXXXXXXXXXXX&X-Plex-Language=en
Unknown Title
Unknown Title
Unknown Title
Unknown Title
Unknown Title
Unknown Title
Unknown Title
Unknown Title
Unknown Title
Unknown Title
Unknown Title
Unknown Title
Unknown Title
Journy's Bourdain All Day
Journy's Bourdain All Day
Journy's Bourdain All Day
Journy's Bourdain All Day
Journy's Bourdain All Day
Journy's Bourdain All Day
Journy's Bourdain All Day
Journy's Bourdain All Day
Journy's Bourdain All Day
Journy's Bourdain All Day
Journy's Bourdain All Day
Journy's Bourdain All Day
Journy's Bourdain All Day
Journy's Bourdain All Day
..
..
Best TV Ever ID
Return of the Androids
Lure of Light
Freezing Torture
The Destroying Ray
Traceback (most recent call last):
File "/storage/emulated/0/IPTV/web2tv-20210822/plex.py", line 543, in
main()
File "/storage/emulated/0/IPTV/web2tv-20210822/plex.py", line 438, in main
xmltv_ns = f'{xmltv_ns}.{int(program[number]) -1}.'
ValueError: invalid literal for int() with base 10: '0-1'

add roku tv channels

Describe the feature

please can you add rokuTV channels

[plex2xml] Get actual program categories

Script currently selects category based on channel number. One issue with this is plex seems to change the channel numbers. The next issue is the categories defined in the script are very generic. If we pull them from plex the category information will be more accurate.

An example url to get categories/genres/tags from is below.

https://epg.provider.plex.tv/library/metadata/?includeConcerts=1&includeExtras=1&includeOnDeck=1&includePopularLeaves=1&includePreferences=1&includeReviews=1&includeChapters=1&includeStations=1&includeExternalMedia=1&asyncAugmentMetadata=1&asyncCheckFiles=1&asyncRefreshAnalysis=1&asyncRefreshLocalMediaAgent=1&X-Plex-Token=X-Plex-Language=

This will create a significant amount of request to the plex's epg provider, so need to determine a way to cache the results.

plutotv.py error

Describe the Bug

the m3u and epg is not being generated at all

Code Snippets

C:\cygwin64\home\user\xteve2\web2tv-main>python plutotv.py -e 12 -x C:\cygwin64\home\user\xteve2\streamlinkPlutoEpg.xml -m C:\cygwin64\home\user\xteve2\streamlinkPlutoM3U.m3u --streamlink
Loading Grid for PlutoTV
url: https://service-channels.clusters.pluto.tv/v1/guide?start=2022-02-13T04:30:00-00:00&stop=2022-02-13T17:00:00-00:00

Expected Behavior

No response

Additional Context

No response

Operating System and Version

windows 10

Python Version

Python 3.7.9

Add ability to update plex DVRs and channel mappings

Need help with this one. Why am I getting a 404 error when calling the url to refresh the guide?

web2tv/update_plexDVR.py

Line 84 in d77bbe2

r = post_json(url)

Similar javascript example here.
https://github.com/vexorian/dizquetv/blob/f428dbecf0c4fd51f9fba1a91fa98eaaae7bbdb8/src/plex.js#L156

Add Roku

Could be challenging to do this...

Roku provides/displays only information about currently airing content.
Additionally content is loaded/rendered on demand, instead of preloaded. I don't currently know how to handle this, but this link may help. (https://dvenkatsagar.github.io/tutorials/python/2015/10/26/ddlv/)

The following url is to see all live tv. Is the string between browse/ and /see-all a token?
https://therokuchannel.roku.com/browse/w.J1mzrama89hRP9GM2D52iGe2LZZL6lHK5xJdY7PRUbJ42N6416hpR8PeN/see-all

As images are loaded on this page they appear under network inspection tools. An example image url is the following... how to get these urls as they are not part of the page source?
https://images.sr.roku.com/idType/roku/context/trc/id/3b2d767e3df253609c04396f87eb3cd6/images/titan/assets/aHR0cHM6Ly9zdGF0aWMtcy1jZG4ueHVtby5jb20vcm9rdV9hc3NldHMvVFJDJTIwL2JyYW5kVGlsZV9UUkNfdm95YWdlci5wbmc.png/presets/29eb2b5a1e673afec27e5a058a577573

If we parse the trc/id and add it to the following string we get a url that gives us json data!
https://therokuchannel.roku.com/api/v2/homescreen/content/https%3A%2F%2Fcontent.sr.roku.com%2Fcontent%2Fv1%2Froku-trc%2F

example:
https://therokuchannel.roku.com/api/v2/homescreen/content/https%3A%2F%2Fcontent.sr.roku.com%2Fcontent%2Fv1%2Froku-trc%2F3b2d767e3df253609c04396f87eb3cd6

Plutotv.py "encoding" error on m3u write attempt

All requirements installed.

When attempting to write m3u file plutotv.py generates "encoding" error. However --xml argument works fine.

$ python ./plutotv.py --m3u

hours: 10
offset: -0000
timezone: -0000
xmlFile: plutotv.xml
m3uFile: plutotv.m3u
prefix:
startNumber: 1
keepNumber: False
streamlink: False
xml: False
m3u: True
Loading Grid for PlutoTV
url: https://service-channels.clusters.pluto.tv/v1/guide?start=2021-04-17T12:00:00-00:00&stop=2021-04-17T22:30:00-00:00
Channels Found: 301
m3u is ready to write
m3u is being created
Traceback (most recent call last):
File "./plutotv.py", line 615, in
with open(m3u_destination, "w", encoding='utf-8') as f: #https://stackoverflow.com/a/42495690/11214013
TypeError: 'encoding' is an invalid keyword argument for this function

plex.py failing at the moment generating epg

Describe the Bug

plex.py failing at the moment generating epg

Code Snippets

'''Loading Grid for PlexTV, day: 1/2                                               url[1]: https://epg.provider.plex.tv/grid?type=1%2C4&sort=beginsAt&endsAt%3E=1654605000&beginsAt%3C=1654691400&X-Plex-Token=<MyPlexToken>&X-Plex-Language=en                                                                             Traceback (most recent call last):                                                File "/web2tv/plex.py", line 549, in <module>                                     main()                                                                        File "/web2tv/plex.py", line 248, in main                                         metadata_data['added_at'] = grid['MediaContainer']['Metadata'][y]['addedAt']KeyError: 'addedAt'''''

Expected Behavior

generate epf

Additional Context

No response

Operating System and Version

ubuntu

Python Version

Python 3.10.4

error running plex.py

getting an error running plex.py

run.log attached

run.log

plex.py not working

When I execute plex.py I get the following Error when calling the Website (I have an active Plex Pass):

https://epg.provider.plex.tv/grid?type=1%2C4&sort=beginsAt&endsAt%3E=1617451200&beginsAt%3C=1617537600&X-Plex-Token=_MY-TOKEN_&X-Plex-Language=en

<Error statusCode="403" error="Forbidden" message="'_MY_E-MAIL_' is NOT permitted to access restricted endpoint '/grid' on 'Plex Web/Chrome': Feature not permitted! Missing feature(s): cloud-livetv"/>

plex.py (and possibly plutotv.py) have a unicode encode error when running in background cmd.exe

Add argument to not print titles, or not print anything.