wmcbrine / tivodecode-ng Goto Github PK

I'm getting a Failed to process file: Undefined error: 0 with all the TiVo files from the Lifetime channel.

The error occurs in tivo_decoder_ts.cxx line 517:
The switch receives the following paket type:
ts_packet_type ts_packet_pid_types TS_PID_TYPE_RESERVED
and goes to the default case which errors.

TiVo Header :
flags : 61
origin : US
format : Transport Stream
source : HDTV
TiVo unit : Series3
mpeg_offset : 19456
chunks : 3

Creating new stream for PID 0
TS ProgAssocTbl : version changed : �
TS ProgAssocTbl : Program Num : 1
TS ProgAssocTbl : Program PID : 100
Creating new stream for PMT PID 100
TS ProgMapTbl : StreamId �, PID 17, Type 2 : Video
Creating new stream for PID 17
TS ProgMapTbl : StreamId \201, PID 20, Type 1 : Audio
Creating new stream for PID 20
TS ProgMapTbl : StreamId \227, PID 21, Type 3 : PrivateData
Creating new stream for PID 21
Unknown Packet Type: Undefined error: 0
Failed to process file: Undefined error: 0
Program ended with exit code: 9

Hey there... our favorite DJ was doing her final show this morning, 6AM-11AM. Comcast simulcasts the station on one of the 900 channels. So I setup a manual recording. Unfortunately cTivo failed to download the recording. So I used the URL and grabbed the .TiVo file directly, figuring I'd do it the old way.

File downloaded fine. 6 hours of audio-only turns out to be pretty small: 291MB.

The problem is... tivodecode-ng can't seem to make heads-or-tails of the file. I can dump the XML files and they look intact (but offer no clues).

If I skip the decoding, I see:

TiVo Header :
fileType : 54:69:56:6f (TiVo)
dummy_0004 : 0x0004
dummy_0006 : 0x003d
origin : US
format : Transport Stream
source : HDTV
TiVo unit : Series3
dummy_0008 : 0x0000
mpeg_offset : 10240
chunks : 3

When I try decoding, I see:

Creating new stream for PID (0x0000)
TS ProgAssocTbl : Program Num : 1
TS ProgAssocTbl : Program PID : 0x64 (100)
Creating new stream for PMT PID 0x0064
TS ProgMapTbl : StreamId 0x81 (129), PID 0x0ddc, Type 0x01 (1 : Audio)
Creating new stream for PID 0x0ddc
TS ProgMapTbl : StreamId 0x97 (151), PID 0x0ddd, Type 0x03 (3 : PrivateData)
Creating new stream for PID 0x0ddd
do_header did not return 0!
: Undefined error: 0
Packet decrypt fails: Undefined error: 0
Failed to add packet to stream : pktId 3
do_header did not return 0!
: Undefined error: 0

Testing with a short clip of regular video, had no problems converting a file. The main difference is the video stream that is found:

TS ProgMapTbl : StreamId 0x1b (27), PID 0x14c6, Type 0x02 (2 : Video)
Creating new stream for PID 0x14c6`

And... the Private data (keys) never show up with the audio-only file. So my MAK is right... any thoughts? Thanks!

Hi,

I've been using tivodecode 0.4.4 for a while now so that I can download TS files from my TiVo. TS is how they come OTA (I'm in Australia, if that's relevant) and having the TiVo convert them to PS does not always work (particularly for certain channels). I originally used ffmpeg to convert to PS (for better seeking) but these days I do a quick mpeg4 re-encode for faster seeking and better handling of the occasional invalid video chunk.

Anyway, I have a show I just recorded that crashes tivodecode. After looking at a stack trace, I made the below change to reject the packets that are bad. The end result is that I get all the video but audio stops about 10 minute in. The file plays just fine on the TiVo itself.

The things is, I'm not really sure about what's going on with the internals of tivodecode so I don't know if this really is a bad packet (ie. the TiVo has given me crap) or if there's just some kind of decode issue.

I was pointed to this project, which continues from tivodecode 0.4.4 and it has the same issue.

diff --git a/tivo_decoder_ts_stream.cxx b/tivo_decoder_ts_stream.cxx
index 78e85cc..28a1b1e 100644
--- a/tivo_decoder_ts_stream.cxx
+++ b/tivo_decoder_ts_stream.cxx
@@ -74,6 +74,10 @@ bool TiVoDecoderTsStream::addPkt(TiVoDecoderTsPacket *pPkt)
             VVERBOSE("DEQUE : PktID %d from PID 0x%04x\n", pPkt2->packetId,
                     stream_pid);
 
+            int size = (TS_FRAME_SIZE - pPkt2->payloadOffset);
+            if (size <= 0)
+                return false;
+
             std::memcpy(&pesDecodeBuffer[pesDecodeBufferLen],
                         &pPkt2->buffer[pPkt2->payloadOffset],
                         TS_FRAME_SIZE - pPkt2->payloadOffset);

I was also pointed to tivolibre, a Java version of the tivodecode logic. Out of curiosity, I ran the same video through it and it completed successfuly, keeping the audio intact. Interesting... I might take a look at what it's doing and see if I can bring the changes/improvements over to C++.

I'm noticing regular stream corruption (i.e. macroblocking) happening in the video stream for non-HD TS streams.

On my old (Series4) TiVo this would be resolved by re-downloading it again as a PS stream.
On my new Tivo the PS stream is corrupted to almost unreadability, currently resolved by transferring it to the Series4 TiVo and then re-downloading it as a PS stream.

The fact that I'm able to recover a non-corrupted file may help identify where the underlying issue is in any case.

The version I am using is labeled "tivodecode-ng-w64-0.5"

I have a bunch of TiVo recordings (mpeg-ts/H.264) and I'm noticing that the decrypted videos from tivodecode-ng have errors roughly every 10:19s. This seems too regular to be from corruptions in the broadcast...

I have a tivo file that crashes NG. I can try and track it down, but I thought I'd check first if you've seen something similar or if you have any suggestions on where to look based on the output:

Goes along fine through 40% of the file then:

712221088 : stream_id: bd, block_no: 533
712221276 : stream_id: bd, block_no: 533
712221464 : stream_id: bd, block_no: 533
712221652 : stream_id: bd, block_no: 533
Can not locate packet stream by PID: Undefined error: 0
Can not locate packet stream by PID: Undefined error: 0
<<1461 more of these>>
Can not locate packet stream by PID: Undefined error: 0
TS ProgAssocTbl : Program Num : 1
TS ProgAssocTbl : Program PID : 0x64 (100)
Re-use existing stream for PMT PID 0x0064
TS ProgMapTbl   : StreamId 0x02 (2), PID 0x0ea8, Type 0x02 (2 : Video)
Creating new stream for PID 0x0ea8
TS ProgMapTbl   : StreamId 0x81 (129), PID 0x0ea7, Type 0x01 (1 : Audio)
Creating new stream for PID 0x0ea7
TS ProgMapTbl   : StreamId 0x97 (151), PID 0x0ea9, Type 0x03 (3 : PrivateData)
Creating new stream for PID 0x0ea9
do_header did not return 0!
: Undefined error: 0
Packet decrypt fails: Undefined error: 0
Failed to add packet to stream : pktId 3789742
do_header did not return 0!
: Undefined error: 0
Packet decrypt fails: Undefined error: 0
Failed to add packet to stream : pktId 3789770

TiVo Private    : Validator                 : 0x5469566f (TiVo)
TiVo Private    : Unknown                   : 0x81 0x3 0x0 0x7e
TiVo Private    : Stream Bytes              : 40
TiVo private data : matched PID 0x0ea8

Updating PID 0x0ea8 Type 0xe0 Turing Key
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
88 c0 00 a2 d0 00 08 07 ff ff ff fc 00 02 00 01  ................
TiVo Private    : Block No                  : 0
TiVo Private    : Crypted                   : 0x00000000
TiVo Private    : PID                       : 0x0ea8 (3752)
TiVo Private    : Stream ID                 : 0xe0 (224)
TiVo Private    : Turing Key                : 
88 c0 00 a2 d0 00 08 07 ff ff ff fc 00 02 00 01  ................
TiVo private data : matched PID 0x0ea7

Updating PID 0x0ea7 Type 0xbd Turing Key
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
88 c0 00 a2 d0 00 08 07 ff ff ff fc 00 02 00 01  ................
TiVo Private    : Block No                  : 0
TiVo Private    : Crypted                   : 0x00000000
TiVo Private    : PID                       : 0x0ea7 (3751)
TiVo Private    : Stream ID                 : 0xbd (189)
TiVo Private    : Turing Key                : 
88 c0 00 a2 d0 00 08 07 ff ff ff fc 00 02 00 01  ................
712522264 : stream_id: e0, block_no: 534
712523016 : stream_id: e0, block_no: 534
712523768 : stream_id: bd, block_no: 534
tivodecode-ng(73807,0x7fff7c60e000) malloc: *** error for object 0x7f87d3d26370: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug

Funny thing is that it seems to be getting back on track, and then ...

FYI, (and to confirm that the file is definitely messed up), tivolibre's output for this section is:

PacketId: 3,600,000 Type: AUDIO_VIDEO_PRIVATE_DATA PID: 0x0fe3 Position after reading: 676,818,432
PacketId: 3,700,000 Type: AUDIO_VIDEO_PRIVATE_DATA PID: 0x0fe3 Position after reading: 695,618,432
No TransportStream exists with PID 0x0ea7, creating one
Decrypting packet in stream 0x0ea7 failed
No TransportStream exists with PID 0x0ea8, creating one
Decrypting packet in stream 0x0ea7 failed
<<250 more of these>>
Decrypting packet in stream 0x0ea7 failed
Creating a new PRIVATE_DATA stream for PID 0x0ea9 (0x97)
Decrypting packet in stream 0x0ea7 failed
Decrypting packet in stream 0x0ea7 failed
PacketId: 3,800,000 Type: AUDIO_VIDEO_PRIVATE_DATA PID: 0x0ea8 Position after reading: 714,418,432
PacketId: 3,900,000 Type: AUDIO_VIDEO_PRIVATE_DATA PID: 0x0fe3 Position after reading: 733,218,432