How to send producer timestamp about amazon-kinesis-video-streams-producer-sdk-cpp

[zhiyua-git]

Hi ynaka81,

By default, the producer timestamp is provided by you. Can you check what timestamp you put here? Timestamp is from the device itself, maybe the device clock is set to some old time.

Please let me know how it goes. Thanks.
-ZH

[ynaka81]

Hi zhiyua-git. Thank you.
I changed the source code to print pts and dts as following.

diff --git a/kinesis-video-gst-demo/kinesis_video_gstreamer_sample_app.cpp b/kinesis-video-gst-demo/kinesis_video_gstreamer_sample_app.cpp
index 0f3c29a..478e9c4 100644
--- a/kinesis-video-gst-demo/kinesis_video_gstreamer_sample_app.cpp
+++ b/kinesis-video-gst-demo/kinesis_video_gstreamer_sample_app.cpp
@@ -6,6 +6,7 @@
 #include "KinesisVideoProducer.h"
 #include <vector>
 #include <stdlib.h>
+#include <iostream>
 
 using namespace std;
 using namespace com::amazonaws::kinesis::video;
@@ -157,6 +158,7 @@ void create_kinesis_video_frame(Frame *frame, const nanoseconds &pts, const nano
 
 bool put_frame(shared_ptr<KinesisVideoStream> kinesis_video_stream, void *data, size_t len, const nanoseconds &pts, const nanoseconds &dts, FRAME_FLAGS flags) {
     Frame frame;
+    cout << "pts:" << pts.count() << ", dts:" << dts.count() << endl;
     create_kinesis_video_frame(&frame, pts, dts, flags, data, len);
     return kinesis_video_stream->putFrame(frame);
 }

I got pts:3600002555339025, dts:3600002555339025. It seems to be incorrect. Even if I use system_clock instead of steady_clock, the result was not changed.
I found that the timestamp from gstreamer is wrong. It might be related with I used web-cam.

[zhiyua-git]

Hi yanka81,

Would you mind providing more information for letting us solve your issue?

• What USB webcam are you using? Can you provide us the model and manufacturer of your USB webcam?to run the gstream app? Is it getting h264 frames directly or from h264 ecoder in gstreamer? Just wanted to check if the GStreamer getting the h264 encoded frames from the camera or not.
• What kind of data consumption are you planning to do on the data, playback or video analysis?

Thanks,
-ZH

[ynaka81]

Hi zhiyua-git,

• I used equipped camera of my ThinkPad T460s (https://www3.lenovo.com/us/en/laptops/thinkpad/thinkpad-t-series/ThinkPad-T460s/p/22TP2TT460S).
• I just ran $ ./kinesis_video_gstreamer_sample_app example to send video stream to KVS.
• I'm planning to use it for video analysis. As a first step, I checked video preview on AWS Web console.
• I can see video preview in server time stamps mode, but can't see it in producer time stamps mode.

Let me know if you need more information. Thanks.

[zhiyua-git]

Hi ynaka81,

Can you also provide some details about your stream? Like stream name, account Id, and request Id in log?

Request id will look like "DEBUG / KinesisVideo: Received ACK bits: x-amzn-RequestId: 8b9ea9ef-12bd-11e8-8d8a-15e16ecsample" in the log.

It will be great if you can send this information to our support email: [email protected].

Thanks,
-ZH

[ynaka81]

Hi zhiyua-git,
Thank you. I sent to your support email.

[vijaygj]

Hello ynaka81@,

Can you please send us more details about following failure on the same email address?

I can see video preview in server time stamps mode, but can't see it in producer time stamps mode.

We see that the producer time stamps are being assigned to the fragments under your stream but may be the time you are using to "search" is not in sync with time assigned to each fragment. Can you please send the screen shot of the playback issue? We can check further.

Thanks,
Vijay.

[ynaka81]

Hello vijaygj,
I sent to your support mail.

As you say, I think that the time is not in sync with time assigned to each fragment because the timestamp from gstreamer is already incorrect (cf. #35 (comment)).

Oh, I might misunderstand. The producer time stamp is not the time like UTC or JST, but the uptime of producer, isn't it?

[vijaygj]

Thank you for the email. The producer time stamp is indeed local time on the camera device hence in millis since epoch and not uptime of the producer. We will check why producer SDK is not able to capture the correct local time and update the thread.

[ynaka81]

Hello vijaygj,
Thank you for your cooperation. I'm waiting for your update.

[zhiyua-git]

Hi ynaka81,

We just posted a fix in release 1.2.1 for system_clock used by SDK related to producer timestamp playback in console.

This should help. Can you give it a try?

Thanks,
-ZH

[ynaka81]

Hi zhiyua-git,

Thank you for fixing it. It works well!

[zhiyua-git]

Great to hear it works!

[wkentaro]

Hi, I encountered the same (or similar) problem, so could you give me some information to fix it?
I am using v1.3.1 of this SDK, and I can see the frame on web console with both producer and server timestamp option.
But producer timestamp differs from the actual time from 5 seconds to 10 minutes.
(the server timestamp also has ~5seconds time lag but I suppose it is the time spent for the upload)

Note that I have checked the times are synchronized for both the clock in the video and the PC to which the USB camera is connected.

server timestamp (looks ok)

producer timestamp

time lag: ~10minutes

time lag: ~5seconds

[MushMal]

@wkentaro the console playback is a "live" playback and it's not "realtime". It is based on fragments being "assembled", persisted, indexed on the backend, then, downloaded to the console player where the browsers MSE is used to playback the MPEG fragments in the right order. Generally, the observed latency in the console playback is expected to be the fragment duration + ~3 seconds for the end-to-end. Could you please check what's your fragment duration?

As for the producer timestamp playback - it's not yet immediately clear as to what's going on. Is the playback starting 10 minutes after the start or it starts and then drifts? One thing that needs to be noted is that the browser will stop/pause the playback if the screen becomes inactive and will re-start from the paused time when the screen becomes active again.

Could you please provide more information? If this is a 10 minute delay to start the playback then we will need to dive deep and root cause this issue.

[wkentaro]

the console playback is a "live" playback and it's not "realtime"..

Yeah, I think I understand. It takes 5-10 seconds before the playback starts, and usually I observe live-7seconds at the right bottom in the console.

---

I expect the time in the left of this figure (the figure below, 0:28:32) and the time in the frame (0:28:27) would be same with the producer timestamp playback.

I also expect the time of the right bottom in the console and the frame in the console is synchronized.
Maybe, my expectations are not right?

---

I have tried uploading video many times, and it was very rare case that I had 10 minutes delay, so maybe it could be caused by my mistake or browser cache or reload.
But I always have >5seconds delay I described above, and that's a critical issue for me.

[zhiyua-git]

@wkentaro Can you check the Fragment duration in the console page? The delay you will see in console would be your fragment duration + console player buffering time(~3 seconds). So 5-second delay in console you see would be quite common if your fragment duration is default 2-second setting.

If you want realtime playback scenario (), you will need some type of realtime player implementation. You can refer to our realtime playback sample KinesisVideoRendererExample in our parser library.

Let me know if this helps. Thanks.

[wkentaro]

Ah, sorry. I overlooked the advice of @MushMal about Fragment duration.

I checked the Fragment duration on the console page as below.
It is 3.0 seconds, so it seems that it is working correctly as you said.
I will try the realtime player implementation. Thank you for your help!