givo / lib-kurento

A typescript library for simplifying the use of Kurento in Node.js

Lib-Kurento

A typescript library for simplifying the use of Kurento in Node.js.

Motivation

Kurento Media server is controlled through the API it exposes, so we, the application developers use client implementations like kurento-client-js to interact with the media server. The problem with kurento-client-js is that the package was automatically generated, therefore the source code is not readable, hard to use and requires a lot of repetitive code. The API becomes even harder to handle as the application becomes larger and uses more then one streaming protocol. Therefore I have created a simple library that simplifies the initialization process of the common endpoints types (And I even handles some bugs in the library for you).

Install

npm i --save lib-kurento

Usage

General Example

An example for creating a pipeline with two types of sources, RTSP and RTP that are sent to clients through WebRTC:

import * as libKurento from 'lib-kurento';

const kmsUri = "ws://192.168.32.130:8888/kurento";
const rtpSdpOffer: string = "";     // get sdp from somewhere
const clientSdpOffer: string = "";  // get sdp from client using any kind of a signaling communication
const socket: WebSocket;

function sendServerIceCandidate(candidate) {
    // send ice candidate to client
    // for example:
    socket.send(JSON.stringify( { candidate: candidate } ))
}

async function main(){
    // connect to kurento
    const kurentoClient = await libKurento.connectToKurentoServer(kmsUri);

    // create a pipeline
    const pipeline = await libKurento.createPipeline(kurentoClient);

    // create RTSP and RTP endpoints
    const rtpEndpoint = new libKurento.RtpEndpointWrapper(pipeline, rtpSdpOffer);

    const rtspEndpoint = new libKurento.PlayerEndpointWrapper(pipeline, {
        uri: 'rtsp://192.168.1.100/stream1',
        networkCache: 0 /* low latency */ 
    });

    // initialization simplified!
    await rtpEndpoint.init();
    await rtspEndpoint.init();

    // Accessing kurento-client`s instances is allowed as follows:
    await (rtpEndpoint.endpoint as any).setMaxOutputBitrate(0); // unlimited bitrate

    // start receiving feed from the rtsp source
    await rtspEndpoint.play();

    // create a WebRTC endpoint
    let webRtcEndpoint = new libKurento.WebRtcEndpointWrapper(pipeline, clientSdpOffer);

    // when the server's ice candidates are collected send them to the client
    webRtcEndpoint.on("ServerIceCandidate", sendServerIceCandidate);

    // init the WebRTC endpoint, also starts gathering ICE candidates from the media server instance
    await webRtcEndpoint.init();

    // receive client ice candidates
    socket.on('message', (msg: any) => {
        const parsedMsg = JSON.parse(msg);

        // add the client's ICE candidate to it's matching WebRTC endpoint
        // IMPORTANT NOTE: `lib-kurento` stores candidates in a queue in order
        // to add them only when the endpoint is ready
        webRtcEndpoint.addClientIceCandidate(parsedMsg.candidate);
    })

    // connect source endpoints to output endpoints and feed will start flowing to clients
    await rtspEndpoint.connect(webRtcEndpoint);
    await rtpEndpoint.connect(webRtcEndpoint);
});

Recording Example

A very simplified example for recording a RTSP feed from an IP camera to a MKV file:

import * as libKurento from 'lib-kurento';
const kmsUri = "ws://192.168.32.130:8888/kurento";
const cameraRtspUrl = "rtsp://192.168.1.32/channels/1";
const socket: WebSocket;

async function startStreaming(clientSdpOffer: string){
    // connect to kurento
    const kurentoClient = await libKurento.connectToKurentoServer(kmsUri);

    // create a pipeline
    const pipeline = await libKurento.createPipeline(kurentoClient);

    // create a RTSP endpoint
    const rtspEndpoint = new libKurento.PlayerEndpointWrapper(pipeline, { 
        uri: cameraRtspUrl,
        networkCache: 0 /* low latency */ 
    });

    // create recorder
    const recorderEndpoint = new libKurento.RecorderEndpointWrapper(pipeline, {
        uri: '/user/home/recordings/recording1.mkv',
        mediaProfile: 'MKV_VIDEO_ONLY'
    });

    // create a WebRTC endpoint
    let webRtcEndpoint = new libKurento.WebRtcEndpointWrapper(pipeline, clientSdpOffer);

    // when the server's ice candidates are collected send them to the client
    webRtcEndpoint.on("ServerIceCandidate", sendServerIceCandidate);

    // initialization simplified again!
    await rtspEndpoint.init();
    await recorderEndpoint.init();
    await webRtcEndpoint.init();

    // listen to recording events
    recorderEndpoint.on('RecordingStarted', (event) => {
        console.log('recording has started')
    });
    recorderEndpoint.on('RecordingStopped', (event) => {
        console.log('recording has stopped')
    });
    recorderEndpoint.endpoint.addListener('Error', (err) => {
        console.log('recorderEndpoint.endpoint error', err);
    })

    // receive client ice candidates
    socket.on('message', (msg: any) => {
        const parsedMsg = JSON.parse(msg);

        // add the client's ICE candidate to the WebRTC endpoint
        webRtcEndpoint.addClientIceCandidate(parsedMsg.candidate);
    });

    // IP Camera -> WebRTC
    // IP Camera -> MKV file
    await rtspEndpoing.connect(recorderEndpoint);
    await rtspEndpoint.connect(webRtcEndpoint);

    // start receiving feed from IP camera
    await rtspEndpoint.play();

    // start recording
    await recorderEndpoint.record();

    // stop recording after 5s
    setTimeout(async () => {
        await recorderEndpoint.stopRecord();
    }, 5000);
}

lib-kurento's Issues

ICE candidate race

Hey, thanks for the awesome lib!

I'm recreating the one2many kurento demo tutorial, and running into a simple problem: When I receive a "present" message from a client, I immediately set up a libKurento.WebRtcEndpointWrapper with the Sdp, but before it is constructed I also receive "onIceCandidate" from the client, so I am unable to call webRtcEndpoint.addClientIceCandidate on the endpoint to put it in the queue.

There's two ways I can think of fixing this without modifying lib-kurento: 1) Have the client only start sending onIceCandidates AFTER it receives the sdp answer. 2) queue the messages myself and pass them to the WebRtcEndpointWrapper when it is ready (not utilizing its internal queue)

That said, if I could create a libKurento.WebRtcEndpointWrapper WITHOUT specifying an sdp (so it could have it around to start queuing ice candidates whenever they are received), and then when I receive an sdp from the client I can call WebRtcEndpointWrapper.init(sdpOffer), that would allow me to utilize the internal queue of WebRtcEndpointWrapper.

So here's what I'm proposing:

• Construct WebRtcEndpointWrapper with only a pipeline (no SDP)
• Add a sdpOffer arg to init so when receiving an sdpOffer I call WebRtcEndpointWrapper.init(sdpOffer)

Of course I'm new to all this so I might be way off the mark -- please let me know if I'm thinking about this correctly!

RecordingStart event is not fired

// . . . . .

let createRecorderStateLogger = (eventLabel) => {
    let callback = (event) => {
        let descriptionParts = [event.type, event.mediaType, event.state].filter(x => x)
        let description = descriptionParts.join(' ')

        this.log(`recorderEndpoint : ${eventLabel} event: ${description}`)
    }
    callback.bind(this)
    return callback
}

let eventsNamesToLog = [
    'RecordingStarted', 'RecordingStopped',
    'MediaFlowingIn', 'MediaFlowingOut',
    'MediaStoppedFlowingIn', 'MediaStoppedFlowingOut',
]

// logging recorder state changes
for (let eventNameToLog of eventsNamesToLog) {
    this.recorderEndpoint.on(
        eventNameToLog,
        createRecorderStateLogger(eventNameToLog)
    );
}

this.recorderEndpoint.on('RecordingStarted', (evt) => {
    // NOT FIRED!!!
    this.wsMessageToClient({
        category: MessageFromServerCategory.recordingStarted,
    })
});

// start recording
await this.recorderEndpoint.record();

setTimeout(async () => {
    await this.recorderEndpoint.stopRecord();
}, 7000);

[25.12.2019 18:20:59] recorderEndpoint : MediaFlowingIn event: MediaFlowInStateChange AUDIO FLOWING
[25.12.2019 18:20:59] recorderEndpoint : MediaFlowingIn event: MediaFlowInStateChange VIDEO FLOWING
[25.12.2019 18:20:59] recorderEndpoint : RecordingStopped event: Recording
[25.12.2019 18:21:05] recorderEndpoint : RecordingStopped event: Stopped

Video file is saved normally but event object has type = 'Recording' on RecordingStopped event and RecordingStarted event is not fired at all.

// package-lock.json:

"lib-kurento": {
      "version": "0.0.5",
"kurento-client": {
      "version": "6.12.0",

Recording HD videos: override max video bandwidth

If I call setMaxVideoRecvBandwidth() in my code

  await webRtcEndpoint.init()

  // no effect!
  await webRtcEndpoint.endpoint.setMaxVideoRecvBandwidth(50000)

it doesn't take effect (poor quality of recorded videos).

I had to modify SdpEndpointWrapper.init() to run setMaxVideoRecvBandwidth() before processOffer()

    async init() {
        await super.init();

        // mod
        await this._endpoint.setMaxVideoRecvBandwidth(50000)

        // process sdp offer
        this._sdpAnswer = await this._endpoint.processOffer(this._sdpOffer);
        this.emit("SdpAnswerCreated", this._sdpAnswer);
    }

Maybe it's a good idea to make some event on EndpointWrapper.init() right after _endpoint creation to have time to manipulate it before processOffer() begins.

    async init() {
        this._endpoint = await this._pipeline.create(this._endpointName, this._createOptions);
        this.emit("EndpointCreated", event);

or, simpler, no automatic processOffer() on SdpEndpointWrapper.init() (extract it to separate method)

Using [email protected]

Record error handling example

Thanks for sharing awesome library!
I suggest adding these lines to the Recording Example in README.md:

recorderEndpoint.endpoint.addListener('Error', (err) => {
  console.log('recorderEndpoint.endpoint error', err);
 })

It greatly helps to debug recording problems.

Nice. Thanks for sharing

However, can't use your lib in my node ts app. What might go wrong?

import * as libKurento from 'lib-kurento';

src/index.ts(2,29): error TS2307: Cannot find module 'lib-kurento'.

WebSocket.on is not defined

Hi,
In both examples of the README, there's a call to socket.on where socket is of type WebSocket.
However, WebSocket does not have any member on. How do I proceed?

I'm compiling to es5 and have the libraries for es2019 and dom in my tsconfig.json

Thanks in advance.