Encoder? about broadway

[binarykitchen]

Anyone?

[soliton4]

sounds interesting
i was thinking about doing something like that. what is your status? what is the problem?

[binarykitchen]

i tried to port avconv with h264 and libvpx to emscripten, see two gists:

1. https://gist.github.com/binarykitchen/5230096
2. https://gist.github.com/binarykitchen/5329825

you see, i fail at some stage and the resulting JS file got beyond 100MB, forget it.

but have a look at my mad experiment at https://www.videomail.io ... it uses a different approach to encode video and works fine as long as your connection isn't too slow.

[binarykitchen]

oh, i forgot to mention: check out this cool npm module https://www.npmjs.com/package/videomail-client

[soliton4]

let me guess, you get the video from the webrtc api and use avconv to convert it to a mp4 / any video container.

do you use native modules?

[soliton4]

about the kitchen. are you aware of the fact that you can use avconv with the std in and std out streams.
std in and out should be mapable with emscripten.
if i understand your problem you can not access files, but you dont need to access files if you use only streams

[binarykitchen]

correct, videomail = webrtc + websocket streams + ffmpeg

(i switched to ffmpeg because it turned to be more reliable than avconv at many places)

yes, i am aware that avconv (and ffmpeg) would accept streams but that doesn't work either. because with the libvpx and h264 libraries, the JS file after the emscripten process gets beyond 100 MB. too large for a web app.

[soliton4]

maybe we could extradite the part that encodes h264 and create a simple interface that feeds it with raw rgb image data

[binarykitchen]

and all that wrapped in a npm module would be so awesome and change the world one more time!

i would set up such a project with you together ...

(and not just h264, also include webp/libvpx as well since browsers need both video types, mp4 and webm)

[soliton4]

you can use native modules for that in npm

no need to compile to js

did you get my video mail?
i got "error forbidden" when i sent it

so many projects ... so many ideas. i can only offer you consulting at this moment.
if you are new to native modules check out https://github.com/rvagg/nan

it makes it so much easier

[binarykitchen]

oh shit, sorry about that bug - just fixed it. i've been working on security aspects recently, that's why. you should be able to send videomails now. please try again ...

native modules ... i've been thinking about it but i am not sure if you can browserify these for the browser? in this case, ffmpeg for example. correct me if i am wrong here. haven't dealt with nan's module before.

ultimately, my goal with videomail is to enable sign-language-based communication for deaf people at full speed without any drawbacks! any consulting is very welcome. been working very passionately for two years on this cool project and log files show that i am getting more and more regular users every day. cool!

[soliton4]

ill try again once i am fully awake ;)

dont get me wrong. i am a huge fan of using the same sourcecode on the server and the browser.
i mean: https://github.com/soliton4/promiseland/wiki/frames

but if you have c++ code in the first place, at least server sided you put it in a native module. it just makes much more sense.
i am not 100% sure the encoding is needed on the browser side anyways. i mean you are sending it to the server, and there you can just transcode it. so why the need for browser based encoding.
besides the fact that it would be totally badass js ;)

nan is just a compatibility layer.
the thing with native modules is that the api changes from node version to node version. and then there is io.js. nan enables you to simply program a native module that will compile on any version.
i just programmed a native module comercially. it really safes a lot of time. not just compatibility wise, it really makes things simpler.
i 100% recommend it ;)
in fact i gonna star it right now ...

oh so thats why there is no sound ;)

[binarykitchen]

well, having video encoding on the browser would be indeed totally cool and save lots of bandwidth. currently the video recording is limited to two minutes. exactly because of technical limitations. moving ffmpeg with h264 + webm to the browser is the answer to lift this limitation.

regarding nan, do you mean to wrap ffmpeg with nan and publish it as if it were it's own node module with it's own javascript api so that there is no need for apt-get install ffmpeg?

ps: have you checked out https://www.npmjs.com/package/videomail-client yet?

[soliton4]

well i tried your demo.
even sent you a mail ;)

yes the idea is to take the x264 code and put it into a native module. read about native modules here
https://nodejs.org/api/addons.html

i see what your prob is. thought the webm videos are recorded in that super fancy webm high compression codec. what codec does webrtc provide?

[binarykitchen]

thanks for the link, bookmarked!

right, you see the prob. no idea what codes webrtc use because i am not using it directly for videomail. just use the getUserMedia() API, nothing further.

[soliton4]

there must be a way to get the video encoded already.
webrtc was primarily designed to enable browser to browser audio and video communication.
they even have some sort of protocol in place to let the browsers manage what codec they agree upon.

i suggest you read further into the matter.
and pls when you have results let me know ;)

[binarykitchen]

believe me, i've done lots of research here. lots. webrtc is for client-to-client communication only. it is connection oriented. whereas email isn't. and videomail too because it's like an email. you send it, it goes to the server and resided there until the user loads it. webrtc is real time. these two are very different concepts.

i've digged deep in existing webrtc interfaces and there is NO function to extract any recordings and to store it on a single server.

for example you need a PeerConnection beforehand. how can you do this on the server side? looking at the diagram on https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Architecture/Connectivity might help understanding the problem. how can you turn peer b into a server itself?

[soliton4]

how about this:
https://github.com/js-platform/node-webrtc/blob/develop/examples/ping-pong-test.js

[binarykitchen]

i tried that. it won't work between client and server. remember, videomail is client to server communication whereas all WebRTC are client to client.

[soliton4]

so since you tried it all you can tell me what is wrong with this?
https://developer.mozilla.org/en-US/docs/Web/API/MediaRecorder_API

this looks like it works
https://www.webrtc-experiment.com/RecordRTC/

then there is
http://www.kurento.org/

do non of these work?

[binarykitchen]

i like your inputs, really am enjoying this conversation.

1. MediaRecorder
   this is for local instances only. Won't work for videomail because i need the video on the server side to send out the email with the video together. furthermore MediaRecorder isn't supported on any browsers except Firefox.
2. RecordRTC ... forget this, no offense but code is poor and does not have the functionalities i need.
3. Kurento is too proprietary and not really open source.

believe me, i did lots of research but there is nothing out there yet ...

[soliton4]

@binarykitchen
this: http://bgrins.github.io/videoconverter.js/demo/

took 28 sec to convert to h264

use this parameters:

-i input.webm -vf showinfo -vcodec libx264 -pass 1 -coder 0 -bf 0 -flags -loop -wpredp 0 -an -preset ultrafast output.mp4

works on ff only

[binarykitchen]

indeed, i have my eyes on videoconverter.js for quite a long time, see bgrins/videoconverter.js#17

few functionality is still missing before i can integrate that to videomail such as piping images from the webcam, the file api and memory issues (because keeping a whole video in memory before sending it to the server is a huge performance issue)

[soliton4]

looks like encoding is not on the horizon soon