touchdesigner / webrtc-remote-panel-web-demo

A simple React project that can receive a WebRTC video stream from TouchDesigner and send back mouse and keyboard events.

Getting started

A walkthrough is available at the following link.

Get the project running locally

To run the project locally, you will need to first clone this repository.

You need to have installed NodeJS: LTS Version: 18.12.1 (includes npm 8.19.2)

In the project cloned repository folder, use: npm start

Runs the app in the development mode. Open http://localhost:3000 to view it in your browser.

The page will reload when you make changes. You may also see any lint errors in the console.

Secure the project in the development server:

If you want to run the project on your own machine and rely on SSL, you will need to run the development server of your NodeJS project by passing the following environment variables

HTTPS=true
SSL_CRT_FILE=PATH TO FILE.crt
SSL_KEY_FILE=PATH TO FILE.key

where you set a relative path to the .crt and .key files such as ./localhost.crt and ./localhost.key.

As an alternative, you can also edit the package.json file with the following start script:

    "start": "set HTTPS=true&&set SSL_CRT_FILE=PATH TO FILE.crt&&set SSL_KEY_FILE=PATH TO FILE.key&&react-scripts start"

You will need a server certificate. To generate development certificates, you can follow instructions on this page's section "Getting a development certificate".

Use the web demo page

A live web demo page can be used at the following link: https://touchdesigner.github.io/WebRTC-Remote-Panel-Web-Demo/

For the web demo page to connect with a local signalingServer, you will need a server certificate. You can follow instructions on this page's section "Getting a development certificate".

Development certificates come with limitations as they are not recognized by the certificate authority.

If you are familiar with certificate authorities, you can get your own valid certificate installed on the signalingServer.

If you use a development certificate, you will want to add this certificate to your browser (In Chrome: Settings -> Privacy and Security -> Security -> Manage device certificates) or first hit the IP of your signalingServer in a browser using https:// to temporarily add the server certificate to Chrome.

TouchDesigner setup

In TouchDesigner, you need a signalingServer COMP (Palette -> WebRTC), where you could have to turn on the Secure toggle if you use TLS as well as to set the certificate and key parameters.

You will need a signalingClient running locally or on another machine, with the toggle to "Forward to subscribers" turned on.

The WebRTCRemotePanel COMP should be setup as well with its signalingClient parameter set (where the signalingClient you created is referenced) and a compatible panel.

Limitations

Unfortunately, not all components will work nicely with that specific setup, but a large array of simple setups should be covered.

Issues you might sometimes encounter are click not being registered, or dialogs not appearing.

A general rule of thumb is to avoid opening dialogs. I.e. Don't open from your custom component a Parameter dialog, but embed the parameters in your UI through widgets or Parameter COMPs.

Keyboard input is currently not supported (but partly designed in the code for future use).

Touch screens input from phones is partly supported.

Perfect negotation

The project should more or less follow the perfect negotiation as presented in the following links, with small tweaks based on the Signaling messages schemas.

One main difference is how we determine which Signaling Client will be polite.

When using the TouchDesigner Signaling Server, the signalingServer COMP will return a "join time" to the client when acknowledging that the client joined the session.

This is the "join time" that will determine which signalingClient will be polite or unpolite.

• Mozilla
• Google's example
• WebRTC Basics

Getting a development certificate

To get your own development certificate we suggest to use mkcert: https://github.com/FiloSottile/mkcert

Follow the instructions to install.

Once install, open a cmd terminal in the folder of your TouchDesigner project and use the following command to generate a local server certificate:

mkcert -install
mkcert -cert-file tdServer.crt -key-file tdServer.key localhost 127.0.0.1

This will create the files tdServer.crt and tdServer.key at the root of your folder.

You can now use those files on the signalingServer COMP

JSON Schemas

All of TD's own Signaling API is defined at the following repository with various schema files: https://github.com/TouchDesigner/SignalingAPI

Publishing (for devs only and contributors)

Updating the Github hosted page can be done with write permissions using

npm run deploy

As per: https://create-react-app.dev/docs/deployment/

The homepage in package.json should be of the likes of

"homepage": "https://myusername.github.io/my-app",

webrtc-remote-panel-web-demo's Issues

ESLint recommendations

WARNING in [eslint]
src\App.js
Line 1:17:    'useEffect' is defined but never used                                         no-unused-vars
Line 1:28:    'useState' is defined but never used                                          no-unused-vars
Line 9:10:    'getOrDefaultTo' is defined but never used                                    no-unused-vars
Line 235:3:   Expected a default case                                                       default-case
Line 296:24:  Expected '===' and instead saw '=='                                           eqeqeq
Line 648:29:  Array.prototype.some() expects a value to be returned at the end of function  array-callback-return
Line 688:35:  Array.prototype.some() expects a return value from function                   array-callback-return

Fix audit warning with CRA update

Users might notice when installing the project or deploying (dependabot) that they are "security warnings".

Those security warnings can be safely ignored as they affect backend code which the project doesn't rely on. This backend code is used to build the project.

It is widely reported @ Create-React-App / Facebook folks and is awaiting the following pr to be merged facebook/create-react-app#12172

Similar issues were already reported by React users:
facebook/create-react-app#11801

This issue goes extensively into why it can be ignored:
facebook/create-react-app#11174

List and remove dead code using the **unimported** tool

https://github.com/smeijer/unimported

     summary               unimported v1.22.0 (node)
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────       entry file          : src/index.js

       unresolved imports  : 0
       unused dependencies : 7
       unimported files    : 2


─────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────     │ 7 unused dependencies
─────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────   1 │ @mui/icons-material
   2 │ @testing-library/jest-dom
   3 │ @testing-library/react
   4 │ @testing-library/user-event
   5 │ express
   6 │ react-scripts
   7 │ webrtc-adapter
─────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

─────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────     │ 2 unimported files
─────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────   1 │ src/adapter.js
   2 │ src/setupTests.js
─────┴─────────────────────

Code splitting

All the classes currently live in src/App.js

There are currently four classes in the project:

• TDClient
• WebRTC
• Signaling
• App

First step is to split those into separate files.

Use JSX syntax instead of **React.createElement()**

Review use of the Mozilla "adapter.js" WebRTC shim

It's downloaded as a file in src/ and is listed in webrtc-adapter.
It should be used to ensure crossbrowser compatibility, since the WebRTC specs are forever evolving.

adapter.js is a shim to insulate apps from spec changes and prefix differences in WebRTC. The prefix differences are mostly gone these days but differences in behaviour between browsers remain.

NPM Dependency vulnerabilities

npm install complains about potential vulnerabilities.

9 high severity vulnerabilities

To address all issues (including breaking changes), run:
npm audit fix --force

Specify LICENSE

@jetXS what license are you planning on using? It would be set in the package.json file and in the repo.

https://docs.npmjs.com/cli/v8/configuring-npm/package-json

Add comments everywhere

Rename unclear classes and props

Use react hooks instead of class Components

The web is pretty clear about it. Component classes should not be used in React anymore.
https://usehooks.com/
https://dev.to/video-sdk/build-video-calling-app-using-react-hooks-1a79

Review in comment TODOs

Better documentation

Hi, a clearer explanation would be nice: i can't make this connect to nothing, there is no log, no error messages, just nothing that helps to make a working connection.

• Is the certificate needed? the port should be set as 443 in the signaling server in touchdesigner?
• Can i use 9980 port in the web page?
• the project must run on the same machine? do i need a touchdesigner with a signaling client running together?
• using the official demo do i need to open ports on the router?

Having a more precise explanation would be better, since i can make a WebRTC between two Touchdesigner machines without issues.

Review signaling strategy

With Google's and Mozilla's exemples and guidelines.

Migrate to using webrtc-react premade modules

JS newcommers comming from a standard TD use probably have no interest in understanding the WebRTC negotiation tactics.
This demo should be as simple as possible and use less code.