rend-o-matic / renderer

Rend-o-matic render agent - combines videos into a finished output.

Rend-o-matic

A musical collaboration project for disconnected performers.

Rend-o-matic is a Call for Code with The Linux Foundation Project created by developers at IBM. Rend-o-matic is the technology that powers the Choirless platform.

What is Rend-o-matic?

Choirless is a musical collaboration platform built to enter the Call for Code 2020 global competition. It allows music groups to create a video wall recording of a piece of music, where all of the individual submissions are captured separately on the performers' phone, tablet or laptop.

Choirless came about during the 2020 Covid-19 Pandemic, as countries went into lockdown and social-distancing prevented choirs, bands and other musical ensembles from meeting and performing in person. Video meeting platforms such as Zoom and Webex proved useless for live collaboration because of the network latency and the audio being optimised for speech.

Choirless aims to make it very easy for choir leaders to create songs made out of several parts (e.g. alto, tenor, soprano) and to organise choir members to provide renditions of a part. All of the contributed videos are stitched together into a video wall with no special equipment and without employing costly and time-consuming video editing software.

Rend-o-matic is a Call for Code® with The Linux Foundation project that features a set of core Choirless platform components.

Getting Started

Start by reading this Rend-o-matic overview which describes the architecture and projects. We are in the process of updating the documentation for deploying the rendering capabilities locally or on IBM Cloud Functions.

You are also welcome to join our Slack in the #rend-o-matic channel for getting to know the community and discussing issues in real-time, including the creation of a network for your region.

Contributing

Please read our contributing guidelines for details of how you can get involved, and Code of Conduct for information about how to participate.

Authors

• The Choirless team - Initial work - Choirless
• Matt Hamilton
• Sean Tracey
• Glynn Bird

License

This project is licensed under the MIT License, unless otherwise stated. Separate third party code objects invoked within this code pattern are licensed by their respective providers pursuant to their own separate licenses. Contributions are subject to the Developer Certificate of Origin, Version 1.1 (DCO) and the MIT License.

renderer's Issues

Rework alignment logic to use API

Currently the librosa-powered alignment tech calculates the "offset" for each non-leader part and uses it to trim the videos, which make it to the final rendered output. The offset for each part isn't recorded anywhere and isn't tweakable.

• ensure that when a librosa-powered alignment operation is undertaken, it's calculated offset is stored using the API (see below)
• the JS renderer will take the "offset" values for each songpart from the database and save them in the render config JSON
• the Python renderer will respect the offsets it sees in the JSON file
• the trimmed bucket will be decomissioned
• if we choose to tweak an offset in the database (or via a future front-end update) then the results would be reflected in the JSON file and the rendered video.

To record the offset via the API, the following API call is required:

curl -X POST -d'{"choirId":"x", "songId":"y","partId":"z","offset":100}' -H 'Content-type: application/json' "$URL/choir/songpart"

See

• https://github.com/Choirless/choirlessapi/blob/master/API.md#post-choirsongpart
• https://github.com/orgs/Choirless/projects/3#card-43852148

Handle deletion of song parts

As laid out here: https://github.com/orgs/Choirless/projects/3#card-43852025, the front end will soon delete a song part's .webm object from the raw COS bucket when a user deletes a part.

New triggered actions are required on various buckets so that a deletion from the raw bucket triggers the subsequent deletion of all other derivative artefacts, both audio and video.

A further trigger is required so that the whole chain of deletion events causes the final video to be rendered again (without the deleted song part).

Store render status using the API

The render status API allows the serverless back end to store where it is in the render pipeline, so that the front-end knows what's going on.

Details here: Choirless/choirlessapi#53

So several serverless need amending to write to the API at the end of their execution.