openfun / marsha

:clapper: An opensource LTI Learning Content Management System (LCMS)

Marsha, a self-hosted open source video and document provider 🐠

Overview

Marsha is a video management & playback service. It is intended to be operated independently: it's like having your very own YouTube for education.

Marsha also supports hosting documents and distribute them on all your courses.

Instructors & organizations can use Marsha to upload and manage their videos (and associated files, such as subtitles or transcripts) or documents directly from a course as they are creating it.

Once the course is published, learners simply see a video player or documents in the course.

This seamless integration works with any LMS (Open edX, Moodle, ...) thanks to the LTI standard for interoperability.

Here is what Marsha offers out of the box:

Video:

• automatic transcoding of videos to all suitable formats from a single video file uploaded by the instructor;
• adaptive-bitrate streaming playback (both HLS and DASH);
• accessibility through the player itself and support for subtitles, closed captions and transcripts;

Document:

• upload any type of documents;
• prevent disk storage quota by using AWS S3;

Moreover, Marsha provides:

• access control to resources through LTI authentication;
• easy deployment & management of environments through Terraform;

Architecture

Marsha is made up of 3 building blocks: a container-native Django backend, an AWS transcoding and file storage environment, and a React frontend application.

The Django backend

The Django backend is tasked with serving the LTI pages that are integrated into the LMS. It also manages all the objects with their relationships, user accounts and all authentication concerns. It exposes a JSON API to communicate with the part of the infrastructure that operates on AWS lambdas and the React frontend.

It is defined using a docker-compose file for development, and can be deployed on any container environment (such as Kubernetes) for production.

The storage & transcoding environment

Source files (video, documents, subtitles,...) are directly uploaded to an S3 bucket by instructors. Depending the uploaded resource a lambda will be triggered to do different jobs:

• Launch MediaConvert to generate all necessary video files (various formats and fragments & manifests for adaptive-bitrate streaming) into a destination S3 bucket. Those files are then served through the CloudFront CDN.
• Convert any kind of subtitles (also captions and transcripts) in WebVTT format and encode them properly.
• Resize thumbnails in many formats.
• Copy documents from a source to a destination S3 Bucket accessible through the CloudFront CDN.

Lambdas are used to manage and monitor the process and report back to the Django backend.

This storage & transcoding environment requires AWS as it heavily relies on AWS MediaConvert to do the heavy lifting when it comes to transcoding. All the services it relies on are configured through Terraform and can be deployed effortlessly through a make command.

⚠️ Privacy concerns

Please note that the only objects we handle in AWS are the actual video, documents or subtitles files, from the upload to the distribution through transcoding and storage. It is not required to deploy any database or application backend to AWS or send any user's personal information there.

The React frontend

The React frontend is responsible for the interfaces with which users interact in the LTI Iframes. It gets an authenticated token with permissions from the view and interacts with the Django backend to manage objects and directly with AWS s3 to upload files.

It also powers the same resource view when loaded by a learner to display a video player (thanks to videojs) or a document reader.

⚠️ Iframe management

To have the best possible user experience for instructors, we need to be able to change the size of the <iframe> depending on its contents. This can be done through the iframe-resizer library.

iframe-resizer requires to run some JS inside the <iframe> (which we include with our React frontend bundle) and some JS inside the host page. It then communicates through message-passing to adjust the size of the <iframe>.

This means that to have the best interfaces for instructors, you need to include the host-side iframe-resizer JS in your LMS pages. For Open edX, this is already done in our custom LTI consumer Xblock.

If you cannot or do not want to include this host-side JS, you can still run Marsha. It will work exactly the same for learners (provided you adjust the size of the LTI <iframe> for video), and instructors will only have to scroll inside the <iframe> on some occasions.

Getting started

Make sure you have a recent version of Docker and Docker Compose installed on your laptop:

$ docker -v
  Docker version 19.03.6, build 369ce74a3c

$ docker-compose --version
  docker-compose version 1.24.1, build 4667896b

⚠️ You may need to run the following commands with sudo but this can be avoided by assigning your user to the docker group.

The storage & transcoding environment

All tasks related to this environment are run from the ./src/aws directory. We use Terraform to keep this infrastructure configuration as code and easily manage several independent deployments of the whole AWS infrastructure.

Note for Mac users: Marsha's AWS development setup uses getopt. The version that comes with macOS is not suitable for our use case. You need to install the GNU version and add it to your path so it is used by default.

brew install gnu-getopt

echo 'export PATH="/usr/local/opt/gnu-getopt/bin:$PATH"' >> ~/.zshrc

There are 2 Terraform projects in Marsha with two different purposes:

• ./src/aws/shared_resources: this project manages resources common to all marsha environments on the same AWS account. These resources must not live in different workspaces so you must work in the default workspace. To ease the use of this project, a dedicated script is available in ./src/aws/bin/shared-resources which uses and configures the Terraform docker image. You have to run a Terraform command as if you were using the terraform cli. (eg: ./bin/shared-resources plan will execute Terraform's "plan" command).
• ./src/aws: this is the main project we use, most of the infrastructure is managed here (in all *.tf files). This project must use Terraform workspaces and we highly recommand you to not use the default one. With multiple workspaces, you can manage multiple environments for your Marsha instance with a single AWS account. To ease the use of this project, a dedicated script is available in ./src/aws/bin/terraform which uses and configures the Terraform docker image. You have to run a Terraform command as if you were using the terraform cli. (eg: ./bin/terraform plan will execute Terraform's "plan" command).

Terraform state management

Terraform manages a state of your infrastructure. By default this state is stored locally on your machine but it is highly recommanded to use a remote backend.

You will find all you need to configure a remote backend in the Terraform documentation: https://www.terraform.io/docs/configuration/blocks/backends/index.html

⚠ You must configure your state management before running any of the commands hereafter. The first init will initiate your state and after that you will have to deal with state migration if you want to modify it. You can create a file src/aws/state.tf and src/aws/shared-resources/state.tf to configure a backend, there is an example in each project (state.tf.dist file).

🔧 Before you go further, you need to create ./src/aws/env.d/development and replace the relevant values with your own. You can take a look at the environment documentation for more details on this topic. You should use this command to create the file from the existing model:

$ cp ./src/aws/env.d/development.dist ./src/aws/env.d/development

Initialize your Terraform config:

$ cd src/aws
$ make init

The make init command will also create an ECR repository. Before going further you have to build and publish the lambda docker image. Unfortunately AWS doesn't allow to use a public image, so you have to host this one on a private ECR instance. Copy the output of the init command, you will use them in the next step.

Build and publish the lambda image

For this step, we cooked a script to help you build, tag and deploy images. All the scripts are run from the marsha root directory.

🔧 Before you go further, you need to create ./env.d/lambda and replace the relevant values with your own. The ECR url is available in the shared_resources terraform output you copied earlier. You should use this command to create the file from the existing model:

$ cp ./env.d/lambda.dist ./env.d/lambda

You have to successively run these commands :

Build the image:

$ ./bin/lambda build

Tag the image:

$ ./bin/lambda tag

And then publish it:

$ ./bin/lambda publish

Apply all terraform plans

Terraform is split in two parts. The main one, directly in src/aws can work on multiple Terraform workspaces. You will use this feature if you want separate environments (development, staging, preprod and production). We also need some resources available across all workspaces. For this we have an other terraform in src/aws/shared_resources. To apply all plans at once run this command in the src/aws directory.

$ make apply-all

Everything should be set up! You can check on your AWS management console.

You may have noticed that the AWS development environment requires a URL where the Django backend is running. You can easily get a URL that points to your locally running Django app using a tool such as ngrok.

The Django Backend

All tasks related to the Django backend are run from the project root (where this README.md is located).

The easiest way to start working on the project is to use our Makefile:

$ make bootstrap

This command builds the app container, installs back-end dependencies and performs database migrations. It's a good idea to use this command each time you are pulling code from the project repository to avoid dependency-related or migration-related issues.

🔧 Before you go further, you should take a look at the newly created ./env.d/development file and replace the relevant values with your own. You can take a look at the environment documentation for more details on this topic.

Now that your Docker services are ready to be used, start the application by running:

$ make run

You should be able to view the development view at localhost:8060/development/.

The React frontend

All tasks related to the React frontend are run from the ./src/frontend directory.

We use yarn for all those tasks. Make sure you have a recent version installed:

$ yarn --version
  1.13.0

If you need to install yarn, please take a look at the official documentation.

Install all the dependencies:

$ yarn install

The LTI frontend

Run the build and copy the iframe-resizer host-side JS into your local Django assets:

$ yarn build-lti
$ yarn copy-iframe-resizer

The "Standalone" frontend

Run the build for libraries and start the standalone frontend:

$ yarn start-site

Now the standalone frontend is available at localhost:3000.

Testing frontend

The front application is tested using jest. Every js module has its corresponding spec file containing related tests:

├── VideoPlayer
│   ├── index.spec.tsx
│   ├── index.tsx

Run the tests (this will run all tests: for both LTI and standalone frontends and libraries):

$ yarn test

Browser testing provided by:

🗝 Before you go further, you need to create a Consumer Site and Passport in Marsha's admin panel.

You should be all set to make the LTI request on the development view and access Marsha's frontend interface!

Contributing

This project is intended to be community-driven, so please, do not hesitate to get in touch if you have any question related to our implementation or design decisions.

We try to raise our code quality standards and expect contributors to follow the recommandations from our handbook.

License

This work is released under the MIT License (see LICENSE).

marsha's Issues

statics js chunks are not served from cloud-front when AWS is configured to managed statics

Bug Report

Expected behavior/code

In environment allowing to manage statics with AWS (staging, preproduction and production), every statics files should be fetched on the cloud-front CDN

Actual Behavior

First the index.js file is fetched on cloud-front, then every chunks are fetched on the current domain name:

Steps to Reproduce

1. Deploy marsha in an environment listed before using AWS to manage statics
2. Monitor networks in your browser and you will se that every chunk are in 404

Possible Solution

In webpack we use the output.publicpath option to change the static files paths but in the documentation it is also written that when you use a CDN, the CDN URL should be use, but in our case it's not possible.

I don't really know if it's a webpack config issue or if it is related to the manifest generated by the backend during the collecstatic job.

Use frontend URL fragments from variables

Purpose

Right now we're using strings directly (eg. /errors/lti, /errors/:code) to define our URLs and link to them.

This works as it uses react-router's low-level API at its simplest. However, it suffers from two issues:

• It's not DRY, which means it's prone to bugs when a developer changes the magic string (or URL fragment) in one place and forgets to change it accordingly in another
• All the strings are in English and cannot be localized easily (this is a smaller concern for marsha right now as we operate in iframes and do not need to translate URLs)

Proposal

Use machine names (imported variables) wherever we reference a URL. This machine name points to names in any number of languages and ensure the actual URL fragments only exist in one place.

AWS MediaConvert limits us to 100 presets, fails workspace deployment

Bug Report

Expected behavior/code
We should be able to create as many workspaces & deployments as we want (or as we can pay for 😅).

Actual Behavior
make apply fails with an amazon error that warns of an exceeded limit of 100 presets. As we have 21 presets (per workspace) as of now, this limits us to 4 simultaneous workspaces.

Steps to Reproduce
In the terraform folder with everything initialized:

• docker-compose run --rm app /bin/terraform workspace select preset-limit-example
• make apply

Environment
N/A

Possible Solution
I think we could come up with a system to reuse presets across workspaces. In practice, they don't change that often, so we should be able to work within the limit if we can reuse them.
The danger there is that we must be sure we never accidentally delete presets used in production when we deploy other environments.

Additional context/Screenshots

Error: Error applying plan:

1 error(s) occurred:

* data.aws_lambda_invocation.configure_lambda_presets: data.aws_lambda_invocation.configure_lambda_presets: Lambda function (preset-limit-example-marsha-configure) returned error: ({"errorMessage":"Your account already has the maximum number of presets: maximum=100.","errorType":"TooManyRequestsException","stackTrace":["Object.extractError (/var/task/node_modules/aws-sdk/lib/protocol/json.js:48:27)","Request.extractError (/var/task/node_modules/aws-sdk/lib/protocol/rest_json.js:52:8)","Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:105:20)","Request.emit (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:77:10)","Request.emit (/var/task/node_modules/aws-sdk/lib/request.js:683:14)","Request.transition (/var/task/node_modules/aws-sdk/lib/request.js:22:10)","AcceptorStateMachine.runTo (/var/task/node_modules/aws-sdk/lib/state_machine.js:14:12)","/var/task/node_modules/aws-sdk/lib/state_machine.js:26:10","Request.<anonymous> (/var/task/node_modules/aws-sdk/lib/request.js:38:9)","Request.<anonymous> (/var/task/node_modules/aws-sdk/lib/request.js:685:12)"]})

Implement CSP in marsha response

Feature Request

To increase the security and prevent XSS Content Security Policy can be implemented to control which dynamic resources are allowed to load.

Describe the solution you'd like

We have to list all the dynamic resources uses in marsha and see if we want to distinguish resources (allow by resource type like media, style, script, ...) or the same rule for every resources.

A django middleware exists to help the implementation of CSP: https://github.com/mozilla/django-csp

Do you want to work on it through a Pull Request?

I can yes.

create a env.d/development.dist and env.d/terraform_state.dist files in aws repo

Purpose

There is no env.d directory in src/aws when you clone the project and we have no idea what file should be present and what values they should contain.

Proposal

• create a repository env.d in src/aws
• create two files development.dist and terraform.dist
• add fake values
• add a relevant comment for each variable
• document the environment configuration step

Admin users can create invalid passports by failing to fill either a consumer site or a playlist

Bug Report

Expected behavior/code
When creating a passport, the admin user must link it to EITHER a consumer site OR a playlist.

Actual Behavior
It is today possible technically to create a passport linked neither to a consumer site nor to a playlist. Doing so generates 500 errors on launch requests trying to use this passport.

Steps to Reproduce

1. Create a new passport from the admin and link it neither to a consumer site, nor to a playlist
2. Make a launch request using this passport's credentials
3. And then the bug happens!

Environment

• Marsha version: 1.0.0

Possible Solution
We should solve this problem at its root: add a custom form validation to the passport admin update view and check that either the consumer_site or the playlist fields are filled.

Keep only one docker-compose file

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.

Since our recent circle-ci config refacto, we don't use anymore test version of docker-compose and we think that the alpine version can be also removed.

We should try to keep only one docker-compose file and refacto it to handle cases where we have to switch between environment.

Describe the solution you'd like

• remove test docker-compose file
• remove alpine docker-compose file
• keep only one docker-compose at the root.

Discovery, Documentation, Adoption, Migration Strategy

Look at the work made on fun-mooc repo

Keep alpine images ?

Topic

Alpine images are not used at all but we are maintaining them. Sometimes we miss to maintain the Dockerfile and it's the only remaining usage of docker-compose in our CI.

Question

Should we continue to maintain alpine images ? Or should we remove them ?

[xAPI] introduce a configurable completion threshold

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.

completed xAPI event is fired when the video completion threshold reaches 100%, but in some cases, like the last 5 seconds of a video is a black cross-fade or end credits, a user will probably stop the video, the completion threshold will not be reached and the completed event not fired while he completed the video.

@jmaupetit suggested to introduce an arbitrary completion threshold smaller than 100%, something like 96% and iterate to determine if this completion threshold is relevant or not.

Describe the solution you'd like

Find a way to configure the completion threshold value and fire the completed event if this ratio is reached.

Consider moving the gitlint script to bin/

Purpose

We use a dedicated script to lint git commits. It is currently located in gitlint/gitlint_emoji.py. I think we might consider moving it to the bin/ directory as it's a project tool.

Proposal

• git mv gitlint/gitlint_emoji.py bin/
• update .gitlint configuration to add bin to the extra-path

Rework Marsha's styles for iframe resizing

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.
We have several use cases that require (or are at least improved by) resizing the LTI iframe to fit the content:

• video downloads;
• transcripts;
• instructor views.

It's also useful to have a clean experience on smaller screens.

Describe the solution you'd like
We can use the iframe-resizer JS library to open a communication channel between the consumer & marsha, and some hard-coded ratios & heights in marsha to get the best possible dimensions.
Ideally, we'd have a fallback to "responsive" dimensions inside the iframe when the resizing does not happen for whatever reason (so marsha can work in a degraded state as a bare LTI service).

Describe alternatives you've considered
We considered developing this channel ourselves but iframe-resizer does exactly what we need. The only other way would be to drop the LTI iframe and use a script instead, which we are not ready for.

Discovery, Documentation, Adoption, Migration Strategy
This would be transparent for students as well as instructors: the content would be deployed in the best way possible without them having to specify anything.

Do you want to work on it through a Pull Request?
🤷‍♂️

---

Progress

• implement the iframe-resizer in marsha;
• adapt Marsha's styles for the ideal case;
• implement the host iframe-resizer js in xblock-configurable-lti-consumer;
• check origin/referrer at the LTI passport level & whitelist passport domain in iframe-resizer;
• adapt Marsha's styles for the fallback case.

[xAPI] do not mix completed and paused statements

Purpose

The current implementation of the paused event callback introduces a bias in the paused statement statistics: one should not send a paused statement and a completed statement if the player triggers a paused event, but only a completed statement if and only if it has not been sent yet.

Proposal

Check the completion status (progress) before sending the paused statement and do not send a paused statement if the completed statement has already been sent.

Disable text track if language and mode already exists

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.

The backend does not allow to create a timed text if it already exists for the same language and mode.
In the dashboard it is still possible to select the language even it already exists.

Describe the solution you'd like

Remove in the select box every langauage already created for each mode.

Remove dependency to the pylti library

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.
PR #129 fixed the LTI verification to work behind a TLS termination proxy. It was a necessary quick fix that underlined the fragility of the pylti library. It is not doing much for us so we should remove this dependency and use the oauth2 library directly.

Describe the solution you'd like
Remove dependency to pylti and directly use the oauth_server "verify_request" method in the oauth2 library. It requires massaging our request and parameters before calling the library.

Describe alternatives you've considered
Open edX LTI xblock consumer is not using the oauth2 library but oauthlib. We could consider using it instead.

Discovery, Documentation, Adoption, Migration Strategy
Add tests to securize the signature calculation including tests on the proto proxy header, the oauth timestamp,...

Do you want to work on it through a Pull Request?
👍

Link existing thumbail to the player

Bug Report

Expected behavior/code
The player must start as fast as possible and show a thumbnail of the video.

Actual Behavior
Apparently the player does not show a thumbnail we link to it, but something it computes after downloading the first video fragment. This is taking a lot of time with a slow connection.

Environment
Marsha v2.2.1

Possible Solution
The URL of a thumbnail that was pre-computed by AWS is included in the video document that we include in the DOM of the LTI view (or on the video detail API endpoint). Can we link this thumbnail directly to the player?

Use LTI launch_presentation_locale parameter

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.

For now we have no idea about which locale to use in marsha. We use by default english locale and we have nothing to determine which local to use.

Describe the solution you'd like

in the LTI request there is a parameter launch_presentation_locale containing the locale we should use:

launch_presentation_locale=en-US
Language, country and variant as represented using the IETF Best Practices for Tags for Identifying Languages (BCP-47) available at http://www.rfc-editor.org/rfc/bcp/bcp47.txt

Discovery, Documentation, Adoption, Migration Strategy

documentation about this parameter is available in the LTI spec, in the Basic Launch Data chapter (there is no anchor to link it) => http://www.imsglobal.org/specs/ltiv1p1p1/implementation-guide#toc-2

Fix xAPI "completed" event implementation

Purpose

After digging a bit into it, it appears that there was a confusion between the "completed" and "terminated" events. While the first one means that the actor has seen every part of a video, the later one means he has reached the end of it.

Reference: https://liveaspankaj.gitbooks.io/xapi-video-profile/statement_data_model.html

Proposal

• Move the current "completed" event name to "terminated"
• Implement the "completed" event (requires to maintain a viewed segments list in the application state)

Rename lambda-update-state to use a more appropriate name

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.
lambda-update-state is not the ideal name as the encode lambda also does a state update. The real purpose of this lambda is to report the result of transcoding. The name should match that.

Describe the solution you'd like
lambda-report-result
lambda-finish-encode

Describe alternatives you've considered
N/A

Discovery, Documentation, Adoption, Migration Strategy
No impact. Once the rename is complete everything should work as it does now.

Do you want to work on it through a Pull Request?
🤷‍♂️

Add subtitles

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.

We need to activate subtitles and transcripts on videos. This is already supported in the backend but needs to be added in the frontend and in AWS.

We need to transparently allow many subtitle/transcript format sources. At the minimum srt and vtt.

Describe the solution you'd like

👉 for the instructor:

• After downloading a video, the instructor can choose to upload a subtitle file and/or a transcript file,
• The client retrieves an upload policy for this file the same way as it does for videos,
• The file is uploaded directly to Amazon S3 thanks to the upload policy,
• Uploading the file triggers a lambda that converts the subtitle/transcript file to .vtt. I found a library that claims to do this well: https://github.com/papnkukn/subsrt,
• The lambda calls Django to update the status of the subtitle/transcript, the same way as it does for videos.

👉 for the student:

Subtitles are integrated in the video by the player.
Transcripts are shown below the video (several lines visible) and scrolls in sync with the video.

On the backend, we must add a flag is_transcript on the Subtitle object that can be set by the frontend when creating it. This flag should appear in the content of subtitles rendered by the API.

Describe alternatives you've considered

I proposed transcripts to be .vtt files so that we can synchronize scrolling with the video but I think we should also allow downloading simple text files for transcripts for instructors who don't have a .vtt version of it.

Another way we could make it work is by respecting subtitle tracks that may be multiplexed inside the source video. The current encoding presets are just ignoring them... so they are lost after encoding. Allowing this would require Amazon to either:

• respect the subtitles so they stay in the encoded videos and will be available in the player,
• or extract them, convert them to .vtt and create a subtitle object in Marsha via the API.

Add xAPI events

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.
We want the player to emit xAPI events.

Describe the solution you'd like
Add xAPI eventing to the video player. Quick braindump:

• Make the eventing module generic so that the important parts are shared when Marsha becomes multiplayer.
• We should probably use Learning Locker for a POC.
• I found this initiative to try to standardize video events: https://github.com/liveaspankaj/xapi-video-cop

s3 bucket for terraform state region is harcoded

Bug Report

Expected behavior/code

When I create the bucket to store the terraform state I want to choose the region where it will be created

Actual Behavior

The aws region is harcoded in src/aws/create_state_bucket/variables.tf. The value is eu-west-1

Possible Solution

Use the environment variable aws_region in env.d/aws/env.d/state_bucket

Rename/move around thumbnail(s) props on video

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.
Video objects have a thumbnail property that can contain a Thumbnail object, and a thumbnails property inside their urls object that contains urls to a default thumbnail generated automatically from the video.

This can be confusing for developers unfamiliar with the project.

Describe the solution you'd like
Move video.urls.thumbnails to video.thumbnail_default_urls. This should make things clearer.

Describe alternatives you've considered
N/A

Discovery, Documentation, Adoption, Migration Strategy
N/A

Do you want to work on it through a Pull Request?
🤷‍♂️

`jwt` in application state typings is inaccurate

Bug Report

Expected behavior/code
jwt as contained in the RootState and AppData should not be Nullable when the global state is not ERROR (that is, when the LTI request was not a failure).

Actual Behavior
jwt is always Nullable.

Steps to Reproduce
N/A (always present)

Environment

• Marsha version: #master

Possible Solution
Improve the types to reflect the constraint that jwt is non-Nullable when the app state is not ERROR.

Additional context/Screenshots
N/A

Make ABR work with iOS devices

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.
Support for HLS was removed in v2.5.0 because it was not working properly and resulted in videos not being visible at all on iOS devices.

Removing it allows iOS devices to view the mp4 version without ABR.

We now need to find a solution to make ABR work on iOS devices.

Describe the solution you'd like
Test HLS with different encodings and manifests formats.

Remove "Go to dashboard" button on read_only video.

Feature Request

When an instructor is watching a video in read_only mode the button Go to dashboard is disable. We would like to remove it.

Describe the solution you'd like

Remove the button when the video is in read_only mode.

Unused Video.position field ?

Topic

In the video model there is a position field. The documentation describe this field as the video position in the playlist it belongs.

I check and the position is never set and is always equal to 0.

Question

Is there any reason to keep this field?

Choose a side effect model library

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.

We have more and more side effect in the application, when an API call is made we want to reflect a possible state change in the application. As we already have redux installed we could install a middleware responsible of this part.

Describe the solution you'd like

There are two well known library doing this job:

• saga
• thunk

I have no preference between both. Just saga is harder to use from my point of view. Thunk has lot of "plugin" doing the job, at the end you write less code.

They both have typescript definition.

Check LTI role

Bug Report

This is not really a bug report but more a reminder to check if we have a bug or not.

Expected behavior/code

As a pedagogic team member I can create and modify a video.

Actual Behavior

We have to check if the behavior described before is good, maybe only admin member can modify a video.

Steps to Reproduce

1. Create a user with pedagogic team accesses
2. Try to create and then modify a video.
3. Check what role is sent by OpenedX and eventually add it to the roles allowed to modify a video.

Add an audio player

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.
Marsha does not propose an audio player yet.

Describe the solution you'd like
Instructors should be able to:

• upload an audio file in whatever format, which Marsha will then encode with optimal size and quality for the web, in a best practice format to be defined,
• upload a transcript file (bonus: generate the transcript file automatically via speech to text),
• [nice to have] define chapters.

Students should be able to listen to the audio file and view the transcript (similarly as what is currently done on the video player). We should probably use the same Plyr player that we already use for video as it also supports audio...

Add Staging, PreProduction & Production settings

Purpose

At the time of writing, available configurations for Marsha are: Development and Test. We need more settings for missing environments.

Proposal

• Add Production configuration
• Derive Staging and Preproduction configurations from Production

Warn user when an upload is on-going and would be lost

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.
When an instructor starts uploading a video from the LMS with Marsha in an iframe, and changes page in the LMS, the upload is stopped and lost without warning.

Describe the solution you'd like
It would be nice if we can force a warning in the LMS from within the iframe that tells the user he has an on-going upload that would be lost if he confirms changing page.

Describe alternatives you've considered
It would be even nicer if the upload could continue in the background even when we change page...

Allow to search in active transcript

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.

As a user I would like to search occurences in the active transcript. And I would like to do this with a search field.

Describe the solution you'd like

Add a search field in the interface when a transcript is activated.

Add support for chapters in videos

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.
We would like to allow instructors to define chapters in a video and viewers to easily jump to the beginning of a chapter.

Describe the solution you'd like
For instructors, it seems that video chapters are supported in webVTT (see https://www.w3.org/TR/webvtt1/#introduction-chapters). This may be a way to define chapters... but some questions remain: how do we make it work regardless of whether subtitles are activated? Is it a specific type of TimeTextTrack? It is not very user friendly to define chapters in a text file that is uploaded alongside the video.... we should probably provide a Ux to add timestamps for a chapter... a form with start_stamp and name (think of i18n) ?

For viewers, we need to include a navigation button in the player UI similar to what is done in this demo: https://codepen.io/team/rcrooks1969/pen/LJBdPJ

Rename /fun to /marsha in CI

Purpose

In CircleCI, the working directory is called /fun. There should not be any mention of FUN as this project is open source and can be used by others.

Proposal

Rename the working directory from /fun to /marsha

Improve marsha's structure to make it easier to approach

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.
Right now marsha is not easy to approach for new developers getting on the project.
Obviously the main thing is that we're missing docs, but the file structure with 14 directories & 25 files at root level might not be ideal there.

Describe the solution you'd like
marsha is one project made up of one python project, one javascript frontend project & 3 javascript lambda projects. I propose that each of these should live in its own folder under src/.

We can keep all the READMEs & docs as well as global project files (eg. docker, git, ci & maybe terraform) at root.

Here's the proposed structure:

marsha/
    .circleci/
    bin/
        gitlint
        ...
    docker/
    docs/
    env.d/
    src/
        backend/
            marsha/
                core/
                static/
                __init__.py
                settings.py
                ...
            manage.py
            pylintrc
            setup.cfg
            setup.py
        frontend/
            __mocks__/
            node_modules/
            src/
                components/
                index.tsx
                ...
            .babelrc
            .prettierrc
            jest.config.js
            package.json
            tsconfig.json
            tslint.json
            webpack.config.js
            yarn.lock
        lambda-configure/
            node_modules/
            src/
            jest.config.js
            package.json
            yarn.lock
            ...
        lambda-encode/
        lambda-update-state/
    terraform/
    .dockerignore
    .gitignore
    .gitlint
    CHANGELOG.md
    CONTRIBUTORS.md
    Dockerfile
    LICENCE
    Makefile
    README.md
    docker-compose.yml

Describe alternatives you've considered
Using this structure, I could see terraform being in src/.

Alternatively, we could but backend, frontend, lambda-configure, lambda-encode and lambda-update-state at root.

Discovery, Documentation, Adoption, Migration Strategy
The main adoption thing is that developers generally need to be in the relevant folder to work on any part of the project, as is already the case for lambdas & terraform.

Do you want to work on it through a Pull Request?
👍

Write a tool to migrate from videofront to marsha

Purpose

We need to migrate some courses from https://github.com/openfun/videofront to https://github.com/openfun/marsha.

Ultimately, we also want to use this to massively migrate fun-mooc.fr to marsha.

Proposal

The tool could work on the tar.gz export of a course:

• untar the archive,
• scan the course looking for the libcast Xblocks and retrieve the video_id and the xblock_id,
• with the video_id, download the video source file from VideoFront's Amazon S3 bucket,
• simulate a POST LTI launch request to marsha (with the xblock id retrieved from the course, the course key and the domain name of the site). This has the effect to create a video object in marsha and returns a JWT Token. See this test for an example,
• get an upload policy from the marsha API (/api/video/{uuid}/upload_policy) passing the JWT Token in headers. See this test for an example,
• upload the video retrieved from VideoFront to S3 with the credentials contained in the upload policy delivered by marsha,
• marsha is ready. Now modify the course file in the archive to refer to our new LTI xblock,
• tar the archive,
• import the archive to the new Open edX instance in which the LTI Xblock is installed.

In order to massively migrate fun-mooc.fr to marsha, we can repeat this procedure and modify the xblock directly in Mongodb instead of exporting/re-importing courses.

logo contribution

Hello community! In my free time I contribute to open source softwares by designing logos. I think this project doesn't have a logo. So i have a logo idea that will represent the function of the project. If you would like to see this, please do a feedback. Best regards...

Refactor django permissions

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.

In PR #234

I think the permission classes IsRelatedVideoTokenOrAdminUser and IsVideoTokenOrAdminUser have bad names. They are looking for Instructor tokens as opposed to the IsVideoToken class which accepts students.

Describe the solution you'd like

Should we call them something like IsVideoInstructorTokenOrAdminUser et IsVideoRelatedInstructorTokenOrAdminUser?

Export only connected components

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.

In our code base, when a component is connected to redux we have the standalone component in a directory and the connected in an other one, like this:

.
├── AppRoutes
│   └── AppRoutes.tsx
├── AppRoutesConnected
│   └── AppRoutesConnected.tsx
├── Dashboard
│   ├── DashboardInternalHeading.tsx
│   ├── Dashboard.spec.tsx
│   ├── Dashboard.tsx
│   └── route.ts
├── DashboardConnected
│   ├── DashboardConnected.spec.tsx
│   └── DashboardConnected.tsx

We have this architecture to test in one hand the standalone component and in the other hand just to test the mapStateToProps and/or mapDispatchToProps.

Since we decided to use the react-testing-library we would like to test like a user, so the connected component directly.

Describe the solution you'd like

We would like to export only the connected component. By doing this we can remove this dual directory and only keep the one this the component name. We can also remove the component file in index.tsx, we will have shorten and more readable imports.

At the end we will have an architecture like this:

├── VideoPlayer
│   ├── index.spec.tsx
│   ├── index.tsx
│   ├── route.ts

The file index.tsx contain the component and its connect. Only the connect is exported with the name of the component. When we want to use it, we simply import and use it like this:

import { VideoPlayer } from '../VideoPlayer';

...
<VideoPlayer
  video={context.ltiResourceVideo}
  createPlayer={createPlayer}
/>
...

Discovery, Documentation, Adoption, Migration Strategy

There is a PR as a PoC (#371) where it's possible to see how the change will be made

List of components to refactor:

• AppRoutes
• Dashboard
• DashboardThumbnail
• DashboardThumbnailProgressConnected (just rename it I think)
• DashboardTimedTextPane
• DashboardVideoPane
• DashboardVideoPaneDownloadOption
• DashboardVideoPaneProgessConnected (just rename it I think)
• InstructorView
• InstructorWrapper
• RedirectOnLoad
• TimedTextCreationForm
• TimedTextListItem
• TranscriptReader
• Transcripts
• UploadForm
• VideoPlayer

flake8 is not run correctly on circle-ci

Bug Report

Expected behavior/code

flake8 should be running from src/backend directory or the --configuration should be used with the correct configuration file.

Actual Behavior

flake8 configuration is in src/backend and flake8 is run from the root project so the configuration is not used.

Integrate sentry

Purpose

Sentry is a great tool to track and report application errors. Looks like a requirement for any Django project.

Proposal

Add raven as a project requirement and configure sentry.

Reference documentation: https://docs.sentry.io/platforms/python/django/

Adways service support

Topic
Current video player supports Adways service to add interactive content to course video. This is achieved by storing Adways interactive content ID in video xblock and by loading a specific JS library.
Before migrating fun-mooc.fr website to Marsha we'll have to decide how this support should continue.

https://github.com/openfun/libcast-xblock/blob/10cbe572d854d9903d03f6318cbe5203854eb606/libcast_xblock/public/js/videofront.js#L4

Merge marsha's env.d/development & development_secret files

Purpose

Two environment files are currently required to bootstrap a development environment to work on:

• env.d/development
• env.d/development_secret

The first one is committed in the repository and the second one is not. I think this can be improved with a single environment file (e.g. env.d/development) that is committed as a template (with fake values and appropriate comments).

Proposal

• remove the env.d/development file from the repository
• merge those two files as env.d/development.template
• add fake values
• add a relevant comment for each variable
• document the environment configuration step

Allow deactivating signed URLs in CloudFront

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.
In Django there is a setting to deactivate CloudFront signed URLs but this is of no use if we don't give the possibility to deactivate this protection on the CloudFront distribution.

Describe the solution you'd like
Add a setting in Terraform to deactivate signed URLs on the CloudFront distribution. It should consist of removing this line: https://github.com/openfun/marsha/blob/master/src/aws/cloudfront.tf#L53.

Alternative solution
Consider that there is no reason to deactivate signed URLs and remove the DJANGO_CLOUDFRONT_SIGNED_URLS_ACTIVE setting that allows to do so in Django.

Download transcript files

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.

In the same way we can download videos, we would like to download the transcripts associated to the video.

Describe the solution you'd like

When a transcript is activated, a link to download it can be added. If we click on it the file should be downloaded.

Describe alternatives you've considered

• download an archive containing all timed text tracks
• download an archide containing all transcripts
• create a list with all downloadable timed text tracks. Each link start the download of the associated file.

Do not enforce node engine version in lamdba's package.json

Purpose

Lambda's package.json restrict their node engine compatibility even if they may be compatible with higher major releases.

Proposal

Remove the engines section from lamdba's package.json file.

Videos are not displayed on iPhone

Bug Report

Expected behavior/code

On iphone I should see the video

Actual Behavior

The video is not loaded

Steps to Reproduce

1. Take an iphone
2. Try to open marsha on it
3. You won't see the video

Possible Solution

In <VideoPlayer /> we should check the compatibility before loading dashjs

Improve i18n process

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.

The i18n process is not really easy to use, especially in the react application. We have a mapping made in front application that can be easily broken.

Describe the solution you'd like

Copy what we made in richie by creating a more reliable mapping in the backend application

Upload a custom thumbnail

Feature Request

Is your feature request related to a problem or unsupported use case? Please describe.

From PR #242

The feature that universities have already requested is to be able to upload their own image to be used as a poster in front of the video. That's one of the important functionalities we should add next.

Describe the solution you'd like

In the dashboard we can upload a custom thumbnail, this new thumbnail will replace the AWS computed thumbnail.

Update CONTRIBUTORS at each new release

Purpose

The CONTRIBUTORS.md file seems outdated.

Proposal

• add this file update to the release process documentation