mottaquikarim / spectacles

initial spec for how content should be defined for problems / tests

Spectacles

^ Deployed app - however will only really work for collaborators. Pls open an issue if interested in contributing to the content for this repo!

Spec and scripts for generation of practice problems.

Purpose

Although there are many independent schools teaching software development, there are few consistent formats for the creation and consumption of programming practice problems.

Spectacles is a series of scripts and a simple Express server that enforces a predefined spec for practice problem generation. It also provides a UI for quickly and easily creating this content and the required unit tests for the content itself.

Through Spectacles, we are asserting a specification for how practice problem content can and should be created. Ideally, if this spec if followed and Github is used as a single source of truth for all practice problem content, it can and would be trivial to write multiple clients that consume this data for various purposes.

For instance, on the student-facing side, we can write homework / curriculum generation services that draw on this spec to reliably pull in practice content, parse it and do whatever is necessary as required by business logic.

Additionally, having a strongly formal spec can allow others (ie: a coding school in Guatemala) to draw from this content and build custom client implementations that fit their needs, which could be different from our use cases.

Usage

Clone this repo

$ git clone https://github.com/mottaquikarim/spectacles && cd spectacles

CD into /app and run npm install

$ cd app/ && npm install

Running tasks

Run npm run-script new-problem to create new problem

$ npm run-script new-problem

Should create a /content repo with subfolder, .contentrc, etc

Running Server

Run npm run-script run-app

$ npm run-script run-app

In browser, navigating to:

http://localhost:3000/content/${UUID}

would display JSON result of file contents if directory exists.

From terminal, CURLing:

curl -X POST "http://localhost:3000/content/${UUID}" -H "Content-Type: application/json" -d '{"prompt.js": "test"}'

Would update the prompt.js key to test in file system.

When happy with content, simply git push to commit to SCM and allow others to use as well.

Example

First,

$ npm run-script run-app

(There is a pre-made one committed as of now)

In browser, go here. Expect the following:

Click on Run on top right hand corner. Expect the following:

spectacles's Issues

envvars vs envvars.sample

Currently, we have an envvars.sample that is supposed to be a template for any required / important envvars that docker environment needs to run.

As of now, this is mainly empty, project is super simple rn but if we ever wanted to pull out envvars from Travis CI and pipe into build, this would be the way to do it.

The main issue is, docker requires envvars file, not envvars.sample, the sample extension exists mainly to discourage accidental pushes of sensitive secrets to scm. This should not be a problem when using make and npm-scripts targets for various processes, but it may become an issue if one wants to run docker compose cmds directly /do something custom.

The ALTERNATIVE would be to simple commit envvars to SCM and tack on a bid ol' warning on top of file to NOT commit sensitive material.

Whatdoyathink? I can go both ways.

@mmosayed? Anyone else? (@crymall already looked at this, wanted to defer to community at large).

Design

Functional Tests Framework

• Propose a framework for handling ftests, Robot Framework is great for python - is there something similar for node?
• If not, what are the tradeoffs for using Robot Framework for this project as well...?
• Set up CI level configs required to make ftests run on every build
• Set up way for devs to run ftests locally as well

Implement tagging support

Add field to FE for user to load in, search, and apply existing or new tags to a problem. The act of adding a tag should call a separate API endpoint which updates .contentrc as appropriate.

ALSO: implement this API endpoint

Implement OAuth2.0

Replace personal access token approach with a github login

NOTE: this will require webtask and/or some other type of proxy

Write Functional tests

Given that functional tests framework is up and running, write ftests that assert:

• local express API endpoints work as expected
• scripts run as expected
• validate content folder spec requirements

Implement unit tests and functional testing framework

For unit tests, ava.js seems quite attractive. However, open to other libs as well - as long as we have a good case for why to use. We should implement unit tests across the board, for the scripts, the server and the FE

As part of this task, put together a reasonable framework for functional testing - Robot Framework is quite attractive - there may be a version for NodeJS as well. We want to functional test the API endpoints and the scripts, minimally.

Implement human readable routes for the FE

Essentially, in a meta.json associate human readable title with UUID

Set up codeclimate for test coverage

• Configure cc to track test coverage metrics
• May require istanbul / nyc npm projects to track test coverage locally
• Set up webhooks on github project to ensure that on PR and merge test coverage is correctly calculated.

On `npm run-script new-problem` or `npm run-script update-problem`, run the express app and open URL with UUID in browser

Update the app/index.js to open the FE page so user can view the content in a UI for editing upon successful completion of the new-problem and update-problem scripts

Configure the `npm run-script udpate-contentrc` command to run in Travis CI `after_success`

Ideally, we want this to only run on master. This should ensure there are no conflicts between various problems added on merge.

Consider this resource as an approach to implementation: https://gist.github.com/willprice/e07efd73fb7f13f917ea

Implement webhook to generate .compiled versions of each problem

Essentially, on git commit / push, we want to walk through all existing content items and regenerate .compiled files that concatenate all the subfiles that exist in it.

This is mainly a measure for efficiency, should we need it in the future.

As an alternative, this could be just a script that is run in CI/CD pipeline on deployment to "prod".

Implement CI/CD pipeline

Implement a CI pipeline, travis/buildkite are both good options (travis being quite popular).

There should be a testing stage and a deployment stage involved.

Update FE app to implement basic CRUD functionalities: *esp* the ability to list all created problems

If user opens up FE app without UUID query param, retrieve a reasonable number of created problems and implement pagination for the ability to browse,

Automatically create PR from Workspace

Allow user to submit a PR directly from the workspace UI

Implement the `update-problem` script

User must supply valid (ie: in existence) UUID, something like:

$ npm run-script update-problem -uuid=XXX-XXXX-XXXX

If UUID is found, open FE with content displayed, ready for updates

Set up Travis CI pipeline with dockerization

• Set up docker image to pull in latest NodeJS and run unit tests.
• Set up Travis to run these tests within pipeline.
• Set up webhooks on gitproject so that every PR kicks off Travis build as does every merge to master.

Set up simple testing framework

Mainly to unit test the scripts and utils, consider using ava.js or mocha/chai

Set up badges to indicate build status and test coverage

Badges should be added to the top of the README file

TravisCI script to assert spec is valid

only runs in content- prefixed branches

Get unit test coverage up to 100%

*SUPER IMPORTANT*: implement webhook when merging to master that recomputes the .contentrc

Otherwise, there will be length mismatches and more menacingly, lost data re: tags.

Alternatively, recompute this every time a script is run, keeping track of commit SHA or some other tracking metric.

Create static FE that consumes the express GET/POST endpoints

In public, we wish to create a simple FE app that will allow a user to edit all content per problem.

• Query param pointing to valid UUID must be passed to URL, otherwise throw error message
• If found, perform a GET request to server and display results on page
• User is able to edit and have edits stored locally in localstorage
• On save, make a POST request to update the actual files themselves