cu-dbmi / mondaydotcom-notebooks

Center-specific notebook logic, mostly with MDC data

mondaydotcom-notebooks

Center-specific notebook logic, mostly with MDC data

Requirements

This uses the poetry tool.

Install Poetry

In my environment I run entirely "path-less", with a number of scripts for each version of Python (e.g python39.cmd, python310.cmd). After each of those scripts is run, I install poetry in each, and proceed from there. Your experience may/will vary.

pip install poetry

To work in notebooks

poetry run jupyter notebook

To run a notebook on the command line

Note, the _output folder must already exist.

# from the source folder
cd mondaydotcom-notebooks

poetry run papermill --no-report-mode --log-output "Mondaydotcom Set Task Integration Status.ipynb" "_output\Mondaydotcom Set Task Integration Status.ipynb" 

Secrets and keys

Keys for MDC and SS are kept in files called .env-<environment>, where environment is a parameter passed into the notebook. For example, the environment variable file for dev is .env-dev. These files are located in the same folder as the notebooks.

Use the Ansible playbook to create the secrets files,

ansible-playbook -i inventory/prod create_environment_file.yaml --ask-vault-pass

The vault secret is in the team LastPass.

Example .env file

MONDAY_KEY=1234124313109482304238409328402384032840238404380482390
SMARTSHEET_KEY=123980123890123980

Cleaning and Linting with Dagger

Development may be assisted using Dagger and related files within this repo. Use the following steps to get started:

1. Install Dagger
2. Open a terminal and navigate to the directory of this source.
3. Use dagger project update to populate dependencies.
4. Use the following to clean, lint, or test with Dagger:
   • Clean: dagger do clean (perform various auto-formatting on notebooks)
   • Lint: dagger do lint (perform various linting on notebooks and project)