A logical, reasonably standardized, but flexible project structure for doing and sharing data work.
- Python 2.7 or later
- Cookiecutter Python package >= 1.4.0: This can be installed with pip by or conda depending on how you manage your Python packages:
$ pip install cookiecutter pathlib
or
$ conda config --add channels conda-forge
$ conda install cookiecutter pathlib
cookiecutter git+ssh://[email protected]/anoru/data-ci-cd-template
or
cookiecutter https://github.com/anoru/data-ci-cd-template
The directory structure of your new project depends on your choices during the above cmd line.
-
Automate deployment and testing with databricks notebook mlflow
-
Continuous integration and continuous delivery with databricks
-
Continuous integration and delivery on Databricks using Jenkins
-
Continuous Delivery of ML-Enabled Pipelines on Databricks using MLflow
-
Databricks Labs CI/CD Templates: Automated Databricks CI/CD pipeline creation and deployment
-
Databricks Labs CI/CD Templates Tutorial - lendingclubsscoringdemo