pins-python
The pins package publishes data, models, and other python objects, making it easy to share them across projects and with your colleagues. You can pin objects to a variety of pin boards, including folders (to share on a networked drive or with services like DropBox), RStudio Connect, and Amazon S3. Pins can be automatically versioned, making it straightforward to track changes, re-run analyses on historical data, and undo mistakes.
Installation
python -m pip install pins
Usage
See the documentation for getting started.
Development
Install pins with dev dependencies
python -m pip install -e .[dev]
Install pre-commit hooks
This project uses pre-commit to check and format each commit.
You can set it up by running the following code in this repo:
python -m pip install pre-commit
pre-commit install
Setting version number
This project uses setuptools_scm to
automatically track and change version numbers within the pins
package.
It works by checking the last tagged commit.
In order to set the version number, create a tag like the following.
git tag v0.0.1
In order to see the version number being used for the current commit, run:
python -m setuptools_scm
Test
Tests can be run using pytest:
pytest pins
# run all tests except those for Rstudio Connect
pytest pins -m 'not fs_rsc'
# run only local filesystem backend tests
pytest pins -m 'fs_file'
There are two important details to note for testing:
- Backends. pins can write to backends like s3, azure, and RStudio Connect, so you will need to set credentials to test against them.
- Pytest Marks. You can disable tests over a specific backend through pytest's
-m
flag. For example...- Skip S3:
pytest pins -m 'not fs_s3'
- Test only s3:
pytest pins -m 'fs_s3'
- List all marks:
pytest pins --markers
- Skip S3:
Configuring backends
- Copy
.env.dev
to be.env
- Modify
.env
to file in environment variables (e.g. AWS_ACCESS_KEY_ID) - Be careful not to put any sensitive information in
.env.dev
!
Setting up RStudio Connect tests
# Be sure to set RSC_LICENSE in .env
make dev