neuroscout / pyns

Python neuroscout API library

pyNS 🌲

pyNS is a python library to easily interact with the Neuroscout API.

pyNS enables advanced used cases not supported by the neuroscout.org web-based analysis builder, such as batch-creation of analyses, meta-analytic applications, as well as supporting the operations of the neuroscout-cli

Installation

Use pip to install it:

pip install pyns

Documentation 📚

Full documentation can be found at readthedocs.

For API docs, check out the Swagger API UI.

For a tutorial see this jupyter notebook example.
You can also follow the Neuroscout Paper's analyses in this interactive jupyter book.

Testing

We use pytest for testing, and betamax to record HTTP requests used in tests.
To re-run tests locally set theUSER_TEST_EMAIL and USER_TEST_PWD environment variables with valid API credentials.

pyns's Issues

Add generic handler for run id identifier, to avoid manual querying of runs

Add decorator to avoid having to specify run_ids for calls such as predictor-events predictors etc... and it queries and find the appropriate ids for you

search user predictors in `create_analysis`

create_analysis only searches public predictors, not private user uploaded predictors. Add this ability.

see: neuroscout/neuroscout#812

Don't add 'Subject' level to model if only 1 sub per run

E.g. HealthyBrainNetwork

upgrade requests to latest version

older version seemed to have SSL cert issues

Decide if/how to handle flask/requests Client differences

One idea with this library is that it could also be used internally in neuroscout to test the API itself.
In fact, this library is so far based on the internal flask client wrapper I use in the neuroscout tests.

However, one difficulty is that the flask client (which is used to test in flask without testing the live URL), is slightly different from requests.

We need to decide if its worthwhile to design the library to deal with both clients. There may be a way to subclass the client over in the main neuroscout repo to deal with this, but push off the responsibility over there so that this library stays cleaner.

upload fails for analyses with same name

append unique id to analysis name so that unique collection can be created on neurovault and images can be uploaded successfully

Use consistent method names that match API routes

Currently, we have ns.analyses.full() and ns.analyses.get_report(). Methods should probably all follow the same naming conventions, and ideally match the API (so, e.g., the first above should be ns.analyses.get_full().

jwt decode regression

see: Azure/azure-sdk-for-python#16035

requests slowing things down?

At least some routes in pyns are extremely slow compared to directly hitting the API (e.g., via the browser). It looks like the bottleneck is in requests, because just doing something like requests.get(https://neuroscout.org/api/analyses/Mzq74/full) is very slow on my machine (several seconds). Probably worth looking into.

Use HTTPs route

neuroscout is now on https, not http

Make an example that successfully runs an analysis

Add basic documentation

Fill out the README and add a few examples

Update AutoContrasts to DummyContrasts

Update model creator to new model type

Convert predictor-events to pandas dfs

Add helper functions for this

Make copy of transformations in create_analysis

Otherwise, the transformations list is mutated, which causes problems outside of this function, especially when iterating across models

In depth tutorials/notebooks

Let's add some jupyter notebooks or tutorials covering some use cases of pyNS.

One example is how to deal with categorical variables.

Add single sign on

The frontend supports SSO, but pyNS currenly only supports users created through our db.

Adding SSO directly from pyNS would be useful.

Add models for all basic resources in the API

E.g. User, Dataset, Analysis

For now, just return JSON response, but in the future could consider developing a more advanced model.

Passing list of runs only pulls first

It seems query strings may not be formatted correctly

Release to PyPI

Push library to PyPI when good enough

Add convenience function to fill/clear/push/pull

Currently it's a bit cumbersome:

paranoia = life.clone()
paranoia.dataset_id = 19
paranoia.runs = []
paranoia.predictors = []
paranoia.push()
paranoia.fill()

That could all be made into a little helper function in which you provide a dataset_id

Upload t-maps first

For a cleaner look in the collection

add 'hrf' option to dummy contrasts

Always specific subject level in models

Related to neuroscout/neuroscout#641

Add exception handling for requests

Requests usually will just return an error code, instead of throw exception.

Instead, lets throw exceptions, and have the calls return the content.

jwt not installed with package

Installing neuroscout doesn't install the jwt package, resulting in failure when importing Neuroscout.

Sort out install_requires vs requirements.txt

pyJWT is listed in requirements.txt but not installed, and install_requires lists requests, but requirements does not

Add _repr_ to objects

Currently, they are not terribly useful:
<pyns.models.analysis.Analysis at 0x7f163c718c50>

support all entities for neurovault collection loading

pyNS/pyns/models/analysis.py

Line 439 in 1828c11

def _get_entities(path):

Add upload progress bar

Such as:
https://www.programcreek.com/python/example/106103/requests_toolbelt.MultipartEncoderMonitor

Ideally its nested (e.g. file progress + overall progress)

Predictor meta-data corrupted when resampling

some rows say subject is 0.0

look into this

Add way to search existing public results

Will need to make changes to the API to make this more performant.

Add source as filter for predictors

Add betamax support

Add betamax to cache requests in the tests. Also, think about whether the current "test user" is a good idea, or might be insecure.

Truncate collection names

to prevent NV upload failures

Allow user name / password through env vars

NEUROSCOUT_USER NEUROSCOUT_PASSWORD

pin altair/vegalite version

Analysis reports are generated for the following versions:
altair 4.0.0
vega-lite 2.6.0

Note that when upgrading to these versions, there's no need to call notebook renderer.

inconsistent name for RMS predictor

RMS is coded with name field = 'rms' for a bunch of datasets (SchematicNarrative, Sherlock, SherlockMerlin) and as 'rmse' for all others, which yields issues with pyNS-based queries.

Add OO Interface for Analysis creation

To make pyNS more user friendly, it would be useful if the library returns python objects that mirror the JSON response. These objects could have methods associated with them to save to the API directly.

For example

analysis = api.Analysis(name='new_analysis')
analysis.create()
print(analysis.hash_id)
    5xdj45
analysis.predictors = [124, 421, 246]
analysis.save()
analysis.compile()
print(analysis.status())
   PENDING
print(analysis.status())
   FINISHED

Add Travis-CI

It would be good to add Travis-CI support for CI testing.

Add analysis /fill route

Load predictor events into pandas df

link to API docs not working

current link to api docs in README (http://neuroscout.org/swagger-ui) is broken

Add /predictor/create

For uploading custom predictors

Add build to compile as arg

Ability to not build when compiling

Cache requests

We should consider caching requests in the client, as some routes can take a long time to retrieve. Since you're already using requests (which btw should be added to requirements), a super simple solution I see no downside to would be to add requests-cache.

Automatically re-authorize

Currently, once your api token is stale, protected calls will just fail, and you have to manually re-authorize.

It'd be nice if the API checks that the token is expired, and automatically re-authorizes when possible (i.e. the keys are in the os environment)

Access predictor_events by predictor name

Get design matrix should fail is report is failed

Currently, it will return FAILED as string, instead it should throw error.