belbio / bel_api Goto Github PK

Starting point: https://openbel.api-docs.io/1.0.0/welcome/welcome

Using Swagger to build the specification using JSON Schema definitions of critical objects such as Nanopubs. Swagger-Edit and Swagger UI docker instances are available to edit and host the swagger spec with external $refs.

We will be integrating with the GraphDati NanopubStore, EdgeStore and NetworkStore so that functionality should not be part of BELbio API.

We also need to add the ability to manage terminologies (load, search, autocomplete). Some of the namespace/annotation endpoints/functionality from the OpenBEL API will be migratable. We also need to be able to load/delete whole terminologies as JSON/YAML objects.

The BEL statement parsing/transformation/etc services will be backed by the bel.py python module (in development).

We need all of the following endpoints for BEL manipulation:

• convert a BEL statement into an AST and back again
• migrate BEL V.x to V.y
• validate (syntactically and semantically)
• convert or update namespaces
• expand to BEL Edges
• break a statement into Subject, Relation, Object and vice versa

BEL Nanopubs:

• validate structure
• full validation: BEL Statement, Namespaces, Annotations
• convert between JSON and YAML

As querystring params.

Also - figure out how to handle /orthologs so that you can skip canonical/decanonical for the use case in bel_lang.bel.orthologize which could come after canonicalization.

Allow partial matches of terms/synonyms, etc while filtering for namespace, entity_types, annotation_types, or species. Is this just the completion endpoint?

add populationAbundance() function

From Natalie:
Can’t I disambiguate “CD3Z” or “CD3-Zeta” to HGNC? Looks like “CD3Z” is listed as a previous name, but not a synonym (“CD3Q” works, but “CD3 zeta” does not, though both should match on synonyms)

Can't complete the annotation MESH:Blood

it will find phrases with blood in it but not just blood - blood is a MESH term and is in the database in the Anatomy annotation_type

Add basics to get started using py.test for bel_api code testing - both unit tests and functional/api tests

Add DOI endpoint to collect DOI metadata for bel nanopub.

https://github.com/CrossRef/rest-api-doc

https://github.com/sckott/habanero
https://github.com/fabiobatalha/crossrefapi

Refactor term completion

Add a settings collection to Arango DB and add things like authenticated endpoint, canonicalization targets, etc. Anything that the API needs to know and might need to change or update.

Allow different types of annotations:

• namespaces
• p-value (e.g. .001 or <0.05)
• cohort name

Requires changes to nanopub_bel JSONSchema, belbio_conf, bel_api and front-end tools.

Best way to handle this is probably to provide BEL API configurable annotation field requirements.

Change meaning of annotation type to: NSArg, P-value, Cohort, etc

Add schemas for each annotation type in the belbio_conf configuration file (NSArg and keyvalue would be default annotation types).

E.g. NSArg: {"class": string, "id": string, "label": string}, pvalue: {"name": string, "value": [number, string]}, keyvalue: {"key": string, "value": value}

Examples

• pvalue 'label': X 'value': 0.10
• pvalue 'label': X 'value': <1E-5
• Cohort Description 'value':
• Cohort Size 'value':

Syntax generation is straightforward enough - simply use a Jinja2 generated EBNF from our YAML and pass that to Tatsu. However, semantic classes are a little more detailed oriented - the function signatures require specific functions to take specific parameters. How do we go about this? The semantics file must be in a .py file which contain a semantics class for use in our parser. One idea is to use Jinja2 to generate this .py as we did with syntax. However, our EBNF template was mostly finished (just had to add functions, relations, etc. from YAML). How can we accomplish the same with a semantic class file?

Create script to generate Orthology BEL statements probably as a CSV file for efficiency loading into EdgeStore -- see OpenBEL github repo resource-generator for example script collecting this information

Child of #84

Natalie reported:

If the outer BEL function is selected from the autocomplete, it includes the closing parenthesis, which must be removed for the value autocomplete. This really trips up newer users, who are most likely to need the BEL function help.

https://github.com/rdidyk/falcon-swagger-ui

Create script to generate gene scaffold BEL statements e.g. g(HGNC:"A1BG-AS1") transcribedTo r(HGNC:"A1BG-AS1") and r(HGNC:"HLA-E") translatedTo p(HGNC:"HLA-E")

See openbel-framework-resources/resource/gene_scaffolding_document_9606_10090_10116.bel for an example

https://blog.realkinetic.com/building-minimal-docker-containers-for-python-applications-37d0272c52f3

Should probably be an array of orthologs for /orthologs/{gene_id}/{species_id}

Add another endpoint for /orthologs/{gene_id} without species_id

Format should be sourced from NanopubStore:

If you are interested in the format of the Search Response the nanopub supports - Look at the Nanopub Service code - Models\GDSearchResponse, it uses the Facets.cs class The SearchProvider DynamicSearch is what is called to execute the elastic request. Since the code is using the LowLevel elastic calls the FacetDict must be created from the LLFacetDict.

https://europepmc.org/AnnotationsApi;jsessionid=EC2691317021D77277727B88D8C87499

Check into using these for proto BEL Assertions and Annotations for the BEL Editor or sources of BEL Entities to add to BEL Assertions.

This is a bel python module issue as well - e.g. see bel/lang/bel_utils/convert_nsarg.

http://pybel.readthedocs.io/en/latest/overview.html#explicit-node-labels

Good idea - prefer to take labels from terminologies, but as indicated in the example in pyBEL - that won't always work

Change all API routes to their plural form

Create a terminology json-schema to capture a terminology (e.g. HGNC - human genes namespace)

*terminology metadata
** name
** label - e.g. HGNC
** domain - e.g. Gene, mRNA, Protein
** source
** source_url
** description
** version
** date_modified

• terms
  ** official term name
  ** synonyms
  ** equivalences to other terminologies
  ** hierarchy if available
  ** orthology to other organisms if available
  ** prior terms used for this term if available

This format will be used to capture different namespaces and annotations

Create scripts to generate the namespace and annotation terminology files as JSON/YAML using the schemas defined in the bel_api/docs/api/schemas.

Add better examples to Swagger doc and fix any errors in docs - use Swagger Editor online to check our Swagger doc.

See /nanopub/validate for an example that is not showing up correctly in the Swagger UI.

Look into this as a performance enhancement for bel and bel_api projects

add ability to grab env vars and override config

Swap the name and abbreviations in the default namespace section of the bel_v2.yaml file.

Namespace values are not being checked in Assertions.

Bad namespace values are not being returned as WARNING messages.

with https://github.com/OpenBEL/openbel-platform/blob/master/terminology.md

Add namespace metadata to the types endpoint. Namespace full name, description, uri, uri template, etc. This probably needs to be added to the arangodb database when loading resources.

NLM BioNLP APIs
https://www.ncbi.nlm.nih.gov/research/bionlp/APIs/

Pubtator API - all bioconcepts available in JSON format: https://www.ncbi.nlm.nih.gov/CBBresearch/Lu/Demo/RESTful/tmTool.cgi/BioConcept/19894120/JSON/

The species is the TAX namespace. The other IDs, e.g. Chemical:D005978 or Gene:4363 are MESH ids unless they have another namespace, e.g. Chemical:CHEBI:16412.
MedlineXML2JSON: https://gist.github.com/richard-jones/1852712
https://github.com/ldbib/MEDLINEXMLToJSON/blob/master/lib/app.js

To support methylation http://varnomen.hgvs.org/bg-material/consultation/svd-wg005/

See also pyBEL: http://pybel.readthedocs.io/en/latest/overview.html#syntax-for-epigenetics

Needed for BEL Editor - adding multiple objects for example - given list of mRNA names

• find NSArgs for RNA entity_type
• decanonicalize
• wrap them in given function (e.g. rnaAbundance())

Add bel_lang functionality as /api/bel endpoints

Add the pubmed publication types to the /pubmed/ endpoint result.

e.g.

BioDati Studio would like to add them as Annotations.

SFAM support? No autocomplete in BEL terms, but appear in Annotations autocomplete.

Needs to show up in BEL namespace completion

From Natalie's suggestion:

Allow entering naked namespace values, complete to NS:VAL and then provide completion suggestions taking into account possible functions that can wrap the NS:VAL based on the NS:VAL entity types and the parameter location of the NS:VAL if it's already in a function.

Examples ('|' indicates the cursor):

AKT1| -> HGNC:AKT1| -> ['p(HGNC:AKT1)', 'r(HGNC:AKT1)', 'g(HGNC:AKT1)']

p(AKT1|) -> p(HGNC:AKT1|) -> [p(fus(pHGNC:AKT1, |))]

Use 204 instead of 404 for missing resource?

https://github.com/AndreLouisCaron/pytest-docker

To support testing, need to have ability to select between different environments from the same config file which looks to be overly complicated using the layeredconfig module.

Go back to just using python code in Config.py to create the settings and remove the conf-api.yml file.

Autocomplete for loc() modifier only works if loc is improperly used as an outer function. (Try “loc(mitochondria” vs “p(HGNC:AKT1, loc(mitochondria”)

/ortholog/HGNC:A1BG/TAX:0

Check other boundary conditions as well.

http://v1-5.archive.docs.traefik.io/configuration/acme/#acmehttpchallenge

Let's Encrypt HTTP-01 issue

Getting an error parsing/validating the following:

belstmt validate "abundance(CHEBI:antioxidant) decreases (abundance(SCHEM:"7-Ketocholesterol") increases biologicalProcess(GO:"apoptotic process"))"

Improve the error reporting for loading BEL resources using the endpoint. Add option for email address to get error reports.