GithubHelp home page GithubHelp logo

cbiit / bento-meta Goto Github PK

View Code? Open in Web Editor NEW
5.0 10.0 2.0 18.99 MB

Bento Metamodel

Perl 11.21% Dockerfile 0.02% Python 26.10% Shell 0.02% Jupyter Notebook 0.10% Cypher 62.43% Batchfile 0.06% Makefile 0.05%
icdc ctdc bento graph

bento-meta's Issues

missing and superfluous arguments in `object_map.py`

There appears to be an unused parameter albl that not used in the query in object_map.py lines 449 - 453

bento-meta/python/bento_meta/object_map.py

Line 449 in eda7de2

return "MATCH (n:{lbl}){rel}(a) WHERE id(n)={neoid} DELETE r RETURN id(n),id(a)".format( 

          return "MATCH (n:{lbl}){rel}(a) WHERE id(n)={neoid} DELETE r RETURN id(n),id(a)".format(
            lbl=self.cls.mapspec()["label"],
            albl=end_lbls[0],
            rel=rel,
            neoid=obj.neoid)

and a missing parameter aneoid and unused parameter albl in the query in object_map.py lines 475 - 480

bento-meta/python/bento_meta/object_map.py

Line 475 in eda7de2

qry = "MATCH (n:{lbl}){rel}(a) WHERE id(n)={neoid} AND id(a)={aneoid} AND ({cond}) DELETE r RETURN id(n),id(a)".format( 

            qry = "MATCH (n:{lbl}){rel}(a) WHERE id(n)={neoid} AND id(a)={**aneoid**} AND ({cond}) DELETE r RETURN id(n),id(a)".format(
              lbl=self.cls.mapspec()["label"],
              albl=end_lbls[0],
              neoid=obj.neoid,
              cond=cond,
              rel=rel)

Cannot save a model containing a 'tag' - `dput()` returns AttributeError

I can use bento_meta to create a tag for a node.
I cannot save the tag to the database; or, I cannot save the model after a tag is created - trying to execute dput() returns an Attribute Error:

Traceback (most recent call last):
  File "tag_issue.py", line 45, in <module>
    icdc_model.dput()
  File "/Users/bensonml/.pyenv/versions/pyenv3.8/lib/python3.8/site-packages/bento_meta/model.py", line 506, in dput
    do_(e)
  File "/Users/bensonml/.pyenv/versions/pyenv3.8/lib/python3.8/site-packages/bento_meta/model.py", line 481, in do_
    obj.dput()
  File "/Users/bensonml/.pyenv/versions/pyenv3.8/lib/python3.8/site-packages/bento_meta/entity.py", line 436, in dput
    return type(self).object_map.put(self)
  File "/Users/bensonml/.pyenv/versions/pyenv3.8/lib/python3.8/site-packages/bento_meta/object_map.py", line 229, in put
    for qry in ObjectMap(cls=type(val), drv=self.drv).put_q(val):
  File "/Users/bensonml/.pyenv/versions/pyenv3.8/lib/python3.8/site-packages/bento_meta/object_map.py", line 414, in put_q
    if getattr(obj, pr) is None:
  File "/Users/bensonml/.pyenv/versions/pyenv3.8/lib/python3.8/site-packages/bento_meta/entity.py", line 247, in __getattr__
    raise AttributeError(
AttributeError: get: attribute 'key' neither private nor declared for subclass Tag

Sample script:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# demonstrate issue of saving a tag to db w/ dput()
import os
from neo4j import GraphDatabase
from bento_meta.model import Model
from bento_meta.objects import (
    Node,
    Property,
    Edge,
    Term,
    ValueSet,
    Concept,
    Origin,
    Tag,
)


def get_neo4j_db():
    uri = "bolt://localhost:7687"
    user = os.environ.get("NEO4J_MDB_USER")
    password = os.environ.get("NEO4J_MDB_PASS")

    ndbr = GraphDatabase.driver(uri, auth=(user, password))

    return ndbr


drv = get_neo4j_db()
icdc_model = Model("ICDC", drv)

# get data
print("getting from db")
icdc_model.dget()

# get case node, for example
case_node = icdc_model.nodes["case"]

# show that the tags are empty, as expected
# {}
print("tags before assignment")
print(case_node.tags)

# save model without changes - no problems expected
print("saving without changes")
icdc_model.dput()

# create and assign "core" as tag to 'case' node
tag_string = "core"
case_node.tags[tag_string] = Tag({"value": tag_string})

# show that 'case' node has tag, as expected
# {'core': <bento_meta.objects.Tag object at 0x107c7ae50>}
print("tags after assignment")
print(case_node.tags)

# Attempt to save db - now that model includes a tag
# AttributeError: get: attribute 'key' neither private nor declared for ...
print("saving to db")
icdc_model.dput()

# Never reached
print('done saving')

set_with_node doesn't access Node mapspec dict

bento-meta/python/bento_meta/entity.py

Line 178 in 90efa1b

patt = type(self).mapspec()['property'][att]

If the attribute "nanoid" is added to class Node(Entity)

    attspec = {
        "handle": "simple",
        "model": "simple",
        "category": "simple",
        "nanoid": "simple",
        "concept": "object",
        "props": "collection",
    }
    mapspec_ = {
        "label": "node",
        "key": "handle",
        "property": {"handle": "handle", "model": "model", "category": "category", "nanoid": "nanoid"},
        "relationship": {
            "concept": {"rel": ":has_concept>", "end_cls": "Concept"},
            "props": {"rel": ":has_property>", "end_cls": "Property"},
        },
    }

Then a KeyError is raised:

>           patt = type(self).mapspec()["property"][att]
E           KeyError: 'nanoid'

.tox/py3/lib/python3.7/site-packages/bento_meta/entity.py:227: KeyError

the mapspec will instead hold (note no nanoid)

{'key': '_id',
 'label': 'node',
 'property': {'_from': '_from',
              '_id': 'id',
              '_to': '_to',
              'category': 'category',
              'desc': 'desc',
              'handle': 'handle',
              'model': 'model'},

However, this problem does not appear to exist with other entities such as class Property(Entity), or class Edge(Entity)

Proof of workaround is in branch pep8. See https://github.com/CBIIT/bento-meta/blob/pep8/python/bento_meta/entity.py

            # WARNING: the node.mapspec()["property"] is NOT reading
            #          what is defined in objects.py class Node(Entity)
            #          can add 'nanoid' to attspec and mapspec_ but it will
            #          NOT show up here, not yet read?
            # Ergo, code needs to check that the attribute att actually
            # exists in the mapspec()["property"] dictionary, or else it will
            # raise KeyError here
            if att in (type(self).mapspec()["property"]):
                patt = type(self).mapspec()["property"][att]
                if patt in init:
                    setattr(self, att, init[patt])
                else:
                    setattr(self, att, None)

            # NOTE: 'one-off' munge to allow 'nanoid' attspec for Node
            # P.S.: don't let mom see I coded this...
            # TODO fix this!
            if (att == "nanoid" and type(self) == "Node"):
                patt = "nanoid"
                if patt in init:
                    setattr(self, att, init[patt])
                else:
                    setattr(self, att, None)

Reloading data from MDB?

I cannot seam to be able to 'reload' data from MDB into Bento-meta.

Use case: using Bento-meta for modeling data in web server. End-user Anna goes to "sts.com" and sees the data that is stored in the MDB for the value set "disease". A few minutes later, end-user Bob goes to "sts.com" and makes an update to the disease value set., adding a term to a value set. When Anna revisits the the value set a few minutes later, she should see the now-updated value set.

When it comes to using "Bento-meta", I cannot find a way to reload data. Any attempts to "reload" result in error:
$m->load_all_db_models($bolt_url);. I need to be able to have Bento-meta continually polling MDB very quickly to get any changes.

error:

Can't call method "handle" on an undefined value at /Users/bensonml/0_SRC/bento-meta/perl/lib/Bento/Meta/Model/Edge.pm line 30.

The problem is compounded by the fact that it takes about 100 seconds to run $m->load_all_db_models($bolt_url);, which means that each time a person wants to go to a new page, or look at a different value set, or node and I have to reload the model for each query, it will take 1:40 to load and display each page.

Reloading error example snippet:

   $bolt_url //= 'bolt://54.156.191.24:7687';
    print "Using Bento::Meta version=$Bento::Meta::VERSION\n";

    my $m = Bento::Meta->new;

    $m->load_all_db_models($bolt_url);
    my $icdc = $m->model('ICDC');
    print "---A---\n";

    $m = undef;
    $icdc = undef;
    $m = undef;
    $m = Bento::Meta->new;

    print "---B---\n";
    $m->load_all_db_models($bolt_url);
    print "---C---\n";
    $icdc = $m->model('ICDC');
    print "---D---\n";

    my @icdc__nodes = $icdc->nodes();
    #print Dumper (@icdc__nodes);
    foreach my $n (@icdc__nodes) {
        print "---E---\n";
        #my $n_ = $m->modelnode($n);
        #print Dumper ($n_);
        #print "\t\txxxxxxxx\n\n";
        my $n_handle_ = $n->handle();
        print "\t - node: $n_handle_\n";
        my @n_props = $n->props();
        foreach my $n_prop (@n_props){
            print "---F---\n";
            my $n_p_h = $n_prop->handle();
            print "\t\t-node-property: $n_p_h\n";
        }
        #my $name = $n->handle();
        #print "   found $name\n";
    }

    print "---G---\n";

yields the following error:

❯ perl simple.pl
Unescaped left brace in regex is deprecated here (and will be fatal in Perl 5.32), passed through in regex; marked by <-- HERE in m/^([:$?])|({ <-- HERE [^}]+}$)/ at /Users/bensonml/perl5/perlbrew/perls/perl-5.30.2/lib/site_perl/5.30.2/Neo4j/Cypher/Abstract/Peeler.pm line 542.
Unescaped left brace in regex is deprecated here (and will be fatal in Perl 5.32), passed through in regex; marked by <-- HERE in m/^([:$?])|({ <-- HERE [^}]+}$)/ at /Users/bensonml/perl5/perlbrew/perls/perl-5.30.2/lib/site_perl/5.30.2/Neo4j/Cypher/Abstract/Peeler.pm line 120.
---A---
---B---
Can't call method "handle" on an undefined value at /Users/bensonml/0_SRC/bento-meta/perl/lib/Bento/Meta/Model/Edge.pm line 30.

Add 'Model' as new Entity

Add Model as a class in objects.py - enable loaders to create these automatically (as "orphan" nodes)

`get_by_id` broken

object.nanoid not yet implemented (it is implemented in branch feat/nanoidvsid)

object_map.get_by_id() is broken

  • using Neo4j identifier id is not stable, lookups should use nanoid instead
  • function- doesn't work if called on an instance or class
bento_meta.object_map.get_by_id('wze6rN')
n = bento_meta.object.Node()
n.get_by_id('wze6rN')

Docs build breaking with mergespec() update

Discovered that the docs build is breaking. The issue has to do with being able to access the subclass specific (i.e., not the Entity class) attributes separately. Fixing objects.py to account for this.

This will lead to a very slight change in how one declared the attspec (basically, use attspec_, not attspec in the subclass def). The fix will also update the docs to account for this.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.