dfci / matchminer-api Goto Github PK
View Code? Open in Web Editor NEWA platform for matching patient-specific genomic profiles to precision cancer medicine clinical trials
Home Page: https://matchminer.org/
A platform for matching patient-specific genomic profiles to precision cancer medicine clinical trials
Home Page: https://matchminer.org/
My pymongo is 2.9.1 (as in the requirements.txt)
I ran these two successfully as far as I can tell:
$ docker run --name match-mongo -d -p 27017:27017 mongo mongod --smallfiles --replSet=rs0
$ docker exec -it match-mongo mongo matchminer --eval "rs.initiate();"
Now:
$ python2.7 pymm_run.py restore
[INFO] settings: DEV
[INFO] settings: mongodb://localhost:27017/matchminer?replicaSet=rs0
/Users/ipozdnya/matchminer-api/cerberus1/cerberus.py:1272: UserWarning: No validation schema is defined for the arguments of rule 'consented'
"'%s'" % '_'.join(method_name.split('_')[2:]))
/Users/ipozdnya/matchminer-api/cerberus1/cerberus.py:1272: UserWarning: No validation schema is defined for the arguments of rule 'match'
"'%s'" % '_'.join(method_name.split('_')[2:]))
/Users/ipozdnya/matchminer-api/cerberus1/cerberus.py:1272: UserWarning: No validation schema is defined for the arguments of rule 'normalized'
"'%s'" % '_'.join(method_name.split('_')[2:]))
/Users/ipozdnya/matchminer-api/cerberus1/cerberus.py:1272: UserWarning: No validation schema is defined for the arguments of rule 'unique'
"'%s'" % '_'.join(method_name.split('_')[2:]))
[INFO] settings: DEV
[INFO] settings: mongodb://localhost:27017/matchminer?replicaSet=rs0
[INFO] mongorestore --host rs0/localhost:27017 --db matchminer --drop /Users/ipozdnya/matchminer-api/data/matchminer
2019-02-21T11:46:41.458+0100 building a list of collections to restore from /Users/ipozdnya/matchminer-api/data/matchminer dir
2019-02-21T11:46:41.465+0100 reading metadata file from /Users/ipozdnya/matchminer-api/data/matchminer/genomic.metadata.json
2019-02-21T11:46:41.465+0100 restoring matchminer.genomic from file /Users/ipozdnya/matchminer-api/data/matchminer/genomic.bson
2019-02-21T11:46:41.468+0100 reading metadata file from /Users/ipozdnya/matchminer-api/data/matchminer/clinical.metadata.json
2019-02-21T11:46:41.468+0100 restoring matchminer.clinical from file /Users/ipozdnya/matchminer-api/data/matchminer/clinical.bson
2019-02-21T11:46:41.532+0100 restoring indexes for collection matchminer.clinical from metadata
2019-02-21T11:46:41.534+0100 finished restoring matchminer.clinical (92 documents)
2019-02-21T11:46:44.459+0100 [#####################...] matchminer.genomic 77.1 MB/84.8 MB (91.0%)
2019-02-21T11:46:45.128+0100 restoring indexes for collection matchminer.genomic from metadata
2019-02-21T11:46:45.129+0100 finished restoring matchminer.genomic (87980 documents)
2019-02-21T11:46:45.129+0100 done
Traceback (most recent call last):
File "pymm_run.py", line 467, in <module>
args.func(args)
File "pymm_run.py", line 186, in bootstrap_restore
db = get_db()
File "/Users/ipozdnya/matchminer-api/matchminer/database.py", line 50, in get_db
connection[MONGO_DBNAME].add_user(MONGO_USERNAME, MONGO_PASSWORD)
File "/Users/ipozdnya/miniconda3/envs/matchminer/lib/python2.7/site-packages/pymongo/database.py", line 966, in add_user
(not uinfo["users"]), name, password, read_only, **kwargs)
File "/Users/ipozdnya/miniconda3/envs/matchminer/lib/python2.7/site-packages/pymongo/database.py", line 892, in _create_or_update_user
read_preference=ReadPreference.PRIMARY, **opts)
File "/Users/ipozdnya/miniconda3/envs/matchminer/lib/python2.7/site-packages/pymongo/database.py", line 535, in command
codec_options, **kwargs)[0]
File "/Users/ipozdnya/miniconda3/envs/matchminer/lib/python2.7/site-packages/pymongo/database.py", line 440, in _command
msg, allowable_errors)
File "/Users/ipozdnya/miniconda3/envs/matchminer/lib/python2.7/site-packages/pymongo/helpers.py", line 213, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: command SON([('createUser', 'user'), ('pwd', u'<some encoded password here>'), ('digestPassword', False), ('roles', ['dbOwner']), ('writeConcern', {})]) on namespace matchminer.$cmd failed: Use of SCRAM-SHA-256 requires undigested passwords
That happened to me before when on another machine and then I fixed it by getting into mongo and creating an admin user:
> use admin
> db.createUser({ user:"admin", pwd:"abc123", roles:[{role:"userAdminAnyDatabase",db:"admin"}], passwordDigestor:"server" })
This time it did not help. Dropping and re-creating that user did not help either.
Running $ nosetests tests
results in the same error.
Please suggest a workaround. Thanks!
Deco docs:
Link https://drive.google.com/open?id=0B334b-HHVrwGc3djZUdvMktNekU gives 404
This link in demo docs gives 404:
https://drive.google.com/open?id=0B334b-HHVrwGc3djZUdvMktNekU
After installing requirements (with saml2 package commented out, see issue 6) I tried "restore" step:
matchminer-api$ python --version
Python 2.7.15
matchminer-api$ python pymm_run.py restore
Traceback (most recent call last):
File "pymm_run.py", line 7, in <module>
from matchminer import managment
File "/Users/ipozdnya/matchminer-api/matchminer/managment.py", line 10, in <module>
from matchminer import events
File "/Users/ipozdnya/matchminer-api/matchminer/events.py", line 12, in <module>
from matchminer.hooks import UtilHooks
File "/Users/ipozdnya/matchminer-api/matchminer/hooks/__init__.py", line 1, in <module>
from matchminer.database import get_db
File "/Users/ipozdnya/matchminer-api/matchminer/database.py", line 2, in <module>
from flask.ext.pymongo import MongoClient
ImportError: No module named ext.pymongo
I double-checked flask_pymongo, seems to be in place:
matchminer-api$ pip2.7 install flask_pymongo
Collecting flask_pymongo
Downloading https://files.pythonhosted.org/packages/d5/c8/22ddacfe05893884dceef5b9ecfa683f947ba155bd63cd9d841aea29b7b7/Flask_PyMongo-2.2.0-py2.py3-none-any.whl
Requirement already satisfied: Flask>=0.11 in /usr/local/lib/python2.7/site-packages (from flask_pymongo) (1.0.2)
Requirement already satisfied: PyMongo>=3.0 in /usr/local/lib/python2.7/site-packages (from flask_pymongo) (3.7.2)
Please advise.. Thanks
MacOS 10.14.1
conda env with python 2.7
Locally installed matchminer-ui and matchminer-api, started as described in the docs.
mongo in a container as described in the matchminer-api docs.
When the UI frontpage loads (8001) it gives correct number of patients (91) so it communicates with mongo OK.
When I inspect network traffic in the browser's development tools, there are several errors trying to hit /api like this:
http://localhost:8001/api/user/577cf6ef2b9920002cef0337?cacheBuster=1553031094438
When I hit that in browser, I get
"unauthorized access".
Same message if I change the port to 5000 (as it's set in config.json).
Here is the relevant part of config.json
:
"dev": {
"ENV": {
"name": "development",
"elasticsearch": {
"host": "http://localhost:9200",
"proxy": "http://localhost:8001/elasticsearch",
"index": "matchminer"
},
"api": {
"host": "http://localhost:5000",
"endpoint": "/api"
},
"samlAuthentication": false,
"devUser": {
"token" : "some token here,
"user_id" : "some user id here"
},
"certificate" : {
"key": "../certificates/matchminer.key",
"cert": "../certificates/matchminer.crt"
},
.....
Both certificates files (matchminer.key and matchminer.crt) are in place
The messages might be coming from pymm_run.py
from here:
@app.errorhandler(401)
def my_own_error_msg(err):
return make_response("unauthorized access", 497)
What can this "unauthorized access" can by caused by?
There is an option to start matchminer-api with --no-auth
switch. When I do that, on the front page I get red stripe across with "You are currently not authorized to access this page." message, the patient count is still correct. However, the api calls from the page produce different error:
e.g.
http://localhost:8001/api/user/577cf6ef2b9920002cef0337?cacheBuster=1553032900445
{
_status: "ERR",
_error: {
message: "The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.",
code: 404
}
}
This happens with both 5000 and 8001 ports
Hitting an API endpoint with no parameters (
http://localhost:8001/api/user/
) gives this:
{
_items: [ ],
_links: {
self: {
href: "user",
title: "user"
},
parent: {
href: "/",
title: "home"
}
},
_meta: {
max_results: 100000,
total: 0,
page: 1
}
}
UPDATE:
This might be a data issue, user data in particular. For example, http://localhost:8001/api/clinical/
being pasted in the browser returns the clinical data ok.
However http://localhost:8001/api/user/577cf6ef2b9920002cef033
call points at non-existing user id "577cf6ef2b9920002cef033" in a non-existing collection "user". Where are those two entities coming from in the angular frontend? Should "user" collection with that id exist before the ui is started? mongorestore creates only "clinical" and "genomic" collections, no "user". Or maybe that api call should not get executed on the frontpage load? The problem is this error disables everything in the interface, makes search fields not clickable.
UPDATE 2
The troublesome user API call from the frontpage goes away if:
python2.7 matchengine.py load -t examples/trial.example.yml -c examples/clinical.example.csv -g examples/genomic.example.csv --mongo-uri mongodb://localhost:27017
. It does not create any new collections in mongo, but it does bump up the number of patients from 91 to 110python pymm_run.py serve
WITHOUT --no-auth
The search still does not work (typing any char in the search fields gives another error) but that's a subject of a different issue. I guess since there is a workaround now this issue can be closed, leaving open just in case if the maintainer would like to comment on my install procedure.
Would be nice to have a docker-compose file spin up a dev environment with one command
MacOS 10.14.1
conda env with python 2.7
Locally installed matchminer-ui and matchminer-api, started as described in the docs.
mongo in a container as described in the matchminer-api docs.
When the UI frontpage loads (8001) it gives correct number of patients (91) so it communicates with mongo OK.
When I try to type anything in "Disease Center" search box (or any search box for that matter) in developer tools/network I see these errors:
Performing suggest!
angular.js:11756 POST http://localhost:8001/elasticsearch/matchminer/trial/_search 400 (Bad Request)
(anonymous) @ angular.js:11756
sendReq @ angular.js:11517
serverRequest @ angular.js:11227
processQueue @ angular.js:15961
(anonymous) @ angular.js:15977
$eval @ angular.js:17229
$digest @ angular.js:17045
processQueue @ angular-material.js:1734
(anonymous) @ angular.js:19157
completeOutstandingRequest @ angular.js:5869
(anonymous) @ angular.js:6145
setTimeout (async)
Browser.self.defer @ angular.js:6143
timeout @ angular.js:19155
nextTick @ angular-material.js:1711
handleAsyncResults @ angular-material.js:26145
fetchResults @ angular-material.js:26136
handleQuery @ angular-material.js:26253
focus @ angular-material.js:25824
fn @ VM541:4
expensiveCheckFn @ angular.js:15485
callback @ angular.js:25018
$eval @ angular.js:17229
$apply @ angular.js:17329
(anonymous) @ angular.js:25023
dispatch @ jquery.js:5183
elemData.handle @ jquery.js:4991
angular.js:13550 Error occurred while searching for suggestions Error: [illegal_argument_exception] no mapping found for field [_suggest.disease_center_suggest]
at respond (http://localhost:8001/bower_components/elasticsearch/elasticsearch.angular.js:15491:16)
at checkRespForFailure (http://localhost:8001/bower_components/elasticsearch/elasticsearch.angular.js:15450:8)
at http://localhost:8001/bower_components/elasticsearch/elasticsearch.angular.js:319:8
at processQueue (http://localhost:8001/bower_components/angular/angular.js:15961:28)
at http://localhost:8001/bower_components/angular/angular.js:15977:27
at Scope.$eval (http://localhost:8001/bower_components/angular/angular.js:17229:28)
at Scope.$digest (http://localhost:8001/bower_components/angular/angular.js:17045:31)
at Scope.$apply (http://localhost:8001/bower_components/angular/angular.js:17337:24)
at http://localhost:8001/bower_components/angular/angular.js:17633:22
at completeOutstandingRequest (http://localhost:8001/bower_components/angular/angular.js:5869:10)
I suspect there should be performed some "elasticsearch configuration"-related step, some mapping / indexing for the elasticsearch.
I followed the setup instructions, but it didn't work out of the box:
$ git clone [email protected]:dfci/matchminer-api.git
$ cd matchminer-api
$ ./setup.sh
I ran into and resolved two errors:
mongo
executable not being shipped with the mm-mongo
container (mongo
image). So, I replaced it with mongosh
.numpy 1.23.3
not being available when the mm-api
container tried to run pip install -r requirements.txt
.Resolved these with
--- a/requirements.txt
+++ b/requirements.txt
@@ -91,7 +91,8 @@ nose==1.3.7
# via
# -r requirements.in
# oncotreenx
-numpy==1.23.3
+numpy==1.21.6
# via pandas
oauth2client==2.0.2
# via -r requirements.in
--- a/setup.sh
+++ b/setup.sh
@@ -24,7 +24,7 @@ echo "*****************"
sleep 5
echo "Add dev user to database to bypass authentication"
-docker-compose exec mm-mongo mongo matchminer --eval 'db.user.replaceOne({
+docker-compose exec mm-mongo mongosh matchminer --eval 'db.user.replaceOne({
But now,
$ ./setup
...
Using Mongosh: 1.6.1
...
⠿ Container mm-elastic Started [382/530]
[WARNING] settings:
[WARNING] settings: mongodb://mm-mongo:27017/matchminer
/matchminerAPI/cerberus1/cerberus.py:1272: UserWarning: No validation schema is defined for the arguments
of rule 'consented'
"'%s'" % '_'.join(method_name.split('_')[2:]))
/matchminerAPI/cerberus1/cerberus.py:1272: UserWarning: No validation schema is defined for the arguments
of rule 'match'
"'%s'" % '_'.join(method_name.split('_')[2:]))
/matchminerAPI/cerberus1/cerberus.py:1272: UserWarning: No validation schema is defined for the arguments
of rule 'normalized'
"'%s'" % '_'.join(method_name.split('_')[2:]))
/matchminerAPI/cerberus1/cerberus.py:1272: UserWarning: No validation schema is defined for the arguments
of rule 'unique'
"'%s'" % '_'.join(method_name.split('_')[2:]))
/usr/local/lib/python3.7/site-packages/cerberus/validator.py:1402: UserWarning: No validation schema is d
efined for the arguments of rule 'consented'
"'%s'" % method_name.split('_', 2)[-1])
/usr/local/lib/python3.7/site-packages/cerberus/validator.py:1402: UserWarning: No validation schema is d
efined for the arguments of rule 'match'
"'%s'" % method_name.split('_', 2)[-1])
/usr/local/lib/python3.7/site-packages/cerberus/validator.py:1402: UserWarning: No validation schema is d
efined for the arguments of rule 'normalized'
"'%s'" % method_name.split('_', 2)[-1])
[WARNING] NO AUTHENTICATION IS ENABLED - SKIPPING HIPAA LOGGING
[WARNING] settings:
[WARNING] settings: mongodb://mm-mongo:27017/matchminer
/usr/local/lib/python3.7/site-packages/eve/flaskapp.py:310: UserWarning: XML setting is deprecated and wi
ll be removed in future release. Please use RENDERERS instead.
warnings.warn(msg.format("XML"))
[INFO] reset_elasticsearch
[INFO] http://mm-elastic:9200/matchminer
[INFO]
[INFO]
[INFO] delete_elasticsearch_index
[WARNING] DELETE http://mm-elastic:9200/matchminer [status:N/A request:3.052s]
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 159, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
OSError: [Errno 113] No route to host
Any suggestions?
Additionally, what version of mongo
was docker-compose supposed to retrieve?
docker-compose.yml
did not specify a version number, and the current matchminer-api setup is apparently not compatible with the latest version of mongo, because it relies on the deprecated mongo
executable rather than mongosh
.
$ uname -a
Linux durus 6.1.4-arch1-1 #1 SMP PREEMPT_DYNAMIC Sat, 07 Jan 2023 15:10:07 +0000 x86_64 GNU/Linux
$ docker version
Client:
Version: 20.10.22
API version: 1.41
Go version: go1.19.4
Git commit: 3a2c30b63a
Built: Tue Dec 20 20:43:40 2022
OS/Arch: linux/amd64
Context: default
Experimental: true
Server:
Engine:
Version: 20.10.22
API version: 1.41 (minimum version 1.12)
Go version: go1.19.4
Git commit: 42c8b31499
Built: Tue Dec 20 20:42:46 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.6.14
GitCommit: 9ba4b250366a5ddde94bb7c9d1def331423aa323.m
runc:
Version: 1.1.4
GitCommit:
docker-init:
Version: 0.19.0
GitCommit: de40ad0
$ docker-compose version
Docker Compose version 2.14.2
I'm running into issues when trying to run matchminer UI locally. I can't find in the documentation where the docker-compose yaml files are for the elasticsearch docker containers. Can you advise where I should find that information.
Thanks
matchminer-api$ nosetests tests
Traceback (most recent call last):
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 235, in load_module
return load_source(name, filename, file)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 172, in load_source
module = _load(spec)
File "", line 693, in _load
File "", line 673, in _load_unlocked
File "", line 697, in exec_module
File "", line 222, in _call_with_frames_removed
File "/Users/ipozdnya/matchminer-api/tests/unit/test_api/test_neg_genomic.py", line 5, in
from matchminer.database import get_db
File "/Users/ipozdnya/matchminer-api/matchminer/database.py", line 4, in
from matchminer.settings import *
File "/Users/ipozdnya/matchminer-api/matchminer/settings.py", line 56, in
from matchminer.settings_dev import *
File "/Users/ipozdnya/matchminer-api/matchminer/settings_dev.py", line 20, in
TOKEN_TIMEOUT = sys.maxint
AttributeError: module 'sys' has no attribute 'maxint'
Traceback (most recent call last):
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 235, in load_module
return load_source(name, filename, file)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 172, in load_source
module = _load(spec)
File "", line 693, in _load
File "", line 673, in _load_unlocked
File "", line 697, in exec_module
File "", line 222, in _call_with_frames_removed
File "/Users/ipozdnya/matchminer-api/tests/unit/test_api/test_response.py", line 7, in
from rfc822 import formatdate
ImportError: No module named 'rfc822'
Traceback (most recent call last):
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 235, in load_module
return load_source(name, filename, file)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 172, in load_source
module = _load(spec)
File "", line 693, in _load
File "", line 673, in _load_unlocked
File "", line 697, in exec_module
File "", line 222, in _call_with_frames_removed
File "/Users/ipozdnya/matchminer-api/tests/unit/test_api/test_status.py", line 6, in
from matchminer.database import get_db
File "/Users/ipozdnya/matchminer-api/matchminer/database.py", line 4, in
from matchminer.settings import *
File "/Users/ipozdnya/matchminer-api/matchminer/settings.py", line 56, in
from matchminer.settings_dev import *
File "/Users/ipozdnya/matchminer-api/matchminer/settings_dev.py", line 20, in
TOKEN_TIMEOUT = sys.maxint
AttributeError: module 'sys' has no attribute 'maxint'
Traceback (most recent call last):
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 235, in load_module
return load_source(name, filename, file)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 172, in load_source
module = _load(spec)
File "", line 693, in _load
File "", line 673, in _load_unlocked
File "", line 693, in exec_module
File "", line 799, in get_code
File "", line 759, in source_to_code
File "", line 222, in _call_with_frames_removed
File "/Users/ipozdnya/matchminer-api/tests/unit/test_api/test_trial.py", line 621
"site": [{"id": [01], "value": "Dana-Farber Cancer Institute"}],
^
SyntaxError: invalid token
Traceback (most recent call last):
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 235, in load_module
return load_source(name, filename, file)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 172, in load_source
module = _load(spec)
File "", line 693, in _load
File "", line 673, in _load_unlocked
File "", line 697, in exec_module
File "", line 222, in _call_with_frames_removed
File "/Users/ipozdnya/matchminer-api/tests/unit/test_api/test_utilities.py", line 4, in
from matchminer.database import get_db
File "/Users/ipozdnya/matchminer-api/matchminer/database.py", line 4, in
from matchminer.settings import *
File "/Users/ipozdnya/matchminer-api/matchminer/settings.py", line 56, in
from matchminer.settings_dev import *
File "/Users/ipozdnya/matchminer-api/matchminer/settings_dev.py", line 20, in
TOKEN_TIMEOUT = sys.maxint
AttributeError: module 'sys' has no attribute 'maxint'
Traceback (most recent call last):
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 235, in load_module
return load_source(name, filename, file)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 172, in load_source
module = _load(spec)
File "", line 693, in _load
File "", line 673, in _load_unlocked
File "", line 697, in exec_module
File "", line 222, in _call_with_frames_removed
File "/Users/ipozdnya/matchminer-api/tests/test_cbioengine/test_engine.py", line 4, in
from tcm import CBioEngine
File "/Users/ipozdnya/matchminer-api/tcm/init.py", line 1, in
from engine import CBioEngine
ImportError: No module named 'engine'
Traceback (most recent call last):
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 245, in load_module
return load_package(name, filename)
File "/Users/ipozdnya/ve_matchminer/lib/python3.5/imp.py", line 217, in load_package
return _load(spec)
File "", line 693, in _load
File "", line 673, in _load_unlocked
File "", line 697, in exec_module
File "", line 222, in _call_with_frames_removed
File "/Users/ipozdnya/matchminer-api/tests/test_matchminer/init.py", line 10, in
from rfc822 import formatdate
ImportError: No module named 'rfc822'
Looks like it's trying to import python3 packages for some reason even though my python is aliased to 2.7.
When I force it to run with python2.7, ( python2.7 -m nose tests
) another error happens:
ERROR: Failure: ImportError (No module named pytest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/Cellar/numpy/1.15.4_2/libexec/nose/lib/python2.7/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/usr/local/Cellar/numpy/1.15.4_2/libexec/nose/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/usr/local/Cellar/numpy/1.15.4_2/libexec/nose/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/ipozdnya/Library/Python/2.7/lib/python/site-packages/tests/test_transport_http.py", line 35, in <module>
import pytest
ImportError: No module named pytest
----------------------------------------------------------------------
Ran 28 tests in 0.963s
I checked for pytest:
$ python2.7 -m pip install pytest
Requirement already satisfied: pytest in /usr/local/lib/python2.7/site-packages (4.1.0)
hey there. where can we find information about the api endpoints?
Step pip install -r requirements.txt
dies with this (probably python-package version mismatch):
Collecting dm.xmlsec.binding==1.3.2 (from python-saml==2.1.5->-r requirements.txt (line 46))
Downloading https://files.pythonhosted.org/packages/0c/47/c8e17a5fec7bc86b092ba6841c2db7ce5e828327eb72d6a8c4c93a19f62e/dm.xmlsec.binding-1.3.2.tar.gz (119kB)
100% |████████████████████████████████| 122kB 10.7MB/s
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/private/var/folders/4z/6tmqh9hx68v3081lpb6j33k00000gs/T/pip-install-pmczeksh/dm.xmlsec.binding/setup.py", line 72
print "Warning : cflag %s skipped" % flag
^
SyntaxError: Missing parentheses in call to 'print'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/4z/6tmqh9hx68v3081lpb6j33k00000gs/T/pip-install-pmczeksh/dm.xmlsec.binding/
aliasing python to python2.7 inside the VE does not help.
commenting out saml package in requirements/.txt lets it finish with no other issues
Trying to install all the matchminer bits on a new server (without docker) and I found that the elastic search indexes do not seem to be set up. So I tried running python pymm_run.py reset-elasticsearch in the matchminer-api directory.
It seemed to be going along fine until it came to this error:
Any suggestions on what to do next?
Thanks!
Wes
[WARNING] POST http://localhost:9200/_bulk [status:400 request:0.005s]
Traceback (most recent call last):
File "pymm_run.py", line 87, in <module>
args.func(args)
File "pymm_run.py", line 81, in <lambda>
subp_p.set_defaults(func=lambda x: reset_elasticsearch())
File "/opt/matchminer-api/matchminer/elasticsearch.py", line 189, in reset_elasticsearch
add_all_trials_to_elasticsearch()]
File "/opt/matchminer-api/matchminer/elasticsearch.py", line 44, in add_all_trials_to_elasticsearch
add_trial_id_list_to_elasticsearch(chunked_trial_ids)
File "/opt/matchminer-api/matchminer/elasticsearch.py", line 50, in add_trial_id_list_to_elasticsearch
add_prepared_trials_to_elasticsearch(prepared_trials)
File "/opt/matchminer-api/matchminer/elasticsearch.py", line 67, in add_prepared_trials_to_elasticsearch
helpers.bulk(get_es_client(), prepared_trials)
File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/helpers/actions.py", line 314, in bulk
for ok, item in streaming_bulk(client, actions, *args, **kwargs):
File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/helpers/actions.py", line 244, in streaming_bulk
**kwargs
File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/helpers/actions.py", line 130, in _process_bulk_chunk
raise e
File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/helpers/actions.py", line 126, in _process_bulk_chunk
resp = client.bulk("\n".join(bulk_actions) + "\n", *args, **kwargs)
File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/client/utils.py", line 101, in _wrapped
return func(*args, params=params, **kwargs)
File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/client/__init__.py", line 1581, in bulk
headers={"content-type": "application/x-ndjson"},
File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/transport.py", line 409, in perform_request
timeout=timeout,
File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/connection/http_urllib3.py", line 252, in perform_request
self._raise_error(response.status, raw_data)
File "/opt/matchminer-api/env/lib/python3.7/site-packages/elasticsearch/connection/base.py", line 254, in _raise_error
status_code, error_message, additional_info
elasticsearch.exceptions.RequestError: RequestError(400, 'illegal_argument_exception', 'Action/metadata line [1] contains an unknown parameter [_type]')
Hi,
When I try to access https://localhost:5555, it returns This site cannot be reached
. I checked log, it displayed the error below. I'm wondering if this file is missed. If yes, could you upload this file? Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.