GithubHelp home page GithubHelp logo

minotour-docker's Introduction

Running minoTour with Docker

minoTour is developed as a docker container, so docker must be installed and correctly set up if this is how you wish to run minotour.

Docker is a set of platform as a service products that use OS-level virtualisation to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files; they can communicate with each other through well-defined channels.

If Docker is not installed - instructions for installing can be found here. Make sure Docker is installed and running at the end of the installation. You can check this by running the command below:

docker --version

Docker Engine version 19.03+ is required, with 20+ preferred.

Running pre-built minotour

First, clone our Docker minotour project from here

git clone https://github.com/LooseLab/docker-compose-test.git

Change to the docker-compose-test folder

cd docker-compose-test

Start the images (-d means daemon, so the images run in the background)

docker-compose up -d

First time setup

If the images are not present they will be downloaded.

After a few seconds, you should be able to access minoTour at http://localhost:8100/

To create an admin user:

docker exec -it web python manage.py createsuperuser

Or to create a standard user just use the GUI!

Uploading references, primer schemes and Centrifuge indexes to minoTour

References less than 256Mb in size can be uploaded using the minoTour GUI. If your reference is bigger, you will need to add it on the command line.

Inside your docker-compose-test directory there exists a folder named minotour_files. This folder is mounted into the docker container, and thus minoTour can access files inside this folder. To start with the folder should currently have a structure of:

minotour_files
|
|-- primer_schemes
|   |-- nCoV-2019
|   |   |-- V1
|   |   |-- V2
|   |   |-- V3
|   |   `-- V4
|   |-- Nipah
|   |   `-- V1
|   |-- SARS-CoV-2 -> nCoV-2019
|   |-- scripts
|   `-- ZaireEbola
|       |-- V1
|       |-- V2
|       `-- V3
`-- references
    |-- minimap2_indexes
    `-- reference_files

To add references over 256Mb

Note: You will need your API-KEY, found in your profile details after login

Copy the reference (or sym-link if that's your flavour) into the minotour_files/references/reference_files directory and then run the following command

docker exec web python manage.py add_reference -k <API-KEY> /var/lib/minotour/apps/data/references/reference_files/<REF_FILE_NAME>

To add Primer schemes for the artic pipeline

For a scheme two files are required:

  • <new_species>.scheme.bed
  • <new_species>.reference.fasta

Where new species has the same name. Their is a management page for primer schemes in the right hand side bar of minoTour. Drag and drop the two files (or use the built in browser selector), give them a version name and upload them. And you're good to go!

To add Centrifuge indexes

Create a centrifuge index folder in the minotour_files folder and add the .cf files for your index, for example -

minotour_files
|
|-- centrifuge_indexes
    |-- example.1.cf
    |-- example.2.cf
    |-- example.3.cf
    |-- example.4.cf

We do not distribute any indexes, a compressed pre-made index containing bacteria and viruses can can be dowloaded from JHU.

Stop the containers currently running

docker-compose stop

Set the MT_CENTRIFUGE_INDEX variable in the docker-compose.yml to /var/lib/minotour/data/centrifuge_indexes/example where example is the name of your index, excluding any numbers and file suffix.

Restart your containers -

docker-compose up -d

Debugging

Please refer to Debugging.md

minotour-docker's People

Contributors

adoni5 avatar mattloose avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

minotour-docker's Issues

minFQ won't run - what am I doing wrong?

minotour (docker) is running on localhost, 8100

When I try to run minFQ with key and port, I get an error - "failed to connect to all addresses". Any tips or suggestions?

minFQ -k 201d23868f2e9722066a15459889f1d80c894b0c -p 8100
<_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses"
debug_error_string = "{"created":"@1654699347.250973082","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3217,"referenced_errors":[{"created":"@1654699347.250971952","description":"failed to connect to all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":165,"grpc_status":14}]}"

best regards
R

[2022-06-08T15:02:40] INFO [minFQ.minknow_connection.init:402] Installed minknow API Version 5.0.0.1
[2022-06-08T15:02:40] DEBUG [minknow_api.load_grpc_credentials:341] Reading ssl certificate
[2022-06-08T15:02:40] DEBUG [minknow_api.load_grpc_credentials:361] Getting local token
[2022-06-08T15:02:40] DEBUG [root.get_local_authentication_token_file:53] Unable to connect to manager on port '9502' to retrieve local auth token path
Traceback (most recent call last):
File "/home/brug.r/Documents/virtualPythonEnv/minotour/lib/python3.8/site-packages/minknow_api/manager.py", line 51, in get_local_authentication_token_file
return service.local_authentication_token_path().path
File "/home/brug.r/Documents/virtualPythonEnv/minotour/lib/python3.8/site-packages/minknow_api/manager_service.py", line 1095, in local_authentication_token_path
return run_with_retry(self._stub.local_authentication_token_path,
File "/home/brug.r/Documents/virtualPythonEnv/minotour/lib/python3.8/site-packages/minknow_api/manager_service.py", line 85, in run_with_retry
result = MessageWrapper(method(message, timeout=timeout), unwraps=unwraps)
File "/home/brug.r/Documents/virtualPythonEnv/minotour/lib/python3.8/site-packages/grpc/_channel.py", line 946, in call
return _end_unary_response_blocking(state, call, False, None)
File "/home/brug.r/Documents/virtualPythonEnv/minotour/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses"
debug_error_string = "{"created":"@1654714960.599588044","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3217,"referenced_errors":[{"created":"@1654714960.599587075","description":"failed to connect to all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":165,"grpc_status":14}]}"

Problems with GUI

I have a problem with the graphical user interface. I used the Docker solution to install minoTour. I´ve also created an account - a normal user as well as an admin - but it is just possible for me to login in minoTour from the computer where I´ve installed the normal user and the admin. I cannot login in minoTour from another computer although the installation of the other computer was succesful.

local files on rorys computer not available on matts computer...

I think we've had this issue before....

Running ARTIC:

    return func(*args, **kwargs)
  File "/home/rory/Projects/minotourapp/artic/views.py", line 462, in get_artic_summary_table_data
    amplicon_stats = get_amplicon_stats(
  File "/home/rory/Projects/minotourapp/artic/utils.py", line 544, in get_amplicon_stats
    raise e
  File "/home/rory/Projects/minotourapp/artic/utils.py", line 541, in get_amplicon_stats
    with open(coverage_path, "rb") as fh:
FileNotFoundError: [Errno 2] No such file or directory: '/home/rory/Projects/data/Artic/artic/Temp_results/86_470_artic/unclassified/unclassified.coverage.dat'
"GET /api/v1/artic/visualisation/summary-table-data/?flowcellId=86&_=1624443820418 HTTP/1.1" 500 24337
(base) root@db42b19e2189:/var/lib/minotour/apps/minotourapp# tail -f 200 minotour.log
tail: cannot open '200' for reading: No such file or directory
==> minotour.log <==
  File "/home/rory/envs/minotour/lib/python3.8/site-packages/rest_framework/decorators.py", line 50, in handler
    return func(*args, **kwargs)
  File "/home/rory/Projects/minotourapp/artic/views.py", line 462, in get_artic_summary_table_data
    amplicon_stats = get_amplicon_stats(
  File "/home/rory/Projects/minotourapp/artic/utils.py", line 544, in get_amplicon_stats
    raise e
  File "/home/rory/Projects/minotourapp/artic/utils.py", line 541, in get_amplicon_stats
    with open(coverage_path, "rb") as fh:
FileNotFoundError: [Errno 2] No such file or directory: '/home/rory/Projects/data/Artic/artic/Temp_results/86_470_artic/unclassified/unclassified.coverage.dat'
"GET /api/v1/artic/visualisation/summary-table-data/?flowcellId=86&_=1624443820418 HTTP/1.1" 500 24337
^C```

Using minFQ with ip (other than 127.0.0.1) seem to have port issues (failed to connect all addresses)?

When I try to run minFQ / minotour on Docker, I get RPC error.
minFQ -k 2645d7e2ebalablabla -ip 145.xxx.xxx.xxx -hn localhost -p 8100 -nf
->
<_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses"
debug_error_string = "{"created":"@1653160488.112296657","description":"Failed to pick subchannel",

Debugging -> tail -f -n 100 /var/lib/minotour/logs/celery.log
16624:[2022-05-20 23:02:08,011: ERROR/ForkPoolWorker-1] artic.utils.update_pangolin[06bde82d-5482-4173-87ed-5103c3a353c6]: b'pangolin updated to v4.0.6\npangolearn updated to 2022-04-22\nconstellations updated to v0.1.10\nscorpio updated to v0.3.17\npango-designation updated to v1.9\n'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.