Greetings!

I have some questions about MeDuSa runs:

1. Is it necessary or even possible to modify run parameters in the underlying MUMmer software from the medusa command line prompt syntax? Specifically a discussion thread here talks about how considering only unique seeds, increase the minimum match length, and increase the minimum cluster length can reduce run times. OR is this only possible via manipulation of the underlying MUMmer parameter file - in which case, which one?

2. I am thinking of shaving off short contigs drastically (filter out shorter than N50, for example) and then used 5 of these filtered drafts with 50 - 100 scaffolds each, to assemble my target genome. Do you see any drawbacks using these sizefiltered reference drafts?

3. If the target and reference (pairwise comparison) have converged after a few rounds, then does the software return a message as below?
"Building the network...SORRY: No information found. Are you sure to have MUMmer packedge location in your PATH? If yes, the chosen drafts genomes don't provide sufficient information for scaffolding the target genome."

I am trying out medusa with a 900Mb genome. I think I am having issues with MUMmer - iI know not your software, but this may represent a hard limit in the genome size that one can scaffold. Please advise.

This is using medusa code I cloned from GitHub today.

java -jar /share/medusa/medusa.jar -f ../assemblies/long/ -i ../assemblies/reference/asm.long.fa -o meduasa.fasta -v
INPUT FILE:../assemblies/reference/asm.long.fa
------------------------------------------------------------------------------------------------------------------------
Running MUMmer...1: PREPARING DATA
2,3: RUNNING mummer AND CREATING CLUSTERS
# reading input file "asm_AllPaths.ntref" of length 812730787
# construct suffix tree for sequence of length 812730787
# (maximum reference length is 536870908)
# (maximum query length is 4294967295)
# process 8127307 characters per dot
/usr/bin/mummer: suffix tree construction failed: textlen=812730787 larger than maximal textlen=536870908
ERROR: mummer and/or mgaps returned non-zero
ERROR: Could not parse delta file, asm_AllPaths.delta
error no: 400

I am trying to run medusa on my university HPC cluster and I receive some STDOUT messages that look like warnings / errors: (please see attached file with complete STDOUT)

EthFoc173_Vs_6_MeDuSa-11374332.txt

WARNING: there was a problem deleting the temporary output files

File "medusa_scripts/netcon_mummer.py", line 5, in
import networkx as nx
ImportError: No module named networkx

And in the file nucmer.error created afresh, it reads:
20170405|132334| 6288| WARNING: there was a problem deleting the temporary output files

Could you please help me diagnose the list of things I need to fix on my compute cluster.
I've got this working this on my local machine, so I suspect some of this has to do with file / folder permissions. Yes?

Thanks!

Hi,

I want to use medusa to scaffold a (small) plant genome based on a close relative. I am getting a mummer error, is there a minimum version required? The system that I am working has "NUCmer (NUCleotide MUMmer) version 3.1". I do not have admin rights on this system, but I could install the newer version MUMmer locally. How can I tell medusa to use the local version instead?

This is the error I encountered:

20171109|142440| 20204| ERROR: The following critical files could not be used
20171109|142440| 20204| /opt/bio/MUMmer/postnuc
20171109|142440| 20204| /opt/bio/MUMmer/prenuc
20171109|142440| 20204| Check your paths and file permissions and try again

Thank you,

Beatriz

Hi,

I am running medusa for scaffolding 15 genomes, sizes from 5Mb to 12Mb. It went fine for 14 of them, taking a few hours each, but there is one stuck in the "Building the network..." phase - it's been running for 555 hours by now, with low memory and cpu usage.

Is there any way to check progress, more detailed than the normal output?

Regards,
Carlos

In the program parameters' description, it says:

The option -w2 is optional and allows for a sequence similarity based weighting scheme. Using a different weighting scheme may lead to better results.

I am a bit confused with the wordings. Which option would lead to better results? Turning -w2 on or off?

Hello,

I have finished running some tests using default parameters and it appears that the scaffolds being generated do not put a 100 "N" gap between the contigs but place the contigs right one after the other with no GAP. This is probably an error in the script, do I need to explicitly put -d 100 to force this behaviour?

Kind regards,

Juan Montenegro

The medusa.jar file seems to be missing from the project? Does it have to be compiled by the user? If so could you please point me to the instructions that can walk me through it?

It would be really great if a gexf (created by a previous medusa run) could be provided as an input.
or example, I now want to re-run, but this time use the flag -d or a different , so if I could use the graph as an input it would save a lot of running time.

I launched the program and after several days of work got the error:

4: FINISHING DATA
ERROR: The reference file may contain sequences with non-unique
       header Ids, please check your input files and try again
ERROR: postnuc returned non-zero
done.
------------------------------------------------------------------------------------------------------------------------
Building the network...

I check that reference not contain non-unique header:
how should I fix this?

[mag@smp medusa]$ cat reference_genomes/Canis_familiaris.fa | grep '>'
>10 dna_rm:chromosome chromosome:CanFam3.1:10:1:69331447:1 REF
>11 dna_rm:chromosome chromosome:CanFam3.1:11:1:74389097:1 REF
>12 dna_rm:chromosome chromosome:CanFam3.1:12:1:72498081:1 REF
>13 dna_rm:chromosome chromosome:CanFam3.1:13:1:63241923:1 REF
>14 dna_rm:chromosome chromosome:CanFam3.1:14:1:60966679:1 REF
>15 dna_rm:chromosome chromosome:CanFam3.1:15:1:64190966:1 REF
>16 dna_rm:chromosome chromosome:CanFam3.1:16:1:59632846:1 REF
>17 dna_rm:chromosome chromosome:CanFam3.1:17:1:64289059:1 REF
>18 dna_rm:chromosome chromosome:CanFam3.1:18:1:55844845:1 REF
>19 dna_rm:chromosome chromosome:CanFam3.1:19:1:53741614:1 REF
>1 dna_rm:chromosome chromosome:CanFam3.1:1:1:122678785:1 REF
>20 dna_rm:chromosome chromosome:CanFam3.1:20:1:58134056:1 REF
>21 dna_rm:chromosome chromosome:CanFam3.1:21:1:50858623:1 REF
>22 dna_rm:chromosome chromosome:CanFam3.1:22:1:61439934:1 REF
>23 dna_rm:chromosome chromosome:CanFam3.1:23:1:52294480:1 REF
>24 dna_rm:chromosome chromosome:CanFam3.1:24:1:47698779:1 REF
>25 dna_rm:chromosome chromosome:CanFam3.1:25:1:51628933:1 REF
>26 dna_rm:chromosome chromosome:CanFam3.1:26:1:38964690:1 REF
>27 dna_rm:chromosome chromosome:CanFam3.1:27:1:45876710:1 REF
>28 dna_rm:chromosome chromosome:CanFam3.1:28:1:41182112:1 REF
>29 dna_rm:chromosome chromosome:CanFam3.1:29:1:41845238:1 REF
>2 dna_rm:chromosome chromosome:CanFam3.1:2:1:85426708:1 REF
>30 dna_rm:chromosome chromosome:CanFam3.1:30:1:40214260:1 REF
>31 dna_rm:chromosome chromosome:CanFam3.1:31:1:39895921:1 REF
>32 dna_rm:chromosome chromosome:CanFam3.1:32:1:38810281:1 REF
>33 dna_rm:chromosome chromosome:CanFam3.1:33:1:31377067:1 REF
>34 dna_rm:chromosome chromosome:CanFam3.1:34:1:42124431:1 REF
>35 dna_rm:chromosome chromosome:CanFam3.1:35:1:26524999:1 REF
>36 dna_rm:chromosome chromosome:CanFam3.1:36:1:30810995:1 REF
>37 dna_rm:chromosome chromosome:CanFam3.1:37:1:30902991:1 REF
>38 dna_rm:chromosome chromosome:CanFam3.1:38:1:23914537:1 REF
>3 dna_rm:chromosome chromosome:CanFam3.1:3:1:91889043:1 REF
>4 dna_rm:chromosome chromosome:CanFam3.1:4:1:88276631:1 REF
>5 dna_rm:chromosome chromosome:CanFam3.1:5:1:88915250:1 REF
>6 dna_rm:chromosome chromosome:CanFam3.1:6:1:77573801:1 REF
>7 dna_rm:chromosome chromosome:CanFam3.1:7:1:80974532:1 REF
>8 dna_rm:chromosome chromosome:CanFam3.1:8:1:74330416:1 REF
>9 dna_rm:chromosome chromosome:CanFam3.1:9:1:61074082:1 REF
>X dna_rm:chromosome chromosome:CanFam3.1:X:1:123869142:1 REF

Moreover, I see the file "nucmer.error" which contains

20150623|091725| 29188| ERROR: postnuc returned non-zero

I wonder if parallelization of the underlying MUMmer would accelerate MeDuSa runs to make it much more attractive. For example, I wonder if this https://github.com/fritzsedlazeck/sge_mummer might be something to easily incorporate in your existing MeDuSa analysis pipelines. Thoughts?

Hello
I just tried the last version of Medusa. I first ran medusa using the test dataset, as indicated in the README file. Unfortunately I get stuck with "building network...." forever.
I have a HP workstation with 32 Gb RAM running with Debian Jessie.
Do you have an idea of what is going wrong?
Thank you in advance
Regards
Christophe

Hi!
I have cloned the medusa's folder. But I did not find the medusa.jar file when I cloned the medusa folder.

Regards

Thanks for the tool!

On python 3.6 the script netcon_mummer.py doesn't work because pickle has changed. Replacing the line from cPickle import dump to from pickle import dump fix the issue.

Regards

Hi,

this looks very interesting.

Is the .inp file ending required for the input fasta/s ? It seems that way from the example and from my first attempts, but I didn't see this documented anywhere.

Thanks,
Colin

Hi,
I'm trying to run medus, but I'm stuck with error message during running test. I've attached a screenshot of the terminal, could you please help me to fix this issue?

Thanks!

Hello,

I get the following error when attempting to build scaffolds even when I have the latest version of Biopython installed:

.....

reading input file "/home/janani/NovelStreptomycesWGS.May18/new6T.draft.Apr18/6T.direct.assembly/ref.genomes/GCA_002192455.1_ASM219245v1_genomic.fna" of length 8144417

matching query-file "/home/janani/NovelStreptomycesWGS.May18/new6T.draft.Apr18/6T.direct.assembly/ref.genomes/GCA_002192455.1_ASM219245v1_genomic.fna"

against subject-file "6T_GCA_002192455.ntref"

COMPLETETIME /opt/anaconda2/opt/mummer-3.23/mummer 6T_GCA_002192455.ntref 10.45

SPACE /opt/anaconda2/opt/mummer-3.23/mummer 6T_GCA_002192455.ntref 16.41

4: FINISHING DATA
done.

File "medusa_scripts/netcon_mummer.py", line 247, in
G=initialize_graph(query_genome)
File "medusa_scripts/netcon_mummer.py", line 154, in initialize_graph
from Bio.SeqIO import parse
ModuleNotFoundError: No module named 'Bio'
Exception in thread "main" java.lang.RuntimeException: Error: Network construction failed.
at Scaffolder.scaffolder(Unknown Source)
at Scaffolder.runOnTerminal(Unknown Source)
at Scaffolder.(Unknown Source)
at Scaffolder.main(Unknown Source)

Any suggestions on how to fix this error? Thanks!

First, I want to thank you for developing this tool!

I have ran into the following issue while using your tool:
When using the -gexf option, I see that the edges don't have the weight information. Is there a way to save the full graph information including the weight of each edge?

Here is some information on what I did (in case this is my fault):

I used 106 reference genomes, and I ran the following command:
java -jar medusa.jar -f 01_FASTA/ -i myFASTA.fa -v -gexf -o myFASTA-medusa-fixed.fa

I'm running the following software version and dependencies:
Medusa version 1.6
Java version 1.8.0_101-b13
Biopython version 1.69
mummer version 3.0
Python version 2.7.13
networkx version 1.11

Thank you very much!

Hi,

I tried running the test example and I kept getting the following error message:

Building the network...Traceback (most recent call last):
File "medusa_scripts/netcon_mummer.py", line 227, in
compute_distances(G,method=gap)
File "medusa_scripts/netcon_mummer.py", line 76, in compute_distances
if method==1: dist=distanceEstimation_mean(distances,outlier)
File "medusa_scripts/netcon_mummer.py", line 93, in distanceEstimation_mean
distance=v[-mask].mean()
TypeError: The numpy boolean negative, the - operator, is not supported, use the ~ operator or tead.
Exception in thread "main" java.lang. RuntimeException: Error: Network construction failed.
at Scaffolder.scaffolder(Unknown Source)
at Scaffolder.runOnTerminal(Unknown Source)
at Scaffolder.(Unknown Source)
at Scaffolder.main(Unknown Source)

I tried many ways to fix it, updated numpy and pandas as per suggestions online to no avail.

However, when I changed the '-' in line 93 to '~', distance=v[-mask].mean() in the netcon_mummer.py , it all worked well.

I wanted to let you know about the issue nevertheless, in case others have the same problem as well. Apologies if this is very easy/obvious, I am very new to the area.

Many thanks,

Yoana

I was getting an error when I was trying to in a small set of contigs and compare it to another genome.

The full run log looked like this

$ java -Xmx11g -jar medusa.jar -i contigs.fa -f ~/genome/ -o my_new_scaffolds.fa -v
INPUT FILE:contigs.fa
------------------------------------------------------------------------------------------------------------------------
Running MUMmer...1: PREPARING DATA
2,3: RUNNING mummer AND CREATING CLUSTERS
# reading input file "contigs_GCF_001624265.ntref" of length 2888899
# construct suffix tree for sequence of length 2888899
# (maximum reference length is 2305843009213693948)
# (maximum query length is 18446744073709551615)
# process 28888 characters per dot
#....................................................................................................
# CONSTRUCTIONTIME /opt/software/MUMmer/3.23--GCC-4.4.5/mummer contigs_GCF_001624265.ntref 0.84
# reading input file "/mnt/home/me/genome/GCF_001624265.1_ASM162426v1_genomic.fa" of length 777364094
# matching query-file "/mnt/home/me/genome/GCF_001624265.1_ASM162426v1_genomic.fa"
# against subject-file "contigs_GCF_001624265.ntref"
# COMPLETETIME /opt/software/MUMmer/3.23--GCC-4.4.5/mummer contigs_GCF_001624265.ntref 532.68
# SPACE /opt/software/MUMmer/3.23--GCC-4.4.5/mummer contigs_GCF_001624265.ntref 754.05
4: FINISHING DATA
done.
------------------------------------------------------------------------------------------------------------------------
Building the network...done.
------------------------------------------------------------------------------------------------------------------------
Cleaning the network...done.
------------------------------------------------------------------------------------------------------------------------
Exception in thread "main" java.lang.NullPointerException
        at graphs.MyGraph.reverseComplement(Unknown Source)
        at graphs.MyGraph.scaffoldStringSeq(Unknown Source)
        at graphs.MyGraph.readScaffoldsSeq(Unknown Source)
        at Scaffolder.scaffolder(Unknown Source)
        at Scaffolder.runOnTerminal(Unknown Source)
        at Scaffolder.<init>(Unknown Source)
        at Scaffolder.main(Unknown Source)

Using python 2.7.8, biopython 1.69, java 8, mummer 3.23

If there's any other info that can help let me know. I didn't see any line numbers in the stack trace so I can't see anything further myself

When using the -d option (for distance estimation) could there be a way to get the distance estimation as an output (for example in the SUMMARY file)?

Hi,

I had apparently very good results -still yet to verify them- scaffolding about 5000 large Pacbio contigs vs a 750MB genome with results inside 1-2 days.

I would like to scaffold ~200,000 much smaller Illumina contigs versus the same genome reference. Do you know how much time this may take ? I ran two assemblies for one week until the server had to be shut down for other reasons, but want to know how much time this may take. Is there any way to check the progress of the algorithm especially in the postnuc stage ?

Thanks,
Colin

Hello,

My anchoring failed after the software detected non-reference genomes in the reference genome folder. How do I restart and keep using the xxx.delta and xxx.coords files?

Thanks,
Liam

hello everyone,

I installed medusa in an environment in conda, but I'm having difficulty using it. Can anyone help?
best.

(medusa) kayobianco@ARQUEAS-XPS:~$ medusa -f /media/kayobianco/UBUNTU-HD/Illumina/Genomas/Neisseria_debora/N_meningitidis_CP021520 -i /media/kayobianco/UBUNTU-HD/Illumina/Genomas/Neisseria_debora/2779/unicycler_output/2779_assembly.fasta -o 2779_medusa_assembled -v
INPUT FILE:/media/kayobianco/UBUNTU-HD/Illumina/Genomas/Neisseria_debora/2779/unicycler_output/2779_assembly.fasta

Running MUMmer.../home/kayobianco/miniconda3/envs/medusa/share/medusa-1.6-2/script//mummerRunner.sh: linha 11: nucmer: command not found
/home/kayobianco/miniconda3/envs/medusa/share/medusa-1.6-2/script//mummerRunner.sh: linha 12: show-coords: command not found
done.

Building the network...Traceback (most recent call last):
File "/home/kayobianco/miniconda3/envs/medusa/share/medusa-1.6-2/script//netcon_mummer.py", line 6, in
from cPickle import dump
ModuleNotFoundError: No module named 'cPickle'
Exception in thread "main" java.lang.RuntimeException: Error: Network construction failed.
at Scaffolder.scaffolder(Unknown Source)
at Scaffolder.runOnTerminal(Unknown Source)
at Scaffolder.(Unknown Source)
at Scaffolder.main(Unknown Source)

Python2.7 is out of support. Is there any plan for a python 3 support?

Hi,
It would be very useful to use update the software to use the multithreaded version of Mummer (v 4.0+) to increase the speed of the program

Hi there

I was running the program and got an error at this stage

Building the network...  File "medusa_scripts/netcon_mummer.py", line 170
    G[n1][n2]['orientation_max']=list({tuple(i) for i in G[n1][n2]['orientation'] if G[n1][n2]['orientation'].count(i)==max_count})

Under python2 .6 it is a syntax error but works in 2.7

I just wanted to prove it to myself and made this small code which fails under 2.6 but works in 2.7

>>> r=[(1,1),(2,2),(3,3)]
>>> list({tuple(i) for i in r })

so in effect could be worth making the README say python2 .7 instead of 2.6 :)

Hi! I download the last version of medusa, but when I run the examples I get the error:

$ java -jar medusa.jar -f test/reference_genomes/ -i test/Rhodobacter_target.fna
...
Building the network... File "medusa_scripts/netcon_mummer.py", line 165
G[n1][n2]['orientation_max']=list({tuple(i) for i in G[n1][n2]['orientation'] if G[n1][n2]['orientation'].count(i)==max_count})
^
SyntaxError: invalid syntax
Exception in thread "main" java.lang.RuntimeException: Error: Network construction failed.
at Scaffolder.scaffolder(Unknown Source)
at Scaffolder.runOnTerminal(Unknown Source)
at Scaffolder.(Unknown Source)
at Scaffolder.main(Unknown Source)

Hi,
I just installed Medusa 1.6 today from this link, and Mummer from dmg for Mac OS X yesterday.
I tested and received error as follow. What do I need to fix (for installation)? Please see below.
Does this look like a bad MumMER installation a the underlying cause?
Thanks,
A

rri301as908:medusa anand$ sudo java -jar medusa.jar -f test/reference_genomes/ -i test/Rhodobacter_target.fna -v
INPUT FILE:test/Rhodobacter_target.fna

Running MUMmer...1: PREPARING DATA
2,3: RUNNING mummer AND CREATING CLUSTERS

reading input file "Rhodobacter_target_Rhodobacter_SW2_uid40865.ntref" of length 4386797

construct suffix tree for sequence of length 4386797

(maximum reference length is 536870908)

(maximum query length is 4294967295)

process 43867 characters per dot

#....................................................................................................

CONSTRUCTIONTIME /usr/local/bin/MUMmer3.23/mummer Rhodobacter_target_Rhodobacter_SW2_uid40865.ntref 1.45

reading input file "/medusa-master/medusa/test/reference_genomes/Rhodobacter_SW2_uid40865.fna.inp" of length 3514400

matching query-file "/medusa-master/medusa/test/reference_genomes/Rhodobacter_SW2_uid40865.fna.inp"

against subject-file "Rhodobacter_target_Rhodobacter_SW2_uid40865.ntref"

COMPLETETIME /usr/local/bin/MUMmer3.23/mummer Rhodobacter_target_Rhodobacter_SW2_uid40865.ntref 4.24

SPACE /usr/local/bin/MUMmer3.23/mummer Rhodobacter_target_Rhodobacter_SW2_uid40865.ntref 7.66

4: FINISHING DATA
1: PREPARING DATA
2,3: RUNNING mummer AND CREATING CLUSTERS

reading input file "Rhodobacter_target_Rhodobacter_sphaeroides_WS8N.ntref" of length 4386797

construct suffix tree for sequence of length 4386797

(maximum reference length is 536870908)

(maximum query length is 4294967295)

process 43867 characters per dot

#....................................................................................................

CONSTRUCTIONTIME /usr/local/bin/MUMmer3.23/mummer Rhodobacter_target_Rhodobacter_sphaeroides_WS8N.ntref 1.49

reading input file "/medusa-master/medusa/test/reference_genomes/Rhodobacter_sphaeroides_WS8N.fna.inp" of length 4417791

matching query-file "/medusa-master/medusa/test/reference_genomes/Rhodobacter_sphaeroides_WS8N.fna.inp"

against subject-file "Rhodobacter_target_Rhodobacter_sphaeroides_WS8N.ntref"

COMPLETETIME /usr/local/bin/MUMmer3.23/mummer Rhodobacter_target_Rhodobacter_sphaeroides_WS8N.ntref 4.45

SPACE /usr/local/bin/MUMmer3.23/mummer Rhodobacter_target_Rhodobacter_sphaeroides_WS8N.ntref 8.53

4: FINISHING DATA
done.

Building the network...Traceback (most recent call last):
File "medusa_scripts/netcon_mummer.py", line 5, in
import networkx as nx
ImportError: No module named networkx
Exception in thread "main" java.lang.RuntimeException: Error: Network construction failed.
at Scaffolder.scaffolder(Unknown Source)
at Scaffolder.runOnTerminal(Unknown Source)
at Scaffolder.(Unknown Source)
at Scaffolder.main(Unknown Source)

/bin/python2.7 is hard coded into medusa/src/Scaffolder.java - if you're still making updates, could you please address this to be compatible with python 3?