MITE-Hunter produces no results about edta

[liu3zhenlab]

Shujun, thank you very much. This helps. I will update and test it.

[oushujun]

No you don't have to. The change is to copy over TIR results directly and pretend it's MITE result, which will be removed entirely in the next combination step. MITE-Hunter does not contribute a lot (usually just a couple sequences) so you won't loss too much of sensitivity.

…

On Mon, Aug 26, 2019, 2:28 AM Philipp Bayer ***@***.***> wrote: Thanks for that! So I'm still running the split-up EDTA_raw.pl with the four different types, after pulling the changes MITE just finishes immediately with the MITE-Hunter warning. Should I rerun the TIR step separately since that's where the changes seem to have happened? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#12?email_source=notifications&email_token=ABNX4NG7JOASUM5VKDHN7W3QGOAYNA5CNFSM4IMHJQHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5DQ3WY#issuecomment-524750299>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABNX4NC5WZKRGOIZOR4TMMLQGOAYNANCNFSM4IMHJQHA> .

[csxie-666]

Hi,shujun
Im using EDTA 1.5 on a nematode genome.The first running i got TIR errors(but i cant remmber error message....).Then I increase max user processes , TIR is running.
And i met same issue like QiushiLi, MITE's blastall produces no results. I change the version of blast from 2.2.6 to 2.2.2. MITE is running.

#################################################################
2019年 08月 25日 星期日 17:07:57 CST EDTA_raw: Check files and dependencies, prepare working directories.

2019年 08月 25日 星期日 17:07:57 CST Start to find LTR candidates.

2019年 08月 25日 星期日 17:07:57 CST Identify LTR retrotransposon candidates from scratch.

2019年 08月 25日 星期日 18:38:13 CST Finish finding LTR candidates.

2019年 08月 25日 星期日 18:38:13 CST Start to find TIR candidates.

2019年 08月 25日 星期日 18:38:13 CST Identify TIR candidates from scratch.

Species: others
Finish finding TIR candidates.

2019年 08月 25日 星期日 20:55:41 CST Start to find MITE candidates.

2019年 08月 25日 星期日 20:55:41 CST Identify MITE candidates from scratch.

2019年 08月 26日 星期一 00:00:38 CST Finish finding MITE candidates.

2019年 08月 26日 星期一 00:00:38 CST Start to find Helitron candidates.

2019年 08月 26日 星期一 00:00:38 CST Identify Helitron candidates from scratch.

2019年 08月 26日 星期一 02:04:48 CST Finish finding Helitron candidates.

2019年 08月 26日 星期一 02:04:48 CST Execution of EDTA_raw.pl is finished!

#################################################################
success~

[baozg]

After changing blast version from 2.2.26 to 2.2.22, the MITE finally has its result. It worked for me.
Thanks! @csxie-666

[oushujun]

Dear Sanzhen,

MITE-Hunter sometimes chokes and emits random errors. Normally a fresh rerun can solve such issues. You can just run the MITE module with EDTA_raw.pl under the same job directory.
Assume your genome file is genome.fa
Run:
rm -rf ./genome.fa.EDTA.raw/MITE
perl EDTA_raw.pl -genome genome.fa -type mite -t 10

Hope this helps!
Shujun

[oushujun]

I just pushed some new updates to EDTA, mainly to fix the TIR-Learner issue. Please reinstall EDTA and rerun it in the same work folder. Existing results will be reused so there is essentially no waste of time. Thank you for your patience and support!

[QiushiLi]

Dear Shujun,
In one of my fish cases, the MITE-Hunter choked on the blastall for more than 72h.

Here is the screen output

Sun Aug 18 03:38:01 MDT 2019 Dependency checking:
All passed!
Sun Aug 18 03:38:12 MDT 2019 Obtain raw TE libraries using various structure-based programs:
Sun Aug 18 03:38:12 MDT 2019 EDTA_raw: Check files and dependencies, prepare working directories.

Sun Aug 18 03:38:12 MDT 2019 Start to find LTR candidates.

Sun Aug 18 03:38:12 MDT 2019 Identify LTR retrotransposon candidates from scratch.

Sun Aug 18 04:53:51 MDT 2019 Finish finding LTR candidates.

Sun Aug 18 04:53:51 MDT 2019 Start to find TIR candidates.

Sun Aug 18 04:53:51 MDT 2019 Identify TIR candidates from scratch.

Species: others
Finish finding TIR candidates.

Sun Aug 18 08:17:50 MDT 2019 Start to find MITE candidates.

Sun Aug 18 08:17:50 MDT 2019 Identify MITE candidates from scratch.

The cmd hanging there:

blastall -i genome_raw_NC_200 -d genome_raw_NC_200 -e 1e-10 -p blastn -o genome_raw_NC_200.self -m 8 -v 60 -b 60 -a 24 -F F -G 4 -E 2 -q -3 -r 2 

I tried to rerun this specific cmd, and the error reproduced.
Will see if a refresh rerun can solve the issue.

Thanks,
Qiushi

[liu3zhenlab]

My rerunning with the updated EDTA failed at the TIR step again. Will check in more details to see what was going on.

[oushujun]

Hi Sanzhen,

Sorry the TIR issue is not resolved. Please let me know the specific error message so that I can work on it.

Best,
Shujun

[liu3zhenlab]

Shujun, here is the error message:

Fri Aug 16 21:13:30 CDT 2019 Identify TIR candidates from scratch.
Species: Maize
cat: xxx/TIR/Module1/TIR-Learner/*-select.csv: No such file or directory
Can't open ./TIR-Learner-Result/TIR-Learner_FinalAnn.fa: No such file or directory at /homes/liu3zhen/local/conda/EDTA2/EDTA/util/rename_tirlearner.pl line 18.
Warning: LOC list A.fasta.TIR.ext30.list is empty.
Warning: The TIR result file has 0 bp!

No output was generated for Module2 or Module3.

thanks,

-Sanzhen

[oushujun]

Hi @liu3zhenlab, @QiushiLi and @philippbayer,

I updated the TIR-Learner to v1.23 and hope to solve this issue. I also abandoned MITE-Hunter for its poor performance. Please update EDTA (git pull in /EDTA) and try again. Please let me know if it doesn't work on your side.

Best,
Shujun

[philippbayer]

Thanks for that!
So I'm still running the split-up EDTA_raw.pl with the four different types, after pulling the changes MITE just finishes immediately with the MITE-Hunter warning.
Should I rerun the TIR step separately since that's where the changes seem to have happened?

[liu3zhenlab]

Thanks a lot Shujun. MITE-Hunter is also slow. Glad you found an alternative solution.

[liu3zhenlab]

Shujun, I tested the v1.5 with a small data set. It showed errors as:

########################################################

Extensive de-novo TE Annotator (EDTA) v1.5

Shujun Ou ([email protected])

########################################################

Mon Aug 26 12:33:52 CDT 2019 Dependency checking:
All passed!
Mon Aug 26 12:33:57 CDT 2019 Obtain raw TE libraries using various structure-based programs:
Mon Aug 26 12:33:57 CDT 2019 EDTA_raw: Check files and dependencies, prepare working directories.

Mon Aug 26 12:33:57 CDT 2019 Start to find LTR candidates.

Mon Aug 26 12:33:57 CDT 2019 Identify LTR retrotransposon candidates from scratch.

    Usage: perl cleanup.pl -f sample.fa [options] > sample.cln.fa 
Options:
	-misschar	n	Define the letter representing unknown sequences; case insensitive; default: n
	-Nscreen	[0|1]	Enable (1) or disable (0) the -nc parameter; default: 1
	-nc		[int]	Ambuguous sequence len cutoff; discard the entire sequence if > this number; default: 0
	-nr		[0-1]	Ambuguous sequence percentage cutoff; discard the entire sequence if > this number; default: 1
	-minlen		[int]	Minimum sequence length filter after clean up; default: 100 (bp)
	-cleanN		[0|1]	Retain (0) or remove (1) the -misschar taget in output sequence; default: 0
	-trf		[0|1]	Enable (1) or disable (0) tandem repeat finder (trf); default: 1
	-trf_path	path	Path to the trf program

cp: cannot stat ‘TF05-1v012.fasta.mod.retriever.scn.adj’: No such file or directory
cp: cannot stat ‘TF05-1v012.fasta.LTRlib.fa’: No such file or directory
cp: cannot stat ‘TF05-1v012.fasta.LTRlib.fa’: No such file or directory
Error: LTR results not found!

ERROR: Raw LTR results not found in TF05-1v012.fasta.EDTA.raw/TF05-1v012.fasta.LTR.raw.fa at /homes/liu3zhen/.conda/envs/EDTA3/EDTA/EDTA.pl line 176.

[liu3zhenlab]

Seems we have a problem for numpy in our cluster. I changed to another server and now I can fully run EDTA :). Thanks Shujun.

[oushujun]

I will put the requirement for blast-legacy=2.2.22 in the next update. Thank you all for testing and reporting.