Comments (8)
Would this result in a (HTTP error code) 500 error?
EDIT (googles white screen of death.... yep, OK, thats our problem)
The user im troubleshooting with will get an HTTP 500 error and not be able to display the job page if he sets the max hit limit to 250. (Setting it to 500 and your failsafe kicks in).
Setting to 100, and the page loads.
We can check the logs and see that the blast job ran successfully in all cases.
So it sounds like for HWG implementing your module, 250 is somewhere between your fix and what our server can manage. I can change the code on our local branch to cut off at 101 instead of at 500.
from tripal_blast.
Possible solutions:
-
As suggested, use a streaming XML parser like XMLReader. But this requires a PHP package that isn't typically included in a standard PHP installation. The number of results displayed will still need to be limited, and it isn't simply a matter of limiting the number of hits as a single hit can contain many hsps.
-
Limit the number of hits when the XML is created from the ASN result file but save the full results in the other file formats (HTML, tab, GFF) . This can be done with a parameter to blast_formatter. But this only limits the hits; there can still be lots and lots of hsps for each hit.
-
Since HSPs can be limited by the BLAST command, add an hsp limit to the configuration options. But this prevents people from searching with repetitive sequence, for example, to look at the distribution of a particular repeat sequence.
-
Detect if there are too many results after running BLAST and converting ASN into HTML, tab, and GFF outputs. If too many, run BLAST again with hsps limited to: (# hits)/(total # HSPs) and generate the XML from those results. But this greatly slows down the job.
Reactions? Any other ideas?
from tripal_blast.
Hmmmmm.... my gut reaction is that #4 is the safest but I worry since BLAST is a heuristic we would end up with results shown on the page that are not in the downloadable files :-(
from tripal_blast.
I have an operational proof-of-concept for option 4, but the concern about getting different results from the second BLAST execution is very valid ... and quit likely to happen. Unfortunately, blast_formatter won't permit limiting hsps like the BLAST programs do.
from tripal_blast.
What about if we do a combination of #2 & #3?
A. create an advanced option for the blast form that limits HSPs and set its default to something reasonable. That way if they need to do repeat analysis they can change it but most people should not.
B. When creating the XML from the ASN we limit the number of hits.
A takes care of the HSPs and B takes care of the hits keeping us in a safe zone and ensuring we have the same results on the page as in the files. There is still a little bit of rope for people to hang themselves but it is MUCH better then where we are currently at. Thoughts?
from tripal_blast.
Hmm. I'll play with this idea for a bit.
from tripal_blast.
Ok, thanks for taking this one on and Yay for coffee and a fresh morning ;-)
from tripal_blast.
Hello. I tried blast recently with a huge file and am getting the same problem. Sample given below
Query:
CM030740.1 Citrus sinensis isolate HZAU_DHSO_2021 chromosome 1, whole genome shotgun sequence
AATTTTACCATTAAACATTTAGTTAATTGGAATATGAAGTTTAGGACCGCCAACTTCATATTGTAATGCT
CTAGTTAATAATTAAAATTTTATTAAGTTAGTGATTGTAGATTTCATGTTAACATTAGAAGGGTGGAATT
ATTTTAGGTTCAATAATAATGGTTTCATGTTATAGTCAATGATTTTAGGTTAAACATTTAGAATATTTTT
AGTTAATGGTCTGTAAATATACGGATGTGATGTTTCGGGTTCAGTGTCGTACTCAAAGTAAGACTGTAAT
ATACTTTGTTCTCTAGTATATAGTGATTATACATATATCAATGCATACTAAGTTAAACTCTATTAAAAAA
TGCATGGAACAAGCTCGTGGATGGTGCAAGGTTGTATTGCATTAAATTACTTTTCATGTTACATAAATAT
AATATTGTAGTCGACGTTATTATCCTATATTATAATAACTAATAAAAATGATTATTCAGTTTCTTATAAT
ATAATAAGCTTAAACTATATAACAACATATTGTAATATTTAATTATTTATTAATGGTTTAGTATTACGTT
AAAACATTTCGCAAGCCAATAGCCTAATATAACTATTTGTAGGTTTAATTAAAAAATGAATAACAAACCC
CGCAAATATTAATGCATCTAAAATAGTATGAATTTAATTTGAACGGTATTATATAAACTATAAATATAGC
ATAAAATTTAATTTGTTAAATAATACAGTTCATAATTATTCGAAAACAATGATATATCATAATACAATTC
AAGATAATGCAATATAGCCAAAAAAATTATGTTAATAGTCCACAGTGCTTACAATTGAATATGTTGTAAT
GTTAATTGTAACTGTCTCATTTTTATTTTTTTAATGATTAAATACAATAACATAAAATTAATATATTTAA
AAATGGATATTATATTTCAATAATGATTTCAAATCTATTTGTTACTATGAAAATAATTAATGGACTATGA
ATTGTAGGAATTGAATTACATAAAATATAGTTACTCTAATTACTATCTTATTCTAAAAAACAAAATTAGA
ATTATTTTTCTTTAAAAAATGCTATAACTTTAAAATCCAGCTTATTTGCCAAAATCTTCCCATTTTTTGG
TTCGCCCTCCACAAAAGGGCTTCAAGCCTGTTGATTAGGTTGCCCAACATTCTAAACAAAACAATTCAGC
CGTTGTTCCATCCATTATCACTTCTGGACATATTAAATATCATCCATTAGATAAGATTTTAACATATTCA
GTGGTTGAGATTTTGTTGACTGTAAGTTACATTTAACGTAACTTAGGGATTGCCTACAAAAATGAAGCTA
Target:
scaffold00001 length=5927163
TTTTGTATTCTATGTCCTCTGATCTTTATACTTCTTCATTTTGTCTTTGCAAGAACCGGA
ATTATGGGTACATCACAAATTCTCTAGGTGTGACTTGTGTTGTGGGGCCTTTTTTTtACA
TTTCCATATTGCAAGTATTTTTTTGCTACCATTGGTATATTTGTCTGTTAAAATCAATCT
GCTTTCACTTATGTTCGTGCGTTCTTGTTCCCTCGCCTTGCAATTGCATATCTCAAATTA
TCTTTCTTACTTTGATTTAGATGGCCAAGGTTTTAAGCTAACTTTTTACAATGCCAATTT
TTAAATGGTTTTCTAATGCTGTTCAAAGTTGCAGCCTTTACTTCGTATATTTGTCAGGTT
CTGACGGGTGCGGTCGGCGGCGGGGGCTATAGCATGCGGTCTCGAGAGCCGCAAAGAAAA
ATGGGTGGTTTTCCCGGTTTCGGCCATAACTCGTGATCGGGGCCTCCGATTCTGGTTCCG
TTTCGTCCCACGGGACCAGCCGGGCGGGGGCATCGGATTGCAAAAGTCTTTAAATTTGAA
TTTGATTTAAGTTTATATAGTTTGAACACAAAAACTAGCCATTACGGACAAAAACAACAA
ATAGTCGGCTAGCCTATTAATTAGCCAGATCGCCTCTTAATACAGTGCAAGTTACCGTTG
CAATTTGAATTTTGCTGCAGTGATGCTATAGTAACACTATTTTTtAAAATTTCATTGTTA
CCTAAAACTTTTTTATAATTTGACTATGACCCAAAATGTCATAAAATTTTGCAAATATAT
CAAATTTCAGAATTTCTAAATAATGCGCGTTATTCTTAAAACTTTTTGAAATTATGCTAT
GGCCTAAAACTTTATAATATTTTTCAAAGAGATTCTTCTCAGAATTTTAACATAATGCTT
ATTTATTTCAAAGTTCCCAAAATCTTTTTCAGTTTAATCCAAACTTTGAAAAACACTCAA
ATCCTCAAAATACTCGTCTTATAAATATAAAATCTTTTTGTTTATAAAAAGTAATGATTT
ATTAAATAAAATCTTGAGCTTTTTCAATGCTAAACTATACATATATCAAATCATACTGGC
TTTATAAGAATTTGTTGCAATAATGACTCCGCAGAGCTAAACTTTGCTCTTGATCAAGCC
I am either getting a memory error such as
Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 324671885 bytes) in /var/www/html/teak-wood-genes-drupal7-test/includes/database/database.inc on line 2284
Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 23072768 bytes) in Unknown on line 0
Warning: Unknown: Cannot call session save handler in a recursive manner in Unknown on line 0
Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 98570240 bytes) in /var/www/html/teak-wood-genes-drupal7-test/includes/bootstrap.inc on line 3876
or a "gatekeeping timeout" error. But the files are getting generated and stored...
I was asked to contact the developers/maintainers to see what can be done. I am using Tripal v3.100.
Edit 1: I tried blast in blast website using the same files and got this error
"Length limit exceeded. Please reduce your query/subject sequence length to 10,000,000 letters or less."
So is my file too large? I still need to use that file so how to I configure so?
Edit 2: When I tried blasting in the web ui using a very small query size, I am getting some results but I am getting more errors (hundreds which are basically repetitions of the same few)
Deprecated function: imagefilledpolygon(): Using the $num_points parameter is deprecated in generate_blast_hit_image() (line 654 of /var/www/html/teak-wood-genes-drupal7-test/sites/all/modules/tripal_blast/api/blast_ui.api.inc).
Deprecated function: imagefilledpolygon(): Using the $num_points parameter is deprecated in generate_blast_hit_image() (line 654 of /var/www/html/teak-wood-genes-drupal7-test/sites/all/modules/tripal_blast/api/blast_ui.api.inc).
Deprecated function: Implicit conversion from float 299.4699930400503 to int loses precision in generate_blast_hit_image() (line 671 of /var/www/html/teak-wood-genes-drupal7-test/sites/all/modules/tripal_blast/api/blast_ui.api.inc).
Deprecated function: Implicit conversion from float 50.219298245614034 to int loses precision in generate_blast_hit_image() (line 672 of /var/www/html/teak-wood-genes-drupal7-test/sites/all/modules/tripal_blast/api/blast_ui.api.inc).
Deprecated function: Implicit conversion from float 50.219298245614034 to int loses precision in generate_blast_hit_image() (line 696 of /var/www/html/teak-wood-genes-drupal7-test/sites/all/modules/tripal_blast/api/blast_ui.api.inc).
Warning: A non-numeric value encountered in include() (line 92 of /var/www/html/teak-wood-genes-drupal7-test/sites/all/modules/tripal_blast/theme/blast_report_alignment_row.tpl.php).
Warning: A non-numeric value encountered in include() (line 106 of /var/www/html/teak-wood-genes-drupal7-test/sites/all/modules/tripal_blast/theme/blast_report_alignment_row.tpl.php).
Warning: A non-numeric value encountered in include() (line 111 of /var/www/html/teak-wood-genes-drupal7-test/sites/all/modules/tripal_blast/theme/blast_report_alignment_row.tpl.php).
from tripal_blast.
Related Issues (20)
- Error - "File is not accessible" HOT 2
- max_target_seqs parameter HOT 3
- Overly aggressive regex to remove database extension in api/blast_ui.api.inc HOT 1
- Undefined variable HOT 1
- Trimming of input sequence HOT 1
- Issues with online docs HOT 2
- Blast analyses failed: "We encountered an error and are unable to load your BLAST results" HOT 4
- Upgrade to Tripal 4 + Drupal 9 HOT 10
- Progressive Display Controls HOT 1
- PHP8 Deprecation warnings HOT 1
- PHP8 catches mismatched parameter names HOT 1
- Tripal blast not run smoothly when using 'drush trp-run-jobs' command line HOT 4
- Modify queue priority of BLAST jobs
- Tripal Blast doesn't show up in the Extensions listing
- Blast Overview Javascript issues
- Update blastx, tblastn, blastp forms HOT 2
- Current Documentation HOT 1
- Unable to install via Composer
- Undefined array key "cvitjs_location" in blast_ui HOT 2
- 'File is not accessible' issue in tripal blast (related to sites/default/files/tripal_blast/ or /tmp?) HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tripal_blast.