ropolomx / flumatch

Automating Prokka annotations, BLAST runs and generation of BLAST reports

FluMatch

Automating Prokka annotations, BLAST runs and generation of BLAST reports

Process

The script does the following:

1. Reads in your FASTA format file containing contig sequences
2. Annotates your contigs with Prokka
3. Performs a BLAST search of the annotated coding sequences generated by Prokka with BLAST vs. the local database of your preference
4. Generates a report table with the following fields:

Query	Matching Strain	Query Coverage	Percent ID	Identities	Alignment Length	Query Start	Query End	Subject End	Query Length	Subject Length	E-value

Dependencies

The script has been written on Python 3.4.3 and it has been tested on Linux only. Windows implementation is being investigated, but it is not available yet. The installation of a Linux virtual machine (preferrably Ubuntu) on your Windows computer is strongly recommended for the meanwhile.

In order to run this script, you would need to have the following installed on your system:

1. Python 3

2. NCBI BLAST+

This is the standalone version of BLAST that you can run locally on your machine. flumatch.py calls the blastn algorithm.

Instructions for Linux:

sudo apt-get install ncbi-blast+

3. Prokka: microbial annotation software

Please follow the installation instructions in the official Prokka repository

BLAST database set up

flumatch.py will do a BLAST search of your annotated sequences vs. a local database. You need to build said database. The instructions to do this are:

1. Download the sequences from NCBI or FluDB that you want to compare to your contigs. In my case I downloaded all the Influenza Virus A sequences that existed in NCBI on May 26, 2016 (taxid:11320). I have a Perl script that you can use to do that from the command line.
2. Once you have the database you want, you need to type in the following commands:

makeblastdb -in filename.fasta -dbtype nucl -title filename -out filename

For example, if you have a FASTA file named avian.fasta or avian.fna which contains the sequences that you want to use as your BLAST database, you could do the following:

makeblastdb -in avian.fasta -dbtype nucl -title avian -out avian

After you run that command you will see that three new files will be created in the directory with the extensions *.nhr, *.nin, and *.nsq. These are the files that BLAST will use to search your sequences against.

Usage

python3 flumatch.py --blast-db /path/to/blastdb -r name_of_report_file.txt -p name_of_prokka_folder contigs.fasta

Arguments and options:

The most important thing to keep in mind is that the name of the FASTA file that contains the contigs you want to analyze should be written at the very end of the command.

--blast-db: This argument is required The local BLAST database that you want to search your annotated sequences against. See the Set up section above.

-p or --prokka-dir: The name of the directory where the Prokka output will be stored. Default = the program will create a sub-directory with the name of the file you are

-t or --top-hits : The number of top BLAST hits for each annotated CDS that you want to see in the final report. Default = 10

-r or --report-out: The name of the report text file. This is a tab-separated file that you can open in R or Excel. Default = TopBLASThits.txt

Sample data

I have included a folder with sample data to try the script. The folder contains the following files and directories

• La_Habana_test.fasta which contains the 8 segments of the publicly available strain A/swine/La/Habana/130/2010/H1N1 (Genbank Accession numbers HE584753.1 to HE584760.1).

• TopBLASThits.txt: an example report of the BLAST report of the annotated strain. The version of blastn used for this was 2.2.28+. The current version of online BLAST is 2.3.1+.

• A directory called La_Habana_test that has the output of the Prokka annotation process

flumatch's Issues

Updating local blast databases periodically

Very related to Issue #11 .

Two avenues can be taken here:

1. Use NCBI's Perl script and run a cron job as suggested here; or
2. Use the solution proposed by @dorbarker

for i in $(seq -w 0 99); do
    wget -q --continue --wait 10 --random-wait ftp://ftp.ncbi.nlm.nih.gov/blast/db/nt.${i}.tar.gz &&
    wget -q --continue --wait 10 --random-wait ftp://ftp.ncbi.nlm.nih.gov/blast/db/nt.${i}.tar.gz.md5 &&
    echo "Successfully retrieved nt.${i}.tar.gz" ||
    echo "nt.${i}.tar.gz failed or does not exist.";
done

Split BLAST subject names

It would be good to parse and split the BLAST subject names so that each category would be displayed in a separate column/field

Current:

Query	Subject
La_Habana_test_00001 Polymerase basic protein 2	Influenza A virus (A/swine/Pinar del Rio/3/2010(H1N1)) segment 1, complete genome

Desirable:

Query	Subject Accession	Subject Strain Name	Subject Subtype	Subject Segment / Gene
La_Habana_test_00001 Polymerase basic protein 2	HE589463.1	Influenza A virus A/swine/Pinar del Rio/3/2010	H1N1	Segment 1

Retrieve additional metadata from Genbank

Currently, the script returns a table with the annotated contig (query), target strain and other BLAST coverage information. It would be very nice to add metadata such as country by retrieving it from Genbank records. This could perhaps be done by feeding a list of accessions to a Genbank entrez search (in the style of this example. Alternatively, we could have a reference metadata table from FluDB, and then just cross-reference a list of accessions from the BLAST results to the information of that table

If flumatch can't find local BLAST DB then run makeblastdb or download pre-formatted DBs

1. Specify path to blast files
2. Check if there are databases there
3. Error if they provided wrong path

Use os.osdir

Show error message if FASTA headers are longer than character limit allowed by Prokka

Currrently, the script stops (hangs up, really) at "Please rename your contigs or use --centre XXX to generate clean contig names" if FASTA headers are longer than 20 characters (prokka 1.11). If this happens, the program needs to exit.

Better yet: since this is a very common occurrence, a regex or a sed command could be integrated into the script so that this is taken care of during program run. Alternatively, sharing a sed command with the user to shorten the FASTA header could be a short-term solution.

Add concatenation of files

Python or cat command ... or for loop from shell

Integrate pathotyping

Design method to differentiate between H5 and H7 hi-path and low-path and classify strains accordingly. This can be part of the same script or part of a downstream application.

Specify working directory

Use os.makedir to create and os.access to check the directories exist

Make script Python 3 friendly

Make few changes to libraries imported and syntax

Run BLAST from *.ffn file generated by Prokka

Currently the BLAST search uses the user-defined contig FASTA (i.e. args.contigs) as the query. However, using the strain/strain.ffn file generated by Prokka as the BLAST query is important because the FASTA headers of that file contain the annotation designated by Prokka.

Desired output:

Query (Prokka annotation)	Strain
La_Habana_test_00001 Polymerase basic protein 2	Target Strain Name

Current output:

Query (FASTA header from user-defined file)	Strain
HE584753.1	Target Strain Name

Alternatively, including both columns is also desirable:

Contig name	Prokka annotation	Strain
HE584753.1	La_Habana_test_00001 Polymerase basic protein 2	Target Strain Name

Integrate SPADES assembly

It would be great to allow users to either start with raw reads and perform assembly with SPAdes or to start from assembled contigs. If we start with SPAdes assembly, then we should integrate annotation, BLAST search and report generation in the same script (the same way it is currently done if one starts with assembled contigs).