All of my various bioinformatic scripts fit for public consumption (loosely defining "fit")
License: MIT License
@taylorreiter ran into an issue while pushing updates yesterday, addition of Bio or biopython as modules through toml does not create a version of KEGG-decoder 'tanglegram' that produces an output. Any insight as to the correct module to install?
Other elements still run correctly.
Hi,
Thanks a lot for the nice tool, apologies if this is not where to ask this sort of question. For ubiquinol-cytochrome c reductase you are using K00411 (PetA) and K00410 (FbcH). K00410 is a fusion of K00412 (PetB) and K00413 (PetC). Almost nothing has FbcH except a few proteobacteria as far as I can tell, so ubiquinol-cytochrome c reductase is almost always incomplete in your tables. Following the kegg presence logic in this module might be best as a general representation for complex III-like things? https://www.genome.jp/dbget-bin/www_bget?M00151 with the definition ((K00412+K00413,K00410)+K00411),K03886+K03887+K03888,K03890+K03891+K03889, but at the very least probably best to do PetABC (K00411,12,13) with 0.33 presence value each instead of 0.5 value for PetA and FbcH.
Also not sure what the significance is of 859-860 in KEGG_decoder.py since its in the ubiquinol-cytochrome c reductase section but is doing stuff with 'NADH-quinone oxidoreductase'
Best,
Gray
Hello,
I have two issues popping up while trying to run KEGGDecoder. Firstly, the -v static option runs without issue, but the heatmap generated does not include the amino acid pathways at the end.
Secondly, when trying to run -v interactive with modified and concatenated kofamKOALA outputs from several genomes, I get the error:
(KEGGdecoder) Rebeccas-MacBook-Pro:kofamKOALA Becca$ KEGG-decoder -i all.txt -o out/all.ko -v interactive
Traceback (most recent call last):
File "/Users/Becca/miniconda3/envs/KEGGdecoder/bin/KEGG-decoder", line 8, in <module>
sys.exit(main())
File "/Users/Becca/miniconda3/envs/KEGGdecoder/lib/python3.8/site-packages/KEGGDecoder/KEGG_decoder.py", line 1447, in main
plotly_viz(genome, os.path.splitext(filehandle)[0] + ".html")
File "/Users/Becca/miniconda3/envs/KEGGdecoder/lib/python3.8/site-packages/KEGGDecoder/Plotly_viz.py", line 19, in plotly_viz
Most_Least_genome_df = hClust_most_least(genome_df)
File "/Users/Becca/miniconda3/envs/KEGGdecoder/lib/python3.8/site-packages/KEGGDecoder/KEGG_clustering.py", line 36, in hClust_most_least
genome_df = genome_df.ix[sort_dex]
File "/Users/Becca/miniconda3/envs/KEGGdecoder/lib/python3.8/site-packages/pandas/core/generic.py", line 5465, in __getattr__
return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'ix'
Any idea why I am getting this error and why I cannot visualize the amino acid pathways?
Thanks!
Hiya,
I'm here from the BVCN youtube channel and hoping to use KEGG-decoder for a project. I was hoping to use the TSV output to create my own heatmaps but the format I'm getting doesn't seem to be columns of pathways and rows of samples. I attached one of my function_out.list files.
kegg-decoder-output_matanushka1.txt
Hi there,
thanks for this neat tool!
Unfortunately, I am getting an error when executing KEGG-decoder -i test -o test.out -v static.
Traceback (most recent call last):
File "/home/user/user/py3-venv/bin/KEGG-decoder", line 10, in <module>
sys.exit(main())
File "/home/user/user/py3-venv/lib/python3.7/site-packages/KEGGDecoder/KEGG_decoder.py", line 1375, in main
genome = hClust_euclidean(genome)
File "/home/user/user/py3-venv/lib/python3.7/site-packages/KEGGDecoder/KEGG_clustering.py", line 10, in hClust_euclidean
linkage_matrix = linkage(genome_df, method='average', metric='euclidean')
File "/home/user/user/py3-venv/lib/python3.7/site-packages/scipy/cluster/hierarchy.py", line 716, in linkage
n = int(distance.num_obs_y(y))
File "/home/user/user/py3-venv/lib/python3.7/site-packages/scipy/spatial/distance.py", line 2276, in num_obs_y
raise ValueError("The number of observations cannot be determined on "
ValueError: The number of observations cannot be determined on an empty distance matrix.
The file test looks like this:
GB1UC4431350_v1_1640001|ID:61449066| K21498
GB1UC4431350_v1_1650001|ID:61449067| K07480
GB1UC4431350_v1_1680001|ID:61449069| K03630
GB1UC4431350_v1_1770001|ID:61449078| K03630
Hi,
I tried to install KEGGdecoder with pip and it would not work with python 3.8. Switching to python 3.7 solved the problem.
I think the error came from the installation of numpy (from what I understand of the long error log from pip).
I just thought this might be useful for others.
Hugo
Hi! Thank you for this simple to implement tool!
I haven't been able to find the version information of the KO-pathways database being used. Is this based on an online look-up or is there a database version that downloads with the conda install?
Thanks,
Robin
Heya!
Thanks all for the work on keggdecoder, is it super-helpful in grouping/summarizing :)
Some data led me to looking more closely at "methanogenesis via trimethylamine", which is currently flagged as complete with solely K14083 being detected, which is a trimethylamine methyltransferase.
In my limited looking into this, i think we need methyl coenzyme M reductase also? Sticking to KEGG and looking on the module page, it looks like K14083 (and maybe needing K14084 also(?)) is listed as taking trimethylamine to this intermediate C03920 compound, which then is expected to require K00399, K00401, and K00402 to perform reaction R04541 to actually generate methane.
It also seems something similar might be going on for the other things listed as "methanogenesis via ...", where these are being marked as complete if genes are present for just this first step, and not requiring the follow-up that produces methane.
If i'm not confusing myself here and the above looks right to others, do you think it makes more sense to have the modules like "methanogenesis via trimethylamine" require K00399, K00401, and K00402 also? Or, if they are deliberately kept separate, with the intent that the enzyme those 3 make up is represented in the "coenzyme M reduction to methane" grouping, then maybe the others should just be labeled as, e.g., "trimethylamine methyltransferase" rather than "methanogenesis via trimethylamine"?
I'm happy to submit a pull request with the appropriate modifications if that'd be helpful, but I also wanted to have a second set of eyes on what i'm thinking to make sure i'm not just confused.
Thanks!
Hello! I think the KEGG-Decoder is very useful. But it seem that there is a typo in the script KEGG_decoder.py.
line265-266
#citryl-CoA synthetase AND citryl-CoA lyase
if ('K15232' in ko_match and 'K15232' in ko_match and 'K15234' in ko_match):
One of the K15232 should be K15233. And, in my opinion, it is better to define this enzyme as:
if (('K15232' in ko_match or 'K15233' in ko_match) and 'K15234' in ko_match):
What do you think about it?
Best regards,
Cheng
Hi there
It seems that the estimation of the completeness of rTCA is only based on the last block (k15230, k15231..) in the current code.
Best,
Hi,
Sometimes, I prefer naming my MAGs with a format "sample_serial". If a sample name is SP, then the MAGs will be named SP_01, SP_02, and so on. In this case, a merged KEGG output file would look like below.
SP_01_00001 K00370
SP_01_00002 K00371
SP_01_00003 K00372
SP_02_00001 K01230
SP_02_00002 K01231
SP_02_00003 K01234
If I perform KEGG-Decoder with this file, the output will contain only one genome named "SP".
So, I tried modifying KEGG_decoder.py. When I modified lines 1410, 1411, 1413 (for the current version) as below, the result came out as I wanted.
info[0].split("_")[0] --> info[0].rsplit("_",1)[0]
Because I have little knowledge in Python and this tip is from just Google searches, I wish someone would check whether this modification has no unintended effects on the results.
Thanks.
hey!
I am running the following code using python/3.5.2-
$KEGG-decoder --input 272-14-prokka.faa-kopfam --output 272-14.list --vizoption static
On my input file which looks like this-
LAKMIEEK_00001 K06997
LAKMIEEK_00002 K09772
LAKMIEEK_00003 K01696
LAKMIEEK_00003 K01817
LAKMIEEK_00004 K01695
LAKMIEEK_00005 K03526
But I keep getting an error-
Traceback (most recent call last):
File "/home/j/jigyasa-arora/.local/bin/KEGG-decoder", line 11, in
sys.exit(main())
File "/home/j/jigyasa-arora/.local/lib/python3.5/site-packages/KEGGDecoder/KEGG_decoder.py", line 1288, in main
import pandas as pd
File "/home/j/jigyasa-arora/.local/lib/python3.5/site-packages/pandas/init.py", line 55, in
from pandas.core.api import (
File "/home/j/jigyasa-arora/.local/lib/python3.5/site-packages/pandas/core/api.py", line 5, in
from pandas.core.arrays.integer import (
File "/home/j/jigyasa-arora/.local/lib/python3.5/site-packages/pandas/core/arrays/init.py", line 1, in
from .array_ import array # noqa: F401
File "/home/j/jigyasa-arora/.local/lib/python3.5/site-packages/pandas/core/arrays/array_.py", line 7, in
from pandas.core.dtypes.common import (
File "/home/j/jigyasa-arora/.local/lib/python3.5/site-packages/pandas/core/dtypes/common.py", line 11, in
from pandas.core.dtypes.dtypes import (
File "/home/j/jigyasa-arora/.local/lib/python3.5/site-packages/pandas/core/dtypes/dtypes.py", line 53, in
class Registry:
File "/home/j/jigyasa-arora/.local/lib/python3.5/site-packages/pandas/core/dtypes/dtypes.py", line 84, in Registry
self, dtype: Union[Type[ExtensionDtype], str]
File "/apps/free72/python/3.5.2/lib/python3.5/typing.py", line 552, in getitem
dict(self.dict), parameters, _root=True)
File "/apps/free72/python/3.5.2/lib/python3.5/typing.py", line 512, in new
for t2 in all_params - {t1} if not isinstance(t2, TypeVar)):
File "/apps/free72/python/3.5.2/lib/python3.5/typing.py", line 512, in
for t2 in all_params - {t1} if not isinstance(t2, TypeVar)):
File "/apps/free72/python/3.5.2/lib/python3.5/typing.py", line 1077, in subclasscheck
if super().subclasscheck(cls):
File "/apps/free72/python/3.5.2/lib/python3.5/abc.py", line 225, in subclasscheck
for scls in cls.subclasses():
TypeError: descriptor 'subclasses' of 'type' object needs an argument
Any idea why I'm getting the following error with the attached input file?
$ KEGG-decoder -i test.txt
test.txt
-o test.out -v static
Traceback (most recent call last):
File "/home/nels329/anaconda3/envs/KEGG-Decoder/bin/KEGG-decoder", line 8, in
sys.exit(main())
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/KEGGDecoder/KEGG_decoder.py", line 1443, in main
genome = hClust_euclidean(genome)
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/KEGGDecoder/KEGG_clustering.py", line 15, in hClust_euclidean
clust = dendrogram(linkage_matrix, no_plot=True, labels=names, get_leaves=True)
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/scipy/cluster/hierarchy.py", line 3347, in dendrogram
_dendrogram_calculate_info(
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/scipy/cluster/hierarchy.py", line 3653, in _dendrogram_calculate_info
_dendrogram_calculate_info(
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/scipy/cluster/hierarchy.py", line 3653, in _dendrogram_calculate_info
_dendrogram_calculate_info(
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/scipy/cluster/hierarchy.py", line 3653, in _dendrogram_calculate_info
_dendrogram_calculate_info(
[Previous line repeated 10 more times]
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/scipy/cluster/hierarchy.py", line 3620, in _dendrogram_calculate_info
_dendrogram_calculate_info(
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/scipy/cluster/hierarchy.py", line 3653, in _dendrogram_calculate_info
_dendrogram_calculate_info(
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/scipy/cluster/hierarchy.py", line 3653, in _dendrogram_calculate_info
_dendrogram_calculate_info(
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/scipy/cluster/hierarchy.py", line 3653, in _dendrogram_calculate_info
_dendrogram_calculate_info(
[Previous line repeated 22 more times]
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/scipy/cluster/hierarchy.py", line 3620, in _dendrogram_calculate_info
_dendrogram_calculate_info(
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/scipy/cluster/hierarchy.py", line 3653, in _dendrogram_calculate_info
_dendrogram_calculate_info(
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/scipy/cluster/hierarchy.py", line 3653, in _dendrogram_calculate_info
_dendrogram_calculate_info(
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/scipy/cluster/hierarchy.py", line 3653, in _dendrogram_calculate_info
_dendrogram_calculate_info(
[Previous line repeated 950 more times]
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/scipy/cluster/hierarchy.py", line 3620, in _dendrogram_calculate_info
_dendrogram_calculate_info(
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/scipy/cluster/hierarchy.py", line 3550, in _dendrogram_calculate_info
_append_singleton_leaf_node(Z, p, n, level, lvs, ivl,
File "/home/nels329/anaconda3/envs/KEGG-Decoder/lib/python3.9/site-packages/scipy/cluster/hierarchy.py", line 3409, in _append_singleton_leaf_node
lvs.append(int(i))
RecursionError: maximum recursion depth exceeded while calling a Python object
Is there a typo for option 2 in the Wood Ljungdahl section of the KOALA_definitions.txt file?
Should it be:
acsB (K14138) AND CODH (cooS; K00198) OR aerobic CODH (coxSML; K03518 + K03519 + K03520)
Rather than:
acsB + cooS + coxSML
Hi,
I recently ran the module on my data. My query is how the range (0-1) for heatmap (txt and figure) is assigned.
For e.g., this function_out.list
Function | glycolysis | gluconeogenesis | TCA Cycle | NAD(P)H-quinone oxidoreductase | NADH-quinone oxidoreductase
genecall | 1 | 1 | 1 | 0.98 | 0.98
On what basis these numbers are decided. If it is related to completion of a pathway, how to infer the results?
Hi again.
As you mentioned in the last issue, I re-did the analysis using two genomes (Kofamscan run on two genomes separately (with headers like Prefix_sequentialnumber) and concatenate the two KO lists).
Concerning the svg output: Besides that the colors are not red instead of blue (I prefer blue ;-) ), There is no legend drawn with the command -v static. However, when I run the decode/expand code with KEGG_expander.py, the legend IS drawn.
Concerning the html output: This file looks faulty. The function names are all cropped and only one genome is displayed, without any heatmap. Have a look at the screenshot of part of the html file. If you need the files let me know.
Hi,
I installed KEGGDecoder and it ran well on one of my files but truncate the heatmap of the other with no error reported. Could not figure out why since the small file is just a subset the other one.
I have python 3.7 and a conda env, pip installed KEGGdecoder.
Thanks.
conda activate Decoder
KEGG-decoder --input 1.txt --output 1.list --vizoption interactive
#UBA9217, with new r95, 49 were assign as UBA9217
KEGG-decoder --input 2.txt --output 2.list --vizoption interactive
#the truncated heatmap
this is my file, with 1.txt fine and 2.txt truncate
2.txt
Hey!
I want to ask that based on the heatmap output, what is the best cut-off to use for "almost complete pathway" analysis in metagenomes?
If I have pathways with completeness greater than 0.6, would that be a good estimate of "almost complete" pathways?
Would you recommend only using "complete" pathways i.e. completeness =1 for downstream analysis?
Hi, we are using KEGG-decoder with KOfamscan's output. The heatmap generated has several pathways in beige. I'm assuming it's because only one or very few genes have been detected. Is there a way to filter out the pathways for which no sample has a value higher than a certain threshold, say 0.05 ?
Hi!
While checking KEGG_decoder.py I noticed that in the carotenoid backbone biosynthesis section K02291 is listed twice. I think crtI should be K10027.
Each pathway is manually curated an reflects biological knowledge. Some of these reflect KEGG modules, but it would be good if we could find citations to associate with the pathways. I know this is a big job, but something to think about!
This could also be a requirement for the addition of new pathways eventually.
import itertools
from Bio import Phylo
import tanglegram as tg
from scipy.spatial.distance import pdist, squareform
Hi,
The KEGG-decoder.py is very helpful to census pathways of the genome.
While I am a little confused about How to calculate the value of pathway.
For example:
#norBC if ('K04561' in ko_match and 'K02305' in ko_match): out_data['nitric oxide reduction'] = 1
#nifKDH
# if ('K02586' in ko_match and 'K02591' in ko_match and 'K02588' in ko_match):
# out_data['nitrogen fixation'] = 1
if ('K02586' in ko_match): out_data['nitrogen fixation'] += 0.33 if ('K02591' in ko_match): out_data['nitrogen fixation'] += 0.33 if ('K02588' in ko_match): out_data['nitrogen fixation'] += 0.33
For nitric oxide reduction, the value of nitric oxide reduction is one when both K04561 and K02305 must be present.
While for nitrogen fixation, the value of nitrogen fixation is the added value of each KO.
Best
chunxu
Hi,
When i run , kegg decoder with my MAG ko list i get the following error. I t strange because i have successfully used the same program a few days ago on much bigger data.
Terminal output
KEGG-decoder -i SN_MAG_00001-gene_calls.txt -o SN_MAG_00001-ko_FUNCTION_OUT.list -v static
Traceback (most recent call last):
File "/home/mcs/miniconda3/envs/vibrant/bin/KEGG-decoder", line 8, in
sys.exit(main())
File "/home/mcs/miniconda3/envs/vibrant/lib/python3.7/site-packages/KEGGDecoder/KEGG_decoder.py", line 1649, in main
genome = hClust_euclidean(genome)
File "/home/mcs/miniconda3/envs/vibrant/lib/python3.7/site-packages/KEGGDecoder/KEGG_clustering.py", line 15, in hClust_euclidean
clust = dendrogram(linkage_matrix, no_plot=True, labels=names, get_leaves=True)
File "/home/mcs/miniconda3/envs/vibrant/lib/python3.7/site-packages/scipy/cluster/hierarchy.py", line 3347, in dendrogram
above_threshold_color=above_threshold_color)
File "/home/mcs/miniconda3/envs/vibrant/lib/python3.7/site-packages/scipy/cluster/hierarchy.py", line 3633, in _dendrogram_calculate_info
above_threshold_color=above_threshold_color)
File "/home/mcs/miniconda3/envs/vibrant/lib/python3.7/site-packages/scipy/cluster/hierarchy.py", line 3633, in _dendrogram_calculate_info
above_threshold_color=above_threshold_color)
File "/home/mcs/miniconda3/envs/vibrant/lib/python3.7/site-packages/scipy/cluster/hierarchy.py", line 3633, in _dendrogram_calculate_info
above_threshold_color=above_threshold_color)
[Previous line repeated 57 more times]
File "/home/mcs/miniconda3/envs/vibrant/lib/python3.7/site-packages/scipy/cluster/hierarchy.py", line 3600, in _dendrogram_calculate_info
above_threshold_color=above_threshold_color)
File "/home/mcs/miniconda3/envs/vibrant/lib/python3.7/site-packages/scipy/cluster/hierarchy.py", line 3633, in _dendrogram_calculate_info
above_threshold_color=above_threshold_color)
File "/home/mcs/miniconda3/envs/vibrant/lib/python3.7/site-packages/scipy/cluster/hierarchy.py", line 3633, in _dendrogram_calculate_info
above_threshold_color=above_threshold_color)
File "/home/mcs/miniconda3/envs/vibrant/lib/python3.7/site-packages/scipy/cluster/hierarchy.py", line 3633, in _dendrogram_calculate_info
above_threshold_color=above_threshold_color)
[Previous line repeated 929 more times]
File "/home/mcs/miniconda3/envs/vibrant/lib/python3.7/site-packages/scipy/cluster/hierarchy.py", line 3600, in _dendrogram_calculate_info
above_threshold_color=above_threshold_color)
File "/home/mcs/miniconda3/envs/vibrant/lib/python3.7/site-packages/scipy/cluster/hierarchy.py", line 3512, in _dendrogram_calculate_info
leaf_label_func, i, labels)
File "/home/mcs/miniconda3/envs/vibrant/lib/python3.7/site-packages/scipy/cluster/hierarchy.py", line 3369, in _append_singleton_leaf_node
lvs.append(int(i))
RecursionError: maximum recursion depth exceeded while calling a Python object
KO file
SN_MAG_00001-gene_calls.txt
which will easily support multiple runs within the same directory
Hi!
The KO number for 'Mixed acid: Ethanol, Acetylaldehyde to Ethanol' is not counting properly.
In line 1041 of KEGG_decoder.py, out_data['Mixed acid: Ethanol, Acetylaldehyde to Ethanol'] == 1, not out_data['Mixed acid: Ethanol, Acetylaldehyde to Ethanol'] = 1 instead of out_data['Mixed acid: Ethanol, Acetylaldehyde to Ethanol'] == 1?
Hi!
I love the program and the visualizations! I have a question regarding using/running a customized
KEGG_decoder.py. I created a copy of the KEGG_decoder.py from my /anaconda3/envs/KEGGDecoder/lib/python3.6/site-packages/KEGGDecoder directory and added KO terms associated with the pentose phosphate pathway and methyltransferases associated in the Wood-Ljungdahl pathway. I then ran python KEGG_decoder_cs_copy.py --input GhostKO_JGI_KEGGDecoder_copy.txt --output GhostKO_JGI_KEGGDecoder_heatmap -v interactive in that same directory and got a ModuleNotFoundError: No module named 'Plotly_viz.py'; 'Plotly_viz' is not a package error. It repeats a similar error when I use -v static
I am not an experienced python coder but I would like to be able to use the customized KEGG_decoder_cs_copy.py. My question is how can I tweak the script so that it can "call" plotly_viz, make_tanglegram and hClust_euclidean.
I have attached the main directory as a zip file, which includes the KEGG_decoder_cs_copy.py.
Mahalo nui loa for any suggestions!
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.