GithubHelp home page GithubHelp logo

Failed to run gemBS extract about gembs HOT 24 CLOSED

heathsc avatar heathsc commented on August 13, 2024
Failed to run gemBS extract

from gembs.

Comments (24)

heathsc avatar heathsc commented on August 13, 2024

from gembs.

chhylp123 avatar chhylp123 commented on August 13, 2024

Hello, Can you post the output of the command: gemBS --loglevel debug extract ? Thanks, Simon
…
On 18 Mar 2019, at 04:20, chhylp123 @.***> wrote: Hi, When I am running gemBS extract, it reported: : No BCF files are available for methylation extraction. However, the gemBS call seems worked successfully, it reported: : Command call started at 2019-03-18 10:15:43.098498 : : ----------- Methylation Calling -------- : Reference : /home/nhpcc502/ssd/gemBS_reference/Homo_sapiens.GRCh38.dna.primary_assembly.fa : Species : Homo_sapiens : Right Trim : 0 : Left Trim : 0 : Chromosomes : ['KI270303.1', 'GL000225.1', 'KI270396.1', 'KI270713.1', '3', '5', 'GL000195.1', 'KI270747.1', 'KI270710.1', 'KI270394.1', 'KI270708.1', 'KI270727.1', '9', 'KI270539.1', 'KI270310.1', 'KI270371.1', 'KI270311.1', 'KI270519.1', 'KI270374.1', '8', 'KI270723.1', 'GL000226.1', 'KI270751.1', '7', 'GL000216.2', 'KI270438.1', 'KI270518.1', 'KI270510.1', 'KI270590.1', 'KI270384.1', 'KI270724.1', 'KI270728.1', 'KI270362.1', 'KI270411.1', 'KI270337.1', 'KI270390.1', 'KI270583.1', 'KI270378.1', 'KI270517.1', 'KI270364.1', '15', 'KI270740.1', 'KI270516.1', 'KI270717.1', 'KI270333.1', 'KI270722.1', 'KI270754.1', 'KI270731.1', 'KI270742.1', '16', 'GL000008.2', 'KI270393.1', 'KI270584.1', 'KI270580.1', 'KI270363.1', 'KI270389.1', 'KI270373.1', 'KI270442.1', 'GL000213.1', 'KI270733.1', 'KI270329.1', 'KI270316.1', 'X', 'KI270423.1', 'KI270320.1', 'KI270755.1', 'KI270746.1', 'KI270741.1', 'KI270709.1', 'KI270712.1', 'KI270528.1', 'KI270544.1', 'GL000220.1', 'KI270757.1', 'GL000214.1', '1', 'KI270465.1', '11', 'KI270418.1', 'KI270756.1', 'KI270530.1', 'KI270511.1', 'KI270512.1', 'GL000221.1', 'KI270467.1', 'KI270302.1', 'KI270334.1', 'KI270739.1', 'KI270417.1', 'KI270330.1', 'KI270735.1', 'KI270340.1', 'KI270507.1', 'KI270714.1', 'GL000194.1', 'KI270422.1', 'KI270752.1', 'KI270468.1', '13', 'KI270748.1', 'KI270737.1', 'KI270429.1', 'KI270381.1', 'KI270375.1', 'KI270386.1', 'KI270749.1', 'KI270412.1', 'KI270305.1', 'KI270726.1', 'KI270391.1', 'KI270508.1', '22', 'KI270466.1', 'KI270732.1', 'KI270338.1', 'KI270315.1', 'KI270716.1', 'KI270322.1', '14', 'KI270383.1', 'KI270448.1', 'KI270425.1', 'KI270424.1', 'KI270715.1', 'KI270372.1', 'GL000219.1', 'KI270743.1', 'KI270515.1', 'KI270335.1', '19', 'KI270711.1', 'KI270317.1', 'KI270582.1', 'KI270366.1', 'KI270589.1', '10', 'KI270304.1', 'GL000009.2', 'KI270385.1', 'KI270387.1', 'KI270379.1', 'KI270509.1', 'GL000224.1', 'KI270753.1', 'KI270729.1', 'KI270312.1', 'GL000208.1', 'KI270738.1', 'KI270336.1', 'KI270435.1', 'KI270419.1', '12', 'KI270376.1', 'KI270587.1', 'KI270395.1', 'KI270579.1', 'GL000218.1', 'KI270581.1', 'KI270521.1', '2', 'KI270721.1', 'KI270744.1', 'Y', 'KI270414.1', 'KI270420.1', '20', 'KI270736.1', 'MT', '6', 'KI270593.1', 'KI270730.1', 'KI270725.1', 'KI270720.1', '21', 'KI270734.1', 'KI270591.1', 'KI270718.1', 'KI270522.1', 'GL000205.2', 'KI270548.1', 'KI270750.1', 'KI270529.1', 'KI270706.1', 'KI270707.1', 'KI270388.1', '17', 'KI270382.1', '18', 'KI270538.1', 'KI270588.1', '4', 'KI270392.1', 'KI270719.1', 'KI270745.1'] : Threads : 4 : Sample: SRR5453774 Bam: ./mapping/SRR5453774/SRR5453774.bam : : Methylation Calling... : Methylation call done, samples performed: SRR5453774 And the bcf folder has the following files: bs_call_SRR5453774_10.err bs_call_SRR5453774_4.err contigs_SRR5453774_18.bed SRR5453774_10.bcf SRR5453774_18.bcf SRR5453774_4.bcf bs_call_SRR5453774_11.err bs_call_SRR5453774_5.err contigs_SRR5453774_19.bed SRR5453774_10.json SRR5453774_18.json SRR5453774_4.json bs_call_SRR5453774_12.err bs_call_SRR5453774_6.err contigs_SRR5453774_1.bed SRR5453774_11.bcf SRR5453774_19.bcf SRR5453774_5.bcf bs_call_SRR5453774_13.err bs_call_SRR5453774_7.err contigs_SRR5453774_20.bed SRR5453774_11.json SRR5453774_19.json SRR5453774_5.json bs_call_SRR5453774_14.err bs_call_SRR5453774_8.err contigs_SRR5453774_21.bed SRR5453774_12.bcf SRR5453774_1.bcf SRR5453774_6.bcf bs_call_SRR5453774_15.err bs_call_SRR5453774_9.err contigs_SRR5453774_22.bed SRR5453774_12.json SRR5453774_1.json SRR5453774_6.json bs_call_SRR5453774_16.err bs_call_SRR5453774_X.err contigs_SRR5453774_2.bed SRR5453774_13.bcf SRR5453774_20.bcf SRR5453774_7.bcf bs_call_SRR5453774_17.err bs_call_SRR5453774_Y.err contigs_SRR5453774_3.bed SRR5453774_13.json SRR5453774_20.json SRR5453774_7.json bs_call_SRR5453774_18.err contigs_SRR5453774_10.bed contigs_SRR5453774_4.bed SRR5453774_14.bcf SRR5453774_21.bcf SRR5453774_8.bcf bs_call_SRR5453774_19.err contigs_SRR5453774_11.bed contigs_SRR5453774_5.bed SRR5453774_14.json SRR5453774_21.json SRR5453774_8.json bs_call_SRR5453774_1.err contigs_SRR5453774_12.bed contigs_SRR5453774_6.bed SRR5453774_15.bcf SRR5453774_22.bcf SRR5453774_9.bcf bs_call_SRR5453774_20.err contigs_SRR5453774_13.bed contigs_SRR5453774_7.bed SRR5453774_15.json SRR5453774_22.json SRR5453774_9.json bs_call_SRR5453774_21.err contigs_SRR5453774_14.bed contigs_SRR5453774_8.bed SRR5453774_16.bcf SRR5453774_2.bcf SRR5453774_X.bcf bs_call_SRR5453774_22.err contigs_SRR5453774_15.bed contigs_SRR5453774_9.bed SRR5453774_16.json SRR5453774_2.json SRR5453774_X.json bs_call_SRR5453774_2.err contigs_SRR5453774_16.bed contigs_SRR5453774_X.bed SRR5453774_17.bcf SRR5453774_3.bcf SRR5453774_Y.bcf bs_call_SRR5453774_3.err contigs_SRR5453774_17.bed contigs_SRR5453774_Y.bed SRR5453774_17.json SRR5453774_3.json SRR5453774_Y.json Could you help me to solve this problem? Thank you in advance. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#49>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPdyK3Ht9JpQVc2rlw6XFV8ABoj_vWks5vXwXrgaJpZM4b44CB.

It still reported:

: No BCF files are available for methylation extraction.

Thank you very much.

from gembs.

heathsc avatar heathsc commented on August 13, 2024

from gembs.

chhylp123 avatar chhylp123 commented on August 13, 2024

And from the command: gemBS call --dry-run ? Simon
…
On 18 Mar 2019, at 06:34, chhylp123 @.> wrote: Hello, Can you post the output of the command: gemBS --loglevel debug extract ? Thanks, Simon … x-msg://3/# On 18 Mar 2019, at 04:20, chhylp123 @.> wrote: Hi, When I am running gemBS extract, it reported: : No BCF files are available for methylation extraction. However, the gemBS call seems worked successfully, it reported: : Command call started at 2019-03-18 10:15:43.098498 : : ----------- Methylation Calling -------- : Reference : /home/nhpcc502/ssd/gemBS_reference/Homo_sapiens.GRCh38.dna.primary_assembly.fa : Species : Homo_sapiens : Right Trim : 0 : Left Trim : 0 : Chromosomes : ['KI270303.1', 'GL000225.1', 'KI270396.1', 'KI270713.1', '3', '5', 'GL000195.1', 'KI270747.1', 'KI270710.1', 'KI270394.1', 'KI270708.1', 'KI270727.1', '9', 'KI270539.1', 'KI270310.1', 'KI270371.1', 'KI270311.1', 'KI270519.1', 'KI270374.1', '8', 'KI270723.1', 'GL000226.1', 'KI270751.1', '7', 'GL000216.2', 'KI270438.1', 'KI270518.1', 'KI270510.1', 'KI270590.1', 'KI270384.1', 'KI270724.1', 'KI270728.1', 'KI270362.1', 'KI270411.1', 'KI270337.1', 'KI270390.1', 'KI270583.1', 'KI270378.1', 'KI270517.1', 'KI270364.1', '15', 'KI270740.1', 'KI270516.1', 'KI270717.1', 'KI270333.1', 'KI270722.1', 'KI270754.1', 'KI270731.1', 'KI270742.1', '16', 'GL000008.2', 'KI270393.1', 'KI270584.1', 'KI270580.1', 'KI270363.1', 'KI270389.1', 'KI270373.1', 'KI270442.1', 'GL000213.1', 'KI270733.1', 'KI270329.1', 'KI270316.1', 'X', 'KI270423.1', 'KI270320.1', 'KI270755.1', 'KI270746.1', 'KI270741.1', 'KI270709.1', 'KI270712.1', 'KI270528.1', 'KI270544.1', 'GL000220.1', 'KI270757.1', 'GL000214.1', '1', 'KI270465.1', '11', 'KI270418.1', 'KI270756.1', 'KI270530.1', 'KI270511.1', 'KI270512.1', 'GL000221.1', 'KI270467.1', 'KI270302.1', 'KI270334.1', 'KI270739.1', 'KI270417.1', 'KI270330.1', 'KI270735.1', 'KI270340.1', 'KI270507.1', 'KI270714.1', 'GL000194.1', 'KI270422.1', 'KI270752.1', 'KI270468.1', '13', 'KI270748.1', 'KI270737.1', 'KI270429.1', 'KI270381.1', 'KI270375.1', 'KI270386.1', 'KI270749.1', 'KI270412.1', 'KI270305.1', 'KI270726.1', 'KI270391.1', 'KI270508.1', '22', 'KI270466.1', 'KI270732.1', 'KI270338.1', 'KI270315.1', 'KI270716.1', 'KI270322.1', '14', 'KI270383.1', 'KI270448.1', 'KI270425.1', 'KI270424.1', 'KI270715.1', 'KI270372.1', 'GL000219.1', 'KI270743.1', 'KI270515.1', 'KI270335.1', '19', 'KI270711.1', 'KI270317.1', 'KI270582.1', 'KI270366.1', 'KI270589.1', '10', 'KI270304.1', 'GL000009.2', 'KI270385.1', 'KI270387.1', 'KI270379.1', 'KI270509.1', 'GL000224.1', 'KI270753.1', 'KI270729.1', 'KI270312.1', 'GL000208.1', 'KI270738.1', 'KI270336.1', 'KI270435.1', 'KI270419.1', '12', 'KI270376.1', 'KI270587.1', 'KI270395.1', 'KI270579.1', 'GL000218.1', 'KI270581.1', 'KI270521.1', '2', 'KI270721.1', 'KI270744.1', 'Y', 'KI270414.1', 'KI270420.1', '20', 'KI270736.1', 'MT', '6', 'KI270593.1', 'KI270730.1', 'KI270725.1', 'KI270720.1', '21', 'KI270734.1', 'KI270591.1', 'KI270718.1', 'KI270522.1', 'GL000205.2', 'KI270548.1', 'KI270750.1', 'KI270529.1', 'KI270706.1', 'KI270707.1', 'KI270388.1', '17', 'KI270382.1', '18', 'KI270538.1', 'KI270588.1', '4', 'KI270392.1', 'KI270719.1', 'KI270745.1'] : Threads : 4 : Sample: SRR5453774 Bam: ./mapping/SRR5453774/SRR5453774.bam : : Methylation Calling... : Methylation call done, samples performed: SRR5453774 And the bcf folder has the following files: bs_call_SRR5453774_10.err bs_call_SRR5453774_4.err contigs_SRR5453774_18.bed SRR5453774_10.bcf SRR5453774_18.bcf SRR5453774_4.bcf bs_call_SRR5453774_11.err bs_call_SRR5453774_5.err contigs_SRR5453774_19.bed SRR5453774_10.json SRR5453774_18.json SRR5453774_4.json bs_call_SRR5453774_12.err bs_call_SRR5453774_6.err contigs_SRR5453774_1.bed SRR5453774_11.bcf SRR5453774_19.bcf SRR5453774_5.bcf bs_call_SRR5453774_13.err bs_call_SRR5453774_7.err contigs_SRR5453774_20.bed SRR5453774_11.json SRR5453774_19.json SRR5453774_5.json bs_call_SRR5453774_14.err bs_call_SRR5453774_8.err contigs_SRR5453774_21.bed SRR5453774_12.bcf SRR5453774_1.bcf SRR5453774_6.bcf bs_call_SRR5453774_15.err bs_call_SRR5453774_9.err contigs_SRR5453774_22.bed SRR5453774_12.json SRR5453774_1.json SRR5453774_6.json bs_call_SRR5453774_16.err bs_call_SRR5453774_X.err contigs_SRR5453774_2.bed SRR5453774_13.bcf SRR5453774_20.bcf SRR5453774_7.bcf bs_call_SRR5453774_17.err bs_call_SRR5453774_Y.err contigs_SRR5453774_3.bed SRR5453774_13.json SRR5453774_20.json SRR5453774_7.json bs_call_SRR5453774_18.err contigs_SRR5453774_10.bed contigs_SRR5453774_4.bed SRR5453774_14.bcf SRR5453774_21.bcf SRR5453774_8.bcf bs_call_SRR5453774_19.err contigs_SRR5453774_11.bed contigs_SRR5453774_5.bed SRR5453774_14.json SRR5453774_21.json SRR5453774_8.json bs_call_SRR5453774_1.err contigs_SRR5453774_12.bed contigs_SRR5453774_6.bed SRR5453774_15.bcf SRR5453774_22.bcf SRR5453774_9.bcf bs_call_SRR5453774_20.err contigs_SRR5453774_13.bed contigs_SRR5453774_7.bed SRR5453774_15.json SRR5453774_22.json SRR5453774_9.json bs_call_SRR5453774_21.err contigs_SRR5453774_14.bed contigs_SRR5453774_8.bed SRR5453774_16.bcf SRR5453774_2.bcf SRR5453774_X.bcf bs_call_SRR5453774_22.err contigs_SRR5453774_15.bed contigs_SRR5453774_9.bed SRR5453774_16.json SRR5453774_2.json SRR5453774_X.json bs_call_SRR5453774_2.err contigs_SRR5453774_16.bed contigs_SRR5453774_X.bed SRR5453774_17.bcf SRR5453774_3.bcf SRR5453774_Y.bcf bs_call_SRR5453774_3.err contigs_SRR5453774_17.bed contigs_SRR5453774_Y.bed SRR5453774_17.json SRR5453774_3.json SRR5453774_Y.json Could you help me to solve this problem? Thank you in advance. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#49 <#49>>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPdyK3Ht9JpQVc2rlw6XFV8ABoj_vWks5vXwXrgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPdyK3Ht9JpQVc2rlw6XFV8ABoj_vWks5vXwXrgaJpZM4b44CB. It still reported: : No BCF files are available for methylation extraction. Thank you very much. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPdzUztjVC0nNcR4X0G3F52BHI_D38ks5vXyV9gaJpZM4b44CB.

It reported:

gemBS call -b SRR5453774 --pool @pool_1 --no-merge -r -u -U -k
gemBS merge-bcfs -b SRR5453774 -r

Thank you.

from gembs.

heathsc avatar heathsc commented on August 13, 2024

from gembs.

chhylp123 avatar chhylp123 commented on August 13, 2024

OK, so the calling did not finish for some reason. If the calling had finished correctly then you would see only the merged bcf file for the sample rather that then the individual chromosome bcfs. gemBS extract looks for this merged bcf file, which is why you are getting the error message. You seem to have the bcfs for the chromosomes, but you don’t have the calls for pool1 which groups together the minor contigs. If you run the gemBS call command again (with your normal options) it will perform the single outstanding call operation and merge the bcfs to create the sample bcf file. Simon

You mean I need to run gemBS call twice? I have already delete all bcf files, and try to gemBS call again, but it still cannot generate a single bcf file.

from gembs.

heathsc avatar heathsc commented on August 13, 2024

from gembs.

chhylp123 avatar chhylp123 commented on August 13, 2024

No, you shouldn’t need to run it twice, but the first time it did not complete for some reason. If you re-ran the gemBS call command it would have performed just the required operations to complete the calling (or given you an error as to why it did not work). That is what the output of the gemBS call --dry-run command tells us - it shows the operations that gemBS needs to do to complete the calling. In your example, the output showed that it only needed to perform one call and the merge step. If you deleted the existing BCFs then you will have to run gemBS db-sync before running gemBS call. Simon
…
On 18 Mar 2019, at 07:03, chhylp123 @.***> wrote: OK, so the calling did not finish for some reason. If the calling had finished correctly then you would see only the merged bcf file for the sample rather that then the individual chromosome bcfs. gemBS extract looks for this merged bcf file, which is why you are getting the error message. You seem to have the bcfs for the chromosomes, but you don’t have the calls for pool1 which groups together the minor contigs. If you run the gemBS call command again (with your normal options) it will perform the single outstanding call operation and merge the bcfs to create the sample bcf file. Simon You mean I need to run gemBS call twice? I have already delete all bcf files, and try to gemBS call again, but it still cannot generate a single bcf file. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB.

Thank you. I will try as you said. If it works, I will tell you as soon as possible.

from gembs.

chhylp123 avatar chhylp123 commented on August 13, 2024

No, you shouldn’t need to run it twice, but the first time it did not complete for some reason. If you re-ran the gemBS call command it would have performed just the required operations to complete the calling (or given you an error as to why it did not work). That is what the output of the gemBS call --dry-run command tells us - it shows the operations that gemBS needs to do to complete the calling. In your example, the output showed that it only needed to perform one call and the merge step. If you deleted the existing BCFs then you will have to run gemBS db-sync before running gemBS call. Simon
…
On 18 Mar 2019, at 07:03, chhylp123 @.***> wrote: OK, so the calling did not finish for some reason. If the calling had finished correctly then you would see only the merged bcf file for the sample rather that then the individual chromosome bcfs. gemBS extract looks for this merged bcf file, which is why you are getting the error message. You seem to have the bcfs for the chromosomes, but you don’t have the calls for pool1 which groups together the minor contigs. If you run the gemBS call command again (with your normal options) it will perform the single outstanding call operation and merge the bcfs to create the sample bcf file. Simon You mean I need to run gemBS call twice? I have already delete all bcf files, and try to gemBS call again, but it still cannot generate a single bcf file. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB.

Hi,

I have ran gemBS call and gemBS extract successfully, but a new problem existed. The result files of gemBS extract are too small:

mextr_SRR5453774.err 0
SRR5453774_cpg.txt.gz 88K
SRR5453774_cpg.txt.gz.tbi 19K
tabix_SRR5453774_cpg.err 0

However, both the bam file and the bcf file are 14GB. And my commands of gemBS call and gemBS extract are:

gemBS call -j 7 -t 4 -g 0 -f 0
gemBS extract -j 28

Could you help me to solve this problem? It is very important for me. Thank you very much!

from gembs.

heathsc avatar heathsc commented on August 13, 2024

from gembs.

chhylp123 avatar chhylp123 commented on August 13, 2024

What coverage do you have? If you have low coverage (< 30x) then the number of CpGs that can be reliably called can be very low. Since the cpg files only contain confidently called CpGs, you can therefore end up with few results. There are two ways to get around this: (a) Use the Encode BedMethyl output format (-B option for gemBS extract). This will give an output for every CpG in the reference that is covered in the experiment. (b) Use the reference bias option for the calling (i.e., gemBS call -R 1000). This will heavily weight the genotype calling to favour homozygous reference calls. This will require removing the existing bcf files, running gemBS db-sync and then re-calling. Hope this helps, Simon
…
On 18 Mar 2019, at 11:56, chhylp123 @.> wrote: No, you shouldn’t need to run it twice, but the first time it did not complete for some reason. If you re-ran the gemBS call command it would have performed just the required operations to complete the calling (or given you an error as to why it did not work). That is what the output of the gemBS call --dry-run command tells us - it shows the operations that gemBS needs to do to complete the calling. In your example, the output showed that it only needed to perform one call and the merge step. If you deleted the existing BCFs then you will have to run gemBS db-sync before running gemBS call. Simon … x-msg://19/# On 18 Mar 2019, at 07:03, chhylp123 @.> wrote: OK, so the calling did not finish for some reason. If the calling had finished correctly then you would see only the merged bcf file for the sample rather that then the individual chromosome bcfs. gemBS extract looks for this merged bcf file, which is why you are getting the error message. You seem to have the bcfs for the chromosomes, but you don’t have the calls for pool1 which groups together the minor contigs. If you run the gemBS call command again (with your normal options) it will perform the single outstanding call operation and merge the bcfs to create the sample bcf file. Simon You mean I need to run gemBS call twice? I have already delete all bcf files, and try to gemBS call again, but it still cannot generate a single bcf file. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 (comment) <#49 (comment)>>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB. Hi, I have ran gemBS call and gemBS extract successfully, but a new problem existed. The result files of gemBS extract are too small: 0 March 18 18:17 mextr_SRR5453774.err 88K March 18 18:41 SRR5453774_cpg.txt.gz 19K March 18 18:41 SRR5453774_cpg.txt.gz.tbi 0 March 18 18:41 tabix_SRR5453774_cpg.err However, both the bam file and the bcf file are 14GB. And my commands of gemBS call and gemBS extract are: gemBS call -j 7 -t 4 -g 0 -f 0 gemBS extract -j 28 Could you help me to solve this problem? It is very important for me. Thank you very much! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB.

Thank you very much! My dataset is about 10X. So you mean since it is low coverage, maybe just thousands of CpG sites are reliable?

As for -B option, I have tried:

gemBS extract -B -j 28

But it reported:

:
: Command extract started at 2019-03-18 19:45:05.924688
:
: Methylation Extraction...
2019-03-18 19:45:06,197 ERROR: Process '/usr/local/lib/python3.5/dist-packages/gemBS/bin/bcftools' finished with -11
Traceback (most recent call last):
File "/usr/local/bin/gemBS", line 13, in
load_entry_point('gemBS==3.2.7', 'console_scripts', 'gemBS')()
File "/usr/local/lib/python3.5/dist-packages/gemBS/commands.py", line 157, in gemBS_main
instances[args.command].run(args)
File "/usr/local/lib/python3.5/dist-packages/gemBS/production.py", line 1397, in run
self.do_filter(v)
File "/usr/local/lib/python3.5/dist-packages/gemBS/production.py", line 1491, in do_filter
snps=snps,snp_list=self.snp_list,snp_db=self.snp_db)
File "/usr/local/lib/python3.5/dist-packages/gemBS/init.py", line 1173, in methylationFiltering
raise ValueError("Error while extracting methylation calls.")
ValueError: Error while extracting methylation calls.

May I ask how can I fix this?

from gembs.

heathsc avatar heathsc commented on August 13, 2024

from gembs.

chhylp123 avatar chhylp123 commented on August 13, 2024

I mean at 10x you can not expect to call genotypes and methylation from your sample (I should probably make this clearly in the documentation), so we have to assume no sequence variants. The error you are seeing, however, is a bug in the extraction. Is there any possibility that you could share the bcf file (with [email protected] mailto:[email protected]) so that I can track down what is triggering the problem? Simon
…
On 18 Mar 2019, at 12:48, chhylp123 @.> wrote: What coverage do you have? If you have low coverage (< 30x) then the number of CpGs that can be reliably called can be very low. Since the cpg files only contain confidently called CpGs, you can therefore end up with few results. There are two ways to get around this: (a) Use the Encode BedMethyl output format (-B option for gemBS extract). This will give an output for every CpG in the reference that is covered in the experiment. (b) Use the reference bias option for the calling (i.e., gemBS call -R 1000). This will heavily weight the genotype calling to favour homozygous reference calls. This will require removing the existing bcf files, running gemBS db-sync and then re-calling. Hope this helps, Simon … x-msg://21/# On 18 Mar 2019, at 11:56, chhylp123 @.> wrote: No, you shouldn’t need to run it twice, but the first time it did not complete for some reason. If you re-ran the gemBS call command it would have performed just the required operations to complete the calling (or given you an error as to why it did not work). That is what the output of the gemBS call --dry-run command tells us - it shows the operations that gemBS needs to do to complete the calling. In your example, the output showed that it only needed to perform one call and the merge step. If you deleted the existing BCFs then you will have to run gemBS db-sync before running gemBS call. Simon … x-msg://19/# On 18 Mar 2019, at 07:03, chhylp123 @.***> wrote: OK, so the calling did not finish for some reason. If the calling had finished correctly then you would see only the merged bcf file for the sample rather that then the individual chromosome bcfs. gemBS extract looks for this merged bcf file, which is why you are getting the error message. You seem to have the bcfs for the chromosomes, but you don’t have the calls for pool1 which groups together the minor contigs. If you run the gemBS call command again (with your normal options) it will perform the single outstanding call operation and merge the bcfs to create the sample bcf file. Simon You mean I need to run gemBS call twice? I have already delete all bcf files, and try to gemBS call again, but it still cannot generate a single bcf file. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 <#49> (comment) <#49 (comment) <#49 (comment)>>>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB. Hi, I have ran gemBS call and gemBS extract successfully, but a new problem existed. The result files of gemBS extract are too small: 0 March 18 18:17 mextr_SRR5453774.err 88K March 18 18:41 SRR5453774_cpg.txt.gz 19K March 18 18:41 SRR5453774_cpg.txt.gz.tbi 0 March 18 18:41 tabix_SRR5453774_cpg.err However, both the bam file and the bcf file are 14GB. And my commands of gemBS call and gemBS extract are: gemBS call -j 7 -t 4 -g 0 -f 0 gemBS extract -j 28 Could you help me to solve this problem? It is very important for me. Thank you very much! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 (comment) <#49 (comment)>>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB. Thank you very much! My dataset is about 10X. So you mean since it is low coverage, maybe just thousands of CpG sites are reliable? As for -B option, I have tried: gemBS extract -B -j 28 But it reported: : : Command extract started at 2019-03-18 19:45:05.924688 : : Methylation Extraction... 2019-03-18 19:45:06,197 ERROR: Process '/usr/local/lib/python3.5/dist-packages/gemBS/bin/bcftools' finished with -11 Traceback (most recent call last): File "/usr/local/bin/gemBS", line 13, in load_entry_point('gemBS==3.2.7', 'console_scripts', 'gemBS')() File "/usr/local/lib/python3.5/dist-packages/gemBS/commands.py", line 157, in gemBS_main instances[args.command].run(args) File "/usr/local/lib/python3.5/dist-packages/gemBS/production.py", line 1397, in run self.do_filter(v) File "/usr/local/lib/python3.5/dist-packages/gemBS/production.py", line 1491, in do_filter snps=snps,snp_list=self.snp_list,snp_db=self.snp_db) File "/usr/local/lib/python3.5/dist-packages/gemBS/init.py", line 1173, in methylationFiltering raise ValueError("Error while extracting methylation calls.") ValueError: Error while extracting methylation calls. May I ask how can I fix this? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd5cA69q9p9YeW2j7ZsxptVyOzVF_ks5vX30IgaJpZM4b44CB.

I am sorry for misunderstanding the documents. May I ask if I want to get the CpG sites in Bismark style with .bedGraph format, the -B option must be used?

How can I send the bcf file to you? It is about 14GB.

from gembs.

heathsc avatar heathsc commented on August 13, 2024

from gembs.

heathsc avatar heathsc commented on August 13, 2024

from gembs.

chhylp123 avatar chhylp123 commented on August 13, 2024

Can you send a contact email address to [email protected] mailto:[email protected] Simon
…
On 18 Mar 2019, at 13:42, Simon Heath @.> wrote: Yes the -B option will give bedGraph output. I can share a google drive folder with you and you should be able to upload it there. Alternatively, you could extract 1 chromosome from the bcf and upload just that which will be faster. Simon > On 18 Mar 2019, at 13:39, chhylp123 @. @.>> wrote: > > I mean at 10x you can not expect to call genotypes and methylation from your sample (I should probably make this clearly in the documentation), so we have to assume no sequence variants. The error you are seeing, however, is a bug in the extraction. Is there any possibility that you could share the bcf file (with @. @.> @. @.>) so that I can track down what is triggering the problem? Simon > … x-msg://28/# > On 18 Mar 2019, at 12:48, chhylp123 @.> wrote: What coverage do you have? If you have low coverage (< 30x) then the number of CpGs that can be reliably called can be very low. Since the cpg files only contain confidently called CpGs, you can therefore end up with few results. There are two ways to get around this: (a) Use the Encode BedMethyl output format (-B option for gemBS extract). This will give an output for every CpG in the reference that is covered in the experiment. (b) Use the reference bias option for the calling (i.e., gemBS call -R 1000). This will heavily weight the genotype calling to favour homozygous reference calls. This will require removing the existing bcf files, running gemBS db-sync and then re-calling. Hope this helps, Simon … x-msg://21/# x-msg://21/# On 18 Mar 2019, at 11:56, chhylp123 @.> wrote: No, you shouldn’t need to run it twice, but the first time it did not complete for some reason. If you re-ran the gemBS call command it would have performed just the required operations to complete the calling (or given you an error as to why it did not work). That is what the output of the gemBS call --dry-run command tells us - it shows the operations that gemBS needs to do to complete the calling. In your example, the output showed that it only needed to perform one call and the merge step. If you deleted the existing BCFs then you will have to run gemBS db-sync before running gemBS call. Simon … x-msg://19/# x-msg://19/# On 18 Mar 2019, at 07:03, chhylp123 @.> wrote: OK, so the calling did not finish for some reason. If the calling had finished correctly then you would see only the merged bcf file for the sample rather that then the individual chromosome bcfs. gemBS extract looks for this merged bcf file, which is why you are getting the error message. You seem to have the bcfs for the chromosomes, but you don’t have the calls for pool1 which groups together the minor contigs. If you run the gemBS call command again (with your normal options) it will perform the single outstanding call operation and merge the bcfs to create the sample bcf file. Simon You mean I need to run gemBS call twice? I have already delete all bcf files, and try to gemBS call again, but it still cannot generate a single bcf file. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 <#49> <#49 <#49>> (comment) <#49 <#49> (comment) <#49 (comment) <#49 (comment)>>>>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB. Hi, I have ran gemBS call and gemBS extract successfully, but a new problem existed. The result files of gemBS extract are too small: 0 March 18 18:17 mextr_SRR5453774.err 88K March 18 18:41 SRR5453774_cpg.txt.gz 19K March 18 18:41 SRR5453774_cpg.txt.gz.tbi 0 March 18 18:41 tabix_SRR5453774_cpg.err However, both the bam file and the bcf file are 14GB. And my commands of gemBS call and gemBS extract are: gemBS call -j 7 -t 4 -g 0 -f 0 gemBS extract -j 28 Could you help me to solve this problem? It is very important for me. Thank you very much! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 <#49> (comment) <#49 (comment) <#49 (comment)>>>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB. Thank you very much! My dataset is about 10X. So you mean since it is low coverage, maybe just thousands of CpG sites are reliable? As for -B option, I have tried: gemBS extract -B -j 28 But it reported: : : Command extract started at 2019-03-18 19:45:05.924688 : : Methylation Extraction... 2019-03-18 19:45:06,197 ERROR: Process '/usr/local/lib/python3.5/dist-packages/gemBS/bin/bcftools' finished with -11 Traceback (most recent call last): File "/usr/local/bin/gemBS", line 13, in load_entry_point('gemBS==3.2.7', 'console_scripts', 'gemBS')() File "/usr/local/lib/python3.5/dist-packages/gemBS/commands.py", line 157, in gemBS_main instances[args.command].run(args) File "/usr/local/lib/python3.5/dist-packages/gemBS/production.py", line 1397, in run self.do_filter(v) File "/usr/local/lib/python3.5/dist-packages/gemBS/production.py", line 1491, in do_filter snps=snps,snp_list=self.snp_list,snp_db=self.snp_db) File "/usr/local/lib/python3.5/dist-packages/gemBS/init.py", line 1173, in methylationFiltering raise ValueError("Error while extracting methylation calls.") ValueError: Error while extracting methylation calls. May I ask how can I fix this? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 (comment) <#49 (comment)>>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd5cA69q9p9YeW2j7ZsxptVyOzVF_ks5vX30IgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd5cA69q9p9YeW2j7ZsxptVyOzVF_ks5vX30IgaJpZM4b44CB. > > I am sorry for misunderstanding the documents. May I ask if I want to get the CpG sites in Bismark style with .bedGraph format, the -B option must be used? > > How can I send the bcf file to you? It is about 14GB. > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub <#49 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd6dXz4nRxYgTj7yRiT3IGe_LU2JSks5vX4j2gaJpZM4b44CB. >

I have sent an email to you by gmail. And may I ask one more question? I see in Section 7.2 of your document:

The ENCODE files produce methylation estimates for cytosines in CpG, CHG and CHH contexts

If I only want to cytosines in CpG context, what should I do? Thank you very much.

from gembs.

heathsc avatar heathsc commented on August 13, 2024

from gembs.

chhylp123 avatar chhylp123 commented on August 13, 2024

Hi Simon,

The bcf file has been uploaded. If you have any question, please contact me.

I have shared a folder with you where you can upload the BCF file. Currently it is not possible to select only CpGs - the -B flag turns on ENCODE compatible output which requires all 3 outputs. If there is demand I could add additional options to allow more granular control. Cheers, Simon
…
On 18 Mar 2019, at 13:54, chhylp123 @.> wrote: Can you send a contact email address to @. @.> @. @.> Simon … x-msg://32/# On 18 Mar 2019, at 13:42, Simon Heath @.> wrote: Yes the -B option will give bedGraph output. I can share a google drive folder with you and you should be able to upload it there. Alternatively, you could extract 1 chromosome from the bcf and upload just that which will be faster. Simon > On 18 Mar 2019, at 13:39, chhylp123 @.*** @.>> wrote: > > I mean at 10x you can not expect to call genotypes and methylation from your sample (I should probably make this clearly in the documentation), so we have to assume no sequence variants. The error you are seeing, however, is a bug in the extraction. Is there any possibility that you could share the bcf file (with @. @.> @. @.>) so that I can track down what is triggering the problem? Simon > … x-msg://28/# > On 18 Mar 2019, at 12:48, chhylp123 @.> wrote: What coverage do you have? If you have low coverage (< 30x) then the number of CpGs that can be reliably called can be very low. Since the cpg files only contain confidently called CpGs, you can therefore end up with few results. There are two ways to get around this: (a) Use the Encode BedMethyl output format (-B option for gemBS extract). This will give an output for every CpG in the reference that is covered in the experiment. (b) Use the reference bias option for the calling (i.e., gemBS call -R 1000). This will heavily weight the genotype calling to favour homozygous reference calls. This will require removing the existing bcf files, running gemBS db-sync and then re-calling. Hope this helps, Simon … x-msg://21/# x-msg://21/# On 18 Mar 2019, at 11:56, chhylp123 @.> wrote: No, you shouldn’t need to run it twice, but the first time it did not complete for some reason. If you re-ran the gemBS call command it would have performed just the required operations to complete the calling (or given you an error as to why it did not work). That is what the output of the gemBS call --dry-run command tells us - it shows the operations that gemBS needs to do to complete the calling. In your example, the output showed that it only needed to perform one call and the merge step. If you deleted the existing BCFs then you will have to run gemBS db-sync before running gemBS call. Simon … x-msg://19/# x-msg://19/# On 18 Mar 2019, at 07:03, chhylp123 @.> wrote: OK, so the calling did not finish for some reason. If the calling had finished correctly then you would see only the merged bcf file for the sample rather that then the individual chromosome bcfs. gemBS extract looks for this merged bcf file, which is why you are getting the error message. You seem to have the bcfs for the chromosomes, but you don’t have the calls for pool1 which groups together the minor contigs. If you run the gemBS call command again (with your normal options) it will perform the single outstanding call operation and merge the bcfs to create the sample bcf file. Simon You mean I need to run gemBS call twice? I have already delete all bcf files, and try to gemBS call again, but it still cannot generate a single bcf file. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 <#49> <#49 <#49>> <#49 <#49> <#49 <#49>>> (comment) <#49 <#49> <#49 <#49>> (comment) <#49 <#49> (comment) <#49 (comment) <#49 (comment)>>>>>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB. Hi, I have ran gemBS call and gemBS extract successfully, but a new problem existed. The result files of gemBS extract are too small: 0 March 18 18:17 mextr_SRR5453774.err 88K March 18 18:41 SRR5453774_cpg.txt.gz 19K March 18 18:41 SRR5453774_cpg.txt.gz.tbi 0 March 18 18:41 tabix_SRR5453774_cpg.err However, both the bam file and the bcf file are 14GB. And my commands of gemBS call and gemBS extract are: gemBS call -j 7 -t 4 -g 0 -f 0 gemBS extract -j 28 Could you help me to solve this problem? It is very important for me. Thank you very much! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 <#49> <#49 <#49>> (comment) <#49 <#49> (comment) <#49 (comment) <#49 (comment)>>>>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB. Thank you very much! My dataset is about 10X. So you mean since it is low coverage, maybe just thousands of CpG sites are reliable? As for -B option, I have tried: gemBS extract -B -j 28 But it reported: : : Command extract started at 2019-03-18 19:45:05.924688 : : Methylation Extraction... 2019-03-18 19:45:06,197 ERROR: Process '/usr/local/lib/python3.5/dist-packages/gemBS/bin/bcftools' finished with -11 Traceback (most recent call last): File "/usr/local/bin/gemBS", line 13, in load_entry_point('gemBS==3.2.7', 'console_scripts', 'gemBS')() File "/usr/local/lib/python3.5/dist-packages/gemBS/commands.py", line 157, in gemBS_main instances[args.command].run(args) File "/usr/local/lib/python3.5/dist-packages/gemBS/production.py", line 1397, in run self.do_filter(v) File "/usr/local/lib/python3.5/dist-packages/gemBS/production.py", line 1491, in do_filter snps=snps,snp_list=self.snp_list,snp_db=self.snp_db) File "/usr/local/lib/python3.5/dist-packages/gemBS/init.py", line 1173, in methylationFiltering raise ValueError("Error while extracting methylation calls.") ValueError: Error while extracting methylation calls. May I ask how can I fix this? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 <#49> (comment) <#49 (comment) <#49 (comment)>>>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd5cA69q9p9YeW2j7ZsxptVyOzVF_ks5vX30IgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd5cA69q9p9YeW2j7ZsxptVyOzVF_ks5vX30IgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd5cA69q9p9YeW2j7ZsxptVyOzVF_ks5vX30IgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd5cA69q9p9YeW2j7ZsxptVyOzVF_ks5vX30IgaJpZM4b44CB. > > I am sorry for misunderstanding the documents. May I ask if I want to get the CpG sites in Bismark style with .bedGraph format, the -B option must be used? > > How can I send the bcf file to you? It is about 14GB. > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub <#49 (comment) <#49 (comment)>>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd6dXz4nRxYgTj7yRiT3IGe_LU2JSks5vX4j2gaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd6dXz4nRxYgTj7yRiT3IGe_LU2JSks5vX4j2gaJpZM4b44CB. > I have sent a email to you by gmail. And may I ask one more question? I see in Section 7.2 in your document: The ENCODE files produce methylation estimates for cytosines in CpG, CHG and CHH contexts If I only want to cytosines in CpG context, what should I do? Thank you very much. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPdxiCi8fb0c9toKZhJri40CqWxaJxks5vX4xqgaJpZM4b44CB.

from gembs.

heathsc avatar heathsc commented on August 13, 2024

from gembs.

heathsc avatar heathsc commented on August 13, 2024

from gembs.

heathsc avatar heathsc commented on August 13, 2024

from gembs.

chhylp123 avatar chhylp123 commented on August 13, 2024

Thank you very much! Should I re-implement map and call?

I think the bug has been fixed now on github. Please test and let me know, Simon
…
On 19 Mar 2019, at 11:19, Simon Heath @.> wrote: Could you try running: gemBS extract -B -c and see if this runs? Simon > On 19 Mar 2019, at 08:25, Simon Heath @. @.>> wrote: > > I see the file. I will take a look today and let you know. > > Simon > > > On Tue, 19 Mar 2019, 08:22 chhylp123, @. @.>> wrote: > Hi Simon, > > The bcf file has been uploaded. If you have any question, please contact me. > > I have shared a folder with you where you can upload the BCF file. Currently it is not possible to select only CpGs - the -B flag turns on ENCODE compatible output which requires all 3 outputs. If there is demand I could add additional options to allow more granular control. Cheers, Simon > … x-msg://48/#m_-5300883036892069154_ > On 18 Mar 2019, at 13:54, chhylp123 @.> wrote: Can you send a contact email address to @.*** @.> @. @.> Simon … x-msg://32/# x-msg://32/# On 18 Mar 2019, at 13:42, Simon Heath @.> wrote: Yes the -B option will give bedGraph output. I can share a google drive folder with you and you should be able to upload it there. Alternatively, you could extract 1 chromosome from the bcf and upload just that which will be faster. Simon > On 18 Mar 2019, at 13:39, chhylp123 @.*** @.>> wrote: > > I mean at 10x you can not expect to call genotypes and methylation from your sample (I should probably make this clearly in the documentation), so we have to assume no sequence variants. The error you are seeing, however, is a bug in the extraction. Is there any possibility that you could share the bcf file (with @. @.> @. @.>) so that I can track down what is triggering the problem? Simon > … x-msg://28/# x-msg://28/# > On 18 Mar 2019, at 12:48, chhylp123 @.> wrote: What coverage do you have? If you have low coverage (< 30x) then the number of CpGs that can be reliably called can be very low. Since the cpg files only contain confidently called CpGs, you can therefore end up with few results. There are two ways to get around this: (a) Use the Encode BedMethyl output format (-B option for gemBS extract). This will give an output for every CpG in the reference that is covered in the experiment. (b) Use the reference bias option for the calling (i.e., gemBS call -R 1000). This will heavily weight the genotype calling to favour homozygous reference calls. This will require removing the existing bcf files, running gemBS db-sync and then re-calling. Hope this helps, Simon … x-msg://21/# x-msg://21/# x-msg://21/# x-msg://21/# On 18 Mar 2019, at 11:56, chhylp123 @.> wrote: No, you shouldn’t need to run it twice, but the first time it did not complete for some reason. If you re-ran the gemBS call command it would have performed just the required operations to complete the calling (or given you an error as to why it did not work). That is what the output of the gemBS call --dry-run command tells us - it shows the operations that gemBS needs to do to complete the calling. In your example, the output showed that it only needed to perform one call and the merge step. If you deleted the existing BCFs then you will have to run gemBS db-sync before running gemBS call. Simon … x-msg://19/# x-msg://19/# x-msg://19/# x-msg://19/# On 18 Mar 2019, at 07:03, chhylp123 @.> wrote: OK, so the calling did not finish for some reason. If the calling had finished correctly then you would see only the merged bcf file for the sample rather that then the individual chromosome bcfs. gemBS extract looks for this merged bcf file, which is why you are getting the error message. You seem to have the bcfs for the chromosomes, but you don’t have the calls for pool1 which groups together the minor contigs. If you run the gemBS call command again (with your normal options) it will perform the single outstanding call operation and merge the bcfs to create the sample bcf file. Simon You mean I need to run gemBS call twice? I have already delete all bcf files, and try to gemBS call again, but it still cannot generate a single bcf file. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 <#49> <#49 <#49>> <#49 <#49> <#49 <#49>>> <#49 <#49> <#49 <#49>> <#49 <#49> <#49 <#49>>>> (comment) <#49 <#49> <#49 <#49>> <#49 <#49> <#49 <#49>>> (comment) <#49 <#49> <#49 <#49>> (comment) <#49 <#49> (comment) <#49 (comment) <#49 (comment)>>>>>>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd3CSc_AA2gim7WRBY5nEGtgdU5_Hks5vXywhgaJpZM4b44CB. Hi, I have ran gemBS call and gemBS extract successfully, but a new problem existed. The result files of gemBS extract are too small: 0 March 18 18:17 mextr_SRR5453774.err 88K March 18 18:41 SRR5453774_cpg.txt.gz 19K March 18 18:41 SRR5453774_cpg.txt.gz.tbi 0 March 18 18:41 tabix_SRR5453774_cpg.err However, both the bam file and the bcf file are 14GB. And my commands of gemBS call and gemBS extract are: gemBS call -j 7 -t 4 -g 0 -f 0 gemBS extract -j 28 Could you help me to solve this problem? It is very important for me. Thank you very much! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 <#49> <#49 <#49>> <#49 <#49> <#49 <#49>>> (comment) <#49 <#49> <#49 <#49>> (comment) <#49 <#49> (comment) <#49 (comment) <#49 (comment)>>>>>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd22LMuCq-7c9q0YgWxttOoRedkm3ks5vX3DFgaJpZM4b44CB. Thank you very much! My dataset is about 10X. So you mean since it is low coverage, maybe just thousands of CpG sites are reliable? As for -B option, I have tried: gemBS extract -B -j 28 But it reported: : : Command extract started at 2019-03-18 19:45:05.924688 : : Methylation Extraction... 2019-03-18 19:45:06,197 ERROR: Process '/usr/local/lib/python3.5/dist-packages/gemBS/bin/bcftools' finished with -11 Traceback (most recent call last): File "/usr/local/bin/gemBS", line 13, in load_entry_point('gemBS==3.2.7', 'console_scripts', 'gemBS')() File "/usr/local/lib/python3.5/dist-packages/gemBS/commands.py", line 157, in gemBS_main instances[args.command].run(args) File "/usr/local/lib/python3.5/dist-packages/gemBS/production.py", line 1397, in run self.do_filter(v) File "/usr/local/lib/python3.5/dist-packages/gemBS/production.py", line 1491, in do_filter snps=snps,snp_list=self.snp_list,snp_db=self.snp_db) File "/usr/local/lib/python3.5/dist-packages/gemBS/init.py", line 1173, in methylationFiltering raise ValueError("Error while extracting methylation calls.") ValueError: Error while extracting methylation calls. May I ask how can I fix this? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 <#49> <#49 <#49>> (comment) <#49 <#49> (comment) <#49 (comment) <#49 (comment)>>>>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd5cA69q9p9YeW2j7ZsxptVyOzVF_ks5vX30IgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd5cA69q9p9YeW2j7ZsxptVyOzVF_ks5vX30IgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd5cA69q9p9YeW2j7ZsxptVyOzVF_ks5vX30IgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd5cA69q9p9YeW2j7ZsxptVyOzVF_ks5vX30IgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd5cA69q9p9YeW2j7ZsxptVyOzVF_ks5vX30IgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd5cA69q9p9YeW2j7ZsxptVyOzVF_ks5vX30IgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd5cA69q9p9YeW2j7ZsxptVyOzVF_ks5vX30IgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd5cA69q9p9YeW2j7ZsxptVyOzVF_ks5vX30IgaJpZM4b44CB. > > I am sorry for misunderstanding the documents. May I ask if I want to get the CpG sites in Bismark style with .bedGraph format, the -B option must be used? > > How can I send the bcf file to you? It is about 14GB. > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub <#49 <#49> (comment) <#49 (comment) <#49 (comment)>>>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPd6dXz4nRxYgTj7yRiT3IGe_LU2JSks5vX4j2gaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd6dXz4nRxYgTj7yRiT3IGe_LU2JSks5vX4j2gaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd6dXz4nRxYgTj7yRiT3IGe_LU2JSks5vX4j2gaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPd6dXz4nRxYgTj7yRiT3IGe_LU2JSks5vX4j2gaJpZM4b44CB. > I have sent a email to you by gmail. And may I ask one more question? I see in Section 7.2 in your document: The ENCODE files produce methylation estimates for cytosines in CpG, CHG and CHH contexts If I only want to cytosines in CpG context, what should I do? Thank you very much. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 (comment) <#49 (comment)>>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPdxiCi8fb0c9toKZhJri40CqWxaJxks5vX4xqgaJpZM4b44CB https://github.com/notifications/unsubscribe-auth/ADHPdxiCi8fb0c9toKZhJri40CqWxaJxks5vX4xqgaJpZM4b44CB. > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub <#49 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/ADHPdzPVWXKBaJflpmFBgicCBY97nCJgks5vYJBLgaJpZM4b44CB. >

from gembs.

heathsc avatar heathsc commented on August 13, 2024

from gembs.

chhylp123 avatar chhylp123 commented on August 13, 2024

The problem has been solved. Thanks very much for your great help!

from gembs.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.