GithubHelp home page GithubHelp logo

bcl2fastq's Introduction

NextSeq .bcl Conversion

bcl_to_fastq runs bcl2fastq with optional effects to the Sample Sheet and concatenates reads across lanes into R1 and R2 by sample. By default, Undetermined and reads across individual lanes are removed on success and all reads are placed in BaseCalls directory.

Tested on bcl2fastq2 Conversion Software v2.17.1.14 and Python 2.7 and 3.5.

Running

$ cd /data/nextseq/170111_NS500409_0130_AHHGTMAFXX/SampleSheet.csv
$ bcl_to_fastq --reverse-complement --processing 80
[2017-01-14 19:07:57 - INFO] Using /data/nextseq/170111_NS500409_0130_AHHGTMAFXX/SampleSheet.csv
[2017-01-14 19:07:57 - INFO] Processing /data/nextseq/170111_NS500409_0130_AHHGTMAFXX/SampleSheet.csv
[2017-01-14 19:07:58 - INFO] Found 384 samples for run 170111_NS500409_0130_AHHGTMAFXX
[2017-01-14 19:07:58 - INFO] Run complete.
[2017-01-14 19:07:58 - INFO] Converting .bcl to .fastq using: $>bcl2fastq -r 12 -d 12 -p 80 -w 12 --barcode-mismatches 0 --no-lane-splitting -R .
[2017-01-14 20:54:02 - INFO] .bcl Conversion successful
[2017-01-14 20:54:02 - INFO] Generating demultiplexing stats file

Results

In the run folder, SampleSheet.csv.bak is a backup copy of the original SampleSheet.csv and is accompanied by:

bcl2fastq.log

$ head bcl2fastq.log
2017-01-14 19:07:58 [f82880] INFO: Create FASTQs for index reads: NO
BCL to FASTQ file converter
bcl2fastq v2.17.1.14
Copyright (c) 2007-2015 Illumina, Inc.

2017-01-14 19:07:58 [16f2880] Command-line invocation: bcl2fastq -r 12 -d 12 -p 80 -w 12 --barcode-mismatches 0 --no-lane-splitting -R .
2017-01-14 19:07:58 [16f2880] INFO: Minimum log level: INFO
2017-01-14 19:07:58 [16f2880] INFO: Sample sheet: './SampleSheet.csv'
2017-01-14 19:07:59 [16f2880] INFO: Runfolder path: '.'
2017-01-14 19:07:59 [16f2880] INFO: Input path: './Data/Intensities/BaseCalls/'
etc...

demultiplexing_stats.csv

$ head demultiplexing_stats.csv
AAA003-K10,102570
AAA007-J07,72566
AAA240-I17,146605
AAA240-J05,197833
etc...

Fastq files (<sample>_R?.fastq.gz) are available in RunFolder/Data/Intensities/BaseCalls along with a file named SAMPLES which merely listed the sample IDs that were processed.

Help

$ bcl_to_fastq -h
Usage: bcl_to_fastq [OPTIONS]

  Runs bcl2fastq2, creating fastqs and concatenating fastqs across lanes.
  Original fastq files and Undetermined files are deleted.

Options:
  --runfolder TEXT              path to directory containing run data
                                [default: .]
  --loading INTEGER             number of threads used for loading BCL data
                                [default: 12]
  --demultiplexing INTEGER      number of threads used for demultiplexing
                                [default: 12]
  --processing INTEGER          number of threads used for processing
                                demultiplexed data  [default: 24]
  --writing INTEGER             number of threads used for writing FASTQ data
                                [default: 12]
  --barcode-mismatches INTEGER  number of allowed mismatches per index
                                [default: 0]
  --keep-tmp                    save fastqs across lanes as well as
                                Undetermined  [default: False]
  --reverse-complement          reverse complement index 2 of the sample sheet
                                [default: False]
  --no-wait                     process the run without checking its
                                completion status  [default: False]
  -h, --help                    Show this message and exit.

Requires

Install

git clone [email protected]:brwnj/bcl2fastq.git
cd bcl2fastq
python setup.py install

bcl2fastq's People

Contributors

brwnj avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

bcl2fastq's Issues

troobleshooting with UseBasesMask option

Hi,

I'am trying do convert bcl files ion fastq of 10x genomics data using the option UseBasesMask.

Here is the option in the command that I use :

--use-bases-mask=y28*,i8*,y76*

and here the error log of bcl2fastq that say that the option is incorect because of an asterisk must be preceded by either a 'y', 'i', or 'n'. But the I have put the good syntaxe so I don't understand the mistake.

Command-line invocation: bcl2fastq --minimum-trimmed-read-length 8 --mask-short-adapter-reads 8 --create-fastq-for-index-reads --ignore-missing-positions --ignore-missing-filter --ignore-missing-bcls --use-bases-mask=y28*,i8*,y76* -R /NetScratch/cpichot/singleCell_RNAseq/scRNAseq_herma_fullRun_20200916/rawdata/200915_NB501040_0273_AHLC22BGXG --output-dir=/NetScratch/cpichot/singleCell_RNAseq/scRNAseq_herma_fullRun_20200916/snRNAseq_herma_20200923_testOverideIndexSizeIssue/MAKE_FASTQS_CS/MAKE_FASTQS/BCL2FASTQ_WITH_SAMPLESHEET/fork0/chnk0-u39f1720e5a/files/fastq_path --interop-dir=/NetScratch/cpichot/singleCell_RNAseq/scRNAseq_herma_fullRun_20200916/snRNAseq_herma_20200923_testOverideIndexSizeIssue/MAKE_FASTQS_CS/MAKE_FASTQS/BCL2FASTQ_WITH_SAMPLESHEET/fork0/chnk0-u39f1720e5a/files/interop_path --sample-sheet=/NetScratch/cpichot/singleCell_RNAseq/scRNAseq_herma_fullRun_20200916/snRNAseq_herma_20200923_testOverideIndexSizeIssue/MAKE_FASTQS_CS/MAKE_FASTQS/PREPARE_SAMPLESHEET/fork0/chnk0-u39f1720e50/files/samplesheet.csv -p 4 -r 4 -w 4 --barcode-mismatches 1 
2020-09-28 18:25:03 [7f446485c7c0] INFO: Minimum log level: INFO
2020-09-28 18:25:03 [7f446485c7c0] INFO: Sample sheet: '/NetScratch/cpichot/singleCell_RNAseq/scRNAseq_herma_fullRun_20200916/snRNAseq_herma_20200923_testOverideIndexSizeIssue/MAKE_FASTQS_CS/MAKE_FASTQS/PREPARE_SAMPLESHEET/fork0/chnk0-u39f1720e50/files/samplesheet.csv'
2020-09-28 18:25:03 [7f446485c7c0] INFO: Runfolder path: '/NetScratch/cpichot/singleCell_RNAseq/scRNAseq_herma_fullRun_20200916/rawdata/200915_NB501040_0273_AHLC22BGXG'
2020-09-28 18:25:03 [7f446485c7c0] INFO: Input path: '/NetScratch/cpichot/singleCell_RNAseq/scRNAseq_herma_fullRun_20200916/rawdata/200915_NB501040_0273_AHLC22BGXG/Data/Intensities/BaseCalls/'
2020-09-28 18:25:03 [7f446485c7c0] INFO: Intensities path: '/NetScratch/cpichot/singleCell_RNAseq/scRNAseq_herma_fullRun_20200916/rawdata/200915_NB501040_0273_AHLC22BGXG/Data/Intensities/'
2020-09-28 18:25:03 [7f446485c7c0] INFO: Output path: '/NetScratch/cpichot/singleCell_RNAseq/scRNAseq_herma_fullRun_20200916/snRNAseq_herma_20200923_testOverideIndexSizeIssue/MAKE_FASTQS_CS/MAKE_FASTQS/BCL2FASTQ_WITH_SAMPLESHEET/fork0/chnk0-u39f1720e5a/files/fastq_path'
2020-09-28 18:25:03 [7f446485c7c0] INFO: InterOp path: '/NetScratch/cpichot/singleCell_RNAseq/scRNAseq_herma_fullRun_20200916/snRNAseq_herma_20200923_testOverideIndexSizeIssue/MAKE_FASTQS_CS/MAKE_FASTQS/BCL2FASTQ_WITH_SAMPLESHEET/fork0/chnk0-u39f1720e5a/files/interop_path'
2020-09-28 18:25:03 [7f446485c7c0] INFO: Stats path: '/NetScratch/cpichot/singleCell_RNAseq/scRNAseq_herma_fullRun_20200916/snRNAseq_herma_20200923_testOverideIndexSizeIssue/MAKE_FASTQS_CS/MAKE_FASTQS/BCL2FASTQ_WITH_SAMPLESHEET/fork0/chnk0-u39f1720e5a/files/fastq_path/Stats/'
2020-09-28 18:25:03 [7f446485c7c0] INFO: Reports path: '/NetScratch/cpichot/singleCell_RNAseq/scRNAseq_herma_fullRun_20200916/snRNAseq_herma_20200923_testOverideIndexSizeIssue/MAKE_FASTQS_CS/MAKE_FASTQS/BCL2FASTQ_WITH_SAMPLESHEET/fork0/chnk0-u39f1720e5a/files/fastq_path/Reports/'
2020-09-28 18:25:03 [7f446485c7c0] INFO: Detected CPUs: 12
2020-09-28 18:25:03 [7f446485c7c0] INFO: Loading threads: 4
2020-09-28 18:25:03 [7f446485c7c0] INFO: Processing threads: 4
2020-09-28 18:25:03 [7f446485c7c0] INFO: Writing threads: 4
2020-09-28 18:25:03 [7f446485c7c0] INFO: Allowed barcode mismatches: 1 
2020-09-28 18:25:03 [7f446485c7c0] INFO: Tiles: <ALL>
2020-09-28 18:25:03 [7f446485c7c0] INFO: Minimum trimmed read length: 8
2020-09-28 18:25:03 [7f446485c7c0] INFO: Use bases masks: 
	y28*,i8*,y76*
2020-09-28 18:25:03 [7f446485c7c0] INFO: Mask short adapter reads: 8
2020-09-28 18:25:03 [7f446485c7c0] INFO: Adapter stringency: 0.9
2020-09-28 18:25:03 [7f446485c7c0] INFO: Adapter trimming method: Allow matches with indels
2020-09-28 18:25:03 [7f446485c7c0] INFO: Ignore missing BCLs: YES
2020-09-28 18:25:03 [7f446485c7c0] INFO: Ignore missing filters: YES
2020-09-28 18:25:03 [7f446485c7c0] INFO: Ignore missing positions: YES
2020-09-28 18:25:03 [7f446485c7c0] INFO: Ignore missing controls: NO
2020-09-28 18:25:03 [7f446485c7c0] INFO: Include non-PF clusters: NO
2020-09-28 18:25:03 [7f446485c7c0] INFO: Create FASTQs for index reads: YES
2020-09-28 18:25:03 [7f446485c7c0] INFO: Use bgzf compression for FASTQ files: YES
2020-09-28 18:25:03 [7f446485c7c0] INFO: FASTQ compression level: 4
2020-09-28 18:25:03 [7f446485c7c0] INFO: RunInfo.xml: '"/NetScratch/cpichot/singleCell_RNAseq/scRNAseq_herma_fullRun_20200916/rawdata/200915_NB501040_0273_AHLC22BGXG/RunInfo.xml"'
2020-09-28 18:25:03 [7f446485c7c0] INFO: Lane: 1
2020-09-28 18:25:03 [7f446485c7c0] INFO: Mask: y28*,i8*,y76*
2020-09-28 18:25:03 [7f446485c7c0] ERROR: bcl2fastq::common::Exception: 2020-Sep-28 18:25:03: Success (0): /opt/conda/conda-bld/bcl2fastq2_1548424849859/work/src/cxx/lib/layout/UseBasesMask.cpp(164): Throw in function void bcl2fastq::layout::UseBasesMask::basicValidate(const string&) const
Dynamic exception type: boost::exception_detail::clone_impl<bcl2fastq::layout::UseBasesMaskFormatError>
std::exception::what: UseBasesMask formatting error. An asterisk must be preceded by either a 'y', 'i', or 'n'. Use base mask: 'y28*'

Do you have any suggestion please ?

Thanks in advance

bcl2fastq, all the fastq files are of zero size

Hello,
When i try to demultiplex using bcl2fastq, the process ran successfully but outputting process got completed with zero error and 0 warnings. But all the fastq files have 0 size. All the reads went undetermined.
Details:
8 samples, single-end
what could be the reason?

Cannot install via conda on python 3

Hi,
I tried to install your wrapper through conda (https://anaconda.org/bioconda/bcl2fastq-nextseq) to use it in a snakemake pipeline but it fails, as it apparently requires python 2.7 :
`Specifications:

  • bcl2fastq-nextseq -> python[version='2.7.*|<3|>=2.7,<2.8.0a0']
    Your python: python=3.5
    `
    It's written that it as been tested on python 3.5 in the readme, do you have more info ?
    Thanks ,

Releases?

Hello,

Do you consider this stable? If so could you please add a git tag so that versions can be reliably created?

Thanks,
Martin

lane splitting

Should use option --no-lane-splitting properly rather than relying on concatenation.

run complete, nextseq 2k

check for existence of RTAComplete.txt rather than the status within the XML in order to support nextseq 2k. add some amount of time once file is discovered to account for changes happening in InterOp and elsewhere.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.