differences in output for preCluster.py depending on batchsize about cdna_cupcake

[Magdoll]

This is because I've set an limit on how many records a single minimap2 run can report, currently set to -N 2000000 as seen in code here.

So, when you put in a big batch like 500k sequences, even if every sequence is homologous, you'd get an average of 4 hits per sequence. This -N limit is necessary to prevent minimap2 from blowing up in targeted Iso-Seq cases while still maintaining a reasonable tradeoff with eventual pre-cluster size.

The default batch size of 1000000 was selected to work well for a whole transcriptome (high diversity) case, specifically for inputs of FL reads though. Your input is supposedly 71k HQ isoforms that already has redundancy reduced, so the diversity would be even higher than the usual case.

I'm curious - did you notice a significant run-time difference between a batch size of 5000 vs 500000?

--Liz

[HenrivdGeest]

Thanks. I see the reasoning for batching in relation with minimap -N.
However for me the amount of orphan reads is the real question.

First, why do the number of orphan reads differ between batch sizes?
Second, I can reason that the orphan reads are determined per batch, meaning that the smaller the batches, the more orphan reads you will get. However, my 500k batch setting makes much more orphans than the 5k batch setting, which is then counter-intuitive.
As a final result, I would add the orphans to the cogent output, so it makes a huge difference whether I add 7k or 23k orphans to the final output.

• I assume that orphans are like singletons, so there is only 1 transcript for a gene which has no close family gene in the dataset.

[Magdoll]

Hi @HenrivdGeest ,

My intuition is actually the opposite - the smaller the batch, the less orphans, but probably there's a balance.

Because of the -N (reporting limit) in minimap2, when the batch is smaller, each hit will be reported and clustered together --> why small batches result in fewer orphans.

However, when the batch size is too small, it's also possible no clusters get formed because by chance nothing is a hit when the batch is too small.

Below is an example of a Cogent input. All input is HQ isoforms. There are 641,653 HQ isoforms. It confirms my hypothesis. When the batch size is too high (500k, when -N is set to 2000k, meaning on average only 4 hits are reported per read), the number of orphans is very high. The batch size between 50-200k don't seem very different.

batchsize_50k/preCluster_out.orphans.fasta:17168
batchsize_100k/preCluster_out.orphans.fasta:19309
batchsize_200k/preCluster_out.orphans.fasta:16382
batchsize_500k/preCluster_out.orphans.fasta:71678