Weird looking UMAP for spectral flow data about uwot

[SamGG]

Hi,

1. Long story short: IMHO the data are not transformed. Details below.
2. If you use a pipeline, please submit an issue in the github of the pipeline, not in the github of a tool used in the pipeline, especially when the tool (uwot::umap) proved its usefulness and robustness so many times.
3. Please, consider closing this issue once it's solved.

Hope the following will help you.
Best.

I don't see where the classical asinh(x/cofactor) is applied to the channel intensities x, with a cofactor typically equal to 5000.
It seems you used the pipeline described at https://github.com/HdBraanker/Spectral_Flow_Workflow/blob/main/script.Rmd from the article of den Braanker et al..
From your code, I think your code lacks of the transformation step either by automatic estimation of cofactors

cofactors <- estParamFlowVS(fcs_data_small, channels=markerstotransform)
fcs_transform <- transFlowVS(fcs_data, channels = markerstotransform, cofactors)

or by setting manually cofactors

cofactors<- rep(3000, length(markerstotransform))
fcs_transform <- transFlowVS(fcs_data, channels = markerstotransform, cofactors)

Following the code, some of the channels are retained in the transformed data

fcs_transform <- fcs_transform[c(1:4,10,16)] #samples from batch A
md <- md[c(1:4,10,16),] # samples from batch A

Then this data is subsampled (splitted being more exact, line 286)

Subsampling_FlowSet(fcs_transform,0.25, md=md) 

Here, the code is misleading because the Subsampling_FlowSet function is creating variables (fcs_train, md_train, fcs_test, md_test). So you must execute this function if you want to follow the pipeline and use its next steps from line 294.
Be careful with this "pipeline". In fact, it is a code exploring many aspects of the spectral data processing, but not a pipeline to be executed step by step. If you are looking for something more standard (but without batch correction), I recommend the CATALYST or FlowSOM pipelines and vignettes.

[fingeram]

Hi,

I am sorry, I was just looking for help not to criticize the uwot package. I would like to clarify, that I DID transform the data (I just did not think it was relevant to my question to include the code, especially since I mentioned that the data I use for umapping is indeed transformed with arcsinh) with channel optimized arcsinh cofactors. Please find the transformation pipeline, as well as examples of what the data looks like before and after below:

markerstotransform <- panel$fcs_colname[c(8:18,20)]
cofactors <- estParamFlowVS(fcs_data, channels=markerstotransform)
fcs_transform <- transFlowVS(fcs_data, channels=markerstotransform, cofactors)

I did try batch correction but I did not see any batch effects that needed to be corrected. I also performed quality control with PeacoQC.

That being said, do you have any other recommendations or suggestions why my UMAPs might look this way?

Thank you very much!

PS. Here is the uwot::umap I generated used the same code as above but for NON-transformed spectral flow data (just using raw expression values). You see that the clustering looks different but the pattern with the cloud of cells around the clusters looks very similar. The reason I am posting this here and not in the gitub of the other pipelines I used for preprocessing the data, is that I believe the preprocessing is ok but maybe not compatible with how the umapping algorithm works. That's mostly because spectral flow data can be quite different from sequencing data (and cyTOF data). However, I can also reach out to people who generated pipelines for the preprocessing.

[SamGG]

Sorry, still not convinced that your code is doing what you think it should do.
If I was you, I would start by testing my code on the FCS used by dan Braanker. I would select one or a few FCS that look like mine. For peaking these FCS and checking the similarity with my FCS, I would use a point and click software such as FlowJo.
And even before that, I would use FlowJo (that you or your staff used for compensation FJComp prefix in channel names) for doing a tSNE or UMAP. Please do post such a UMAP here.
There is no much difference between spectral data and CyTOF data once properly pre-processed. There are a few articles showing a good agreement between these technologies, and none (as far as I know) showing disagreement.
Best.