kechrislab / msprep Goto Github PK

Easiest task ever. :)

Remove old code files and check locations and gen scripts of datasets used in package and testing.

Should allow for none or some (any) user-defined number of replicates.

I checked Biocondcutor and there were only 31 packages with the "metabolomics" search term.
I removed some that were not obviously related. Attached is a list, but some of those may not be as relevant either - I don't have time now to look at each of them.

But if it helps, we are looking at data from LC/MS - which is related to GC/MS or other mass spectrometry (MS) based methods - but not related to NMR methods.

Packages I know better are 'xcms', which is widely used. The authors of "metabomxr" visited last year and we're in touch with them, they could be a good resource.

Other relevant packages from the Emory group that we work with:
http://web1.sph.emory.edu/apLCMS/
https://sourceforge.net/projects/xmsanalyzer/

• change name to 'sample_info'
• ensure that it takes continuous (e.g. blood pressure could be a phenotype)

Deprecate groupingvars arguments and functions if so.

Doesn't seem to be working across the entire pipeline.

All of these should have reasonable default values so that, for example, an experiment without batches treats all samples as belonging to a single batch. Default value should be NULL and have dummy variables filled in internally. Per @mmulvahill 's suggestion, this needs to be done as a separate branch and the downstream implications thoroughly tested before merging.

Definitely remove the file read -- should just use read.csv, read_csv, etc.

Want to be able to quickly provide interface for incorporating new methods with simple wrappers. Definitely for imputation methods, but probably other functions too.

When running msFilter using summarized experiment, column names are changed from just sample_id to one that has all of the colData variables that are pasted together with separation "_". How do i preserve the sample_ID of the original colnames without having to delete all colData

Some values in the quant/spike dataset have colons: 3.2672226:1 and 3.2817035:2. Is this a ratio that needs to be calculated before converting to numeric? Currently this column is a factor and is inconsistently implemented. Resolve in wide_matrix_to_data function and ms_prepare.

@kechrisk

1. One imputation method is the half-minimum approach, where we impute 0's with half the minimum value for that compound. Do we actually want to use the minimum across all patients & replicates, or should we do this using the minimum value within each patient?
2. In the old code, when BPCA imputes a number < 0, we replace that negative value with the half-minimum imputation -- resulting in a combined BPCA/half-min imputation method. Is this what we want to do? Or should these negative values be considered 'true missing'? Or, should these be two separate options? (1. BPCA, assumed 0 if neg. and 2. BPCA, assumed below threshold)

For reference, from the manuscript:

Missing Data: There are three primary modes of missing data in metabolomics datasets and each mode has different implications for subsequent analysis; therefore, different imputation routines and statistical methods are required and three are offered in the MSPrep package. The three modes are truly not present, present below the detectable limit of the instrument and absent owing to error in pre-processing algorithms. The MSPrep package implements three methods of managing missing data: (i) No imputation assumes the mode of missing is true zeros and therefore assigns the missing values as zeros. This dataset could be useful for PCA analysis, cluster analysis and methods that account for clustering at zero. Unless a stringent filter is applied, normalization routines may have poor performance, as most have assumptions about underlying distributions that are not valid with zero clustered data. (ii) The second option assumes missing compounds were below the detectable limit and imputes a value of one half of the minimum observed value for that compound (Xia et al., 2009). (iii) The final method is a call to the Bayesian PCA (BPCA) imputation algorithm (Oba et al., 2003) from the PCAMethods R package (Stacklies et al., 2007) and assumes that the compound is present but failed to be accurately detected. This algorithm estimates the missing value by a linear combination of principal axis vectors, where the parameters of the model are identified by a Bayesian estimation method and is not sensitive to the quantity of missing data.

• Add a dataframe object that has column for pipeline step (prepare, filter, impute, etc.) and a locigal column for conducted y/n.
• Then create a function that defines acceptable paths through pipeline, giving error when invalid path

e.g., imputation steps

There are 3 or 4 versions -- get down to one set.

https://edwinth.github.io/blog/dplyr-recipes/

Also, consider using .data$varname in dplyr funs to avoid env scoping issues.

Currently ms_impute uses the default parameters for Bayesian PCA as implemented in impute_bpca. All parameters passed through impute_bpca to pca need to be available to the user in the arguments to ms_impute.

filter --> impute --> normalize???? Forgot to grab sticky note from Katerina's office...

The n_comp and n_control parameters for ms_normalize are not well documented. The first is only for CRMN, while the second is passed to several functions. I will track down what both of these do (especially n_control) and document thoroughly, including arguments passed to functions in other packages.

This should be a matter of having a parameter (e.g. neighbor_type=c("metabolite", "sample")) in impute_knn. c5c0464 implies that this has been implemented but I don't see a user-controllable option. Need to check with @tuh8888 to find out current status.

MSPrep R package

Summary from meeting w/ Domink:

• Start with read_data
• Sean's version (in sean branch, _sj()) has more code than the one in
  develop/master
• Separate out tasks is the first key thing to do

Notes from meeting w/ Dominik:

• Dominik uses:
  • read_data_sj()
    • fn doesn't allow more than 3 technical replicates -- should allow Inf
    • fn summarizes, but this should be a separate fn -- probabaly summary()
    • option for whether data is already log-xfrmed or not -
    • option to load/convert data existing in workign environment
    • ID variable ("lcms...") is hardcoded, shoudln't be
• batch correction --
  • hasn't used
  • should this be done before summaries, imputation, normalization
• cvmax -
• missing data typically represented by 0 or 1 (log(1)=0)

• Grouping
  • change name to 'sample_info'
  • ensure that it takes continuous (e.g. blood pressure could be a phenotype)
• Check if sva and ruv have a 'covariate' option -- should be a user option whether sample_info should be passed to it.
• Test users: Harrison, Sean Jacobson
• Ask Sean for the Emory dataset (cc: Katerina)

• diagnosticgen()
• graphimputations()

So far not adding these features, but consider doing so later.

Add option for data transformation -- should it be whether data is already transformed (if so do we need to know what that transformation is?) or should it be what transformation to apply to the data (transform = c(NULL, 'log', 'sqrt', etc.))?