Comments (5)
Hi,
There are some discussions about this question in Bioconconductor' forum. To make the story short, Smyth & Lun are not in favor and rather propose to do independent FDR correction for each the contrast. This means that if you have 3 classes A, B, C, then you specify that you are interested in all differences A-C, B-C, A-B and then you compute a multiple tests correction for each those 3 contrasts. The alternatives are to merge all them before the correction (what is called global in limma) or to do a hierarchical approach.
I was asked myself this question and it was good to think about how I would apprehend the solution. In fact, the question is coping with 2-dimensional multiple tests. Because FDR handle many tests at once (typically across genes, ie rows) and Tukey approach handle multiple tests across groups (ie columns) within a single gene. So we can compute the individual p-values of all contrasts then either compute q-value per contrast (limma's default) or q-value after having pooled them all (limma's global). We can get the p-value of an ANOVA on each line, then apply a FDR filter, then apply Tukey-like on each remaining row (sounds like limma's hierarchical, but too conservative IMHO). What else? I don't know.
I think this came to the conclusion that the simple solution is to used contrasts separately. But in fact, I didn't look at the code of the great Rob. Tibshirani to know his point of view. Whatever he did, I think he did it well.
HTH
from sam.
Dear SamGG:
Thank you very much for your such comprehensive explanation!!
However I can't understand some of your details and I want to ask further.
First, you said:"So we can compute the individual p-values of all contrasts then either compute q-value per contrast (limma's default)." As your intruction, FDR handle many tests at row and Tukey at columns, but in the limma's default I think it only do FDR using BH adjustion method as default and doesn't do Tukey in the multiple contrasts. The code I used is:
->topTableOutput <- topTable(fit2,coef = 1, adjust="BH", n = Inf)
Am I right? Or it has been done Tukey in limma?
Second, you said:”to merge all them before the correction (what is called global in limma)”. I thought I had got the global table by the code:
->write.table(fit2,"limma.txt")
If this is what you said the pull pooled them all, should I need to do both FDR and Tukey to these p values in this table? Is the column “fit2$F.p.value” has been done any adjustment like FDR or Tukey?
Furthermore, in this global table I got the “fit2$F.p.value” of every gene in every group like the one-way ANOVA. You see in an ordinary ANOVA analysis, I can only get this column and there isn’t every p value before post-hoc has been down. So I don’t know whether the “fit2$F.p.value” has been down any multiple tests, as well as whether you said could be taken on other multiple way analysis except limma (like ANOVA or SAM, et al.), Because only one value for a total contrast existed, like ABC and there isn’t any value A-B,B-C,A-C. In this situation, we can’t get the answer by pull pooled them all isn’t it? So we can only use contrasts separately and do 2-dimensional multiple tests.
Third, I want to know are the p-values in “limma global” of the 3 group contrast (more than 2 group) comes from the 3 group like ANOVA or direct 3 separate contrasts like t-test? If it is the former, it doesn’t need Tukey further, isn’t it? If it is the latter, it needs 2-dimensional multiple tests. I would like to know, if the 2-dimensional multiple rule is universal to all multiple group analysis? If it is like this, we can do the 2-dimensional after multiple to separate contrasts and get the results of all differences like you said.
Finally, do you know the independent code to FDR and Tukey code which fit for the common statistic analysis? If yes, is it means the problem has been resolved?
Thank you again for all your kindly help!!!
Sincerely,
Minzhi Zhao
from sam.
Dear Minzhi,
I was talking about the decideTest function. Look at a discussion at https://support.bioconductor.org/p/120072/#120105 that was very instructive for me. The code at https://github.com/cran/limma/blob/master/R/decidetests.R was also interesting to understand what's behind the scene and what the options.
If you go with limma, you should ask your initial question at https://support.bioconductor.org after having checked the long limma documentation.
Best,
Samuel
from sam.
Dear Samuel,
Thanks for your detailed information! After red the pages you suggested carefully, I found myself more confused. But I think your description is more clear. This is realy a huge job and I understand you want to see a published paper. Now I want to focus on one tip"false discovery rate control at the gene level or at the contrast level".
As we know the adjustion of p value (at the gene level) have some method like BH or permutation (q value?) based FDR. All these can be taken only with the existing of P value it self. But at the contrast level, also called post-hoc test, like Tukey, can such adjustion done independent with the base analysis, like two group t-test? I found this test must have the original expression value of each contrast. In your discussion I found all of you use t-test directly to describe such post-hoc test. There are other post-hoc tests like LSD, Bonferroni. I want to find a adjustion method which could be fit for all test method or algorithm, not only for ANOVA and t-test (in my understanding limma is also based on t-test?). For example, the method named "edge" which using optimal discovery procedure (ODP) frame work as likelihood ratio test in t-test and ANOVA. So it is better to indenpent with the original test method. Do you know such method? And do you know the essential difference of the multiple tests between "at the gene level" and "at the contrast level" ?
Thank you again for all your kindly help!!
Sincerely,
Minzhi Zhao
from sam.
Dear Samuel,
I find a tool named "Perseus" for proteomic data analysis. And it's process for ANOVA and post-goc test is a bit like your said "hierarchical". I don't know whether it is your interest and I paste the paper link below. I think it should be a common question because we usually do many multiple group contrasts.
https://link.springer.com/protocol/10.1007%2F978-1-4939-7493-1_7#Sec16
from sam.
Related Issues (20)
- multiclass GSA: "contrast" for gene sets and saving output
- Question on quantitative.func
- there is no package called ‘shiny’ HOT 9
- There is no package called "samr" HOT 2
- cannot install samr with R 3.4.3 or R 3.4.4 HOT 6
- Warning: Error in impute.knn HOT 1
- How to get the final value of the after an automatic method
- Possible to parallelize samr?
- Error in attempt to compute delta.table
- DataTables warning HOT 1
- Normalization needed for RNAseq data analysis? HOT 1
- Issue on SAM interface HOT 3
- can not allocate vector of size 20.3Mb HOT 1
- Seeing significant genes on sam shiny
- Error in library(dqshiny) : there is no package called ‘dqshiny’ HOT 1
- function 'Rcpp_precious_remove' not provided by package 'Rcpp'
- Error running RNAseq time course analysis using SAMseq
- Error when saving HOT 1
- Error in if (assay.type == "seq" & cond) { : the condition has length > 1 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sam.