• Sign In
  1. Requests

Benjamini-Hocjburg

support

View Request

view list print
Benjamini-Hocjburg sstoychev  2018-08-22 22:41
 

Am I correct that correction for multiple testing for group comparisons within Skyline is done using the Bonferoni method? I assume this is because most MRM/SRM experiments will have a small number of "tests" to correct hence Bonferoni does not result in a too strict of a cut-off?

In the case of DIA experiments Benajmini-Hocjburg would be the preferred method for multiple testing correction. I think this is the method used by the external MSStats tool? Is it possible to add Benajmini-Hocjburg as an option to group comparisons within Skyline or is this too much work to do?
 
 
Brendan MacLean responded:  2018-08-22 22:53

No. I am afraid you are mistaken. Nowhere in Skyline do we use Bonferoni. The Adjusted P Value in Skyline for group comparisons uses the same Benjamini-Hochberg Procedure (http://www.statisticshowto.com/benjamini-hochberg-procedure/) as MSstats.

MSstats differs from Skyline in other ways, but not in multiple hypothesis correction.

Thanks for checking. And sorry to be the bearer of bad news. (If you are asking, you likely are not happy with the number of targets within your chosen estimated false discovery rate cut-off, but it isn't because we use Bonferoni.)

--Brendan
 
Nick Shulman responded:  2018-08-22 22:55
Skyline uses the BH method. This is the same as what MSstats uses (I copied it).
Here's the Skyline code:
https://github.com/ProteoWizard/pwiz/blob/master/pwiz_tools/Shared/Common/DataAnalysis/PValues.cs
 
sstoychev responded:  2018-08-23 11:45
Thanks for the clarification Brendan.

Hits are not necessarily fewer but in some instances different. From the little comparison I've done it seems high fold change/ low p-value are same but at the "borders" there are differences. Will have a look at the info you sent.

Out of curiosity, why the difference between "internal" vs "external" MSStats?
 
Brendan MacLean responded:  2018-08-23 13:53
Sorry about the confusion. The original intent of the MSstats logo in this Skyline Group Comparison form was to give credit to the MSstats team for working with us on this feature and provide a link to its more advanced statistical analysis capabilities. It was not intended to imply that our Group Comparison feature was part of MSstats. But, I think we need to remove the logo because it gets misinterpreted that way.

MSstats is written in R and therefore has access to a much wider range of statistical functions. It implements more sophisticated methods of imputation for missing values, and it is capable of dealing with more complex experimental designs, i.e. more than just pairwise comparison and things like time-courses.

Hope this helps to clarify.

--Brendan
 
sstoychev responded:  2018-08-23 14:40
Thanks Brendan, makes sense.

It's a shame since group comparisons within Skyline is much more "interactive" with ability to easily pick out a protein from the volcano plot and interrogate it's peptides/ transitions. Also small things like not having info on number of peptides per protein in the "TestingResults" export from MSStats can be frustrating but maybe I'm being too picky... you and your team have delivered a fantastic tool for DIA data :)