1. I believe there is not a good reason for not being able to uncheck the "Intensity" score. Originally, the "Intensity" was the "bootstrap score" (there's an explanation of what that means in the mProphet paper), but currently, you ought to be able to uncheck that score. I will try to fix this in an upcoming release of Skyline-Daily.

2. In the PowerPoint that you attached, your "Reverse decoy" scores are much much better than your "Reverse decoy + second best". There is very good separation between the scores of your decoys and the scores of your targets, and both of those histograms are gaussian shaped, which is what you want.
Decoys are nearly always better than "second best peak". The main reason to use "second best peaks" would be if you did not want to collect the extra data necessary to have decoys. This is not an issue in a DIA experiment, since the decoys can always be generated from the data that you do have. For PRM or SRM, you might not want to make the instrument collect the decoy data, and so you might use "second best peaks".

-- Nick

Hi Antony,
Can I quickly check that this is MS1-only data and that you are expecting to see the Library dot-product, Product mass error and Precursor-Product shape scores disable?

If this is actually PRM or DIA data, then you really want to figure out why the Library dot-product score is disabled. It should be one of your most powerful scores. The way you describe it as "...high-quality data set (in total ca. 1000 spectra..." makes me curious. the use of "spectra" especially makes me wonder if you are expecting to be matching to library MS/MS spectra, which is definitely not happening in your mProphet model.

One thing we maybe should be enforcing is that I don't really think it is valid to use retention time scores when you are using second-best peaks, because these scores are not independent between your targets and decoys. The fact that you expect your targets to show up close to the predicted retention times means that the second best peaks have less available time at which to be close to predicted RT themselves. This is just a feeling I have never tested myself, but the scores really should be independent.

So, I lean toward trusting the model from the true decoys when possible, and when not possible excluding RT scores in a model that uses second-best peaks.

Hope this helps. Nick is certainly right that there is no inherent reason we can't make it possible to exclude the Intensity score. I will look into getting that enabled for our next release.

--Brendan

Dear Nick and Brendan,

Many thanks to your quick and very helpful reply!

I am actually trying analyzing some PRM data. Unfortunately, no decoy data were monitored in this PRM data set. I just generated the reverse or shuffle decoy and then reimported the data to train mProphet models. Most of the decoy peptides do not have MS2 spectra in tis case, and it is the reason that 3 subscores (library intensity dotp, product mass error, precursor -product shape score) are disabled. Now I realized it was wrong.
 
I just tested using only 2nd best peaks to train mProphet models (see attached slides) and almost all subscores are available. Are they good enough to be applied?

Thanks again,
Antony

That looks much better, Antony. I would go with the model that excludes the retention time scores, which are anyway very week scores when they are included. You are right that you can't use reversed of shuffled decoys in this case. You are actually limited to what you have done, and we did exactly the same thing for a paper with our collaborators at the Buck Institute.

Good luck with your research. Thanks for the clear screenshots.

--Brendan