Score type of xQuest scores for building .ssl files

support
Score type of xQuest scores for building .ssl files kmk  2018-05-02 07:31
 
Hello,

I am creating a ssl file according to: https://skyline.ms/wiki/home/software/BiblioSpec/page.view?name=BiblioSpec%20input%20and%20output%20file%20formats

The software I use for identification is called xQuest (http://proteomics.ethz.ch/cgi-bin/xquest2_cgi/index.cgi)

xQuest exports a (linear discriminant) score for each peptide based on the xProphet metrics. I'd like to import these scores into the Skyline library I'm building, so I have to choose the correct score type. Since there is no xQuest score type mentioned on the linked web page, I am wondering which score type to choose.

Right now I am using "PEPTIDE PROPHET SOMETHING" as a type since that software also uses a (different) kind of ld-score describing the probability of a positive identification.

Also, I am quite curious how Skyline exactly uses these imported scores.

Thank you very much for any help!
Kai
 
 
Brian Pratt responded:  2018-05-02 08:14
Hi Kai,

You went to the right place for documentation, and it sounds like you dug deep enough to find the BiblioSpec schema: https://raw.githubusercontent.com/ProteoWizard/pwiz/master/pwiz_tools/BiblioSpec/tests/reference/tables.check

which contains this:
probabilityType VARCHAR(128) -- detail about the cutoff logic used by each score type, PROBABILITY_THAT_IDENTIFICATION_IS_CORRECT, PROBABILITY_THAT_IDENTIFICATION_IS_INCORRECT, or NOT_A_PROBABILITY_VALUE

So the purpose of the score is to decide what makes it into the library and what doesn't, and the score type determines if the cutoff is for values that are too large, or too small. It sounds like you have chosen a reasonable score type for your data. Note that there is an assumption that the score range is from 0 to 1.

Brian
 
Brendan MacLean responded:  2018-05-02 08:50
So, be aware that PEPTIDE PROPHET SOMETHING is one of the scores where 1.0 is the best possible score and 0 is the worst. Unfortunately, not all "Prophets" behave like this. For example, mProphet produces q values where 0 is best and 1.0 is the worst. It sounds like you are saying that xProphet may be more like Peptide Prophet, iProphet and Protein Prophet producing posterior error probabilities instead of q values or expectation scores. If this is true, then you are probably right that PEPTIDE PROPHET SOMETHING is the closest type for you to use. We use it for both Peptide Prophet and iProphet.

It is definitely the wrong type to use, if 0 is best for your score and you should consider using one of the q value or expectation score types.

Also, I would like to direct your attention to the Cross-link Transition Calculator:

https://skyline.ms/skyts/home/software/Skyline/tools/details.view?name=Cross-link%20Transition%20Calculator

It uses a particular tabular format to produce transition lists in Skyline for cross-linking search results. I am not positive that it yet connects well with a spectral library, but if you are interested in helping with that effort, we would like that to work eventually.

This workflow was co-developed by Jimmy Eng and Yuval Boss on the Skyline team. Exciting to see a new cross-link search tool from ETH. I have asked Jimmy to provide more information on how to achieve support for Cross-link Transition Calculator for a new software tool like this.

--Brendan
 
engj responded:  2018-05-02 11:00
Just to follow-up on Brendan's reply, the Cross-Link Transition Calculator is a Skyline external tool plug-in that is meant to facilitate quantitative cross-link analysis. The input is a text file where the short-arm and long-arm masses (0.0 for both for DSS or 54.01 and 103.99 for DSSO) are specified followed by a list of cross-linked peptide pairs. With a list of cross-linked peptides defined in the input text file, the plug-in calculates PRM transitions for both peptides as well as the intact precursor and long-arms (if applicable).

If this might possibly be useful for you, I would encourage you to first browse through the plug-in's PDF file under "Documentation". You might also look at this publication below which describes its application. Note that the ReACT2prm standalone tool referenced in the publication is what we implemented as the Cross-Link Transition Calculator plug-in.

   http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0167547

The targeted workflow would be to take a list of identified cross-links, translate those to the input file format specified for this plug-in (likely requiring some scripting/programming), and perform quantitative analysis on those cross-links using Skyline.

Jimmy
 
kmk responded:  2018-05-03 01:39
Thank you everyone for your responses!

@Brian
You're correct, I dug into the BiblioSpec schema, that's where I picked that scoring scheme. However, xQuest's score is not normalized to range like 0-1. The higher the score, the more confidence it has in an identification of a crosslinked peptide. "Good" peptides usually have scores > 25. Is there any score type already implemented in Skyline reflecting such a behavior?
Just reading the tables.check I cannot really infer that. If that is not the case I could also normalize the scores for each experiment to lie between 0-1 before importing them into Skyline but I am not sure this is the right approach.

@Brendan, Jimmy
While xQuest uses some m/xProphet scoring metrics the resulting score behaves as I explained avbove and is not a q-value.

Thank you both for your input on the transition tool. Right now I linearize crosslinked peptides (received from xQuest) using my own script and save those to a custom ssl file I then import into Skyline. I roughly follow the protocol described in https://link.springer.com/article/10.1007/s13361-017-1837-2

So if I understand you correctly rather than importing my linearized peptides directly into Skyline, I'd convert them to the transition tool input format and from there go into Skyline for quantitation? Does that offer any advantage compared to the process I described above? I'm sorry if the question is a bit naive but I'm still rather new to using Skyline.

Kai
 
Brian Pratt responded:  2018-05-03 07:51
Hi Kai,

In that case you'll want to go with NOT_A_PROBABILITY_VALUE, with a cutoff of 25 (using the -c option in blibbuild.exe). This will give you a .blib spectral library file containing only "good" peptides. I think you can just use UNKNOWN as your type instead of PEPTIDE_PROPHET_SOMETHING.

I don't know enough about the crosslinking tools to comment about any advantages, but with this understanding of cutoffs in blibbuilder I think you should be able to at least keep moving with your current approach. I expect others will offer informed opinions about other options.

Brian
 
kmk responded:  2018-05-04 05:48
Thank you Brian!

Using your approach I'd just use the scores for as a cut-off, right? Or is there any further use in Skyline itself when importing them?
If the former is the case, then I can also directly apply the cut-off in my conversion script. I just thought that Skyline was using the scores in some way for peptide quality evaluation.

Kai
 
Brendan MacLean responded:  2018-05-05 17:04
Hi Kai,
Skyline uses the scores both for the cut-off and to know when a score is better than another score, or maybe that only happens in BlibFilter, which takes a redundant .blib file and produces a non-redundant .blib file. BlibFilter definitely needs to know the direction of a score. That is why we have both:

PROBABILITY_THAT_IDENTIFICATION_IS_INCORRECT
PROBABILITY_THAT_IDENTIFICATION_IS_CORRECT

As far as NOT_A_PROBABILITY_VALUE goes, I guess we have not yet seen a score that is not a probability value where smaller is better. So, we must just always assume that larger is better, and that seems to be the case with your score. If we ever see one where smaller is better, we will have to create another variant for that type of score.

It is a good idea to do your filtering in your tool and not rely on Skyline or BlibBuild to apply your cut-off, if your score is not a probability value. That way, if you get it supported in Skyline, then Skyline users can simply enter zero (0) as their cut-off and they will get a valid library.

Thanks for your feedback and the pointer to the JASMS 2017 paper. I had missed that one. Looks pretty interesting.

--Brendan