SKYLINE BUG for display and peak area reporting when Non-Quantitative transitions are included

support
SKYLINE BUG for display and peak area reporting when Non-Quantitative transitions are included Will Thompson  2020-01-14 12:50
 

Hi Brian, Brendan et al

Please see attached to this message a reasonably detailed account of a pretty bad bug in peak area reporting which seems to occur when non-quantitative transitions are included in Skyline documents. It affects reported total peak areas, such that external measurement of results (such as stats or against calibration curves) will not match up what is returned by Skyline.

As always, let us know if you have any questions.

Cheers

Will

 
 
Nick Shulman responded:  2020-01-14 14:02
Will,

When Skyline calculates the ratio of light to heavy, Skyline has to figure out which light transition corresponds to which heavy transition. The reason that Skyline needs to do this is so that if a peak is missing from the numerator (because the user maybe did "Remove Peak"), the corresponding peak area does not get added to the denominator.

If small molecule transitions only have m/z values, the way that Skyline matches light transitions to heavy transitions is by the order in which they appear in the Targets tree.
In the example that you provided us, there are 4 transitions for the light precursor and 2 transitions for the heavy precursor.
Skyline therefore pairs the first light transition with the first heavy transition and the second light transition with the second heavy transition. The last two light transitions are seen as having no corresponding heavy transitions.

You have marked your first and third light transitions as non-quantitative. Because of this, when Skyline needs to calculate the ratio of light to heavy, Skyline gives you the ratio of the second light transition to the second heavy transition.

This is clearly not what you were expecting.

If your light and heavy precursors have a different set of transitions from each other, we recommend that you give each of the transition names so that Skyline will know how to match them up. You can right-click on the transition in the Targets tree and choose "Modify" and fill in the "Name" textbox on the "Modify Transition" dialog.

In your document, you might name your light transitions "A", "B", "C", and "D". In that case, you should name your heavy transitions "B", "D". In that way, Skyline will know that the Ratio of Light to Heavy should be calculated by summing the "B" and "D" transition peak areas for the light and heavy precursors. (See attached "WithNames.sky.zip")

Note that when you delete a transition from the light precursor, Skyline deletes what it thinks is the corresponding heavy transition. So, when you tried to remove the non-quantitative transitions from your Skyline document, the reason that you still did not get the correct numbers is that Skyline deleted the heavy 181.1100 transition when you deleted the non-quantitative light 181.2100 transition.

Currently, Skyline does not pay attention to Quantitativeness when matching up heavy and light transitions. The only thing that matters (if the transitions do not have names) is the order in which the Transitions appear in the Targets tree. We will look into whether we can change anything about this in the future.

For now, you should try to give your Transitions names. If that is not feasible, you should make it so that the transitions that are common between light and heavy appear earlier in the list than the non-quantitative transitions that are intended to be unmatched.

Note that this matching of light and heavy transitions only happens when Skyline is doing the ratio or precursors in the same molecule. If you have a Surrogate Standard, then the Normalized Area will always be the ratio of Total Areas.

Sorry for the confusion and I hope this helps.
Let me know if anything else seems suspicious or that I might have missed something.
-- Nick
 
Will Thompson responded:  2020-01-14 14:27
Hi Nick

I was under the assumption that at the precursor level, skyline will calculate the ratios by summing up the intensity of the product ions for either heavy or light, then calculating the ratio. It is pretty important for some special cases in small molecules that it not require that EVERY product ion have a matching heavy and light transition. There are a number of cases in quantification where we might want to have more transitions in heavy than light...or vice versa...to throw away those that don't have a direct corollary seems to be a capricious. Among these cases are:

having interference in heavy or light but not both
needing to use 'echo summing' of transitions to improve sensitivity
heavy and light molecules fragment

Giving the transitions names again seems like it should not be necessary...why not just enable an option to make the precursor ratio the actual ratio at the precursor level of the sum of all transitions? I am pretty sure this is how most vendor software processes the data (not that 'because the vendor does it' is a particularly good reason in all cases, just saying that you are pushing against the grain here).

I also think you misunderstand what i reported...though now i am not sure i understand completely what skyline is doing. REgardless of how the ratios are calculated, why would removing a LIGHT transition, cause the raw peak area for the HEAVY molecule to change? This is what happened...i removed two light transitions, and the HEAVY precursor peak area changed. Most interestingly, the ratio did not change, and is the same (and I think correct) before and after i deleted the transitions.

Perhaps a call would be helpful?

Cheers

W
 
Nick Shulman responded:  2020-01-14 15:04
Will,

The reason that the heavy peak area changed when you deleted a light transition is that Skyline deleted a heavy transition at the same time.
That is, in your screenshot on page 2 of your .docx, the 181.1100 transition was deleted from the heavy precursor.

It sounds like Skyline should only do this clever matching up of light and heavy transitions if the transitions really have been given Names. If any of the transitions does not have a Name then the ratio will just be calculated by doing the ratio of the two Total Areas, and the Ratio Dot Product value will be blank.
(Also, we won't delete a heavy transition if a nameless light one gets deleted).
-- Nick
 
Will Thompson responded:  2020-01-15 10:38
Hi Nick

Can you say whether or not it would be possible to add a feature to Skyline where the ratio is calculated based on the sum of all transitions (light) over sum of all transitions (heavy)? My understanding of what you stated above is that the ratio light/heavy is currently calculated as:

Light/Heavy (skyline) = AVG (lightT1/heavyT1, lightT2/heavyT2, lightT3/HeavyT3...etc), where T1 represents first matched transition, T2 second matched, etc.

In some of the cases I mentioned above, and in most software that i am aware, the ratio is calculated as:

Light/Heavy = Sum (LightT1, LightT2, LightT3...n) / Sum (HeavyT1, HeavyT2, HeavyT3).

Does this makes sense?

Will
 
Will Thompson responded:  2020-01-15 10:40
In other words, yes it would make sense to me to have the sums used (to answer your query in the second paragraph). But maybe instead of Skyline just guessing which method should be used, this should/could be a selectable option in the Transitions menu under quantification.

Cheers

W
 
Brendan MacLean responded:  2020-01-15 10:48
That is not how Skyline calculates the ratio. It does not use a mean average, as you expressed. It uses:

Light/Heavy = Sum (LightT1, LightT2, LightT3) / Sum (HeavyT1, HeavyT2, HeavyT3)

Where apparently Nick as added a requirement that the transitions match each other, and any unpaired transitions get excluded from the sums. I am not sure when that change got made. In practice, I don't think it impacts a lot of people, certainly not in proteomics where unpaired transitions are generally considered a mistake and users asked for us to add features to make it harder to end up with unpaired transitions.

It sounds like you are voting for looser restrictions on what gets included in the sums for the numerator and denominator.
 
Will Thompson responded:  2020-01-15 10:51
Final Follow up.

In the current skyline version, if i have two light transitions and one heavy transition, is there a way for me to make it use both light transitions in the ratio calculation?

Cheers

W
 
Brendan MacLean responded:  2020-01-15 10:53
Nick's proposed solution of looser handling for unpaired unnamed transitions sounds like it would solve your problem. I don't really agree that we want to add a new option for this in quantification. The ratio calculation is what you and everyone else wants and based on experiments from the original CPAC program. What is at issue is which transitions get included in the ratio. It is certainly news to us that it is common to expect a ratio of unmatched transitions.
 
Brendan MacLean responded:  2020-01-15 10:59
"is there a way for me to make it use both light transitions in the ratio calculation?"

I seriously doubt it from Nick's responses. I am sure the code I wrote originally was this simplistic, but Nick has added helpful features like rdotp and stronger transition matching for peptides. Apparently the matching got build into the TotalAreaRatio calculation.

I am sure we can fix this in a Skyline-daily soon.
 
Will Thompson responded:  2020-01-15 10:59
Thanks for the clarification, Brendan. I would be happy if it used the quantitative transitions that are displayed, in the ratio. I thought that was why you guys enabled the 'automated' deletion of the matched y/b ions in heavy/light when you delete one...but it seems a bit counter-intuitive to not be using all the displayed transitions in the ratio. It would seem that the most common assumption would be that if it is in the document and it is quantitative (i.e. not greyed out), then it is used in the ratio. I can't speak for everyone but i can say that we have several use cases where it is important for us to use more transitions for light than for heavy, and our working assumption has been that if transitions are displayed in the tree, they are used. Otherwise, if a quantitative transition in the tree is NOT being used to calculate the ratio, it should at least be flagged as "this transition is not being used" or something.

Thanks for the discussion, always learning.

Cheers

W
 
Will Thompson responded:  2020-01-15 11:41
I think a workaround for using differing numbers of transitions in the light and heavy compound is to break them apart in the molecule tree (different names) and use the surrogate standard quant strategy. Not saying this is a good or preferred way to go, but its a workaround for the time being.

W
 
Will Thompson responded:  2020-01-31 09:05
Hi Brendan and Nick

Just following up on this. Did it make onto the "worklist" for Skyline-daily release? We have some use cases where it will be important to have Skyline utilize all the transitions shown as quantitative in the molecule tree for quantification. I just wanted to know if this made it onto the 'to do' list or if I need to work to provide further justification for this feature.

As an aside, after our discussion I was so surprised that I checked with Sue Abbatiello and she had the same assumption that I did; all transitions shown are used for quantification, not just those with a matching heavy/light.

Cheers

W
 
Nick Shulman responded:  2020-02-04 13:46
I have created an Issue so that we will not forget to fix this behavior in Skyline 20.2.
https://skyline.ms/issues/home/issues/details.view?issueId=716
-- Nick