Skyline-Daily v. 3.7.1.11208 CLI mProphet NullPointerException

support
Skyline-Daily v. 3.7.1.11208 CLI mProphet NullPointerException wbarshop  2017-08-24 20:18
 
Hello Skyline Team!
The aforementioned version of Skyline-Daily fails to correctly apply mProphet model to the data, returning a NullPointerException (as shown in the partial output below). The same command set//data works just fine under the stable v. 3.7 release. This is only the case when the analysis is done through the CLI (SkylineCmd).

For reference, the command is included here:
SkylineCmd --timestamp --in=combined_analysis.sky --full-scan-rt-filter-tolerance=2.0 --add-library-path=filtered.blib --import-fasta=C:\pulsar\files\staging\1953\inputs\dataset_3491.dat_corrected.fasta --decoys-add=reverse --import-all=. --reintegrate-model-name=REVERSEcombinedanalysis180434961000 --reintegrate-create-model --reintegrate-overwrite-peaks --report-add=WOHL_MSSTATS_REPORT.skyr --report-conflict-resolution=overwrite --report-name=Wohl_MSstats_Input --report-file=MSstats_input.csv --chromatogram-file=C:\pulsar\files\staging\1953\outputs\dataset_3732.dat --chromatogram-precursors --chromatogram-products --chromatogram-base-peaks --chromatogram-tics --out=docker_protection_temporary.sky


Please let me know if you would like me to upload my data directly to the Skyline team to use as a test case.
==========================================================================


[2017/08/24 19:49:43]    0% - Updating peak statistics

[2017/08/24 19:49:45]    6%

[2017/08/24 19:49:48]    12%

[2017/08/24 19:49:50]    18%

[2017/08/24 19:49:52]    24%

[2017/08/24 19:49:54]    30%

[2017/08/24 19:49:57]    36%

[2017/08/24 19:49:59]    43%

[2017/08/24 19:50:01]    49%

[2017/08/24 19:50:03]    55%

[2017/08/24 19:50:05]    61%

[2017/08/24 19:50:07]    67%

[2017/08/24 19:50:09]    73%

[2017/08/24 19:50:12]    79%

[2017/08/24 19:50:14]    84%

[2017/08/24 19:50:16]    90%

[2017/08/24 19:50:18]    95%

[2017/08/24 19:50:49]    100%

[2017/08/24 19:50:49]    Creating scoring model REVERSEcombinedanalysis180434961000

[2017/08/24 19:50:51]    Calculating peak group scores

[2017/08/24 19:50:54]    5%

[2017/08/24 19:50:56]    7%

[2017/08/24 19:50:58]    9%

[2017/08/24 19:51:01]    12%

[2017/08/24 19:51:03]    14%

[2017/08/24 19:51:05]    16%

[2017/08/24 19:51:08]    19%

[2017/08/24 19:51:11]    22%

[2017/08/24 19:51:14]    25%

[2017/08/24 19:51:16]    28%

[2017/08/24 19:51:19]    31%

[2017/08/24 19:51:21]    34%

[2017/08/24 19:51:24]    37%

[2017/08/24 19:51:26]    40%

[2017/08/24 19:51:28]    43%

[2017/08/24 19:51:31]    46%

[2017/08/24 19:51:34]    49%

[2017/08/24 19:51:37]    52%

[2017/08/24 19:51:40]    55%

[2017/08/24 19:51:43]    58%

[2017/08/24 19:51:46]    61%

[2017/08/24 19:51:49]    64%

[2017/08/24 19:51:51]    67%

[2017/08/24 19:51:54]    69%

[2017/08/24 19:51:56]    71%

[2017/08/24 19:51:58]    73%

[2017/08/24 19:52:00]    75%

[2017/08/24 19:52:02]    77%

[2017/08/24 19:52:04]    79%

[2017/08/24 19:52:06]    81%

[2017/08/24 19:52:09]    84%

[2017/08/24 19:52:11]    87%

[2017/08/24 19:52:14]    90%

[2017/08/24 19:52:16]    92%

[2017/08/24 19:52:18]    94%

[2017/08/24 19:52:20]    96%

[2017/08/24 19:52:22]    98%

[2017/08/24 19:52:23]    100%

[2017/08/24 19:52:26]    0% - Training peak scoring model

[2017/08/24 19:52:30]    9% - Training scoring model (iteration 1 of 10)

[2017/08/24 19:52:33]    18% - Training scoring model (iteration 2 of 10 - 242,066 peaks at 2% FDR)

[2017/08/24 19:52:36]    27% - Training scoring model (iteration 3 of 10 - 245,356 peaks at 2% FDR)

[2017/08/24 19:52:39]    36% - Training scoring model (iteration 4 of 10 - 245,739 peaks at 2% FDR)

[2017/08/24 19:52:42]    45% - Training scoring model (iteration 5 of 10 - 245,791 peaks at 2% FDR)

[2017/08/24 19:52:45]    54% - Training scoring model (iteration 6 of 10 - 245,824 peaks at 2% FDR)

[2017/08/24 19:52:48]    63% - Training scoring model (iteration 7 of 10 - 245,828 peaks at 2% FDR)

[2017/08/24 19:52:51]    72% - Training scoring model (iteration 8 of 10 - 245,831 peaks at 2% FDR)

[2017/08/24 19:52:54]    81% - Training scoring model (iteration 9 of 10 - 245,831 peaks at 2% FDR)

[2017/08/24 19:52:57]    90% - Training scoring model (iteration 10 of 10 - 245,832 peaks at 2% FDR)

[2017/08/24 19:52:57]    100%

[2017/08/24 19:53:05]    7% - Calculating score contributions

[2017/08/24 19:53:05]    100%

[2017/08/24 19:53:05]    Intensity: -0.3061 (-11.6 %)

[2017/08/24 19:53:05]    Retention time difference: -0.3580 (5.6 %)

[2017/08/24 19:53:05]    Retention time difference squared: 0.0057 (-0.1 %)

[2017/08/24 19:53:05]    Library intensity dot-product: 5.1910 (49.3 %)

[2017/08/24 19:53:05]    Shape (weighted): 1.7624 (20.8 %)

[2017/08/24 19:53:05]    Co-elution (weighted): -0.0448 (1.3 %)

[2017/08/24 19:53:05]    Co-elution count: -0.0136 (-0.6 %)

[2017/08/24 19:53:05]    Signal to noise: 0.1551 (5.1 %)

[2017/08/24 19:53:05]    Product mass error: -0.0513 (-0.4 %)

[2017/08/24 19:53:05]    Precursor-product shape score: -0.0254 (-0.3 %)

[2017/08/24 19:53:05]    Precursor mass error: -0.1419 (-0.4 %)

[2017/08/24 19:53:05]    Precursor isotope dot product: 2.7389 (26.1 %)

[2017/08/24 19:53:05]    Identified count: 0.5870 (5.2 %)

[2017/08/24 19:53:09]    Adjusting peak boundaries

[2017/08/24 19:53:11]    6%

[2017/08/24 19:53:13]    12%

[2017/08/24 19:53:15]    18%

[2017/08/24 19:53:17]    24%

[2017/08/24 19:53:20]    30%

[2017/08/24 19:53:22]    36%

[2017/08/24 19:53:24]    42%

[2017/08/24 19:53:26]    48%

[2017/08/24 19:53:28]    55%

[2017/08/24 19:53:31]    62%

[2017/08/24 19:53:33]    68%

[2017/08/24 19:53:35]    74%

[2017/08/24 19:53:37]    80%

[2017/08/24 19:53:40]    85%

[2017/08/24 19:53:42]    90%

[2017/08/24 19:53:44]    95%

[2017/08/24 19:53:45]    100%

[2017/08/24 19:53:45]    Error: Failed to reintegrate peaks successfully.

[2017/08/24 19:53:46]    System.NullReferenceException: Object reference not set to an instance of an object.

   at pwiz.Skyline.Model.DocNodeChildren.IsOrderSame(DocNodeChildren previousChildren) in c:\proj\pwiz_x64\pwiz_tools\Skyline\Model\DocNodeChildren.cs:line 45

   at pwiz.Skyline.Model.DocNodeChildren..ctor(IEnumerable`1 items, IList`1 previous) in c:\proj\pwiz_x64\pwiz_tools\Skyline\Model\DocNodeChildren.cs:line 20

   at pwiz.Skyline.Model.DocNodeParent.set_Children(IList`1 value) in c:\proj\pwiz_x64\pwiz_tools\Skyline\Model\DocNode.cs:line 384

   at pwiz.Common.SystemUtil.Immutable.ChangeProp[TIm](TIm immutable, SetLambda`1 set) in c:\proj\pwiz_x64\pwiz_tools\Shared\Common\SystemUtil\Immutable.cs:line 192

   at pwiz.Skyline.Model.DocNodeParent.ChangeChildren(IList`1 children, IList`1 counts, Int32 indexReplaced) in c:\proj\pwiz_x64\pwiz_tools\Skyline\Model\DocNode.cs:line 0

   at pwiz.Skyline.Model.DocNodeParent.ChangeChildrenChecked(IList`1 children, Boolean changeAutoManage) in c:\proj\pwiz_x64\pwiz_tools\Skyline\Model\DocNode.cs:line 1170

   at pwiz.Skyline.Model.SrmDocument.RegroupMolecules(IList`1 children, PeptideDocNode[] moleculeNodes, Func`3 rankChildren) in c:\proj\pwiz_x64\pwiz_tools\Skyline\Model\SrmDocument.cs:line 803

   at pwiz.Skyline.Model.SrmDocument.ChangeSettingsInternal(SrmSettings settingsNew, SrmSettingsChangeMonitor progressMonitor) in c:\proj\pwiz_x64\pwiz_tools\Skyline\Model\SrmDocument.cs:line 995

   at pwiz.Skyline.Model.MProphetResultsHandler.ChangePeaks(IProgressMonitor progressMonitor) in c:\proj\pwiz_x64\pwiz_tools\Skyline\Model\MProphetResultsHandler.cs:line 177

   at pwiz.Skyline.CommandLine.Reintegrate(ModelAndFeatures modelAndFeatures, Boolean isOverwritePeaks, Boolean logTraining) in c:\proj\pwiz_x64\pwiz_tools\Skyline\CommandLine.cs:line 1511
 
 
Nick Shulman responded:  2017-08-24 21:02
Yes, it would be helpful if you could upload your files.
You can upload them here:
https://skyline.ms/files.url

I cannot quite tell from the callstack what is going wrong. I think it might have something to do with an optimization that we have when running in the commandline when we null out some objects when we think they are no longer needed in order to reduce the amount of memory Skyline is hanging onto.
 
wbarshop responded:  2017-08-25 09:30
Nick,
You should be able to find all the necessary data to reproduce this in the uploaded file "sky-daily-mprophet-nullreference.7z" in the file share that you linked.

You should be able to replicate the error by running the following command in the extracted folder (assuming SkylineCmd is in path):

SkylineCmd --timestamp --in=document_with_decoys.sky --full-scan-rt-filter-tolerance=2.0 --import-process-count=10 --import-all=. --reintegrate-model-name=failureModel --reintegrate-create-model --reintegrate-overwrite-peaks --out=mProphet_choke.sky
 
wbarshop responded:  2017-08-25 14:40
Nick,

Sorry about the large number of targets, etc, in the file above.
I have made a vastly slimmed down skyline file with far, far fewer targets.

This should help in speeding up hunting out this problem, and make it tractable for a spinning disk system!

Let me know if you have any questions (and don't forget to change the --in=________ portions of the command).

The file name in the same upload location is "tiny_skyline_with_decoys.sky"


Best,
William
 
Brendan MacLean responded:  2017-08-29 10:56
Thanks for the files, William. I have fixed this on our side and implemented a test that would have caught this before it got released. It was a command-line only bug, and apparently we didn't have enough testing of the mProphet reintegration case in our automated tests.

Thanks to your help, that has improved a little. The next Skyline-daily release should fix this issue for you.

--Brendan
 
wbarshop responded:  2017-08-29 11:55
Brendan and Nick,
As always, huge props to you guys and the rest of the Skyline team.

Thanks for your tremendous work!
-wb