Not able to get report using SkylineDailyRunner

support
Not able to get report using SkylineDailyRunner veerupaksh  2019-05-29 11:42
 

Hi Skyline Team,

I am trying to import mass spectrometry result using SkylineDailyRunner.

A brief description of the procedure:

  1. Have created a template, template.sky (attached below)
  2. I open that file and import a transition list (attached below)
  3. Then I import results, there are 143 of those. Each named SHRSP_xxx.d where xxx varies from 001 to 143. (have attached a .zip containing SHRSP_001.d)
  4. Export the report using a custom PythonTemplate.skyr (attached below)

I am able to do this flawlessly using the Skyline Daily GUI, and a correct report is generated as expected. However, when I try to replicate the procedure by creating a .bat file for SkylineDailyRunner, it throws the error mentioned below. I have attached the .bat file as well.

Code for the bat file

C:/MSIC_Master/SkylineDailyRunner.exe --in=C:/MSIC_Master/template.sky --import-process-count=4 --import-transition-list=C:\MSIC_Master\Data\Ruwan_SHRSP_MPfeatures_13_TL.csv --import-all-files=C:\MSIC_Master\Data --report-name=PythonTemplate --report-file=C:\MSIC_Master\Data\Ruwan_SHRSP_MPfeatures_13.tsv --report-format=TSV --out=C:\MSIC_Master\Data\Ruwan_SHRSP_MPfeatures_13.sky

It throws a similar error for all 143 files.

Error: Failed importing the results file SHRSP_001.d.
Message: System.IO.IOException: Error: Failed importing the results file C:\MSIC_Master\Data\SHRSP_001.d.
Message: pwiz.Skyline.Model.Results.MissingDataException: No scans in C:\MSIC_Master\Data\SHRSP_001.d match the current filter settings. ---> pwiz.Skyline.Model.Results.NoFullScanDataException: No scans in C:\MSIC_Master\Data\SHRSP_001.d match the current filter settings.
   at pwiz.Skyline.Model.Results.SpectraChromDataProvider.ExtractChromatogramsLocked() in C:\proj\pwiz_x64\pwiz_tools\Skyline\Model\Results\SpectraChromDataProvider.cs:line 362
   at pwiz.Skyline.Model.Results.SpectraChromDataProvider.ExtractChromatograms() in C:\proj\pwiz_x64\pwiz_tools\Skyline\Model\Results\SpectraChromDataProvider.cs:line 215
   at pwiz.Skyline.Model.Results.SpectraChromDataProvider.SetRequestOrder(IList`1 chromatogramRequestOrder) in C:\proj\pwiz_x64\pwiz_tools\Skyline\Model\Results\SpectraChromDataProvider.cs:line 548
   at pwiz.Skyline.Model.Results.ChromCacheBuilder.Read(ChromDataProvider provider) in C:\proj\pwiz_x64\pwiz_tools\Skyline\Model\Results\ChromCacheBuilder.cs:line 403
   at pwiz.Skyline.Model.Results.ChromCacheBuilder.BuildCache() in C:\proj\pwiz_x64\pwiz_tools\Skyline\Model\Results\ChromCacheBuilder.cs:line 263
   --- End of inner exception stack trace ---


   at pwiz.Skyline.Model.Results.ChromatogramCache.RunProcess(ProcessStartInfo psi, String importProgressPipe, IProgressStatus status, ILoadMonitor loader) in C:\proj\pwiz_x64\pwiz_tools\Skyline\Model\Results\ChromatogramCache.cs:line 683
   at pwiz.Skyline.Model.Results.ChromatogramCache.Run(MsDataFileUri msDataFileUri, String documentFilePath, String cachePath, IProgressStatus status, ILoadMonitor loader) in C:\proj\pwiz_x64\pwiz_tools\Skyline\Model\Results\ChromatogramCache.cs:line 644
   at pwiz.Skyline.Model.Results.ChromatogramCache.Build(SrmDocument document, String documentFilePath, ChromatogramCache cacheRecalc, String cachePath, MsDataFileUri msDataFileUri, IProgressStatus status, ILoadMonitor loader, Action`2 complete) in C:\proj\pwiz_x64\pwiz_tools\Skyline\Model\Results\ChromatogramCache.cs:line 607

Would be really thankful if someone could provide a possible solution/ cause of error.

Regards
Veerupaksh

 
 
Brendan MacLean responded:  2019-05-29 12:16

Hi Veerupaksh,
Nice. Thanks for reporting this. It seems like a bug in our code to import small molecule transition lists from the command-line. I reproduced your report that I can import the same transition list in the Skyline UI and I correctly get all of the precursor ions specified as MS1 precursors, and the file you provided imports just fine, extracting from the MS1 spectra. However, when I try this from the command line, I get the same results as you. So, I removed everything but importing the transition list and saving to an output .sky file. When I open up that file, I can see that the precursor ions are switched to MS/MS ions, as indicated by the fact that they all have a "rank" (and not an "irank" value), which is also somewhat strange, because I would not expect this ion to be ranked without a matching spectral library spectrum.

But, the bug is clearly in the way the transitions get imported differently between the command-line and the graphic user interfaces.

We will try to get this fixed quickly in an upcoming Skyline-daily for you.

--Brendan

 
veerupaksh responded:  2019-05-29 12:26

Brendan,

Thanks for the quick response. Would be looking forward to the updated Skyline Daily.

Is there any work around this issue in the command line (maybe an older version?) as of now, or I need to work with the GUI as of now?

Veerupaksh

 
Brendan MacLean responded:  2019-05-29 17:19

No. It turns out I was wrong on this one. In order to use --import-process-count, you need to make sure the file you are importing into is already saved.

You can certainly argue that we should be better about enforcing this, but there is a workaround. Just break your command-line into two commands or use two lines in a --batch-commands file.

I have attached a batch file that worked for me. It separates the transition list import and save into one SkylineDailyRunner call and the data import and report export into a second call.

I will look at changes that do a better job at protecting this approach. It seems reasonable to:

  1. Save the file before raw data import when --save or --out are used in conjunction with --import-process-count
  2. Error if --import-process-count is used in combination with operations that modify the document without also specifying --save and --out

The issue is that --import-process-count starts up new Skyline processes which themselves need to read in the Skyline document in order to import results. If you modify the document in memory but don't save it, then these separate processes are using the document before your changes.

And, currently, --save and --out only cause a single save of the file after all operations that change the document have occurred.

Thanks again for reporting this. I hope this explanation and the workaround helps you for now. Sorry for my misunderstanding at first.

--Brendan

 
veerupaksh responded:  2019-05-30 11:19

Thanks, it works. The only thing is that the transition list needs to be imported only once, in the first line. Importing the transition list again in the second line is redundant and, in fact, leads to a report which is twice in length, with double the number of lines, half of them being repeated.

Another issue I am facing now is that the SkylineRunner is not able to export the TIC chromatogram in some cases. It just creates a tsv with the header.

Here's the code I am using now

SkylineDailyRunner.exe --in=template.sky --import-transition-list=Ruwan_SHRSP_MPfeatures_13_TL.csv - out=Ruwan_SHRSP_MPfeatures_13.sky
SkylineDailyRunner.exe --in=Ruwan_SHRSP_MPfeatures_13.sky --save --import-process-count=4 --import-all-files=Data --report-name=PythonTemplate --report-add=PythonTemplate.skyr --report-file=Ruwan_SHRSP_MPfeatures_13.tsv --report-format=TSV --chromatogram-file=Ruwan_SHRSP_MPfeatures_13_TIC.tsv --chromatogram-tics

Any possible reason?

 
Brendan MacLean responded:  2019-05-30 11:37

Yeah. Must have made a mistake in saving the .bat file or something, or I just verified my changes too quickly. I had not intended to import the transition list twice. Don't really have time to look at the TIC issue right now. Do you get them all when you use File > Export > Chromatograms?

 
veerupaksh responded:  2019-05-30 11:43

I think there might an issue with the data. The TIC Chromatograms do not export correctly with the GUI as well.

However, both the GUI and the command line work for a different dataset.

Thanks for all the help. Best wishes for the Skyline User Group Meeting, and congratulations on the 10th Anniversary of Skyline!