Can I get more Verbose output from SkylineCmd.exe ?

support
Can I get more Verbose output from SkylineCmd.exe ? lparsons  2017-09-11 13:11
 
I don't see a verbosity argument in the documentation I have for Skyline; is there a way to get more verbose output? I have been trying to get it to build a file using the results of a large number (30) of raw files and once the last one reaches 96% there is no longer any sign of anything happening. If I check Windows Task Manager the most CPU activity I see from a SkylineCmd.exe" process is less than 10% (only 4 or 5 are listed the rest are 0% CPU). None of them are using more than ~120MB RAM either (on a system with 64GB).
I have a SkylineCmd.exe process that seems to have been stalled now for well over a week. It doesn't seem to have crashed, but it doesn't seem to be making any progress either. No new files have been written or started since starting the run.
I plan to try this again with far fewer files - when I run this same command with just one Thermo RAW file everything is fine - but I was wondering if there is any way to get more information from a CLI run.

thank you
Lee
 
 
Nick Shulman responded:  2017-09-12 12:47
Can you send us your files? I will try to figure out what is going wrong.
You can upload your files here:
https://skyline.ms/files.url

It sounds like you have a lot of files, but it also sounds like this is a bug in Skyline that we would definitely want to fix.

I do not know of any way to increase the verbosity level of SkylineCmd.
 
Brendan MacLean responded:  2017-09-12 12:56
Hi Lee,
That SkylineCmd.exe is not going to make any further progress. You might as well kill it.

What version of Skyline are you using? And are you attempting to use multi-process import? There was a bug in Skyline 3.7 for multi-process import where it would fail without reporting the error that caused the failure. That case might have come out looking something like what you are reporting.

Of course, there could be other problems, and I am sure Nick can help once he has your files, but if you are trying to use Skyline 3.7 for multi-process import the solution might be either to revert to single process import with 3.7 or try again with Skyline-daily, which has a fix to improve its logging of error messages during multi-process import.

Thanks for posting to the Skyline support board. Sorry about the 1 week wait. In the future, you should never need to wait that long. Mostly if you go more than even 10 minutes without any new output, you can assume something has gotten stuck and begin the process of reporting it to us.

--Brendan
 
lparsons responded:  2017-09-13 11:54
Brendan, Nick

Thank you for getting back to me. I suspect the problem may have actually been rooted in file access / file locking. I think I may have inadvertently had one of the files open in another program and skyline was unable to access it, ending up being left stalled out indefinitely.
When SkylineCmd is opening a .raw file is it opening it with write permission? I see that it creates a .skyd file to go with each .raw file, but if it was trying to open the .raw for writing I could see this problem coming up in windows.
I think for now it is working correctly and it was apparently user error.

Lee
 
Brendan MacLean responded:  2017-09-13 12:04
Hi Lee,
Well, we would like Skyline to deliver you an error message in that case, and not just hang like that. Were you actually using multi-process import? We have seen the error it should have given you before. So, it is a bit surprising that you got the behavior you described, unless you were using older multi-process import. In that case, we know it was swallowing error messages and maybe just hanging after that.

Still curious. It may have been user error, but we don't want users hanging around waiting for days for such a small mistake.

Skyline is not opening for writing, but apparently some instrument vendor readers open files with exclusive access. So, we can't even read them when they have them open. This is what Excel does as well.

Thanks for getting back to us. If you could give us a bit more information, I would sure like to understand this better.

--Brendan
 
lparsons responded:  2017-09-14 12:23
Brendan

I'm pretty sure now that the problem was with individual files attempting to be opened by multiple programs (or instances of the same program). At one point when I was experimenting with the SkylineCmd.exe syntax I had multiple runs of it going simultaneously (user error here; I neglected to see the other command prompt window I had open!) and they ended up colliding. More recently I took all the files I was trying to go through, copied them to a new directory, and then loaded them as single processes and everything went fine.

If I may make one suggestion for the documentation, I realized through trial and error (after some thought on previous results) that if the template skyline file already has a transition list in it, that list will be used and the "--import-transition-list" switch is not required if that is the list one wants to use for the data files that are being imported. In hindsight this is perhaps obvious but the documentation does not state that. When I added the transition list this way and it was identical to transitions already used in the template the result was a .sky file that was almost exactly twice as large as it was when the transition list was not imported on top.

thank you!
Lee
 
Brendan MacLean responded:  2017-09-14 12:59
Hi Lee,
Thanks for the update. I guess we can play around with this to see if we can get it to hang as you describe. We would much prefer you getting errors about files being in use by other processes than you waiting on a hung process, as the former is more likely to help you figure out what you are doing wrong, even when it is user error.

Interesting point on --import-transition-list. As we originally conceived it, the Skyline command-line interface was mostly intended to start with a fully formed Skyline document and then import files into it and export reports. It was only after that was in successful use and people started realizing the potential that we started adding ways for the command-line interface to be more of a settings template and import even the targets through FASTA, transition lists, assay libraries, etc. Which tend to be most useful in cases where other upstream command-line tools are used to generate a spectral library and the targets are not known at the beginning of the processing.

But, okay, point taken. We need to make it clearer that you can start with a fully formed document containing all of the targets you intend to use.

Thanks again for the update and feedback.

--Brendan