Hi William,
I actually think at its heart, Skyline is entirely side-by-side installation. It does not modify any registry settings. I guess it does have prerequisites in the Windows environment. (e.g. .NET runtimes)

I would have a look at the "unplugged" installation (there is always a link on every installation page), which is packaged in a ZIP file. It is my belief that if you start with a system that contains the prerequisites, and you get the contents of the directory containing Skyline.exe into a single directory on the target computer (or Docker container), then you can just run Skyline.exe from there....

Hmmm... But, if you then want to run Skyline itself from a command-line using SkylineRunner.exe things get ever so slightly more complicated. Though, we could probably create a SkylineRunner.exe for that case, if you are interested.

Maybe first confirm my theory about just getting all the files onto a computer in the same directory structure as you find Skyline.exe in the unplugged ZIP file. I just tried it myself and it seemed to work, but my computer obviously is very far from pristine.

Interesting project. Jarrett Egertson in our lab is beginning to work on a similar deployment. This will probably help move us in a direction that seems interesting to quite a few people.

--Brendan

Brendan,
Thanks for taking some time to respond!

I have code already set up to grab the unplugged installation, unzip, install .NET dependencies...

Based on your suggestion, I manually dropped the Skyline(Daily)Runner.exe into the skyline folder on a machine which has never seen the Skyline installer. Below is the output. A modified version of the Runner shim executable would be terrific, such that it would look into the local directory instead of the default install locations. (Out of curiosity, is there a static link I could use to reliably grab the newest "Daily" runner.exe and newest "Daily" unplugged build?)

I'm happy to hear that you guys are exploring similar avenues! Virtualization seems to be so important these days for handling computational infrastructure.

Cheers,
William

C:\Users\Administrator\skyline-app>SkylineDailyRunner
Error: Unable to find Skyline program at any of the following locations:
C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\MacCoss Lab, UW\Skyline-daily.appref-ms
C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Skyline-daily\Skyline-daily.appref-ms

I just posted updated SkylineDailyRunner.exe files to the Skyline-daily installation pages that now check the application directory as a place to find Skyline-daily.exe. Download and let me know if this works for you. I only gave it very light testing myself.

You shouldn't really need to worry about having the latest Skyline[Daily]Runner.exe. It is just a very thin shim around Skyline[-daily].exe to make running it from the command line work. All of the actual command-line processing code lives inside the Skyline[-daily].exe.

For instance, until tonight the last time we made a meaningful change to SkylineRunner.exe was June 19, 2015 when support was added for it return non-zero when SkylineRunner fails to start or connect to Skyline.exe. Other than that it has been relatively stable for nearly 2 years.

Let me know how this works for you and I will also update SkylineRunner with this change.

--Brendan

Did the new SkylineDailyRunner.exe work for you?

Brendan,
Unfortunately it looks like the app is actually looking for
".\{executable name}\Skyline-Daily"

============================================================================================
C:\Users\Administrator\skyline-app>SkylineDailyRunner
Error: Unable to find Skyline program at any of the following locations:
C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\MacCoss Lab, UW\Skyline-daily.appref-ms
C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Skyline-daily\Skyline-daily.appref-ms
C:\Users\Administrator\skyline-app\SkylineDailyRunner.exe\Skyline-daily
============================================================================================

Instead of just
".\Skyline-Daily" (or ".\Skyline.exe" as the exe extracted from the Application Files directory of the -Daily unplugged .zip is named)

Cheers,
-William

See what I mean! Yeah, in my quick local test before posting I didn't even uninstall Skyline-daily. So, of course it worked.

This time I uninstalled and first proved that got the error "Unable to find Skyline program", and then fixed the error by only copying a new SkylineDailyRunner.exe into the place I was running my test.

So, I feel quite a bit more confident that this one will work (newly posted), but please confirm that it works for you.

Thanks for posting your request and your help in validating the change.

--Brendan

Hi William,
We are now heading in a different direction for solving this request. SkylineRunner.exe is nice for people actually installing Skyline.exe using the self-updating installer, which puts Skyline.exe in a location where it would be very difficult to ask people to just run a .exe from the command-line. SkylineRunner was expressly designed to bridge that gap.

However, once you make the assumption that the user knows the path to Skyline.exe, it becomes much easier to solve the one issue of Skyline.exe being a Windows EXE and not a command-line EXE. So, Nick, on the Skyline team, created a new shim EXE for running Skyline[-daily] when you know its path. That will soon be Skyline[Daily]Cmd.exe and it will always live in the same directory as Skyline[-daily].exe.

Running SkylineCmd.exe will also be simpler than running SkylineRunner.exe, where you end up with two processes communicating through named pipes. SkylineCmd.exe will only require the one process for most operations (excluding new processing coming soon where Skyline will start a separate process for every file being imported into the document, which has proven to have positive performance impact on servers with many processors).

Thanks for your feedback getting us to rethink your use case. This improvement should be available in the next Skyline-daily release.

--Brendan

I will hold onto this magical SkylineDailyRunner for the time being!

If there will not be a method to run the "local directory" Skyline, will there be a "quiet"/CLI install enabled for Skyline so that I can set it up on a headless box? I have not been able to get Skyline to install on a headless system like a Docker container. Am I missing a way to do so?

I am excited for the multiprocessing implementation over the threaded import.

Thanks for all the hard work!

-wb

Hi William,
I am not quite sure what you mean now by "install" Skyline. You can run it, right? Just unzip the files and run. I mostly think of installing Skyline as something that puts it where a UI user can find it, in the start menu or quick-searchable by name. So, I am not clear what you are missing now, but maybe if you give me a clearer description, we can make it possible.

Glad that it sounds like the SkylineDailyRunner changes work for you. They kind of worked for me also, but I ran into problems with SkylineDailyRunner holding onto my log file causing subsequent commands in my batch file to fail because the log file was locked. Hence the move to the SkylindCmd.exe simplification. Still not sure why running a local copy would change that, as SkylineDailyRunner.exe still works without issue on my installed versions.

If you really want Skyline "installed" into its normal location <user root>\AppData\Local\Apps\... then you may still need SkylineRunner to find it and run it. Just not clear why you would want that for a headless installation. SkylineCmd.exe is making my life easier for the implement and debug cycle, since I don't need to install anything to run it.

Multiprocess is particularly nice on a system with many cores. I am currently running 12 concurrent processes on a NUMA server with 48 logical cores and 192 GB of RAM and this allows me to run the Rosenberger pan-human search on 21 files in about 1.5 hours.

Though, you can also distribute the single file import to many different computers with what is available today by using --import-no-join. This is what multi-process Skyline is using internally to import a single file with each process and then join everything together, which you would do with a single call to SkylineRunner at the end, requesting the import of all of the files that were imported individually, but without the --import-no-join argument. This should allow you to do larger scale cluster processing, which I have not yet done myself, but I have tested the concept with batch file code like this:

rem Simulate distributed processing on a cluster
for %%i in (%ROOT_DIR%\%DATA_DIR%\*.mz5) do %SKYLINE_RUNNER% --timestamp --dir="%ROOT_DIR%" --in="%SKYLINE_FILE%" --import-file="%%i" --import-no-join >> %LOG% 2>&1

rem Join everything and complete processing
%SKYLINE_RUNNER% --timestamp --dir="%ROOT_DIR%" --in="%DATA_DIR%\%MODEL_NAME%\%SKYLINE_FILE%" --import-all="%DATA_DIR%" --import-naming-pattern="_([^_]*)$" --reintegrate-model-name="%MODEL_NAME%" --reintegrate-create-model --save --report-name=SWATHbenchmark --report-file="%DATA_DIR%\%MODEL_NAME%\SWATHbenchmark.csv" --report-invariant >> %LOG%

Maybe something to think about in a dockerized implementation. Very interested in hearing where you end up with this.

--Brendan