release: adding nupkg cpack build option, updating win deployment script #6262

muffins · 2020-02-21T06:13:39Z

Summary:
This brings the NuGet packaging option to cpack. We can now produce Chocolatey/NuGet packages as well as MSI packages for Windows.

Test Plan:

Clean install of osquery

PS C:\Users\thor\Desktop\osquery_release_bins\4.2.0> choco install -yf osquery -s . --version 4.2.0 --params='/InstallService'
Chocolatey v0.10.15
Installing the following packages:
osquery
By installing you accept licenses for the packages.

osquery v4.2.0 (forced)
osquery package files install completed. Performing other installation steps.
C:\Program Files\osquery\log
True
osqueryd
PATH environment variable does not have C:\Program Files\osquery in it. Adding...
Environment Vars (like PATH) have changed. Close/reopen your shell to
 see the changes (or in powershell/cmd.exe just type `refreshenv`).
 ShimGen has successfully created a shim for osqueryi.exe
 ShimGen has successfully created a shim for osqueryd.exe
 The install of osquery was successful.
  Software install location not explicitly set, could be in package or
  default install location if installer.

Chocolatey installed 1/1 packages.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

PS C:\Users\thor\Desktop\osquery_release_bins\4.2.0> get-service osqueryd

Status   Name               DisplayName
------   ----               -----------
Running  osqueryd           osqueryd

PS C:\Users\thor\Desktop\osquery_release_bins\4.2.0> ls 'C:\Program Files\osquery\log\'


    Directory: C:\Program Files\osquery\log


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----        2/20/2020  10:00 PM            287 osqueryd.INFO.20200220-220024.20600
-a----        2/20/2020  10:00 PM              0 osqueryd.results.log

Upgrade of osquery, installation from upstream choco latest stable version

PS C:\Users\thor\Desktop\osquery_release_bins\4.2.0> choco upgrade -yf -s . osquery --version 4.2.0
Chocolatey v0.10.15
Upgrading the following packages:
osquery
By upgrading you accept licenses for the packages.

You have osquery v4.0.2 installed. Version 4.2.0 is available based on your source(s).
True
True

osquery v4.2.0 (forced)
osquery package files upgrade completed. Performing other installation steps.
True
True
C:\Program Files\osquery\log
True
True
True
True
Environment Vars (like PATH) have changed. Close/reopen your shell to
 see the changes (or in powershell/cmd.exe just type `refreshenv`).
 ShimGen has successfully created a shim for osqueryi.exe
 ShimGen has successfully created a shim for osqueryd.exe
 The upgrade of osquery was successful.
  Software install location not explicitly set, could be in package or
  default install location if installer.

Chocolatey upgraded 1/1 packages.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

PS C:\Users\thor\Desktop\osquery_release_bins\4.2.0> Get-Service osqueryd

Status   Name               DisplayName
------   ----               -----------
Running  osqueryd           osqueryd

PS C:\Users\thor\Desktop\osquery_release_bins\4.2.0> ls 'C:\Program Files\osquery\log\'


    Directory: C:\Program Files\osquery\log


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----        2/20/2020  10:11 PM            287 osqueryd.INFO.20200220-221131.16872
-a----        2/20/2020  10:10 PM              0 osqueryd.results.log

Uninstalling osquery from system:

PS C:\Users\thor\Desktop\osquery_release_bins\4.2.0> choco uninstall -yf osquery
Chocolatey v0.10.15
Uninstalling the following packages:
osquery
True
True

osquery v4.2.0 (forced)
@{ReturnValue=0; PSComputerName=}
 Skipping auto uninstaller - No registry snapshot.
 osquery has been successfully uninstalled.
Environment Vars (like PATH) have changed. Close/reopen your shell to
 see the changes (or in powershell/cmd.exe just type `refreshenv`).

Chocolatey uninstalled 1/1 packages.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

muffins · 2020-02-21T06:26:05Z

I'm going to follow-up this PR with some documentation on how to build this package.

muffins · 2020-02-22T04:43:46Z

This addresses #6248

muffins · 2020-02-25T21:31:56Z

cmake/packaging.cmake

+        - Check that the file hash of the osqueryd.exe binary is correct:
+        ```
+        PS C:\\Users\\thor> (Get-FileHash -Algorithm sha256 'C:\\Program Files\\osquery\\osqueryd\\osqueryd.exe').Hash
+        <Fill in with osqueryd.exe Sha256 before publishing to Chocolatey>


I don't really like doing it like this, but I'm not sure howe to get the file digest of the osqueryd.exe binary before cpack is invoked. Right now we already need to manually swap out the binaries with the signed versions before calling cpack anyway, so I'm going to say we just manually update the verification.txt, unless someone smarter than I in cmake can tell me how to get the sha256 of osqueryd.exe at this point :)

You can add a custom command that runs after the build of a target, check this https://cmake.org/cmake/help/v3.16/command/add_custom_command.html and POST_BUILD.

EDIT: It's important to note though that this will run only if the target has not already been built, so for instance if you have a case where this has to be run always before packaging (but on the result of a target), and a previous saved value/result is not correct, this won't work.
If though is possible to create some dependency on some file which if it's not there, then the command has to be launched, then you can use a normal custom command that depends on such file and that runs each time it's missing.
There are also some other ways, depending on the specific behavior you want to obtain.

theopolis · 2020-03-04T01:03:26Z

cmake/packaging.cmake

-    file(COPY "${CMAKE_SOURCE_DIR}/tools/wel/osquery.man" DESTINATION "${CMAKE_BINARY_DIR}/package/wix")
-    install(FILES "${CMAKE_BINARY_DIR}/package/wix/osquery.man" DESTINATION .)
+      file(WRITE "${CMAKE_BINARY_DIR}/package/${win_packager}/VERIFICATION.txt" "
+VERIFICATION


Can you put this file into ./tools (into an appropriate sub folder) and then copy it similar to how CMake is copying other files.

I could, the issue I was originally having, and why this is here, is I wanted to grab the hashes of the built binaries and have them be included in this file. For now I'll just remove the pre-computed hashes and maybe that can be a follow-up for us, as I don't think it's required.

directionless · 2020-03-18T19:02:43Z

Where did we leave this?

muffins · 2020-03-19T03:35:12Z

@directionless thanks for the ping! I didn't have a huge amount of time to get things fixed up, I think there's one or two more small bits that need addressing, I'll take another look tonight but I believe we could get this shipped and maybe cut a follow-up PR/Issue to track the additional bits that might need doing.

muffins · 2020-05-08T16:05:11Z

I believe all of the issues with this have been addressed. Any chance I could get another review and we could get this shipped? I'm VERY behind on shipping the osquery 4.3.0 chocolatey packages, and having this landed upstream would be extremely useful for anyone else wishing to build choco packages. Sorry for my delays y'all!

muffins added deployment Windows release labels Feb 21, 2020

muffins force-pushed the cpack-nupkg-release branch from 9cbbf21 to f17a839 Compare February 21, 2020 06:18

muffins force-pushed the cpack-nupkg-release branch from f17a839 to ee3d624 Compare February 21, 2020 22:48

muffins force-pushed the cpack-nupkg-release branch from ee3d624 to 8a737cd Compare February 22, 2020 22:06

muffins mentioned this pull request Feb 25, 2020

Move nupkg to cpack #6248

Closed

muffins force-pushed the cpack-nupkg-release branch from 8a737cd to f1cfa6b Compare February 25, 2020 21:29

muffins commented Feb 25, 2020

View reviewed changes

muffins force-pushed the cpack-nupkg-release branch 2 times, most recently from aed6295 to a4d0054 Compare February 26, 2020 00:03

theopolis reviewed Mar 4, 2020

View reviewed changes

directionless mentioned this pull request Mar 18, 2020

Build MSI Powershell - make_windows_package.ps1 #6309

Closed

muffins force-pushed the cpack-nupkg-release branch 3 times, most recently from e2bd231 to 056af4b Compare March 20, 2020 18:20

muffins force-pushed the cpack-nupkg-release branch from 056af4b to cd51446 Compare May 8, 2020 16:04

release: adding nupkg cpack build option, updating win deployment script

3619b2a

muffins force-pushed the cpack-nupkg-release branch from cd51446 to 3619b2a Compare May 21, 2020 16:50

theopolis approved these changes May 21, 2020

View reviewed changes

theopolis merged commit d1feab7 into osquery:master May 25, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

release: adding nupkg cpack build option, updating win deployment script #6262

release: adding nupkg cpack build option, updating win deployment script #6262

muffins commented Feb 21, 2020 •

edited

muffins commented Feb 21, 2020

muffins commented Feb 22, 2020

muffins Feb 25, 2020

Smjert Feb 25, 2020 •

edited

theopolis Mar 4, 2020

muffins Mar 19, 2020

directionless commented Mar 18, 2020

muffins commented Mar 19, 2020

muffins commented May 8, 2020

release: adding nupkg cpack build option, updating win deployment script #6262

release: adding nupkg cpack build option, updating win deployment script #6262

Conversation

muffins commented Feb 21, 2020 • edited

Clean install of osquery

Upgrade of osquery, installation from upstream choco latest stable version

Uninstalling osquery from system:

muffins commented Feb 21, 2020

muffins commented Feb 22, 2020

muffins Feb 25, 2020

Choose a reason for hiding this comment

Smjert Feb 25, 2020 • edited

Choose a reason for hiding this comment

theopolis Mar 4, 2020

Choose a reason for hiding this comment

muffins Mar 19, 2020

Choose a reason for hiding this comment

directionless commented Mar 18, 2020

muffins commented Mar 19, 2020

muffins commented May 8, 2020

muffins commented Feb 21, 2020 •

edited

Smjert Feb 25, 2020 •

edited