Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration and adaptation of tier 1 manager, agent and WPK package generation tools #23508

Merged
merged 85 commits into from
May 27, 2024

Conversation

jotacarma90
Copy link
Member

@jotacarma90 jotacarma90 commented May 17, 2024

jotacarma90 and others added 26 commits May 22, 2024 17:16
* Migrated filebeat template and create wofkflow to generate filebeat package

* Change revision variable name to version, due to more accurated
* Migrate WPK directory to wazuh/wazuh repository

* Add workflow to create WPK packages

* Add mechanism to dont build WPK generation images and download from ghcr

* Fix generate WPK script when no key dir passed

* Add tag parameter to generate wpk script and workflow related

* Remove workflows triggers on pull_request

* Multiple fixes in workflow:

- Added new input and functionality to format WPK name
- Check if WPK with same name exists, and exit the workflow
- Added -j parameters to generate scripts
- Add conditions to check if some package will be generate, to avoid some steps

* Add some info into the readme file

* Some changes related to format parameter inputs
…h/wazuh repository (#22435)

* Add packages for deb, rpm and windows

* Add macos to packages

* Add build.sh and generate_package.sh

* Add short hash for package_names and clean generate_packages.sh

* Adds short hash to package_name

* Fix rpm package name to follow the name convention

* Rename rpm arch from x86_64 to amd64

* Added arch name fix in rpm packages creation

* Added Dockerfiles for debs managers

* Added Dockerfiles for rpm managers

* Standarized arch naming

* Updated SPECS

* Updated package naming

* Removed unused Vagrantfile

* Fixed rpm packages names

* Migrated ghcr scripts

* Added upload-deb-manager-images.yml workflow

* Updated packages-upload-images.yml and helper scripts

* Added packages-build-amd64.yml

* Added on-demand package generation

* Added _pkg prefix to CONTAINER_NAME

* Added runner selector by arch

* Fix metadata problem in rpm packages

* Improved manager builder workflows

* Sync with agent migration development

* Fix manager package buiding via workflow call

* Uploaded variables name and added package build condition

* Updates deb and rpm SPECS

* Updated packages naming procedure

* Improved workflows parametrization

* Fixed rpm --src option

* Added workflow abort if package already exists

* Updated docker tag management

* Migrated testing tools from wazuh-packages

* Added install test before package upload

* Fixed paths for docker test

* Updated scripts and workflows with required changes

* Updated multi server builder workflow

* Updated inputs

* Addressed comments

* Fixed dockerfile path for arm64 and ppc64le

* Removed unused if in manager package builder

* Use dpkg in test install

* Fix: removed arm64/aarch64 architecture option

* Fix: applied minor fixes and added git as dependency in debs/amd64 Dockerfile

* Fix: removed non manager-related files

* Fix: fixed some problems regarding previous rebase

* Address some minors comments

---------

Co-authored-by: Luis Enrique Chico Capistrano <luis.chico@wazuh.com>
* Add packages for deb, rpm and windows

Add macos to packages

Add build.sh and generate_package.sh

Add short hash for package_names and clean generate_packages.sh

Adds short hash to package_name

Fix rpm package name to follow the name convention

Rename rpm arch from x86_64 to amd64

Added arch name fix in rpm packages creation

Rename aarch64->arm64, armv7hl->armhf

Fix issue cause of rename architecture

Move build_deps.sh from rpms/armhf/agent to rpms/utils

Improve macos script and update pkg name to follow the new convention

Update file name convention for zip windows pkg

Workflows testing

Upload docker images

Fix newline not accepted

Fix macos generate_wazuh_packages and fix windows name convension

Fix macos specs

Fix macos dependency issues

Testing push docker changging tag

Rename container_name testing permission issues

Testing using workflowdis_patch

Remove dev flag

Update packages upload workflow

Remove referenced to package-build workflow

Rename package-buil to package-build-linux-agent

Fix copy the correct helpers

Testing

Revert "Testing"

This reverts commit 26b6b5a.

Add release name option

Add flag release package for deb

Update workflow for push agent and docker images

Update workflow

Undo release package flag, back again

Update agent workflow, added package-release flag

Add legacy and debug the name for rpm developer

Export only the source package or the package file, add checksum option

Add legacy dockerfile

Update workflow adding a pull_request

Revert "Update workflow adding a pull_request"

This reverts commit 2dc9766.

Update workflow

Fix error trying to generate docker container for legacy

Remove rtr accions

Test workflow functionalities

Update scripts and upload images

Update linux agent workflow

Fix abort workflow

mend

Avoid push the same docker image

Remove branch option from workflow and add README.md

Remove check if the image or package alredy exist

Update README.md

Introduce the new changes

Add architecture step and pending docker_builder script for vagrant build

Update architecture values

Add test and clean the code

Fix test, path incorrect

Tried to fix i386 tests

Fix to use of docker builder for testing

Fix IS_STAGE bug, and avoid remove legacy/docker_builder.sh

Migrate and adapt macos wazuh agent packages subsystem (#22707)

* Added debug option and release naming

* Added release and develop package name

* Added wazuh repository code use

* Added macos package generation workflow

* Added install test step

* Add smoke test check to macos package generation

* Fix some inputs descriptions and jobs name

* Fix some descriptions and modify generate script to set commit hash with git rev-parse

---------

Co-authored-by: Jotacarma90 <jose.carreras@wazuh.com>

Migrate and adapt Windows wazuh agent packages subsystem (#22631)

* Modified windows generate package script and create workflow to use it

* Introduce the new changes and fixes

* Fix zip name

* Generate and push msi to s3

* Check download step

* Adapted Windows package workflow to generate .zip as artifact

* Modified generate msi script to set msi name with parameter

* Added smoke test install after msi creation

* Fix some jobs name and inputs descriptions

---------

Co-authored-by: Luis Enrique Chico Capistrano <luis.chico@wazuh.com>

Improve package install validation and job title

Improve workflow title for uploard images

Update comments

Change some descriptions to better understand

Testing exit 1 on workflow

Revert "Testing exit 1 on workflow"

This reverts commit e093042.

Fix validation to dpkg command and upload docker image yml

Address Jose comment's

* Fixed problems regarding previous rebase

* Addressed Manu's comments

* Modify manager and agent specs in rpm package generation to use metadata and allow no stage nomenclature

* Modified get_package_and_checksum function to use full name of the package in rpm

* Fix: Readme documentation

* Fix: For developer tag, if the branch name has "/" change change by "… (#22973)

* Fix: For developer tag, if the branch name has "/" change change by "--".

* Address Manu's comments

* docs: improving description to avoid errors

---------

Co-authored-by: Marcel Kemp <marcel.kemp@wazuh.com>

* Removed migrated workflows

* Fix pkg_name variable in macOS package generation script

---------

Co-authored-by: Jotacarma90 <jose.carreras@wazuh.com>
Co-authored-by: Marcel Kemp <marcel.kemp@wazuh.com>
Co-authored-by: Manuel <manuel.cano@wazuh.com>
… some problems in the new retag_image.sh script
Co-authored-by: Marcel Kemp Muñoz <marcel.kemp@wazuh.com>
@MarcelKemp MarcelKemp force-pushed the migration/21755-wazuh-packages-redesign branch from a0d8a4e to fc99bb7 Compare May 22, 2024 15:32
Copy link
Member

@lchico lchico left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments.

.github/actions/ghcr-pull-and-push/retag_image.sh Outdated Show resolved Hide resolved
.github/workflows/packages-filebeat.yml Outdated Show resolved Hide resolved
.github/workflows/packages-upload-manager-images.yml Outdated Show resolved Hide resolved
packages/README.md Outdated Show resolved Hide resolved
packages/README.md Show resolved Hide resolved
packages/macos/generate_wazuh_packages.sh Outdated Show resolved Hide resolved
packages/macos/generate_wazuh_packages.sh Outdated Show resolved Hide resolved
packages/macos/generate_wazuh_packages.sh Show resolved Hide resolved
packages/macos/package_files/build.sh Outdated Show resolved Hide resolved
lchico
lchico previously approved these changes May 23, 2024
Copy link
Member

@lchico lchico left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Member Author

@jotacarma90 jotacarma90 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment on lines 66 to 68
# TODO: Remove the "pkg" appended to the container name.
# This was necessary because we don't have permission to overwrite an existing container.
# Also, ensure that the same adjustment is made in the else condition.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this TO DO, as it is not a problem.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

packages/wpk/README.md Outdated Show resolved Hide resolved
Comment on lines 61 to 62
3. Build a legacy RPM package for CentOS 5 (manager):
./wazuh_package_builder.sh -t manager -l -s /tmp --system rpm
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generate for a legacy agent.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Member

@mjcr99 mjcr99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@MarcelKemp MarcelKemp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. GJ team!

@MarcelKemp MarcelKemp merged commit 4159c29 into 4.9.0 May 27, 2024
48 checks passed
@MarcelKemp MarcelKemp deleted the migration/21755-wazuh-packages-redesign branch May 27, 2024 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
7 participants