Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Modify AWS tests to use isolated resources #23419

Merged
merged 10 commits into from
May 22, 2024
Merged

Modify AWS tests to use isolated resources #23419

merged 10 commits into from
May 22, 2024

Conversation

fdalmaup
Copy link
Member

@fdalmaup fdalmaup commented May 14, 2024 •
edited

Related issue
#22514

Description

Finished adapting the AWS module integration tests to use isolated resources created and deleted during the execution:

  • For each test requiring an S3 bucket or a CloudWatch Logs log group, the test creates the necessary resources which are destroyed at the end of the test. This allows tests to be launched in parallel without colliding due to using the same resources.
  • The original tiers for the different test and test cases have not been modified.
  • It allows using different AWS environments without specifying the profile in the tests nor the configuration (it can be set as the AWS_PROFILE environment variable)

Tests

========================================================================================= short test summary info =========================================================================================
FAILED wazuh/tests/integration/test_aws/test_regions.py::test_regions[cloudtrail_inexistent_region] - AssertionError: The AWS module did not show correct message about non-existent region
FAILED wazuh/tests/integration/test_aws/test_regions.py::test_regions[vpc_inexistent_region] - AssertionError: The AWS module did not show correct message about non-existent region
FAILED wazuh/tests/integration/test_aws/test_regions.py::test_regions[config_inexistent_region] - AssertionError: The AWS module did not show correct message about non-existent region
FAILED wazuh/tests/integration/test_aws/test_regions.py::test_regions[alb_inexistent_region] - AssertionError: The AWS module did not show correct message about non-existent region
FAILED wazuh/tests/integration/test_aws/test_regions.py::test_regions[clb_inexistent_region] - AssertionError: The AWS module did not show correct message about non-existent region
FAILED wazuh/tests/integration/test_aws/test_regions.py::test_regions[nlb_inexistent_region] - AssertionError: The AWS module did not show correct message about non-existent region
=============================================================== 6 failed, 188 passed, 3 skipped, 2 xfailed, 1 warning in 5265.21s (1:27:45) ===============================================================

Note

The failing tests should be modified once the behavior of the AWS module regarding unexistent regions is checked in the #23431 issue.

Important

This PR must be merged along with wazuh/qa-integration-framework#25 which contains several modifications to the Framework used for the tests

@fdalmaup fdalmaup self-assigned this May 14, 2024
@fdalmaup fdalmaup linked an issue May 15, 2024 that may be closed by this pull request
Complete isolated generation of resources in AWS IT #22514
Closed
@fdalmaup fdalmaup force-pushed the 17788-aws-integration-test-migration-and-refactor branch from 4b33cc7 to e495436 Compare May 15, 2024 07:28
@fdalmaup fdalmaup force-pushed the enhancement/22514-complete-aws-its-resources-isolation branch from 083255b to 6587a67 Compare May 15, 2024 07:36
@fdalmaup fdalmaup marked this pull request as ready for review May 15, 2024 07:46
@fdalmaup fdalmaup force-pushed the enhancement/22514-complete-aws-its-resources-isolation branch 2 times, most recently from 1e4df04 to df2280e Compare May 20, 2024 15:45
@fdalmaup fdalmaup mentioned this pull request May 20, 2024
Closed
1 task
nico-stefani
nico-stefani requested changes May 20, 2024
View reviewed changes
Copy link
Member

@nico-stefani nico-stefani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good Job!
I left you with some small improvements to make.

tests/integration/test_aws/configurator.py Outdated Show resolved Hide resolved
tests/integration/test_aws/conftest.py Outdated Show resolved Hide resolved
...on/test_aws/data/test_cases/only_logs_after_test_module/cases_cloudwatch_multiple_calls.yaml Outdated Show resolved Hide resolved
tests/integration/test_aws/conftest.py Outdated Show resolved Hide resolved
tests/integration/test_aws/conftest.py Outdated Show resolved Hide resolved
tests/integration/test_aws/conftest.py
Comment on lines +274 to +287
s3_client (boto3.resources.base.ServiceResource): S3 client used to manage the bucket resources.
ec2_client (Service client instance): EC2 client to manage VPC resources.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aren't these the same type?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The instanced objects are from different classes. The first one is a ServiceResource (called client since it has a similar use as a client) and the second is a Service client instance

tests/integration/test_aws/conftest.py Outdated Show resolved Hide resolved
tests/integration/test_aws/conftest.py Outdated Show resolved Hide resolved
tests/integration/test_aws/conftest.py Outdated Show resolved Hide resolved
nico-stefani
nico-stefani approved these changes May 21, 2024
View reviewed changes
Copy link
Member

@nico-stefani nico-stefani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Selutario
Selutario reviewed May 22, 2024
View reviewed changes
Copy link
Member

@Selutario Selutario left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job @fdalmaup 🎉 I added just one question

tests/integration/conftest.py Outdated Show resolved Hide resolved
@fdalmaup fdalmaup force-pushed the 17788-aws-integration-test-migration-and-refactor branch 2 times, most recently from bac68b6 to c8cf993 Compare May 22, 2024 09:22
@fdalmaup fdalmaup force-pushed the enhancement/22514-complete-aws-its-resources-isolation branch from 4637420 to 45dd6ea Compare May 22, 2024 09:30
Selutario
Selutario approved these changes May 22, 2024
View reviewed changes
Copy link
Member

@Selutario Selutario left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Selutario Selutario merged commit 4c0da89 into 17788-aws-integration-test-migration-and-refactor May 22, 2024
10 of 38 checks passed
@Selutario Selutario deleted the enhancement/22514-complete-aws-its-resources-isolation branch May 22, 2024 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nico-stefani nico-stefani nico-stefani approved these changes

@Selutario Selutario Selutario approved these changes

Assignees

@fdalmaup fdalmaup

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Complete isolated generation of resources in AWS IT
3 participants
@fdalmaup @nico-stefani @Selutario