Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add missing validation to fix last table #6147

Merged
merged 20 commits into from
Jan 14, 2020
Merged

Conversation

joaogodinho
Copy link
Contributor

@joaogodinho joaogodinho commented Dec 23, 2019

A validation was added to the genLastAccess function (#5274) which would filter for login events only (USER_PROCESS). This change breaks the intent of the last table, which should show logins and logouts.

This PR updates extracts the last implementation to its own function, which is OS independent, enabling tests to be made to the implementation, as well as adding a test for the content (if existing) of the last table.

The reasoning for the validation in the first place is to remove information unrelated to user login/logout (check man wtmp for more info).

@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Dec 23, 2019

CLA Check
The committers are authorized under a signed CLA.

@Smjert
Copy link
Member

Smjert commented Dec 23, 2019

@joaogodinho thanks for this PR, looks good!

Please fix that small issue with commas and remember to sign the CLA.

@directionless
Copy link
Member

make format and make format-check mat be handy

@joaogodinho
Copy link
Contributor Author

make format and make format-check mat be handy

Hi @directionless, the tests failure is related to the containers not populating the wtmp file. I'm working on refactoring the table code so that implementation tests are run on all platforms, but table tests only if the table actually contains something.

I've discussed this with Stefano on the code-review channel on slack. As soon as I have the code, I'll also update the first comment to reflect the choices and what was discussed.

@Smjert
Copy link
Member

Smjert commented Jan 3, 2020

The unit test idea looks good, but it does have an important downside, which is that the integration test will never test the conditions on the CI, because the system file won't have records in it.

Unfortunately since the implementation of a table is with free functions and we cannot change the parameters of the generate functions that sqlite has to call, the only possible alternative I see is to use a global variable in the table code, which stores the path to the file the table has to parse.
This will be accessed via extern in the test code and changed to a test controlled path, where you would use the utmp functions to write on it and then retrieve the values.

In theory we could have the CI use a custom wtmp file, put in the system path, but this might not always work when a user launches tests locally.
So the approach I propose also has the added benefit that we have control on what the file contains and we do not alter the system where the test is run on.

@joaogodinho
Copy link
Contributor Author

Correct, this won't test the table itself in any CI that's run inside a container.

I've spent a good time trying to find a way for the test to call the wtmp lib directly to some tmp file, but I couldn't implement one that wouldn't had multiple #ifdef due the different OSes, which would in the end limit the tests as well, because in Apple and FreeBSD (if setutxdb is not available) we can't set which wtmp file to parse. If we wanted to be pedantic about the tests, we'd still need to have a test for when we can read from the file, and one for when we can't.

Copy link
Member

@Smjert Smjert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, this won't test the table itself in any CI that's run inside a container.

I've spent a good time trying to find a way for the test to call the wtmp lib directly to some tmp file, but I couldn't implement one that wouldn't had multiple #ifdef due the different OSes, which would in the end limit the tests as well, because in Apple and FreeBSD (if setutxdb is not available) we can't set which wtmp file to parse. If we wanted to be pedantic about the tests, we'd still need to have a test for when we can read from the file, and one for when we can't.

Unfortunately I don't think that running them outside a container would change anything, because the VM it's run on it's destroyed after each pipeline run, so it would most likely not have any record in it.
My request was due to the fact that we rely on tests not failing in the CI, beyond manual review, to decide if a PR can be merged, or if there are regressions.
A test that never really run on the CI is a tad less useful.

Though I understand that the current situation is not ideal, we should have another structure for implementing table code; so thanks for trying!

Lets keep this structure, but please address my comment below.

osquery/tables/system/posix/last.cpp Outdated Show resolved Hide resolved
osquery/tables/system/tests/posix/last_tests.cpp Outdated Show resolved Hide resolved
osquery/tables/system/tests/posix/last_tests.cpp Outdated Show resolved Hide resolved
osquery/tables/system/tests/posix/last_tests.cpp Outdated Show resolved Hide resolved
osquery/tables/system/tests/posix/last_tests.cpp Outdated Show resolved Hide resolved
@joaogodinho
Copy link
Contributor Author

Unfortunately I don't think that running them outside a container would change anything, because the VM it's run on it's destroyed after each pipeline run, so it would most likely not have any record in it.

The MacOS VM does (check here), but it's the only one, since Linux is using containers.

My request was due to the fact that we rely on tests not failing in the CI, beyond manual review, to decide if a PR can be merged, or if there are regressions.
A test that never really run on the CI is a tad less useful.

Though I understand that the current situation is not ideal, we should have another structure for implementing table code; so thanks for trying!

I agree, unfortunately I don't have the resources right now to learn more about C++ and OSQuery structure itself.

Lets keep this structure, but please address my comment below.

I will, I also have a minor fix for the test that I probably discard accidentally. 🙏

Copy link
Member

@Smjert Smjert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last round :).
Also remember to format the code with make format from the build folder before committing.

osquery/tables/system/posix/last.cpp Outdated Show resolved Hide resolved
osquery/tables/system/posix/last.h Outdated Show resolved Hide resolved
joaogodinho and others added 4 commits January 6, 2020 11:40
Co-Authored-By: Stefano Bonicatti <smjert@gmail.com>
Co-Authored-By: Stefano Bonicatti <smjert@gmail.com>
Co-Authored-By: Stefano Bonicatti <smjert@gmail.com>
Co-Authored-By: Stefano Bonicatti <smjert@gmail.com>
@joaogodinho
Copy link
Contributor Author

ping @Smjert 😃

Copy link
Member

@Smjert Smjert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
Sorry for the delay.

@Smjert Smjert merged commit a6ffa37 into osquery:master Jan 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants