Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for gathering python packages from any user Python installations #7414

Merged
merged 7 commits into from
Feb 15, 2022

Conversation

iko1
Copy link
Contributor

@iko1 iko1 commented Dec 13, 2021

This PR fixes the bug reported at #7350. It adds a check whether 'queryKey' returns major error(e.g. access denied) or path not found.
The "path not found" error should be handled when writing registry globs like "HKEY_USERS\%\SOFTWARE\Python\PythonCore\%\InstallPath" The first glob wildcard is trivial to be found, but the other once depends on whether user installs any Python software within its registry tree.

I tested the fix with installing latest Python version for local-user only. My Python installation is stored under "C:\Users\Amir\AppData\Local\Programs\Python\Python310\python.exe" it's not a system location( e.g. c:\program files) .
After my fix the python packages populate to the table.

any feedback is more than welcome, Thanks!

@iko1 iko1 requested review from a team as code owners December 13, 2021 07:16
Copy link
Member

@Smjert Smjert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iko1 sorry for the delay and thank you for debugging this and fixing it!

May I ask you to also add some tests for the populateSubkeys function here https://github.com/osquery/osquery/blob/master/osquery/tables/system/tests/windows/registry_tests.cpp?

This fix modifies the behavior of the function when the list of keys to check contains only non existing keys, since previously it would return with failure, now it will return with success and depending on the replaceKeys boolean it will return either an empty set or with the same set of input keys.

Can we test that:

  1. Given an input set of keys with only one key in it, which is known to exists and has some subkeys, populateSubkeys called with replaceKeys to false returns with success and in the set you have the same input key + some additional keys (no need to check which exactly they are)
  2. Given an input set of keys with only one key in it, which is known to be an invalid key, populateSubkeys called with replaceKeys to false returns with failure.
  3. Given an input set of keys with 3 keys in it, where the first and the last are known to exists and have subkeys, but the middle one it's known to be invalid, populateSubkeys called with replaceKeys to false returns with success and the set of keys now contains the input keys + at least one subkey per valid input key

osquery/tables/system/windows/registry.cpp Outdated Show resolved Hide resolved
@iko1 iko1 force-pushed the fix-windows-python-packages branch from e7dd150 to c4ade62 Compare December 26, 2021 15:13
@iko1 iko1 requested a review from Smjert December 26, 2021 15:17
@iko1
Copy link
Contributor Author

iko1 commented Dec 27, 2021

Fixed the PR according to @Smjert comments.

@directionless directionless added this to the 5.2.0 milestone Dec 27, 2021
@Smjert
Copy link
Member

Smjert commented Dec 29, 2021

Fixed the PR according to @Smjert comments.

@iko1 Thank you very much for the tests!
I realized now that the second test as I've expressed it is incorrect, I'm sorry; I was thinking of a key which would actually cause the internal API's to fail for a cause different then it not being found.
In fact this caused you to add that atLeastOneKeyExists case.
I'm not entirely sure how to cause such failure; we can defer this test.

I don't think we should create a special case for "all the passed keys are not existing", and instead see that as success with an empty return if replaceKeys is true or the input keys if false.
I don't see the need to do such differentiation in this API.

So to clarify, I would remove the atLeastOneKeyExists logic and change the second test to check that the result is successful and that the returned keys only contain the input key.

@iko1
Copy link
Contributor Author

iko1 commented Dec 30, 2021

this caused you to add that atLeastOneKeyExists case. I'm not entirely sure how to cause such failure; we can defer this test.

It's possible to use gmock to overload the 'queryKey' helper function(what populateSubkeys uses).
The "queryKey" mock version will return error status other than "key doesn't exist".
This change isn't straightforward and some changes in the file "registry.cpp" should be done, so i postpone it for now.

So to clarify, I would remove the atLeastOneKeyExists logic and change the second test to check that the result is successful and that the returned keys only contain the input key.

I've committed the changes as you requested, please review.

Copy link
Member

@Smjert Smjert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you again!

We will address the other API errors separately, because I suspect that it might make more sense to actually "ignore" all the keys that have errors, and do not interrupt the loop, so as to provide the most complete view possible.
But this might cause side effects around that need to be carefully evaluated.

@directionless directionless modified the milestones: 5.2.0, 5.3.0 Jan 6, 2022
@Smjert Smjert merged commit 4ea9f2c into osquery:master Feb 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants