Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use total_size within watchdog on Windows #7157

Merged
merged 2 commits into from
Jun 22, 2021
Merged

Conversation

theopolis
Copy link
Member

@theopolis theopolis commented Jun 12, 2021

See #6537, we should at least be using PrivateBytes for our definition of resident_size.

Update: based on discussion here, we are choosing to address the issue by using total_size within the watchdog logic.

@Smjert
Copy link
Member

Smjert commented Jun 12, 2021

Shouldn't we just use the total_size column in the watchdog instead (at least for now on Windows)?

resident_size is the RSS on Linux which are the active pages in physical memory, so as a concept it sort of maps to the Working Set of Windows.

I believe the issue talks about Windows, but given that Private Bytes also accounts for paged out memory, we should also change the behavior on the other platforms for consistency.

On Linux you might need to sum other counters like VmSwap or similar (I only had a brief look at this).

@Smjert
Copy link
Member

Smjert commented Jun 12, 2021

Also, this is slightly tangential, but now that I see, we don't have a "1:1" mapping between Linux and Windows on total_size too, because on Linux we use the virtual address space size (VmSize) which is not the same as Windows's Private Bytes, but indeed maps to the size of Virtual Address space (which also includes shared memory for instance).

I'm not totally sure on Windows how you would take that though.

@theopolis
Copy link
Member Author

Right, perhaps we need a new column "private_size". Then we can switch back to using Working Set for Windows. I am unsure how to get the virtual size to accurately set "total_size".

The private size on Linux would be a summation of Private Dirty values within smaps. (My best guess at the most accurate representation.)

@Smjert
Copy link
Member

Smjert commented Jun 12, 2021

Right, perhaps we need a new column "private_size". Then we can switch back to using Working Set for Windows. I am unsure how to get the virtual size to accurately set "total_size".

I'm in favor of adding such a column, but what I was trying to say is that the expectation and definition of resident_size is the memory that is kept in physical memory, so setting it to the value of Private Bytes would be incorrect, because it also contains paged out memory.

Also given that we already have such value in the total_size column, wouldn't it be better, for this specific watchdog fix, to actually change the watchdog so that it looks at total_size instead of resident_size, on Windows?

And I mean, specifically changing here:

change.footprint = tryTo<long long>(r.at("resident_size")).takeOr(0LL);

and here:
{"parent", "user_time", "system_time", "resident_size"},

to get total_size.

@theopolis
Copy link
Member Author

Yeap, I agree

@theopolis theopolis changed the title Use PrivateUsage for windows resident_size Use total_size within watchdog on Windows Jun 13, 2021
@theopolis
Copy link
Member Author

Introducing a new private_size column is a bit complex at this point in time. Let's go with the half-measure band aid and update the watcher logic for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants