feat(pool): Expose utilization and activity metrics from the pool #4658
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
iambriccardo
changed the title
Dav1dde
reviewed
Apr 9, 2025
Dav1dde
approved these changes
Apr 10, 2025
relay-threading/src/metrics.rs
Outdated
| /// | ||
| /// Note that this metric is collected and updated for each thread when the main future is polled, | ||
| /// thus if no work is being done, it will not be updated. | ||
| pub fn utilization(&self) -> u8 { |
There was a problem hiding this comment.
Maybe call that cpu_utilization and expose a total_utilization which does max(activity, cpu_utilization) which can be used in the auto scaler thingy?
There was a problem hiding this comment.
I am thinking of the edge case where we have 100% activity since we are doing lots of I/O and 0% max utilization (assuming no cpu bound work is being done). Do we really want to scale in this case? Scaling might not help.
There was a problem hiding this comment.
Up to you, without thinking too much about I would have said yes, because scaling is probably the safer option.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR updates the metrics emitted by the async pool:
utilization-> it now calculates how busy the pool is by checking the CPU-bound work it is doing.activity-> it checks how active the pool is (this was the previous utilization metric). The activity determines how many futures are currently being driven by the pool w.r.t. the total capacity of futures it can hold.To make this possible I have reused the
ServiceMonitorwhich has been renamed toTimedFutureand it has been made agnostic of any service related internals. Additionally, the future now relies on aTimedFutureReceiverto emit lifecycle and state changes.Closes: https://github.com/getsentry/team-ingest/issues/689