Optimize service event query #284
Open
+131
−45
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
himdel
reviewed
Nov 25, 2025
metrics_utility/library/collectors/controller/main_jobevent_service.py
Outdated
Show resolved
Hide resolved
MilanPospisil
force-pushed
the
milan_optimize_queries2
branch
from
88b699b to
5abd68f
Compare
MilanPospisil
force-pushed
the
milan_optimize_queries2
branch
from
42a2621 to
2a2f927
Compare
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[AAP-59250]
Speedup for main_jobevent_service and main_jobevent in library using LATERAL.
main_jobevent service now reduces number of pairs in jobs:
This will contain all jobs finished in since-until, which can be tens of thousands of jobs!
This can easily bloat. We introduce new approach. Since we can not join on jobs according to job_created, because in order to work with partitions prunning on job_created (job_created is houlrly partition in event table), we need to do very clear literal explicit filter on event table (thus WHERE with contstant job_created).
Improvements:
job_created is simplified - awx uses hourly partitions, so we split into hourly job_created and put it into set, this keeps max hundreds of hours
Then we also makes groups of nearby hours to reduce it even more, for example 2025-06-01 10:00:00 and 2025-06-01 11:00:00 will be concatenated and the resulted range filter will be job_created => min AND job_created < max.
Also removed cyclic import.