Skip to content

Aim higher than target cpu usage #4750

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: oss-fuzz
Choose a base branch
from
Open

Aim higher than target cpu usage #4750

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 1 addition & 6 deletions src/clusterfuzz/_internal/cron/schedule_fuzz.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,6 @@
# TODO(metzman): Actually implement this.
CPUS_PER_FUZZ_JOB = 2

# Pretend like our CPU limit is 3% higher than it actually is so that we use the
# full CPU capacity even when scheduling is slow.
CPU_BUFFER_MULTIPLIER = 1.03


def _get_quotas(creds, project, region):
compute = discovery.build('compute', 'v1', credentials=creds)
Expand Down Expand Up @@ -110,7 +106,7 @@ def get_cpu_usage(creds, project: str, region: str) -> int:
# We need this because us-central1 and us-east4 have different numbers of
# cores alloted to us in their quota. Treat them the same to simplify things.
limit = quota['limit']
limit = min(limit, 100_000)
limit = min(limit, 110_000)
return limit, quota['usage']


Expand Down Expand Up @@ -265,7 +261,6 @@ def get_available_cpus(project: str, regions: List[str]) -> int:
# Only worry about queueing build up if we are above 95% utilization.
count_args = ((project, region) for region in regions)
with multiprocessing.Pool(2) as pool:
target *= CPU_BUFFER_MULTIPLIER
# These calls are extremely slow (about 30 minutes total).
result = pool.starmap_async( # pylint: disable=no-member
batch.count_queued_or_scheduled_tasks, count_args)
Expand Down
Loading