Skip to content

Commit

Permalink
Upgrade to Taskgraph 7 (#316)
Browse files Browse the repository at this point in the history
* Bump to taskgraph 7.0

* Taskgraph 7.0 changes
  • Loading branch information
bhearsum authored Dec 21, 2023
1 parent 2d4530d commit d7f725e
Show file tree
Hide file tree
Showing 46 changed files with 42 additions and 47 deletions.
2 changes: 1 addition & 1 deletion .taskcluster.yml
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,7 @@ tasks:
features:
taskclusterProxy: true

image: mozillareleases/taskgraph:decision-5483484ad45a3d27a0f5bd05f1c87d90e08df67a3713605d812b851a8a5bd854@sha256:ef132cc5741539f846a85bbe0cebc3c9ead30b8f24c1da46c55363f2170c3993
image: mozillareleases/taskgraph:decision-5483484ad45a3d27a0f5bd05f1c87d90e08df67a3713605d812b851a8a5bd854@sha256:2dd667994aa13fccdcdcede85c570a5eb4a5247e42875a9e95a135ef774ee469
maxRunTime: 1800

command:
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ loader: taskgraph.loader.transform:loader

transforms:
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ loader: taskgraph.loader.transform:loader
transforms:
- translations_taskgraph.transforms.from_datasets:per_dataset
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ loader: taskgraph.loader.transform:loader

transforms:
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ loader: taskgraph.loader.transform:loader
transforms:
- translations_taskgraph.transforms.from_datasets:per_dataset
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ loader: taskgraph.loader.transform:loader

transforms:
- translations_taskgraph.transforms.from_datasets:mono
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ loader: taskgraph.loader.transform:loader
transforms:
- taskgraph.transforms.task_context
- taskgraph.transforms.from_deps
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ loader: taskgraph.loader.transform:loader
transforms:
- taskgraph.transforms.from_deps
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ loader: taskgraph.loader.transform:loader
transforms:
- taskgraph.transforms.from_deps
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ loader: taskgraph.loader.transform:loader
transforms:
- translations_taskgraph.transforms.from_datasets:per_dataset
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ transforms:
- translations_taskgraph.transforms.from_datasets:per_dataset
- translations_taskgraph.transforms.worker_env
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ transforms:
- translations_taskgraph.transforms.worker_env
- taskgraph.transforms.from_deps
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ transforms:
- taskgraph.transforms.task_context
- translations_taskgraph.transforms.cast_to
- taskgraph.transforms.chunking
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ loader: taskgraph.loader.transform:loader

transforms:
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ transforms:
- taskgraph.transforms.task_context
- translations_taskgraph.transforms.cast_to
- taskgraph.transforms.chunking
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ loader: taskgraph.loader.transform:loader

transforms:
- translations_taskgraph.transforms.fetch:transforms
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- taskgraph.transforms.task:transforms

task-defaults:
Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ transforms:
- translations_taskgraph.transforms.marian_args:transforms
- translations_taskgraph.transforms.worker_env
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ loader: taskgraph.loader.transform:loader
transforms:
- translations_taskgraph.transforms.find_upstreams:by_locales
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ loader: taskgraph.loader.transform:loader
transforms:
- translations_taskgraph.transforms.find_upstreams:by_locales
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ loader: taskgraph.loader.transform:loader
transforms:
- taskgraph.transforms.task_context
- translations_taskgraph.transforms.find_upstreams:mono
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ loader: taskgraph.loader.transform:loader

transforms:
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ loader: taskgraph.loader.transform:loader

transforms:
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ loader: taskgraph.loader.transform:loader
transforms:
- translations_taskgraph.transforms.worker_env
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ loader: taskgraph.loader.transform:loader

transforms:
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ loader: taskgraph.loader.transform:loader

transforms:
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ loader: taskgraph.loader.transform:loader

transforms:
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ loader: taskgraph.loader.transform:loader

transforms:
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- taskgraph.transforms.task:transforms

task-defaults:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ kind-dependencies:
- fetch

transforms:
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ transforms:
- translations_taskgraph.transforms.marian_args:transforms
- translations_taskgraph.transforms.worker_env
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ transforms:
- translations_taskgraph.transforms.marian_args:transforms
- translations_taskgraph.transforms.worker_env
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ transforms:
- taskgraph.transforms.task_context
- translations_taskgraph.transforms.cast_to
- taskgraph.transforms.chunking
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ loader: taskgraph.loader.transform:loader

transforms:
- taskgraph.transforms.task_context
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ transforms:
- translations_taskgraph.transforms.cast_to
- taskgraph.transforms.chunking
- taskgraph.transforms.from_deps
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ transforms:
- translations_taskgraph.transforms.cast_to
- taskgraph.transforms.chunking
- taskgraph.transforms.from_deps
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ transforms:
- taskgraph.transforms.task_context
- translations_taskgraph.transforms.cast_to
- taskgraph.transforms.chunking
- taskgraph.transforms.job:transforms
- taskgraph.transforms.run:transforms
- translations_taskgraph.transforms.cached_tasks:transforms
- taskgraph.transforms.task:transforms

Expand Down
2 changes: 1 addition & 1 deletion taskcluster/requirements.in
Original file line number Diff line number Diff line change
@@ -1 +1 @@
taskcluster-taskgraph>=6.2.0
taskcluster-taskgraph>=7
11 changes: 3 additions & 8 deletions taskcluster/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -261,12 +261,7 @@ requests==2.28.2 \
--hash=sha256:98b1b2782e3c6c4904938b84c0eb932721069dfdb9134313beff7c83c2df24bf
# via
# cookiecutter
# requests-unixsocket
# taskcluster-taskgraph
requests-unixsocket==0.3.0 \
--hash=sha256:28304283ea9357d45fff58ad5b11e47708cfbf5806817aa59b2a363228ee971e \
--hash=sha256:c685c680f0809e1b2955339b1e5afc3c0022b3066f4f7eb343f43a6065fc0e5d
# via taskcluster-taskgraph
six==1.16.0 \
--hash=sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926 \
--hash=sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254
Expand All @@ -275,9 +270,9 @@ slugid==2.0.0 \
--hash=sha256:a950d98b72691178bdd4d6c52743c4a2aa039207cf7a97d71060a111ff9ba297 \
--hash=sha256:aec8b0e01c4ad32e38e12d609eab3ec912fd129aaf6b2ded0199b56a5f8fd67c
# via taskcluster-taskgraph
taskcluster-taskgraph==6.2.0 \
--hash=sha256:101c0ac0ce26a90c05a74b4f296424425409ffd4d89bbc4daf8bd4c24bc54c22 \
--hash=sha256:4e589bdaa8caac37c0bbd598b274eea7968380dbff396ce0e00bfbddcd34088f
taskcluster-taskgraph==7.0.2 \
--hash=sha256:d72bc5c8a63a8817e2c17d23e4dd5f197af71218388db4aff036502bf7ff84bc \
--hash=sha256:dcfb4edefcbd8090adb51e9db09d2ecc18f39ae7ad6e0b083aa839afc5bd58d1
# via -r requirements.in
taskcluster-urls==13.0.1 \
--hash=sha256:5e25e7e6818e8877178b175ff43d2e6548afad72694aa125f404a7329ece0973 \
Expand Down
4 changes: 2 additions & 2 deletions taskcluster/translations_taskgraph/target_tasks.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from taskgraph.target_tasks import _target_task
from taskgraph.target_tasks import register_target_task


@_target_task("train-target-tasks")
@register_target_task("train-target-tasks")
def train_target_tasks(full_task_graph, parameters, graph_config):
training_config = parameters["training_config"]
stage = training_config["target-stage"]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from taskgraph import files_changed
from taskgraph.transforms.base import TransformSequence

KIND_DIR = Path(__file__).parent.parent.parent / "ci"
KIND_DIR = Path(__file__).parent.parent.parent / "kinds"

# Kinds are slightly special - there are some kinds that don't affect the pipeline,
# and changing them shouldn't force the pipeline to run.
Expand Down

0 comments on commit d7f725e

Please sign in to comment.