v0.49.0

This release includes an improvement to integration plugin connection handling, the addition of detectors in more log messages, task termination control improvements in the executor, improvements to Bookkeeping and Kafka clients, and other fixes.

• Integration client connections:

  • [core] Use connectivity.Shutdown for plugin client connection check

• Detectors in log messages:

  • [core] Output detector field to InfoLogger
  • [core] Make sure deployment error is forwarded + print detectors
  • [executor] Output detector field to InfoLogger
  • [executor] Attach detector to task PID event

• Bookkeeping improvements:

  • [OCTRL-695] AliECS Bookkeeping plugin passes the raw runtype as string for run details.
  • [OCTRL-677] AliECS Bookkeeping plugin passes odc_topology_fullname to BK for run details.

• Miscellaneous:

  • [core] Improved output for "no resource demands for descriptor" issue
  • [core] Warn before deployment if there are claimable or orphan tasks
  • [core] Kafka: Remove the deprecated PublishUpdate callback
  • [OCTRL-691][core] add a before_activity_start topic in kafka
  • [OCTRL-691][core] add a before_activity_start topic in kafka

v0.48.2

This release patches a critical issue in plugin interaction caused by a recent bump in gRPC. It also brings fixes to Bookkeeping timestamp fields.

• [core] Do not check for gRPC client connection before plugin calls
• [core] O2 End and TRG End timestamps can be set as late as DESTROY if the info was not available before.
• [OCTRL-690] O2 End Time timestamp is correctly set in case of early run failure.
• [OCTRL-690] O2 End time timestamp is only set once per run.

v0.48.1

This patch release fixes a critical issue with incoming events in the DCS integration client.

• Revert "[core] Do not loop forever on DCS global stream bad response"

v0.48.0

This release greatly improves log output, associating an environment ID with InfoLogger messages wherever possible. It also brings important improvements to the Bookkeeping component, as well as important fixes throughout the core, including a change that allows hook calls to share variables within the same transition. Finally, it includes a cleanup and update of all dependencies and bumps the compiler version to Go 1.18 (up from 1.14), bringing many improvements including generics.

• Log output:

  • [common] DeviceEvents can now carry Labels, including environmentId
  • [core] Add envId to all servent/envman/commandqueue messages
  • [core] Add environment ID to log messages in scheduler and task.Task
  • [core] Add missing environment ID to TRG plugin IL output
  • [core] Even more environmentId in InfoLogger output
  • [executor] Include environment ID in log messages
  • [executor] Split out actions.go from handlers file
  • [executor] Carry environment ID in DeviceEvents and output

• Bookkeeping:

  • [core] The number of warning messages of the BK plugin is reduced.
  • [core] AliECS Bookkeeping plugin returns the requested number of EPNs for a run.
  • [core] UpdateRun is adjusted to match the one from Bookkeeping API.
  • [core] go.mod and go.sum files point to Bookkeeping@main.
  • [OCTRL-676] Bookkeeping stores the EPN Workflow script in the log entry.
  • [OCTRL-657][OCTRL-677] Lhc_period and odc_topology_fullname are added to the BK run details with UpdateRun.
  • [OCTRL-687] Trigger values are set and passed to BK depending on trg_enabled and trg_global_enabled.

• Build improvements:

  • [build] go mod tidy - finally successful, including bump to Go 1.18
  • [build] Update all dependencies
  • [build] Bump Go compiler in GH build action
  • [build] Bump to v0.48.0
  • [common] Yank out spf13/viper remote package and all its deps
  • [core] Correct version of protobuf package

• Miscellaneous:

  • [core] Add total host count to EnvironmentInfo
  • [core] AliECS plugin calls instanciate the varStack within their call to access runtime variable from other plugins.
  • [core] Improve performance of o2-aliecs-shmcleaner
  • [core] Transition to ERROR in case of failed deployment or configuration, before tearing down.
  • [core] Do not loop forever on DCS global stream bad response

v0.47.0

This release adds a variety of bug fixes and improvements, mostly focused on integration, including a fix for a high impact deployment failure.

• Bookkeeping and TRG timestamps:

  • [core] allow to override the start of run time in GRP objects
  • [core] AliECS trigger plugin returns the correct TRG Stop timestamp on RunStop and Cleanup.
  • [core] Trigger timestamps are cleared before run start.

• Integration plugins:

  • [core] Add support for pdp_force_run_as_raw variable in ODC
  • [core] DCS override "ddl_list: default" with _default_ddl_list
  • [core] Fix nil pointer dereference in DCS plugin

• Miscellaneous:

  • [coconut] Show task claimability
  • [core] Refactor task.Class into its own package + thread safety
  • [core] Restrict task claimability criterion to avoid trash
  • [core] Add task claimability property to API
  • [executor] Fix gRPC socket GC issue that caused failed deployments
  • Revert "[task] Add a random sleep of up to 10s before each task for debugging purposes"

v0.46.0

This release adds a variety of bug fixes and improvements, mostly focused on integration, as well as an update of the run types system.

• Run types:

  • [common] COSMIC->COSMICS plus add SYNTHETIC to run types
  • [core] Special case for COSMICS run type in DCS

• ODC integration:

  • [core] ODC GEN_TOPO_PARTITION parameter is no more
  • [core] ODC add RUNTYPE and GEN_TOPO_CACHE_HASH to script generator
  • [core] Push orbit-reset-time to all FairMQ devices on FLP and EPN

• Bookkeeping:

  • [OCTRL-680] Bookkeeping run O2End and TrgEnd times are only updated once.
  • [OCTRL-680] Bookkeeping plugin UpdateRun function checks whether it is a start or a stop call.
  • [OCTRL-680] A warning is issued if there is no run number found in the pending runs of the BK plugin cache.
  • [OCTRL-679] O2 times and Trg times should be the correct ones for BK run details.

v0.45.3

This releases fixes an issue with the bookkeeping plugin:

• [O2B-560] Empty time objects are handled correctly and pass -1 to Bookkeeping if empty.

v0.45.2

This release improves Bookkeeping integration & error handling, Kafka logging, and includes some minor bugfixes.

• Bookkeeping

  • [core] Error are handled better for the plugin.
  • [core] Bookkeeping plugin run errors are better handled in case of DESTROY or GO_ERROR transition.
  • [O2B-560] Variable naming is more consistent, and parsing errors do not result in exit.
  • [O2B-560] AliECS Bookkeeping plugin is looking at the branch main from Bookkeeping.
  • [O2B-560] UpdateRun passes test as default run quality to Bookkeeping.
  • [O2B-560] Run details are updated successfully in Bookkeeping.
  • [O2B-560] A Bookkeeping run is created as early as possible with as many parameters as available, and updated afterwards with the necessary information.
  • [core][O2B-560] Bookkeeping plugin is ready to return required fields for Run information in Bookkeeping.
  • [core][bookkeeping-plugin] Some debug fields are added and unused Run_Quality is corrected.

• Kafka

  • Kafka: more logs, require ACK from all brokers

• Bugfixes / improvements

  • [repos] Local repo accessibility check before PlainOpen
  • [config] Empty repos ssh key by default
  • [dcs] More desciptive messages for some DCS failures
  • [core] taskmanager: Channel can remain open until deletion
  • [core] taskmanager: Channel should be closed by sender

v0.45.1

This release extends the JIT functionality for DPL workflow generation and catches an unhandled error on the task manager.

• JIT:

  • [jit] Also check QC version for jit hash
  • [jit] Include env vars in resolved command trace
  • [jit] Use JIT env vars during generation
  • [jit] Add option for DPL commands contained in files

• taskmanager:

  • [core] taskmanager: Don't close a closed channel

v0.45.0

This release adds improved debug facilities and fixes to the core and executor, as well as improvements to the EOR sequence and error handling thereof.

• EOR sequence:

  • [auto-stop] Transition into ERROR if stop fails
  • [OCTRL-667][core] Write down EOR time also on RUNNING->ERROR

• Debug:

  • [core] Fix cmdq.commit debug message
  • [core] Improved debug output in commandqueue and task.Manager
  • [core] Ensure an executor failure reports a hostname
  • [core] Add machine counter to scheduler.go offers handler
  • [executor] Fix command output in debug messages from controllabletask