Skip to content

Releases: dmwm/WMCore

WMCore 2.0.1 central services production release

10 Mar 19:14
Compare
Choose a tag to compare

This release brings in many new features and feature changes to the microservices. It also includes a few bug fixes required for the DBSServer migration to the Go implementation. Lastly, usual code and documentation enhancements have been provided.

Release date: 10 March 2022.
Changes since release: 2.0.0.

Central services

Software stack

Features and/or feature changes

  • Update requirements.txt and PyPI build scripts for py3 stack (Erik Gough) #10976
  • Add parameters to rucio datasets (Jhonatan Amado) #10989
  • [MSTransferor] Send alerts to prometheus when a workflow fails to be processed. Rename notifyLargeData method. Fixes #10215 (Kenyi Hurtado) #10959
  • [MSUnmerged] Add rucioConMonErrors counter. (Todor Ivanov) #10993
  • [MSUnmerged] Simplify counters logic && Substitute rucioConMonErrors with rucioConMonStatus. (Todor Ivanov) #10993
  • [MSUnmerged] Swap NotMasterError with NotPrimaryError. (Todor Ivanov) #11016
  • [MSUnmerged] Add rse=ALL_DOCS to the MSUnmerged info REST API. (Todor Ivanov) #11025
  • [MSOutput] Make MSOutput Disk expression configurable (Alan Malta Rodrigues) #11018
  • [MSOutput] MSOutput: read RelVal output data policy from configuration (Alan Malta Rodrigues) #11024

Bug Fixes

  • Fix a few DBS3Reader APIs, likely unused ones (Alan Malta Rodrigues) #10990
  • [MSUnmerged] Remove files from mongoProjection (Todor Ivanov) #10987
  • [MSUnmerged] Allow MongoDB to be instantiated without mongomock library (Alan Malta Rodrigues) #11023
  • [MSUnmerged] Reorder RSE's status and record age checks from RucioConMon. (Todor Ivanov) #11029
  • [MSTransferor] Fix MSTransferor typo calling getName method (Alan Malta Rodrigues) #11031
  • [MSTransferor] MSTransferor: fix positional and kwargs passed to sendAlert API (Alan Malta Rodrigues) #11032

Enhancements

  • Update mocked DBS APIs and data (Alan Malta Rodrigues) #10990
  • Remove leftover requirements_py3.txt; update contribution doc (Alan Malta Rodrigues) #10997
  • fix uninitialized templates variable (Dirk Hufnagel) #11004
  • Add mockMongoDB to MSUnmerged and MongoDB Class init options (Todor Ivanov) #11007
  • Enhancements to the GH issues and PRs workflows (Alan Malta Rodrigues) #11021

WMAgent

Features and/or feature changes

  • Add parameters to rucio datasets in RucioInjector (Jhonatan Amado) #10989
  • Add lifetime to RucioContainers shipping to disk sites (Jhonatan Amado) #10950
  • make sure proxy is 2048 bit. Fix #11005 (belforte) #11006
  • Explicitly create proxies with 2048 bits (Alan Malta Rodrigues) #11008

Bug Fixes

  • Fix a few DBS3Reader APIs, likely unused ones (Alan Malta Rodrigues) #10990

Enhancements

  • Update mocked DBS APIs and data (Alan Malta Rodrigues) #10990
  • Enhancements to the GH issues and PRs workflows (Alan Malta Rodrigues) #11021

WMCore 2.0.0 central services production release

07 Feb 12:37
Compare
Choose a tag to compare

This is our first WMCore release which no longer supports Python2 stack, and we will keep gradually removing python2 from our baseline code (including the future library).
This release has a major bug fix for MSUnmerged as well as other bug-fixes and enhancements to the overall system.

Release date: 8 February 2022.
Changes since release: 1.5.7.

Central services

Software stack

  • Update Python3 pycurl library from 7.43.0.3 to 7.43.0.6 (Alan Malta Rodrigues) cms-sw/cmsdist#7600

Features and/or feature changes

  • Aggregate DBS results from Go server (Valentin Kuznetsov) #10963

Bug Fixes

  • Fix MSUnmerged logic and MSUnmergedRSE format to account for deletion errors. (Todor Ivanov) #10687

Enhancements

  • Check that sdir exists on a file system (Valentin Kuznetsov) #10938
  • Added MSUnmerged to the WM schematic (Alan Malta Rodrigues) #10974
  • Update diagrams and contribution documentation for end of support for Py2 (Alan Malta Rodrigues) #10968

WMAgent

Features and/or feature changes

Bug Fixes

  • DBS3Upload: check error message from the response body attribute (Alan Malta Rodrigues) #10961

Enhancements

  • Update deploy-wmagent for 1.5.7.patch1; remove deprecated py2 option (Alan Malta Rodrigues) #10966
  • Update error message for jobs killed from site state transition (Alan Malta Rodrigues) #10958

Python3 related changes

WMCore 1.5.7.patch1 WMAgent production release

27 Jan 16:23
Compare
Choose a tag to compare

This is the last WMAgent stable release supporting both Python2 and Python3. This release brings in some important new features and bug fixes, in addition to the continuous minor enhancements. It also comes with a few very important software stack changes.

Release date: 27 January 2022.
Changes since release: 1.5.4.

WMAgent

Software stack

  • Upgrade MariaDB spec from 10.1.21 to 10.6.5 (Alan Malta Rodrigues) #7205
  • Update pylint version required from 2.6.0 to 2.7.0 (Alan Malta Rodrigues) #10913
  • Bump dbs3-client to 4.0.5 (Alan Malta Rodrigues) #10931
  • Bump dbs3-pycurl to 3.17.7 (Alan Malta Rodrigues) #10931
  • Update CMSCouchapp version for python3 stack (Alan Malta Rodrigues) #10910
  • Update python3 pip to 20.3.3 and setuptools to 51.3.3 (Alan Malta Rodrigues) #7418

Features and/or feature changes

  • Remove trailing slash from DBS URL (Alan Malta Rodrigues) #10930
  • Add two new CMSSW exit code: 8034 and 8501 (Stefano Belforte) #10850
  • Remove PhEDEx library and any other use of PhEDEx (Alan Malta Rodrigues) #10856
  • Stop using password for MariaDB localhost users (Alan Malta Rodrigues) #10932
  • Use siteLocalConfig to get LogCollect stage out params (germanfgv) #10905
  • Add --deep option to getReplicaInfoForBlocks. Add slicing to deleteBlocks (germanfgv) #10902
  • Set request_GPUs classad according to RequiresGPU (Alan Malta Rodrigues) #10849
  • Support different architectures during job submission and runtime (Alan Malta Rodrigues) #10853

Bug Fixes

  • DBS3Upload: check error message from the response body attribute #10961
  • Properly update workqueue elements from JobUpdater (Alan Malta Rodrigues) #10944
  • Skip ErrorHandler cycle if agent configuration fails to be retrieved - take2 (Alan Malta Rodrigues) #10852
  • Remove leading ../../ from logArchive tarballs (Alan Malta Rodrigues) #10868
  • Check whether WMStep object has gpu attribute (Alan Malta Rodrigues) #10866
  • Use the right COMP scramArch for PowerPC workflows (Alan Malta Rodrigues) #10871
  • CredentialProxy - convert to bytes the input of hashlib.sha1() (Dario Mapelli) #10918
  • REST/Server.py - encode to bytes input of hashlib.sha1 (Dario Mapelli) #10917
  • Fix path for Py2/Py3 WMAgent logs (Alan Malta Rodrigues) #10897

Enhancements

  • Update error message for jobs killed from site state transition #10958
  • Create the MariaDB wmcore unittest database with the current unix user (Alan Malta Rodrigues) #10934
  • fix DbsUrl in test modules and templates (Alan Malta Rodrigues) #10930
  • Fix python version in subprocess call for inject-test-wfs (Alan Malta Rodrigues) #10847
  • Update Rucio Int url (Alan Malta Rodrigues) #10855
  • Update job priority only for processing and production task types (Kenyi Hurtado) #10812
  • Updating workflow json templates; adding working PY3 templates (Alan Malta Rodrigues) #10761
  • Update wmagent-deploy script to deploy Py3 WMAgent by default; update to cycle 1.5.4 (Alan Malta Rodrigues) #10859
  • Update campaigns used in our test setup (Alan Malta Rodrigues) #10859

WMCore 1.5.7 central services and WMAgent production release

18 Jan 20:58
Compare
Choose a tag to compare

This release brings in the latest dbs3-client version, compatible with both the Python and Golang server implementations.
The software stack also had a major MariaDB upgrade (from 10.1.21 to 10.6.5).
It also has a few bug fixes and improvements for central services and WMAgent.

Release date: 18 January 2021.
Changes since release: 1.5.6.

Central services

Software stack

  • Upgrade MariaDB spec from 10.1.21 to 10.6.5 (Alan Malta Rodrigues) #7205
  • Update pylint version required from 2.6.0 to 2.7.0 (Alan Malta Rodrigues) #10913
  • Bump dbs3-client to 4.0.5 (Alan Malta Rodrigues) #10931
  • Bump dbs3-pycurl to 3.17.7 (Alan Malta Rodrigues) #10931

Features and/or feature changes

  • Remove trailing slash from DBS URL (Alan Malta Rodrigues) #10930

Bug Fixes

  • [MSTransferor] Fix logic which drops input blocks to be transferred (Alan Malta Rodrigues) #10928
  • [MSUnmerged] REST APIs to fetch RSE data from MongoDB. (Todor Ivanov) #10923

Enhancements

  • fix DbsUrl in test modules and templates (Alan Malta Rodrigues) #10930

WMAgent

Features and/or feature changes

  • Remove trailing slash from DBS URL (Alan Malta Rodrigues) #10930
  • Stop using password for MariaDB localhost users (Alan Malta Rodrigues) #10932
  • Create the MariaDB wmcore unittest database with the current unix user (Alan Malta Rodrigues) #10934

Bug Fixes

  • Properly update workqueue elements from JobUpdater (Alan Malta Rodrigues) #10944

Enhancements

  • Update job priority only for processing and production task types, rather than every single subtask name, to reduce condor_qedit load. (Kenyi Hurtado) #10812

Python3 related changes

  • CredentialProxy - convert to bytes the input of hashlib.sha1() (Dario Mapelli) #10918
  • REST/Server.py - encode to bytes input of hashlib.sha1 (Dario Mapelli) #10917

WMCore 1.5.6 central services production release

18 Jan 20:56
Compare
Choose a tag to compare

First version of MSUnmerged with a MongoDB database backend.

Release date: 10 November 2021.
Changes since release: 1.5.5.

Central services

Software stack

  • Update CMSCouchapp version for python3 stack (Alan Malta Rodrigues) #10910

Features and/or feature changes

  • [MSUnmerged] Initial integration of MongoDB. (Todor Ivanov) #10895
  • [MSUnmerged] Add replicaset opotion to the MongoDB wrapper. (Todor Ivanov) #10908
  • [MSUnmerged] Adapt MSUnmerged to sync RSE data with MongoDB properly (Todor Ivanov) #10906

Bug Fixes

Enhancements

  • Update request state diagram with archival done by MS (Alan Malta Rodrigues) #10899

WMAgent

Features and/or feature changes

  • Use siteLocalConfig to get LogCollect stage out params (germanfgv) #10905
  • Add --deep option to getReplicaInfoForBlocks. Add slicing to deleteBlocks (germanfgv) #10902

Bug Fixes

Enhancements

Python3 related changes

  • Fix path for Py2/Py3 WMAgent logs (Alan Malta Rodrigues) #10897

WMCore 1.5.5 central services production release

10 Nov 11:21
Compare
Choose a tag to compare

This release no longer ships any of the PhEDEx libraries. The running-open and running-close statuses have been slightly redesigned.
Last but not least, some important bug fixes and enhancements have been made.

Release date: 10 November 2021.
Changes since release: 1.5.4.

Central services

Software stack

  • Update dbs3-client to 3.17.6 and dbs3-pycurl 3.17.4
  • Update python3 pip to 20.3.3 and setuptools to 51.3.3

Features and/or feature changes

  • add two new CMSSW exit code: 8034 and 8501 (Stefano Belforte) #10850
  • Remove PhEDEx library and any other use of PhEDEx (Alan Malta Rodrigues) #10856
  • [MSTransferor] Remove unused Unified/MSTransferor code (Alan Malta Rodrigues) #10876
  • Add validation to the SubRequestType parameter (Alan Malta Rodrigues) #10890
  • Refactored conditions for running-open/running-closed workflow statuses (Alan Malta Rodrigues) #10693
  • Temporary fix for the StatusChangeTasks ReqMgr2 thread (Alan Malta Rodrigues) #10693

Bug Fixes

  • [MSUnmerged] Remove check of RSE dump root_failed attr in MSUnmerged (Alan Malta Rodrigues) #10894

Enhancements

  • Fix python version in subprocess call for inject-test-wfs (Alan Malta Rodrigues) #10847

WMAgent

Features and/or feature changes

  • Set request_GPUs classad according to RequiresGPU (Alan Malta Rodrigues) #10849
  • Update Rucio Int url (Alan Malta Rodrigues) #10855
  • Support different architectures during job submission and runtime (Alan Malta Rodrigues) #10853

Bug Fixes

  • Skip ErrorHandler cycle if agent configuration fails to be retrieved - take2 (Alan Malta Rodrigues) #10852
  • Remove leading ../../ from logArchive tarballs (Alan Malta Rodrigues) #10868
  • Check whether WMStep object has gpu attribute (Alan Malta Rodrigues) #10866
  • Use the right COMP scramArch for PowerPC workflows (Alan Malta Rodrigues) #10871

Enhancements

  • Updating workflow json templates; adding working PY3 templates (Alan Malta Rodrigues) #10761
  • Update wmagent-deploy script to deploy Py3 WMAgent by default; update to cycle 1.5.4 (Alan Malta Rodrigues) #10859
  • update campaigns used in our test setup (Alan Malta Rodrigues) #10859

Python3 related changes

WMCore 1.5.4.patch2 WMAgent production release

12 Oct 16:29
Compare
Choose a tag to compare

This is the latest stable WMAgent branch, which already includes to patch releases on top of it.
The highlight of this release is the support to GPU and PowerPC workflows/jobs. In terms of the software stack,
it brings in a new oracle client version 19, and HTCondor python bindings version 9.2.0.

Release date: 12 October 2021.
Changes since release: 1.5.2.

WMAgent

Software stack

  • Oracle client version 19.11
  • Update Python3 requirements list with Sphinx and Jinja2 #10794
  • Update py3-dbs3-client/dbs3-pycurl to 3.16.2001 cms-sw/cmsdist#7384
  • New HTCondor specs from PyPi; update version to 9.2.0 cms-sw/cmsdist#7383

Features and/or feature changes

  • Support different architectures during job submission and runtime (Alan Malta Rodrigues) #10863
  • Update Rucio Int url (Alan Malta Rodrigues) #10855
  • Add two new CMSSW exit code: 8034 and 8501 (Stefano Belforte) #10850
  • Set request_GPUs classad according to RequiresGPU (Alan Malta Rodrigues) #10849
  • Propagate GPU requirements to job creation and submission (Alan Malta Rodrigues) #10811
  • Provide request type via job classad (Alan Malta Rodrigues) #10821
  • Permanently stop staging out log tarball to CASTOR (Alan Malta Rodrigues) #10712
  • CouchDB 3: Add new Error HTTP Status Codes to CMSCouch.py (phenomax) #10785

Bug Fixes

  • JobAccountant workaround for StepChain jobs with duplicate files - wmagent branch (Alan Malta Rodrigues) #10863
  • [ReqMgr2/Tier0] PhysicsSkim issue (Jhonatan Amado) #10809
  • Skip ErrorHandler cycle if agent configuration fails to be retrieved (Alan Malta Rodrigues) #10817 #10852
  • Fix missing parenthesis during PSet tweak (Kenyi Hurtado) #10771

Enhancements

  • Updating workflow json templates; adding working PY3 templates (Alan Malta Rodrigues) #10761
  • Fix python version in subprocess call for inject-test-wfs (Alan Malta Rodrigues) #10847
  • Define pylint message template in the pylintrc file (Alan Malta Rodrigues) #10776

Python3 related changes

  • [py2py3] modernize WMCore_t/REST_t/Api_t.py (Dario Mapelli) #10757
  • #10349 - JobSubmitter_t and JobSubmitterCaching_t - test (Dario Mapelli) #10608
  • #10371 check - fix JobCreator_t.py - test (Dario Mapelli) #10724
  • Modernization of leftover files - src (Dario Mapelli) #10768
  • RESTFormatter.to_string returns 'native str' (Dario Mapelli) #10753

WMCore 1.5.4 central services production release

05 Oct 07:33
Compare
Choose a tag to compare

Thin central production release providing mostly WMAgent-related changes.
This release brings in the ability to create and submit GPU jobs.

Release date: 5 October 2021.
Changes since release: 1.5.3.

Central services

Software stack

  • Oracle client version 19.11

Features and/or feature changes

Bug Fixes

  • [ReqMgr2/Tier0] PhysicsSkim issue (Jhonatan Amado) #10809

Enhancements

  • Add a step to change rucio instance during deployment. (Todor Ivanov) #10803

WMAgent

Features and/or feature changes

  • Propagate GPU requirements to job creation and submission (Alan Malta Rodrigues) #10811
  • Provide request type via job classad (Alan Malta Rodrigues) #10821

Bug Fixes

  • Skip ErrorHandler cycle if agent configuration fails to be retrieved (Alan Malta Rodrigues) #10817

Enhancements

  • Fix python version in subprocess call for inject-test-wfs (Alan Malta Rodrigues) #10847

Python3 related changes

  • [py2py3] modernize WMCore_t/REST_t/Api_t.py (Dario Mapelli) #10757

WMCore 1.5.3 central services production release

16 Sep 07:30
Compare
Choose a tag to compare

This release contains many feature changes and bug fixes. It's worth highlighting that support
to GPU workflows has been introduced for ReReco and TaskChain request specs. The remaining critical
issues with the unmerged microservice have been fixed and it's now fully functional.
Last but not least, the WMCore codebase is now fully dual stack (python2 and python3).

Release date: 16 September 2021.
Changes since release: 1.5.1.

Central services

Software stack

  • Update Python3 requirements list with Sphinx and Jinja2 (Alan Malta Rodrigues) #10794

Features and/or feature changes

  • [MSOutput]: only produce documents for workflows not yet in the database (Alan Malta Rodrigues) #10773
  • [ReqMgr2] Override factoryWorkloadConstruction in the Resubmission spec (Alan Malta Rodrigues) #10781
  • [ReqMgr2] Support GPU parameters at StdBase/ReReco; validation functions implemented (Alan Malta Rodrigues) #10799
  • [ReqMgr2] Add support to GPUs in TaskChain spec (Alan Malta Rodrigues) #10805

Bug Fixes

  • [MSOutput] Skip inexistent containers in the Tape data placement (Alan Malta Rodrigues) #10717
  • [MSUnmerged] Make rucioConMon parameter mandatory (Alan Malta Rodrigues) #10715
  • [MSUnmerged] Split initial data fetching steps && Return from handled exceptions. (Todor Ivanov) #10762
  • [MSTransferor] Fix undefined variable in MSTransferor (Alan Malta Rodrigues) #10769
  • [ReqMgr2] Fix spec validation for Multicore/Memory/Streams/TpE (Alan Malta Rodrigues) #10781
  • [ReqMgr2] Keep Resubmission spec and json data consistent (Alan Malta Rodrigues) #10789
  • [ReqMgr2] Fix Multicore/EventStreams override in Resubmission of StepChains (Alan Malta Rodrigues) #10795
  • [MSUnmerged] Delete file lists in slices. (Todor Ivanov) #10792

Enhancements

  • Define pylint message template in the pylintrc file (Alan Malta Rodrigues) #10776

WMAgent

Features and/or feature changes

  • Stop staging out log tarball to CASTOR (Alan Malta Rodrigues) #10712
  • Created new job wrapper script for Python3 environment (Alan Malta Rodrigues) #10726
  • CouchDB 3: Add new Error HTTP Status Codes to CMSCouch.py (phenomax) #10785

Bug Fixes

  • Encode a few job classads from unicode to bytes (Alan Malta Rodrigues) #10711
  • Revert CouchDB replication port forward (Alan Malta Rodrigues) #10727
  • Fix issue with merge process and output labels (Merged and MergedError) (Kenyi Hurtado) #10593
  • Define funcName variable in the right place (Alan Malta Rodrigues) #10744
  • Decode byte string to unicode for json.dumps (Alan Malta Rodrigues) #10740
  • Provide standard/unicode string to the PSet handlePileup method (Alan Malta Rodrigues) #10740
  • Cast performance values to float in MathAlgos (Alan Malta Rodrigues) #10750
  • Fix job priority at submission time, taking into account maxTaskPriority, as it was originally. (Kenyi Hurtado) #10760
  • Fix missing parenthesis (Kenyi Hurtado) #10771

Enhancements

  • Add new attribute for T0 Container rules (Jhonatan Amado) #10738
  • Define cms_type=int for rucio rules created against Rucio Int (Alan Malta Rodrigues) #10756
  • Convert a WorkQueueManagerCleaner log info to debug (Alan Malta Rodrigues) #10755

Python3 related changes

  • #10349 - JobSubmitter_t and JobSubmitterCaching_t - test (Dario Mapelli) #10608
  • Properly deal with strings location in the File module (Alan Malta Rodrigues) #10723
  • #10371 check - fix JobCreator_t.py - test (Dario Mapelli) #10724
  • Load workload pickle object during runtime in binary mode (Alan Malta Rodrigues) #10729
  • Create job package with pickle protocol=4 for Python3 agents (Alan Malta Rodrigues) #10731
  • modernize use of pickle.load/dump - src (Dario Mapelli) #10742
  • modernize use of mylist.sort() in REST/Server.py (Dario Mapelli) #10743
  • Implement py3 sorting for JobSplitting (Erik Gough) #10720
  • Pass byte string to subprocess.stdin in Scram runtime (Alan Malta Rodrigues) #10736
  • Fix Services_t/pycurlFileUpload_t and WebTools/REST_t - src (Dario Mapelli) #10650
  • Modernize DQMUpload.py (Dario Mapelli) #9934
  • Modernize scripts that rely on urllib (Dario Mapelli) #10754
  • More job runtime places to encode pickledarguments to bytes (Alan Malta Rodrigues) #10766
  • Modernization of leftover files - src (Dario Mapelli) #10768
  • RESTFormatter.to_string returns 'native str' (Dario Mapelli) #10753

WMCore 1.5.2 WMAgent production release

21 Aug 07:05
Compare
Choose a tag to compare

This is the first WMAgent release with full support to python3. It also brings in a major refactoring made for the job PSet tweaking, where a new dependency on cmssw-wm-tools has been introduced, fully decoupling WMAgent python from CMSSW.

Release date: 21 August 2021.
Changes since release: 1.5.1.

Central services

Software stack

Features and/or feature changes

  • Split initial data fetching steps && Return from handled exceptions. (Todor Ivanov) #10762

Bug Fixes

  • MSOutput: skip inexistent containers in the Tape data placement (Alan Malta Rodrigues) #10717
  • MSUnmerged: make rucioConMon parameter mandatory (Alan Malta Rodrigues) #10715

Enhancements

WMAgent

Features and/or feature changes

  • Add new attribute for T0 Container rules (Jhonatan Amado) #10738
  • Permanently stop staging out log tarball to CASTOR (Alan Malta Rodrigues) #10712

Bug Fixes

  • Fix issue with merge process and output labels (Merged and MergedError) (Kenyi Hurtado) #10593 #10744
  • Revert CouchDB replication port forward (Alan Malta Rodrigues) #10727
  • Encode a few job classads from unicode to bytes (Alan Malta Rodrigues) #10711
  • Fix job priority at submission time, taking into account maxTaskPriority, as it was originally. (Kenyi Hurtado) #10760

Enhancements

  • Define cms_type=int for rucio rules created against Rucio Int (Alan Malta Rodrigues) #10756
  • Convert a WorkQueueManagerCleaner log info to debug (Alan Malta Rodrigues) #10755

Python3 related changes

  • [py2py3] modernize DQMUpload.py (Dario Mapelli) #9934
  • [py2py3] fix Services_t/pycurlFileUpload_t and WebTools/REST_t - src (Dario Mapelli) #10650
  • PY3: Cast performance values to float in MathAlgos (Alan Malta Rodrigues) #10750
  • PY3: Pass byte string to subprocess.stdin in Scram runtime (Alan Malta Rodrigues) #10736
  • PY3: Provide standard/unicode string to the PSet handlePileup method (Alan Malta Rodrigues) #10740
  • PY3: Decode byte string to unicode for json.dumps (Alan Malta Rodrigues) #10740
  • PY3: Implement py3 sorting for JobSplitting (Erik Gough) #10720
  • [py2py3] modernize use of mylist.sort() in REST/Server.py (Dario Mapelli) #10743
  • [py2py3] modernize use of pickle.load/dump - src (Dario Mapelli) #10742
  • PY3: Created new job wrapper script for Python3 environment (Alan Malta Rodrigues) #10726
  • PY3: Create job package with pickle protocol=4 for Python3 agents (Alan Malta Rodrigues) #10731
  • PY3: Load workload pickle object during runtime in binary mode (Alan Malta Rodrigues) #10729
  • [py2py3] Properly deal with strings location in the File module (Alan Malta Rodrigues) #10723
  • [py2py3] modernize scripts that rely on urllib (Dario Mapelli) #10754
  • [py2py3] More job runtime places to encode pickledarguments to bytes (Alan Malta Rodrigues) #10766

Important changes from the latest production WMAgent release (1.4.7)

  • Many many python3 modernization changes
  • DBFormatter formatDict/formatList to native string instead of bytes (Alan Malta Rodrigues) #10639
  • Create workload sandbox efficiently (Alan Malta Rodrigues) #10612
  • Decouple CMSSW and WMCore python versions from PSet tweak (khurtado) #10179
  • Retrieve only a summary of the condor jobs in DrainStatusPoller (Alan Malta Rodrigues) #10458
  • Only create Rucio connection in PileupFetcher if there is a pileup dataset (Alan Malta Rodrigues) #10470