Releases: dmwm/WMCore
Releases · dmwm/WMCore
WMCore 1.5.1 central services production release
This is the second stable Python3 release of the central services.
It also brings in a new MicroService - called MSUnmerged - to clean up orphan/leftover files from the sites unmerged area.
Last but not least, this release also brings in some relevant feature changes and bug-fixes for other microservices.
Release date: 3 August 2021.
Changes since release: 1.5.0.
Central services
Software stack
- [k8s] Manifests created for MSUnmerged: dmwm/CMSKubernetes#604 dmwm/CMSKubernetes#612 dmwm/CMSKubernetes#665 dmwm/CMSKubernetes#667 dmwm/CMSKubernetes#669 dmwm/CMSKubernetes#671 dmwm/CMSKubernetes#697
- [k8s] New cron proxy Docker container for MSUnmerged: dmwm/CMSKubernetes#672 dmwm/CMSKubernetes#682
- [k8s] New Docker image for MSUnmerged + gfal2: dmwm/CMSKubernetes#615 dmwm/CMSKubernetes#668
Features and/or feature changes
- Add RucioConMon API to WMCore.Services. (Todor Ivanov) #10640 #10689
- [MSUnmerged] Add logic for cutting the path depth as provided by RucioConMon (Todor Ivanov) #10666
- [MSUnmerged] Add lfn2pfn resolve logic. (Todor Ivanov) #10671 #10679
- [MSUnmerged] Limit PFNList entries logged (Todor Ivanov) #10685
- [MSUnmerged] Fix unnecessary traceback dumps. (Todor Ivanov) #10691
- [MSUnmerged] support detail and rse_type in the RESTful APIs (Alan Malta Rodrigues) #10700
- [MSUnmerged] Introduce include and exclude directory filters. (Todor Ivanov) #10706
- [MSOutput] Keep all relval output datatiers on disk by bypassing all MSOutput configurations (haozturk) #10669
- [MSTransferor] Provide a configurable weight value for MSTransferor Rucio rules (Alan Malta Rodrigues) #10696
Bug Fixes
- Fix object type passed to time.mktime (Alan Malta Rodrigues) #10662
- [MS] Import micro services on demand in MSManager. (Todor Ivanov) #10698
- [MSOutput] MSOutput: do not fail workflow if there are containers unknown to Rucio (Alan Malta Rodrigues) #10699
- [MS] Fix error message reported via MSUnmerged/MSRuleCleaner REST API (Alan Malta Rodrigues) #10700
Enhancements
WMAgent
Features and/or feature changes
Bug Fixes
- Fix broken variable reference in TestMonitor (Alan Malta Rodrigues) #10665
Enhancements
- Performance improvements for WorkQueueManagerCleaner thread (Alan Malta Rodrigues) #10695
Python3 related changes
WMCore 1.5.0 central services production release
This is the first Python3 release deployed to CMSWEB Production.
This release also brings the initial skeleton for a new MicroService to perform the cleanup of unneeded unmerged files, called MSUnmerged.
It has also a few important feature changes, required to make the behavior consistent between Python2 and Python3. Last but not least,
many other Python3-related fixes and improvements are part of this release.
Release date: 2 July 2021.
Changes since release: 1.4.9.
Central services
Software stack
Features and/or feature changes
- MSUnmerged basic skeleton. (Todor Ivanov) #10598
- MSUnmerged construction around functional pipelines and configurable RSE lists. (Todor Ivanov) #10621
- Replace built-in hash function by hashlib.sha1 (Alan Malta Rodrigues) #10605
- Make output module ordering consistent between Py2/Py3 (Alan Malta Rodrigues) #10610
- Force pickle protocol=0 for the spec creation to be compatible with Py2/Py3 (Alan Malta Rodrigues) #10628
- Encode CMSSW picklearguments to bytes before loading it (Alan Malta Rodrigues) #10648
Bug Fixes
- Deal with EventStreams=None in StepChain step level (Alan Malta Rodrigues) #10594
Enhancements
- Fix string type for DBS mock calls in python2 (Alan Malta Rodrigues) #10588
- Update code creating DBS mocked data; ditched unicode signatures (Alan Malta Rodrigues) #10585
- Create function to elements being acquired in WorkQueue (Alan Malta Rodrigues) #10605
- Print request name in the InvalidStateTransition exception (Alan Malta Rodrigues) #10649
WMAgent
Features and/or feature changes
- Remove repack slots flag (germanfgv) #10629
- DBFormatter formatDict/formatList to native string instead of bytes (Alan Malta Rodrigues) #10639
Bug Fixes
- Fix problem with T0 processes (Kenyi Hurtado) #10527
- Fix WMAgent component Daemon read/write operations (Alan Malta Rodrigues) #10632 #10645
Enhancements
- Create workload sandbox efficiently (Alan Malta Rodrigues) #10612
Python3 related changes
- Fixes for slice 13: file open mode; pickle operations; stdout/stderr (Alan Malta Rodrigues) #10585
- [py2py3] fix unittests from #10289 - src (Dario Mapelli) #10580
- [py2py3] fix unittest from #10262 - src (Dario Mapelli) #10590
- Ensure input for md5 algorithm is a bytes-object in Python3 (Alan Malta Rodrigues) #10595
- [py2py3] WMCore/Services/Service - suggestion (Dario Mapelli) #10584
- Format REST response to bytes in Python3 (Alan Malta Rodrigues) #10601
- [py2py3] Fixing unittests from #10329 - src (Dario Mapelli) #10606
- [py2py3] fix some unittests from #10307 - src (Dario Mapelli) #10603
- [py2py3] Fix AgentStatusWatcher subprocess.Popen calls (Alan Malta Rodrigues) #10635
- [py2py3] Open file in binary mode for pickle load/dump (Alan Malta Rodrigues) #10637
WMCore 1.4.9 central services production release
This release contains major Python3 code modernization, while keeping compatibility with Python2.
In addition to that, there is the removal of all the old Dashboard/DashboardMonitor libraries. Last but not least, it also contains a few bug-fixes and overall enhancements.
Release date: 15 June 2021.
Changes since release: 1.4.8.
Central services
Software stack
- PY3: Ramp up Rucio Client vesion (Todor Ivanov) #10567 cms-sw/cmsdist#6959
- PY3: added Cheetah3 to requirements_py3.txt (Dario Mapelli) #10491
- Replace mox dependency with mock. Remove deprecated testMkdir test (khurtado) #10484
- Removed pystack from our Python2 COMP stack (Alan Malta Rodrigues) dmwm/deployment#1056
- Create new spec for py2-cmscouchapp; remove old couchapp/restkit specs (Alan Malta Rodrigues) cms-sw/cmsdist#6892 cms-sw/cmsdist#6897
- Created new spec for py3-cmscouchapp, version (Alan Malta Rodrigues) 1.3.4 cms-sw/cmsdist#6984
- Created new spec for CouchDB 1.6 with python3 CMSCouchapp (Alan Malta Rodrigues) cms-sw/cmsdist#6919
- Created new spec for Python3 DBS clients (Todor Ivanov) cms-sw/cmsdist#6837 cms-sw/cmsdist#6946
- Created new spec for py3-mysqclient (Todor Ivanov) cms-sw/cmsdist#6926
- Add condor python3 version and make wmagentpy3 require it (khurtado) cms-sw/cmsdist#6925
Features and/or feature changes
- Remove Services/Dashboard package; WMRuntime DashboardMonitor and DashboardInterface (Alan Malta Rodrigues) #10494
- Adjust couchapp push command to match CMSCouchapp (Alan Malta Rodrigues) #10485
- Remake MSTransferor alert subject; increase alert expiration time to 1h (Alan Malta Rodrigues) #10475
- Add couchdb manage function to update couchapps (Erik Gough) #1045 #1057
Bug Fixes
- Fix DID existence check in global workqueue (Alan Malta Rodrigues) #10557
Enhancements
WMAgent
Features and/or feature changes
- Remove Services/Dashboard package; WMRuntime DashboardMonitor and DashboardInterface (Alan Malta Rodrigues) #10494
- Decouple CMSSW and WMCore python versions from PSet tweak (khurtado) #10179
- Add support to OverrideCatalog, allowing use of different storage (germanfgv) #10344
- Adjust couchapp push command to match CMSCouchapp (Alan Malta Rodrigues) #10485
- Add couchdb manage function to update couchapps (Erik Gough) #1045 #1057
Bug Fixes
- PY3: source init.sh script from the wmagentpy3 directory (Alan Malta Rodrigues) #1055
- PY3: Fix WMCORE_ROOT environment variable for WMAgent PY3 (Alan Malta Rodrigues) #1052
- PY3: Fix python interpreter to be used in wmagentpy3 spec (Alan Malta Rodrigues) #6922
Enhancements
- Fix Rucio-related unit tests (Alan Malta Rodrigues) #10576
- Flowchart with the WMCore Jenkins CI setup (Alan Malta Rodrigues) #10560
- Python3 workflow test templates (Alan Malta Rodrigues) #10480
- PY3: Add devtools variant to wmagentpy3 deploy script, for unittests (Alan Malta Rodrigues) #1053
Python3 related changes
- fix unittests from #10295 - src (Dario Mapelli) #10581
- Sort files by name in EventAwareLumiByWork; fix slice 7 unit tests (Alan Malta Rodrigues) #10583
- fix unittests from Slice 3 in py3 (Dario Mapelli) #10579
- Fix unittests from Slice 1 in py3 (Dario Mapelli) #10575
- Fix Py3 unit tests for slice 6, CouchService_t (Alan Malta Rodrigues) #10578
- Fix unittest from #10529, Proxy_t (Erik Gough) #10577
- Open parsed file in bytes. (Todor Ivanov) #10571
- Fix DBFormatter unit test (Alan Malta Rodrigues) #10574
- Re-execute REST/Main with the correct python version (Alan Malta Rodrigues) #10573
- fix unittests from WMCore/Datastructs in py3 (Dario Mapelli) #10562
- Adapt to renaming of assertItemsEqual to assertCountEqual in py3 (Dario Mapelli) #10563
- List of unstable python3 unit tests (Alan Malta Rodrigues) #10565
- fix unittests from WMCore/Algorithms in py3 (Dario Mapelli) #10561
- fix unittests from WMCoreConfigCache (Dario Mapelli) #10522
- Fix missing exceptions module in python3 (Todor Ivanov) #10524
- Fix some problems with string handling in src/python/WMCore/Services/Requests.py (Dario Mapelli) #10508
- futurize src/python/WMCore/Configuration.py (Dario Mapelli) #9974
- fix unittests in test/python/Utils_t - #10500 - src (Dario Mapelli) #10500
- new variables Utils.PythonVersion.PY2/3 (Dario Mapelli) #10518
- MySQLCore - update use of arguments in list.sort() to reflect py3-compatible interface (Dario Mapelli) #10506
- fix stuck unittest in test/python/Utils_t (Dario Mapelli) #10503
- Do not convert unicode to bytes when parsing data from Databases (Dario Mapelli) #10504
- pycurl_manager: converting response to unicode before parsing - src (Dario Mapelli) #10501
- setup_test.py compatible with both py2 and py3 (Dario Mapelli) #10499
- new unittest deploy scripts for wmagentpy3 (Dario Mapelli) #10496
- Modernized bin/ scripts to support py3 (Dario Mapelli) #10482
WMCore 1.4.8 central services production release
This release contains mostly bug-fixes and a few python3 modernization changes.
Release date: 3 May 2021.
Changes since release: 1.4.7.
Central services
Software stack
Features and/or feature changes
- Force cmsweb-prod in DBS3Reader (Alan Malta Rodrigues) #10444
Bug Fixes
- Revert changes to src/python/WMCore/Configuration.py from slice 17 (Dario Mapelli) #10443
- Wrap sendAlert() in a try/except block. (Todor Ivanov) #10467
- Do not delete transfer documents if bystatus couchview response is empty (Alan Malta Rodrigues) #10471
- Properly evaluate stuck rules in MSTransferor (Alan Malta Rodrigues) #10474
Enhancements
- Remove py package from the requirements list (Alan Malta Rodrigues) #10462
WMAgent
Features and/or feature changes
- Force cmsweb-prod in DBS3Reader (Alan Malta Rodrigues) #10444
- Remove unused tweak file (Kenyi Hurtado) #10434
Bug Fixes
- Revert changes to src/python/WMCore/Configuration.py from slice 17 (Dario Mapelli) #10443
- Retrieve only a summary of the condor jobs in DrainStatusPoller (Alan Malta Rodrigues) #10458
- Only create Rucio connection in PileupFetcher if there is a pileup dataset (Alan Malta Rodrigues) #10470
Enhancements
- Create symbolic links in deploy-wmagent (tivanov) #10454
- Bump WMAgent deployment example to 1.4.7 series (Alan Malta Rodrigues) #10457
Python3 related changes
WMAgent 1.4.7.patch2 production release
This is a bug fix patch release. For full release cycle changes, please refer to: https://github.com/dmwm/WMCore/releases/tag/1.4.7.patch1
Changes since release: 1.4.7.patch1.
Release date: 19 April 2021.
Bug Fixes
- Temporary fix to let CouchDB replications go through the standard port - wmagent branch (Alan Malta Rodrigues) #10453
WMAgent 1.4.7.patch1 production release
First WMAgent production release on the 1.4.7 series.
This release has close to 100% of our codebase compatible with both python2 and python3. It is also important to mention that the PhEDExInjector component has been deprecated. Furthermore, many other developments and bug fixes went into this release as well. See all the changes compared to the previous WMAgent production release mentioned below.
Changes since release: 1.4.5.patch1.
Release date: 15 April 2021.
WMAgent
Software stack
- Bump httplib2 from 0.18.1 to 0.19.0 #10271
Features and/or feature changes
- Forward WMCore.Services.Requests class to port 8443 for cmsweb services. (Todor Ivanov) #10330 #10364
- Remove PhEDExInjector component and its config/deploy code (Alan Malta Rodrigues) #10366
- Remove datatier-based functionality from RucioInjector (Alan Malta Rodrigues) #10369
- Remove switch between Rucio and PhEDEx (Alan Malta Rodrigues) #10283
- Update production DBS URL to cmsweb-prod (Alan Malta Rodrigues) #10269
Bug Fixes
- Fix values for the "submitted" key in JobSummary (Alan Malta Rodrigues) #10325
- fix bug in selecting stagein for UL logcollect, fixes ##10294 (Dirk Hufnagel) #10296
- Check if DID exists before creating a replication rule (Alan Malta Rodrigues) #10265
- Trap SSL_connect error and Service Unavailable in DBSUploadPoller (Alan Malta Rodrigues) #10252
Enhancements
- added script to query Rucio and create mock data; added mock data (Alan Malta Rodrigues) #10342
- Add check and different path for wmagent component. (tivanov) #10327
- Remove PhEDEx backwards compatibility from the Rucio module (Alan Malta Rodrigues) #10273
- Make WMAgent job wrapper slightly more verbose; fix one error message (Alan Malta Rodrigues) #10261
Python3 related changes
- [py2py3] src/python/A/B/C slice 5 - issue #10116 - src (Dario Mapelli) #10262
- [py2py3] src/python/A/B/C slice 6 - issue #10127 - src (Dario Mapelli) #10277
- [py2py3] src/python/A/B/C slice 7 - issue #10131 - src (Dario Mapelli) #10284
- [py2py3] src/python/A/B/C slice 8 - issue #10132 - src (Dario Mapelli) #10289
- [py2py3] src/python/A/B/C slice 9 - issue #10133 - src (Dario Mapelli) #10290
- [py2py3] src/python/A/B/C slice 10 - issue #10134 - src (Dario Mapelli) #10295
- [py2py3] pycurl_manager imports encoder to Bytes from Utilities - src (Dario Mapelli) #10250
- [py2py3] Utils/Utilities.py - encoder Unicode to Bytes - src (Dario Mapelli) #10250
- [py2py3] src/python/A/B/C slice 11 - issue #10135 - src (Dario Mapelli) #10299
- [py2py3] src/python/A/B/C slice 12 - issue #10136 - src (Dario Mapelli) #10307
- [py2py3] src/python/A/B/C slice 13 - issue #10137 - src (Dario Mapelli) #10310
- [py2py3] src/python/A/B/C slice 14 - issue #10138 - src (Dario Mapelli) #10313
- [py2py3] src/python/A/B/C slice 15 - issue #10139 - src (Dario Mapelli) #10319
- [py2py3] src/python/A/B/C slice 16 - issue #10140 - src (Dario Mapelli) #10321
- [py2py3] src/python/A/B/C slice 17 - issue #10141 - src (Dario Mapelli) #10329
- [py2py3] src/python/A/B/C slice 18 - issue #10142 - src (Dario Mapelli) #10331
- [py2py3] src/python/A/B/C slice 19 - issue #10143 - src (Dario Mapelli) #10348
- [py2py3] src/python/A/B/C slice 20 - issue #10144 - src (Dario Mapelli) #10349
- [py2py3] src/python/A/B/C slice 21 - issue #10145 - src (Dario Mapelli) #10361
- [py2py3] src/python/A/B/C slice 24 - issue #10148 - src (Dario Mapelli) #10371
- [py2py3] src/python/A/B/C slice 22 - issue #10146 - src (Dario Mapelli) #10362
- [py2py3] src/python/A/B/C slice 23 - issue #10147 - src (Dario Mapelli) #10363
- [py2py3] src/python/A/B/C slice 25 - issue #10149 - src (Dario Mapelli) #10379
WMCore 1.4.7 central services and WMAgent production release
In this release, almost all the WMCore source code has been modernized to work with python2 and python3 (still a couple very minor modules to be carefully modernized, and third-party libraries).
It also brings in a new Prometheus AlertManager wrapper module, and its integration to the MicroServices. It's worth mentioning that WMCore services should now - at the very low pycurl layer - be using port 8443 for every HTTP request.
Last but not least, there has been many bug fixes to the central services.
NOTE: we are working in a new patch version for WMAgent, which will fix a bug originated from the python3 code modernization.
Changes since release: 1.4.6.
Release date: 3 Mar 2021.
Central services
Software stack
- Add py3-cmsmonitoring to the list of python3 requirements (Alan Malta Rodrigues) #10355
Features and/or feature changes
- Dont map pileup location to the campaign top level site lists (Alan Malta Rodrigues) #10338
- Add AlertManager API (Erik Gough) #10308 #10358
- Switch microservices to use AlertManager (Erik Gough) #10346
- Forward WMCore.Services.Requests class to port 8443 for cmsweb services. (Todor Ivanov) #10330 #10364
- Account for tape rule status in MSRulecleaner. (Todor Ivanov) #10370
Bug Fixes
- Fix unhashable types in the ReqMgr2 request API with mask (Alan Malta Rodrigues) #10312
- Fix to unhashable types in the filtered_requests REST API (Alan Malta Rodrigues) #10311
- Validate status transition for cascade option (Alan Malta Rodrigues) #10309
- Reject blocks without any Rucio files in GQ (Alan Malta Rodrigues) #10342
- MSTransferor: iterate over a dictionary as a list when it can change (Alan Malta Rodrigues) #10337
- Change regex when building list of PR numbers in buildrelease (Alan Malta Rodrigues) #10372
Enhancements
- added script to query Rucio and create mock data; added mock data (Alan Malta Rodrigues) #10342
- allow to run REST/Main in frontend via env.var. fix #10266 (belforte) #10339
- Reorganize microservices package structure; remove unused (Alan Malta Rodrigues) #10357 #10373
- Update CMS WM overview schematic (Alan Malta Rodrigues) #10438
WMAgent
Features and/or feature changes
- Forward WMCore.Services.Requests class to port 8443 for cmsweb services. (Todor Ivanov) #10330 #10364
- Remove PhEDExInjector component and its config/deploy code (Alan Malta Rodrigues) #10366
- Remove datatier-based functionality from RucioInjector (Alan Malta Rodrigues) #10369
Bug Fixes
- Fix values for the "submitted" key in JobSummary (Alan Malta Rodrigues) #10325
Enhancements
- Add check and different path for wmagent component. (tivanov) #10327
Python3 related changes
- [py2py3] src/python/A/B/C slice 11 - issue #10135 - src (Dario Mapelli) #10299
- [py2py3] src/python/A/B/C slice 12 - issue #10136 - src (Dario Mapelli) #10307
- [py2py3] src/python/A/B/C slice 13 - issue #10137 - src (Dario Mapelli) #10310
- [py2py3] src/python/A/B/C slice 14 - issue #10138 - src (Dario Mapelli) #10313
- [py2py3] src/python/A/B/C slice 15 - issue #10139 - src (Dario Mapelli) #10319
- [py2py3] src/python/A/B/C slice 16 - issue #10140 - src (Dario Mapelli) #10321
- [py2py3] src/python/A/B/C slice 17 - issue #10141 - src (Dario Mapelli) #10329
- [py2py3] src/python/A/B/C slice 18 - issue #10142 - src (Dario Mapelli) #10331
- [py2py3] src/python/A/B/C slice 19 - issue #10143 - src (Dario Mapelli) #10348
- [py2py3] src/python/A/B/C slice 20 - issue #10144 - src (Dario Mapelli) #10349
- [py2py3] src/python/A/B/C slice 21 - issue #10145 - src (Dario Mapelli) #10361
- [py2py3] src/python/A/B/C slice 24 - issue #10148 - src (Dario Mapelli) #10371
- [py2py3] src/python/A/B/C slice 22 - issue #10146 - src (Dario Mapelli) #10362
- [py2py3] src/python/A/B/C slice 23 - issue #10147 - src (Dario Mapelli) #10363
- [py2py3] src/python/A/B/C slice 25 - issue #10149 - src (Dario Mapelli) #10379
WMCore 1.4.6 central services production release
This release contains a major new feature in MSRuleCleaner, being able to delete input data placement rules for input data no longer locked. In addition to that, it comes with many python3 modernisation changes.
Release date: 3 Mar 2021.
Changes since release: 1.4.5.
Central services
Software stack
- Bump httplib2 from 0.18.1 to 0.19.0 #10271
Features and/or feature changes
- Include logic for cleaning input data rules. (Todor Ivanov) #10224 #10320
- Remove switch between Rucio and PhEDEx (Alan Malta Rodrigues) #10283
- Skip RSE quota check for RelVal input data placement; added rule meta data (Alan Malta Rodrigues) #10275
- Update production DBS URL to cmsweb-prod (Alan Malta Rodrigues) #10269
Bug Fixes
Enhancements
- add new dataset to mock in DBS (Alan Malta Rodrigues) #10283
- mock Rucio getPileupLockedAndAvailable API; patch PileupFetcher_t (Alan Malta Rodrigues) #10283
- Properly set rucio.cfg in our docker unittest setup (Alan Malta Rodrigues) #10292
- Remove reportWorkflowToDashboard method from StdBase (Alan Malta Rodrigues) #10281
- Completely remove SiteDB, including emulators and tests (Alan Malta Rodrigues) #10280
- Remove PhEDEx backwards compatibility from the Rucio module (Alan Malta Rodrigues) #10273
- Update snapshot of Unified campaigns; update a couple json templates (Alan Malta Rodrigues) #10242
- add back WMBS to crabtaskworker dependencies and remove asyncstageout (belforte) #10246
WMAgent
Features and/or feature changes
Bug Fixes
- fix bug in selecting stagein for UL logcollect, fixes ##10294 (Dirk Hufnagel) #10296
- Check if DID exists before creating a replication rule (Alan Malta Rodrigues) #10265
- Trap SSL_connect error and Service Unavailable in DBSUploadPoller (Alan Malta Rodrigues) #10252
Enhancements
- Make WMAgent job wrapper slightly more verbose; fix one error message (Alan Malta Rodrigues) #10261
- Update deployment script to accept python3 WMAgent service (Alan Malta Rodrigues) #10302
Python3 related changes
- Python3 requirements for WMAgent and WMAgent-Dev (Alan Malta Rodrigues) #10247 #10268
- [py2py3] src/python/A/B/C slice 10 - issue #10134 - src (Dario Mapelli) #10295
- [py2py3] src/python/A/B/C slice 9 - issue #10133 - src (Dario Mapelli) #10290
- [py2py3] src/python/A/B/C slice 7 - issue #10131 - src (Dario Mapelli) #10284
- [py2py3] src/python/A/B/C slice 8 - issue #10132 - src (Dario Mapelli) #10289
- [py2py3] src/python/A/B/C slice 5 - issue #10116 - src (Dario Mapelli) #10262
- [py2py3] src/python/A/B/C slice 6 - issue #10127 - src (Dario Mapelli) #10277
- [py2py3] pycurl_manager imports encoder to Bytes from Utilities - src (Dario Mapelli) #10250
- [py2py3] Utils/Utilities.py - encoder Unicode to Bytes - src (Dario Mapelli) #10250
WMAgent 1.4.5.patch1 production release
First WMAgent production release on the 1.4.5 series. This release is built on top of the system OpenSSL library and it contains a large amount of python3 modernization changes. Many other developments and bug fixes went into this release as well, see all the changes compared to the previous WMAgent production release.
Changes since release: 1.4.3.patch2.
Release date: 8 Feb 2021.
WMAgent
Software stack
- OpenSSL updated from
1.0.1rto version1.0.2k-fips; and it now relies on the system-based libraries - Python cx-Oracle library updated from
5.2.1to7.3.0 - Python rucio-clients library updated from
1.23.11to1.24.2.post1
Features and/or feature changes
- Reuse Rucio object in WorkQueue start policy modules (Alan Malta Rodrigues) #10099
- add all Rucio lfns2pfns args. to getPFN. Fix #10056 (belforte) #10122
- Add requiresApproval method to Rucio class. Manage RSE rule approval requirements in insertContainerRules (germanfgv) #10105
- add option for preloader for xrdcp stageout (Dirk Hufnagel) #10226
- Update production DBS URL to cmsweb-prod (Alan Malta Rodrigues) #10269
Bug Fixes
- Fix pileup location with multi-RSE rules (Alan Malta Rodrigues) #10194
- Fix servicekey permission in the agent deployment; remove central services url option (Alan Malta Rodrigues) #10163
- Add only merged parents to the ACDC docs (Alan Malta Rodrigues) #9997
- Fix JobAccountant missing parent dbsbuffer file (Alan Malta Rodrigues) #9997
- Only resolve RSE requires_approval for T0 data placement (Alan Malta Rodrigues) #10130
- Catch UnicodeDecodeError in JobAccountant (Alan Malta Rodrigues) #10152
- Trap SSL_connect error and Service Unavailable in DBSUploadPoller #10252
- Check if DID exists before creating a replication rule #10265
Enhancements
- Remove badly encoded single quotes from Rucio module (Alan Malta Rodrigues) #10124
- Minor updates to some json test templates (Alan Malta Rodrigues) #10113
- Add 2021 validation campaigns. (Todor Ivanov) #10210
- Make WMAgent job wrapper slightly more verbose; fix one error message #10261
Python3 related changes
- [py2py3] WMCore/WMException.py - issue 10173 - src (Dario Mapelli) #10174
- [py2py3] src/python/A/B/C slice 2 - issue 10098 - src (Dario Mapelli) #10103
- [py2py3] src/python/A/B/C slice 1 - issue 10055 - src (Dario Mapelli) #10057
- [py2py3] src/python/A/B/C slice 3 - issue 10114 - src (Dario Mapelli) #10117
- [py2py3] issue #10173, PR #10187 - src (Dario Mapelli) #10187
- [py2py3] WMCore/WMSpec/ConfigSectionTree.py - issue #10218 - src (Dario Mapelli) #10221
- [py2py3] src/python/A/B/C slice 4 - issue 10115 - src (Dario Mapelli) #10165
- [py2py3] fix pre-py2.6:
type() ==->isinstance(Dario Mapelli) #10172 - [py2py3] Database/DBFormatter.py - src - fix #10235 (Dario Mapelli) #10238
- [py2py3] pycurl_manager - better handling of curl.setopts() input arguments #10250
WMCore 1.4.5 services and WMAgent production release
This release is built on top of the SLC7 OpenSSL library, thus no longer shipping our own CMS COMP OpenSSL package. The cx-Oracle library went through an upgrade from version 5.2.1 to 7.3.0 as well.
In addition to that, it comes with many python3 modernization changes.
Changes since release: 1.4.4.
Release date: 12 Jan 2021.
Central services
Features and/or feature changes
- All RPMs built on top of system-based OpenSSL version cms-sw/cmsdist#6440
Bug Fixes
- Catch UnboundLocalError exception when EmailAlert fails connecting to the SMTP server #10244
- MSTransferor: Convert Campaign pileup data location to PSNs to match against the SiteWhitelist #10243
Enhancements
WMAgent
Features and/or feature changes
- add option for preloader for xrdcp stageout #10226
Bug Fixes
Enhancements
- Add 2021 validation campaigns. #10210
Python3 related changes
- [py2py3] src/python/A/B/C slice 1 - issue 10055 - src #10057
- [py2py3] src/python/A/B/C slice 3 - issue 10114 - src #10117
- [py2py3] issue #10173, PR #10187 - src #10187
- [py2py3] WMCore/WMSpec/ConfigSectionTree.py - issue #10218 - src #10221
- [py2py3] src/python/A/B/C slice 4 - issue 10115 - src #10165
- [py2py3] fix pre-py2.6:
type() ==->isinstance#10172 - [py2py3] Database/DBFormatter.py - src - fix #10235 #10238