Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #44 and #182 Build Once and Remove .version #184

Merged
merged 1 commit into from
Oct 21, 2022

Conversation

rustydb
Copy link
Contributor

@rustydb rustydb commented Jun 30, 2022

Summary and Scope

This PR changes canu and its pipeline so that it only builds once and uses git-tags for defining stable. Currently canu builds twice in the pipeline (once before the RPM to make the binary, and again for the Docker image).

Full list of changes:

  • Removes .version and canu/.version, instead versioning is done via git-tags
  • Stable RPMs are only built on a git-tag
  • Migrates from the old-style requirements files, using setup.py entirely
  • Ceases using the cdrx/pyinstaller-linux:python3 image for building the binary, the binary is now built in the .spec file using the running build environment (csm-docker-sle-python:3.10)
  • Various tweaks to the README.md as well as the pull_request_template.md
  • Speeds up GitHub actions, the 10minute workflow was broken up into 3 parallelized workflows that run in a lot less time (2-3minutes)
  • Speeds up the Docker image build, instead of rebuilding canu the image build now installs the canu RPM (thus skipping a second build of the canu source)
  • The RPM now creates a canu user, this provides the canu user in the Docker image (it will also live on a system whether or not it is used but it is harmless)
  • Updated the README.md with information about the various versions that report (e.g. stable, unstable, and working directory); .dev1 means canu is running from a dirty directory (with uncommitted files), .postX means canu is not stable

PR checklist (you may replace this section):

  • I have run nox locally and all tests, linting, and code coverage pass.
  • I have added new tests to cover the new code
  • My code follows the style guidelines of this project
  • If adding a new file, I have updated pyinstaller.py
  • I have updated the appropriate Changelog entries in README.md
  • I have incremented the version in the README.md

Now feature/develop RPMs, Docker images, canu --version, and the generated switch config headers will look like this:

canu-1.6.20~1~g6b38cd2-1.x86_64.rpm
cray-canu:1.6.20_1_g6b38cd2
canu, version 1.6.20.post1.dev1

###############################################################################
# CSM version:  1.3
# CANU version: 1.6.20.post1.dev1
###############################################################################

After taking a new git-tag, that git-tag's build will produce an RPM, Docker image, canu --version, and the generated switch config headers will look like this:

canu-1.6.21.x86_64.rpm
cray-canu:1.6.21
canu, version 1.6.21

###############################################################################
# CSM version:  1.3
# CANU version: 1.6.21
###############################################################################

I tested my changes with the following commands on redbull, which produced a working ccj.json and generated switch configs:

canu validate shcd -a v1 --shcd ./Shasta\ River\ Redbull\ CCD_RevA9.xlsx --tabs 40G_10G,NMN,HMN --corners I12,S39,I9,S20,I20,S31 --json --out ccj.json
canu generate network config --ccj ccj.json --csm 1.3 --sls-file redbull/sls_input_file.json --folder generated
Architecture Dell and Mellanox Architecture is deprecated
sw-spine-001 Config Generated
sw-spine-002 Config Generated
sw-leaf-bmc-001 Config Generated

Warning

The following devices typically exist, but were not found in the current configuration.
If the network is supposed to have these devices, check the SHCD to ensure all tabs were included.
If the network does not contain these devices, disregard this warning.
------------------------------------------------------------
leaf
edge
cdu
cec
uan
cmm

NOTE The return from the following commands was large so I did not attach it.

canu validate switch config --ip 10.254.0.2 --generated ./generated/sw-spine-001.cfg
canu validate switch config --ip 10.254.0.2 --generated ./generated/sw-spine-001.cfg --remediation

Issues and Related PRs

Testing

Tested on:

  • List of virtual system tested.
  • List of physical systems tested.

@rustydb rustydb force-pushed the 44-rpm-enhancements branch from eafcbda to f1c906d Compare July 7, 2022 16:00
@rustydb rustydb force-pushed the 44-rpm-enhancements branch 10 times, most recently from 7b198a1 to 971e286 Compare July 18, 2022 14:08
@rustydb rustydb marked this pull request as ready for review July 18, 2022 14:08
@rustydb rustydb requested a review from a team July 18, 2022 14:08
@rustydb rustydb force-pushed the 44-rpm-enhancements branch from 971e286 to 137c830 Compare July 18, 2022 14:11
@rustydb rustydb changed the title WIP 44 Build Once and Use Git-Tags Build Once and Use Git-Tags Jul 18, 2022
@rustydb rustydb force-pushed the 44-rpm-enhancements branch 3 times, most recently from b87fdf6 to a092c97 Compare July 18, 2022 14:32
@rustydb rustydb changed the title Build Once and Use Git-Tags Build Once and Remove .version Jul 18, 2022
@rustydb rustydb force-pushed the 44-rpm-enhancements branch 11 times, most recently from 928e0cd to 01f5dc7 Compare July 18, 2022 20:43
@rustydb rustydb force-pushed the 44-rpm-enhancements branch 5 times, most recently from 11286c4 to 06aa701 Compare October 11, 2022 23:06
@rustydb rustydb force-pushed the 44-rpm-enhancements branch 7 times, most recently from d782cf9 to 7e68582 Compare October 13, 2022 20:37
@rustydb rustydb requested review from trad511 October 13, 2022 21:02
@rustydb rustydb linked an issue Oct 13, 2022 that may be closed by this pull request
@rustydb rustydb changed the title Fixes #44 Build Once and Remove .version Fixes #44 and #182 Build Once and Remove .version Oct 13, 2022
@rustydb rustydb force-pushed the 44-rpm-enhancements branch 3 times, most recently from 49cea0b to 20e62bf Compare October 14, 2022 15:45
This removes the `.version` files and uses git-tags. `canu` now versions
itself within `setup.py` based on `git describe --tags`.

This also builds canu inside of the RPM `specfile` using `pyinstaller`
and no longer uses a special `pyinstaller` image.

The spec file will create a `canu` user.

All of the `requirements` files were converted into `setup.py`, this
tidies up the repository by using less files.

All references to the `.version` files were replaced by leveraging the
`pkg_resources` module to fetch the version from the module `PKG_INFO`.

This also breaks apart the GitHub actions for lint, docs, and tests into
their own workflows allowing for more parallelization.

Lastly this adds an automated, weekly dependabot scan for making PRs to
update dependencies.
@rustydb rustydb force-pushed the 44-rpm-enhancements branch from 20e62bf to f148f40 Compare October 14, 2022 19:23
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 1 Code Smell

No Coverage information No Coverage information
2.6% 2.6% Duplication

@rustydb rustydb requested a review from lukebates123 October 14, 2022 20:20
@trad511 trad511 merged commit 819c745 into develop Oct 21, 2022
@trad511 trad511 deleted the 44-rpm-enhancements branch October 21, 2022 16:13
lukebates123 pushed a commit that referenced this pull request Apr 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove requirements*.txt rpm creation enhancements
3 participants