HISTORY.rst

History

1.26.7 (2024-10-17)

• Hotfix to update for git modules. Now that we are using tags, we need to fetch rather than pull if it is a tag, i.e., HEAD is detached.

1.26.5 (2024-10-16)

• Modifying how we inspect git modules before checkout and how we find unique branch names so that we only do the git clone once. Previously, this could have been done up to 3 times per checkout.

1.26.4 (2024-08-26)

• pyam-build now changes the directory permissions of the temporary directory to 775. That way, the directory is accessible by others.

1.26.3 (2024-07-25)

• pyam-build now just prints info to stderr rather than trying to send an email at the end. If modules don't build then it exits with error code 2.

1.26.1 (2024-07-20)

• Made more robust the handling of mdules whose name includes the '-' character (eg. dev-utils).

1.26 (2024-07-05)

• Adding pyam-format to the list of scripts.

1.25.25 (2024-06-25)

• Minor fix to pyam save so that the modules using git only add versioned files to their commits when merging.

1.25.24 (2024-05-16)

• Fixing corner case where sys.stdin is not defined. Needed when running pyam from a daemon process that does not have stdin.

1.25.21 (2024-04-15)

• Fixing content transfer type to use 8bit. This what was there before, but got missed in the port to aiosmtp.

1.25.20 (2024-04-15)

• Updating to use aiosmtpd rather than smtpd. aiosmtpd replaces smtpd in newer Python versions.

1.25.19 (2024-04-13)

• Fixed a 'pyam sync' so that when it is run on a tagged module and a branch is specified, the module is synced to a branch with that name.

1.25.18 (2024-03-03)

• Updated paths to not use the obsolete dlab/ file system partition.

1.25.17 (2024-02-26)

• Fixing bug with conditional that was preventing SSH modules from being created with pyam register-new-module.

1.25.16 (2024-02-13)

• Fixed 'pyam-format' to filter out extra messages coming from 'pyam diff' which were being interpreted as the presence of unreleased commits for the modules, and causing this command to fail.
• Updating so that external git modules can be accessed via SSH as well. Previously, only HTTPS was supported.

1.25.15 (2023-12-25)

• Fixed 'pyam diff' to use the root of the branch to diff from when on a branch. Previously it was incorrectly diffing against the latest release.

1.25.14 (2023-12-06)

• Minor updates to improve error messages for git when there are main branch commits.

1.25.12 (2023-09-14)

• Minor updates so that users can enter a release message in their editor when pyam saving a git module; similar to what they can already do for an svn module.

1.25.8 (2023-06-07)

• When updating to a new tag, do a fetch first to ensure the local repo has that tag. Otherwise, the repo may try to update to a tag it does not know about.

1.25.7 (2023-05-26)

• For a tagged module, sync return right away if the module is already at the latest version.

1.25.5 (2023-05-18)

• Now pyam save-package creates a symbolic for the ReleaseNotes in the change logs folder for viewing from the releases web page.

1.25.3 (2023-05-14)

• Updated regtest process to make temp folders writeable even when the checked out pyam is a read only tagged module.
• Updated the releses web page to allow toggling of column sorting, and also to add page navigation links at the top of the table as well.

1.25.2 (2023-05-14)

• Fixed a bug in the syncing of link modules where due to a bad path, pyam was detecting that the target link module did not exsist and was checking out a tagged version instead.
• Added --baseline option to the register-new-package, save-package and sync commands.
• For the register-new-package command, this option will create a new BaselinePkg package with just the 'SiteDefs' module to be used for baseline releases.
• For the save-pacakge command, this option will make a release of the BaselinePkg using the module releases defined by the (required) yam config file.
• For the sync command, this option will cause the module syncing to use module releases associated with the specified baseline release for each module sync.

1.25.1 (2023-05-07)

• Replaced use of the obsolete egrep with grep -E in regtests and other places.

1.25 (2023-05-02)

• Fixing tagged work module sync issue for git modules. Now, syncing should work rather than creating a branch with the same name as the tag you want.

1.24.23 (2023-03-08)

• Fixing module branch data for tagged git modules so that pyam-rebuild works with tagged git modules.

1.24.22 (2023-03-08)

• Updated pyam-format to support git modules in addition to svn ones.

1.24.21 (2023-03-08)

• Updated pyam-build's default command to reverte change files to support git modules in addition to svn ones.

1.24.20 (2023-03-08)

• Added the --exclude option to the 'obsolete-builds' command to filter out non-essential modules.

1.24.19 (2023-03-02)

• Fixing some corner cases that appear in the git revsision control system's working_copy_exists method.

1.24.18 (2023-03-02)

• Updating hasMainTrunkCommits in git revision system. Previous version had a bug that prevented pyam-build from working properly with git modules.

1.24.17 (2023-03-01)

• Updating git's _check_out_1 method so that repos are cloned into the correct place. There were cases when using pyam-build where this was not happening.

1.24.16 (2023-02-01)

• Hotfix: removing unecessary changes.

1.24.15 (2023-02-01)

• Updating pyam diff for git modules so a git fetch is performed before the diff. This ensures that remote branches/tags are present. Without doing a fetch first, one can get errors related to missing brances/tags.

1.24.14 (2023-01-16)

• Pyam rebuild now works with Git modules.
• Handling corner case when repository keyword is None.

1.24.13 (2023-01-12)

• Pyam now recognizes any repository whose keyword contains "git" or "Git" as a git-based repository.

1.24.12 (2023-01-10)

• Git modules can now be housed on a remote server, e.g., GitLab or GitHub. When registering a module, if it is a git module and the repository keyword begins with https://, pyam will assume it already exists on a remote server.

1.24.8 (2022-11-11)

• Fixed error in git revision control system where branch names with a dash in them were being truncated. This was negatively affecting operaions for usernames that had a dash in them.

1.24.5 (2022-11-07)

• Added the --no-diff option to 'pyam save'. This option skips the generation of change diff for inclusion in the noification email message. This option can be handy when the diff is very large.

1.23.19 (2022-07-05)

• Added pyam-format utility for making automated releases of formatted modules.
• Added the --emai-tag option to 'pyam save' command for inserting strings into release notification email subject lines
• Now if the 'vcs' field defaults to 'svn' if it is empty for a module in the modulePackages database table.

1.23.17 (2022-05-30)

• Added preliminary support for modules version controlled in git repos as an alternative to svn repos. This is work in progress.
• During module creation via 'pyam register-new-module` we can request the module to be a git module by specifying 'git' as the value to its '--repository-keyword' option. Each module will have its own dedicated git repository in a common area for all git repos.
• This does require adding a vcs column to the packageModules table.
• The repositories paths in the pyam configuration file (or command line option) must include an entry for the 'git` repositories path.
• The git Python module must be installed for use of this feature.
• Current support is limited to the local file system protocol. Network protocols support will be added in the future.

1.22.24 (2021-08-01)

• Now a branched module sync aborts if there are any uncommitted files in the module. It used to abort earlier too when there were uncommited files, but only after the merge had begun - leading to partially merged and messed up modules.

1.22.23 (2021-07-23)

• Fixed handling of case where YAM_SITE is undefined, This fixes the make based building where this si the case.
• Added --jobs option to use specify the option for :make -j" for parallel builds. A negative value (the default) cause pyam to detect and use the number of cpus on the system.

1.22.22 (2021-05-31)

• Updated help messages
• Made pyam setup more robust to handling undefined release directory.

1.22.21 (2021-05-25)

• Fixed minor issue with --dependencies option for 'pyam checkout'
• Fixed check for bad module/package names specified on the command line to throw exception for unrecognized ones.

1.22.20 (2021-05-24)

• Added the -all-except option to pyam sync. When syncing a sandbox, this option can be used to skip syncing some of the modules.

1.22.19 (2020-12-30)

• The checkout --dependencies option will check out tagged work modules for the latest releases of the dependency modules if the link modules are not available.

1.22.18 (2020-12-25)

• Now the 'dependents' and 'dependencies' subcommands take multiple module name arguments and return the overall dependencies for the all the specified modules.
• The 'dependents' subcommand now takes a --recursive option to include dependent modules multiple levels away.
• The 'checkout' subcommand now takes a --dependencies option to add modules that the specified modules depend on as link modules to the sandbox.
• Now pyam prints out module version info when checking out a module.
• Added the --from-release and -to-release options to the 'diff' subcommand. These options can be used to create patch files to go from one module release to another.

1.22.17 (2020-12-07)

• When checking out a sandbox, a tagged module is created for missing link modules instead of creating a branched module.

1.22.16 (2020-11-28)

• pyam sync now only rebuilds the synced modules instead of all work modules in the sandbox after syncing.
• Now pyam scrap can handle command line module names with trailing '/' in the name.

1.22.15 (2020-11-22)

• Now module saves store more information about header file API changes that might make build releases of other modules obsolete.

1.22.14 (2020-11-19)

• Now disallow branches with '-' character in them
• Fixed up creation of ChangeLog etc links for build releases

1.22.13 (2020-11-16)

• Now module build releases save ChangeLog etc info to the database and also create symbolic links for access from the releases page.

1.22.12 (2020-11-11)

• Fixed up 'pyam sync' to allow syncing to a maintenance module release.
• Fixed up 'pyam scrap --remove' to not complain if work module directory is not there under src/

1.22.11 (2020-11-10)

• Removed unnecessary check for the existence of a release directory when checking out maintenance releases

1.22.10 (2020-11-06)

• Fixed up 'pyam scrap' to work properly when removing multiple work modules.

1.22.9 (2020-11-02)

• Refactored code to use enum.Flag. With this, a minimum of Python 3.6 is required to using pyam.

1.22.8 (2020-11-01)

• Now package releases also save the module release (relatives) information in the database. This feature from the previous YaM was never implemented in pyam. This data is used to view a package release's content on the online releases page.

• Added a few subcommads to the 'pyam util' information and utility command. Key subcommands are:

  -- all-modules: display a table of all current modules -- all-packages: display a table of all current packages -- package-release: display a table of module releases in a package relesase

1.22.7 (2020-10-30)

• Fixed up the handling of the --work-modules when checking out a specific package release sandbox. Now modules versions in the package release are checked out instead of the latest versions.
• Fixed up save-package command to handle maintenance release modules specified in the module configuration for the release.

1.22.6 (2020-10-29)

• Internal refactoring. Added the module_configuration_utils.py module to consolidate the various utility methods for maniuplating a sandbox's link/work module configuration.

1.22.5 (2020-09-16)

• Now the .d dependency file parsing process skips over .d files that it cannot parse (may not be dependency files at all)
• Restored save sub-command's --no-pre-save-hook option.

1.22.3 (2020-06-10)

• Updated to work with Python 3.7

1.21.25 (2020-05-27)

• Added --repository-password option. If specified, it causes pyam to disable subversion's password caching system. Since version 1.12, subversion has switched to using system level password storing capabilities and this new option can be used to bypass this process.

1.21.24 (2020-04-08)

• Fixed pyam-build to once again work with Python 3.4. Updates for Python 3.7 had broken the compatibility.

1.21.23 (2020-03-02)

• Updated module releases from branch to pick up any unreleased commit messages on the main trunk when generating ChangeLog entry.

1.21.22 (2020-03-02)

• Now skip check on latest modules in sandbox when making a maintenance release
• Disallow doing regular and maintenance releases together
• sync will error out if any of the modules is on a maintenance branch

1.21.21 (2020-02-19)

• Improved documentation for various commands.

1.21.20 (2019-11-26)

• Now 'pyam config' option --all-to-work can be used together with the --update-links, --to-link etc options
• Added the --current-modules option to 'pyam config' to create a YAM.config file with latest link modules for all non-obsolete modules.

1.21.19 (2019-11-25)

• 2019-11-25 Added new package creation process to do all error checking first before modifying the database or the repository. Added further error checking.
• 2019-11-18 In dbutil add-release, added check that the release exists in the version control repository. Added dbutil pyam command for repairing database
• 2019-11-14 Added more checks to ensure that pyam catches errors in YAM.config files.
• 2019-10-29 The recent changes to package releases meant that the release YAM.config files now only contain link modules. Updated setup command so that the --work-modules, --tagged-work-modules, --main-work-modules now work so that can get work modules for the modules in such package releases. Added check for sync that the release argument is legal.
• 2019-10-23 Added support for creating external symbolic links for module ChangeLog, ReleaseNotes files after a release for viewing from the on-line releases page. Now sync for a main trunk module will throw an error if there are commits on the main trunk.

1.21.9 (2019-10-17)

• Now mergeinfo after a module sync is automatically committed.

1.21.8 (2019-10-16)

• Removed the --latest-modules and --no-latest-modules options from save-package, and instead added the --config-file option. By default, save-package will make a package release with the latest releases of the modules. To make a release with a specific combination of module releases, create a YAM.config file with this combination of work/link module releases, and use the --config-file option to point to it (see https://dartslab.jpl.nasa.gov/qa/1030).

1.21.6 (2019-10-02)

• Added --release and --branch options to sync to allow syncing to a branch other than the latest one (see https://dartslab.jpl.nasa.gov/qa/1536)
• Now sync works with main trunk modules too and will switch the module to the specified branch
• Now can switch a tagged module to a branch using the --release/--branch options

1.21.4 (2019-09-29)

• This catch up history entry summarizes key pyam updates since 1.4.1 version (1.18.d release) from Oct 27, 2016.
• Added support for maintenance branches and releases (see https://dartslab.jpl.nasa.gov/qa/1530)
• Now config only transforms YAM.config files, and does not need or work on sandboxes. Also added the --update-links option to bump up link module versions to the latest releases
• Added the scrap subcommand for demoting a work module to a link module, and its --remove option for removing a work/link module from the sandbox. See https://dartslab.jpl.nasa.gov/qa/1534 for usage examples.
• Added the history subcommand for listing recent release history for a module
• Now generate verbose yamlog* log files in ~/yamlogs with trace of command execution (for post mortem debugging if needed). Can disable with --no-log option. Use yamlog.el in emacs for better viewing of these log files.
• Disabled pre-save checks during save for now
• Now pyam-build sends an email for modules it failed to release
• Added the --no-keep-release option to pyam to prevent moving the saved module to the release area (instead it is renamed in sandbox)
• Addded --recall-from option to config to allow the creation of a package's module releases configuration at a specific time in the past
• Now release emails skip diffs information from uninteresting files (eg. auto-generated files)
• pyam save now works with tagged release work modules and results in a build release
• pyam-build no longer creates branches
• Added --recursive option to pyam dependencies
• Now pyam setup checks out tagged work modules for link modules that are no longer available
• Added --release and --branch options to pyam checkout (see https://dartslab.jpl.nasa.gov/qa/526)
• pyam save never deletes work module. If not moved to the release area, the directory is renamed in the sanbox with a time stamp
• Removed requirement that site be defined. It was getting the way of using SITECONFIGDIR for laptop/VM based pyam use.
• Now version reported by pyam --version matches the pyam release number (since pyam 1.18d release which was advertised as 1.4.1 version)

1.4.1 (2016-10-27)

• Correct sync --all so that it syncs up link modules too.

1.4 (2016-10-24)

• In pyam-build output build log to build.log file within the temporary sandbox.

1.3 (2016-09-30)

• In latest, count column length correctly when formatting text.
• Use PAGER environment variable, if defined, instead of less directly.

1.2 (2016-09-23)

• In sync, make files under revision control read only after checking them out.
• Add --tagged-work-modules/-t to sync.
• Add out-of-date work module information to status.
• Handle errors more gracefully in status.
• Fix bug where sync would sync all link modules even if only one was specified.
• In checkout, support checking out tagged modules as branched modules.

1.1 (2016-09-18)

• In sync, support tagged work modules.

1.0 (2016-02-23)

• In save-package, require either --latest-modules or --no-latest-modules.

0.66.8 (2016-02-16)

• Remove redundant relink in setup.

0.66.7 (2016-02-12)

• Make sync module_name update link modules too.
• Add --to-tagged to save.
• Add --no-latest-modules to save-package. This prevents pyam from populating the YAM.config with the latest module versions. This allows for the user to commit the YAM.config themselves.

0.66.6

• Only modify umask if configured to do so.
• Further isolate Python installation in standalone.bash.
• Support setup --no-build correctly.

0.66.5

• Add global --no-create-link-modules option. This can be enabled in the *.pyamrc via create-link-modules=.
• Show progress messages during pyam status.
• Improve performance of pyam status.

0.66.4

• Improve handling of merge conflict. Only modify the sandbox configuration if needed. Previously, we would always move the conflicted module to the main branch in the configuration. But sometimes, merge conflicts happen before we switch to the main branch. In those cases, we should not modify the configuration.

0.66.3

• Improve Unicode-error handling in pyam diff.

0.66.2

• Improve pyam diff documentation.

0.66.1

• Fix bug in pyam diff -a when in a sandbox with modules on the main branch.

0.66

• Add global --no-yam-make-build-system.

0.65.2

• Handle error gracefully when running latest before initialize is ever called.

0.65.1

• Print location of sandbox upon completion of setup.
• Skip module during pyam sync --all and pyam diff --all if it is on the main or a tagged branch.

0.65

• Add --no-release-notes option to disable prompt for and creation of release notes.

0.64.3

• Handle more encoding problems gracefully on Python 2. Python 3 handled this fine by default.

0.64.2

• Improve command-line option sanity checking. --work-modules and --tagged-work-modules are mutually exclusive options.

0.64.1

• Avoid corrupting shell if errors during pyam diff.

0.64

• Support mysql-connector-python 2.x.

0.63.7

• Handle "txn-current-lock" error gracefully.

0.63.6

• Add more information into the crash log.

0.63.5

• Replace a Perl-dependent script in SiteDefs.

0.63.4

• Handle bad line-of-code change statistics gracefully.

0.63.3

• Handle machine resource errors gracefully.

0.63.2

• Handle bad work modules more gracefully in pyam diff.

0.63.1

• Allow --text-editor to configured via configuration file.

0.63

• Add --require-bug-id to require --bug-id with save.

0.62.1

• Handle case where repository access is not available during pyam register-new-module.

0.62

• Add dependents command.

0.61

• Add latest-package command.

0.60.7

• Show progress of pre-save checks.
• Run dangling-link check last.

0.60.6

• Only relink necessary modules during rebuild.

0.60.5

• Add missing build step during sync.
• Handle more database-connectivity errors gracefully.

0.60.4

• Avoid Subversion bug (http://subversion.tigris.org/issues/show_bug.cgi?id=4099).

0.60.3

• Wrap long change-log messages if they are composed of a single long line.
• Check out modules in alphabetical order (except for SiteDefs). This gives the user a better indication of when it will complete. Note that we leave the order alone if the user specified the modules explicitly.
• Gracefully handle sync --commit failure due to merge conflict.

0.60.2

• Add --undo flag to pyam unregister-module.
• Add --test-email to test command.

0.60.1

• Wrap some long release note messages. Only wrap if the input message from the user is composed of only a single line. Trying to wrap multi-line text would break column alignment. And it could result in garbled output.

0.60

• Make --main-work-modules and --tagged-work-modules explicitly mutually exclusive.
• Add --no-lock to ignore existing locks.
• Fix save-package so that it actually commits the sandbox configuration.

0.59.7

• In test, only test build server if asked explicitly via --test-build-server.
• Remove creation of some useless files.

0.59.6

• Simplify sandbox creation code to avoid edge cases.

0.59.5

• Detect incomplete Subversion working copy rather than crashing. This can happen if the user sends a keyboard interrupt during the initial check out of the module.
• Use buffered MySQL connection for reading results to avoid error about unread results during keyboard interrupt.

0.59.4

• Make directories user/group writable by default during register-new-module and register-new-package. This is configurable via --umask.

0.59.3

• Improve performance of status by optimizing some pysvn parameters.

0.59.2

• Test database write permission when beginning save procedure.
• Show modules with uncommitted files in status.

0.59.1

• Only mark files as read-only after the first check out of a tagged work module.

0.59

• Add --database-gateway for tunnelling database connection through ssh port forwarding.

0.58.2

• Check out branched work module if link module is missing during setup.

0.58.1

• Add --no-build to rebuild. This is for purposes of checking out the modules to reflect the sandbox configuration, but not building them.

0.58

• During setup, check out missing link modules as work modules.
• Add --no-build to setup.

0.57

• Take into account missing link modules during setup.
• Relink after config --to-link.
• Deprecate config --to-work in favor of checkout.

0.56.3

• Add --commit/-c option to sync.

0.56.2

• If the build release is missing in the release area, reuse the latest build ID listed in the database. This allows pyam to play nicely with multiple sites that each have their own release area.
• Add --build-id to save.

0.56.1

• Make obsolete-builds detect missing builds in the release area too.
• Align branch names in sandbox configuration file.
• Add equivalent short options for options in setup.

0.56

• Take build release existence on local file system into account. This changes the behavior of config, latest, save, status, sync. Previously we assumed that the build information listed in the database had some relevant meaning. This is not the case if you have multiple distinct release areas (for different sites).
• Make repository permission error explicit.

0.55.6

• Separate out --email-from-address. Deprecate it in favor of --email-from-username and --email-from-hostname.

0.55.5

• Handle pysvn Unicode bug. This happens when it encounters non-Unicode lines in a diff.

0.55.4

• Make sandbox suffix configurable via global --sandbox-format-string. This can be set in the *.pyamrc file.
• Add --work-modules, --main-work-modules, --tagged-work-modules options to setup.
• Add --edit option to setup to allow configuration in a text editor before checking out modules.

0.55.3

• Work around bug in pysvn 1.7.8.

0.55.2

• Speed up initial working copy check during save.

0.54

• Rename pyam summary to pyam status.
• Improve efficiency of pyam rebuild.

0.53

• Add a very incomplete implementation of pyam summary.
• Disentangle --no-check-links.

0.51

• Allow adding release notes during pyam save-package.
• Make sure directories/files are readable after module move.
• Handle directory removal IO errors gracefully.

0.50.1

• Improve sync efficiency.

0.50

• Add Python 3 support.
• Rebuild modified modules after sync.
• Set the number of jobs to the number of cores. This avoids overloading machines with not as much resources.
• Handle SVN permission errors gracefully.
• Add webpage and Unicode support (☃).
• Generalize email exception catching.
• Gracefully handle permission errors and Unicode.
• Remove an innocuous error message.
• Gracefully, handle file permission error.

0.48.6

• Add preemptive module check in checkout.
• This prevents the problem of partial checkout due to misspelling of a module name.

0.48.5

• Add pyam setup --configuration.
• This creates a new sandbox that replicates the passed in configuration.
• This is useful for avoiding manually copy over an existing YAM.config file and rebuilding.
• Add pyam sync --link-modules.
• This deprecates pyam config --update-links. The other two sync options are --all and --work-modules. --all will trigger both link module and work module updates.

0.48.4

• Add pyam checkout --link. Usually, checkout adds work modules to the sandbox. The --link option causes link modules to be added instead. This is useful if you want to add a link module to the sandbox without having to manually edit the sandbox configuration file.

0.48.3

• Mark release as obsolete if no release directory is available. This will let remote build servers know that a link module is missing.

0.48.2

• Fix crash caused by missing variable declaration.

0.48

• Avoid unnecessary build releases. A build release is useless if there is no release directory.

• Add bash tab-completion support. This allows tab completion of commands/options. Activate this by putting the following in ~/.bashrc:

  eval "$(register-python-argcomplete pyam)"
  

• Add pyam help. This prints subcommand usage only without showing the multitude of top-level options.

• Improve build efficiency.

0.47

• Allow saving and checking out modules without a release directory.

• Always check out SiteDefs first. SiteDefs must be checked out first to ensure that make works.

• The SiteDefs-first order was previously not preserved in the save --to-work case due to use of dict rather than OrderedDict.

• Allow forcing a specific tag in save-package:

  $ pyam save-package --revision-tag=R9876-99z MyPackage
  

• Handle left over files on module move.

0.45

• Add support for build dependency detection.
• GCC-generated dependency files are parsed and used to determine module build dependencies. These dependencies are written to the database. This information is used to determine when a module needs to be rebuilt by the pyam-build background build script.