RELEASE.txt

# This number should track progressive changes in this software, and correspond to git tags
# denoting releases. It is entirely independent of the the Pavilion's VERSION.
RELEASE=2.4

# Release History

## 2.5 Pre-release notes
 - In the 'modules' section of 'run' or 'build' in test configs, modules specified without a version
   no longer verify that the default was loaded.
 - Added the ability to configure module wrappers in test/host configs.
   - This breaks backwards compatibility with existing module wrapper plugins. The 'load()' method
     now takes an addtional, required 'name' argument.

 - Added the ability to extend lists via inheritance across all test configs.
 - Added `pav series sets` and `pav series cancel` commands.

   - `pav series history` is now `pav series `states`.
   - Added `-c` 'overrides' to `pav series run`, and an overrides section to the series config.
   - `pav cancel` no longer cancels series, just tests.
 - Series now only generate tests as needed, rather than all in advance, and handles test creation,
   build, and run errors gracefully.

 - The `SCHED_RUNNING` state is now `SCHED_STARTUP` - existing scheduler plugins that use
   `SCHED_RUNNING` should still work.
 - Added an '--all-passed' option to the results command
 - Added `pav log states` command, which lists all states a test has had.

## 2.4 Release Notes
 - Minimum supported python version is now 3.6
 - All IO or Processor intensive parts of Pavilion have been made multi-threaded or multi-process,
   vastly speeding up many Pavilion operations.
 - Tests now have id's of the form '<label>.<id>'. The default label is 'main',
   so tests from your 'main' (PAV_CONFIG_DIR defined) pavilion config directory
   can leave off the label. Labels aren't saved, but determined when configs
   are loaded.
 - Every config directory can (and should) now have a 'config.yaml' that sets the label for tests
   built from that config directory, as well as the working_directory for tests created
   from those configs. If this does not exist, a default label is applied to the tests,
   and the general pavilion.yaml defined working directory is used.
 - Pavilion has removed all permissions management, except to operate with 'shared_group' as
   the default group. Given multiple working directories, it is now up to the user
   to set sticky bits on those directories if specific permissions are desired for those tests.
 - Test series files now have a 'repeat' config item, to specify the number of times the tests
   in the series should be repeated.
 - The code for test Series has been refactored and greatly cleaned up.
 - Scheduler plugins have been completely revamped (see below).
   - Machines can be 'chunked', and those chunks can be permuted over for smaller run sizes that
     still cover the entire system.
   - Chunk node distributions can be contiguous, random, distributed, and more.
   - You can specify a percentage of the system for chunk size or node count.
 - All builtin plugins are now directly integrated into the Pavilion code, to speed plugin search
   time.
 - Test nodes are (generally) chosen before a test is created, so you can permute over
   `sched.test_node_list` and similar.
 - Added `pav series ls command`
 - Added `pav config` commands.

 - Backwards compatibility breakso
   - Scheduler configuration has been unified under the 'schedule' key in test configs (see below).
   - Imports for the base plugin libraries have changed (but those libraries have not).
   - Tests can no longer have skip conditions that are deferred.
   - The '--wait' argument of the run command has been removed.
   - Tests can no longer set test specific group and permissions
   - The 'restart' test series config item is no longer supported.
   - Many 'sched' variables in Pavilion test configs have changed names. See
     `pav show sched --vars slurm` (or 'raw').
   - The Slurm MPI scheduler plugin has been merged with Slurm. Use the slurm specific 'mpi_cmd'
     parameter to use `mpirun` instead of `srun`. Additional `mpirun` options are also available.
   - In test series files, the 'series' key is now 'test_sets'.
   - The `pav series` command is now 'pav series run`

### Upgrade guide

#### Scheduler config

This update will unfortunately break most tests and plugins, but the fixes are simple.

 - The individual scheduler config sections are now just 'schedule'.
   - Most schedule parameters are universal, and most kept their names from when they
     were under the `slurm` section.
   - The big exception is `num_nodes` is now just 'nodes'.
     - It no longer supports node ranges (Use 'min_nodes')
   - Non-universal arguments are still in scheduler specific sections, but under `schedule`.
     See `pav show sched --config` for the full format.

In general, this means you should rename all 'slurm' sections in test/host/mode configs to
'schedule', and all 'num_nodes' to just 'nodes'. Some config values (like 'up_states') will need
to be moved as well. Also `nodes` no longer takes a range. Use `min_nodes` instead.

For example, given the following:

```
mytest:
  run:
    cmds: "echo 'I am an example!'"

  slurm:
    num_nodes: 50
    partition: foo
    up_states: ['IDLE', 'NOT_ON_FIRE']
```

would become:

```
mytest:
  run:
    cmds: "echo 'I am an example!'"

  schedule:
    nodes: 50
    partition: foo
    slurm:
        up_states: ['IDLE', 'NOT_ON_FIRE']
```

#### Scheduler Variables

Scheduler variables have also changed considerable.

  - Almost all are universal.
  - All slurm `alloc_*` variables are now `test_*`.
  - Most can be converted directly.

## 2.3 Release Notes
 - Added 'flatten_results' option to the base Pavilion config. Allows for
   producing a separate result log line for each 'per_file' result, making
   charting in Splunk significantly easier. The results can be significantly
   larger, however, expecially on large machines.
 - Added unified test run filtering in 'status' and 'result'.
 - Added the 'list' command for searching test runs and series
 - Added 'maint' command for maintanence operations.
 - Results now include a 'uuid' key - A completely unique id for each test.
 - Added spack options to test config. Allows for spack packages to be built
   and/or loaded inside of a pavilion test. Requires a valid spack path to
   be provided in the Pavilion config under the new key `spack_path`.
 - Modules may now have nested paths (True lmod module organization)
 - Updated permissions management.
 - Added test series files for organizing multiple test runs.
 - Added 'pav graph' command.
 - Added 'pav show hosts' and 'pav show modes' commands.
 - Switched to using epoch time for all timestamps.
 - Test run attributes are now indexed for faster 'pav status' and 'pav result'
   searches.
 - Backwards compatibility breaks
   - 'fn' and 'n' result fields have been merged into 'per_file'.
   - The 'status', and 'result' commands now search all available test runs
     and select recent (1 day) tests by the current user on the current
     machine. For the old behavior, use ``pav status last``, etc.
   - Old tests will lack some expected properties for searching,
     as that information has been consolidated into the 'attributes' file
     which they won't have.
   - 'test_node_list' in the results has been replaced with the much shorter
     'test_node_list_short' key.
   - The results command '-l' (--show-logs) option is now '-L'.
   - The results command '--json' argument and '--oneline' argments were removed.
     Use --summary instead.

## 2.2 Release notes
 - All new test config parser.
 - Variable references are now 'expressions'.
   - In addition to variables, they can contain math and functions.
     - `'hello {{world * 2}}'`
     - Assuming 'world' has a value of 5, would be: 'hello 10'
 - Iterations (formerly sub-strings) can now contain more than one variable.
   This results in all combinations of the variables being used.
   - `'Greetings: [~hello {{name}}{{num}}~,]'`
   - With `name: ['bob', 'janet']` and `num: [1,2]` would result in:
   - 'Greetings: hello bob1, hello bob2, hello janet1, hello janet2'
 - per test permissions settings
   ```yaml
   mytest:
     group: secret_group
     umask: '007'
   ```
 - Backwards compatibility breaks
   - All duplicate keys in configs are now detected and cause errors.
   - Default values may no longer be specified using '{{foo?}}'
   - Dashes are no longer allowed in Pavilion variable names.
     - `'{{name-len}}'`
   - You can no longer have nested 'sub-strings' (now called iterations).
   - Not everything can be escaped in Pavilion strings anymore. This is now
     limited to all normal python escapes
     (https://docs.python.org/2.0/ref/strings.html),
     plus '[', '{', and '~'.
   - The results section is now 'result_parse'
   - Result parsers plugins now have explicit argument default and validation settings,
     and the _check_args method must return the modified argument dictionary.
     (rather than None before)

   ```yaml
   mytest:
     variables:
       unit_base: 100
     result_parse:
       regex:
          speed:
            regex: 'speed (\d+)'
     result_evaluate:
       speed_normalized: "speed / {{unit_base}}"
       another_thing: '"some don''t string"'
   ```
   - build.source_location has changed.
   ```yaml
   mytest:
     build:
       source_path: mytest.zip
       source_url: http://github.com/hpc/mytest/latest.zip
       source_download: latest

 - Build are now built in their final location, rather than in a tmp
   location. This fixes the issues with symlinks (as they no longer
   move, so they can be absolute), and issues with non-relocatable
   builds.

 - New config keys:
   -
   ```yaml
   mytest:
     maintainer:
       name: Paul Ferrell
       email: pferrell@lanl.gov
    ```

## 2.1.2
 - Fixed some regressions from the 2.1.1 release.

## 2.1.1
 - Fixed speed of draw_table with new algorithm
 - Fixed some name conflicts with the 'per_fullname' and 'per_name' result parser options.
 - Fixed issue with pipe exceptions in pav results
 - Fixed issue with slurm scheduler node selection.
 - Fixed clean command exceptions
 - Updated reserved result parser key list. Renamed 'errors' to 'pav_result_errors'.
 - and many smaller bug fixes.
 - Updated yaml_config version.
 - Updated the ls and cat commands, merged the tree command into ls.
 - Moved build code into its own module
 - Added include/exclude_nodes option to slurm.
 - Added a 'build' command. Like the 'run' command, but only builds.
 - Added '--rebuild' option to both build and run (and the capability to do so)
 - Added better build tracking output and verbosity under the 'run' command.


## 2.1
 - Quite a few bugfixes.
 - Simplified time output (and dropped pytz dependency)
 - Updated regex parser.
 - Migrated documentation to Readthedocs and restructured text format.
   - Added API documentation
   - Added some tutorials
 - Add ls and cat commands
 - Moved exception tracking to a single log file.
 - Added the concept of hidden tests.
 - Re-organized how deferred variables work.
   - run.sh files are now regenerated when a test _run's
   - test variables are now saved in the test run directory.
 - Fixed RUN_COMPLETE files. They are now made when a test is finished,
   fails, is cancelled, and in most error cases.

## 2.0

 - All new Pavilion 2.0 redesign
   - Pavilion was completely redesigned and reimplemented for this release.