0.16.0

CHANGES

1. Proxy module refresh_nornir behavior updated to not do saltutil.sync_all and saltutil.refresh_pillar as call pillar.items updates pillar regardless, that should improve nr.nornir refresh calls speed, as pillar no longrer updated three times. Had to ad this change as new netbox pillar netbx-secretstore was failing due to salt was trying to refresh secrets simultaneously due to previous behaviour.
2. Dependencies updated and tested:

• pydantic 1.9.1 -> 1.10.2
• jmespath 1.0.0 -> 1.0.1
• nornir-scrapli 2022.1.30 -> 2022.07.30
• pygnmi 0.8.4 -> 0.8.9
• requests 2.27.1 -> 2.28.1
• scrapli 2022.1.30 -> 2022.07.30
• scrapli-community 2022.1.30 -> 2022.07.30
• scrapli-netconf 2022.1.30 -> 2022.07.30
• tabulate 0.8.9 -> 0.8.10
• lxml 4.8.0 -> 4.9.1
• jinja2 3.0.3 -> 3.1.2

FEATURES

1. Added new pillar module salt_nornir_netbox to source proxy minion pillar data from Netbox and to source hosts data from Netbox, this module has integration with netbox-secretstore plugin to source secrets from Netbox.
2. Execution module nr.test added support for Jinja2 templates test suites

0.15.0

CHANGES

1. Added pursenmp and pynetbox into "nr.nornir version" call output
2. Improved "\n" newline and "\n" escaped newline handling for inline salt cli arguments to address issue #21

FEATURES

1. Added execution module nr.netbox function to enable various interactions between Netbox and Nornir
2. Added support to honour "jinja_env" configuration for proxy minion rendering to address issue #21
3. Added "multiline" argument to nr.cfg_gen function to return either as single multiline string or a list of commands
4. Added support to supply any arbitrary data for rendering using "job_data" argument, that is to keep pillar data small,
   allowing to load any parameters from files on demand for rendering within templates
5. Added capability to define Nornir "user_defined" configuration in Salt-Nornir proxy pillar under "configuration" key.

TESTING

1. Tested 0.15.0 release with SaltStack 3005 - all 389 tests are passing.

0.14.0

FEATURES

1. Added nr.snmp execution module function to manage devices over SNMP

0.13.2

BUGS

1. Fixed execution module nr.cfg and nr.cfg_gen to accept "commands" argument
2. State module nr.workflow fixed handling of FL as a string in common filters, "FL": "ceos1" was not working before
3. State module nr.workflow hcache/dcache handling improved to only target hosts that have been matched by workflow filters, previously was clearing cache for all hosts unconditionaly

CHANGES

1. Test docker environment updated to start salt-api process for salt-master machine
2. Added tests for salt-api cherrypy
3. nrp3 and fakenos inventories updated to spun up 402 Arista fake hosts for stress testing
4. Improved state module nr.workflow logging

0.13.1

CHANGES

1. nr.nc execution module function improvement to render "rpc" argument value, added nr.nc custom render argument list - #11
2. Dependency update - PyGNMI from 0.6.9 to 0.8.4

0.13.0

FEATURES

1. Added nr.diagram function to Salt-Nornir Runner plugin to produce L2, L3/IP, OSPF, ISIS network diagrams in yed, drawio or v3d format using N2G module
2. Execution module nr.nornir refresh added support for workers_only=True to refresh only nornir workers and their inventory to address issue

CHANGES

1. Runner module changed the way to collect job results using client.get_cli_returns function as it automatically checks jobs cache for results in addition to listening for job return events
2. Added N2G 0.3.* to optional dependencies in poetry pyproject.toml file
3. Added textfsm == 1.1.2 dependency, 1.1.3 version was failing to install due to lack of six package
4. Dependencies update:

• Ncclient from 0.6.9 to 0.6.13
• Netmiko from 3.4.1 to 4.1.1
• NAPALAM from 3.4.0 to 4.0.0
• nornir-netmiko from 0.1.0 to 0.2.0
• nornir-napalm from 0.2.0 to 0.3.0
• TTP from 0.8.4 to 0.9.*
• TTP-Templates from 0.1.* to 0.3.*
• NTC-Templates from 1.7.0 to 3.0.0
• PyATS and Genie updated to install using 'full' extras to address issue

5. Enhancement for error handling for jobs tat called while proxy process refreshing, this is to address issue

BUGS

1. Fixed nr.nornir dir function as it was broken after adding Pydantic models schema, it was referring to non existing list, fix is to use enum object to list availabel functions
2. Execution module nr.file function was failing if no args provided, fixed it to check args length before trying to extract values

0.12.0

CHANGES

0. Deprecated Python 3.6 support, pinned requirement for Python >=3.7,<4.0
1. Updated dependency - Nornir from 3.2.0 to 3.3.0
2. Updated dependency - jmespath from 0.10.0 to 1.0.0
3. Updated dependency - NAPALM from 3.3.1 to 3.4.1
4. Updated dependency - nornir-napalm from 0.1.2 to 0.2.0
5. Updated dependency - paramiko from 2.9.2 to 2.11.0
6. Updated dependency - pygnmi from 0.6.8 to 0.6.9
7. Updated dependency - xmltodict from 0.12.0 to 0.13.0
8. Pinned lxml dependency to version 4.8.0
9. Added dependency extras - netmiko, napalm, scrapli, pyats, netconf, gnmi, restconf, dataprocessor - to simplify installation using pip install nornir-salt[xyz]
10. Moved development environment to rockylinux, removed centos7 docker related files
11. Updated dependency - scrapli from 2021.7.30 to 2022.1.30

0.11.0

Major change in this release - introduction of Pydantic models to validate execution/sate/runner modules arguments to implement fail fast approach and significantly improve overall reliability as all arguments are now validated by Pydantic ensuring semantic and functional validity.

Another change are a switch to Poetry for dependency management and switch to Rocky Linux with Python3.9 as a main development and testing environment.

CHANGES

1. Added pydantic==1.9.0 to base requirements
2. Added these pydantic models to verify execution/state/runner modules function arguments using yangdantic module from nornir-salt:

• model_exec_nr_cli
• model_exec_nr_task
• model_exec_nr_cfg
• model_exec_nr_tping
• model_exec_nr_test
• model_exec_nr_nc
• model_exec_nr_http
• model_exec_nr_do
• model_exec_nr_file
• model_exec_nr_learn
• model_exec_nr_find
• model_exec_nr_diff
• model_exec_nr_nornir_fun
• model_exec_nr_gnmi
• model_state_nr_workflow
• model_runner_nr_inventory
• model_runner_nr_call
• model_runner_nr_event
• model_runner_nr_cfg
• model_nornir_config

3. Proxy Module uses nornir-salt modelTestsProcessorSuite Pydantic model to verify tests suite items prior to running them, that way nr.test will fail immediately if any of the test items is not compliant
4. nr.do changed to omit searching for actions in master config, minion config and grains, it only sources them from pillar or text files from master, this is to speed up nr.do dir calls and overall nr.do actions execution
5. Added multiprocessing lock to files download, now only one worker at a time can download files from master
6. File downloading from master with cp.get_url was updated to use files cached on Minion, instead of downloading files over and over again from master if content is the same.
7. Moved dependency management to Poetry, removed all requirements files replacing them with pyproject.toml definition
8. For test environment added fakenos container to run scale/performance tests
9. Moved test environment to Rocky Linux container running Python 3.9 - all development will be done using that setup
10. Runner nr.call makes use of _get_hosts_minions function to get a list of hosts and proxy minion to target before running the task, that is to ensure that we only targeting alive proxy minions without unnecessary retry attempts on disconnected.downed minions
11. Test/Development containers environment updated to run on Rocky Linux using Python 3.9

FEATURES

1. Added Pydantic package and Python versions to nr.nornir version report
2. Introduced Pydantic models for Salt-Nornir modules functions' input validation, that is to make sure that parameters supplied comply with required format - increase Salt-Nornir proxy robustness/reliability and enforce fail-fast approach
3. Salt-Nornir updated to use Poetry for packaging and dependency management and for virtual env management
4. Added tree output to runner nr.inventory to visualize output for minions, devices and machines where they are hosted
5. Added model_nornir_config pydantic model to run configuration parameters verification prior to starting proxy minion

BUGS

1. Fixed issue #9 bug with simultaneous files download from master
2. When doing 'nr.file rm filegrouop_name' was getting tf_index file overwritten probably by various task threads trying to remove files simultaneously, introduced file lock support mechanism passing it onto the 'files' task and 'ToFileProcessor' from proxy minion
3. Fixed nr.cfg runner to honor Fx filters while filtering hosts to configure
4. nr.cfg runner fixed batches calculations to account for cases when using odd numbers of hosts and of batch size e.g. 7 / 3 should produce 3 batches while previously was resulted in 2 batches

0.10.1

CHANGES

1. Cleaned up base installation requirement leaving only nornir, nornir-salt and psutil.
2. Added installation flavors:

• dev - packages for development
• prodmin - minimum set of tested packages for production use to interact with devices over NETCONF, SSH or RESTCONF using Ncclient, Netmiko and requests respectively
• prodmax - set of tested packages to incorporate support of all salt-nornir supported features

3. Nornir Proxy Module render and download function changed, now, instead of throwing error and stopping altogether, new behavior is to continue with task execution but exclude failed hosts, once task completed, failed hosts added back to the results with exception details
4. Previously, when calling nr.cfg_gen, nr.cfg or nr.cli with filename argument pointing to non existing file it returns traceback, now it returns errors on a per-host basis and continue execution for non failed hosts. This is the result of implementing change 3.
5. nr.workflow updated to include hard requirement for global filter options to be set if proxy minion nornir_filter_required parameter is True, if nornir_filter_required is True, and no filters given in workflow global options, it will fail to target any hosts and will exit with error.
6. Updated runner behavior to identify hosts to minions mapping before sending tasks out, that allows to target only minions that actually manage given hosts.
7. Runner nr.event replaced mode argument with progress argument
8. Execution module - added user key to job identity dictionary to include it in events emitted by SaltEventProcessor.
9. State module, updated to generate uuid jid for the tasks for jobs tracking, by default state modules does not get jid id assigned, also, extracted user name value from opts to include it in identity dictionary.
10. nr.event runner extended to display user name for the progress bars and logs modes
11. Dependency updated nornir-scrapli==2022.1.30a1, previously was nornir-scrapli==2021.7.30, reason - scrapli-netconf==2022.1.30a1 fails to work with nornir-scrapli==2021.7.30 netconf tasks if doing custom tasks.
12. Changed requirement from psutil==5.8.* to psutil>=5.8.*, <=5.9.* to allow newer psutil version, tested it with 3002 and 3004 SaltStack versions

BUGS

1. nr.workflow state module updated to exit with comment if global filters option provided but matched no hosts, previously it was continuing with execution returning empty report
2. Change 3 fixed nr.workflow if first step is to get file from master and it fails for the host, this error was not included in report.
3. Change 3 fixed nr.workflow if first step fails and it targeted only one host, empty report produced
4. Nornir Proxy module list_hosts function updated to honor nornir_filter_required parameter to raise an error if no filters provided but nornir_filter_required set to true.
5. Fixing bug 4 fixed issue with nr.workflow when it was continuing on with tasks execution when nornir_filter_required was True but no global filters provided in options.
6. Runner nr.event was fixed to better handle events for failed hosts, previously it was failing with key error
7. Runner fixed bug to raise command execution error when no minions matched by tgt and tgt_type
8. Fixed docs spelling using pylint spelling check plugin (pyenchant) and sphinxcontrib-spelling RST plugin
9. When was running tasks with runner other than RetryRunner netmiko_send_command failed with - TypeError: send_command() got an unexpected keyword argument 'connection_name', added check if not using RetryRunner to remove connection_name argument from task kwargs.

FEATURES

1. Added support for nr.workflow to produce text tables in github, jira or some pretty format out of workflow report summary section using tabulate module.
2. Proxy module render function updated to support arguments that refer to dictionary keyed by host name with values being config templates to render.
3. Added new runner function nr.cfg that supports loading devices configuration from directory with files or dictionary, it also supports interactive mode allowing operator to control tasks execution.
4. Updated proxy module and SaltEvetProcessor to send worker_id in events
5. Created runner nr.make_plugin function to create boilerplate code for various Nornir Plugins, added Nornir task and test - TestsProcessor custom function
6. Runner module added bars progress mode to nr.event, as a result nr.call and nr.cfg also support bars mode. bars mode uses Rich library progress bars.
7. nr.workflow added support to specify common step kwargs in options section, this is to simplify adding common arguments to all steps, like event_progress
8. Added new stats - tasks_completed and tasks_failed counters, as well as worker_tasks_completed and worker_tasks_failed to nr.nornir stats and nr.nornir workers stats
   functions output respectively.
9. Execution module table brief and terse format now support headers argument e.g. table=brief headers="host, result"
10. Added nr.nornir connect function to be able to connect to devices on demand, as well as to retry various connection parameters such as credentials, port number etc. using reconnect argument list
11. Execution module tasks added support for RetryRunner task parameters - run_connect_retry, run_task_retry, run_creds_retry, run_num_workers, run_num_connectors to influence Nornir task run logic
12. Replaced cp.get_file_str with cp.get_url for downloading functions to allow sourcing files from any of supported URL schemes: salt://, http://, https://, ftp://, s3://, swift:// and file:// (local filesystem). Updated proxy module and execution modules to provide that support.

0.9.0

Significant changes in this release to improve task execution throughput and unveil new capabilities to manage network devices inspired by tbotnz's netpalm project workers handling.

BUGS

1. Fixed nr.nornir unknown function call - string formatter was broken
2. Fixed nr.test handling of integer patterns converting the to string
3. Fixed bug with 'nr.nornir refresh' function not updating proxy parameters
4. Fixed proxy_always_alive is False, connection through the proxy does not terminated completely and becomes unusable after first task
5. Fixed Salt3002 local client destroy method call

FEATURE

1. Added support to execution module for tf_skip_failed ToFileProcessor argument to not save results for failed tasks
2. Added support for multiple Nornir Worker instances to improve task execution throughput
3. Added proxy parameter nornir_workers to control the number of Nornir Worker Instances, default is 3
4. Added support for worker execution module argument to delegate task execution to certain Nornir worker
5. Added nr.nornir workers stats function to retrieve workers stats
6. Added dir function to nr.nornir to list supported functions
7. Added new proxy connections_idle_timeout to control for how long devices' connections can stay open, by default it never timeouts
8. Added queues_utils function to proxy module
9. Added workers_utils function to proxy module
10. Added results_queue_dump to nr.nornir execution module function

CHANGES

1. Due to introduction of multiple Nornir workers had to make these changes:

• Task identity now includes worker number if worker=all
• hcache and dcache handling updated to make sure all worker instances have same data cached
• nr.nornir inventory add/remove/read hosts done for all workers by default now
• nr.nornir connections returns connections data for all workers by default now
• each worker tracks its own job stats, global stats computed using per-worker stats now

2. Updated requirements to latest releases for test environment:

• paramiko==2.9.2 was 2.7.2
• nornir-utils - removed
• nornir-netmiko==0.1.2 was 0.1.1
• nornir==3.2.0 was 3.1.1
• genie==22.1 was 21.10
• pyats==22.1 was 21.10
• pygnmi==0.6.8 was 0.5.3
• ttp==0.8.4 was 0.8.2
• ttp-templates==0.1.3 was 0.1.2
• psutil==5.9.0 was 5.9.0
• pyyaml==6.0 was 5.4.1

3. Execution module nr.learn updated to use tf_skip_failed=True by default to not save failed tasks