Support pillar merging for list items

[mikemol]

Zabbix agent pillar data structure doesn't support pillar merging for things like Server, ServerActive or UserParameter. See saltstack/salt#28394.

For example, looking an excertp of pillar.example:

zabbix-agent:
  server:
    - localhost
  serveractive:
    - localhost
  userparameters:
    - net.ping[*],/usr/bin/fping -q -c3 $1 2>&1 | sed 's,.*/\([0-9.]*\)/.*,\1,'
    - custom.vfs.dev.discovery,/usr/local/bin/dev-discovery.sh

Anywhere you use - to build a set, Saltstack won't perform pillar merge.

If you structure the data like this:

zabbix-agent:
  server:
    localhost:
  serveractive:
    localhost:
  userparameters:
    'net.ping[*]': /usr/bin/fping -q -c3 $1 2>&1 | sed 's,.*/\([0-9.]*\)/.*,\1,'
    custom.vfs.dev.discovery: /usr/local/bin/dev-discovery.sh

then pillar data merges can happen, allowing Zabbix agent configuration to be composed more easily.

[mchugh19]

Salt can perform pillar merging of lists natively. We are doing it currently, but it requires that you set the pillar_merge_lists config

At the moment, we have a list grain that holds a list of zabbix template names. During a highstate, applying the ntp state to a host also sets the template app ntp value in our monitoring_groups grain. At the end of the highstate run, if that grain was updated it sends an event to the salt-master. This kicks a reactor to have the master ensure that the minion is properly monitored in zabbix by setting/removing all host properties and templates using the corresponding zabbix states in salt.

In our case, all of our zabbix userparameter configs are placed inside the state doing the provisioning. So setting the ntp state for a host, might also add any zabbix-agent userparameters needed to monitor that service. Thus we have to use the pillar_merge_list option to aggregate the various state's zabbix-agent UserParameters.