v1.6.1

README.md

@@ -133,6 +133,9 @@ Contents:
 		    - [(( x509genkey(spec) ))](#-x509genkeyspec-)
 		    - [(( x509publickey(key) ))](#-x509publickeykey-)
 		    - [(( x509cert(spec) ))](#-x509certspec-)
+		- [Wireguard Functions](#wireguard-functions)
+            - [(( wggenkey() ))](#-wggenkey-)
+        	- [(( wgpublickey(key) ))](#-wgpublickey-)
 	- [(( lambda |x|->x ":" port ))](#-lambda-x-x--port-)
 	    - [Positional versus Named Argunments](#positional-versus-named-arguments)
 	    - [Scopes and Lambda Expressions](#scopes-and-lambda-expressions)
@@ -147,6 +150,7 @@ Contents:
 		- [(( map[list|idx,elem|->dynaml-expr] ))](#-maplistidxelem-dynaml-expr-)
 		- [(( map[map|key,value|->dynaml-expr] ))](#-mapmapkeyvalue-dynaml-expr-)
 		- [(( map{map|elem|->dynaml-expr} ))](#-mapmapelem-dynaml-expr-)
+		- [(( map{list|elem|->dynaml-expr} ))](#-maplistelem-dynaml-expr-)
 		- [(( select[expr|elem|->dynaml-expr] ))](#-selectexprelem-dynaml-expr-)
 		- [(( select{map|elem|->dynaml-expr} ))](#-selectmapelem-dynaml-expr-)
 	- [Aggregations](#aggregations)
@@ -182,7 +186,7 @@ Contents:
 
 # Installation
 
-Official release executable binaries can be downloaded via [Github releases](https://github.com/mandelsoft/spiff/releases) for Darwin and Linux machines (and virtual machines).
+Official release executable binaries can be downloaded via [Github releases](https://github.com/mandelsoft/spiff/releases) for Darwin, Linux and PowerPC machines (and virtual machines).
 
 Some of spiff's dependencies have changed since the last official release, and spiff will not be updated to keep up with these dependencies.  Working dependencies are vendored in the `Godeps` directory (more information on the `godep` tool is available [here](https://github.com/tools/godep)).  As such, trying to `go get` spiff will likely fail; the only supported way to use spiff is to use an official binary release.
 
@@ -227,10 +231,16 @@ The ` merge` command offers several options:
   separate documen. The _yaml_ format uses as usual `---` as separator line.
   The _json_ format outputs a sequence of _json_ documents, one per line.
 
-- With `--select <field path>` is is possible to select a dedicated field of the
+- With `--select <field path>` it is possible to select a dedicated field of the
   processed document for the output
 
+- With `--evaluate <dynaml expression>` it is possible to evaluate a given dynaml
+  expression on the processed document for the output. The expression is evaluated
+  before the selection path is applied, which will then work on the evaluation
+  result.
+
 - The option `--state <path>` enables the state support of _spiff_. If the
+  given file exists it is put on top of the configured stub list for the
   given file exists it is put on top of the configured stub list for the
   merge processing. Additionally to the output of the processed document
   it is filtered for nodes marked with the [`&state` marker](#-state-).
@@ -598,7 +608,8 @@ Another way to compose lists based on expressions are the functions
 
 Any expression may be preluded by any number of explicit _scope literals_. A
 scope literal describes a map whose values are available for relative reference 
-resolution of the expression (static scope). 
+resolution of the expression (static scope). It creates an additional local
+binding for given names.
 
 A scope literal might consist of any number of field assignments separated by a
 comma `,`. The key as well as the value are given by expressions, whereas the
@@ -2160,7 +2171,7 @@ value: foobar
 The argument passed to this function must either be a reference literal or
 an expression evaluating to either a string denoting a reference or a string
 list denoting the list of path elements for the reference.
-If no argument is given, the actual field path is used.
+If no argument or an undefined (`~~`) is given, the actual field path is used.
 
 Alternatively the `merge` operation could be used, for example `merge foo.bar`. The difference is that `stub` does not merge, therefore the field will still be merged (with the original path in the document).
 
@@ -2771,8 +2782,12 @@ The following validators are available:
 | `ca`|  none | certificate for CA |
 | `type`| list of accepted type keys | at least one [type key](#-typefoobar-) must match |
 | `valueset` | list argument with values | possible values |
-| `value` | `=` | value | check dedicated value |
-| `match` | `~=` | regular expression | string value matching regular expression |
+| `value` or `=` | value | check dedicated value |
+| `gt` or `>` | value | greater than (number/string) |
+| `lt` or `<` | value | less than (number/string) |
+| `ge` or `>=` | value | greater or equal to (number/string) |
+| `le` or `<=` | value | less or equal to (number/string) |
+| `match` or `~=` | regular expression | string value matching regular expression |
 | `list` | optional list of entry validators | is list and entries match given validators |
 | `map` | [[ &lt;key validator&gt;, ] &lt;entry validator&gt; ] | is map and keys and entries match given validators |
 | `mapfield` | &lt;field name&gt; [ , &lt;validator&gt;] | required entry in map |
@@ -2845,8 +2860,8 @@ val: (( validate( 6, [|x,m|-> [x > m, "is larger than " m, "is less than or equa
 ```
 
 Just to mention, the validator specification might be given inline as shown
-in the examples above, but as reference expressions, also. The `and` and `or`
-validators accept deeply nested validator specifications.
+in the examples above, but as reference expressions, also. The `not`, `and` 
+and `or` validators accept deeply nested validator specifications.
 
 e.g.:
 
@@ -3452,6 +3467,50 @@ cert:
   validity: 99  # yepp, that's right, there has already time passed since the creation
 ```
 
+### Wireguard Functions
+
+spiff supports some useful functions to work with _wireguard_ keys.
+Please refer also to the [Useful to Know](#useful-to-know) section to find some
+tips for providing state.
+
+#### `(( wggenkey() ))`
+
+This function can be used generate private wireguard key. The result will
+base64 encoded.
+
+e.g.:
+
+```yaml
+keys:
+  key: (( wggenkey() ))
+```
+
+resolves to something like
+
+```yaml
+key: WH9xNVJuSuh7sDVIyUAlmxc+woFDJg4QA6tGUVBtGns=
+```
+
+#### `(( wgpublickey(key) ))`
+
+For a given key (for example generated with the [wggenkey](#-wggenkey-)
+function) this function extracts the public key and returns it again in base64 format-
+
+e.g.:
+
+```yaml
+keys:
+  key: (( wggenkey() ))
+  public: (( wgpublickey(key)
+```
+
+resolves to something like
+
+```yaml
+key: WH9xNVJuSuh7sDVIyUAlmxc+woFDJg4QA6tGUVBtGns=
+public: n405KfwLpfByhU9pOu0A/ENwp0njcEmmQQJvfYHHQ2M=
+```
+
 ## `(( lambda |x|->x ":" port ))`
 
 Lambda expressions can be used to define additional anonymous functions. They
@@ -4055,7 +4114,8 @@ keys:
 ### `(( map{map|elem|->dynaml-expr} ))`
 
 Using `{}` instead of `[]` in the mapping syntax, the result is again a map
-with the old keys and the new entry values.
+with the old keys and the new entry values. As for a list mapping additionally
+a key variable can be specified in the variable list.
 
 ```yaml
 persons:
@@ -4076,6 +4136,27 @@ older:
 
 An alternate way to express the same is to use `sum[persons|{}|s,k,v|->s { k = v + 1 }]`.
 
+### `(( map{list|elem|->dynaml-expr} ))`
+
+Using `{}` instead of `[]` together with a list in the mapping syntax, the result is again a map
+with the list elements as key and the mapped entry values. For this all list entries must be strings.
+As for a list mapping additionally an index variable can be specified in the variable list.
+
+```yaml
+persons:
+  - alice
+  - bob
+length: (( map{persons|x|->length(x)} ) ))
+```
+
+just creates a map mapping the list entries to their length:
+
+```yaml
+length:
+  alice: 5
+  bob: 3
+```
+
 ### `(( select[expr|elem|->dynaml-expr] ))`
 
 With `select` a map or list can be filtered by evaluating a boolean expression

cmd/merge.go

@@ -22,6 +22,7 @@ import (
 var asJSON bool
 var outputPath string
 var selection []string
+var expr string
 var split bool
 var processingOptions flow.Options
 var state string
@@ -63,6 +64,7 @@ func init() {
 	mergeCmd.Flags().StringVar(&bindings, "bindings", "", "yaml file with additional bindings to use")
 	mergeCmd.Flags().StringArrayVarP(&values, "define", "D", nil, "key/value bindings")
 	mergeCmd.Flags().StringArrayVar(&selection, "select", []string{}, "filter dedicated output fields")
+	mergeCmd.Flags().StringVar(&expr, "evaluate", "", "evaluation expression")
 }
 
 func createValuesFromArgs(values []string) (map[string]string, error) {
@@ -263,6 +265,24 @@ func merge(stdin bool, templateFilePath string, opts flow.Options, json, split b
 					log.Fatalln(fmt.Sprintf("cannot write state file %q", stateFilePath))
 				}
 			}
+
+			if len(expr) > 0 {
+				e, err := dynaml.Parse(expr, []string{}, []string{})
+				if err != nil {
+					log.Fatalln(fmt.Sprintf("invalid expression %q: %s", expr, err))
+				}
+				if m, ok := flowed.Value().(map[string]yaml.Node); ok {
+					binding := flow.NewNestedEnvironment(nil, "context", binding).WithLocalScope(m)
+					v, err := flow.Cascade(binding, yaml.NewNode(e, "<expr>"), flow.Options{})
+					if err != nil {
+						log.Fatalln(fmt.Sprintf("expression %q failed: %s", expr, err))
+					}
+					flowed = v
+				} else {
+					log.Fatalln("no map document")
+				}
+			}
+
 			if len(selection) > 0 {
 				new := map[string]yaml.Node{}
 				for _, p := range selection {
@@ -276,6 +296,7 @@ func merge(stdin bool, templateFilePath string, opts flow.Options, json, split b
 				}
 				flowed = yaml.NewNode(new, "")
 			}
+
 			if split {
 				if list, ok := flowed.Value().([]yaml.Node); ok {
 					for _, d := range list {

cmd/root.go

@@ -17,7 +17,7 @@ var cfgFile string
 var rootCmd = &cobra.Command{
 	Use:     "spiff",
 	Short:   "YAML in-domain templating processor",
-	Version: "v1.6.0",
+	Version: "v1.6.1",
 }
 
 // Execute adds all child commands to the root command and sets flags appropriately.
@@ -44,7 +44,11 @@ func ReadFile(file string) ([]byte, error) {
 		if err != nil {
 			return nil, fmt.Errorf("error getting [%s]: %s", file, err)
 		} else {
-			defer response.Body.Close()
+			if response.StatusCode != http.StatusOK {
+				defer response.Body.Close()
+				msg, _ := ioutil.ReadAll(response.Body)
+				return nil, fmt.Errorf("[status %d]: %s", response.StatusCode, msg)
+			}
 			return ioutil.ReadAll(response.Body)
 		}
 	} else {

dynaml/archive.go

@@ -55,7 +55,7 @@ func func_archive(arguments []interface{}, binding Binding) (interface{}, Evalua
 						return info.Error("%s", err)
 					}
 				default:
-					file, _, content, ok := getData(file, false, file, val.Value(), true)
+					file, _, content, ok := getData(file, WriteOpts{}, file, val.Value(), true)
 					if !ok {
 						return info.Error("invalid file content type %s", ExpressionType(val))
 					}
@@ -106,9 +106,10 @@ func getFileEntry(file *string, info map[string]yaml.Node) (*FileEntry, error) {
 	var e FileEntry
 	var err error
 
-	binary := false
+	wopt := WriteOpts{
+		Permissions: 0644,
+	}
 
-	e.mode = 0644
 	field, ok := info["path"]
 	if ok {
 		if field == nil || field.Value() == nil {
@@ -131,18 +132,12 @@ func getFileEntry(file *string, info map[string]yaml.Node) (*FileEntry, error) {
 		if field == nil || field.Value() == nil {
 			return nil, fmt.Errorf("mode field must not be nil")
 		}
-		switch v := field.Value().(type) {
-		case int64:
-			e.mode = v
-		case string:
-			e.mode, binary, err = getWriteOptions(v, e.mode)
-			if err != nil {
-				return nil, fmt.Errorf("permissions must be given as int or int string: %s", err)
-			}
-		default:
-			return nil, fmt.Errorf("mode field must be an integer, found %s", ExpressionType(field))
+		wopt, err = getWriteOptions(field.Value(), wopt, false)
+		if err != nil {
+			return nil, err
 		}
 	}
+	e.mode = wopt.Permissions
 
 	field, ok = info["base64"]
 	if ok {
@@ -163,7 +158,7 @@ func getFileEntry(file *string, info map[string]yaml.Node) (*FileEntry, error) {
 			if field == nil || field.Value() == nil {
 				return nil, fmt.Errorf("data field must not be nil")
 			}
-			_, _, content, ok := getData("", binary, e.path, field.Value(), true)
+			_, _, content, ok := getData("", wopt, e.path, field.Value(), true)
 			if !ok {
 				return nil, fmt.Errorf("invalid data field")
 			}

dynaml/cond.go

@@ -20,6 +20,7 @@ func (e CondExpr) Evaluate(binding Binding, locally bool) (interface{}, Evaluati
 
 	a, info, ok := ResolveExpressionOrPushEvaluation(&e.C, &resolved, &info, binding, false)
 	if !ok {
+		// fmt.Printf("*** COND failed: %s\n", info.Issue)
 		return nil, info, false
 	}
 	if !resolved {