commit command

README.MD

@@ -2,42 +2,37 @@
 
 *OPNsense CLI* is a command-line utility for FreeBSD, Linux, MacOS and Windows that empowers administrators and power users to manage, configure, and monitor OPNsense firewall systems. The CLI provides an alternative method to browser-based GUI to interact with the firewall system.
 
-## Features and Benefits
-- *Versatility*: Can operate both locally and remotely (via SSH),and is suitable for various deployment scenarios.
-- *Transparency and Control*: All opnsense-cli Commands are bash scripts (and not API calls), with interactive confirmation for changes (bypassable with the --force flag).
-- *Cross-Platform Support*: Works with macOS, Windows, Linux, and OpenBSD.
-- *Streamlined Operations*: Facilitates repeatable configurations, troubleshooting and complex automations.
+[Why this thing exists?](/doc/scope.md)
 
-## Why Use opnsense CLI?
-
-Administrators  typically only two options how to manage the firewall: either through browser-based GUI, or using FreeBSD commands in bash. The opnsense CLI utility provides the middle option by using command line to quickly perform common tasks:
-
-- A quick view of firewall settings such as configuration, backups, system vitals.
-- Controlled changes of `conf/config.xml`, including staging and rollback options.
-- Controlled execution of any OPNsense command available through `configctl`.
 
 ## Usage
 
 `opnsense [flags] command [parameters]`
 
 ### Commands
 
-- **`sysctl [<xpath>]`**: Retrieves system information from the firewall.
-- **`show [<xpath>]`**: Displays xpath segment of config.xml.
-- **`backups [<backup.xml>]`**: Lists available backup configs or displays a specific backup.
-- **`run <service> <command>`**: Executes commands on OPNsense.
-- **`set <xpath> value <value>`**: Sets a value of a specific node in staging.xml file.
-- **`commit`**: Moves staging.xml to active config.xml.
-- **`compare [<staging.xml>] [<config.xml>]`**: Compares two config files.
-- **`discard [<xpath>]`**: Discards a value (or all changes) in the staging.xml.
-- **`save [<backup.xml>]`**: Creates a new backup.xml.
-- **`load [<backup.xml>]`**: Restores config.xml from a specific backup.xml. (alias: `restore`)
+- **`show [<xpath>]`**: Displays config.xml or the Xpath segment in it
+- **`compare [<staging.xml>] [<config.xml>]`**: Compares two config files
+- **`set <xpath> [value] [(attribute)]`**: Adds a new branch, value and/or attribute
+- **`set <xpath> [value] [(attribute)] -d`**: Deletes branch, value and/or attribute
+- **`discard [<xpath>]`**: Discards a value (or all changes) in the 'staging.xml'
+
+- **`commit`**: Moves staging.xml to active 'config.xml'
+
+- **`export [<source.xml>] [<target.xml>]`**: Extracts a patch file
+- **`import [patch.xml]`**: Reads provided XML patch and injects it into 'staging.xml'
+
+- **`backup [<backup.xml>]`**: Lists available backup configs or displays a specific backup
+- **`restore [<backup.xml>]`**: Restores config.xml from a specific backup.xml. (alias: `load`)
+- **`save [<file.xml>]`**: Creates a new /conf/backup/file.xml
 - **`delete <backup.xml>`**: Deletes a specific backup.xml.
 - **`delete age [days]`**: Deletes all backups older than specified days
 - **`delete keep [count]`**: Keeps specified number of backups and deletes the rest
 - **`delete trim [count]`**: Deletes number of the oldest backups
-- **`set <xpath> [value] [(attribute)]`**: Adds a new branch, value and/or attribute
-- **`set <xpath> [value] [(attribute)] -d`**: Deletes branch, value and/or attribute
+
+- **`sysinfo [<xpath>]`**: Retrieves system information from the firewall
+- **`run <service> <command>`**: Executes commands on OPNsense.
+
 
 ### Flags
 

cmd/commit.go

@@ -42,7 +42,7 @@ var commitCmd = &cobra.Command{
 			fmt.Println("no staging.xml detected - nothing to commit.")
 			return
 		}
-		bash = `diff -q "` + configfile + `" "` + stagingfile + `" >& /dev/null && echo "same" || echo "diff"`
+		bash = `cmp -s "` + configfile + `" "` + stagingfile + `" && echo "same" || echo "diff"`
 		filesame := internal.ExecuteCmd(bash, host)
 		if strings.TrimSpace(filesame) != "diff" {
 			fmt.Println("staging.xml and config.xml are the same - nothing to commit.")
@@ -55,16 +55,43 @@ var commitCmd = &cobra.Command{
 		fmt.Println("\nChanges to be commited:")
 		internal.PrintDocument(deltadoc, "opnsense")
 
-		internal.Log(2, "commiting %s to %s", stagingfile, configfile)
+		internal.Log(2, "commiting %s to %s and reloading all services", stagingfile, configfile)
 
 		// copy config.xml to /conf/backup dir
 		backupname := internal.GenerateBackupFilename()
 		bash = `sudo cp -f ` + configfile + ` /conf/backup/` + backupname + ` && sudo mv -f /conf/staging.xml ` + configfile
 		internal.ExecuteCmd(bash, host)
 
-		fmt.Println("time to reload OPNSense!")
-
-		//TODO: run php /usr/local/etc/rc.reload_all
+		include := "php -r \"require_once('/usr/local/etc/inc/config.inc'); require_once('/usr/local/etc/inc/interfaces.inc'); require_once('/usr/local/etc/inc/filter.inc'); require_once('/usr/local/etc/inc/auth.inc'); require_once('/usr/local/etc/inc/rrd.inc'); require_once('/usr/local/etc/inc/util.inc'); require_once('/usr/local/etc/inc/system.inc'); require_once('/usr/local/etc/inc/interfaces.inc'); "
+
+		var result string
+
+		result = internal.ExecuteCmd(include+"system_firmware_configure(true); \"", host)
+		fmt.Println(result)
+		result = internal.ExecuteCmd(include+"system_trust_configure(true); \"", host)
+		fmt.Println(result)
+		result = internal.ExecuteCmd(include+"system_login_configure(true); \"", host)
+		fmt.Println(result)
+		result = internal.ExecuteCmd(include+"system_cron_configure(true); \"", host)
+		fmt.Println(result)
+		result = internal.ExecuteCmd(include+"system_timezone_configure(true); \"", host)
+		fmt.Println(result)
+		result = internal.ExecuteCmd(include+"system_hostname_configure(true); \"", host)
+		fmt.Println(result)
+		result = internal.ExecuteCmd(include+"system_resolver_configure(true); \"", host)
+		fmt.Println(result)
+		result = internal.ExecuteCmd(include+"interfaces_configure(true); \"", host)
+		fmt.Println(result)
+		result = internal.ExecuteCmd(include+"system_routing_configure(true); \"", host)
+		fmt.Println(result)
+		result = internal.ExecuteCmd(include+"rrd_configure(true); \"", host)
+		fmt.Println(result)
+		result = internal.ExecuteCmd(include+"filter_configure(true); \"", host)
+		fmt.Println(result)
+		result = internal.ExecuteCmd(include+"plugins_configure('vpn', true); \"", host)
+		fmt.Println(result)
+		result = internal.ExecuteCmd(include+"plugins_configure('local', true); \"", host)
+		fmt.Println(result)
 
 	},
 }

cmd/export.go

@@ -75,11 +75,12 @@ var exportCmd = &cobra.Command{
 		olddoc := internal.LoadXMLFile(oldconfig, host, false)
 		newdoc := internal.LoadXMLFile(newconfig, host, true)
 		if newdoc == nil {
-			newdoc = olddoc
+			newdoc = olddoc.Copy()
 		}
 
 		deltadoc := internal.DiffXML(olddoc, newdoc, false)
 		internal.RemoveChgSpace(deltadoc.Root())
+
 		output := internal.ConfigToXML(deltadoc, path)
 		fmt.Print(output)
 	},

cmd/import.go

@@ -16,84 +16,76 @@ limitations under the License.
 package cmd
 
 import (
-	"bufio"
 	"fmt"
-	"io"
 	"os"
 
 	"github.com/beevik/etree"
 	"github.com/mihakralj/opnsense/internal"
 	"github.com/spf13/cobra"
 )
 
-var execute bool
-
 // compareCmd represents the compare command
 var importCmd = &cobra.Command{
-	Use:   "import",
+	Use:   "import [patch.xml]",
 	Short: `Import XML patch and stage it for configuraiton change`,
 	Long: `The 'import' command allows bulk import of configuration changes by injecting an XML patch file that specifies what to add, or delete in the current configuration. Patch file is in the standard XML format generated by the 'export' command, using namespace tags indicating the type of change (e.g., add:, del:).
 
-The command reads the patch file from the standard input. You can pipe the patch file into this command:
-
-  cat patch.xml | opnsense import
-
-Or insert patch using I/O redirection:
-
-  opnsense import < patch.xml
-
-Once the patch is imported, it is added to currently staged changes in 'staging.xml'. You can review these changes using 'opnsense compare -c'  and apply them using 'opnsense commit' when ready.`,
+Once the patch is imported, it is added to currently staged changes in 'staging.xml'. You can review staged changes using 'opnsense compare --compact'  and apply them using 'opnsense commit' when ready.`,
 
 	Run: func(cmd *cobra.Command, args []string) {
+		if !cmd.Flag("depth").Changed {
+			depth = 5
+		}
 		internal.SetFlags(verbose, force, host, configfile, nocolor, depth, xmlFlag, yamlFlag, jsonFlag)
 
 		patchdoc := etree.NewDocument()
 
-		stat, _ := os.Stdin.Stat()
-		if (stat.Mode() & os.ModeCharDevice) == 0 {
-			reader := bufio.NewReader(os.Stdin)
-			var output []rune
+		if len(args) > 0 {
+			//check parameters
+			if len(args) > 0 {
+				importfilename := args[0]
+				if _, err := os.Stat(importfilename); os.IsNotExist(err) {
+					internal.Log(1, "import file %s does not exist\n", importfilename)
+				}
+				// Read file contents into a string
+				fileContents, err := os.ReadFile(importfilename)
+				if err != nil {
+					internal.Log(1, "failed to read file %s: %v\n", importfilename, err)
+					return
+				}
+				fileString := string(fileContents)
 
-			for {
-				input, _, err := reader.ReadRune()
-				if err != nil && err == io.EOF {
-					break
+				err = patchdoc.ReadFromString(fileString)
+				if err != nil {
+					internal.Log(1, "%s is not an XML file", importfilename)
 				}
-				output = append(output, input)
-			}
-			err := patchdoc.ReadFromString(string(output))
-			if err != nil {
-				internal.Log(1, "received stdin is not in XML format")
 			}
-
 		} else {
-			internal.Log(1, "No data received on stdin, please pipe the XML file into this command")
+			internal.Log(1, "No patch XML file provided")
 		}
 
 		internal.Checkos()
 		configdoc := internal.LoadXMLFile(configfile, host, false)
+
 		stagingdoc := internal.LoadXMLFile(stagingfile, host, true)
 		if stagingdoc == nil {
-			stagingdoc = internal.LoadXMLFile(stagingfile, host, true)
+			stagingdoc = configdoc.Copy()
 		}
 
 		internal.PatchElements(patchdoc.Root(), stagingdoc)
 		deltadoc := internal.DiffXML(configdoc, stagingdoc, false)
 
-		if !execute {
-			fmt.Println("Preview of modifications scheduled for imported into staging.xml:")
-		}
+		fmt.Println("Preview of patch scheduled for imported into staging.xml:")
+
 		internal.PrintDocument(deltadoc, "opnsense")
 
-		if execute {
-			internal.SaveXMLFile(stagingfile, stagingdoc, host, true)
-			fmt.Println("\nModifications imported into staging.xml")
-		}
+		internal.Log(2, "importing patch into staging.xml")
+		internal.SaveXMLFile(stagingfile, stagingdoc, host, true)
+		fmt.Println("\nModifications imported into staging.xml")
 	},
 }
 
 func init() {
-	importCmd.Flags().IntVarP(&depth, "depth", "d", 1, "Specifies number of depth levels of returned tree (default: 1)")
-	importCmd.Flags().BoolVarP(&execute, "execute", "e", false, "Apply the changes to the staging.xml, rather than just previewing them.")
+	importCmd.Flags().IntVarP(&depth, "depth", "d", 5, "Specifies number of depth levels of returned tree (default: 5)")
 	rootCmd.AddCommand(importCmd)
 }

cmd/root.go

@@ -24,7 +24,7 @@ import (
 )
 
 var (
-	Version     string = "0.13.0"
+	Version     string = "0.14.0"
 	verbose     int
 	force       bool
 	host        string
@@ -84,8 +84,8 @@ To streamline remote operations, add your private key to the SSH agent using 'ss
 }
 
 func Execute() {
-	rootCmd.CompletionOptions.DisableDefaultCmd = true
-	rootCmd.CompletionOptions.DisableNoDescFlag = true
+	//rootCmd.CompletionOptions.DisableDefaultCmd = true
+	//rootCmd.CompletionOptions.DisableNoDescFlag = true
 	if err := rootCmd.Execute(); err != nil {
 		fmt.Fprintf(os.Stderr, "Whoops. There was an error while executing CLI '%s'", err)
 		os.Exit(1)

cmd/set.go

@@ -62,13 +62,12 @@ The XPath parameter offers node targeting, enabling you to navigate to the exact
 		if !strings.HasPrefix(path, "opnsense/") {
 			path = "opnsense/" + path
 		}
-		path = strings.ReplaceAll(path, "[", ".")
-		path = strings.ReplaceAll(path, "]", "")
-
 		if matched, _ := regexp.MatchString(`\[0\]`, path); matched {
 			internal.Log(1, "XPath indexing of elements starts with 1, not 0")
 			return
 		}
+		// convert list targeting in path from item[1] to item.1
+		path = internal.EnumeratePath(path)
 
 		var attribute, value string
 		if len(args) == 2 {
@@ -98,16 +97,19 @@ The XPath parameter offers node targeting, enabling you to navigate to the exact
 			}
 		}
 
+		// stagingdoc is converted to enumeratedXML, path is converted to enumerated path
 		element := stagingdoc.FindElement(path)
 		if !deleteFlag {
 			if element == nil {
 				element = stagingdoc.Root()
 				parts := strings.Split(path, "/")
+
 				for i, part := range parts {
 					part = fixXMLName(part)
 					if i == 0 && part == "opnsense" {
 						continue
 					}
+
 					if element.SelectElement(part) == nil {
 						if element.SelectElement(part+".1") != nil {
 							var maxIndex int
@@ -183,8 +185,12 @@ The XPath parameter offers node targeting, enabling you to navigate to the exact
 			}
 		}
 
+		path = internal.ReverseEnumeratePath(path)
+
+		internal.ReverseEnumerateListElements(configdoc.Root())
+		internal.ReverseEnumerateListElements(stagingdoc.Root())
+
 		deltadoc := internal.DiffXML(configdoc, stagingdoc, true)
-		//internal.FullDepth()
 
 		internal.PrintDocument(deltadoc, path)
 		internal.SaveXMLFile(stagingfile, stagingdoc, host, true)

doc/scope.md

@@ -0,0 +1,40 @@
+# opnsense-cli features
+
+### Scope and intent
+
+There is a gap between using OPNsense web GUI that offers fail-safe (but limited) configuration capabilities and using FreeBSD command terminal that offers direct access to all functionality of FreeBSD and OPNsense but exposes a great risk of messing things up for anyone that is not well versed in shell commands.
+
+__opnsense-cli__ utility bridges this gap by providing command-line access to local or remote OPNsense firewall. For remote access, it requires `ssh` service to be enabled, as it uses ssh to communicate with the firewall. Every action of __opnsense-cli__ is translated to a shell command that is then executed on OPNsense.
+
+### Features and Benefits
+- **Versatility**: Can operate both locally and remotely (via ssh),and is suitable for various deployment scenarios.
+- **Transparency and Control**: All opnsense-cli Commands are translated to shell scripts (not API calls), with interactive confirmation for critical changes (bypassable with the --force flag).
+- **Cross-Platform Support**: Works on macOS, Windows, Linux, and OpenBSD.
+- **Streamlined Operations**: Facilitates repeatable configurations, troubleshooting and complex automations.
+
+### Mechanics
+
+__opnsense-cli__ is focusing on `config.xml` manipulation of OPNsense. All configuration settings are stored in `config.xml` file and OPNSense web GUI actions primarily change data in config XML elements. To protect the integrity of configuration, __opnsense-cli__ is not changing `config.xml` directly - all changes are staged in a separate `staging.xml` file. Configuration elements can be added, removed, modified, discarded and imported - all changes will impact only `staging.xml` until 'commit' command is issued. That's when __opnsense-cli__ will create a backup of `config.xml` and replace it with content from `staging.xml`.
+
+__opnsense-cli__ is also providing commands to manage backup copies in `/conf/backup` directory of OPNsense. It can show all available backups, display details of a specific backup file (including XML diffs between backup file and config.xml), save, restore, load and delete backup files. It can trim number of backup files based on age and desired count of files in the directory.
+
+__opnsense-cli__ also offers (very basic) system management commands. `sysinfo` will display core information about OPNsense instance, `run` command will list and execute all commands that are available through __configctl__ process on OPNsense.
+
+### using ssh identity with __opnsense-cli__
+
+When connecting remotely to OPNsense using ssh, __opnsense-cli__ will try to use private key stored in `ssh-agent` to authenticate. Only when no identities are present or match the public key on OPNsense server, the fallback to *password* will be initiated. As __opnsense-cli__ stores no data locally, the password request will pop-up every time when __opnsense-cli__ initiates the ssh call. Very annoying.
+
+To use ssh identity, both server and client need to be configured with the access key. OPNsense server requires the public key in the format `ssh-rsa AAAAB3NC7we...wIfhtcSt==` and is assigned to a specific user (under System/Access/Users) in the field 'Authorized keys'.
+
+Client needs to support `ssh-agent` and accepts the private key in the format:
+```
+-----BEGIN RSA PRIVATE KEY-----
+[BASE64 ENCODED DATA]
+-----END RSA PRIVATE KEY-----
+```
+The command to add the private key to `ssh-agent`:
+```
+eval "$(ssh-agent -s)"  # Start the ssh-agent in the background
+ssh-add id_rsa          # Add your SSH private key to the ssh-agent
+```
+