Merge pull request #320 from gravitl/develop

Develop

advanced-client-install.rst

@@ -9,9 +9,9 @@ These steps should be run after the Netmaker server has been created and a netwo
 Introduction to Netclient
 ===============================
 
-At its heart, the netclient is a simple CLI for managing access to various WireGuard-based networks. It manages WireGuard on the host system, so that you don't have to. Why is this necessary?
+At its heart, the netclient is a simple CLI for managing access to various WireGuard-based networks. It manages WireGuard on the host system so that you don't have to. Why is this necessary?
 
-If you are setting up a WireGuard-based virtual network, you must configure each machine with very specific settings, so that every machine can reach it, and it can reach every other machine. Any changes to the settings of any one of these machines can break those connections. Any machine that is added, removed, or modified on the network requires reconfiguring of every peer in the network. This can be very time consuming.
+If you are setting up a WireGuard-based virtual network, you must configure each machine with very specific settings, so that every machine can reach it, and it can reach every other machine. Any changes to the settings of any one of these machines can break those connections. Any machine that is added, removed, or modified on the network requires reconfiguring of every peer in the network. This can be very time-consuming.
 
 The netmaker server holds configuration details about every machine in your network and how other machines should connect to it.
 
@@ -27,13 +27,13 @@ IPv6 requires a minimum MTU of 1280. A lot of router configurations expect a sta
 Notes on Windows
 ==================================
 
-If running the netclient on windows, you must download the netclient.exe binary and run it from Powershell as an Administrator.
+If running the netclient on Windows, you must download the netclient.exe binary and run it from Powershell as an Administrator.
 
 Windows will by default have firewall rules that prevent inbound connections. If you wish to allow inbound connections from particular peers, use the following command:
 
 ``netsh advfirewall firewall add rule name="Allow from <peer private addr>" dir=in action=allow protocol=ANY remoteip=<peer private addr>``
 
-If you want to allow all peers access, but do not want to configure firewall rules for all peers, you can configure access for one peer, and set it as a Relay Server(Professional Edition Feature) from Netmaker GUI. In order to achieve this, netmaker pro edition is required.
+If you want to allow all peers access, but do not want to configure firewall rules for all peers, you can configure access for one peer, and set it as a Relay Server(Professional Edition Feature) from Netmaker GUI. To achieve this, a netmaker pro edition is required.
 
 Running netclient commands
 ----------------------------
@@ -53,27 +53,27 @@ With some versions of WireGuard on Windows, high CPU utilization has been found
 Changing network profile to private
 ------------------------------------
 
-By default the netmaker network profile is added as a public network. This is the default behavior on Windows. For changing it to private, please run Powershell command,
+By default, the netmaker network profile is added as a public network. This is the default behaviour on Windows. To change it to private, please run the Powershell command,
 
 - `Set-NetConnectionProfile -InterfaceAlias 'netmaker' -NetworkCategory 'Private'`
 
 Issue after Windows sleep/hibernate
 -------------------------------------
 
-Sometimes the netclient does not work after the Windows wake up from sleep/hibernate. The root cause is not identified.  Restarting netclient service can fix the issue.
+Sometimes the netclient does not work after the Windows wake up from sleep/hibernation. The root cause is not identified.  Restarting the netclient service can fix the issue.
 
 Irregular netclient restart on Windows 2016 server
 ----------------------------------------------------
 
-There is one issue reported on Windows 2016 server.  The netclient restarted irregularly. The root cause is not identified. But as the feedback from the client, the issue can be fixed by disabling ISATAP adapter and 6to4 feature.
+There is one issue reported on the Windows 2016 server.  The netclient restarted irregularly. The root cause is not identified. However, as per the feedback from the client, the issue can be fixed by disabling the ISATAP adapter and 6to4 feature.
 
 - `Set-Net6to4configuration -state disabled`
 - `Set-Netisatapconfiguration -state disabled`
 - `Set-NetTeredoConfiguration -type disabled`
 
 Event id 0 in Windows Event logs
 ---------------------------------
-netclient service is delegated to Winsw on Windows. An issue is reported that the stop/start/restart events in Event logs show event id as 0 always. It does not impact any netclient functions.
+netclient service is delegated to Winsw on Windows. An issue is reported that the stop/start/restart events in Event logs show the event ID as 0 always. It does not impact any netclient functions.
 
 Modes and System Compatibility
 ==================================
@@ -85,23 +85,23 @@ The netclient can be run in a few "modes". System compatibility depends on which
 CLI
 ------------
 
-In its simplest form, the netclient can be treated as just a simple, manual, CLI tool, which a user can call to configure the machine. The cli can be compiled from source code to run on most systems, and has already been compiled for x86 and ARM architectures.
+In its simplest form, the netclient can be treated as just a simple, manual, CLI tool, which a user can call to configure the machine. The CLI can be compiled from source code to run on most systems and has already been compiled for x86 and ARM architectures.
 
-As a CLI, the netclient should function on any Linux or Unix based system that has the wireguard utility (callable with **wg**) installed.
+As a CLI, the netclient should function on any Linux or Unix-based system that has the wireguard utility (callable with **wg**) installed.
 
 Daemon
 ----------
 
 The netclient is intended to be run as a system daemon. This allows it to automatically retrieve and send updates. To do this, the netclient can install itself as a systemd service, or launchd/windows service for Mac or Windows.
 
-If running the netclient on non-systemd linux, it is recommended to manually configure the netclient as a daemon using whatever method is acceptable on the chosen operating system.
+If running the netclient on non-systemd Linux, it is recommended to manually configure the netclient as a daemon using whatever method is acceptable on the chosen operating system.
 
 Private DNS Management
 -----------------------
 
 To manage private DNS, the netclient relies on systemd-resolved (resolvectl). Absent this, it cannot set private DNS for the machine.
 
-A user may choose to manually set a private DNS nameserver of <netmaker server>:53. However, beware, as netmaker sets split dns, and the system must be configured properly. Otherwise, this nameserver may break your local DNS.
+A user may choose to manually set a private DNS nameserver of <netmaker server>:53. However, beware, as netmaker sets split DNS, the system must be configured properly. Otherwise, this nameserver may break your local DNS.
 
 Prerequisites
 =============
@@ -114,10 +114,13 @@ To obtain the netclient, go to the GitHub releases: https://github.com/gravitl/n
 
 **For Private DNS management:** Resolvectl (systemd-resolved)
 
+Please refer to the `Firewall Rules for Machines Running Netclient <https://docs.netmaker.io/install.html#firewall-rules-for-machines-running-netclient>`_ for more information.
+
+
 Configuration
 ===============
 
-The CLI has information about all commands and variables. This section shows the "help" output for these commands as well as some additional reference.
+The CLI has information about all commands and variables. This section shows the "help" output for these commands as well as some additional references.
 
 CLI Reference
 --------------------
@@ -144,19 +147,19 @@ Installation
 
 To install netclient and join a network, you need to use ``netclient install`` command and get an enrollment key for a particular network from netmaker.
 
-An admin creates an enrollment key in the "Enrollment Keys" section of the UI. Upon creating a key, it can be viewed by clicking onto the key from UI. Some details regarding the key will be visible:
+An admin creates an enrollment key in the "Enrollment Keys" section of the UI. Upon creating a key, it can be viewed by clicking on the key from UI. Some details regarding the key will be visible:
 
 **Key:** The enrollment key to join and authenticate to a netmaker network
 
-**Type:** Type of key determines the usage limitation of a particular key. Possible values are: Unlimited, Time Bound, Limited Number of Uses
+**Type:** The Type of key determines the usage limitation of a particular key. Possible values are: Unlimited, Time Bound, Limited Number of Uses
 
 **Expires at:** Shows the expiration date of the particular enrollment key
 
 **Networks:** Shows which netmaker networks can be joined by using the particular enrollment key
 
 **Install Command:** The CLI command to register with the server using the enrollment key, and join the networks
 
-For first time installations, you can run the Install Command. For additional networks, simply run ``netclient join -t <enrollment key>``.
+For first-time installations, you can run the Install Command. For additional networks, simply run ``netclient join -t <enrollment key>``.
 
 
 Managing Netclient

install.rst

@@ -11,7 +11,7 @@ Prerequisites
 Server
 -----------------
 
-All components of Netmaker can be run on a single server (Virtual Machine or Bare Metal). Here some recommendations for setting up the server:
+All components of Netmaker can be run on a single server (Virtual Machine or Bare Metal). Here are some recommendations for setting up the server:
 
 - We **highly recommend** that Netmaker be deployed in a dedicated networking environment.
 - The machine should have a public, static IP address 
@@ -40,6 +40,80 @@ Your server will host several services (netmaker server, UI, etc.) each of which
 - **Note on Cloudflare:** Many of our users use Cloudflare. Cloudflare will proxy connections, which MQ does not like. This can be disabled in the Cloudflare DNS dashboard. If setting up your Netmaker server using Cloudflare for DNS, be aware that the configuration of Cloudflare proxy may cause problems with Netmaker which must be resolved, and at this point, Netmaker is not providing guidance on this setup.
 
 
+
+Firewall Rules for Netmaker Server
+-------------------------------------
+
+Make sure firewall settings are set for Netmaker both on the VM and with your cloud security groups (AWS, GCP, etc) or with your router/FWA. 
+
+Make sure the following ports are open both on the VM and in the cloud security groups:
+
+- **443, 80 (tcp):** for Caddy, which proxies the Dashboard (UI), REST API (Netmaker Server), and Broker (MQTT)  
+- **51821 (udp and tcp):** for WireGuard - Install script automatically setups a netclient on the server machine with default port as 51821.  
+- **8085 (exporter Pro):** If you are building a Pro server, you need this port open.
+- **1883, 8883 8083, 18083 (if using EMQX):** We use two different types of brokers. There is Mosquitto or EMQX. Mosquitto is our default offering which uses ports 8883 and 1883. If you are setting up EMQX, all four ports mentioned need to be opened for MQTT, SSL MQTT, web sockets, and the EMQX dashboard/REST API.
+- **53 (tcp and udp):** If you set the CoreDNS container, that comes with the Netmaker installation, to 'host' your domain name resolution needs
+
+
+.. code-block::
+
+  sudo ufw allow proto tcp from any to any port 443 
+  sudo ufw allow proto tcp from any to any port 80
+  sudo ufw allow 51821/udp (based on your netclient listen port on server machine)
+  sudo ufw allow 51821/tcp (based on your netclient listen port on server machine)
+
+  #optional: only when hosting DNS on the Netmaker server
+  sudo ufw allow 53
+  
+
+It is also important to make sure the server does not block forwarding traffic (it will do this by default on some providers). To ensure traffic will be forwarded:
+
+.. code-block::
+
+  iptables --policy FORWARD ACCEPT
+
+
+**Again, based on your cloud provider, you may additionally need to set inbound security rules for your server (for instance, on AWS). This will be dependent on your cloud provider. Be sure to check before moving on:**
+  - allow 443/tcp from all
+  - allow 80/tcp from all
+  - allow 51821-51830/udp from all
+  - allow 51821-51830/tcp from all
+  - allow 53 from all (optional)
+
+
+Firewall Rules for Machines Running Netclient
+-------------------------------------------------
+
+As we already know, Netclient manages WireGuard on client devices (nodes). As its name suggests, Netclient is a client in a mesh topology, thus it needs to communicate with the server and with the other clients as well. Netclient will detect local changes and send them to the server when necessary. A change in IP address or port will lead to a network update to keep everything in sync.
+It goes without saying that in almost all cases a firewall must be up and running on any device that is connected to a network, especially the Internet. Firewalls are inherently restrictive for good reasons. And by default, it might not allow traffic that Netclient would use to function properly.
+
+On Windows machines, it is possible to allow programs or applications through the firewall. Thus you might want to allow Netclient and, depending on your setup, WireGuard.
+
+On Linux, these necessary ports are needed to be opened:
+
+- UDP and TCP ports 51821-51830 for inbound and outbound (based on your client's listen port running on the machine)
+- TCP port 443 for outbound
+- UDP ports 19302 & 3478 for STUN outbound requests
+
+For advanced use cases, you might need to view your device's firewall logs, or in the case of Netclients behind a NAT, your Firewall-Appliance/Router's firewall logs. Look for blocked traffic coming in and out having origin/destination IPs of your devices.
+
+For example, in UFW you may do:
+
+.. code-block::
+  
+  #set the firewall to log only the blocked traffic
+  ufw logging low
+
+  #clear out the current logs
+  cat /dev/null | sudo tee /var/log/ufw.log
+
+  #reload ufw
+  ufw reload
+  
+  #filter the logs
+  cat /var/log/ufw.log | grep -e <netmaker server IP> -e <other nodes' IPs> 
+
+
 Quick Install
 ==================
 
@@ -51,31 +125,30 @@ Quick Install
 
 
 **IMPORTANT:** Notes on Installation
-- Due to the high volume of installations, the auto-generated domain has been rate-limited by the certificate provider. For this reason, we **strongly recommend** using your own domain. Using the auto-generated domain may lead to a failed installation due to rate limiting.
+- Due to the high volume of installations, the auto-generated domain has been rate-limited by the certificate provider. For this reason, we **strongly recommend** using your domain. Using the auto-generated domain may lead to a failed installation due to rate limiting.
 
 **IMPORTANT:** From v0.22.0 the install script will install PRO version of netmaker with a 14-day free trial, for you to try out full capabilities of netmaker.
 
 Integrating OAuth
 ====================
 
-Users are also allowed to join a Netmaker server via OAuth. They can do this by clicking the "Login with SSO" button on the dashboard's login page. Check out the :doc:`integrating oauth docs <./oauth>`.
-
-After trial period ends:
-=========================
+Users are also allowed to join a Netmaker server via OAuth. They can do this by clicking the "Login with SSO" button on the dashboard's login page. Check out the:doc:`integrating oauth docs <./oauth>`.
 
-    a. if you wish to continue using PRO :-
+After the trial period ends:
+==============================
 
+    a. if you wish to continue using PRO:-
         i. check these steps to obtain pro license `<https://docs.netmaker.io/pro/pro-setup.html>`_
         ii. Run `/root/nm-quick.sh -u`
 
-    b. if you wish to downgrade to community version
+    b. if you wish to downgrade to the community version
 
         i. Run `/root/nm-quick.sh -d`
 
 
 1. **To get started the easiest way, visit our SaaS platform to set up a netmaker server with just a few clicks** `<https://app.netmaker.io>`_
 
-2. :doc:`check out these steps for manual installation process for on-prem, although we don't recommend this path, instead use the install script mentioned above<./manual-install>`
+2. :doc:`check out these steps for the manual installation process for on-prem, although we don't recommend this path, instead use the install script mentioned above<./manual-install>`
 
 3. :ref:`Highly Available Installation <HAInstall>`
 

manual-install.rst

@@ -87,39 +87,9 @@ At this point you should have all the system dependencies you need.
 3. Open Firewall
 ===============================
 
-Make sure firewall settings are set for Netmaker both on the VM and with your cloud security groups (AWS, GCP, etc). 
-
-Make sure the following ports are open both on the VM and in the cloud security groups:
-
-- **443, 80 (tcp):** for Caddy, which proxies the Dashboard (UI), REST API (Netmaker Server), and Broker (MQTT)  
-- **51821-518XX (udp):** for WireGuard - Netmaker needs one port per network, starting with 51821, so open up a range depending on the number of networks you plan on having. For instance, 51821-51830.  
-- **8085 (exporter Pro):** If you are building a Pro server, you need this port open.
-- **1883, 8883 8083, 18083 (if using EMQX):** We use two different types of brokers. There is Mosquitto or EMQX. if you are setting up EMQX, these four need to be open for MQTT, SSL MQTT, web sockets, and the EMQX dashbaord/REST api.
-
-
-.. code-block::
-
-  sudo ufw allow proto tcp from any to any port 443 
-  sudo ufw allow proto tcp from any to any port 80 
-  sudo ufw allow proto tcp from any to any port 3479
-  sudo ufw allow proto tcp from any to any port 8089 
-  sudo ufw allow 51821:51830/udp
+Please refer to the `Firewall Rules for Netmaker Server <https://docs.netmaker.io/install.html#firewall-rules-for-netmaker-server>`_ for more information.
 
 
-It is also important to make sure the server does not block forwarding traffic (it will do this by default on some providers). To ensure traffic will be forwarded:
-
-.. code-block::
-
-  iptables --policy FORWARD ACCEPT
-
-
-**Again, based on your cloud provider, you may additionally need to set inbound security rules for your server (for instance, on AWS). This will be dependent on your cloud provider. Be sure to check before moving on:**
-  - allow 443/tcp from all
-  - allow 80/tcp from all
-  - allow 3479/tcp from all
-  - allow 8089/tcp from all
-  - allow 51821-51830/udp from all
-
 4. Prepare MQ
 ========================