New network protocol v4 - SAFEGET

The current protocol (v3) uses a `STAT <FILENAME>` request to determine
the file size (among other things), followed by a `GET <FILENAME>`
request to fetch the file content. However, there is a race condition
here. If the file size increases between the two requests, the client
would think that the remaining data after the "file size" offset is a
new response header.

Here, we introduce a new protocol-version "safeget" to address the race
condition. PDUs from the response following a `GET <FILENAME>` request
now contain a protocol header consisting of a NULL-byte terminated
string of four unsigned integers (`uint64_t`). We will refer to the four
integers as ERROR_CODE, FILE_SIZE, FILE_OFFSET, and PAYLOAD_SIZE. The
integers will appear in this order in the protocol header. Each integer
is separated by a whitespace character.

A non-zero ERROR_CODE signals that an error occurred. Furthermore, the
exact value of ERROR_CODE specifies what went wrong on the server side.
FILE_SIZE and FILE_OFFSET are mainly used to determine when the last PDU
is received (i.e., when FILE_SIZE = FILE_OFFSET + PAYLOAD_SIZE).
However, FILE_SIZE is also used to detect if the file is modified at the
source during transmission. The payload starts immediately after the
NULL-terminating byte in the protocol header and consists of
PAYLOAD_SIZE bytes.

Ticket: ENT-12414
Changelog: Fixed race condition when copying remote files.
Signed-off-by: Lars Erik Wik <[email protected]>

libcfnet/README.md

@@ -15,6 +15,7 @@ Names of protocol versions:
 1. `"classic"` - Legacy, pre-TLS, protocol. Not enabled or allowed by default.
 2. `"tls"` - TLS Protocol using OpenSSL. Encrypted and 2-way authentication.
 3. `"cookie"` - TLS Protocol with cookie command for duplicate host detection.
+3. `"safeget"` - Same as cookie, but with an atomic and safe `GET` request.
 
 Wanted protocol version can be specified from policy:
 
@@ -59,3 +60,25 @@ Both server and client will then set `conn_info->protocol` to `2`, and use proto
 There is currently no way to require a specific version number (only allow / disallow version 1).
 This is because version 2 and 3 are practically identical.
 Downgrade from version 3 to 2 happens seamlessly, but crucially, it doesn't downgrade to version 1 inside the TLS code.
+
+
+## Requests
+
+### `GET <FILENAME>` (protocol v4)
+
+The following is a description of the `GET` request at protocol version v4
+`"safeget"` (introduced in CFEngine 3.25).
+
+PDUs from the response following a `GET <FILENAME>` request now contain a
+protocol header consisting of a NULL-byte terminated string of four unsigned
+integers (`uint64_t`). We will refer to the four integers as ERROR_CODE,
+FILE_SIZE, FILE_OFFSET, and PAYLOAD_SIZE. The integers will appear in this order
+in the protocol header. Each integer is separated by a whitespace character.
+
+A non-zero ERROR_CODE signals that an error occurred. Furthermore, the exact
+value of ERROR_CODE specifies what went wrong on the server side. FILE_SIZE and
+FILE_OFFSET are mainly used to determine when the last PDU is received (i.e.,
+when FILE_SIZE = FILE_OFFSET + PAYLOAD_SIZE). However, FILE_SIZE is also used to
+detect if the file is modified at the source during transmission. The payload
+starts immediately after the NULL-terminating byte in the protocol header and
+consists of PAYLOAD_SIZE bytes.

libcfnet/protocol_version.c

@@ -33,6 +33,10 @@ ProtocolVersion ParseProtocolVersionPolicy(const char *const s)
     {
         return CF_PROTOCOL_COOKIE;
     }
+    else if (StringEqual(s, "4") || StringEqual(s, "safeget"))
+    {
+        return CF_PROTOCOL_SAFEGET;
+    }
     else if (StringEqual(s, "latest"))
     {
         return CF_PROTOCOL_LATEST;

libcfnet/protocol_version.h

@@ -39,10 +39,11 @@ typedef enum
     /* --- Greater versions use TLS as secure communications layer --- */
     CF_PROTOCOL_TLS = 2,
     CF_PROTOCOL_COOKIE = 3,
+    CF_PROTOCOL_SAFEGET = 4, /* Here we fix a race condition in `GET <FILENAME>` */
 } ProtocolVersion;
 
 /* We use CF_PROTOCOL_LATEST as the default for new connections. */
-#define CF_PROTOCOL_LATEST CF_PROTOCOL_COOKIE
+#define CF_PROTOCOL_LATEST CF_PROTOCOL_SAFEGET
 
 static inline const char *ProtocolVersionString(const ProtocolVersion p)
 {
@@ -54,6 +55,8 @@ static inline const char *ProtocolVersionString(const ProtocolVersion p)
         return "tls";
     case CF_PROTOCOL_CLASSIC:
         return "classic";
+    case CF_PROTOCOL_SAFEGET:
+        return "safeget";
     default:
         return "undefined";
     }
@@ -89,6 +92,11 @@ static inline bool ProtocolTerminateCSV(const ProtocolVersion p)
     return (p < CF_PROTOCOL_COOKIE);
 }
 
+static inline bool ProtocolSafeGet(const ProtocolVersion p)
+{
+    return (p >= CF_PROTOCOL_SAFEGET);
+}
+
 /**
  * Returns CF_PROTOCOL_TLS or CF_PROTOCOL_CLASSIC (or CF_PROTOCOL_UNDEFINED)
  * Maps all versions using TLS to CF_PROTOCOL_TLS for convenience