Detect and fix decryption of files encrypted with doubled "Salted" pr…

…efixes due to #147

If someone has committed and encrypted a file using Transcrypt
version 2.2.0 on a MacOS 13 Ventura system and using the system-
provided version of `openssl`, the encrypted file will mistakenly
include a doubled "Salted" prefix.

The prefix doubling was caused by Transcrypt applying a work-around
for the OpenSSL project's breaking change (#133) when it didn't need
to, because the LibreSSL project's version 3+ of `openssl` does not
have the same breaking change.

This fix checks for a doubled prefix during decryption (smudge)
operations, and when it finds the mistake will strip out the first
of the doubled prefixes before decrypting.

A proper fix for the issue is to commit a new version of the file
to remove the faulty doubled prefix, but it will be difficult for
users to commit a new version if they only have a faulty decrypted
file to work with.

As an example, a faulty version of this repository's _sensitive_file_
with the doubled prefix would be decrypted like this:

    ��2p͙��g�c�^?Dj6����`�32��\rs to love
    You know the rules and so do I
    A full commitment's what I'm thinking of

With the fix applied in this commit, the decrypted copy becomes:

    We're no strangers to love
    You know the rules and so do I
    A full commitment's what I'm thinking of

CHANGELOG.md

@@ -13,7 +13,7 @@ The format is based on [Keep a Changelog][1], and this project adheres to
 ### Fixed
 
 - Compatibility fix for LibreSSL versions 3 (and above) especially for MacOS
-  13 Ventura to more carefully apply a work-around required for OpenSSL 3+
+  13 Ventura, to more carefully apply a work-around required for OpenSSL 3+
   that isn't required for LibreSSL 3+ (#147 #133)
 
 ## [2.2.0] - 2022-07-09

transcrypt

@@ -175,13 +175,39 @@ git_clean() {
 	fi
 }
 
+# shellcheck disable=SC2005,SC2155
 git_smudge() {
 	tempfile=$(mktemp 2>/dev/null || mktemp -t tmp)
 	trap 'rm -f "$tempfile"' EXIT
 	cipher=$(git config --get --local transcrypt.cipher)
 	password=$(git config --get --local transcrypt.password)
 	openssl_path=$(git config --get --local transcrypt.openssl-path)
-	tee "$tempfile" | ENC_PASS=$password "$openssl_path" enc -d "-${cipher}" -md MD5 -pass env:ENC_PASS -a 2>/dev/null || cat "$tempfile"
+
+	# Write stdin to $tempfile, while skimming the first bytes at the same time
+	local firstbytes=$(tee "$tempfile" | head -c8 | LC_ALL=C tr -d '\0')
+	# If the first bytes are "Salted", then the file is encrypted
+	if [[ $firstbytes == "U2FsdGVk" ]]; then
+		# Fix for file mistakenly encrypted with double "Salted" prefixes due to #147
+		# that causes garbage characters at top of decrypted files.
+		#
+		# Check file header, which we already know starts with "Salted", to see if
+		# it has exactly the same "Salted__XYZ" prefix mistakenly repeated twice
+		local header_decoded=$(echo "$(head -c48 <"$tempfile")" | openssl base64 -d)
+		local first_salt_prefix=$(echo "$header_decoded" | cut -b 1-16)         # First 16 bytes
+		local maybe_second_salt_prefix=$(echo "$header_decoded" | cut -b 17-32) # Second 16 bytes
+
+		# If the salted prefix is repeated -- and not empty, to avoid mistaken match if
+		# base64 decoding fails -- remove the first occurrence before decrypting...
+		if [[ "$first_salt_prefix" && "$first_salt_prefix" == "$maybe_second_salt_prefix" ]]; then
+			openssl base64 -d <"$tempfile" | tail -c+17 | ENC_PASS=$password "$openssl_path" enc -d "-${cipher}" -md MD5 -pass env:ENC_PASS 2>/dev/null
+		# ...otherwise decrypt as normal
+		else
+			ENC_PASS=$password "$openssl_path" enc -d -a "-${cipher}" -md MD5 -pass env:ENC_PASS <"$tempfile" 2>/dev/null
+		fi
+	# If the first bytes are not "Salted", the file is not encrypted so output it unchanged
+	else
+		cat "$tempfile"
+	fi
 }
 
 git_textconv() {