macOS - deprecated RC2-40-CBC prevents successful import of certificate

[Golffies]

Context

Hello,

On macOS, when you want to import an existing certificate into Mumble, chances are that it is already stored in the Keychain. In all likelihood, the Mac user will then try to export it and its private key in the form of a single ciphered pkcs12 file. Unfortunately, even in its most recent versions, the macOS Keychain ciphers the pkcs12 file using the deprecated RC2-40-CBC algorithm. Current versions of the OpenSSL Default Provider no longer include the RC2-40-CBC algorithm.

% openssl pkcs12 -info -nokeys -nocerts -in certificateandkey.p12
Enter Import Password:
Error outputting keys and certificates
C0DDAE43F87F0000:error:0308010C:digital envelope routines:inner_evp_generic_fetch:unsupported:crypto/evp/evp_fetch.c:355:Global default library context, Algorithm (RC2-40-CBC : 0), Properties ()

As a result, Mumble's certificate import wizard fails to open the pkcs12 file as generated by the macOS Keychain. An error message is displayed in red, but gives no clue as to the real cause of the problem. To avoid taking the user down this dead-end path, Mumble could simply read the pkcs12 file with the -legacy option, which allows openssl to use the RC2-40-CBC algorithm again. Here is an example of the same file that can now be read:

% openssl pkcs12 -legacy -info -nokeys -nocerts -in certificateandkey.p12
Enter Import Password:
MAC: sha1, Iteration 1
MAC length: 20, salt length: 8
PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 2048
Certificate bag
PKCS7 Data
Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 2048

Description

It is probably too much to ask the end user to convert manually the PKCS12 file from the obsolete RC2-40-CBC algorithm that was used by the Keychain to the default algorithm used today by openssl. Such a workaround might look like the following, and unfortunately requires openssl to be used on the command line. This is not what the end user of Mumble is looking for.

% openssl pkcs12 -legacy -in certificateandkey.p12 -out certificateandkey.pem
Enter Import Password:
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:

At this stage, the certificate and private key are ciphered and stored in an intermediate file in pem format.

% openssl pkcs12 -export -in certificateandkey.pem -inkey certificateandkey.pem -out certificateandkey.p12
Enter pass phrase for certificateandkey.pem:
Enter Export Password:
Verifying - Enter Export Password:

At the end of this stage, the certificate and private key are newly ciphered and stored in a new pkcs12 file, which overwrites the old one. All of these operations can be carried out using the same password as the one chosen to enable the Keychain to create the original pkcs12 file. This verifies that the pkcs12 file is now readable by openssl without the -legacy option:

% openssl pkcs12 -info -nokeys -nocerts -in certificateandkey.p12        
Enter Import Password:
MAC: sha256, Iteration 2048
MAC length: 32, salt length: 8
PKCS7 Encrypted data: PBES2, PBKDF2, AES-256-CBC, Iteration 2048, PRF hmacWithSHA256
Certificate bag
PKCS7 Data
Shrouded Keybag: PBES2, PBKDF2, AES-256-CBC, Iteration 2048, PRF hmacWithSHA256

The pkcs12 file is now ciphered using the AES-256-CBC algorithm, rather than RC2-40-CBC. Its handling by openssl no longer requires the -legacy option. Mumble is now able to import it successfully. But frankly, it is unreasonable to expect end users to carry out these tasks themselves.

Proposed change

Mumble could read the pkcs12 file with the -legacy option, which allows openssl to use the RC2-40-CBC algorithm.

Mumble component

Client ; observed with Mumble 1.5.634

OS-specific?

Yes ; observed with macOS 14.6.1

Additional information

Workaround tested with OpenSSL 3.4.0