[TLS 1.3] Hybrid PQ/T key establishment #3609

reneme · 2023-06-30T15:54:25Z

Pull Request Dependencies

Things to do

Tests (at least a round-trip test -- e.g. as a new configuration in test_cli.py cli_tls_socket_tests)
Decide and introduce preliminary code points for compatibility with existing PQ/T endpoints -- e.g. cloudflare, Amazon KMS, Microsoft, IBM, ??)
- How to handle support for those?
Try to build/test with hybrid KEX module disabled
Update the BoGo tests, that seem to support Kyber since May '23
Act on [TLS 1.3] Hybrid PQ/T key establishment #3609 (comment)

Description

This (finally) follows up on #2983 and is a more serious attempt in delivering hybrid key exchange based on draft-ietf-tls-hybrid-design-06.

Hybrid key exchange is handled by an adapter class Hybrid_KEM_PrivateKey that combines 2 or more KEX/KEM keys as specified in draft-ietf-tls-hybrid-design-06. To upstream users it provides a KEM interface.

Internally Hybrid_KEM_PrivateKey uses another adapter KEX_to_KEM_Adapter_PrivateKey that takes a single Key Agreement key (e.g. X25519) and exposes a KEM interface based on it. "KEM-Encaps" is implemented by generating a matching ephemeral key pair and using the ephemeral public key as the "encapsulated secret".

There are plenty of TODO where we needed to workaround short-comings of the Public_Key API. Potential improvements are collected in #3706.

IANA did not define code points for PQ and PQ/T key exchange groups in TLS, yet. As a result, different (beta) services define incompatible algorithm identifiers. We're providing a number of algorithm aliases to be compatible to both libOQS and Cloudflare's implementation. This is certainly a moving target that might change in the near future.

To give this a test run and connect to cloudflare.com using PQC (see their blog post -- section "What we deployed" -- for more details):

Create a TLS policy file (pqc_tls.txt):

allow_tls13 = true
allow_tls12 = false
x25519/Kyber-512-r3 secp256r1/Kyber-512-r3 x25519/Kyber-512-r3/cloudflare

Start the CLI like so:

./botan tls_client --tls-version=1.3 --debug cloudflare.com --policy=./pqc_tls.txt

You'll see that the exchanged Client Hello and Server Hello messages in the debug output are unusually large. The kyber public key weighs about 800bytes.

src/lib/tls/tls13_pqc/info.txt

reneme · 2023-07-04T11:05:09Z

With this policy:

allow_tls13 = true
allow_tls12 = false
key_exchange_groups = x25519/Kyber-512-r3 secp256r1/Kyber-512-r3 x25519/Kyber-512-r3/cloudflare

We tested successfully against:

cloudflare.com (documentation, blog)
Unfortunately they use incompatible algo identifiers (hence the x25519/Kyber-512-r3/cloudflare hack in the policy). They use Curve25519 and don't support the NIST curves.
qsc.eu-de.kms.cloud.ibm.com (documentation)
IBM uses the algo identifiers proposed by liboqs. It does not support Curve25519. Apart from the hybrid exchange this endpoint can also handle just Kyber-512-r3.
test.openquantumsafe.org (documentation)
Yeah. Everything should role.

We couldn't negotiate against:

kms.eu-central-1.amazonaws.com (documentation)
This used to work with [TLS 1.3] Post-Quantum Readiness via Hybrid Key Exchange #2983. But the hybrid encoding proposed by IETF draft changed in the mean time. Apparently, Amazon's endpoint still speaks the old draft. (Individual public values were not just concatenated but a length-value-encoding was used.

coveralls · 2023-07-04T11:57:29Z

coverage: 91.67% (-0.07%) from 91.743% when pulling 4486d90 on Rohde-Schwarz:tls13/kem_establishment into ae37d65 on randombit:master.

reneme · 2023-07-10T07:45:32Z

Do we need to establish one or more new key_exchange_method in the policy?

I believe we should. See for example Session_Summary::kex_algo() which would currently report UNDEFINED for hybrid and/or KEM.

Edit: done

reneme · 2023-07-17T12:45:11Z

Rebased to master after merging #3611.

randombit

This is only a partial review - I haven't reviewed the KEX->KEM adapter at all and only partially reviewed hybrid_public_key.cpp. I will finish the review asap.

src/lib/pubkey/pubkey.cpp

src/lib/pubkey/pubkey.h

src/lib/tls/msg_client_hello.cpp

src/lib/tls/tls13_pqc/info.txt

src/lib/tls/tls_algos.h

src/lib/tls/tls13_pqc/hybrid_public_key.cpp

reneme · 2023-07-26T13:14:21Z

Thanks for the review. Unfortunately, I won't be able to look into it before August 7th. Perhaps @FAlbertDev wants to give it a go, though. 😃

FAlbertDev · 2023-07-26T14:43:50Z

Perhaps @FAlbertDev wants to give it a go, though. 😃

Yeah, I'll give it a try 👍

randombit · 2023-08-03T11:06:12Z

Thanks for addressing the review comments @FAlbertDev, on a quick review of just your commit things looked good.

I want to do another full pass review, I'll get this in sometime before next week.

src/lib/tls/tls13_pqc/hybrid_public_key.cpp

reneme · 2023-08-07T10:56:44Z

Rebased to master to solve conflicts in test_cli.py after merging #3618.

reneme · 2023-09-18T15:02:36Z

src/lib/tls/tls13_pqc/kex_to_kem_adapter.cpp

+/**
+ * This helper determines the length of the agreed-upon value depending
+ * on the key agreement public key's algorithm type. It would be better
+ * to get this value via PK_Key_Agreement::agreed_value_size(), but
+ * instantiating a PK_Key_Agreement object requires a PrivateKey object
+ * which we don't have (yet) in the context this is used.
+ *
+ * TODO: Find a way to get this information without duplicating those
+ *       implementation details of the key agreement algorithms.
+ */
+size_t kex_shared_key_length(const Public_Key& kex_public_key) {
+   return std::visit(
+      overloaded{[](const ECDH_PublicKey& ecdh_public_key) { return ecdh_public_key.domain().get_p_bytes(); },
+                 [](const DH_PublicKey& dh_public_key) { return dh_public_key.group().p_bytes(); },
+                 [](const Curve25519_PublicKey&) { return size_t(32) /* TODO: magic number */; }},
+      as_kex_public_key(kex_public_key));
+}


It would be great if we could access certain basic information of specific algorithms statically.

Thought I had left a comment on this earlier? Hm.

Statically might be difficult given how DH/ECDH are defined over arbitrary groups, but it seems pretty possible to have something like virtual size_t raw_shared_key_length() = 0 the challenge is on what type to put it since PK_Key_Agreement_Key is only for the private keys ...

Conceivably we could have something on Public_Key like so

/** * Return the length of the relevant artifact of this key. * * Throws Not_Implemented if the key does not support an operation of this type. * * For KeyAgreement, returns the raw (non-KDF) output length. */ virtual size_t output_length(PublicKeyOperation op) = 0 ```

randombit

Looks very good, left a few comments, ptal

doc/api_ref/tls.rst

src/lib/tls/msg_client_hello.cpp

src/lib/tls/tls13_pqc/hybrid_public_key.cpp

src/lib/tls/tls13_pqc/hybrid_public_key.h

src/lib/tls/tls13_pqc/info.txt

src/lib/tls/tls13_pqc/kex_to_kem_adapter.cpp

reneme · 2023-09-19T10:00:03Z

Done with your latest review comments. I updated the PR description (mainly for future reference) and created #3706 to collect future improvements on the Pubkey-APIs.

I committed the review fixes into new commits for easy review. Let me know if I should squash before we merge eventually.

(draft-ietf-tls-hybrid-design) Co-Authored-By: Fabian Albert <[email protected]>

reneme · 2023-09-29T08:53:51Z

Rebased to master.

randombit

Did a full review, left a few minor comments. Approving to unblock since I didn't notice anything major. This certainly points out some nasty holes in our API surface; in principle it should be possible to implement the hybrid and kex->kem adaptors without referencing any algorithm specific details.

src/lib/tls/tls13_pqc/info.txt

src/lib/tls/tls13_pqc/kex_to_kem_adapter.cpp

src/tests/test_tls_hybrid_kem_key.cpp

src/lib/tls/tls13_pqc/hybrid_public_key.h

reneme marked this pull request as draft June 30, 2023 15:54

reneme mentioned this pull request Jun 30, 2023

[TLS 1.3] Post-Quantum Readiness via Hybrid Key Exchange #2983

Closed

2 tasks

reneme force-pushed the tls13/kem_establishment branch from b452104 to db5c166 Compare July 3, 2023 14:54

reneme mentioned this pull request Jul 3, 2023

[TLS 1.3] Use a KEM interface in the Callbacks #3608

Merged

reneme force-pushed the tls13/kem_establishment branch from db5c166 to 6d58057 Compare July 3, 2023 15:03

reneme commented Jul 3, 2023

View reviewed changes

src/lib/tls/tls13_pqc/info.txt Outdated Show resolved Hide resolved

FAlbertDev force-pushed the tls13/kem_establishment branch from 6d58057 to 5741e18 Compare July 4, 2023 11:13

reneme marked this pull request as ready for review July 4, 2023 12:44

reneme mentioned this pull request Jul 5, 2023

Chore: Update BoGo tests #3616

Merged

reneme force-pushed the tls13/kem_establishment branch from 136c761 to a963bea Compare July 6, 2023 07:27

reneme mentioned this pull request Jul 10, 2023

TLS 1.3 - Current State and TODOs #2990

Open

14 tasks

reneme force-pushed the tls13/kem_establishment branch from e4af46f to 3f7c9c4 Compare July 17, 2023 12:44

randombit mentioned this pull request Jul 25, 2023

There is no reason the types in pubkey.h cannot be movable #3641

Closed

randombit reviewed Jul 25, 2023

View reviewed changes

FAlbertDev added a commit to Rohde-Schwarz/botan that referenced this pull request Jul 27, 2023

Apply review suggestions for randombit#3609

deccc1c

FAlbertDev force-pushed the tls13/kem_establishment branch 3 times, most recently from 4a159c8 to 745dae8 Compare July 28, 2023 12:29

randombit mentioned this pull request Aug 3, 2023

Add PK_KEM_Decryptor::encapsulated_key_length #3652

Closed

randombit reviewed Aug 3, 2023

View reviewed changes

src/lib/tls/tls13_pqc/hybrid_public_key.cpp Outdated Show resolved Hide resolved

FAlbertDev force-pushed the tls13/kem_establishment branch from 745dae8 to 0988317 Compare August 4, 2023 09:57

reneme force-pushed the tls13/kem_establishment branch from 0988317 to fcbf8a6 Compare August 7, 2023 10:55

reneme commented Sep 18, 2023

View reviewed changes

reneme mentioned this pull request Sep 18, 2023

Refactor: Remove MDx_HashFunction #3705

Merged

randombit reviewed Sep 19, 2023

View reviewed changes

reneme force-pushed the tls13/kem_establishment branch from 1cdf85b to 7394ffb Compare September 19, 2023 07:04

reneme mentioned this pull request Sep 19, 2023

Improve API of asymmetric algorithms #3706

Open

5 tasks

reneme requested a review from randombit September 26, 2023 12:11

FAlbertDev and others added 8 commits September 29, 2023 10:52

Use KEM_Encapsulation for TLS callbacks

9e59bb5

Add DH_PublicKey::group()

52d3bb7

FIX: missing fmt() param

2c6247b

introduce Kex_Algo enum entries for (hybrid) PQC

71a368f

Introduce support for TLS 1.3 with hybrid key exchange

ac31a9c

(draft-ietf-tls-hybrid-design) Co-Authored-By: Fabian Albert <[email protected]>

Documentation of TLS 1.3 hybrid KEX

586c07b

Example: TLS 1.3 client offering PQ/T exchange

71068d0

Hybrid_KEM_* keypair requires at least two keys

d7fd540

reneme force-pushed the tls13/kem_establishment branch from 7394ffb to 4486d90 Compare September 29, 2023 08:53

This was referenced Sep 29, 2023

Documents for AP 5: PQ/T Hybrid Key Exchange in TLS 1.3 sehlen-bsi/botan-docs#138

Closed

PQC: FrodoKEM #3679

Merged

randombit approved these changes Oct 5, 2023

View reviewed changes

src/lib/tls/tls13_pqc/info.txt Show resolved Hide resolved

src/lib/tls/tls13_pqc/kex_to_kem_adapter.cpp Outdated Show resolved Hide resolved

src/tests/test_tls_hybrid_kem_key.cpp Show resolved Hide resolved

src/lib/tls/tls13_pqc/hybrid_public_key.h Show resolved Hide resolved

reneme force-pushed the tls13/kem_establishment branch from 512e0ff to 138ee13 Compare October 5, 2023 12:18

reneme added 2 commits October 5, 2023 16:35

FIX: review comments

cbf0fde

FIX: allow selective disabling of KEX methods

413f7a6

reneme force-pushed the tls13/kem_establishment branch from 138ee13 to 413f7a6 Compare October 5, 2023 14:46

reneme merged commit 2f15b3e into randombit:master Oct 5, 2023
38 checks passed

reneme deleted the tls13/kem_establishment branch October 5, 2023 15:44

reneme mentioned this pull request Feb 20, 2024

Feature: Module Lifecycle and Experimental/Deprecated Features #3910

Merged

reneme mentioned this pull request Mar 8, 2024

BSI Project 481 #3108

Open

reneme mentioned this pull request Apr 8, 2024

Introduce Public_Key::raw_public_key_bits() #3985

Merged

reneme mentioned this pull request May 22, 2024

Compile-time output length #4065

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TLS 1.3] Hybrid PQ/T key establishment #3609

[TLS 1.3] Hybrid PQ/T key establishment #3609

reneme commented Jun 30, 2023 •

edited

Loading

reneme commented Jul 4, 2023 •

edited

Loading

coveralls commented Jul 4, 2023 •

edited

Loading

reneme commented Jul 10, 2023 •

edited

Loading

reneme commented Jul 17, 2023

randombit left a comment

reneme commented Jul 26, 2023

FAlbertDev commented Jul 26, 2023

randombit commented Aug 3, 2023

reneme commented Aug 7, 2023

reneme Sep 18, 2023

randombit Oct 5, 2023

randombit left a comment

reneme commented Sep 19, 2023

reneme commented Sep 29, 2023

randombit left a comment

[TLS 1.3] Hybrid PQ/T key establishment #3609

[TLS 1.3] Hybrid PQ/T key establishment #3609

Conversation

reneme commented Jun 30, 2023 • edited Loading

Pull Request Dependencies

Things to do

Description

reneme commented Jul 4, 2023 • edited Loading

coveralls commented Jul 4, 2023 • edited Loading

reneme commented Jul 10, 2023 • edited Loading

reneme commented Jul 17, 2023

randombit left a comment

Choose a reason for hiding this comment

reneme commented Jul 26, 2023

FAlbertDev commented Jul 26, 2023

randombit commented Aug 3, 2023

reneme commented Aug 7, 2023

reneme Sep 18, 2023

Choose a reason for hiding this comment

randombit Oct 5, 2023

Choose a reason for hiding this comment

randombit left a comment

Choose a reason for hiding this comment

reneme commented Sep 19, 2023

reneme commented Sep 29, 2023

randombit left a comment

Choose a reason for hiding this comment

reneme commented Jun 30, 2023 •

edited

Loading

reneme commented Jul 4, 2023 •

edited

Loading

coveralls commented Jul 4, 2023 •

edited

Loading

reneme commented Jul 10, 2023 •

edited

Loading