PQC: Classic McEliece #3883

FAlbertDev · 2024-01-11T12:04:01Z

Pull Request Dependencies

FFI: Loading of raw FrodoKEM keys & FIX: "insufficient buffer handling" in FFI's decapsulate #4373 - for the generic FFI/Python tests

Description

This PR relates to the Classic McEliece KEM as specified in this ISO draft. It also contains the instances defined in the NIST Round 4 submission. The test cases were generated using the NIST submissions reference implementation. Note that Classic McEliece (module cmce) is not the original McEliece Algorithm that is implemented in Botan's mce module. See the Classic McEliece homepage, for a brief comparison.

TODO Tracker

coveralls · 2024-01-11T12:37:18Z

coverage: 91.237% (+0.2%) from 91.077%
when pulling 969d584 on Rohde-Schwarz:pqc/classic_mceliece
into dc326ed on randombit:master.

reneme

This mostly looks at cmce_decaps.cpp, cmce_encaps.cpp and cmce.cpp, noting many minor code style things and a few suggestions for alternative code structuring. Not looking into the Classic McEliece specifics at all here.

src/build-data/oids.txt

src/lib/pubkey/classic_mceliece/cmce_types.h

src/lib/pubkey/classic_mceliece/cmce_matrix.cpp

src/lib/pubkey/classic_mceliece/cmce.cpp

src/lib/pubkey/classic_mceliece/cmce_encaps.cpp

src/lib/utils/strong_type.h

reneme

First pass on cmce_field_orderings.cpp. I'm somewhat concerned that this isn't very efficient. But it may well be "good enough". Should we profile that?

src/lib/pubkey/classic_mceliece/cmce_field_ordering.cpp

reneme

More comments. I didn't look at cmce_parameter_set.*, cmce_parameters_* and cmce_poly.*, yet.

src/lib/pubkey/classic_mceliece/cmce_keys_internal.cpp

src/lib/utils/strong_type.h

src/lib/pubkey/classic_mceliece/cmce_matrix.cpp

src/lib/pubkey/classic_mceliece/cmce.cpp

reneme · 2024-01-26T10:16:03Z

src/lib/pubkey/classic_mceliece/cmce_gf.cpp

+      // TODO: Only for test instances. Remove on final PR
+      size_t m = Classic_McEliece_GF::log_q_from_mod(mod);
+
+      for(int i = static_cast<int>(m) - 2; i >= 0; --i) {
+         x ^= CT::Mask<uint32_t>::expand((uint32_t(1) << (i + m)) & x)
+                 .if_set_return(static_cast<uint32_t>(mod.get()) << i);
+      }
+
+      return GF_Elem(static_cast<uint16_t>(x));


Don't forget to remove (and probably replace by some exception?)

src/lib/pubkey/classic_mceliece/cmce_matrix.cpp

reneme · 2024-01-29T16:11:55Z

src/lib/pubkey/classic_mceliece/cmce_matrix.cpp

+Code_Word Classic_McEliece_Matrix::mul(const Classic_McEliece_Parameters& params, const Error_Vector& e) const {
+   auto s = e.subvector(0, params.pk_no_rows());
+   auto e_T = e.subvector(params.pk_no_rows());
+   auto pk_slicer = BufferSlicer(m_mat_bytes);
+
+   for(size_t i = 0; i < params.pk_no_rows(); ++i) {
+      auto pk_current_bytes = pk_slicer.take(params.pk_row_size_bytes());
+      auto row = secure_bitvector(pk_current_bytes, params.n() - params.pk_no_rows());
+      row &= e_T;
+      s.at(i) = s.at(i) ^ row.has_odd_hamming_weight();
+   }
+
+   BOTAN_ASSERT_NOMSG(pk_slicer.empty());
+   return s.as<Code_Word>();
+}


This produces quite a few copies and allocations. If that's in the implementation's hot path, we should probably want to have another look. Here's which (I believe cause allocations/copies):

.subvector() always creates a new bitvector and copies the content into a newly allocated buffer,

the c'tor of bitvector copies the passed-in buffer (particularly pk_current_bytes

.as<> produces a copy of the bitvector with a new type

For (3): This could already be the right type using e.subvector<Code_Word>(), no?

Some context: This function is called once per encapsulation. Also, e and s are pretty small, while m_mat_bytes is gigantic. Therefore, 1) and 3) are not critical; I'll apply your suggestion for 3) anyway. I don't know how we can prevent 2) without having something like a bitvector view or dropping the bitvector altogether.

I think I'll have a timeboxed look into a bitvector that doesn't own its underlying storage. Perhaps that isn't too hard to achieve, especially when we can limit it to byte-aligned subvectors.

reneme

Done with a first pass on the implementation. Don't get put off by the number of comments. Most are just C++ style nits and smaller programming suggestions.

That's really good work! 😃

reneme · 2024-01-30T07:34:29Z

src/lib/pubkey/classic_mceliece/cmce_parameter_set.h

+   /// Reduced instances for side channel analysis (Self-created test instance with
+   /// m=8, n=128, t=8, f(z)=z^8+z^7+z^2+z+1, F(y)=y^8+y^4+y^3+y^2+1)
+   /// Minimal instance without semi-systematic matrix creation and no plaintext confirmation
+   test,
+   /// Minimal instance with semi-systematic matrix creation and no plaintext confirmation
+   testf,
+   /// Minimal instance without semi-systematic matrix creation and with plaintext confirmation
+   testpc,
+   /// Minimal instance with semi-systematic matrix creation and with plaintext confirmation
+   testpcf


TODO: remove before merging

src/lib/pubkey/classic_mceliece/cmce_parameter_set.h

reneme · 2024-01-30T07:39:16Z

src/lib/pubkey/classic_mceliece/cmce_parameters.h

+/**
+ * @returns ceil(n/d)
+ * TODO: Remove once LMS is merged
+ */
+constexpr size_t ceil_div(size_t n, size_t d) {
+   return (n + d - 1) / d;
+}


Re-evaluate before merging this.

src/lib/pubkey/classic_mceliece/cmce_parameters.h

src/lib/pubkey/classic_mceliece/cmce_poly.cpp

src/lib/pubkey/classic_mceliece/info.txt

src/lib/utils/bit_ops.h

src/lib/pubkey/classic_mceliece/cmce_field_ordering.cpp

FAlbertDev · 2024-02-05T14:48:10Z

Thanks a lot for your extensive review, @reneme! I addressed your suggestions and am optimistic that this PR is ready to drop its Draft status 🎉. Note that some suggestions that depend on other PRs are still open, which are not critical, though. Also, note that an extensive side-channel analysis is still in progress.

aewag · 2024-02-10T22:22:15Z

src/lib/utils/bitvector.h

+            constexpr bitref& operator^=(bool other) noexcept { return assign(this->is_set() ^ other); }
+
+         private:
+            constexpr bitref& assign(bool bit) noexcept { return (bit) ? set() : unset(); }


As discussed, DATA identified this line as a leakage during our SCA review.

Due to the ? operator, a control-flow difference is observed based on the boolean input bit variable.

The assign() routine is used within the push_back() routine, which in turn is used within the decode() routine. This may allow an adversary to observe the error vector e and, hence recover the shared secret.

We suggest to perform both the set() and unset() functions with the input as a mask, for example:

private: constexpr bitref& assign(bool bit) noexcept { const block_type assign_mask = 0 - static_cast<block_type>(bit); this->m_block |= (this->m_mask & assign_mask); this->m_block &= ~(this->m_mask & ~assign_mask); return *this; }

In our case, this results in the following instructions - without any conditional branch based on the input:

[ ... ] const block_type assign_mask = 0 - static_cast<block_type>(bit); 41d397: 41 f7 dc neg %r12d [ ... ] this->m_block |= (this->m_mask & assign_mask); 41d3ab: 44 89 e1 mov %r12d,%ecx 41d3ae: 21 c1 and %eax,%ecx this->m_block &= ~(this->m_mask & ~assign_mask); 41d3b0: f7 d0 not %eax this->m_block |= (this->m_mask & assign_mask); 41d3b2: 0a 0a or (%rdx),%cl this->m_block &= ~(this->m_mask & ~assign_mask); 41d3b4: 44 09 e0 or %r12d,%eax 41d3b7: 21 c8 and %ecx,%eax [ ... ]

Thanks for your analysis and your report! I applied your fix using Botan's constant-time helper class (7a0fe59)

atreiber94 · 2024-02-29T13:26:53Z

src/lib/utils/bitvector.h

@@ -1,6 +1,8 @@
 /*
 * An abstraction for an arbitrarily large bitvector that can


@reneme the code still contains some artefacts for varying the underlying data type that we'd ideally want to get rid of.

- constant time conditional swap with mask - floor_log2 Co-Authored-By: Amos Treiber <[email protected]>

This is an implementation of the Classic McEliece KEM according to the NIST Round 4 submission and the ISO draft 20230419. Co-Authored-By: Amos Treiber <[email protected]>

Co-Authored-By: Jack Lloyd <[email protected]>

Co-Authored-By: Fabian Albert <[email protected]>

This allows extracting subvectors of the bitvector as unsigned integral bit masks.

... introduced in randombit#4170

Use CT::poison/unpoison for CMCE. Simultaneously, fixes some constant-time issues in the CMCE implementation that may leak some information about the pivots (for the semi-systematic matrix form). Co-Authored-By: René Meusel <[email protected]>

reneme · 2024-10-18T07:09:55Z

Rebased and resolved several conflicts after all other PQC algorithms got merged.

randombit

Some review comments. Also I left some comments in July that never got responded too

randombit · 2024-10-27T11:24:33Z

src/lib/pubkey/pk_algs.cpp

@@ -419,6 +429,12 @@ std::unique_ptr<Private_Key> load_private_key(const AlgorithmIdentifier& alg_id,
   }
 #endif

+#if defined(BOTAN_HAS_CLASSICMCELIECE) || defined(BOTAN_HAS_CLASSICMCELIECE)
+   if(alg_name.starts_with("mceliece")) {


Uses ClassicMcEliece elsewhere we should be consistent

We named the algorithm identifiers as defined in the specifications:

The OIDs were taken from the IETF Hackathon GitHub. IMO, we should stay with the same naming. But I agree that it is annoying that these are not named classic...

randombit · 2024-10-27T11:24:44Z

src/lib/pubkey/pk_algs.cpp

@@ -266,6 +270,12 @@ std::unique_ptr<Public_Key> load_public_key(const AlgorithmIdentifier& alg_id,
   }
 #endif

+#if defined(BOTAN_HAS_CLASSICMCELIECE) || defined(BOTAN_HAS_CLASSICMCELIECE)
+   if(alg_name.starts_with("mceliece")) {


ClassicMcEliece?

randombit · 2024-10-27T11:25:12Z

src/build-data/oids.txt

+1.3.6.1.4.1.22554.5.1.7 = mceliece6960119
+1.3.6.1.4.1.22554.5.1.8 = mceliece6960119f
+1.3.6.1.4.1.22554.5.1.9 = mceliece8192128
+1.3.6.1.4.1.22554.5.1.10 = mceliece8192128f


Too avoid confusion these should maybe start with ClassicMcEliece-

randombit · 2024-10-27T11:29:03Z

src/lib/pubkey/classic_mceliece/cmce_gf.h

+      /**
+       * @brief XOR assign a GF_Elem to the element of this. Constant time.
+       */
+      Classic_McEliece_GF& operator^=(CmceGfElem other) {


Shouldn't this be += keeping with the additive notation in GF(2)?

Note that CmceGfElem is a strong type for uint16. If I remember correctly, we wanted to distinguish the operation with Classic_McEliece_GF objects from the operation with a CmceGfElem. However, while looking for its usage, I found that this operator is not used at all. Oops... I'll remove it ;)

randombit · 2024-10-27T11:29:36Z

src/lib/pubkey/classic_mceliece/cmce_decaps.cpp

+      auto g_alpha = goppa_poly(alphas[i]);
+      auto r = (g_alpha * g_alpha).inv();
+
+      auto c_mask = GF_Mask(CT::Mask<uint16_t>::expand(code_word.at(i)));


Should GF_Mask have an overload of expand here?

randombit · 2024-10-27T11:31:11Z

src/lib/pubkey/classic_mceliece/cmce_field_ordering.cpp

+
+}  // anonymous namespace
+
+std::optional<Classic_McEliece_Field_Ordering> Classic_McEliece_Field_Ordering::create_field_ordering(


Does the non-constant time nature of std::optional bring up issues here? If so we should consider using CT::Option, if not a comment that it's fine would be helpful.

A std::optional should suffice. The field ordering creation is performed in a rejection sampling loop. I.e., if the result is a nullopt, all secrets are dismissed anyway and a new rejection sampling loop starts from the beginning.

randombit · 2024-10-27T11:33:38Z

src/lib/utils/bitvector.h

+      /**
+       * Renders this bitvector into a sequence of "0"s and "1"s.
+       */
+      std::string to_string() const {


Fine to leave it here

reneme added this to the Botan 3.4.0 milestone Jan 11, 2024

FAlbertDev force-pushed the pqc/classic_mceliece branch 4 times, most recently from f73829f to e505390 Compare January 19, 2024 14:53

atreiber94 force-pushed the pqc/classic_mceliece branch 2 times, most recently from 04b9314 to 190261d Compare January 19, 2024 15:59

reneme self-requested a review January 22, 2024 09:33

This comment was marked as resolved.

Sign in to view

reneme force-pushed the pqc/classic_mceliece branch 6 times, most recently from 7662592 to 2a3e60f Compare January 23, 2024 15:55

This comment was marked as resolved.

Sign in to view

reneme reviewed Jan 25, 2024

View reviewed changes

src/lib/utils/strong_type.h Outdated Show resolved Hide resolved

reneme reviewed Jan 25, 2024

View reviewed changes

reneme reviewed Jan 29, 2024

View reviewed changes

reneme requested changes Jan 30, 2024

View reviewed changes

github-advanced-security bot found potential problems Feb 2, 2024

View reviewed changes

src/lib/pubkey/classic_mceliece/cmce_field_ordering.cpp Fixed Show resolved Hide resolved

FAlbertDev marked this pull request as ready for review February 5, 2024 14:49

aewag reviewed Feb 10, 2024

View reviewed changes

This was referenced Feb 14, 2024

Classic McEliece Cryptodoc sehlen-bsi/botan-docs#186

Open

Classic McEliece Testspecification sehlen-bsi/botan-docs#187

Open

atreiber94 reviewed Feb 29, 2024

View reviewed changes

FAlbertDev force-pushed the pqc/classic_mceliece branch from 5f0efcc to 54395c2 Compare March 8, 2024 13:24

reneme force-pushed the pqc/classic_mceliece branch 3 times, most recently from e07e615 to b679113 Compare July 23, 2024 15:17

FAlbertDev mentioned this pull request Jul 24, 2024

Add expand_bit method to CT::Mask #4254

Merged

randombit self-requested a review August 3, 2024 01:15

reneme mentioned this pull request Oct 14, 2024

FFI: Loading of raw FrodoKEM keys & FIX: "insufficient buffer handling" in FFI's decapsulate #4373

Merged

reneme force-pushed the pqc/classic_mceliece branch 3 times, most recently from f88fe6b to 5012217 Compare October 15, 2024 06:52

reneme modified the milestones: Botan 3.6.0, Botan 3.7.0 Oct 17, 2024

FAlbertDev and others added 13 commits October 18, 2024 08:42

Utility functions for Classic McEliece

2558f68

- constant time conditional swap with mask - floor_log2 Co-Authored-By: Amos Treiber <[email protected]>

bitvector<> with basic functionality

d1b70b4

Classic McEliece implementation

b3ad495

This is an implementation of the Classic McEliece KEM according to the NIST Round 4 submission and the ISO draft 20230419. Co-Authored-By: Amos Treiber <[email protected]>

Review comments on bitvector<>

6afafe6

Co-Authored-By: Jack Lloyd <[email protected]>

add ct_reverse_bits<>

5ed7e6a

Co-Authored-By: Fabian Albert <[email protected]>

add bitvector<>::subvector<uintX_t>()

c032a7a

This allows extracting subvectors of the bitvector as unsigned integral bit masks.

Refactor: use nwe Strong<> unwrapping helpers

8cdfd40

... introduced in randombit#4170

Use Valgrind for CMCE

82b27e8

Use CT::poison/unpoison for CMCE. Simultaneously, fixes some constant-time issues in the CMCE implementation that may leak some information about the pivots (for the semi-systematic matrix form). Co-Authored-By: René Meusel <[email protected]>

remove superfluous Botan_Tests:: namespace

f78f277

bitvector<>::subvector_replace()

5948dd7

Use subvector_replace() in CMCE

cfc5094

Fix clang17 side-channel

15759b0

Integrate C. McEliece into FFI and Python bindings

083e153

reneme force-pushed the pqc/classic_mceliece branch from 5012217 to 083e153 Compare October 18, 2024 07:09

randombit reviewed Oct 27, 2024

View reviewed changes

FAlbertDev added 2 commits October 29, 2024 10:46

Randomize bitvector tests

14b247f

Applied review suggestions

969d584

FAlbertDev force-pushed the pqc/classic_mceliece branch from 97cc674 to 969d584 Compare October 29, 2024 09:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PQC: Classic McEliece #3883

PQC: Classic McEliece #3883

FAlbertDev commented Jan 11, 2024 •

edited by reneme

Loading

coveralls commented Jan 11, 2024 •

edited

Loading

This comment was marked as resolved.

This comment was marked as resolved.

reneme left a comment

reneme left a comment

reneme left a comment

reneme Jan 26, 2024

reneme Jan 29, 2024

FAlbertDev Feb 2, 2024

reneme Feb 6, 2024

reneme left a comment

reneme Jan 30, 2024

reneme Jan 30, 2024

FAlbertDev commented Feb 5, 2024

aewag Feb 10, 2024

FAlbertDev Feb 12, 2024

atreiber94 Feb 29, 2024

reneme commented Oct 18, 2024

randombit left a comment

randombit Oct 27, 2024

FAlbertDev Oct 29, 2024

randombit Oct 27, 2024

FAlbertDev Oct 29, 2024

randombit Oct 27, 2024

FAlbertDev Oct 29, 2024

randombit Oct 27, 2024

FAlbertDev Oct 29, 2024

randombit Oct 27, 2024

randombit Oct 27, 2024

FAlbertDev Oct 29, 2024

randombit Oct 27, 2024

		@@ -1,6 +1,8 @@
		/*
		* An abstraction for an arbitrarily large bitvector that can


		} // anonymous namespace

		std::optional<Classic_McEliece_Field_Ordering> Classic_McEliece_Field_Ordering::create_field_ordering(

PQC: Classic McEliece #3883

Are you sure you want to change the base?

PQC: Classic McEliece #3883

Conversation

FAlbertDev commented Jan 11, 2024 • edited by reneme Loading

Pull Request Dependencies

Description

TODO Tracker

coveralls commented Jan 11, 2024 • edited Loading

This comment was marked as resolved.

This comment was marked as resolved.

reneme left a comment

Choose a reason for hiding this comment

reneme left a comment

Choose a reason for hiding this comment

reneme left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reneme left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

FAlbertDev commented Feb 5, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reneme commented Oct 18, 2024

randombit left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

FAlbertDev commented Jan 11, 2024 •

edited by reneme

Loading

coveralls commented Jan 11, 2024 •

edited

Loading