Point to test vectors rather than reproduce them

The current plan is to reproduce the test vectors in the draft once we're ready for RFC. However the value of these test vectors for implementers is not clear. It would be better to publish machine-readable test vectors somewhere somewhere, point to it from the draft, and publish a hash of the test vectors in the draft so that implementers can use to verify their veracity. This commit implements the following idea: point to a commit hash for the draft and describe how to consume the test vectors. This commit also moves the VDAF test vectors to their own sub-directory so that it is convenient to point to them.
cfrg · Sep 20, 2024 · 3e837b6 · 3e837b6
1 parent bb01e52
commit 3e837b6
Show file tree

Hide file tree

Showing 16 changed files with 58 additions and 62 deletions.
diff --git a/draft-irtf-cfrg-vdaf.md b/draft-irtf-cfrg-vdaf.md
@@ -5675,54 +5675,50 @@ useful feedback on and contributions to the spec.
 # Test Vectors {#test-vectors}
 {:numbered="false"}
 
-(TO BE REMOVED BY RFC EDITOR: machine-readable test vectors can be found at
-https://github.com/cfrg/draft-irtf-cfrg-vdaf/tree/main/poc/test_vec.)
-
-Test vectors cover the generation of input shares and the conversion of input
-shares into output shares. Vectors specify the verification key, measurements,
-aggregation parameter, and any parameters needed to construct the VDAF. (For
-example, for `Prio3Sum`, the user specifies the number of bits for representing
-each summand.)
-
-Byte strings are encoded in hexadecimal. To make the tests deterministic, the
-random inputs of randomized algorithms were fixed to the byte sequence starting
-with `0`, incrementing by `1`, and wrapping at `256`:
+Machine readable test vectors can be obtained with the following command:
 
 ~~~
-0, 1, 2, ..., 255, 0, 1, 2, ...
+curl -L https://github.com/cfrg/draft-irtf-cfrg-vdaf/archive/\
+<FIXME>.tar.gz |\
+tar -zxf - "*/test_vec/12/vdaf/*.json"
 ~~~
 
-## Prio3Count {#testvec-prio3count}
-{:numbered="false"}
-
-> TODO Copy the machine readable vectors from the source repository
-> (https://github.com/cfrg/draft-irtf-cfrg-vdaf/tree/main/poc/test_vec) and
-> format them for humans.
-
-## Prio3Sum {#testvec-prio3sum}
-{:numbered="false"}
-
-> TODO Copy the machine readable vectors from the source repository
-> (https://github.com/cfrg/draft-irtf-cfrg-vdaf/tree/main/poc/test_vec) and
-> format them for humans.
-
-## Prio3SumVec {#testvec-prio3sumvec}
-{:numbered="false"}
-
-> TODO Copy the machine readable vectors from the source repository
-> (https://github.com/cfrg/draft-irtf-cfrg-vdaf/tree/main/poc/test_vec) and
-> format them for humans.
-
-## Prio3Histogram {#testvec-prio3histogram}
-{:numbered="false"}
-
-> TODO Copy the machine readable vectors from the source repository
-> (https://github.com/cfrg/draft-irtf-cfrg-vdaf/tree/main/poc/test_vec) and
-> format them for humans.
-
-## Poplar1 {#testvec-poplar1}
-{:numbered="false"}
-
-> TODO Copy the machine readable vectors from the source repository
-> (https://github.com/cfrg/draft-irtf-cfrg-vdaf/tree/main/poc/test_vec) and
-> format them for humans.
+(RFC EDITOR: Update the commit hash in the command above to the commit that
+contains the final version of the test vectors.)
+
+The directory contains a set of JSON files. Each file contains a test vector
+for an instance of `Vdaf` ({{vdaf}}). A test vector covers sharding,
+preparation, aggregation, and unsharding of each of several measurements.
+The test vector schema is as follows:
+
+* whatever parameters are required to instantiate the scheme (see
+  {{vdaf-test-vec-params}})
+* `verify_key`: the verification key encoded in hexadecimal
+* `"agg_param"`: the aggregation parameter of type `Vdaf.AggParam`
+* "prep": a list of objects with the following schema:
+
+    * `"measurement"`: the measurement of type `Vdaf.Measurement`
+    * `"nonce"`: the nonce encoded in hexadecimal
+    * `"rand"`: the sharding randomness encoded in hexadecimal
+    * `"public_share"`: the expected public share encoded in hexadecimal
+    * `"input_shares"`: the expected list of input shares, each incoded in
+      hexadecimal
+    * `"prep_shares"`: the expected list of prep shares generated by each
+      Aggregator at each round of preparation, encoded in hexadecimal
+    * `"prep_messages"`: the expected list of prep messages for each round of
+      preparation, encoded in hexadecimal
+    * `"out_shares"`: the expected list of output shares, encoded in
+      hexadecimal
+
+* `"agg_shares"`: the expected aggregate shares encoded in hexadecimal
+* `"agg_result"`: the expected aggregate result of type `Vdaf.AggResult`
+
+| Scheme                | Parameters                                                     |
+|:----------------------|:---------------------------------------------------------------|
+| Prio3Count            | `shares: int`                                                  |
+| Prio3Sum              | `shares: int, max_measurement: int`                            |
+| Prio3SumVec           | `shares: int, length: int, bits: int, chunk_length: int`       |
+| Prio3Histogram        | `shares: int, length: int, chunk_length: int`                  |
+| Prio3MultihotCountVec | `shares: int, length: int, max_weight: int, chunk_length: int` |
+| Poplar1               | `bits: int`                                                    |
+{: #vdaf-test-vec-params title="Parameters for instantiating each VDAF."}
diff --git a/poc/gen_test_vec.py b/poc/gen_test_vec.py
@@ -11,7 +11,6 @@
 TEST_VECTOR_PATH = os.environ.get('TEST_VECTOR_PATH',
                                   '../test_vec/{:02}'.format(VERSION))
 
-
 # IDPF
 
 def gen_test_vec_for_idpf(idpf: Idpf,
@@ -90,37 +89,39 @@ def gen_test_vec_for_xof(cls: type[Xof]) -> None:
 if __name__ == '__main__':
     from vdaf_poc import idpf_bbcggi21, vdaf_poplar1, vdaf_prio3, xof
 
+    vdaf_test_vec_path = TEST_VECTOR_PATH + "/vdaf/"
+
     # Prio3 variants
     gen_test_vec_for_vdaf(
-        TEST_VECTOR_PATH,
+        vdaf_test_vec_path,
         vdaf_prio3.Prio3Count(2),
         None,
         [1],
         0,
     )
     gen_test_vec_for_vdaf(
-        TEST_VECTOR_PATH,
+        vdaf_test_vec_path,
         vdaf_prio3.Prio3Count(3),
         None,
         [1],
         1,
     )
     gen_test_vec_for_vdaf(
-        TEST_VECTOR_PATH,
+        vdaf_test_vec_path,
         vdaf_prio3.Prio3Sum(2, 255),
         None,
         [100],
         0,
     )
     gen_test_vec_for_vdaf(
-        TEST_VECTOR_PATH,
+        vdaf_test_vec_path,
         vdaf_prio3.Prio3Sum(3, 255),
         None,
         [100],
         1,
     )
     gen_test_vec_for_vdaf(
-        TEST_VECTOR_PATH,
+        vdaf_test_vec_path,
         vdaf_prio3.Prio3SumVec(2, 10, 8, 9),
         None,
         [
@@ -131,7 +132,7 @@ def gen_test_vec_for_xof(cls: type[Xof]) -> None:
         0,
     )
     gen_test_vec_for_vdaf(
-        TEST_VECTOR_PATH,
+        vdaf_test_vec_path,
         vdaf_prio3.Prio3SumVec(3, 3, 16, 7),
         None,
         [
@@ -142,21 +143,21 @@ def gen_test_vec_for_xof(cls: type[Xof]) -> None:
         1,
     )
     gen_test_vec_for_vdaf(
-        TEST_VECTOR_PATH,
+        vdaf_test_vec_path,
         vdaf_prio3.Prio3Histogram(2, 4, 2),
         None,
         [2],
         0,
     )
     gen_test_vec_for_vdaf(
-        TEST_VECTOR_PATH,
+        vdaf_test_vec_path,
         vdaf_prio3.Prio3Histogram(3, 11, 3),
         None,
         [2],
         1,
     )
     gen_test_vec_for_vdaf(
-        TEST_VECTOR_PATH,
+        vdaf_test_vec_path,
         vdaf_prio3.Prio3MultihotCountVec(2, 4, 2, 2),
         None,
         [[0, 1, 1, 0]],
@@ -172,7 +173,7 @@ def gen_test_vec_for_xof(cls: type[Xof]) -> None:
     ]
     for (test_level, prefixes) in tests:
         gen_test_vec_for_vdaf(
-            TEST_VECTOR_PATH,
+            vdaf_test_vec_path,
             vdaf_poplar1.Poplar1(4),
             (test_level, prefixes),
             [0b1101],

diff --git a/poc/vdaf_poc/test_utils.py b/poc/vdaf_poc/test_utils.py
@@ -137,17 +137,16 @@ def gen_test_vec_for_vdaf(
         (public_share, input_shares) = \
             vdaf.shard(measurement, nonce, rand)
 
+        pub_share_hex = vdaf.test_vec_encode_public_share(public_share).hex()
         prep_test_vec: VdafPrepTestVectorDict[Measurement] = {
             'measurement': measurement,
             'nonce': nonce.hex(),
+            'rand': rand.hex(),
+            'public_share': pub_share_hex,
             'input_shares': [],
             'prep_shares': [[] for _ in range(vdaf.ROUNDS)],
             'prep_messages': [],
             'out_shares': [],
-            'rand': rand.hex(),
-            'public_share': vdaf.test_vec_encode_public_share(
-                public_share
-            ).hex()
         }
         for input_share in input_shares:
             prep_test_vec['input_shares'].append(

diff --git a/test_vec/12/Poplar1_0.json → test_vec/12/vdaf/Poplar1_0.json b/test_vec/12/Poplar1_0.json → test_vec/12/vdaf/Poplar1_0.json
diff --git a/test_vec/12/Poplar1_1.json → test_vec/12/vdaf/Poplar1_1.json b/test_vec/12/Poplar1_1.json → test_vec/12/vdaf/Poplar1_1.json
diff --git a/test_vec/12/Poplar1_2.json → test_vec/12/vdaf/Poplar1_2.json b/test_vec/12/Poplar1_2.json → test_vec/12/vdaf/Poplar1_2.json
diff --git a/test_vec/12/Poplar1_3.json → test_vec/12/vdaf/Poplar1_3.json b/test_vec/12/Poplar1_3.json → test_vec/12/vdaf/Poplar1_3.json
diff --git a/test_vec/12/Prio3Count_0.json → test_vec/12/vdaf/Prio3Count_0.json b/test_vec/12/Prio3Count_0.json → test_vec/12/vdaf/Prio3Count_0.json
diff --git a/test_vec/12/Prio3Count_1.json → test_vec/12/vdaf/Prio3Count_1.json b/test_vec/12/Prio3Count_1.json → test_vec/12/vdaf/Prio3Count_1.json
diff --git a/test_vec/12/Prio3Histogram_0.json → test_vec/12/vdaf/Prio3Histogram_0.json b/test_vec/12/Prio3Histogram_0.json → test_vec/12/vdaf/Prio3Histogram_0.json
diff --git a/test_vec/12/Prio3Histogram_1.json → test_vec/12/vdaf/Prio3Histogram_1.json b/test_vec/12/Prio3Histogram_1.json → test_vec/12/vdaf/Prio3Histogram_1.json
diff --git a/test_vec/12/Prio3MultihotCountVec_0.json → ..._vec/12/vdaf/Prio3MultihotCountVec_0.json b/test_vec/12/Prio3MultihotCountVec_0.json → ..._vec/12/vdaf/Prio3MultihotCountVec_0.json
diff --git a/test_vec/12/Prio3SumVec_0.json → test_vec/12/vdaf/Prio3SumVec_0.json b/test_vec/12/Prio3SumVec_0.json → test_vec/12/vdaf/Prio3SumVec_0.json
diff --git a/test_vec/12/Prio3SumVec_1.json → test_vec/12/vdaf/Prio3SumVec_1.json b/test_vec/12/Prio3SumVec_1.json → test_vec/12/vdaf/Prio3SumVec_1.json
diff --git a/test_vec/12/Prio3Sum_0.json → test_vec/12/vdaf/Prio3Sum_0.json b/test_vec/12/Prio3Sum_0.json → test_vec/12/vdaf/Prio3Sum_0.json
diff --git a/test_vec/12/Prio3Sum_1.json → test_vec/12/vdaf/Prio3Sum_1.json b/test_vec/12/Prio3Sum_1.json → test_vec/12/vdaf/Prio3Sum_1.json