CBOR Feature Drop: COSE #2412

JesusMcCloud · 2023-08-18T11:18:06Z

This PR obsoletes #2371 and #2359 as it contains the features of both PRs and many more.

Specifically, this PR contains all feature required to serialize and parse COSE-compliant CBOR (thanks to @nodh). While some canonicalization steps (such as sorting keys) still needs to be performed manually. It does get the job done quite well. Namely, we have successfully used the features introduced here to create and validate ISO/IEC 18013-5:2021-compliant mobile driving license data.

This PR introduces the following features to the CBOR format:

Serial Labels
Tagging of keys and values
Definite length encoding (this is the largest change, as it effectively makes the cbor encoder two-pass)
Globally preferring major type 2 for byte array encoding
Encodes null complex objects as empty map (to be COSE-compliant)

sandwwraith · 2023-08-21T12:54:59Z

Hi, thanks for your PR! Just as a quick heads-up so you know I've seen it — I'm going on vacation soon, so I'll be able to properly review it in September.

sandwwraith

Hi, I appreciate the amount of work you've done — especially for the tests. I didn't look into the implementation (yet), focusing on surface API for now.

Regarding failure on CI — it's connected to the fact that Long.toUnsignedString was added only in Android API 26. However, it is possible to use this declaration on Android with desugaring enabled, so there shouldn't be any problems. You have to crate a special annotation @SuppressAnimalSniffer (it already exists for core and json, but not for cbor, see e.g. here:

kotlinx.serialization/formats/json/commonMain/src/kotlinx/serialization/json/JsonElement.kt

Line 92 in d192d24

@SuppressAnimalSniffer // Long.toUnsignedString(long)

) and use it in configuration here:

kotlinx.serialization/build.gradle

Line 205 in a87b0f1

switch (name) {

formats/cbor/commonMain/src/kotlinx/serialization/cbor/ByteStringWrapper.kt

formats/cbor/commonMain/src/kotlinx/serialization/cbor/Cbor.kt

formats/cbor/commonMain/src/kotlinx/serialization/cbor/SerialLabel.kt

formats/cbor/commonMain/src/kotlinx/serialization/cbor/CborArray.kt

formats/cbor/commonMain/src/kotlinx/serialization/cbor/Tags.kt

formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/Encoding.kt

sandwwraith

I didn't look at the test thoroughly (to understand if there is a need for more test cases), but I highlighted all of the problematic moments from my perspective. Don't forget to press 'show hidden conversations' on Github :)

sandwwraith · 2023-12-11T16:20:51Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/ByteStringWrapper.kt

+ * Note that `equals()` and `hashCode()` only use `value`, not `serialized`.
+ */
+@Serializable(with = ByteStringWrapperSerializer::class)
+public class ByteStringWrapper<T>(


NB: it seems you need to update apiDump since this class is no longer data

sorry, done!

sandwwraith · 2023-12-11T16:30:45Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/ByteStringWrapper.kt

+@Serializable(with = ByteStringWrapperSerializer::class)
+public class ByteStringWrapper<T>(
+    public val value: T,
+    public val serialized: ByteArray = byteArrayOf()


I've skimmed through the diff and can't find any usages of this value. Is there any reason to be a val? In what cases will the user be interested in accessing it? Also, it is very easy to create inconsistent data with this approach (e.g. ByteStringWrapper(original.value, byteArrayOf(garbage))). If it is not necessary to have it, this class can be a value class. Or even no class would be needed, as this can be handled with yet another @SerialInfo annotation.

@nodh can you take care of this, please?

We'll use the CBOR serialization for ISO 18013-5 compliaent mobile driving licences. There verifiers need to check that (the digest of) the serialized byte values appear in another structure (signed by the issuer). So in that case its quite useful to have the bytes as they were serialized (and parsed) in the deserialized ByteStringWrapper object too.

formats/cbor/commonMain/src/kotlinx/serialization/cbor/Cbor.kt

sandwwraith · 2023-12-11T17:48:23Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/CborArray.kt

+@SerialInfo
+@Target(AnnotationTarget.CLASS)
+@ExperimentalSerializationApi
+public annotation class CborArray(@OptIn(ExperimentalUnsignedTypes::class) vararg val tag: ULong)


vararg val tag is still undocumented though

sorry! done!

sandwwraith · 2023-12-12T17:38:22Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/Encoding.kt

            readProperties++
-            descriptor.getElementIndexOrThrow(elemName)
+            descriptor.getElementIndexOrThrow(elemName).also { index ->
+                if (cbor.verifyKeyTags) {


This block of duplicated code can be extracted to function (perhaps local)

sandwwraith · 2023-12-12T17:40:32Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/Encoding.kt

                } else {
+                    if (cbor.verifyKeyTags) {
+                        descriptor.getKeyTags(index)?.let { keyTags ->
+                            if (!(keyTags contentEquals tags)) throw CborDecodingException("CBOR tags $tags do not match declared tags $keyTags")


General observation (for all throw expressions): It is would be much easier to debug errors if they also included locations. Here we don't have a json path analog, but appending descriptor.toString() will already make the job easier significantly.

done where we have access to the descriptor (which we do not always have)

sandwwraith · 2023-12-12T17:52:18Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/Encoding.kt

            readByte()
        }
+
+        return (if (collectedTags.isEmpty()) null else collectedTags.toULongArray()).also { collected ->


IMO also creates unnecessary nesting here.

it needs to be transformed into an ULongArray for comparison and for returning it. the alternative of adding a temp variable just for two accesses seems a bit cumbersome to me

sandwwraith · 2023-12-12T17:54:25Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/Encoding.kt

+
+        return (if (collectedTags.isEmpty()) null else collectedTags.toULongArray()).also { collected ->
+            tags?.let {
+                if (!it.contentEquals(collected)) throw CborDecodingException(


Note that contentEquals has both nullable receiver and parameter. It results in behavior like this: intArrayOf(1,2,3).contentEquals(null) -> false. I'm not sure it is what you intended to have here.

We only want to compare if tags are actually set. Otherwise, we don't care.

I'll also add that comment to the code.

sandwwraith · 2023-12-12T17:54:53Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/Encoding.kt

@@ -633,9 +952,55 @@ private fun Iterable<ByteArray>.flatten(): ByteArray {

 @OptIn(ExperimentalSerializationApi::class)
 private fun SerialDescriptor.isByteString(index: Int): Boolean {
-    return getElementAnnotations(index).find { it is ByteString } != null
+    return kotlin.runCatching { getElementAnnotations(index).find { it is ByteString } != null }.getOrDefault(false)


Why runCatching is needed here?

it's only a theoretical possibility, but an IndexOutOfBoundsException could be thrown. I'll remove it.

well, such a call fails for the WASM target, even though we use runCatching. I have no explanation

develar · 2023-12-17T08:54:22Z

Globally preferring major type 2 for byte array encoding

It would be awesome, as currently it's somewhat surprising that the default encoded size of byte arrays is nearly twice as large.

JesusMcCloud · 2024-01-12T10:59:11Z

WASM still fails, because even runCatching does not catch a runtimeException for an expected IndexOutOfBoundsException when checking for tags. Help would be much appreachiated.

JesusMcCloud · 2024-01-12T14:33:57Z

WASM still fails, because even runCatching does not catch a runtimeException for an expected IndexOutOfBoundsException when checking for tags. Help would be much appreachiated.

Here are the details:

  Current test: testOpenPolymorphism
  Process output:
   wasm://wasm/<org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>-00616f4e:1
  
  
  RuntimeError: array element access out of bounds
      at <org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>.kotlin.Array.get (wasm://wasm/<org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>-00616f4e:wasm-function[2207]:0x4247e)
      at <org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>.kotlinx.serialization.descriptors.SerialDescriptorImpl.getElementAnnotations (wasm://wasm/<org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>-00616f4e:wasm-function[3974]:0x595a3)
      at <org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>.kotlinx.serialization.cbor.internal.getKeyTags (wasm://wasm/<org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>-00616f4e:wasm-function[5369]:0x6b6e2)
      at <org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>.kotlinx.serialization.cbor.internal.Preamble.encode (wasm://wasm/<org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>-00616f4e:wasm-function[5180]:0x68fa4)
      at <org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>.kotlinx.serialization.cbor.internal.Token.encode (wasm://wasm/<org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>-00616f4e:wasm-function[5173]:0x68d56)
      at <org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>.kotlinx.serialization.cbor.internal.CborWriter.encode (wasm://wasm/<org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>-00616f4e:wasm-function[5294]:0x6a0c3)
      at <org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>.kotlinx.serialization.cbor.Cbor.encodeToByteArray (wasm://wasm/<org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>-00616f4e:wasm-function[5042]:0x6743b)
      at <org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>.kotlinx.serialization.cbor.CborPolymorphismTest.testOpenPolymorphism (wasm://wasm/<org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>-00616f4e:wasm-function[6111]:0x79d2a)
      at <org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>.kotlin.wasm.internal.testContainer$kotlinx.serialization.cbor test fun$CborPolymorphismTest test fun$testOpenPolymorphism test fun.invoke (wasm://wasm/<org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>-00616f4e:wasm-function[6715]:0x8440e)
      at <org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>.kotlin.test.TeamcityAdapter$test$lambda.invoke (wasm://wasm/<org.jetbrains.kotlinx:kotlinx-serialization-cbor_test>-00616f4e:wasm-function[5605]:0x6f4a9)

EDIT: seems it's already been reported and there's really not much we can do: https://youtrack.jetbrains.com/issue/KT-59081/WASM-Cant-catch-index-out-of-bounds-and-divide-by-0-exceptions

JesusMcCloud · 2024-06-20T18:40:59Z

@sandwwraith I Implemented your suggested simplified approach according to your snippet. This PR is ready for the next review round.

TL;DR: t is working perfectly well and the code is much simpler. Also performs much faster! But the original composeNumber was abysmal to begin with! Deserializing performance is as you mentioned, but that is fine IMHO.

Long story:
Simplifying the code as you suggested had no impact at all it was exactly as slow as the convoluted approach and when switching out ByteArrayOutput for kotlinx-io Buffer the performance tanked completely (third result) unless write is used instead of copyTo (fourth result).
It got even worse: I've removed all the teeny tiny byteArrays that are allocated during tag encoding to operate in-place on the buffer, only to be back to the very same performance numbers as with my original approach. So I got rid of the buffer again.
And now for the kicker: I went ahead split the indefinite length encoder from the definite length encoder and the speed-up was only 10%.
What finally made a huge impact was to removing all the inefficient creation of teeny tiny ByteArrays from encodeByteArray. Fully optimizing the number encoding to always operate directly on the buffer makes things 5% slower, so I did not go further.

The current state of the PR
* CborBaseline.toBytes    thrpt   10  1125.402 ±  2.414  ops/ms
 
Your suggestion (no more deferred encoding)
* CborBaseline.toBytes    thrpt   10  1107.893 ± 5.853  ops/ms

CopyTo Buffer
* CborBaseline.toBytes    thrpt   10   456.103 ± 4.931  ops/ms

Optimized Buffer
* CborBaseline.toBytes    thrpt   10   983.492 ± 6.253  ops/ms

In-Place Number encoding
* CborBaseline.toBytes  thrpt   10  1160.831 ± 8.129  ops/ms

Indefinite length only encoder:
* CborBaseline.toBytes    thrpt   10  1257.923 ± 6.029  ops/ms

Fastest with the mostest
* CborBaseline.toBytes    thrpt   10  1827.695 ± 6.384  ops/ms

JesusMcCloud · 2024-06-20T18:52:29Z

I'm also happy to report that the JS compiler bug can now only be triggered when using empty @ValueTags or empty @KeyTags. This is nonsensical anyhow and translates to a NOOP, so everything is usable in practice

Isolus · 2024-06-25T21:35:54Z

Hi,

I stumbled upon this great MR while trying to work with some CBOR structures and I have two questions:

The ByteStringWrapper seems to create it's own internal Cbor instance and doesn't follow the settings used to serialize/deserialize the outer structures. Am I overlooking a way to influence the behaviour of ByteStringWrapper?
I see examples to tag classes when I want them to be arrays with @CborArray(Tag()). Can I tag classes when they are maps somehow? And if I use Map or List to work in a more dynamic way can I somehow verify/write tags for the whole map/array and/or the respective elements?

sandwwraith

I really like new approach — I think you can agree that it is much cleaner and requires much less code.

Check the CI failure — you need to add @SuppressAnimalSniffer to CborDecoder.processTags

sandwwraith · 2024-06-26T14:59:03Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/EncodingUtils.kt

+
+@OptIn(ExperimentalSerializationApi::class)
+internal fun SerialDescriptor.isByteString(index: Int): Boolean {
+    return kotlin.runCatching { getElementAnnotations(index).find { it is ByteString } != null }.getOrDefault(false)


Instead of abusing runCatching, it's better to use helper function like this one:

kotlinx.serialization/core/commonTest/src/kotlinx/serialization/TestId.kt

Line 16 in 0e4e0e7

public inline fun <reified A: Annotation> SerialDescriptor.findAnnotation(elementIndex: Int): A? {

Also, in this particular case, you can just do getElementAnnotations(index).any { it is ByteString }

I tried that already before preparing the updated PR. To my surprise, it lead to a slight, but noticeable performance hit according to JMH, so I kept the runCatching

find does not throw exceptions, so runCatching literally does nothing here.

Current benchmark does not write tags (only verifies them, which is incorrect btw — wrote about that separately)

Did you have other benchmark data with labels? I've re-measured PR state vs my diff (attached below) and got no significant difference:

PR Benchmark Mode Cnt Score Error Units CborBaseline.fromBytes thrpt 20 1637.134 ± 17.891 ops/ms CborBaseline.toBytes thrpt 20 2242.036 ± 39.286 ops/ms PR_WITH_PATCH Benchmark Mode Cnt Score Error Units CborBaseline.fromBytes thrpt 20 1641.116 ± 43.964 ops/ms CborBaseline.toBytes thrpt 20 2126.622 ± 96.746 ops/ms

Patch link: https://gist.github.com/sandwwraith/1cc3bf4dc1b1658af34be85dcbb37976

I'll re-check (don't know If I still have the un-squashed commits in another branch locally to re-trace my steps), maybe I mixed something up.

While find does not throw, I seem to remember the runCatching (maybe not in that particular line, but somewhere in what is now EncodingUtils.kt) triggered KT-59081, because getElementAnnotations can throws an OOB exception. Apparently, that was just sloppiness, if it is working fine now.

OOBE from getElementAnnotations means that you've made a mistake with indices somewhere, so I would rather not swallow this exception, since it indicates the bug in the encoder/decoder.

done in 897ff72

sandwwraith · 2024-06-26T15:05:57Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/EncodingUtils.kt

+
+@OptIn(ExperimentalSerializationApi::class)
+internal fun SerialDescriptor.hasArrayTag(): Boolean {
+    return annotations.filterIsInstance<CborArray>().isNotEmpty()


Better to use annotations.any { it is CborArray } to avoid allocation of intermediate array

fixed in 897ff72

sandwwraith · 2024-06-26T15:19:34Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/Decoder.kt

+
+@OptIn(ExperimentalSerializationApi::class)
+private fun SerialDescriptor.getElementNameForCborLabel(label: Long): String? {
+    return elementNames.firstOrNull { getCborLabel(getElementIndex(it)) == label }


using firstOrNull here causes full iteration over descriptor, and since this function is called for each key-value pair in the input, it may lead to a quadratic time complexity — in case of input where labels are predominately used over properties' names. To avoid this, you have to consider caching labels per descriptor. @JsonNames annotation has similar problems; you can take a look how cache for it is implemented here:

kotlinx.serialization/formats/json/commonMain/src/kotlinx/serialization/json/internal/JsonNamesMap.kt

Line 18 in 194a188

private fun SerialDescriptor.buildDeserializationNamesMap(json: Json): Map<String, Int> {

However, to keep this review size reasonable, let's do this after this PR. I assume the performance of CBOR label decoding is in your best interest :)

Or perhaps Protobuf implementation would be closer representation, since @CborLabel does not permit multiple values in a contrary to @JsonNames:

kotlinx.serialization/formats/protobuf/commonMain/src/kotlinx/serialization/protobuf/internal/ProtobufDecoding.kt

Line 41 in 251bca7

public fun populateCache(descriptor: SerialDescriptor) {

So you'll open up an issue after this PR is through?

sandwwraith · 2024-06-26T15:26:54Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/CborLabel.kt

+import kotlinx.serialization.*
+
+/**
+ * CBOR supports *labels*, which work just as `SerialNames`. The key difference is that labels are not strings,


I see the termin 'label' only in COSE specification (https://datatracker.ietf.org/doc/html/rfc8152#section-1.4), while regular CBOR spec does not have this word anywhere. Also, as far as I remember, regular CBOR key can be of any type, not just integer or string. I think this is worth mentioning here in the docs — that labels are mainly used for COSE.

I amended the documentation in 8389089 but decided to keep the name CborLabel and not switch over to CoseLabel.
In the future a more generic annotation would be perfect, which supports arbitrary types as map keys for full compliance. To be honest, though, I don't think anything apart from string and numbers is used in practice.

sandwwraith · 2024-06-26T15:27:30Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/Decoder.kt

@@ -446,8 +363,10 @@ internal class CborDecoder(private val input: ByteArrayInput) {
        return array
    }

-    fun nextFloat(): Float {
-        skipOverTags()
+    fun nextFloat(tag: ULong) = nextFloat(ulongArrayOf(tag))


This function is unused

fixed in a45daf7

sandwwraith · 2024-06-26T16:08:57Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/Tags.kt

+@Target(AnnotationTarget.PROPERTY)
+@ExperimentalSerializationApi
+public annotation class ValueTags(@OptIn(ExperimentalUnsignedTypes::class) vararg val tags: ULong) {
+    public companion object {


Companion undocumented

fixed in 49b3ea2

sandwwraith · 2024-06-26T16:31:53Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/Encoder.kt

+        val label = descriptor.getCborLabel(index)
+
+        if (!descriptor.hasArrayTag()) {
+            if (cbor.writeKeyTags) descriptor.getKeyTags(index)?.forEach { getDestination().encodeTag(it) }


I suggest extracting getDestination() to a local variable to avoid getting it again every time

sandwwraith · 2024-06-26T16:56:18Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/Encoder.kt

+        val label = descriptor.getCborLabel(index)
+
+        if (!descriptor.hasArrayTag()) {
+            if (cbor.writeKeyTags) descriptor.getKeyTags(index)?.forEach { output.encodeTag(it) }


Besides difference in output vs getDestination() this method is identical to CborWriter.encodeElement(). I suggest removing it completely. (and add val output = getDestination() to common version as I suggested there )

done in 682f363

sandwwraith · 2024-06-26T16:58:30Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/Encoder.kt

+}
+
+//optimized definite length encoder
+internal open class DefiniteLengthCborWriter(cbor: Cbor, output: ByteArrayOutput) : CborWriter(cbor, output) {


I think there is no need for it to be open now?

done in 3d8768c

sandwwraith · 2024-06-26T17:00:38Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/Encoder.kt

+//value classes are only inlined on the JVM, so we use a typealias and extensions instead
+private typealias Stack = MutableList<CborWriter.Data>
+
+private fun Stack(vararg elements: CborWriter.Data): Stack = mutableListOf(*elements)


Is argument needs to be vararg though? I see it is only called with 1 element

JesusMcCloud · 2024-06-27T07:08:00Z

Hi,

I stumbled upon this great MR while trying to work with some CBOR structures and I have two questions:
1. The ByteStringWrapper seems to create it's own internal Cbor instance and doesn't follow the settings used to serialize/deserialize the outer structures. Am I overlooking a way to influence the behaviour of ByteStringWrapper?

Nope, that was just an oversight and needs fixing.

2. I see examples to tag classes when I want them to be arrays with @CborArray(Tag()). Can I tag classes when they are maps somehow? And if I use Map or List to work in a more dynamic way can I somehow verify/write tags for the whole map/array and/or the respective elements?

I don't see how that would work. You would need to individually tag specific keys and values, which is just impossible

sandwwraith · 2024-06-27T14:21:16Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/Cbor.kt

    override val serializersModule: SerializersModule
 ) : BinaryFormat {

    /**
     * The default instance of [Cbor]
     */
-    public companion object Default : Cbor(false, false, EmptySerializersModule())
+    public companion object Default :
+        Cbor(false, false, true, true, true, true, false, true, false, EmptySerializersModule())


I see that you enabled new flags (write/verifyTags, preferCborLabels) by default. We can't do that, because it will change output format in the right moment users will get the new version of the library without their explicit consent

fixed in f4b43e7

sandwwraith · 2024-06-27T14:31:21Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/internal/Encoder.kt

+            if ((descriptor.kind !is StructureKind.LIST) && (descriptor.kind !is StructureKind.MAP) && (descriptor.kind !is PolymorphicKind)) {
+                //indices are put into the name field. we don't want to write those, as it would result in double writes
+                if (cbor.preferCborLabelsOverNames && label != null) {
+                    getDestination().encodeNumber(label)


Since we do not need label value unless cbor.preferCborLabelsOverNames is true, it's better to move val name = descriptor.getElementName(index)/ val label = descriptor.getCborLabel(index) line under this if so we won't access annotation list unless it is necessary:

var label: Long? = null if (cbor.preferCborLabelsOverNames) label = descriptor.getCborLabel(index) if (label != null) output.encodeNumber(label) else output.encodeString(descriptor.getElementName(index))

fixed in 2439e5e

JesusMcCloud · 2024-07-02T12:14:24Z

I obviously can't run the CI myself, but I tried to address everything. Ready for the next round of reviews. Thanks everyone for the detailed feedback! It really helped to clear up some things that were not working the way they are supposed to.

whyoleg · 2024-07-02T13:05:46Z

formats/cbor/commonMain/src/kotlinx/serialization/cbor/CborEncoder.kt

+
+package kotlinx.serialization.cbor
+
+public interface CborEncoder {


shouldn't it be two interfaces like CborEncoder and CborDecoder as in JsonEncoder and JsonDecoder

At the current moment it feels like a dangling type

Yeah, naming is not optimal but for whatever reason when I add a dedicated CborDecoder and rename the currently internal CborEncoder, which handles low-level stuff, to LowLevelCborEncoder the WASI tests glitch out with empty error messages and and empty test report, complaining about failing tests.
So while I agree, I don't even know how to debug this weird behaviour, which, to me, is unintelligible.

Managed to split it up by starting from scratch in reverse order. @sandwwraith feel free to use or ignore 82f326f as desired.

JesusMcCloud changed the base branch from master to dev August 18, 2023 11:18

This was referenced Aug 18, 2023

CBOR: Use Serial Labels #2371

Closed

CBOR: Basic Support for Tags #2359

Closed

Support definite-length Arrays and Maps in CBOR #1955

Open

CBOR tagged data can't be deserialized #1560

Open

sandwwraith reviewed Sep 22, 2023

View reviewed changes

JesusMcCloud requested a review from sandwwraith October 6, 2023 18:21

sandwwraith requested changes Dec 12, 2023

View reviewed changes

sandwwraith mentioned this pull request Dec 15, 2023

Support encoding/decoding ByteArrays in parameterized types as ByteStrings. #2383

Open

JesusMcCloud requested a review from sandwwraith January 12, 2024 10:59

sandwwraith self-assigned this Apr 17, 2024

JesusMcCloud and others added 15 commits June 7, 2024 19:30

CBOR basic value tagging

b1182ec

CBOR: handle tagged keys

47d6233

CBOR: make verifying and writing tags optional

87bcda3

CBOR: fix decoding value tags, add tests

d3eac6b

CBOR: refactor Tagged → ValueTags

2788ec2

CBOR: Fixes (docs, whitespace, names, superflous params)

c620fdb

CBOR: serialize cbor to tree

0d1b31b

CBOR: tree fixes

4f82ec8

CBOR: frankensteined data tree

1356666

CBOR: fix direct encoding of primitives

76afa35

CBOR: write definite lengths

ce9eefd

CBOR: experiment with pruning null properties

8b328f4

CBOR: Catch errors on finding annotations

ad97cbb

CBOR: Implement labels to use as map keys

ef1f77d

WIP: Ignore failing tests

83824b6

JesusMcCloud added 5 commits June 20, 2024 20:30

CBOR: Partial performance optimization

ea64d54

CBOR: simplify encoding

674426d

CBOR: split up encoders

7b745f5

CBOR: peak performance

b86f5ca

CBOR: Update API Dump

d7397b2

sandwwraith requested changes Jun 26, 2024

View reviewed changes

sandwwraith reviewed Jun 27, 2024

View reviewed changes

JesusMcCloud added 13 commits July 1, 2024 10:12

CBOR: fix EncodingUtils

897ff72

CBOR: Clarify CborLabel docs

8389089

CBOR: remove unused functions

a45daf7

COSE: Remove ByteStringWrapper

00185f5

CBOR: document CborArray Tags annotation

dc07e3c

CBOR: document predefined tags

49b3ea2

CBOR: revamp tagging

aeb83cf

CBOR: pull up encoder method

682f363

CBOR: clode DefiniteLengthCborWriter

3d8768c

CBOR: fix default instance writing and verifying tags

f4b43e7

CBOR: fix: uncomment tag test

13e2eb0

CBOR: get labels only when preferred over serial names

2439e5e

CBOR: run knit and apiDump

759ffce

JesusMcCloud requested a review from sandwwraith July 2, 2024 12:14

CBOR: fix readme

13e3d5a

whyoleg reviewed Jul 2, 2024

View reviewed changes

CBOR: split encoder and decoder interfaces

82f326f


		package kotlinx.serialization.cbor

		public interface CborEncoder {

CBOR Feature Drop: COSE #2412

Are you sure you want to change the base?

CBOR Feature Drop: COSE #2412

Conversation

JesusMcCloud commented Aug 18, 2023

sandwwraith commented Aug 21, 2023

sandwwraith left a comment

Choose a reason for hiding this comment

sandwwraith left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

develar commented Dec 17, 2023

JesusMcCloud commented Jan 12, 2024

JesusMcCloud commented Jan 12, 2024 • edited Loading

JesusMcCloud commented Jun 20, 2024 • edited Loading

JesusMcCloud commented Jun 20, 2024

Isolus commented Jun 25, 2024

sandwwraith left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JesusMcCloud commented Jun 27, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sandwwraith Jun 27, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JesusMcCloud commented Jul 2, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sandwwraith left a comment •

edited

Loading

JesusMcCloud commented Jan 12, 2024 •

edited

Loading

JesusMcCloud commented Jun 20, 2024 •

edited

Loading

sandwwraith Jun 27, 2024 •

edited

Loading