From 5ae9781df918f15bd7b50bad1a0535b7b5b28290 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bj=C3=B6rn=20Tackmann?= <54846571+Dfinity-Bjoern@users.noreply.github.com> Date: Thu, 5 Aug 2021 14:59:10 +0200 Subject: [PATCH 001/102] Release version 0.18.1 (#364) --- spec/changelog.adoc | 6 +++ spec/index.adoc | 115 +++++++++++++++++++++----------------------- 2 files changed, 60 insertions(+), 61 deletions(-) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 3e7417f9b..a41183e8f 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,12 @@ [#changelog] == Changelog +[#0_18_1] +=== 0.18.1 (2021-08-04) + +* Spec: Support RSA PKCS#1 v1.5 signatures in web authentication +* Spec clarification: Fix various typos and improve textual clarity + [#0_18_0] === 0.18.0 (2021-05-18) diff --git a/spec/index.adoc b/spec/index.adoc index 4fcb061ef..af923d026 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,23 +1,23 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.0 +0.18.1 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ == Introduction -Welcome to the Internet Computer! We speak of “the” Internet Computer, because although under the hood a large number of physical computers are working together in non-trivial ways, in the end we have the appearance of a single, shared, secure and world-wide accessible computer. Much, if not all, of the advanced and complex machinery is hidden from those that use the Internet Computer to run their applications and those who use these applications. +Welcome to the Internet Computer! We speak of “the” Internet Computer, because although under the hood a large number of physical computers are working together in non-trivial ways, in the end we have the appearance of a single, shared, secure and world-wide accessible computer. Developers who want to build applications that run on the Internet Computer and end-users who want to use those applications need to know very little, if anything, about the underlying complexity of how the Internet Computer works. However, knowing some details about the interfaces that the Internet Computer exposes can allow interested developers and architects to take fuller advantages of the unique features that the Internet Computer provides. === Target audience -This documents describes this _external_ view of the Internet Computer, i.e. the low-level interfaces it provides to application developers and users, and what will happen when they use these interfaces. +This document describes this _external_ view of the Internet Computer, i.e. the low-level interfaces it provides to application developers and users, and what will happen when they use these interfaces. NOTE: While this document describes the external interface and behavior of the Internet Computer, it is not end-user or end-developer documentation. Most developers will interact with the Internet Computer through additional tooling like the SDK, Canister Development Kits and Motoko. Please see https://sdk.dfinity.org/ for suitable documentation. The target audience of this document are -* those who use the these low-level interfaces (e.g. implement agents, canister developments kits, emulators, other tooling). +* those who use these low-level interfaces (e.g. implement agents, canister developments kits, emulators, other tooling). * those who implement these low-level interfaces (e.g. developers of the Internet Computer implementation) * those who want to understand the intricacies of the Internet Computer’s behaviour in great detail (e.g. to do a security analysis) @@ -31,11 +31,11 @@ This document tries to be implementation agnostic: It would apply just as well t === Overview of the Internet Computer -If you want to use the Internet Computer as an application developer, you first create a _canister module_ that contains the WebAssembly code and configuration for your application, and deploy it using the <>. You can create canisters using the Motoko language and the SDK, which is more convenient. If you want to use your own tooling, however, then this document describes <> and how the <>. +If you want to use the Internet Computer as an application developer, you first create a _canister module_ that contains the WebAssembly code and configuration for your application, and deploy it using the <>. You can create canisters using the Motoko language and the SDK, which is more convenient. If you want to use your own tooling, however, then this document describes <> and how the <>. -Once your application is running on the Internet Computer, it is a _canister_, and users can interact with it. They can use the <> to interact with the canister according to the <>. +Once your application is running on the Internet Computer, it is a _canister_, and users can interact with it. They can use the <> to interact with the canister according to the <>. -The user can also use the HTTP interface to issue read-only queries, which are faster, but cannot change the state of a canister. +The user can also use the HTTPS interface to issue read-only queries, which are faster, but cannot change the state of a canister. .A typical use of the Internet Computer. (This is a simplified view; some of the arrows represent multiple interaction steps or polling.) [plantuml] @@ -81,7 +81,7 @@ Canisters and users are identified by a _principal_, sometimes also called an _i == Pervasive concepts -Before going into the details of the four public interfaces describes in this document (namely the agent-facing <>, the canister-facing <>, the <> and the <>), this section introduces some concepts that transcend multiple interfaces. +Before going into the details of the four public interfaces described in this document (namely the agent-facing <>, the canister-facing <>, the <> and the <>), this section introduces some concepts that transcend multiple interfaces. === Unspecified constants and limits @@ -102,7 +102,7 @@ Maximum canister cycle balance. Any excess is discarded. Less than 2^64^. [#principal] === Principals -Principal are generic identifiers for canisters, users and possibly other concepts in the future. As far as most uses of the system are concerned they are _opaque_ binary blobs with a length between 0 and 29 bytes, and there is intentionally no mechanism to tell canister id and users ids apart. +Principals are generic identifiers for canisters, users and possibly other concepts in the future. As far as most uses of the system are concerned they are _opaque_ binary blobs with a length between 0 and 29 bytes, and there is intentionally no mechanism to tell canister id and users ids apart. There is, however, some structure to them to encode specific authentication and authorization behavior. @@ -262,9 +262,10 @@ Plain signatures are supported for the schemes [#webauthn] ==== Web Authentication -The only allowed signature scheme for web authentication is +The allowed signature schemes for web authentication are * https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf[*ECDSA*] on curve P-256 (also known as `secp256r1`), using SHA-256 as hash function. +* https://datatracker.ietf.org/doc/html/rfc8017#section-8.2[*RSA PKCS#1v1.5 (RSASSA-PKCS1-v1_5)*], using SHA-256 as hash function. The signature is calculated by using the payload as the challenge in the web authentication assertion. @@ -307,7 +308,7 @@ The IC also supports a scheme where a canister can sign a payload by declaring a This section makes forward references to other concepts in this document, in particular the section <>. -* The public key is a DER-wrapped structure that indicates the _signing canister_, and includes a freely choosable seed. Each choice of seed yields a distinct public key for the canister, and the canister can chose to encode information, such as a user id, in the seed. +* The public key is a DER-wrapped structure that indicates the _signing canister_, and includes a freely choosable seed. Each choice of seed yields a distinct public key for the canister, and the canister can choose to encode information, such as a user id, in the seed. + More concretely, it uses the `SubjectPublicKeyInfo` type used for other types of public keys (see, e.g., https://tools.ietf.org/html/rfc8410#section-4[RFC 8410, Section 4]), with OID 1.3.6.1.4.1.56387.1.2 (iso.org.dod.internet.private.enterprise.dfinity.mechanisms.canister-signature). + @@ -344,7 +345,7 @@ Parts of the system state are publicly exposed (e.g. via <> or Conceptually, the system state is a tree with labeled children, and values in the leaves. Equivalently, the system state is a mapping from paths (sequences of labels) to values, where the domain is prefix-free. -Labels are always blobs (but often with an human readable representation). In this document, paths are written suggestively with slashes as separators; the actual encoding is not actually using slashes as delimitors, and labels may contain the 0x2F byte (ASCII `/`) just fine. Values are either natural numbers, text values or blob values. +Labels are always blobs (but often with a human readable representation). In this document, paths are written suggestively with slashes as separators; the actual encoding is not actually using slashes as delimiters, and labels may contain the 0x2F byte (ASCII `/`) just fine. Values are either natural numbers, text values or blob values. This section specifies the publicly relevant paths in the tree. @@ -417,7 +418,7 @@ The current controllers of the canister. The value consists of a CBOR data item The concrete mechanism that users use to send requests to the Internet Computer is via an HTTPS API, which exposes three endpoints to handle interactions with the internet computer, plus one for diagnostics: * At `/api/v2/canister//call` the user can submit (asynchronous, state-changing) calls. -* At `/api/v2/canister//read_state` the user can read various sytem information. In particular, they can poll for the status of a call here. +* At `/api/v2/canister//read_state` the user can read various system information. In particular, they can poll for the status of a call here. * At `/api/v2/canister//query` the user can perform (synchronous, non-state-changing) query calls * At `/api/v2/status` the user can get additional information about the network @@ -437,7 +438,7 @@ This implies the following high-level workflow: 1. A user submits a call via the <>. No useful information is returned from the node (as such information cannot be trustworthy anyways). 2. For a certain amount of time, the system behaves as if it does not know about the call. -3. The system asks the targetted canister if it is willing to to accept this message and be charged for the expense of processing it. This uses the <> API for normal calls. For calls to the management canister, the rules in <> apply. +3. The system asks the targeted canister if it is willing to accept this message and be charged for the expense of processing it. This uses the <> API for normal calls. For calls to the management canister, the rules in <> apply. 4. At some point, the system may accept the call for processing and set its status to `received`. This indicates that the system as a whole has received the call and plans on processing it (although it may still not get processed if the system is under high load). Furthermore, the user should also be able to ask any endpoint about the status of the pending call. 5. Once it is clear that the call will be acted upon (sufficient resources, call not yet expired), the status changes to `processing`. Now the user has the guarantee that the request will have an effect, e.g. it will reach the target canister. 6. Now the system is processing the call. For some calls this may be atomic, for others this involves multiple internal steps. @@ -475,7 +476,7 @@ State transitions may be instantaneous and not always externally visible. For ex All gray states are _not_ explicitly represented in the system state, and are indistinguishable from “call does not exist”. -The characteristic property of the `received` state is that the call has made it past the (potentionally malicious) endpoint _into to the system_. It is now pointless (but harmless) to submit the (identical) call again. Before reaching that state, submitting the identical call to further nodes might be a useful safeguard against a malicious or misbehaving node. +The characteristic property of the `received` state is that the call has made it past the (potentially malicious) endpoint _into the system_. It is now pointless (but harmless) to submit the (identical) call again. Before reaching that state, submitting the identical call to further nodes might be a useful safeguard against a malicious or misbehaving node. The characteristic property of the `processing` state is _the initial effect of the call has or will happen_. This is best explained by an example: Consider a counter canister. It exports a method `inc` that increases the counter. Assume that the canister is bug free, and is not going to be forcibly removed. A user submits a call to call `inc`. If the user sees request status `processing`, the state change is guaranteed to happen, and the user can stop monitoring the status and does not have to retry submitting. @@ -573,20 +574,20 @@ The `` in the URL paths of requests is the _effective_ de ==== Production instances do not support `provisional_create_canister_with_cycles` anyways. This means that using an effective canister id that could be an existing canister would lead to the request being routed by the edge component to a node the corresponding subnet and produce an error response there, while using an effective canister id that cannot be an existing canister id would cause an error response from the edge component -- either is fine. -In non-production, multi-subnet instances (e.g. testnets), users with priviledged access to `provisional_create_canister_with_cycles` (or possibly similar, internal methods) and knowledge of the mapping from canisters to subnets can use their choice of the effective canister id in the URL to steer the request to a specific subnet. +In non-production, multi-subnet instances (e.g. testnets), users with privileged access to `provisional_create_canister_with_cycles` (or possibly similar, internal methods) and knowledge of the mapping from canisters to subnets can use their choice of the effective canister id in the URL to steer the request to a specific subnet. In local, single-subnet instances, the effective canister id is ignored by the single instance, and thus `aaaaa-aa` can be used. ==== * Else, the effective canister id must be the `canister_id` in the request. -NOTE: The expecation is that user-side agent code shields users and developers from this concept, in analogy to how the system interface shields canister developers from worrying about routing. +NOTE: The expectation is that user-side agent code shields users and developers from this concept, in analogy to how the system interface shields canister developers from worrying about routing. [#authentication] === Authentication -All requests coming in via the HTTP interface need to be either _anonymous_ or _authenticated_ using a cryptographic signature. To that end, the following fields are present in the `content` map in all cases: +All requests coming in via the HTTPS interface need to be either _anonymous_ or _authenticated_ using a cryptographic signature. To that end, the following fields are present in the `content` map in all cases: * `nonce` (`blob`, optional): Arbitrary user-provided data, typically randomly generated. This can be used to create distinct requests with otherwise identical fields. * `ingress_expiry` (`nat`, required): An upper limit on the validity of the request, expressed in nanoseconds since 1970-01-01 (like <>). This avoids replay attacks: The system will not accept requests, or transition requests from status `received` to status `processing`, if their expiry date is in the past. The system may refuse to accept requests with an ingress expiry date too far in the future. This applies to synchronous and asynchronous requests alike (and could have been called `signature_expiry`). @@ -609,7 +610,7 @@ The request id (see <>) is calculated from the content record. This The field `sender_pubkey` contains a public key supported by one of the schemes described in <>. -Signing transactions can be delegated from a one key to another one. +Signing transactions can be delegated from one key to another one. If delegation is used, then the `sender_delegation` field contains an array of delegations, each of which is a map with the following fields: * `delegation` (`map`): Map with fields: @@ -649,9 +650,9 @@ The following encodings of field values as blobs are used [#request-id] === Request ids -When signing requests or querying the status of a request (see <>) in the state tree, the user identifies the request using a _request id_, which the <> of the `content` map of the original request. +When signing requests or querying the status of a request (see <>) in the state tree, the user identifies the request using a _request id_, which is the <> of the `content` map of the original request. -NOTE: The request id is independent of the representation of the request (currenly only CBOR), and does not change if the specification adds further optional field to a request type. +NOTE: The request id is independent of the representation of the request (currently only CBOR), and does not change if the specification adds further optional fields to a request type. NOTE: The recommended textual representation of a request id is a hexadecimal string with lower-case letters prefixed with '0x'. E.g., request id consisting of bytes `[00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 0A, 0B, 0C, 0D, 0E, 0F, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 1A, 1B, 1C, 1D, 1E, 1F]` should be displayed as `0x000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f`. @@ -724,7 +725,7 @@ See <> for details on the precise CBOR encoding of this object. NOTE: Future additions may include local time, geographic location, and other useful implementation-specific information such as blockheight. This data may -possibly signed by the node. +possibly be signed by the node. [#api-cbor] === CBOR encoding of requests and responses @@ -790,7 +791,7 @@ More precisely: to the same canister, they are queued in the order of invocations to `ic0.call_perform`. * Responses (including replies with `ic0.msg_reply`, explicit rejects with `ic0.msg_reject` and system-generated error responses) do _not_ have any ordering guarantee relative to each other or to method calls. * There is no particular order guarantee for ingress messages submitted via - the HTTP interface. + the HTTPS interface. === Synchronicity across nodes @@ -1003,7 +1004,7 @@ The message argument data. ic0.msg_caller_size : () -> i32 ic0.msg_caller_copy : (dst : i32, offset: i32, size : i32) -> () + -The identity of the caller, which may be a canister id or a user id. During canister installation or upgrade, this is the id of of the user or canister requesting the installation or upgrade. +The identity of the caller, which may be a canister id or a user id. During canister installation or upgrade, this is the id of the user or canister requesting the installation or upgrade. * `+ic0.msg_reject_code : () -> i32+` + @@ -1016,7 +1017,7 @@ It returns the special “no error” code `0` if the callback is _not_ invoked ic0.msg_reject_msg_size : () -> i32 ic0.msg_reject_msg_copy : (dst : i32, offset : i32, size : i32) -> () + -The reject message. Traps if there there is no reject message (i.e. if `reject_code` is `0`). +The reject message. Traps if there is no reject message (i.e. if `reject_code` is `0`). [#responding] === Responding @@ -1025,7 +1026,7 @@ Eventually, the canister will want to respond to the original call, either by re * `+ic0.msg_reply_data_append : (src : i32, size : i32) -> ()+` + -Appends idata it to the (initially empty) data reply. +Appends data it to the (initially empty) data reply. + NOTE: This can be invoked multiple times to build up the argument with data from various places on the Wasm heap. This way, the canister does not have to first copy all the pieces from various places into one location. + @@ -1111,11 +1112,11 @@ When handling an update call (or a callback), a canister can do further calls to Begins assembling a call to the canister specified by `callee_src/_size` at method `name_src/_size`. -The system records two mandatory callback functions, represented by a table entry index `fun` and some additional value `env`. When the response comes back, the table is read at the corresponding index, expected to be a function of type `+(env : i32) -> ()+` and passed the corresponding `env` value. +The system records two mandatory callback functions, represented by a table entry index `*_fun` and some additional value `*_env`. When the response comes back, the table is read at the corresponding index, expected to be a function of type `+(env : i32) -> ()+`, and passed the corresponding `*_env` value. The reply callback is executed upon successful completion of the method call, which can query the reply using `ic0.msg_arg_data_*`. -The reject callback is executed if the method calls fails asynchronously or the other canister explicitly rejects the call. The reject code and message can be queried using `ic0.msg_reject_code` and `ic0.msg_reject_msg_*`. +The reject callback is executed if the method call fails asynchronously or the other canister explicitly rejects the call. The reject code and message can be queried using `ic0.msg_reject_code` and `ic0.msg_reject_msg_*`. This deducts `MAX_CYCLES_PER_RESPONSE` cycles from the canister balance and sets them aside for response processing. This will trap if not sufficient cycles are available. @@ -1127,7 +1128,7 @@ Subsequent calls to the following functions set further attributes of that call, -- ic0.call_on_cleanup : (fun : i32, env : i32) -> () -If the a cleanup callback is specified for this call, it is executed if and only if the `reply` or the `reject` callback was executed and trapped (for any reason). +If a cleanup callback (of type `+(env : i32) -> ()+`) is specified for this call, it is executed if and only if the `reply` or the `reject` callback was executed and trapped (for any reason). During the execution of the `cleanup` function, only a subset of the System API is available (namely `ic0.debug_print`, `ic0.trap` and the `ic0.stable_*` functions). The cleanup function is expected to run swiftly (within a fixed, yet to be specified cycle limit) and serves to free resources associated with the callback. @@ -1136,14 +1137,14 @@ If this traps (e.g. runs out of cycles), the state changes from the `cleanup` fu There must be at most one call to `ic0.call_on_cleanup` between `ic0.call_new` and `ic0.call_perform`. -- -* `+ic0.call_data_append : (src : i32, size : i32)+` +* `+ic0.call_data_append : (src : i32, size : i32) -> ()+` + Appends the specified bytes to the argument of the call. Initially, the argument is empty. + This may be called multiple times between `ic0.call_new` and `ic0.call_perform`. -* `+ic0.call_cycles_add : ( amount : i64) -> ()+` +* `+ic0.call_cycles_add : (amount : i64) -> ()+` + This adds cycles onto a call. See <>. + @@ -1153,11 +1154,11 @@ This may be called multiple times between `ic0.call_new` and `ic0.call_perform`. + This call concludes assembling the call. It queues the call message to the given destination, but does not actually act on it until the current WebAssembly function returns without trapping. + -If the system returns `0` as the `err_code`, the system was able to enqueue the call. In this case, the call will either be delivered, returned because the destination canister does not exist or returned because of an out of cycles condition. This also means that exactly one of the reply or reject callback will be executed. +If the system returns `0` as the `err_code`, the system was able to enqueue the call. In this case, the call will either be delivered, returned because the destination canister does not exist or returned because of an out of cycles condition. This also means that exactly one of the reply or reject callbacks will be executed. + If the system returns a non-zero value, the call cannot (and will not be) performed. This can happen due to a lack of system internal resources, but also if it would reduce the current cycle balance to a level below where the canister would be frozen. + -After `ic0.call_perform` and before the next call to `ic0.call_new`, all other `ic0.call_*` function calls trap +After `ic0.call_perform` and before the next call to `ic0.call_new`, all other `ic0.call_*` function calls trap. [#system-api-cycles] === Cycles @@ -1176,7 +1177,7 @@ NOTE: Should a future version of this specification allow `MAX_CANISTER_BALANCE` + returns the amount of cycles that were transferred by the caller of the current call, and is still available in this message. + -Initially, in the update method entry point, this is the amount that the caller passed to the canister. When cycles are accepted (`ic0.msg_cycles_accept`), this reports less cycles accordingly. When the call is responsed to (reply or reject), all available cycles are refunded to the caller, and this will return 0. +Initially, in the update method entry point, this is the amount that the caller passed to the canister. When cycles are accepted (`ic0.msg_cycles_accept`), this reports less cycles accordingly. When the call is responded to (reply or reject), all available cycles are refunded to the caller, and this will return 0. * `ic0.msg_cycles_accept : ( max_amount : i64 ) -> ( amount : i64 )` + @@ -1187,11 +1188,11 @@ This moves cycles from the call to the canister balance. It moves as many cycles * It moves no more cycles than available according to `ic0.msg_cycles_available`, and -* The canister balance afterwards does not exceeed `MAX_CANISTER_BALANCE` minus any possible outstanding balances. The precise logic of this limit is not yet specified here. +* The canister balance afterwards does not exceed `MAX_CANISTER_BALANCE` minus any possible outstanding balances. The precise logic of this limit is not yet specified here. It can be called multiple times, each time possibly adding more cycles to the balance. -The return value indicates how much cycles were actually moved. +The return value indicates how many cycles were actually moved. This does not trap. @@ -1205,7 +1206,7 @@ Example: To accept all cycles provided in a call, invoke `ic0.msg_cycles_accept( + This function moves cycles from the canister balance onto the call under construction, to be transferred with that call. + -The cycles are deducted from the balance as shown by `ic0.canister_cycle_balance` immediately, and moved back if the call cannot be performed (e.g. if `ic0.call_perform` signals an error, or if the canister invokes `ic0.call_new` or returns without callling `ic0.call_perform`). +The cycles are deducted from the balance as shown by `ic0.canister_cycle_balance` immediately, and moved back if the call cannot be performed (e.g. if `ic0.call_perform` signals an error, or if the canister invokes `ic0.call_new` or returns without calling `ic0.call_perform`). + This traps if trying to transfer more cycles than are in the current balance of the canister. @@ -1312,7 +1313,7 @@ The environment may copy out the data specified by `src` and `size`, and log, pr The Internet Computer aims to make the most of the WebAssembly platform, and embraces WebAssembly features. With WebAssembly host references, we can make the platform more secure, the interfaces more abstract and more compositional. The above `ic0` System API does not yet use WebAssembly host references. Once they become available on our platform, a new version of the System API using host references will be available via the `ic` module. The changes will be, at least -1. The introduction of a `api_nonce` reference, which models the capability to use the System API. It is passed as an argument to `canister_init`, `canister_update ` etc., and expected as an argument by almost all system function calls. (The debugging aids remain unconstrainted.) +1. The introduction of a `api_nonce` reference, which models the capability to use the System API. It is passed as an argument to `canister_init`, `canister_update ` etc., and expected as an argument by almost all system function calls. (The debugging aids remain unconstrained.) 2. The use of references, instead of binary blobs, to address principals (user ids, canister ids), e.g. in `ic0.msg_caller` or in `ic0.call_new`. Additional functions will be provided to convert between the transparent binary representation of principals and references. 3. Making the builder interface to create calls build calls identified by a reference, rather than having an implicit partial call in the background. @@ -1492,7 +1493,7 @@ This method is only available in local development instances, and will be remove [#certification] == Certification -Some parts of the system state are exposed to clients in a tamperproof way via certification: the system can reveal a _partial state tree_ which includes just the data of interest, together with a signature on the root hash of the state tree. This means that a client can be sure that the response is correct, even if the client happens to be communicating with a malicious server, or have received the certificate via some other untrusted way. +Some parts of the system state are exposed to clients in a tamperproof way via certification: the system can reveal a _partial state tree_ which includes just the data of interest, together with a signature on the root hash of the state tree. This means that a client can be sure that the response is correct, even if the client happens to be communicating with a malicious server, or has received the certificate via some other untrusted way. To validate a value using a certificate, the client conceptually @@ -1563,7 +1564,7 @@ All state trees include the time at path `/time` (see <>). Clie === Lookup -Given a (verified) tree, the client can fetch the value at a given path, which is a sequence of labels (blobs). In this document, we write paths suggestively with slashes as separators; the actual encoding is not actually using slashes as delimitors. +Given a (verified) tree, the client can fetch the value at a given path, which is a sequence of labels (blobs). In this document, we write paths suggestively with slashes as separators; the actual encoding is not actually using slashes as delimiters. The following algorithm looks up a `path` in a certificate, and returns either @@ -1656,7 +1657,7 @@ The values in the <> are encoded to blobs as follows: === Example -Consider the following tree-shaped data (all single charachter strings denote labels, all other denote values) +Consider the following tree-shaped data (all single character strings denote labels, all other denote values) .... ─┬╴ "a" ─┬─ "x" ─╴"hello" │ └╴ "y" ─╴"world" @@ -1743,7 +1744,7 @@ We use a concatenation operator `·` with various types: to extend sets and maps The shape of values is described using a hand-wavy type system. We use `Foo = Nat` to define type aliases; now `Foo` can be used instead of `Nat`. Often, the right-hand side is a more complex type here, e.g. a record, or multiple possible types separated by a vertical bar (`|`). Partial maps are written as `Key ↦ Value` and the function type as `Argument -> Result`. -NOTE: All values are immutable! State change is specified by describing the new state, not by changing existing state. +NOTE: All values are immutable! State change is specified by describing the new state, not by changing the existing state. Record fields are accessed using dot-notation (e.g. `S.request_id > 0`). To create a new record from an existing record `R` with some fields changed, the syntax `R where field = new_value` is used. This syntax can also be used to create new records with some deeply nested field changed: `R where some_map[key].field = new_value`. @@ -1755,7 +1756,7 @@ For example, the condition `S.messages = Older_messages · M · Younger_messages In this specification, we describe the Internet Computer as a state machine. In particular, there is a single piece of data that describes the complete state of the system, called `S`. -Of course, this is a huge simplification: The real Internet Computer is distributed and has a multi-component architecture, and the state is spread over many different components, some physically separated. But this simplification allows us to have a concise description of the system, and to easily make global decisions (such as, “is there any pending message”), without having to specify the bookkeeping that allows such global decision. +Of course, this is a huge simplification: The real Internet Computer is distributed and has a multi-component architecture, and the state is spread over many different components, some physically separated. But this simplification allows us to have a concise description of the system, and to easily make global decisions (such as, “is there any pending message”), without having to specify the bookkeeping that allows such global decisions. ==== Identifiers @@ -1994,7 +1995,7 @@ hash_of_map: Request -> Request .... For the signatures in a `Request`, we assume that the following function implements signature verification as described in <>. -This function picks the corresponding signature scheme according to the on the DER-encoded metadata in the public key. +This function picks the corresponding signature scheme according to the DER-encoded metadata in the public key. .... PublicKey = Blob Signature = Blob @@ -2133,7 +2134,7 @@ delegation_targets(DS) ==== Effective canister ids -A `Request` has an effective canister id accoridng to the rules in <<#http-effective-canister-id>>: +A `Request` has an effective canister id according to the rules in <<#http-effective-canister-id>>: .... is_effective_canister_id(CanisterUpdateCall {canister_id = ic_principal, arg = candid({canister_id = p, …}), …}, p) @@ -2141,16 +2142,6 @@ is_effective_canister_id(CanisterUpdateCall {canister_id = ic_principal, method is_effective_canister_id(CanisterUpdateCall {canister_id = p, …}, p), if p ≠ ic_principal .... -=== State transitions - -Based on this abstract notion of the state, we can describe the behavior of the system. There are three classes of behaviors: - - * Asynchronous API requests that are submitted via `/api/v2/…/call`. These transitions describes checks that the request must pass to be considered received. - * Spontaneous transitions that model the internal behavior of the system, by describing conditions on the state that allow the transition to happen, and the state after. - * Responses to synchronous requests (i.e. `/api/v2/…/query`). By definition, these do _not_ change the state of the system, and merely describe the response based on the read request and the current system state. - -The state transitions are not complete with regard to error handling. For example, the behavior of sending a request to a non-existent canister is not specified here. For now, we trust implementors to make sensible decisions there. - ==== API Request submission After a node accepts a request via `/api/v2/canister//call`, it gets added to the system in the `Received` state. @@ -2160,7 +2151,7 @@ Due to this check, the envelope is discarded after this point. Requests that have expired are dropped here. -Ingress message inspection is applied, and message that are not accepted by the canister are dropped. +Ingress message inspection is applied, and messages that are not accepted by the canister are dropped. Submitted request:: `E : Envelope` Conditions:: @@ -2241,7 +2232,7 @@ S with A call to a canister which is stopping, stopped, or frozen is automatically rejected. -The (unspecified) function `freezing_limit(S, cid)` determines the freezing threshold in cycles of the canister with id `cid`, given it’s currrent memory footprint, storage cost, memory and compute allocation, and current `freezing_threshold` setting. +The (unspecified) function `freezing_limit(S, cid)` determines the freezing threshold in cycles of the canister with id `cid`, given its current memory footprint, storage cost, memory and compute allocation, and current `freezing_threshold` setting. Conditions:: .... @@ -2282,7 +2273,9 @@ Conditions:: balances[CM.callee] ≥ freezing_limit(S, CM.callee) + MAX_CYCLES_PER_MESSAGE Ctxt_id ∉ dom S.call_contexts .... ++ State after:: ++ .... S with messages = @@ -2420,7 +2413,7 @@ If message execution < If message execution <>, the state is updated and possible outbound calls and responses are enqueued. -Note that returning does _not_ imply that the call associated with this message now _succeeds_ in the sense defined in <>; that would requre a (unique) call to `ic0.reply`. +Note that returning does _not_ imply that the call associated with this message now _succeeds_ in the sense defined in <>; that would require a (unique) call to `ic0.reply`. Note also that the state changes are persisted even when the system is set to synthesize a <> reject immediately afterward (which happens when this returns without calling `ic0.reply` or `ic0.reject`, the corresponding call has not been responded to and there are no outstanding callbacks, see <>). The function `as_update` turns a query function into an update function, this is merely a notational trick to simplify the rule @@ -2757,7 +2750,7 @@ S with S.status[A.canister_id] = Stopping [(M.origin, M.transferred_cycles)] .... -The next two transition record any additional 'stop_canister' requests that arrive at a stopping (or stopped) canister in its status. +The next two transitions record any additional 'stop_canister' requests that arrive at a stopping (or stopped) canister in its status. Conditions:: .... @@ -3106,7 +3099,7 @@ S with requests[M] = (deleted) .... -==== Canister out of cyles +==== Canister out of cycles Once a canister runs out of cycles, its code is uninstalled (cf. <>) and the allocations are set to zero (NB: allocations are currently not modeled in the formal model): @@ -3610,7 +3603,7 @@ copy_from_canister(src : i32, size : i32) blob = Upon _instantiation_ of the WebAssembly module, we can provide the following functions as imports. -The pseudo-code below does _not_ explicitly enforce the restrictions of which imports are available in which contexts; for that the table in <> is authorative, and is assumed to be part of the implementation. +The pseudo-code below does _not_ explicitly enforce the restrictions of which imports are available in which contexts; for that the table in <> is authoritative, and is assumed to be part of the implementation. .... ic0.msg_arg_data_size() : i32 = From d64f93bf2b45d1a91c1a247f6e7a3ea5fe733991 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bj=C3=B6rn=20Tackmann?= <54846571+Dfinity-Bjoern@users.noreply.github.com> Date: Thu, 30 Sep 2021 14:16:44 +0200 Subject: [PATCH 002/102] Release 0.18.2 (#372) * Add heartbeat to spec * Terminology changes in release 0.18.2 * Add support for 64-bit stable memory --- spec/changelog.adoc | 7 + spec/index.adoc | 491 +++++++++++++++++++++++++++++--------------- 2 files changed, 329 insertions(+), 169 deletions(-) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index a41183e8f..013012b81 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,13 @@ [#changelog] == Changelog +[#0_18_2] +=== 0.18.2 (2021-09-29) + +* Spec: Canister heartbeat +* Spec: Terminology changes +* Spec: Support for 64-bit stable memory + [#0_18_1] === 0.18.1 (2021-08-04) diff --git a/spec/index.adoc b/spec/index.adoc index af923d026..427f1bc88 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,39 +1,39 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.1 +0.18.2 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ == Introduction -Welcome to the Internet Computer! We speak of “the” Internet Computer, because although under the hood a large number of physical computers are working together in non-trivial ways, in the end we have the appearance of a single, shared, secure and world-wide accessible computer. Developers who want to build applications that run on the Internet Computer and end-users who want to use those applications need to know very little, if anything, about the underlying complexity of how the Internet Computer works. However, knowing some details about the interfaces that the Internet Computer exposes can allow interested developers and architects to take fuller advantages of the unique features that the Internet Computer provides. +Welcome to _the Internet Computer_! We speak of “the” Internet Computer, because although under the hood a large number of physical computers are working together in a blockchain protocol, in the end we have the appearance of a single, shared, secure and world-wide accessible computer. Developers who want to build decentralized applications (or _dapps_ for short) that run on the Internet Computer blockchain and end-users who want to use those dapps need to know very little, if anything, about the underlying protocol. However, knowing some details about the interfaces that the Internet Computer exposes can allow interested developers and architects to take fuller advantages of the unique features that the Internet Computer provides. === Target audience -This document describes this _external_ view of the Internet Computer, i.e. the low-level interfaces it provides to application developers and users, and what will happen when they use these interfaces. +This document describes this _external_ view of the Internet Computer, i.e. the low-level interfaces it provides to dapp developers and users, and what will happen when they use these interfaces. -NOTE: While this document describes the external interface and behavior of the Internet Computer, it is not end-user or end-developer documentation. Most developers will interact with the Internet Computer through additional tooling like the SDK, Canister Development Kits and Motoko. Please see https://sdk.dfinity.org/ for suitable documentation. +NOTE: While this document describes the external interface and behavior of the Internet Computer, it is not intended as end-user or end-developer documentation. Most developers will interact with the Internet Computer through additional tooling like the SDK, Canister Development Kits and Motoko. Please see https://sdk.dfinity.org/ for suitable documentation. The target audience of this document are * those who use these low-level interfaces (e.g. implement agents, canister developments kits, emulators, other tooling). * those who implement these low-level interfaces (e.g. developers of the Internet Computer implementation) -* those who want to understand the intricacies of the Internet Computer’s behaviour in great detail (e.g. to do a security analysis) +* those who want to understand the intricacies of the Internet Computer’s behavior in great detail (e.g. to do a security analysis) WARNING: This document is a rigorous, technically dense reference. It is not an introduction to the Internet Computer, and as such most useful to those who understand the high-level concepts. Please see more high-level documentation first. === Scope of this document -If you think of the Internet Computer as a distributed execution engine that _provides_ a WebAssembly-based service hosting service, then this document describes exclusively the service hosting aspect of it. To the extent possible, this document will _not_ talk about blockchains, consensus protocols, nodes, subnets, orthogonal persistence or governance. +If you think of the Internet Computer as a distributed engine that executes WebAssembly-based dapps, then this document describes exclusively the aspect of executing those dapps. To the extent possible, this document will _not_ talk about consensus protocols, nodes, subnets, orthogonal persistence or governance. This document tries to be implementation agnostic: It would apply just as well to a (hypothetical) compatible reimplementation of the Internet Computer. This implies that this document does not cover interfaces towards those running the Internet Computer (e.g. data center operators, protocol developers, governance users), as topics like node update, monitoring, logging are inherently tied to the actual _implementation_ and its architecture. === Overview of the Internet Computer -If you want to use the Internet Computer as an application developer, you first create a _canister module_ that contains the WebAssembly code and configuration for your application, and deploy it using the <>. You can create canisters using the Motoko language and the SDK, which is more convenient. If you want to use your own tooling, however, then this document describes <> and how the <>. +Dapps on the Internet Computer, or _IC_ for short, are implemented as _canister smart contracts_, or _canisters_ for short. If you want to build on the Internet Computer as a dapp developer, you first create a _canister module_ that contains the WebAssembly code and configuration for your dapp, and deploy it using the <>. You can create canister modules using the Motoko language and the SDK, which is more convenient. If you want to use your own tooling, however, then this document describes <> and how the <>. -Once your application is running on the Internet Computer, it is a _canister_, and users can interact with it. They can use the <> to interact with the canister according to the <>. +Once your dapp is running on the Internet Computer, it is a canister smart contract, and users can interact with it. They can use the <> to interact with the canister according to the <>. The user can also use the HTTPS interface to issue read-only queries, which are faster, but cannot change the state of a canister. @@ -63,19 +63,20 @@ Sections “<>” and “<>” describe these interf To get some consistency in this document, we try to use the following terms with precision: -We avoid the term “client”, as it could be the client of the Internet Computer or the client inside the distributed network that makes up the Internet Computer. Instead, we use the term _user_ to denote the external entity interacting with the internet computer, even if in most cases it will be some code (sometimes called “agent”) acting on behalf of a (human) user. +We avoid the term “client”, as it could be the client of the Internet Computer or the client inside the distributed network that makes up the Internet Computer. Instead, we use the term _user_ to denote the external entity interacting with the Internet Computer, even if in most cases it will be some code (sometimes called “agent”) acting on behalf of a (human) user. The public entry points of canisters are called _methods_. Methods can be declared to be either _update methods_ (state mutation is preserved) or _query methods_ (state mutation is discarded, no further calls can be made). Methods can be _called_, from _caller_ to _callee_, and will eventually incur a _response_ which is either a _reply_ or a _reject_. A method may have _parameters_, which are provided with concrete _arguments_ in a method call. -Inter-canister calls do not distinguish between update and query methods; inter-canister calls can preserve state mutation and are therefore akin to update method calls. Note that calls from a canister to itself also count as "inter-canister". External calls can be update calls, which can call both kinds of methods, and query calls, which can _only_ call query methods. +External calls can be update calls, which can call both kinds of methods, and query calls, which can _only_ call query methods. +Inter-canister calls can also call both kinds of methods. Note that calls from a canister to itself also count as "inter-canister". Internally, a call or a response is transmitted as a _message_ from a _sender_ to a _receiver_. Messages do not have a response. [[define-wasm-fn]]WebAssembly _functions_ are exported by the WebAssembly module or provided by the System API. These are _invoked_ and can either _trap_ or _return_, possibly with a return value. Functions, too, have parameters and take arguments. -External _users_ interact with the system by issuing _requests_ on the HTTPS interface. Requests have responses which can either be replies or rejects. Some requests cause internal messages to be created. +External _users_ interact with the Internet Computer by issuing _requests_ on the HTTPS interface. Requests have responses which can either be replies or rejects. Some requests cause internal messages to be created. Canisters and users are identified by a _principal_, sometimes also called an _id_. @@ -85,7 +86,7 @@ Before going into the details of the four public interfaces described in this do === Unspecified constants and limits -This specification may refer to certain constants and limits without specifying their concrete value (yet), i.e. they are implementation defined. Many are resource limits which are relevant only to specify the error-handling behaviour of the system (which, as mentioned above, is also not yet precisely described in this document). This list is not complete. +This specification may refer to certain constants and limits without specifying their concrete value (yet), i.e. they are implementation defined. Many are resource limits which are relevant only to specify the error-handling behavior of the IC (which, as mentioned above, is also not yet precisely described in this document). This list is not complete. * `MAX_CYCLES_PER_MESSAGE` + @@ -93,16 +94,16 @@ Amount of cycles that a canister has to have before a message is attempted to be * `MAX_CYCLES_PER_RESPONSE` + -Amount of cycles that the system sets aside when a canister performs a call. This is used to pay for processing the response message, and unused cycles after the execution of the response are refunded. See <>. +Amount of cycles that the IC sets aside when a canister performs a call. This is used to pay for processing the response message, and unused cycles after the execution of the response are refunded. See <>. * `MAX_CANISTER_BALANCE` + -Maximum canister cycle balance. Any excess is discarded. Less than 2^64^. +Maximum canister cycle balance. Any excess is discarded. Less than 2^128^. [#principal] === Principals -Principals are generic identifiers for canisters, users and possibly other concepts in the future. As far as most uses of the system are concerned they are _opaque_ binary blobs with a length between 0 and 29 bytes, and there is intentionally no mechanism to tell canister id and users ids apart. +Principals are generic identifiers for canisters, users and possibly other concepts in the future. As far as most uses of the IC are concerned they are _opaque_ binary blobs with a length between 0 and 29 bytes, and there is intentionally no mechanism to tell canister ids and user ids apart. There is, however, some structure to them to encode specific authentication and authorization behavior. @@ -115,7 +116,7 @@ There are several classes of ids: 1. _Opaque ids_. + -These are always generated by the system and have no structure of interest outside the system. +These are always generated by the IC and have no structure of interest outside of it. + NOTE: Typically, these end with the byte `0x01`, but users of the IC should not need to care about that. @@ -138,7 +139,7 @@ NOTE: Derived IDs are currently not explicitly used in this document, but they m + This has the form `0x04`, and is used for the anonymous caller. It can be used in call and query requests without a signature. -When the system creates a _fresh_ id, it never creates a self-authenticating id, an anonymous id or an id derived from what could be a canister or user. +When the IC creates a _fresh_ id, it never creates a self-authenticating id, an anonymous id or an id derived from what could be a canister or user. [#textual-ids] @@ -182,7 +183,7 @@ function textual_decode() { [#canister-lifecycle] === Canister lifecycle -Services on the Internet Computer are called _canisters_. Conceptually, they consist of the following pieces of state: +Dapps on the Internet Computer are called _canisters_. Conceptually, they consist of the following pieces of state: * A canister id (a <>) * Their _controllers_ (a possibly empty list of <>) @@ -194,7 +195,7 @@ A canister can be _empty_ (e.g. directly after creation) or _non-empty_. A non-e * code, in the form of a canister module * state (memories, globals etc.) - * possibly further system-internal data (e.g. queues) + * possibly further data that is specific to the implementation of the IC (e.g. queues) Canisters are empty after creation and uninstallation, and become non-empty through <>. @@ -220,8 +221,8 @@ NOTE: Once the IC frees the resources of a canister, its id, _cycles_ balance, a The canister status can be used to control whether the canister is processing calls: * In status `running`, calls to the canister are processed as normal. -* In status `stopping`, calls to the canister are rejected by the system, but responses to the canister are processed as normal. -* In status `stopped`, calls to the canister are rejected by the system, and there are no outstanding responses. +* In status `stopping`, calls to the canister are rejected by the IC, but responses to the canister are processed as normal. +* In status `stopped`, calls to the canister are rejected by the IC, and there are no outstanding responses. In all cases, calls to the <> are processed, regardless of the state of the managed canister. @@ -255,9 +256,9 @@ Plain signatures are supported for the schemes - See https://tools.ietf.org/html/rfc8410[RFC 8410] for DER encoding of Ed25519 public keys. - See https://tools.ietf.org/rfc/rfc5480[RFC 5480] for DER encoding of ECDSA public keys; the DER encoding must not specify a hash function. For curve `secp256k1`, the OID 1.3.132.0.10 is used. - The points must be specified in uncompressed form (i.e. `0x04` followed by the big-endian 32byte encodings of `x` and `y`). + The points must be specified in uncompressed form (i.e. `0x04` followed by the big-endian 32-byte encodings of `x` and `y`). -* The signatures are encoded as the concatenation of the 32 byte big endian encodings of the two values _r_ and _s_. +* The signatures are encoded as the concatenation of the 32-byte big endian encodings of the two values _r_ and _s_. [#webauthn] ==== Web Authentication @@ -341,7 +342,7 @@ where `s` is the SHA-256 hash of the `seed` used in the public key and `m` is th [#state-tree] == The system state tree -Parts of the system state are publicly exposed (e.g. via <> or <>) in a verified way (see <> for the machinery for certifying). This section describes the content of the system state abstractly. +Parts of the IC state are publicly exposed (e.g. via <> or <>) in a verified way (see <> for the machinery for certifying). This section describes the content of this system state abstractly. Conceptually, the system state is a tree with labeled children, and values in the leaves. Equivalently, the system state is a mapping from paths (sequences of labels) to values, where the domain is prefix-free. @@ -368,7 +369,7 @@ The public key of the subnet (a DER-encoded BLS key, see <>) [#state-tree-request-status] === Request status -For each asynchronous request known to the system, its status is in a subtree at `/request_status/`. Please see <> for more details on how asynchronous requests work. +For each asynchronous request known to the Internet Computer, its status is in a subtree at `/request_status/`. Please see <> for more details on how asynchronous requests work. * `/request_status//status` (text) + @@ -386,9 +387,9 @@ If the status is `rejected`, then this path contains the reject code (see </controllers` (blob): + -The current controllers of the canister. The value consists of a CBOR data item with major type 6 (“Semantic tag”) and tag value `55799` (see https://tools.ietf.org/html/rfc7049#section-2.4.5[Self-Describe CBOR]), followed by an array of principals in their binary form (CDDL `#6.55799([* bytes .size (0..29)])`) +The current controllers of the canister. The value consists of a CBOR data item with major type 6 (“Semantic tag”) and tag value `55799` (see https://tools.ietf.org/html/rfc7049#section-2.4.5[Self-Describe CBOR]), followed by an array of principals in their binary form (CDDL `#6.55799([* bytes .size (0..29)])`). [#http-interface] == HTTPS Interface -The concrete mechanism that users use to send requests to the Internet Computer is via an HTTPS API, which exposes three endpoints to handle interactions with the internet computer, plus one for diagnostics: +The concrete mechanism that users use to send requests to the Internet Computer is via an HTTPS API, which exposes three endpoints to handle interactions, plus one for diagnostics: * At `/api/v2/canister//call` the user can submit (asynchronous, state-changing) calls. -* At `/api/v2/canister//read_state` the user can read various system information. In particular, they can poll for the status of a call here. -* At `/api/v2/canister//query` the user can perform (synchronous, non-state-changing) query calls -* At `/api/v2/status` the user can get additional information about the network +* At `/api/v2/canister//read_state` the user can read various information about the state of the Internet Computer. In particular, they can poll for the status of a call here. +* At `/api/v2/canister//query` the user can perform (synchronous, non-state-changing) query calls. +* At `/api/v2/status` the user can retrieve status information about the Internet Computer. In these paths, the `` is the <> of the <>. @@ -432,19 +433,19 @@ NOTE: This document does not yet explain how to find the location and port of th === Overview of canister calling Users interact with the Internet Computer by calling canisters. -By the very nature of a distributed implementation, they cannot be acted upon immediately, but only with a delay. +By the very nature of a blockchain protocol, they cannot be acted upon immediately, but only with a delay. Moreover, the actual node that the user talks to may not be honest or, for other reasons, may fail to get the request on the way. This implies the following high-level workflow: -1. A user submits a call via the <>. No useful information is returned from the node (as such information cannot be trustworthy anyways). -2. For a certain amount of time, the system behaves as if it does not know about the call. -3. The system asks the targeted canister if it is willing to accept this message and be charged for the expense of processing it. This uses the <> API for normal calls. For calls to the management canister, the rules in <> apply. -4. At some point, the system may accept the call for processing and set its status to `received`. This indicates that the system as a whole has received the call and plans on processing it (although it may still not get processed if the system is under high load). Furthermore, the user should also be able to ask any endpoint about the status of the pending call. +1. A user submits a call via the <>. No useful information is returned in the immediate response (as such information cannot be trustworthy anyways). +2. For a certain amount of time, the IC behaves as if it does not know about the call. +3. The IC asks the targeted canister if it is willing to accept this message and be charged for the expense of processing it. This uses the <> API for normal calls. For calls to the management canister, the rules in <> apply. +4. At some point, the IC may accept the call for processing and set its status to `received`. This indicates that the IC as a whole has received the call and plans on processing it (although it may still not get processed if the IC is under high load). Furthermore, the user should also be able to ask any endpoint about the status of the pending call. 5. Once it is clear that the call will be acted upon (sufficient resources, call not yet expired), the status changes to `processing`. Now the user has the guarantee that the request will have an effect, e.g. it will reach the target canister. -6. Now the system is processing the call. For some calls this may be atomic, for others this involves multiple internal steps. +6. The IC is processing the call. For some calls this may be atomic, for others this involves multiple internal steps. 7. Eventually, a response will be produced, and can be retrieved for a certain amount of time. The response is either a `reply`, indicating success, or a `reject`, indicating some form of error. -8. In the case that the call has been retained for long enough, but the request has not expired yet, the system can forget the response data and only remember the call as `done`, to prevent a replay attack. -9. Once the expiry time is past, the system can prune the call and its response, and completely forget about it. +8. In the case that the call has been retained for long enough, but the request has not expired yet, the IC can forget the response data and only remember the call as `done`, to prevent a replay attack. +9. Once the expiry time is past, the IC can prune the call and its response, and completely forget about it. This yields the following interaction diagram: @@ -472,21 +473,21 @@ if "" as X then endif .... -State transitions may be instantaneous and not always externally visible. For example, the system may move from `received` via `processing` to `replied` in one go. Similarly, the system may not implement the `done` state at all, and keep calls in state `replied`/`rejected` until they are pruned. +State transitions may be instantaneous and not always externally visible. For example, the state of a request may move from `received` via `processing` to `replied` in one go. Similarly, the IC may not implement the `done` state at all, and keep calls in state `replied`/`rejected` until they are pruned. -All gray states are _not_ explicitly represented in the system state, and are indistinguishable from “call does not exist”. +All gray states are _not_ explicitly represented in the state of the IC, and are indistinguishable from “call does not exist”. -The characteristic property of the `received` state is that the call has made it past the (potentially malicious) endpoint _into the system_. It is now pointless (but harmless) to submit the (identical) call again. Before reaching that state, submitting the identical call to further nodes might be a useful safeguard against a malicious or misbehaving node. +The characteristic property of the `received` state is that the call has made it past the (potentially malicious) endpoint _into the state of the IC_. It is now pointless (but harmless) to submit the (identical) call again. Before reaching that state, submitting the identical call to further nodes might be a useful safeguard against a malicious or misbehaving node. -The characteristic property of the `processing` state is _the initial effect of the call has or will happen_. This is best explained by an example: Consider a counter canister. It exports a method `inc` that increases the counter. Assume that the canister is bug free, and is not going to be forcibly removed. A user submits a call to call `inc`. If the user sees request status `processing`, the state change is guaranteed to happen, and the user can stop monitoring the status and does not have to retry submitting. +The characteristic property of the `processing` state is that _the initial effect of the call has happened or will happen_. This is best explained by an example: Consider a counter canister. It exports a method `inc` that increases the counter. Assume that the canister is bug free, and is not going to be forcibly removed. A user submits a call to call `inc`. If the user sees request status `processing`, the state change is guaranteed to happen. The user can stop monitoring the status and does not have to retry submitting. -A call may be rejected by the system or the canister. In either case, there is no guarantee about how much processing of the call has happened. +A call may be rejected by the IC or the canister. In either case, there is no guarantee about how much processing of the call has happened. To avoid replay attacks, the transition from `done` or `received` to `pruned` must happen no earlier than the call’s `ingress_expiry` field. -Calls must stay in `replied` or `rejected` long enough for polling clients to catch the response. +Calls must stay in `replied` or `rejected` long enough for polling users to catch the response. -When asking the system about the state or call of a request, the user uses the request id (see <>) to read the request status (see <>) from the state tree (see <>). +When asking the IC about the state or call of a request, the user uses the request id (see <>) to read the request status (see <>) from the state tree (see <>). [#http-call] === Request: Call @@ -503,7 +504,7 @@ The HTTP response to this request has an empty body and HTTP status 202, or a HT This request type can _also_ be used to call a query method. A user may choose to go this way, instead of via the faster and cheaper <> below, if they want to get a _certified_ response. -NOTE: The system functionality exposed via the <> can be used this way. +NOTE: The functionality exposed via the <> can be used this way. [#http-read-state] @@ -527,7 +528,7 @@ All requested paths must have one of the following paths as prefix: * `/subnet`. Can be requested by anyone. * `/request_status/`. Can only be read if `/request_id/` is not present in the state tree, or if this `read_state` request was signed by same sender as the original request referenced by ``, and the effective canister id of the original request matches the `` (see <>) in this HTTP request’s path. * `/canisters//module_hash`. Can be requested by anyone, if `` matches ``. - * `/canisters//controllers`. Can be requested by anyone, if `` matches ``. The order is system-defined. + * `/canisters//controllers`. Can be requested by anyone, if `` matches ``. The order may vary depending on the implementation. Note that the paths `/canisters//certified_data` are not accessible with this method; these paths are only exposed to the canister themselves via the System API (see <>). @@ -572,16 +573,16 @@ The `` in the URL paths of requests is the _effective_ de + [NOTE] ==== -Production instances do not support `provisional_create_canister_with_cycles` anyways. This means that using an effective canister id that could be an existing canister would lead to the request being routed by the edge component to a node the corresponding subnet and produce an error response there, while using an effective canister id that cannot be an existing canister id would cause an error response from the edge component -- either is fine. +The Internet Computer blockchain mainnet does not support `provisional_create_canister_with_cycles`. This means that using an effective canister id that could be an existing canister would lead to the request being routed to a node on the corresponding subnet and produce an error response there, while using an effective canister id that cannot be an existing canister id would cause an error response from the node receiving the request -- either is fine. -In non-production, multi-subnet instances (e.g. testnets), users with privileged access to `provisional_create_canister_with_cycles` (or possibly similar, internal methods) and knowledge of the mapping from canisters to subnets can use their choice of the effective canister id in the URL to steer the request to a specific subnet. +In multi-subnet development instances of the Internet Computer Protocol (e.g. testnets), users with privileged access to `provisional_create_canister_with_cycles` (or possibly similar, internal methods) and knowledge of the mapping from canisters to subnets can use their choice of the effective canister id in the URL to steer the request to a specific subnet. -In local, single-subnet instances, the effective canister id is ignored by the single instance, and thus `aaaaa-aa` can be used. +In a local canister execution environment, the effective canister id is ignored, and thus `aaaaa-aa` can be used. ==== * Else, the effective canister id must be the `canister_id` in the request. -NOTE: The expectation is that user-side agent code shields users and developers from this concept, in analogy to how the system interface shields canister developers from worrying about routing. +NOTE: The expectation is that user-side agent code shields users and developers from this concept, in analogy to how the System API interface shields canister developers from worrying about routing. [#authentication] @@ -590,13 +591,13 @@ NOTE: The expectation is that user-side agent code shields users and developers All requests coming in via the HTTPS interface need to be either _anonymous_ or _authenticated_ using a cryptographic signature. To that end, the following fields are present in the `content` map in all cases: * `nonce` (`blob`, optional): Arbitrary user-provided data, typically randomly generated. This can be used to create distinct requests with otherwise identical fields. -* `ingress_expiry` (`nat`, required): An upper limit on the validity of the request, expressed in nanoseconds since 1970-01-01 (like <>). This avoids replay attacks: The system will not accept requests, or transition requests from status `received` to status `processing`, if their expiry date is in the past. The system may refuse to accept requests with an ingress expiry date too far in the future. This applies to synchronous and asynchronous requests alike (and could have been called `signature_expiry`). +* `ingress_expiry` (`nat`, required): An upper limit on the validity of the request, expressed in nanoseconds since 1970-01-01 (like <>). This avoids replay attacks: The IC will not accept requests, or transition requests from status `received` to status `processing`, if their expiry date is in the past. The IC may refuse to accept requests with an ingress expiry date too far in the future. This applies to synchronous and asynchronous requests alike (and could have been called `request_expiry`). * `sender` (`Principal`, required): The user who issued the request. The envelope, i.e. the overall request, has the following keys: * `content` (`record`): the actual request content -* `sender_pubkey` (`blob`, optional): Public key used to authenticate this request. Since a user may in the future have more than one key, this field tells the system which key is used. +* `sender_pubkey` (`blob`, optional): Public key used to authenticate this request. Since a user may in the future have more than one key, this field tells the IC which key is used. * `sender_delegation` (`array` of maps, optional): a chain of delegations, starting with the one signed by `sender_pubkey` and ending with the one delegating to the key relating to `sender_sig`. * `sender_sig` (`blob`, optional): Signature to authenticate this request. @@ -689,11 +690,11 @@ hash_of_map({ request_type: "call", canister_id: 0x00000000000004D2, method_name [#reject-codes] === Reject codes -An API request or inter-canister call that is pending in the system will eventually result in either a _reply_ (indicating success, and carrying data) or a _reject_ (indicating an error of some sorts). A reject contains a _rejection code_ that classifies the error and a hopefully helpful _reject message_ string. +An API request or inter-canister call that is pending in the IC will eventually result in either a _reply_ (indicating success, and carrying data) or a _reject_ (indicating an error of some sorts). A reject contains a _rejection code_ that classifies the error and a hopefully helpful _reject message_ string. Rejection codes are member of the following enumeration: -* `SYS_FATAL` (1): Fatal system error, retry unlikely to be useful. +* `SYS_FATAL` (1): Fatal system error, retry unlikely to be useful. * `SYS_TRANSIENT` (2): Transient system error, retry might be possible. * `DESTINATION_INVALID` (3): Invalid destination (e.g. canister/account does not exist) * `CANISTER_REJECT` (4): Explicit reject by the canister. @@ -703,7 +704,7 @@ The symbolic names of this enumeration are used throughout this specification, b The error message is guaranteed to be a string, i.e. not arbitrary binary data. -When canisters explicitly reject a message (see <>), they can specify the reject message, but _not_ the reject code; it is always `CANISTER_REJECT`. In this sense, the reject code is trustworthy: If the system responds with a `SYS_FATAL` reject, then it really was the system issuing this reject. +When canisters explicitly reject a message (see <>), they can specify the reject message, but _not_ the reject code; it is always `CANISTER_REJECT`. In this sense, the reject code is trustworthy: If the IC responds with a `SYS_FATAL` reject, then it really was the IC issuing this reject. [#api-status] === Status endpoint @@ -713,13 +714,13 @@ Additionally, the Internet Computer provides an API endpoint to obtain various s /api/v2/status .... -For this endpoint, the user performs a GET request, and receives a CBOR value with the following fields. The Internet Computer may include additional implementation-specific fields. +For this endpoint, the user performs a GET request, and receives a CBOR value with the following fields. The IC may include additional implementation-specific fields. * `ic_api_version` (string, mandatory): Identifies the interface version supported, i.e. the version of the present document that the internet computer aims to support, e.g. `0.8.1`. The implementation may also return `unversioned` to indicate that it does _not_ comply to a particular version, e.g. in between releases. -* `impl_source` (string, optional): Identifies the implementation of the Internet Computer, by convention with the canonical location of the source code (e.g. `https://github.com/dfinity/dfinity`). -* `impl_version` (string, optional): If the user is talking to a released version of an Internet Computer implementation, this is the version number. For non-released versions, output of `git describe` like `0.1.13-13-g2414721` would also be very suitable. -* `impl_revision` (string, optional): The precise git revision of the Internet Computer implementation -* `root_key` (blob, only in development instances): The public key (a DER-encoded BLS key) of the root key of this Internet Computer instance. This _must_ be present in short-lived development instances, to allow the client to fetch the public key. In production environments, clients must have an independent trustworthy source for this data, and must not be tempted to fetch it from this insecure location. +* `impl_source` (string, optional): Identifies the implementation of the Internet Computer Protocol, by convention with the canonical location of the source code (e.g. `https://github.com/dfinity/ic`). +* `impl_version` (string, optional): If the user is talking to a released version of an Internet Computer Protocol implementation, this is the version number. For non-released versions, output of `git describe` like `0.1.13-13-g2414721` would also be very suitable. +* `impl_revision` (string, optional): The precise git revision of the Internet Computer Protocol implementation +* `root_key` (blob, only in development instances): The public key (a DER-encoded BLS key) of the root key of this development instance of the Internet Computer Protocol. This _must_ be present in short-lived development instances, to allow the agent to fetch the public key. For the Internet Computer, agents must have an independent trustworthy source for this data, and must not be tempted to fetch it from this insecure location. See <> for details on the precise CBOR encoding of this object. @@ -779,9 +780,8 @@ include::{example}requests.cddl[] === Ordering guarantees -In order to allow for a distributed implementation of the Internet Computer, the order in which the various messages between canisters are delivered and executed is not fully specified. - -The guarantee we do give is that function calls between two canisters are executed in order, so that a canister that requires in-order execution need not wait for the response from an earlier message to a canister before sending a later message to that same canister. +The order in which the various messages between canisters are delivered and executed is not fully specified. +The guarantee provided by the IC is that function calls between two canisters are executed in order, so that a canister that requires in-order execution need not wait for the response from an earlier message to a canister before sending a later message to that same canister. More precisely: @@ -814,9 +814,9 @@ A canister module is simply a https://webassembly.github.io/spec/core/index.html [#system-api] == Canister interface (System API) -The System API is the interface between the running canister and the Internet Computer. It allows the WebAssembly module of a canister to expose functionality to the users (method entry points) and the system (e.g. initialization), and exposes system functionality to the canister (e.g. calling other canisters). Because WebAssembly is rather low-level, it also explains how to express higher level concepts (e.g. binary blobs). +The System API is the interface between the running canister and the Internet Computer. It allows the WebAssembly module of a canister to expose functionality to the users (method entry points) and the IC (e.g. initialization), and exposes functionality of the IC to the canister (e.g. calling other canisters). Because WebAssembly is rather low-level, it also explains how to express higher level concepts (e.g. binary blobs). -We want to leverage advanced WebAssembly features, such as WebAssembly host references. But as they are not yet supported by all tools involved, this section describes an initial System API that does not rely on host references. To emphasize that this is just a preliminary interface, we group the system methods under the module name `ic0`, planning to use `ic` for the real deal. +We want to leverage advanced WebAssembly features, such as WebAssembly host references. But as they are not yet supported by all tools involved, this section describes an initial System API that does not rely on host references. In section <>, we outline some of the proposed uses of WebAssembly host references. [#system-api-module] @@ -830,9 +830,11 @@ In order for a WebAssembly module to be usable as the code for the canister, it * It may have a `(start)` function. * If it exports a function called `canister_init`, the function must have type `+() -> ()+`. * If it exports a function called `canister_inspect_message`, the function must have type `+() -> ()+`. +* If it exports a function called `canister_heartbeat`, the function must have type `+() -> ()+`. * If it exports any functions called `canister_update ` or `canister_query ` for some `name`, the functions must have type `+() -> ()+`. * It may not export both `canister_update ` and `canister_query ` with the same `name`. -* The system may reject WebAssembly modules that declare more than 6000 functions or more than 200 globals. +* It may not export other methods the names of which start with the prefix `canister_` besides the methods allowed above. +* The IC may reject WebAssembly modules that declare more than 6000 functions or more than 200 globals. === Interpretation of numbers @@ -840,30 +842,31 @@ WebAssembly number types (`i32`, `i64`) do not indicate if the numbers are to be === Entry points -The canister provides entry points which are invoked by the system under various circumstances: +The canister provides entry points which are invoked by the IC under various circumstances: * The canister may export a function named `canister_init` and type `+() -> ()+`. * The canister may export a function named `canister_pre_upgrade` and type `+() -> ()+`. * The canister may export a function named `canister_post_upgrade` and type `+() -> ()+`. * The canister may export functions named `canister_inspect_message` with type `+() -> ()+`. +* The canister may export a function named `canister_heartbeat` with type `+() -> ()+`. * The canister may export functions named `canister_update ` and type `+() -> ()+`. * The canister may export functions named `canister_query ` and type `+() -> ()+`. * The canister table may contain functions of type `+(env : i32) -> ()+` which may be used as callbacks for inter-canister calls. -If the execution of any of these entry points traps for any reason, then all changes to the WebAssembly state, as well as the effect of any externally visible system call (like `ic0.msg_reply`, `ic0.msg_reject`, `ic0.call_perform`), are discarded. For upgrades, this transactional behaviour applies to the `canister_pre_upgrade`/`canister_post_upgrade` sequence as a whole. +If the execution of any of these entry points traps for any reason, then all changes to the WebAssembly state, as well as the effect of any externally visible system call (like `ic0.msg_reply`, `ic0.msg_reject`, `ic0.call_perform`), are discarded. For upgrades, this transactional behavior applies to the `canister_pre_upgrade`/`canister_post_upgrade` sequence as a whole. [#system-api-init] ==== Canister initialization -If `canister_init` is present, then this is the first exported WebAssembly function invoked by the system. The argument that was passed along with the canister initialization call (see <>) is available to the canister via `ic0.msg_arg_data_size/copy`. +If `canister_init` is present, then this is the first exported WebAssembly function invoked by the IC. The argument that was passed along with the canister initialization call (see <>) is available to the canister via `ic0.msg_arg_data_size/copy`. -The system assumes the canister to be fully instantiated if the `canister_init` method entry point returns. If the `canister_init` method entry point traps, then canister installation has failed, and the canister is reverted to its previous state (i.e. empty with `install`, or whatever it was for a `reinstall`). +The IC assumes the canister to be fully instantiated if the `canister_init` method entry point returns. If the `canister_init` method entry point traps, then canister installation has failed, and the canister is reverted to its previous state (i.e. empty with `install`, or whatever it was for a `reinstall`). [#system-api-upgrades] ==== Canister upgrades -When a canister is upgraded to a new WebAssembly module, the system: +When a canister is upgraded to a new WebAssembly module, the IC: 1. Invokes `canister_pre_upgrade` (if present) on the old instance, to give the canister a chance to clean up (e.g. move data to <>). 2. Instantiates the new module, including the execution of `(start)`, with a fresh WebAssembly state. @@ -882,10 +885,18 @@ To define a public method of name `name`, a WebAssembly module exports a functio NOTE: The space in `canister_update ` resp. `canister_query ` is intentional. There is exactly one space between `canister_update/canister_query` and the ``. -The argument of the call (e.g. the content of the `arg` field in the <>) is copied into the canister on demand using the System functions shown below. +The argument of the call (e.g. the content of the `arg` field in the <>) is copied into the canister on demand using the System API functions shown below. Eventually, a method will want to send a response, using `ic0.reply` or `ic0.reject` +==== Heartbeat + +For periodic or time-based execution, the WebAssembly module can export a function with name `canister_heartbeat`. If present, the IC will invoke this function at regular intervals. The exact interval is implementation-defined. For each heartbeat invocation, the IC guarantees that the time, as returned by <>, is monotonically increasing. + +`canister_heartbeat` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. + +NOTE: While an implementation will likely try to keep the interval between `canister_heartbeat` invocations to within a few seconds, this is not formally part of this specification. + ==== Callbacks Callbacks are addressed by their table index (as a proxy for a Wasm `funcref`). @@ -896,7 +907,7 @@ In the reply callback of a <>, the a [#system-api-imports] === Overview of imports -The following sections describe various system imports, which we summarize here. +The following sections describe various System API functions, also referred to as system calls, which we summarize here. .... ic0.msg_arg_data_size : () -> i32; // I U Q Ry F @@ -924,7 +935,7 @@ ic0.msg_method_name_size : () -> i32 // F ic0.msg_method_name_copy : (dst : i32, offset : i32, size : i32) -> (); // F ic0.accept_message : () -> (); // F -ic0.call_new : // U Ry Rt +ic0.call_new : // U Ry Rt H ( callee_src : i32, callee_size : i32, name_src : i32, @@ -934,17 +945,21 @@ ic0.call_new : // U reject_fun : i32, reject_env : i32 ) -> (); -ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt -ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt -ic0.call_cycles_add : ( amount : i64 ) -> (); // U Ry Rt -ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt +ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt H +ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt H +ic0.call_cycles_add : ( amount : i64 ) -> (); // U Ry Rt H +ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt H ic0.stable_size : () -> (page_count : i32); // * ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32); // * ic0.stable_write : (offset : i32, src : i32, size : i32) -> (); // * ic0.stable_read : (dst : i32, offset : i32, size : i32) -> (); // * +ic0.stable64_size : () -> (page_count : i64); // * +ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64); // * +ic0.stable64_write : (offset : i64, src : i64, size : i64) -> (); // * +ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> (); // * -ic0.certified_data_set : (src: i32, size: i32) -> () // I G U Ry Rt +ic0.certified_data_set : (src: i32, size: i32) -> () // I G U Ry Rt H ic0.data_certificate_present : () -> i32 // * ic0.data_certificate_size : () -> i32 // * ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> () // * @@ -966,9 +981,10 @@ The comment after each function lists from where these functions may be invoked: * `C`: from a cleanup callback * `s`: the `(start)` module initialization function * `F`: from `canister_inspect_message` +* `H`: from `canister_heartbeat` * `*` = `I G U Q Ry Rt C F` (NB: Not `(start)`) -If the canister invokes a system import from somewhere else, it will trap. +If the canister invokes a system call from somewhere else, it will trap. === Blob-typed arguments and results @@ -1112,7 +1128,7 @@ When handling an update call (or a callback), a canister can do further calls to Begins assembling a call to the canister specified by `callee_src/_size` at method `name_src/_size`. -The system records two mandatory callback functions, represented by a table entry index `*_fun` and some additional value `*_env`. When the response comes back, the table is read at the corresponding index, expected to be a function of type `+(env : i32) -> ()+`, and passed the corresponding `*_env` value. +The IC records two mandatory callback functions, represented by a table entry index `*_fun` and some additional value `*_env`. When the response comes back, the table is read at the corresponding index, expected to be a function of type `+(env : i32) -> ()+`, and passed the corresponding `*_env` value. The reply callback is executed upon successful completion of the method call, which can query the reply using `ic0.msg_arg_data_*`. @@ -1152,11 +1168,11 @@ This may be called multiple times between `ic0.call_new` and `ic0.call_perform`. * `+ic0.call_perform : () -> ( err_code : i32 )+` + -This call concludes assembling the call. It queues the call message to the given destination, but does not actually act on it until the current WebAssembly function returns without trapping. +This concludes assembling the call. It queues the call message to the given destination, but does not actually act on it until the current WebAssembly function returns without trapping. + -If the system returns `0` as the `err_code`, the system was able to enqueue the call. In this case, the call will either be delivered, returned because the destination canister does not exist or returned because of an out of cycles condition. This also means that exactly one of the reply or reject callbacks will be executed. +If the function returns `0` as the `err_code`, the IC was able to enqueue the call. In this case, the call will either be delivered, returned because the destination canister does not exist or returned because of an out of cycles condition. This also means that exactly one of the reply or reject callbacks will be executed. + -If the system returns a non-zero value, the call cannot (and will not be) performed. This can happen due to a lack of system internal resources, but also if it would reduce the current cycle balance to a level below where the canister would be frozen. +If the function returns a non-zero value, the call cannot (and will not be) performed. This can happen due to a lack of resources within the IC, but also if it would reduce the current cycle balance to a level below where the canister would be frozen. + After `ic0.call_perform` and before the next call to `ic0.call_new`, all other `ic0.call_*` function calls trap. @@ -1165,11 +1181,11 @@ After `ic0.call_perform` and before the next call to `ic0.call_new`, all other ` Each canister maintains a balance of _cycles_, the utility token used to pay for platform usage. -NOTE: This specification currently does not go into details about which actions cost how many cycles and/or when. In general, you must assume that the canister’s cycle balance can change arbitrarily between method executions, and during each system function API call, unless explicitly mentioned otherwise. +NOTE: This specification currently does not go into details about which actions cost how many cycles and/or when. In general, you must assume that the canister’s cycle balance can change arbitrarily between method executions, and during each System API function call, unless explicitly mentioned otherwise. * `ic0.canister_cycle_balance : () -> i64` + -indicates the current cycle balance of the canister. It is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add`. After execution of the message, the system may add unused cycles from the reserve back to the balance. +indicates the current cycle balance of the canister. It is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add`. After execution of the message, the IC may add unused cycles from the reserve back to the balance. + NOTE: Should a future version of this specification allow `MAX_CANISTER_BALANCE` >= 2^64^, then this call will report 2^64^-1 if the balance exceeds 2^64^. A new, wider System API will then be provided for canisters that need to deal precisely with large canister balances. @@ -1194,7 +1210,7 @@ It can be called multiple times, each time possibly adding more cycles to the ba The return value indicates how many cycles were actually moved. -This does not trap. +This system call is deprecated and does not trap. [TIP] ===== @@ -1223,34 +1239,74 @@ The stable memory is initially empty. * `ic0.stable_size : () -> (page_count : i32)` + -returns the current size of the stable memory in WebAssembly pages. (One WebAssembly page is 64Ki bytes.) +returns the current size of the stable memory in WebAssembly pages. (One WebAssembly page is 64KiB) ++ +This system call traps if the size of the stable memory exceeds 2^32 bytes. * `ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32)` + tries to grow the memory by `new_pages` many pages containing zeroes. + -If successful, returns the _previous_ size of the memory (in pages). Otherwise, returns `-1`. +This system call traps if the _previous_ size of the memory exceeds 2^32 bytes. ++ +If the _new_ size of the memory exceeds 2^32 bytes or growing is unsuccessful, then it returns `-1`. ++ +Otherwise, it grows the memory and returns the _previous_ size of the memory in pages. * `ic0.stable_write : (offset : i32, src : i32, size : i32) -> ()` + copies the data referred to by `src`/`size` out of the canister and replaces the corresponding segment starting at `offset` in the stable memory. + -This system call traps if `src+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. +This system call traps if the size of the stable memory exceeds 2^32 bytes. ++ +It also traps if `src+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. * `ic0.stable_read : (dst : i32, offset : i32, size : i32) -> ()` + copies the data referred to by `offset`/`size` out of the stable memory and replaces the corresponding bytes starting at `dest` in the canister memory. + +This system call traps if the size of the stable memory exceeds 2^32 bytes. ++ +It also traps if `dst+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory + +* `ic0.stable64_size : () -> (page_count : i64)` ++ +returns the current size of the stable memory in WebAssembly pages. (One WebAssembly page is 64KiB) ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +* `ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64)` ++ +tries to grow the memory by `new_pages` many pages containing zeroes. ++ +If successful, returns the _previous_ size of the memory (in pages). Otherwise, returns `-1`. ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +* `ic0.stable64_write : (offset : i64, src : i64, size : i64) -> ()` ++ +Copies the data from location [src, src+size) of the canister memory to location [offset, offset+size) in the stable memory. ++ +This system call traps if `src+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +* `ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> ()` ++ +Copies the data from location [offset, offset+size) of the stable memory to the location [dst, dst+size) in the canister memory. ++ This system call traps if `dst+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. [#system-api-time] === System time -The canister can query the system for the current time. +The canister can query the IC for the current time. `+ic0.time : () -> i64+` -The time is given as nanoseconds since 1970-01-01. The system guarantees that +The time is given as nanoseconds since 1970-01-01. The IC guarantees that * the time, as observed by the canister, is monotonically increasing, even across canister upgrades. * within an invocation of one entry point, the time is constant. @@ -1262,7 +1318,7 @@ NOTE: While an implementation will likely try to keep the System Time close to t [#system-api-certified-data] === Certified data -For each canister, the system keeps track of “certified data”, a canister-defined blob. For fresh canisters, this blob is the empty blob (`""`). +For each canister, the IC keeps track of “certified data”, a canister-defined blob. For fresh canisters, this blob is the empty blob (`""`). * `+ic0.certified_data_set : (src: i32, size : i32) -> ()+` + @@ -1292,7 +1348,7 @@ This traps if `ic0.data_certificate_present()` returns `0`. === Debugging aids -During local development and execution on a local network, the canister needs a way to emit textual trace messages. On the “real” network, these do not do anything. +In a local canister execution environment, the canister needs a way to emit textual trace messages. On the “real” network, these do not do anything. * `+ic0.debug_print : (src : i32, size : i32) -> ()+` + @@ -1300,7 +1356,7 @@ When executing in an environment that supports debugging, this copies out the da + Semantically, this function is always a no-op, and never traps, even if the `src+size` exceeds the size of the memory, or if this function is executed from `(start)`. If the environment cannot perform the print, it just skips it. -Similarly, the system allows the canister to effectively trap, but give some indication about why it trapped: +Similarly, the System API allows the canister to effectively trap, but give some indication about why it trapped: * `+ic0.trap : (src : i32, size : i32) -> ()+` + @@ -1313,11 +1369,11 @@ The environment may copy out the data specified by `src` and `size`, and log, pr The Internet Computer aims to make the most of the WebAssembly platform, and embraces WebAssembly features. With WebAssembly host references, we can make the platform more secure, the interfaces more abstract and more compositional. The above `ic0` System API does not yet use WebAssembly host references. Once they become available on our platform, a new version of the System API using host references will be available via the `ic` module. The changes will be, at least -1. The introduction of a `api_nonce` reference, which models the capability to use the System API. It is passed as an argument to `canister_init`, `canister_update ` etc., and expected as an argument by almost all system function calls. (The debugging aids remain unconstrained.) +1. The introduction of a `api_nonce` reference, which models the capability to use the System API. It is passed as an argument to `canister_init`, `canister_update ` etc., and expected as an argument by almost all System API function calls. (The debugging aids remain unconstrained.) 2. The use of references, instead of binary blobs, to address principals (user ids, canister ids), e.g. in `ic0.msg_caller` or in `ic0.call_new`. Additional functions will be provided to convert between the transparent binary representation of principals and references. 3. Making the builder interface to create calls build calls identified by a reference, rather than having an implicit partial call in the background. -A canister may only use the old _or_ the new interface; the system detects which interface the canister intends to use based on the names and types of its function imports and exports. +A canister may only use the old _or_ the new interface; the IC detects which interface the canister intends to use based on the names and types of its function imports and exports. [#ic-management-canister] == The IC management canister @@ -1343,27 +1399,27 @@ The binary encoding of arguments and results are as per Candid specification. [#ic-create_canister] === IC method `create_canister` -Before deploying a canister, the administrator of the canister first has to register it with the system, to get a canister id (with an empty canister behind it), and then separately install the code. +Before deploying a canister, the administrator of the canister first has to register it with the IC, to get a canister id (with an empty canister behind it), and then separately install the code. The optional `settings` parameter can be used to set the following settings: * `controllers` (`vec principal`) + -A list of principles. Must be between 0 and 10 in size. This value is assigned to the _controllers_ attribute of the canister. +A list of principals. Must be between 0 and 10 in size. This value is assigned to the _controllers_ attribute of the canister. + -Default value: A list containing only the caller of the `create_canister` call +Default value: A list containing only the caller of the `create_canister` call. * `compute_allocation` (`nat`) + Must be a number between 0 and 100, inclusively. It indicates how much compute power should be guaranteed to this canister, expressed as a percentage of the maximum compute power that a single canister can allocate. -If the system cannot provide the requested allocation, for example because it is oversubscribed, the call will be rejected. +If the IC cannot provide the requested allocation, for example because it is oversubscribed, the call will be rejected. + Default value: 0 * `memory_allocation` (`nat`) + -Must be a number between 0 and 2^48 (i.e 256TB), inclusively. It indicates how much memory the canister is allowed to use in total. Any attempt to grow memory usage beyond this allocation will fail. If the system cannot provide the requested allocation, for example because it is oversubscribed, the call will be rejected. -If set to 0, then memory growth of the canister will be best-effort and subject to the available memory on the network. +Must be a number between 0 and 2^48^ (i.e 256TB), inclusively. It indicates how much memory the canister is allowed to use in total. Any attempt to grow memory usage beyond this allocation will fail. If the IC cannot provide the requested allocation, for example because it is oversubscribed, the call will be rejected. +If set to 0, then memory growth of the canister will be best-effort and subject to the available memory on the IC. + Default value: 0 @@ -1371,9 +1427,9 @@ Until code is installed, the canister is `Empty` and behaves like a canister tha * `freezing_threshold` (`nat`) + -Must be a number between 0 and 2^64-1, inclusively, and indicates a length of time in seconds. +Must be a number between 0 and 2^64^-1, inclusively, and indicates a length of time in seconds. + -A canister is considered frozen whenever the system estimates that the canister would be depleted of cycles before `freezing_threshold` seconds pass, given the canister’s current size and the system’s current cost for storage. +A canister is considered frozen whenever the IC estimates that the canister would be depleted of cycles before `freezing_threshold` seconds pass, given the canister’s current size and the IC’s current cost for storage. + Calls to a frozen canister will be rejected (like for a stopping canister). Additionally, a canister cannot perform calls if that would, due the cost of the call and transferred cycles, would push the balance into frozen territory; these calls fail with `ic0.call_perform` returning a non-zero error code. + @@ -1394,13 +1450,13 @@ This method installs code into a canister. Only _controllers of the canister can install code. -* If `mode = install`, the canister must be empty before. This will instantiate the canister module and invoke its `canister_init` system method (if present), as explained in Section “<>”, passing the `arg` to the canister. +* If `mode = install`, the canister must be empty before. This will instantiate the canister module and invoke its `canister_init` method (if present), as explained in Section “<>”, passing the `arg` to the canister. * If `mode = reinstall`, if the canister was not empty, its existing code and state is removed before proceeding as for `mode = install`. + Note that this is different from `uninstall_code` followed by `install_code`, as that will forcibly reject all unresponded calls. -* If `mode = upgrade`, this will perform an upgrade of a non-empty canister as described in <>, passing `arg` to the `canister_post_upgrade` system method of the new instance. +* If `mode = upgrade`, this will perform an upgrade of a non-empty canister as described in <>, passing `arg` to the `canister_post_upgrade` method of the new instance. This is atomic: If the response to this request is a `reject`, then this call had no effect. @@ -1411,7 +1467,7 @@ NOTE: Some canisters may not be able to make sense of callbacks after upgrades; This method removes a canister’s code and state, making the canister _empty_ again. -Only _controllers of the canister can uninstall code. +Only controllers of the canister can uninstall code. Uninstalling a canister’s code will reject all calls that the canister has not yet responded to, and drop the canister’s code and state. Outstanding responses to the canister will not be processed, even if they arrive after code has been installed again. @@ -1438,7 +1494,7 @@ Only the controllers of the canister can request its status. The controllers of a canister may stop a canister (e.g., to prepare for a canister upgrade). Stopping a canister is not an atomic action. The immediate effect is that the status of the canister is changed to `stopping` (unless the canister is already stopped). -The system will reject all calls to a stopping canister, indicating that the canister is stopping. +The IC will reject all calls to a stopping canister, indicating that the canister is stopping. Responses to a stopping canister are processed as usual. When all outstanding responses have been processed (so there are no open call contexts), the canister status is changed to `stopped` and the management canister responds to the caller of the `stop_canister` request. @@ -1456,7 +1512,7 @@ If the canister was already `running` then the status stays unchanged. This method deletes a canister from the IC. -Only _controllers of the canister can delete it and the canister must already be stopped. Deleting a canister cannot be undone, any state stored on the canister is permanently deleted and its cycles are discarded. Once a canister is deleted, its ID cannot be reused. +Only controllers of the canister can delete it and the canister must already be stopped. Deleting a canister cannot be undone, any state stored on the canister is permanently deleted and its cycles are discarded. Once a canister is deleted, its ID cannot be reused. [#ic-deposit_cycles] === IC method `deposit_cycles` @@ -1473,29 +1529,29 @@ This method takes no input and returns 32 pseudo-random bytes to the caller. The [#ic-provisional_create_canister_with_cycles] === IC method `provisional_create_canister_with_cycles` -As a provisional method, until developers can convert real ICP tokens to provision a new canister with cycles, the system provides the `provisional_create_canister_with_cycles` method. It behaves as `create_canister`, but initializes the canister’s balance with `amount` fresh cycles (using `MAX_CANISTER_BALANCE` if `amount = null`, else capping the balance at `MAX_CANISTER_BALANCE`). +As a provisional method on development instances, the `provisional_create_canister_with_cycles` method is provided. It behaves as `create_canister`, but initializes the canister’s balance with `amount` fresh cycles (using `MAX_CANISTER_BALANCE` if `amount = null`, else capping the balance at `MAX_CANISTER_BALANCE`). Cycles added to this call via `ic0.call_cycles_add` are returned to the caller. -This method is only available in local development instances, and will be removed in the future. +This method is only available in local development instances. [#ic-provisional_top_up_canister] === IC method `provisional_top_up_canister` -As a provisional method, until developers can convert real ICP tokens to a top up an existing canister, the system provides the `provisional_top_up_canister` method. It adds `amount` cycles to the balance of canister identified by `amount` (implicitly capping it at `MAX_CANISTER_BALANCE`). +As a provisional method on development instances, the `provisional_top_up_canister` method is provided. It adds `amount` cycles to the balance of canister identified by `amount` (implicitly capping it at `MAX_CANISTER_BALANCE`). Cycles added to this call via `ic0.call_cycles_add` are returned to the caller. Any user can top-up any canister this way. -This method is only available in local development instances, and will be removed in the future. +This method is only available in local development instances. [#certification] == Certification -Some parts of the system state are exposed to clients in a tamperproof way via certification: the system can reveal a _partial state tree_ which includes just the data of interest, together with a signature on the root hash of the state tree. This means that a client can be sure that the response is correct, even if the client happens to be communicating with a malicious server, or has received the certificate via some other untrusted way. +Some parts of the IC state are exposed to users in a tamperproof way via certification: the IC can reveal a _partial state tree_ which includes just the data of interest, together with a signature on the root hash of the state tree. This means that a user can be sure that the response is correct, even if the user happens to be communicating with a malicious node, or has received the certificate via some other untrusted way. -To validate a value using a certificate, the client conceptually +To validate a value using a certificate, the user conceptually 1. checks the validity of the partial tree using `verify_cert`, 2. looks up the value in the certificate using `lookup` at a given path, which uses the subroutine `lookup_path` on the certificate's tree @@ -1504,7 +1560,7 @@ This mechanism is used in the `read_state` request type, and eventually also for === Root of trust -The root of trust is the _root public key_, which must be known to the client a priori. For temporary instances of the internet computer, e.g. during local development, the key can be fetched via the <> endpoint. +The root of trust is the _root public key_, which must be known to the user a priori. In a local canister execution environment, the key can be fetched via the <> endpoint. === Certificate @@ -1514,7 +1570,7 @@ A certificate consists of * a signature on the tree root hash valid under some _public key_ * an optional _delegation_ that links that public key to _root public key_. -The system will certify states, by issuing certificates where the tree is a partial state tree. The state tree can be pruned by replacing subtrees with their root hashes (yielding a new and potentially smaller but still valid certificate) to only include paths pertaining to relevant data but still preserving enough information to recover the _tree root hash_. +The IC will certify states by issuing certificates where the tree is a partial state tree. The state tree can be pruned by replacing subtrees with their root hashes (yielding a new and potentially smaller but still valid certificate) to only include paths pertaining to relevant data but still preserving enough information to recover the _tree root hash_. More formally, a certificate is described by the following data structure: .... @@ -1560,11 +1616,11 @@ extract_der : Blob -> Blob .... implements DER decoding of the public key, following https://tools.ietf.org/html/rfc5480[RFC4580] using OID 1.3.6.1.4.1.44668.5.3.1.2.1 for the algorithm and 1.3.6.1.4.1.44668.5.3.2.1 for the curve. -All state trees include the time at path `/time` (see <>). Clients that get a certificate with a state tree can look up the timestamp to guard against working on obsolete data. +All state trees include the time at path `/time` (see <>). Users that get a certificate with a state tree can look up the timestamp to guard against working on obsolete data. === Lookup -Given a (verified) tree, the client can fetch the value at a given path, which is a sequence of labels (blobs). In this document, we write paths suggestively with slashes as separators; the actual encoding is not actually using slashes as delimiters. +Given a (verified) tree, the user can fetch the value at a given path, which is a sequence of labels (blobs). In this document, we write paths suggestively with slashes as separators; the actual encoding is not actually using slashes as delimiters. The following algorithm looks up a `path` in a certificate, and returns either @@ -1601,7 +1657,7 @@ find_label(l, []) = Absent find_label(l, _) = Unknown .... -The system will only produce well-formed state trees, and the above algorithm assumes well-formed trees. These have the property that labeled subtrees appear in strictly increasing order of labels, and are not mixed with leaves. More formally: +The IC will only produce well-formed state trees, and the above algorithm assumes well-formed trees. These have the property that labeled subtrees appear in strictly increasing order of labels, and are not mixed with leaves. More formally: .... well_formed(tree) = (tree = Leaf _) ∨ (well_formed_forest(flatten_forks(tree))) @@ -1615,7 +1671,7 @@ well_formed_forest(trees) = [#certification-delegation] === Delegation -To facilitate a distributed implementation of the Internet Computer, the root key can delegate certification authority to other keys. +The root key can delegate certification authority to other keys. A certificate by the root subnet does not have a delegation field. A certificate by other subnets include a delegation, which is itself a certificate that proves that the subnet is listed in the root subnet’s state tree (see <>), and reveals its public key. @@ -1637,7 +1693,8 @@ check_delegations(Delegation d) : public_bls_key = verify_cert(d.certificate) return lookup(["subnet",d.subnet_id,"public_key"],d.certificate) .... -where `root_public_key` is the a priori known root key of the Internet Computer instance. +where `root_public_key` is the a priori known root key. + [#certification-encoding] === Encoding of certificates @@ -1736,7 +1793,7 @@ The design of this abstract specification (e.g. how and where pending messages a === Notation -We specify the behavior of the system using ad hoc pseudocode. +We specify the behavior of the Internet Computer using ad-hoc pseudocode. The manipulated values are primitive values (numbers, text, binary blobs), aggregate values (lists, unordered lists a.k.a. bags, partial maps, records with fixed fields, named constructors) and functions. @@ -1748,15 +1805,15 @@ NOTE: All values are immutable! State change is specified by describing the new Record fields are accessed using dot-notation (e.g. `S.request_id > 0`). To create a new record from an existing record `R` with some fields changed, the syntax `R where field = new_value` is used. This syntax can also be used to create new records with some deeply nested field changed: `R where some_map[key].field = new_value`. -In the state transitions, upper-case variables (`S`, `C`, `Req_id`) are free variables: The state transition may be taken for any possible value of these variables. `S` always refers to the state of the system before. A state transition often comes with a list of _conditions_, which may restrict the values of these free variables. The _state after_ is usually described using the record update syntax by starting with `S where`. +In the state transitions, upper-case variables (`S`, `C`, `Req_id`) are free variables: The state transition may be taken for any possible value of these variables. `S` always refers to the previous state. A state transition often comes with a list of _conditions_, which may restrict the values of these free variables. The _state after_ is usually described using the record update syntax by starting with `S where`. -For example, the condition `S.messages = Older_messages · M · Younger_messages` says that `M` is some message in field `messages` of the record `S`, and that `Younger_messages` and `Older_messages` are the other messages in the system. If the “state after” specifies `S with messages = Older_messages · Younger_messages`, then the message `M` is removed from the state. +For example, the condition `S.messages = Older_messages · M · Younger_messages` says that `M` is some message in field `messages` of the record `S`, and that `Younger_messages` and `Older_messages` are the other messages in the state. If the “state after” specifies `S with messages = Older_messages · Younger_messages`, then the message `M` is removed from the state. === Abstract state -In this specification, we describe the Internet Computer as a state machine. In particular, there is a single piece of data that describes the complete state of the system, called `S`. +In this specification, we describe the Internet Computer as a state machine. In particular, there is a single piece of data that describes the complete state of the IC, called `S`. -Of course, this is a huge simplification: The real Internet Computer is distributed and has a multi-component architecture, and the state is spread over many different components, some physically separated. But this simplification allows us to have a concise description of the system, and to easily make global decisions (such as, “is there any pending message”), without having to specify the bookkeeping that allows such global decisions. +Of course, this is a huge simplification: The real Internet Computer is distributed and has a multi-component architecture, and the state is spread over many different components, some physically separated. But this simplification allows us to have a concise description of the behavior, and to easily make global decisions (such as, “is there any pending message”), without having to specify the bookkeeping that allows such global decisions. ==== Identifiers @@ -1853,6 +1910,7 @@ CanisterModule = { post_upgrade : (CanisterId, StableMemory, Arg, Env) -> Trap | Return WasmState update_methods : MethodName ↦ ((Arg, Env, AvailableCycles) -> UpdateFunc) query_methods : MethodName ↦ ((Arg, Env) -> QueryFunc) + heartbeat : (Env) -> Trap | Return NoResponse callbacks : (Callback, Response, RefundedCycles, Env, AvailableCycles) -> UpdateFunc inspect_message : (MethodName, WasmState, Arg, Env) -> Trap | Return (Accept | Reject) } @@ -1876,7 +1934,7 @@ The concrete mapping of this abstract `CanisterModule` to actual WebAssembly con The Internet Computer provides certain messaging guarantees: If a user or a canister calls another canister, it will eventually get a single response (a reply or a rejection), even if some canister code along the way fails. -To ensure that only one response is generated, and also to detect when no response can be generated any more, the system maintains a _call context_. The `responded` field is set to `true` once the call has received a response. Further attempts to respond will now fail. +To ensure that only one response is generated, and also to detect when no response can be generated any more, the IC maintains a _call context_. The `responded` field is set to `true` once the call has received a response. Further attempts to respond will now fail. .... CallCtxt = { @@ -1895,18 +1953,20 @@ CallOrigin calling_context : CallId; callback: Callback } + | FromHeartbeat .... ==== Calls and Messages -Calls into and within the Internet Computer are implemented as messages passed between canisters. During their lifetime, messages change shape: they begin as a call to a public method, which is resolved to a WebAssembly function that is then executed, potentially generating a response which is then delivered. +Calls into and within the IC are implemented as messages passed between canisters. During their lifetime, messages change shape: they begin as a call to a public method, which is resolved to a WebAssembly function that is then executed, potentially generating a response which is then delivered. Therefore, a message can have different shapes: .... -Queue = Unordered | Queue { from : CanisterId; to : CanisterId } +Queue = Unordered | Queue { from : System | CanisterId; to : CanisterId } EntryPoint = PublicMethod MethodName Principal Blob | Callback Callback Response RefundedCycles + | Heartbeat Message = CallMessage { @@ -1937,7 +1997,7 @@ A reference implementation would likely maintain a separate list of `messages` f ==== API requests -We distinguish between the _asynchronous_ API requests passed to `/api/v2/…/call`, which may be present in the system state, and the _synchronous_ API requests passed to `/api/v2/…/read_state` and `/api/v2/…/query`, which are only ephemeral. +We distinguish between the _asynchronous_ API requests passed to `/api/v2/…/call`, which may be present in the IC state, and the _synchronous_ API requests passed to `/api/v2/…/read_state` and `/api/v2/…/query`, which are only ephemeral. .... Envelope = { @@ -2016,7 +2076,7 @@ SignedDelegation = { ==== The system state -Finally, we can describe the state of the Internet Computer as a record having the following fields: +Finally, we can describe the state of the IC as a record having the following fields: .... S = { @@ -2047,7 +2107,7 @@ CanStatus ==== Initial state -The initial state of the system is +The initial state of the IC is .... { requests = (); @@ -2094,11 +2154,11 @@ The following is an incomplete list of invariants that should hold for the abstr === State transitions -Based on this abstract notion of the state, we can describe the behavior of the system. There are three classes of behaviors: +Based on this abstract notion of the state, we can describe the behavior of the IC. There are three classes of behaviors: * Asynchronous API requests that are submitted via `/api/v2/…/call`. These transitions describe checks that the request must pass to be considered received. - * Spontaneous transitions that model the internal behavior of the system, by describing conditions on the state that allow the transition to happen, and the state after. - * Responses to reads (i.e. `/api/v2/…/read_state`). By definition, these do _not_ change the state of the system, and merely describe the response based on the read request and the current system state. + * Spontaneous transitions that model the internal behavior of the IC, by describing conditions on the state that allow the transition to happen, and the state after. + * Responses to reads (i.e. `/api/v2/…/read_state`). By definition, these do _not_ change the state of the IC, and merely describe the response based on the read request and the current state. The state transitions are not complete with regard to error handling. For example, the behavior of sending a request to a non-existent canister is not specified here. For now, we trust implementors to make sensible decisions there. @@ -2144,7 +2204,7 @@ is_effective_canister_id(CanisterUpdateCall {canister_id = p, …}, p), if p ≠ ==== API Request submission -After a node accepts a request via `/api/v2/canister//call`, it gets added to the system in the `Received` state. +After a node accepts a request via `/api/v2/canister//call`, the request gets added to the IC state as `Received`. This may only happen if the signature is valid and is created with a correct key. Due to this check, the envelope is discarded after this point. @@ -2180,11 +2240,11 @@ S with requests[E.content] = Received .... -NOTE: This is not instantaneous (the system takes some time to agree it accepts the request) nor guaranteed (a node could just drop the request, or maybe it did not pass validation). But once the request has entered the system like this, it will be acted upon. +NOTE: This is not instantaneous (the IC takes some time to agree it accepts the request) nor guaranteed (a node could just drop the request, or maybe it did not pass validation). But once the request has entered the IC state like this, it will be acted upon. ==== Request rejection -The system may reject a received message for internal reasons (high load, low resources) or expiry. The precise conditions are not specified here, but the reject code must indicate this to be a system error. +The IC may reject a received message for internal reasons (high load, low resources) or expiry. The precise conditions are not specified here, but the reject code must indicate this to be a system error. Conditions:: .... @@ -2201,9 +2261,9 @@ S with A first step in processing a canister update call is to create a `CallMessage` in the message queue. -The `request` field of the `FromUser` origin establishes the connection to the api message. One could use the corresponding `hash_of_map` for this purpose, but this formulation is more abstract. +The `request` field of the `FromUser` origin establishes the connection to the API message. One could use the corresponding `hash_of_map` for this purpose, but this formulation is more abstract. -We do not make any guarantees about the order of incoming messages. +The IC does not make any guarantees about the order of incoming messages. Conditions:: .... @@ -2254,18 +2314,20 @@ State after:: ==== Call context creation -Before invoking a message to a public entry point, some bookkeeping is required: - - * A call context is created, and the method is looked up in the list of exports. This happens for both ingress and inter-canister messages. -The canister must be running (so not stopped, or stopping). - -The position of the message in the queue is unchanged. - -This only happens for “real” canisters, not the IC management canister. +Before invoking a heartbeat or a message to a public entry point, a call context is created for bookkeeping purposes. +For these invocations the canister must be running (so not stopped, or stopping). +Additionally, these invocations only happen for "real" canisters, not the IC management canister. This “bookkeeping transition” must be immediately followed by the corresponding <>. +* Call context creation: Public entry points ++ +For a message to a public entry point, the method is looked up in the list of exports. This happens for both ingress and inter-canister messages. ++ +The position of the message in the queue is unchanged. ++ Conditions:: ++ .... S.messages = Older_messages · CallMessage CM · Younger_messages S.canisters[CM.callee] ≠ EmptyCanister @@ -2297,7 +2359,42 @@ S with balances[CM.callee] = balances[CM.callee] - MAX_CYCLES_PER_MESSAGE .... -We can execute any message that is at the head of its queue, i.e. there is no older message with the same abstract `queue` field. The actual message execution, if successful, may enqueue further messages and -- if the function returns a response -- record this response. The new call and response messages are enqueued at the end. +* Call context creation: Heartbeat ++ +If canister `C` exports a method with name `canister_heartbeat`, the IC will add a message to the head of the queue for invoking `canister_heartbeat` and create the corresponding call context. ++ +Conditions:: ++ +.... + S.canisters[C] ≠ EmptyCanister + S.canister_status[C] = Running + balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE + Ctxt_id ∉ dom S.call_contexts +.... ++ +State after:: ++ +.... +S with + messages = + FuncMessage { + call_context = Ctxt_id; + receiver = C; + entry_point = Heartbeat; + queue = Queue { from = System; to = C }; + } + · S.messages + call_contexts[Ctxt_id] = { + canister = C; + origin = FromHeartbeat; + responded = false; + deleted = false; + available_cycles = 0; + } + balances[C] = balances[C] - MAX_CYCLES_PER_MESSAGE +.... + +The IC can execute any message that is at the head of its queue, i.e. there is no older message with the same abstract `queue` field. The actual message execution, if successful, may enqueue further messages and -- if the function returns a response -- record this response. The new call and response messages are enqueued at the end. Note that new messages are executed only if the canister is Running and is not frozen. @@ -2334,6 +2431,10 @@ Conditions:: ( M.entry_point = Callback Callback Response F = Mod.callbacks(Callback, Response, Env, Available) ) + or + ( M.entry_point = Heartbeat + F = Mod.heartbeat(Env) + ) R = F(S.canisters[M.receiver].wasm_state) .... @@ -2402,7 +2503,7 @@ The cycle consumption of executing this message is modeled via the unspecified ` Depending whether this is a call message and a response messages, we have either set aside `MAX_CYCLES_PER_MESSAGE` or `MAX_CYCLES_PER_RESPONSE`, either in the call context creation rule or the Callback invocation rule. -This transition detects certain behaviour that will appear as a trap (and which an implementation may implement by trapping directly in a system call): +This transition detects certain behavior that will appear as a trap (and which an implementation may implement by trapping directly in a system call): * Responding if the present call context has already been responded to * Accepting more cycles than are available on the call context @@ -2414,7 +2515,7 @@ If message execution < If message execution <>, the state is updated and possible outbound calls and responses are enqueued. Note that returning does _not_ imply that the call associated with this message now _succeeds_ in the sense defined in <>; that would require a (unique) call to `ic0.reply`. -Note also that the state changes are persisted even when the system is set to synthesize a <> reject immediately afterward (which happens when this returns without calling `ic0.reply` or `ic0.reject`, the corresponding call has not been responded to and there are no outstanding callbacks, see <>). +Note also that the state changes are persisted even when the IC is set to synthesize a <> reject immediately afterward (which happens when this returns without calling `ic0.reply` or `ic0.reject`, the corresponding call has not been responded to and there are no outstanding callbacks, see <>). The function `as_update` turns a query function into an update function, this is merely a notational trick to simplify the rule .... @@ -2434,11 +2535,12 @@ Note that by construction, a query function will either trap or return with a re [#rule-starvation] ==== Call context starvation -If there is no call, downstream calling context or response that could possibly fulfill a calling context, then a reject is synthesized. The error message below is _not_ indicative. In particular, if the system has an idea about _why_ this starved, it can put that in there (e.g. the initial message handler trapped with an out-of-memory access). +If the call context is not for heartbeat and there is no call, downstream calling context or response that could possibly fulfill a calling context, then a reject is synthesized. The error message below is _not_ indicative. In particular, if the IC has an idea about _why_ this starved, it can put that in there (e.g. the initial message handler trapped with an out-of-memory access). Conditions:: .... S.call_contexts[Ctxt_id].responded = false + S.call_contexts[Ctxt_id].origin ≠ FromHeartbeat ∀ CallMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id ∀ ResponseMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id ∀ ctxt_ids. @@ -2461,11 +2563,17 @@ S with ==== Call context removal -If there is no call, downstream calling context or response that references a call context, and the call context has been replied to, then the call context can be removed. +If there is no call, downstream calling context, or response that references a call context, and the call context has been replied to or the call context corresponds to a heartbeat that had already been executed, then the call context can be removed. Conditions:: .... - S.call_contexts[Ctxt_id].responded = true + ( + S.call_contexts[Ctxt_id].responded = true + ) or + ( + S.call_contexts[Ctxt_id].origin = FromHeartbeat + ∀ FuncMessage M ∈ S.messages. M.call_context ≠ Ctxt_id + ) ∀ CallMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id ∀ ResponseMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id ∀ ctxt_ids. @@ -2481,7 +2589,7 @@ S with ==== IC Management Canister: Canister creation -The system chooses an appropriate canister id and instantiates a new (empty) canister identified by this id. The _controllers_ are set such that the sender of this request is the only controller, unless the `settings` say otherwise. All cycles on this call are now the canister's initial cycles. +The IC chooses an appropriate canister id and instantiates a new (empty) canister identified by this id. The _controllers_ are set such that the sender of this request is the only controller, unless the `settings` say otherwise. All cycles on this call are now the canister's initial cycles. This is also when the System Time of the new canister starts ticking. @@ -2597,7 +2705,7 @@ S with ==== IC Management Canister: Code installation -Only the controllers of the given canister can install code. This transition installs new code over a canister. This involves invoking the `canister_init` system method (see <>), which must succeed. +Only the controllers of the given canister can install code. This transition installs new code over a canister. This involves invoking the `canister_init` method (see <>), which must succeed. Conditions:: .... @@ -2638,7 +2746,7 @@ S with ==== IC Management Canister: Code upgrade -Only the controllers of the given canister can install new code. This changes the code of an _existing_ canister, preserving the state in the stable memory. This involves invoking the `canister_pre_upgrade` system method on the old and `canister_post_upgrade` system method on the new canister, which must succeed and must not invoke other methods. +Only the controllers of the given canister can install new code. This changes the code of an _existing_ canister, preserving the state in the stable memory. This involves invoking the `canister_pre_upgrade` method on the old and `canister_post_upgrade` method on the new canister, which must succeed and must not invoke other methods. Conditions:: .... @@ -3073,7 +3181,7 @@ NB: The refunded cycles, `RM.refunded_cycles` are, by construction, empty. ==== Request clean up -The system will keep the data for a completed or rejected request around for a certain, implementation defined amount of time, to allow clients to poll for the data. After that time, the data of the request will be dropped: +The IC will keep the data for a completed or rejected request around for a certain, implementation defined amount of time, to allow users to poll for the data. After that time, the data of the request will be dropped: Conditions:: .... @@ -3086,7 +3194,7 @@ S with .... -At the same or some later point, the request will be removed from memory of the system. This must happen no earlier than the ingress expiry time set in the request. +At the same or some later point, the request will be removed from the state of the IC. This must happen no earlier than the ingress expiry time set in the request. Conditions:: .... @@ -3255,7 +3363,7 @@ The response is a certificate `cert`, as specified in <>, which p .... lookup(path, cert) = lookup_in_tree(path, state_tree(S)) .... -where `state_tree` constructs the a labeled tree from the system state `S` and the (so far underspecified) set of subnets `subnets`, as per <> +where `state_tree` constructs the a labeled tree from the IC state `S` and the (so far underspecified) set of subnets `subnets`, as per <> .... state_tree(S) = { "time": S.system_time; @@ -3288,7 +3396,7 @@ In Section <> we introduced an abstraction over the interfac ==== The concrete `WasmState` -The abstract `WasmState` above models the WebAssembly _store_ `S`, which encompasses the functions, tables, memories and globals of the WebAssembly program, plus additional data maintained by the system, such as the stable memory: +The abstract `WasmState` above models the WebAssembly _store_ `S`, which encompasses the functions, tables, memories and globals of the WebAssembly program, plus additional data maintained by the IC, such as the stable memory: .... WasmState = { store : S; // a store as per WebAssembly spec @@ -3500,6 +3608,22 @@ This formulation checks afterwards that the system call `ic0.call_perform` was n + By construction, the (possibly modified) `es.wasm_state` is discarded. +* The partial map `heartbeat` of the `CanisterModule` is defined if the WebAssembly program exports a function `func` named `canister_heartbeat`, and has value ++ +.... +heartbeat = λ (sysenv) → λ wasm_state → + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = empty_params with { data = NoData; caller = NoCaller; sysenv } + balance = sysenv.balance + } + try func() with Trap then Trap + if es.cycles_accepted ≠ 0 then Trap + if es.ingress_filter ≠ Reject then Trap + if es.response ≠ NoResponse then Trap + Return NoResponse; +.... + * The function `callbacks` of the `CanisterModule` is defined as follows + .... @@ -3754,24 +3878,53 @@ discard_pending_call() = es.pending_call := NoPendingCall ic0.stable_size() : (page_count : i32) = - return |es.wasm_state.stable_mem| / 64k + if |es.wasm_state.store.mem| > 2^32 then Trap + page_count := |es.wasm_state.stable_mem| / 64k + return page_count ic0.stable_grow(new_pages : i32) : (old_page_count : i32) = - if arbitrary() - then return -1 + if |es.wasm_state.store.mem| > 2^32 then Trap + if arbitrary() then return -1 else old_size := |es.wasm_state.stable_mem| / 64k + if old_size + new_pages > 2^16 then return -1 es.wasm_state.stable_mem := es.wasm_state.stable_mem · repeat(0x00, new_pages * 64k) return old_size ic0.stable_write(offset : i32, src : i32, size : i32) + if |es.wasm_state.store.mem| > 2^32 then Trap if src+size > |es.wasm_state.store.mem| then Trap if offset+size > |es.wasm_state.stable_mem| then Trap es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] ic0.stable_read(dst : i32, offset : i32, size : i32) + if |es.wasm_state.store.mem| > 2^32 then Trap + if offset+size > |es.wasm_state.stable_mem| then Trap + if dst+size > |es.wasm_state.store.mem| then Trap + + es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] + +ic0.stable64_size() : (page_count : i64) = + return |es.wasm_state.stable_mem| / 64k + +ic0.stable64_grow(new_pages : i64) : (old_page_count : i64) = + if arbitrary() + then return -1 + else + old_size := |es.wasm_state.stable_mem| / 64k + es.wasm_state.stable_mem := + es.wasm_state.stable_mem · repeat(0x00, new_pages * 64k) + return old_size + +ic0.stable64_write(offset : i64, src : i64, size : i64) + if src+size > |es.wasm_state.store.mem| then Trap + if offset+size > |es.wasm_state.stable_mem| then Trap + + es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] + +ic0.stable64_read(dst : i64, offset : i64, size : i64) if offset+size > |es.wasm_state.stable_mem| then Trap if dst+size > |es.wasm_state.store.mem| then Trap From e786979fd1df48b753d1b89aac8a9cf824f4fe88 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bj=C3=B6rn=20Tackmann?= <54846571+Dfinity-Bjoern@users.noreply.github.com> Date: Fri, 1 Oct 2021 11:04:00 +0200 Subject: [PATCH 003/102] Publish release 0.18.2 on the web site (#376) * ic-ref-test: Pass cycles with 10-controllers-test * Remove reference implementation (#348) it has been open sourced and moved to https://github.com/dfinity/ic-hs. * Release version 0.18.1 (#364) * Release 0.18.2 (#372) * Add heartbeat to spec * Terminology changes in release 0.18.2 * Add support for 64-bit stable memory Co-authored-by: Joachim Breitner --- default.nix | 233 -- impl/.envrc | 1 - impl/.gitignore | 24 - impl/.hlint.yaml | 8 - impl/cabal.project | 11 - impl/cabal.project.freeze | 168 - impl/cbits/aes.c | 801 ----- impl/cbits/arch.h | 131 - impl/cbits/big_384_58.c | 1677 --------- impl/cbits/big_384_58.h | 635 ---- impl/cbits/bls_BLS12381.c | 176 - impl/cbits/bls_BLS12381.h | 81 - impl/cbits/config_big_384_58.h | 38 - impl/cbits/config_curve_BLS12381.h | 76 - impl/cbits/config_field_BLS12381.h | 45 - impl/cbits/core.h | 789 ----- impl/cbits/ecdh_BLS12381.h | 154 - impl/cbits/ecp2_BLS12381.c | 1005 ------ impl/cbits/ecp2_BLS12381.h | 248 -- impl/cbits/ecp_BLS12381.c | 1762 --------- impl/cbits/ecp_BLS12381.h | 344 -- impl/cbits/fp12_BLS12381.c | 986 ----- impl/cbits/fp12_BLS12381.h | 232 -- impl/cbits/fp2_BLS12381.c | 488 --- impl/cbits/fp2_BLS12381.h | 281 -- impl/cbits/fp4_BLS12381.c | 698 ---- impl/cbits/fp4_BLS12381.h | 332 -- impl/cbits/fp_BLS12381.c | 886 ----- impl/cbits/fp_BLS12381.h | 313 -- impl/cbits/gcm.c | 436 --- impl/cbits/hash.c | 606 ---- impl/cbits/hmac.c | 359 -- impl/cbits/hpke_BLS12381.h | 114 - impl/cbits/mpin_BLS12381.h | 130 - impl/cbits/newhope.c | 510 --- impl/cbits/newhope.h | 58 - impl/cbits/oct.c | 413 --- impl/cbits/pair_BLS12381.c | 1165 ------ impl/cbits/pair_BLS12381.h | 172 - impl/cbits/rand.c | 172 - impl/cbits/randapi.c | 34 - impl/cbits/randapi.h | 46 - impl/cbits/rom_curve_BLS12381.c | 108 - impl/cbits/rom_field_BLS12381.c | 52 - impl/cbits/share.c | 230 -- impl/cbits/x509.h | 109 - impl/default.nix | 2 - impl/example.cbor | Bin 70 -> 0 bytes impl/ic-ref.cabal | 253 -- impl/ic.did | 54 - impl/shell.nix | 2 - impl/src/IC/CBOR/Parser.hs | 45 - impl/src/IC/CBOR/Patterns.hs | 32 - impl/src/IC/CBOR/Utils.hs | 9 - impl/src/IC/Canister.hs | 110 - impl/src/IC/Canister/Imp.hs | 722 ---- impl/src/IC/Canister/Snapshot.hs | 38 - impl/src/IC/Certificate.hs | 17 - impl/src/IC/Certificate/CBOR.hs | 44 - impl/src/IC/Certificate/Validate.hs | 52 - impl/src/IC/Certificate/Value.hs | 31 - impl/src/IC/Constants.hs | 6 - impl/src/IC/Crypto.hs | 123 - impl/src/IC/Crypto/BLS.hsc | 98 - impl/src/IC/Crypto/CanisterSig.hs | 86 - impl/src/IC/Crypto/DER.hs | 91 - impl/src/IC/Crypto/DER/Decode.hs | 20 - impl/src/IC/Crypto/DER_BLS.hs | 61 - impl/src/IC/Crypto/ECDSA.hs | 60 - impl/src/IC/Crypto/Ed25519.hs | 32 - impl/src/IC/Crypto/Secp256k1.hs | 78 - impl/src/IC/Crypto/WebAuthn.hs | 166 - impl/src/IC/DRun/Parse.hs | 65 - impl/src/IC/Debug/JSON.hs | 188 - impl/src/IC/HTTP.hs | 139 - impl/src/IC/HTTP/CBOR.hs | 63 - impl/src/IC/HTTP/GenR.hs | 34 - impl/src/IC/HTTP/GenR/Parse.hs | 84 - impl/src/IC/HTTP/Request.hs | 159 - impl/src/IC/HTTP/RequestId.hs | 33 - impl/src/IC/HTTP/Status.hs | 20 - impl/src/IC/Hash.hs | 12 - impl/src/IC/HashTree.hs | 148 - impl/src/IC/HashTree/CBOR.hs | 27 - impl/src/IC/Id/Forms.hs | 42 - impl/src/IC/Id/Fresh.hs | 15 - impl/src/IC/Management.hs | 45 - impl/src/IC/Purify.hs | 60 - impl/src/IC/Ref.hs | 1192 ------- impl/src/IC/Serialise.hs | 131 - impl/src/IC/StateFile.hs | 158 - impl/src/IC/Test/BLS.hs | 51 - impl/src/IC/Test/ECDSA.hs | 34 - impl/src/IC/Test/HashTree.hs | 152 - impl/src/IC/Test/Options.hs | 23 - impl/src/IC/Test/Secp256k1.hs | 43 - impl/src/IC/Test/Spec.hs | 3156 ----------------- impl/src/IC/Test/Universal.hs | 282 -- impl/src/IC/Test/WebAuthn.hs | 43 - impl/src/IC/Types.hs | 180 - impl/src/IC/Utils.hs | 18 - impl/src/IC/Version.hs | 9 - impl/src/IC/Wasm/Imports.hs | 416 --- impl/src/IC/Wasm/Winter.hs | 136 - impl/src/IC/Wasm/Winter/Persist.hs | 128 - impl/src/IC/Wasm/WinterMemory.hs | 58 - impl/src/SourceId.hs | 47 - impl/src/ic-ref-run.hs | 185 - impl/src/ic-ref-test.hs | 31 - impl/src/ic-ref.hs | 81 - impl/src/ic-request-id.hs | 45 - impl/src/unit-tests.hs | 74 - impl/test-data/trivial.wat | 1 - impl/test-data/universal_canister.wasm | 1 - nix/default.nix | 33 +- nix/generate.nix | 113 - nix/generated/QuickCheck.nix | 29 - nix/generated/base32.nix | 39 - nix/generated/base64-bytestring.nix | 41 - nix/generated/candid.nix | 99 - nix/generated/ic-ref.nix | 185 - nix/generated/leb128-cereal.nix | 28 - nix/generated/megaparsec.nix | 43 - nix/generated/random.nix | 57 - nix/generated/splitmix.nix | 53 - nix/generated/winter.nix | 109 - nix/haskell-packages.nix | 60 - ...e-binary-distribution-which-links-ag.patch | 80 - nix/patches/0002-openblas-0.3.10-0.3.13.patch | 74 - ...3991b26b2b93dece6d09f37041451a5ef4cb.patch | 51 - nix/python-cbor2.nix | 23 - shell.nix | 1 - spec/changelog.adoc | 13 + spec/index.adoc | 572 +-- universal-canister/.envrc | 1 - universal-canister/.gitignore | 1 - universal-canister/Cargo.lock | 69 - universal-canister/Cargo.toml | 8 - universal-canister/default.nix | 2 - universal-canister/shell.nix | 2 - universal-canister/src/api.rs | 280 -- universal-canister/src/lib.rs | 1 - universal-canister/src/main.rs | 384 -- 143 files changed, 373 insertions(+), 29456 deletions(-) delete mode 100644 impl/.envrc delete mode 100644 impl/.gitignore delete mode 100644 impl/.hlint.yaml delete mode 100644 impl/cabal.project delete mode 100644 impl/cabal.project.freeze delete mode 100644 impl/cbits/aes.c delete mode 100644 impl/cbits/arch.h delete mode 100644 impl/cbits/big_384_58.c delete mode 100644 impl/cbits/big_384_58.h delete mode 100644 impl/cbits/bls_BLS12381.c delete mode 100644 impl/cbits/bls_BLS12381.h delete mode 100644 impl/cbits/config_big_384_58.h delete mode 100644 impl/cbits/config_curve_BLS12381.h delete mode 100644 impl/cbits/config_field_BLS12381.h delete mode 100644 impl/cbits/core.h delete mode 100644 impl/cbits/ecdh_BLS12381.h delete mode 100644 impl/cbits/ecp2_BLS12381.c delete mode 100644 impl/cbits/ecp2_BLS12381.h delete mode 100644 impl/cbits/ecp_BLS12381.c delete mode 100644 impl/cbits/ecp_BLS12381.h delete mode 100644 impl/cbits/fp12_BLS12381.c delete mode 100644 impl/cbits/fp12_BLS12381.h delete mode 100644 impl/cbits/fp2_BLS12381.c delete mode 100644 impl/cbits/fp2_BLS12381.h delete mode 100644 impl/cbits/fp4_BLS12381.c delete mode 100644 impl/cbits/fp4_BLS12381.h delete mode 100644 impl/cbits/fp_BLS12381.c delete mode 100644 impl/cbits/fp_BLS12381.h delete mode 100644 impl/cbits/gcm.c delete mode 100644 impl/cbits/hash.c delete mode 100644 impl/cbits/hmac.c delete mode 100644 impl/cbits/hpke_BLS12381.h delete mode 100644 impl/cbits/mpin_BLS12381.h delete mode 100644 impl/cbits/newhope.c delete mode 100644 impl/cbits/newhope.h delete mode 100644 impl/cbits/oct.c delete mode 100644 impl/cbits/pair_BLS12381.c delete mode 100644 impl/cbits/pair_BLS12381.h delete mode 100644 impl/cbits/rand.c delete mode 100644 impl/cbits/randapi.c delete mode 100644 impl/cbits/randapi.h delete mode 100644 impl/cbits/rom_curve_BLS12381.c delete mode 100644 impl/cbits/rom_field_BLS12381.c delete mode 100644 impl/cbits/share.c delete mode 100644 impl/cbits/x509.h delete mode 100644 impl/default.nix delete mode 100644 impl/example.cbor delete mode 100644 impl/ic-ref.cabal delete mode 100644 impl/ic.did delete mode 100644 impl/shell.nix delete mode 100644 impl/src/IC/CBOR/Parser.hs delete mode 100644 impl/src/IC/CBOR/Patterns.hs delete mode 100644 impl/src/IC/CBOR/Utils.hs delete mode 100644 impl/src/IC/Canister.hs delete mode 100644 impl/src/IC/Canister/Imp.hs delete mode 100644 impl/src/IC/Canister/Snapshot.hs delete mode 100644 impl/src/IC/Certificate.hs delete mode 100644 impl/src/IC/Certificate/CBOR.hs delete mode 100644 impl/src/IC/Certificate/Validate.hs delete mode 100644 impl/src/IC/Certificate/Value.hs delete mode 100644 impl/src/IC/Constants.hs delete mode 100644 impl/src/IC/Crypto.hs delete mode 100644 impl/src/IC/Crypto/BLS.hsc delete mode 100644 impl/src/IC/Crypto/CanisterSig.hs delete mode 100644 impl/src/IC/Crypto/DER.hs delete mode 100644 impl/src/IC/Crypto/DER/Decode.hs delete mode 100644 impl/src/IC/Crypto/DER_BLS.hs delete mode 100644 impl/src/IC/Crypto/ECDSA.hs delete mode 100644 impl/src/IC/Crypto/Ed25519.hs delete mode 100644 impl/src/IC/Crypto/Secp256k1.hs delete mode 100644 impl/src/IC/Crypto/WebAuthn.hs delete mode 100644 impl/src/IC/DRun/Parse.hs delete mode 100644 impl/src/IC/Debug/JSON.hs delete mode 100644 impl/src/IC/HTTP.hs delete mode 100644 impl/src/IC/HTTP/CBOR.hs delete mode 100644 impl/src/IC/HTTP/GenR.hs delete mode 100644 impl/src/IC/HTTP/GenR/Parse.hs delete mode 100644 impl/src/IC/HTTP/Request.hs delete mode 100644 impl/src/IC/HTTP/RequestId.hs delete mode 100644 impl/src/IC/HTTP/Status.hs delete mode 100644 impl/src/IC/Hash.hs delete mode 100644 impl/src/IC/HashTree.hs delete mode 100644 impl/src/IC/HashTree/CBOR.hs delete mode 100644 impl/src/IC/Id/Forms.hs delete mode 100644 impl/src/IC/Id/Fresh.hs delete mode 100644 impl/src/IC/Management.hs delete mode 100644 impl/src/IC/Purify.hs delete mode 100644 impl/src/IC/Ref.hs delete mode 100644 impl/src/IC/Serialise.hs delete mode 100644 impl/src/IC/StateFile.hs delete mode 100644 impl/src/IC/Test/BLS.hs delete mode 100644 impl/src/IC/Test/ECDSA.hs delete mode 100644 impl/src/IC/Test/HashTree.hs delete mode 100644 impl/src/IC/Test/Options.hs delete mode 100644 impl/src/IC/Test/Secp256k1.hs delete mode 100644 impl/src/IC/Test/Spec.hs delete mode 100644 impl/src/IC/Test/Universal.hs delete mode 100644 impl/src/IC/Test/WebAuthn.hs delete mode 100644 impl/src/IC/Types.hs delete mode 100644 impl/src/IC/Utils.hs delete mode 100644 impl/src/IC/Version.hs delete mode 100644 impl/src/IC/Wasm/Imports.hs delete mode 100644 impl/src/IC/Wasm/Winter.hs delete mode 100644 impl/src/IC/Wasm/Winter/Persist.hs delete mode 100644 impl/src/IC/Wasm/WinterMemory.hs delete mode 100644 impl/src/SourceId.hs delete mode 100644 impl/src/ic-ref-run.hs delete mode 100644 impl/src/ic-ref-test.hs delete mode 100644 impl/src/ic-ref.hs delete mode 100644 impl/src/ic-request-id.hs delete mode 100644 impl/src/unit-tests.hs delete mode 100644 impl/test-data/trivial.wat delete mode 120000 impl/test-data/universal_canister.wasm delete mode 100644 nix/generate.nix delete mode 100644 nix/generated/QuickCheck.nix delete mode 100644 nix/generated/base32.nix delete mode 100644 nix/generated/base64-bytestring.nix delete mode 100644 nix/generated/candid.nix delete mode 100644 nix/generated/ic-ref.nix delete mode 100644 nix/generated/leb128-cereal.nix delete mode 100644 nix/generated/megaparsec.nix delete mode 100644 nix/generated/random.nix delete mode 100644 nix/generated/splitmix.nix delete mode 100644 nix/generated/winter.nix delete mode 100644 nix/haskell-packages.nix delete mode 100644 nix/patches/0001-ghc865-binary-Use-binary-distribution-which-links-ag.patch delete mode 100644 nix/patches/0002-openblas-0.3.10-0.3.13.patch delete mode 100644 nix/patches/fb063991b26b2b93dece6d09f37041451a5ef4cb.patch delete mode 100644 nix/python-cbor2.nix delete mode 100644 shell.nix delete mode 100644 universal-canister/.envrc delete mode 100644 universal-canister/.gitignore delete mode 100644 universal-canister/Cargo.lock delete mode 100644 universal-canister/Cargo.toml delete mode 100644 universal-canister/default.nix delete mode 100644 universal-canister/shell.nix delete mode 100644 universal-canister/src/api.rs delete mode 100644 universal-canister/src/lib.rs delete mode 100644 universal-canister/src/main.rs diff --git a/default.nix b/default.nix index 827cf1728..e35311795 100644 --- a/default.nix +++ b/default.nix @@ -6,222 +6,7 @@ let nixpkgs = import ./nix { inherit system; }; in let stdenv = nixpkgs.stdenv; in let subpath = p: import ./nix/gitSource.nix p; in -let naersk = nixpkgs.callPackage nixpkgs.sources.naersk {}; in -let universal-canister = (naersk.buildPackage rec { - name = "universal-canister"; - src = subpath ./universal-canister; - root = ./universal-canister; - CARGO_TARGET_WASM32_UNKNOWN_UNKNOWN_LINKER = "${nixpkgs.llvmPackages_9.lld}/bin/lld"; - RUSTFLAGS = "-C link-arg=-s"; # much smaller wasm - cargoBuildOptions = x : x ++ [ "--target wasm32-unknown-unknown" ]; - doCheck = false; - release = true; -}).overrideAttrs (old: { - postFixup = (old.postFixup or "") + '' - mv $out/bin/universal_canister $out/universal_canister.wasm - rmdir $out/bin - ''; -}); in - - -let haskellPackages = nixpkgs.haskellPackages.override { - overrides = import nix/haskell-packages.nix nixpkgs subpath; -}; in - -let - ic-ref = nixpkgs.haskell.lib.dontCheck ( - haskellPackages.ic-ref.overrideAttrs (old: { - installPhase = (old.installPhase or "") + '' - cp -rv test-data $out/test-data - # replace symlink with actually built - rm -f $out/test-data/universal_canister.wasm - cp ${universal-canister}/universal_canister.wasm $out/test-data - ''; - # variant of justStaticExecutables that retains propagatedBuildInputs - postFixup = "rm -rf $out/lib $out/share/doc"; - }) - ); - - # This is a static build of the ic-ref tool only, - # for distribution independent of nix - ic-ref-dist = - if nixpkgs.stdenv.isDarwin - # on Darwin, use dylibbundler to include non-system libraries - then nixpkgs.runCommandNoCC "ic-ref-dist" { - buildInputs = [ nixpkgs.macdylibbundler nixpkgs.removeReferencesTo ]; - allowedRequisites = []; - } '' - mkdir -p $out/bin - cp ${ic-ref}/bin/ic-ref $out/bin - chmod u+w $out/bin/ic-ref - dylibbundler \ - -b \ - -x $out/bin/ic-ref \ - -d $out/bin \ - -p '@executable_path' \ - -i /usr/lib/system \ - -i ${nixpkgs.darwin.Libsystem}/lib - - # there are still plenty of nix store references - # but they should not matter - remove-references-to \ - -t ${nixpkgs.darwin.Libsystem} \ - -t ${nixpkgs.darwin.CF} \ - -t ${nixpkgs.libiconv} \ - $out/bin/* - - # sanity check - $out/bin/ic-ref --version - '' - - # on Linux, build statically using musl - # and until we are open source, also using integer-simple - # (once we can use ghc-9.0 we can maybe use ghc-bignum native, which should be faster) - else - let - muslHaskellPackages = nixpkgs.pkgsMusl.haskell.packages.integer-simple.ghc884.override { - overrides = self: super: - import nix/haskell-packages.nix nixpkgs subpath self super - // { - cryptonite = super.cryptonite.overrideAttrs(old: { - configureFlags = "-f-integer-gmp"; - doCheck = false; # test suite too slow without integer-gmp - }); - # more test suites too slow withour integer-gmp - scientific = nixpkgs.haskell.lib.dontCheck super.scientific; - math-functions = nixpkgs.haskell.lib.dontCheck super.math-functions; - }; - }; - ic-ref-musl = - muslHaskellPackages.ic-ref.overrideAttrs ( - old: { - configureFlags = [ - "--ghc-option=-optl=-static" - "--extra-lib-dirs=${nixpkgs.pkgsMusl.zlib.static}/lib" - "--extra-lib-dirs=${nixpkgs.pkgsMusl.libffi.overrideAttrs (old: { dontDisableStatic = true; })}/lib" - ]; - } - ); - in nixpkgs.runCommandNoCC "ic-ref-dist" { - allowedRequisites = []; - } '' - mkdir -p $out/bin - cp ${ic-ref-musl}/bin/ic-ref $out/bin - ''; - - - # We run the unit test suite only as part of coverage checking (saves time) - ic-ref-coverage = nixpkgs.haskell.lib.doCheck (nixpkgs.haskell.lib.doCoverage ic-ref); -in - - - rec { - inherit ic-ref; - inherit ic-ref-dist; - inherit ic-ref-coverage; - inherit universal-canister; - - ic-ref-test = nixpkgs.runCommandNoCC "ic-ref-test" { - nativeBuildInputs = [ ic-ref ]; - } '' - function kill_ic_ref () { kill %1; } - ic-ref --pick-port --write-port-to port & - trap kill_ic_ref EXIT PIPE - sleep 1 - test -e port - mkdir -p $out - LANG=C.UTF8 ic-ref-test --endpoint "http://0.0.0.0:$(cat port)/" --html $out/report.html - - mkdir -p $out/nix-support - echo "report test-results $out report.html" >> $out/nix-support/hydra-build-products - ''; - - coverage = nixpkgs.runCommandNoCC "ic-ref-test" { - nativeBuildInputs = [ haskellPackages.ghc ic-ref-coverage ]; - } '' - function kill_ic_ref () { kill %1; } - ic-ref --pick-port --write-port-to port & - trap kill_ic_ref EXIT PIPE - sleep 1 - test -e port - LANG=C.UTF8 ic-ref-test --endpoint "http://0.0.0.0:$(cat port)/" - kill -INT %1 - trap - EXIT PIPE - sleep 5 # wait for ic-ref.tix to be written - - find - LANG=C.UTF8 hpc markup ic-ref.tix --hpcdir=${ic-ref-coverage}/share/hpc/vanilla/mix/ic-ref --srcdir=${subpath ./impl} --destdir $out - - mkdir -p $out/nix-support - echo "report coverage $out hpc_index.html" >> $out/nix-support/hydra-build-products - ''; - - # The following two derivations keep the impl/cabal.products.freeze files - # up to date. It is quite hacky to get the package data base for the ic-ref - # derivation, and then convince Cabal to use that... - cabal-freeze = (nixpkgs.haskell.lib.doCheck haskellPackages.ic-ref).overrideAttrs(old: { - nativeBuildInputs = (old.nativeBuildInputs or []) ++ [ nixpkgs.cabal-install ]; - phases = [ "unpackPhase" "setupCompilerEnvironmentPhase" "buildPhase" "installPhase" ]; - buildPhase = '' - rm -f cabal.project.freeze cabal.project - unset GHC_PACKAGE_PATH - mkdir .cabal - touch .cabal/config # empty, no repository - HOME=$PWD cabal v2-freeze --ghc-pkg-options="-f $packageConfDir" --offline --enable-tests || true - ''; - outputs = ["out"]; # no docs - installPhase = '' - mkdir -p $out - echo "-- Run nix-shell .. -A check-cabal-freeze to update this file" > $out/cabal.project.freeze - cat cabal.project.freeze >> $out/cabal.project.freeze - ''; - }); - - check-cabal-freeze = nixpkgs.runCommandNoCC "check-cabal-freeze" { - nativeBuildInputs = [ nixpkgs.diffutils ]; - expected = cabal-freeze + /cabal.project.freeze; - actual = ./impl/cabal.project.freeze; - cmd = "nix-shell . -A check-cabal-freeze"; - shellHook = '' - dest=${toString ./impl/cabal.project.freeze} - rm -f $dest - cp -v $expected $dest - chmod u-w $dest - exit 0 - ''; - } '' - diff -r -U 3 $actual $expected || - { echo "To update, please run"; echo "nix-shell . -A check-cabal-freeze"; exit 1; } - touch $out - ''; - - check-generated = nixpkgs.runCommandNoCC "check-generated" { - nativeBuildInputs = [ nixpkgs.diffutils ]; - expected = import ./nix/generate.nix { pkgs = nixpkgs; }; - dir = ./nix/generated; - } '' - diff -r -U 3 $expected $dir - touch $out - ''; - - # A simple license check: Check that all used Haskell packages - # declare a liberal (non-GPL) license. - # This does not necessarily cover imported C libraries! - license-check = haskellPackages.ic-ref.overrideAttrs(old: { - name = "ic-ref-license-check"; - phases = [ "unpackPhase" "setupCompilerEnvironmentPhase" "buildPhase" "installPhase" ]; - buildPhase = '' - cd $packageConfDir - ! grep -i '^license:' *.conf | grep -v 'BSD\|Apache\|MIT\|ISC' - ''; - outputs = ["out"]; # no docs - installPhase = '' - touch $out - ''; - }); - - interface-spec = nixpkgs.stdenv.mkDerivation { name = "interface-spec"; @@ -281,25 +66,7 @@ rec { all-systems-go = nixpkgs.releaseTools.aggregate { name = "all-systems-go"; constituents = [ - ic-ref - ic-ref-dist - ic-ref-test - ic-ref-coverage - universal-canister interface-spec - check-generated ]; }; - - # include shell in default.nix so that the nix cache will have pre-built versions - # of all the dependencies that are only depended on by nix-shell. - ic-ref-shell = - let extra-pkgs = [ - nixpkgs.cabal-install - nixpkgs.ghcid - ]; in - - haskellPackages.ic-ref.env.overrideAttrs (old: { - propagatedBuildInputs = (old.propagatedBuildInputs or []) ++ extra-pkgs ; - }); } diff --git a/impl/.envrc b/impl/.envrc deleted file mode 100644 index be81feddb..000000000 --- a/impl/.envrc +++ /dev/null @@ -1 +0,0 @@ -eval "$(lorri direnv)" \ No newline at end of file diff --git a/impl/.gitignore b/impl/.gitignore deleted file mode 100644 index 44f27d5c5..000000000 --- a/impl/.gitignore +++ /dev/null @@ -1,24 +0,0 @@ -dist -dist-* -cabal-dev -*.o -*.hi -*.hie -*.chi -*.chs.h -*.dyn_o -*.dyn_hi -.hpc -.hsenv -.cabal-sandbox/ -cabal.sandbox.config -*.prof -*.aux -*.hp -*.eventlog -.stack-work/ -cabal.project.local -*~ -.HTF/ -.ghc.environment.* -.tasty-rerun-log diff --git a/impl/.hlint.yaml b/impl/.hlint.yaml deleted file mode 100644 index af6f0abea..000000000 --- a/impl/.hlint.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# Warnings currently triggered by your code -- ignore: {name: "Use uncurry"} -- ignore: {name: "Avoid lambda"} -- ignore: {name: "Use camelCase"} -- ignore: {name: "Eta reduce"} -- ignore: {name: "Use >=>"} -- ignore: {name: "Use const"} -- ignore: {name: "Avoid lambda using `infix`"} diff --git a/impl/cabal.project b/impl/cabal.project deleted file mode 100644 index c9f2d7334..000000000 --- a/impl/cabal.project +++ /dev/null @@ -1,11 +0,0 @@ -packages: . - -source-repository-package - type: git - location: https://github.com/dfinity-side-projects/winter - tag: 2cc31576fe029d85c37d21fc9ea4902c5c64b5a9 - -source-repository-package - type: git - location: https://github.com/nomeata/haskell-candid - tag: 26b6aacbe04fa59b977a0e8e3038e9ddb8f5196c diff --git a/impl/cabal.project.freeze b/impl/cabal.project.freeze deleted file mode 100644 index 68ff25acc..000000000 --- a/impl/cabal.project.freeze +++ /dev/null @@ -1,168 +0,0 @@ --- Run nix-shell .. -A check-cabal-freeze to update this file -constraints: any.FloatingHex ==0.4, - any.HUnit ==1.6.0.0, - any.MonadRandom ==0.5.2, - any.QuickCheck ==2.14.2, - any.StateVar ==1.2, - any.aeson ==1.4.7.1, - any.ansi-terminal ==0.10.3, - any.ansi-wl-pprint ==0.6.9, - any.appar ==0.1.8, - any.array ==0.5.4.0, - any.asn1-encoding ==0.9.6, - any.asn1-parse ==0.9.5, - any.asn1-types ==0.3.4, - any.async ==2.2.2, - any.atomic-write ==0.2.0.7, - any.attoparsec ==0.13.2.4, - any.auto-update ==0.1.6, - any.base ==4.13.0.0, - any.base-compat ==0.11.1, - any.base-compat-batteries ==0.11.1, - any.base-orphans ==0.8.2, - any.base16-bytestring ==0.1.1.7, - any.base32 ==0.1.1.2, - any.base64-bytestring ==1.1.0.0, - any.basement ==0.0.11, - any.bifunctors ==5.5.7, - any.binary ==0.8.7.0, - any.bindings-DSL ==1.0.25, - any.blaze-builder ==0.4.1.0, - any.blaze-html ==0.9.1.2, - any.blaze-markup ==0.8.2.7, - any.bsb-http-chunked ==0.0.0.4, - any.byte-order ==0.1.2.0, - any.byteorder ==1.0.4, - any.bytestring ==0.10.10.1, - any.call-stack ==0.2.0, - any.candid ==0.1, - any.case-insensitive ==1.2.1.0, - any.cborg ==0.2.4.0, - any.cereal ==0.5.8.1, - any.clock ==0.8, - any.colour ==2.3.5, - any.comonad ==5.0.6, - any.connection ==0.3.1, - any.constraints ==0.12, - any.containers ==0.6.2.1, - any.contravariant ==1.5.2, - any.cookie ==0.4.5, - any.crc ==0.1.0.0, - any.cryptonite ==0.27, - any.data-default-class ==0.1.2.0, - any.data-fix ==0.2.1, - any.deepseq ==1.4.4.0, - any.directory ==1.3.6.0, - any.distributive ==0.6.2, - any.dlist ==0.8.0.8, - any.easy-file ==0.2.2, - any.ed25519 ==0.0.5.0, - any.exceptions ==0.10.4, - any.fast-logger ==3.0.1, - any.filepath ==1.4.2.1, - any.generic-deriving ==1.13.1, - any.generic-lens ==2.0.0.0, - any.generic-lens-core ==2.0.0.0, - any.ghc-boot-th ==8.8.4, - any.ghc-prim ==0.5.3, - any.half ==0.3, - any.hashable ==1.3.0.0, - any.hex-text ==0.1.0.0, - any.hourglass ==0.2.12, - any.http-client ==0.6.4.1, - any.http-client-tls ==0.3.5.3, - any.http-date ==0.0.8, - any.http-types ==0.12.3, - any.http2 ==2.0.5, - ic-ref -release, - any.indexed-profunctors ==0.1, - any.integer-gmp ==1.0.2.0, - any.integer-logarithms ==1.0.3, - any.iproute ==1.7.9, - any.leb128-cereal ==1.2, - any.lifted-base ==0.2.3.12, - any.megaparsec ==8.0.0, - any.memory ==0.15.0, - any.microlens ==0.4.11.2, - any.microlens-ghc ==0.4.12, - any.microlens-mtl ==0.2.0.1, - any.microlens-platform ==0.4.1, - any.microlens-th ==0.4.3.5, - any.mime-types ==0.1.0.9, - any.monad-control ==1.0.2.3, - any.mtl ==2.2.2, - any.nats ==1.1.2, - any.network ==3.1.1.1, - any.network-byte-order ==0.1.5, - any.network-uri ==2.6.3.0, - any.old-locale ==1.0.0.7, - any.old-time ==1.1.0.3, - any.optparse-applicative ==0.15.1.0, - any.parallel ==3.2.2.0, - any.parsec ==3.1.14.0, - any.parser-combinators ==1.2.1, - any.pem ==0.2.4, - any.pretty ==1.1.3.6, - any.prettyprinter ==1.6.2, - any.primitive ==0.7.0.1, - any.primitive-unaligned ==0.1.1.1, - any.process ==1.6.9.0, - any.profunctors ==5.5.2, - any.psqueues ==0.2.7.2, - any.quickcheck-io ==0.2.0, - any.random ==1.2.0, - any.resourcet ==1.2.4.2, - any.row-types ==0.4.0.0, - any.rts ==1.0, - any.scientific ==0.3.6.2, - any.semigroups ==0.19.1, - any.serialise ==0.2.3.0, - any.simple-sendfile ==0.2.30, - any.socks ==0.6.1, - any.split ==0.2.3.4, - any.splitmix ==0.1.0.3, - any.stm ==2.5.0.0, - any.streaming-commons ==0.2.2.1, - any.tagged ==0.8.6, - any.tasty ==1.2.3, - any.tasty-ant-xml ==1.1.6, - any.tasty-html ==0.4.1.2, - any.tasty-hunit ==0.10.0.2, - any.tasty-quickcheck ==0.10.1.1, - any.tasty-rerun ==1.1.17, - any.template-haskell ==2.15.0.0, - any.temporary ==1.3, - any.text ==1.2.4.0, - any.th-abstraction ==0.3.2.0, - any.time ==1.9.3, - any.time-compat ==1.9.3, - any.time-manager ==0.0.0, - any.tls ==1.5.4, - any.transformers ==0.5.6.2, - any.transformers-base ==0.4.5.2, - any.transformers-compat ==0.6.5, - any.type-equality ==1, - any.unbounded-delays ==0.1.1.0, - any.unix ==2.7.2.2, - any.unix-compat ==0.5.2, - any.unix-time ==0.4.7, - any.unliftio-core ==0.1.2.0, - any.unordered-containers ==0.2.10.0, - any.utf8-string ==1.0.1.1, - any.uuid-types ==1.0.3, - any.vault ==0.3.1.4, - any.vector ==0.12.1.2, - any.void ==0.7.3, - any.wai ==3.2.2.1, - any.wai-extra ==3.0.29.2, - any.wai-logger ==2.3.6, - any.warp ==3.3.13, - any.wcwidth ==0.0.2, - any.winter ==1.0.0, - any.word8 ==0.1.3, - any.x509 ==1.7.5, - any.x509-store ==1.6.7, - any.x509-system ==1.6.6, - any.x509-validation ==1.6.11, - any.xml ==1.3.14, - any.zlib ==0.6.2.2 diff --git a/impl/cbits/aes.c b/impl/cbits/aes.c deleted file mode 100644 index 71af57f6c..000000000 --- a/impl/cbits/aes.c +++ /dev/null @@ -1,801 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - - -/* - * Implementation of the NIST Advanced Ecryption Standard - * - * SU=m, SU is Stack Usage - */ - -#include -#include "arch.h" -#include "core.h" - -/* this is fixed */ -#define NB 4 - -/* Rotates 32-bit word left by 1, 2 or 3 byte */ - -#define ROTL8(x) (((x)<<8)|((x)>>24)) -#define ROTL16(x) (((x)<<16)|((x)>>16)) -#define ROTL24(x) (((x)<<24)|((x)>>8)) - -static const uchar InCo[4] = {0xB, 0xD, 0x9, 0xE}; /* Inverse Coefficients */ - -static const uchar ptab[] = -{ - 1, 3, 5, 15, 17, 51, 85, 255, 26, 46, 114, 150, 161, 248, 19, 53, - 95, 225, 56, 72, 216, 115, 149, 164, 247, 2, 6, 10, 30, 34, 102, 170, - 229, 52, 92, 228, 55, 89, 235, 38, 106, 190, 217, 112, 144, 171, 230, 49, - 83, 245, 4, 12, 20, 60, 68, 204, 79, 209, 104, 184, 211, 110, 178, 205, - 76, 212, 103, 169, 224, 59, 77, 215, 98, 166, 241, 8, 24, 40, 120, 136, - 131, 158, 185, 208, 107, 189, 220, 127, 129, 152, 179, 206, 73, 219, 118, 154, - 181, 196, 87, 249, 16, 48, 80, 240, 11, 29, 39, 105, 187, 214, 97, 163, - 254, 25, 43, 125, 135, 146, 173, 236, 47, 113, 147, 174, 233, 32, 96, 160, - 251, 22, 58, 78, 210, 109, 183, 194, 93, 231, 50, 86, 250, 21, 63, 65, - 195, 94, 226, 61, 71, 201, 64, 192, 91, 237, 44, 116, 156, 191, 218, 117, - 159, 186, 213, 100, 172, 239, 42, 126, 130, 157, 188, 223, 122, 142, 137, 128, - 155, 182, 193, 88, 232, 35, 101, 175, 234, 37, 111, 177, 200, 67, 197, 84, - 252, 31, 33, 99, 165, 244, 7, 9, 27, 45, 119, 153, 176, 203, 70, 202, - 69, 207, 74, 222, 121, 139, 134, 145, 168, 227, 62, 66, 198, 81, 243, 14, - 18, 54, 90, 238, 41, 123, 141, 140, 143, 138, 133, 148, 167, 242, 13, 23, - 57, 75, 221, 124, 132, 151, 162, 253, 28, 36, 108, 180, 199, 82, 246, 1 -}; - -static const uchar ltab[] = -{ - 0, 255, 25, 1, 50, 2, 26, 198, 75, 199, 27, 104, 51, 238, 223, 3, - 100, 4, 224, 14, 52, 141, 129, 239, 76, 113, 8, 200, 248, 105, 28, 193, - 125, 194, 29, 181, 249, 185, 39, 106, 77, 228, 166, 114, 154, 201, 9, 120, - 101, 47, 138, 5, 33, 15, 225, 36, 18, 240, 130, 69, 53, 147, 218, 142, - 150, 143, 219, 189, 54, 208, 206, 148, 19, 92, 210, 241, 64, 70, 131, 56, - 102, 221, 253, 48, 191, 6, 139, 98, 179, 37, 226, 152, 34, 136, 145, 16, - 126, 110, 72, 195, 163, 182, 30, 66, 58, 107, 40, 84, 250, 133, 61, 186, - 43, 121, 10, 21, 155, 159, 94, 202, 78, 212, 172, 229, 243, 115, 167, 87, - 175, 88, 168, 80, 244, 234, 214, 116, 79, 174, 233, 213, 231, 230, 173, 232, - 44, 215, 117, 122, 235, 22, 11, 245, 89, 203, 95, 176, 156, 169, 81, 160, - 127, 12, 246, 111, 23, 196, 73, 236, 216, 67, 31, 45, 164, 118, 123, 183, - 204, 187, 62, 90, 251, 96, 177, 134, 59, 82, 161, 108, 170, 85, 41, 157, - 151, 178, 135, 144, 97, 190, 220, 252, 188, 149, 207, 205, 55, 63, 91, 209, - 83, 57, 132, 60, 65, 162, 109, 71, 20, 42, 158, 93, 86, 242, 211, 171, - 68, 17, 146, 217, 35, 32, 46, 137, 180, 124, 184, 38, 119, 153, 227, 165, - 103, 74, 237, 222, 197, 49, 254, 24, 13, 99, 140, 128, 192, 247, 112, 7 -}; - -static const uchar fbsub[] = -{ - 99, 124, 119, 123, 242, 107, 111, 197, 48, 1, 103, 43, 254, 215, 171, 118, - 202, 130, 201, 125, 250, 89, 71, 240, 173, 212, 162, 175, 156, 164, 114, 192, - 183, 253, 147, 38, 54, 63, 247, 204, 52, 165, 229, 241, 113, 216, 49, 21, - 4, 199, 35, 195, 24, 150, 5, 154, 7, 18, 128, 226, 235, 39, 178, 117, - 9, 131, 44, 26, 27, 110, 90, 160, 82, 59, 214, 179, 41, 227, 47, 132, - 83, 209, 0, 237, 32, 252, 177, 91, 106, 203, 190, 57, 74, 76, 88, 207, - 208, 239, 170, 251, 67, 77, 51, 133, 69, 249, 2, 127, 80, 60, 159, 168, - 81, 163, 64, 143, 146, 157, 56, 245, 188, 182, 218, 33, 16, 255, 243, 210, - 205, 12, 19, 236, 95, 151, 68, 23, 196, 167, 126, 61, 100, 93, 25, 115, - 96, 129, 79, 220, 34, 42, 144, 136, 70, 238, 184, 20, 222, 94, 11, 219, - 224, 50, 58, 10, 73, 6, 36, 92, 194, 211, 172, 98, 145, 149, 228, 121, - 231, 200, 55, 109, 141, 213, 78, 169, 108, 86, 244, 234, 101, 122, 174, 8, - 186, 120, 37, 46, 28, 166, 180, 198, 232, 221, 116, 31, 75, 189, 139, 138, - 112, 62, 181, 102, 72, 3, 246, 14, 97, 53, 87, 185, 134, 193, 29, 158, - 225, 248, 152, 17, 105, 217, 142, 148, 155, 30, 135, 233, 206, 85, 40, 223, - 140, 161, 137, 13, 191, 230, 66, 104, 65, 153, 45, 15, 176, 84, 187, 22 -}; - -static const uchar rbsub[] = -{ - 82, 9, 106, 213, 48, 54, 165, 56, 191, 64, 163, 158, 129, 243, 215, 251, - 124, 227, 57, 130, 155, 47, 255, 135, 52, 142, 67, 68, 196, 222, 233, 203, - 84, 123, 148, 50, 166, 194, 35, 61, 238, 76, 149, 11, 66, 250, 195, 78, - 8, 46, 161, 102, 40, 217, 36, 178, 118, 91, 162, 73, 109, 139, 209, 37, - 114, 248, 246, 100, 134, 104, 152, 22, 212, 164, 92, 204, 93, 101, 182, 146, - 108, 112, 72, 80, 253, 237, 185, 218, 94, 21, 70, 87, 167, 141, 157, 132, - 144, 216, 171, 0, 140, 188, 211, 10, 247, 228, 88, 5, 184, 179, 69, 6, - 208, 44, 30, 143, 202, 63, 15, 2, 193, 175, 189, 3, 1, 19, 138, 107, - 58, 145, 17, 65, 79, 103, 220, 234, 151, 242, 207, 206, 240, 180, 230, 115, - 150, 172, 116, 34, 231, 173, 53, 133, 226, 249, 55, 232, 28, 117, 223, 110, - 71, 241, 26, 113, 29, 41, 197, 137, 111, 183, 98, 14, 170, 24, 190, 27, - 252, 86, 62, 75, 198, 210, 121, 32, 154, 219, 192, 254, 120, 205, 90, 244, - 31, 221, 168, 51, 136, 7, 199, 49, 177, 18, 16, 89, 39, 128, 236, 95, - 96, 81, 127, 169, 25, 181, 74, 13, 45, 229, 122, 159, 147, 201, 156, 239, - 160, 224, 59, 77, 174, 42, 245, 176, 200, 235, 187, 60, 131, 83, 153, 97, - 23, 43, 4, 126, 186, 119, 214, 38, 225, 105, 20, 99, 85, 33, 12, 125 -}; - -static const unsign32 rco[] = -{1, 2, 4, 8, 16, 32, 64, 128, 27, 54, 108, 216, 171, 77, 154, 47}; - -static const unsign32 ftable[] = -{ - 0xa56363c6, 0x847c7cf8, 0x997777ee, 0x8d7b7bf6, 0xdf2f2ff, 0xbd6b6bd6, - 0xb16f6fde, 0x54c5c591, 0x50303060, 0x3010102, 0xa96767ce, 0x7d2b2b56, - 0x19fefee7, 0x62d7d7b5, 0xe6abab4d, 0x9a7676ec, 0x45caca8f, 0x9d82821f, - 0x40c9c989, 0x877d7dfa, 0x15fafaef, 0xeb5959b2, 0xc947478e, 0xbf0f0fb, - 0xecadad41, 0x67d4d4b3, 0xfda2a25f, 0xeaafaf45, 0xbf9c9c23, 0xf7a4a453, - 0x967272e4, 0x5bc0c09b, 0xc2b7b775, 0x1cfdfde1, 0xae93933d, 0x6a26264c, - 0x5a36366c, 0x413f3f7e, 0x2f7f7f5, 0x4fcccc83, 0x5c343468, 0xf4a5a551, - 0x34e5e5d1, 0x8f1f1f9, 0x937171e2, 0x73d8d8ab, 0x53313162, 0x3f15152a, - 0xc040408, 0x52c7c795, 0x65232346, 0x5ec3c39d, 0x28181830, 0xa1969637, - 0xf05050a, 0xb59a9a2f, 0x907070e, 0x36121224, 0x9b80801b, 0x3de2e2df, - 0x26ebebcd, 0x6927274e, 0xcdb2b27f, 0x9f7575ea, 0x1b090912, 0x9e83831d, - 0x742c2c58, 0x2e1a1a34, 0x2d1b1b36, 0xb26e6edc, 0xee5a5ab4, 0xfba0a05b, - 0xf65252a4, 0x4d3b3b76, 0x61d6d6b7, 0xceb3b37d, 0x7b292952, 0x3ee3e3dd, - 0x712f2f5e, 0x97848413, 0xf55353a6, 0x68d1d1b9, 0x0, 0x2cededc1, - 0x60202040, 0x1ffcfce3, 0xc8b1b179, 0xed5b5bb6, 0xbe6a6ad4, 0x46cbcb8d, - 0xd9bebe67, 0x4b393972, 0xde4a4a94, 0xd44c4c98, 0xe85858b0, 0x4acfcf85, - 0x6bd0d0bb, 0x2aefefc5, 0xe5aaaa4f, 0x16fbfbed, 0xc5434386, 0xd74d4d9a, - 0x55333366, 0x94858511, 0xcf45458a, 0x10f9f9e9, 0x6020204, 0x817f7ffe, - 0xf05050a0, 0x443c3c78, 0xba9f9f25, 0xe3a8a84b, 0xf35151a2, 0xfea3a35d, - 0xc0404080, 0x8a8f8f05, 0xad92923f, 0xbc9d9d21, 0x48383870, 0x4f5f5f1, - 0xdfbcbc63, 0xc1b6b677, 0x75dadaaf, 0x63212142, 0x30101020, 0x1affffe5, - 0xef3f3fd, 0x6dd2d2bf, 0x4ccdcd81, 0x140c0c18, 0x35131326, 0x2fececc3, - 0xe15f5fbe, 0xa2979735, 0xcc444488, 0x3917172e, 0x57c4c493, 0xf2a7a755, - 0x827e7efc, 0x473d3d7a, 0xac6464c8, 0xe75d5dba, 0x2b191932, 0x957373e6, - 0xa06060c0, 0x98818119, 0xd14f4f9e, 0x7fdcdca3, 0x66222244, 0x7e2a2a54, - 0xab90903b, 0x8388880b, 0xca46468c, 0x29eeeec7, 0xd3b8b86b, 0x3c141428, - 0x79dedea7, 0xe25e5ebc, 0x1d0b0b16, 0x76dbdbad, 0x3be0e0db, 0x56323264, - 0x4e3a3a74, 0x1e0a0a14, 0xdb494992, 0xa06060c, 0x6c242448, 0xe45c5cb8, - 0x5dc2c29f, 0x6ed3d3bd, 0xefacac43, 0xa66262c4, 0xa8919139, 0xa4959531, - 0x37e4e4d3, 0x8b7979f2, 0x32e7e7d5, 0x43c8c88b, 0x5937376e, 0xb76d6dda, - 0x8c8d8d01, 0x64d5d5b1, 0xd24e4e9c, 0xe0a9a949, 0xb46c6cd8, 0xfa5656ac, - 0x7f4f4f3, 0x25eaeacf, 0xaf6565ca, 0x8e7a7af4, 0xe9aeae47, 0x18080810, - 0xd5baba6f, 0x887878f0, 0x6f25254a, 0x722e2e5c, 0x241c1c38, 0xf1a6a657, - 0xc7b4b473, 0x51c6c697, 0x23e8e8cb, 0x7cdddda1, 0x9c7474e8, 0x211f1f3e, - 0xdd4b4b96, 0xdcbdbd61, 0x868b8b0d, 0x858a8a0f, 0x907070e0, 0x423e3e7c, - 0xc4b5b571, 0xaa6666cc, 0xd8484890, 0x5030306, 0x1f6f6f7, 0x120e0e1c, - 0xa36161c2, 0x5f35356a, 0xf95757ae, 0xd0b9b969, 0x91868617, 0x58c1c199, - 0x271d1d3a, 0xb99e9e27, 0x38e1e1d9, 0x13f8f8eb, 0xb398982b, 0x33111122, - 0xbb6969d2, 0x70d9d9a9, 0x898e8e07, 0xa7949433, 0xb69b9b2d, 0x221e1e3c, - 0x92878715, 0x20e9e9c9, 0x49cece87, 0xff5555aa, 0x78282850, 0x7adfdfa5, - 0x8f8c8c03, 0xf8a1a159, 0x80898909, 0x170d0d1a, 0xdabfbf65, 0x31e6e6d7, - 0xc6424284, 0xb86868d0, 0xc3414182, 0xb0999929, 0x772d2d5a, 0x110f0f1e, - 0xcbb0b07b, 0xfc5454a8, 0xd6bbbb6d, 0x3a16162c -}; - -static const unsign32 rtable[] = -{ - 0x50a7f451, 0x5365417e, 0xc3a4171a, 0x965e273a, 0xcb6bab3b, 0xf1459d1f, - 0xab58faac, 0x9303e34b, 0x55fa3020, 0xf66d76ad, 0x9176cc88, 0x254c02f5, - 0xfcd7e54f, 0xd7cb2ac5, 0x80443526, 0x8fa362b5, 0x495ab1de, 0x671bba25, - 0x980eea45, 0xe1c0fe5d, 0x2752fc3, 0x12f04c81, 0xa397468d, 0xc6f9d36b, - 0xe75f8f03, 0x959c9215, 0xeb7a6dbf, 0xda595295, 0x2d83bed4, 0xd3217458, - 0x2969e049, 0x44c8c98e, 0x6a89c275, 0x78798ef4, 0x6b3e5899, 0xdd71b927, - 0xb64fe1be, 0x17ad88f0, 0x66ac20c9, 0xb43ace7d, 0x184adf63, 0x82311ae5, - 0x60335197, 0x457f5362, 0xe07764b1, 0x84ae6bbb, 0x1ca081fe, 0x942b08f9, - 0x58684870, 0x19fd458f, 0x876cde94, 0xb7f87b52, 0x23d373ab, 0xe2024b72, - 0x578f1fe3, 0x2aab5566, 0x728ebb2, 0x3c2b52f, 0x9a7bc586, 0xa50837d3, - 0xf2872830, 0xb2a5bf23, 0xba6a0302, 0x5c8216ed, 0x2b1ccf8a, 0x92b479a7, - 0xf0f207f3, 0xa1e2694e, 0xcdf4da65, 0xd5be0506, 0x1f6234d1, 0x8afea6c4, - 0x9d532e34, 0xa055f3a2, 0x32e18a05, 0x75ebf6a4, 0x39ec830b, 0xaaef6040, - 0x69f715e, 0x51106ebd, 0xf98a213e, 0x3d06dd96, 0xae053edd, 0x46bde64d, - 0xb58d5491, 0x55dc471, 0x6fd40604, 0xff155060, 0x24fb9819, 0x97e9bdd6, - 0xcc434089, 0x779ed967, 0xbd42e8b0, 0x888b8907, 0x385b19e7, 0xdbeec879, - 0x470a7ca1, 0xe90f427c, 0xc91e84f8, 0x0, 0x83868009, 0x48ed2b32, - 0xac70111e, 0x4e725a6c, 0xfbff0efd, 0x5638850f, 0x1ed5ae3d, 0x27392d36, - 0x64d90f0a, 0x21a65c68, 0xd1545b9b, 0x3a2e3624, 0xb1670a0c, 0xfe75793, - 0xd296eeb4, 0x9e919b1b, 0x4fc5c080, 0xa220dc61, 0x694b775a, 0x161a121c, - 0xaba93e2, 0xe52aa0c0, 0x43e0223c, 0x1d171b12, 0xb0d090e, 0xadc78bf2, - 0xb9a8b62d, 0xc8a91e14, 0x8519f157, 0x4c0775af, 0xbbdd99ee, 0xfd607fa3, - 0x9f2601f7, 0xbcf5725c, 0xc53b6644, 0x347efb5b, 0x7629438b, 0xdcc623cb, - 0x68fcedb6, 0x63f1e4b8, 0xcadc31d7, 0x10856342, 0x40229713, 0x2011c684, - 0x7d244a85, 0xf83dbbd2, 0x1132f9ae, 0x6da129c7, 0x4b2f9e1d, 0xf330b2dc, - 0xec52860d, 0xd0e3c177, 0x6c16b32b, 0x99b970a9, 0xfa489411, 0x2264e947, - 0xc48cfca8, 0x1a3ff0a0, 0xd82c7d56, 0xef903322, 0xc74e4987, 0xc1d138d9, - 0xfea2ca8c, 0x360bd498, 0xcf81f5a6, 0x28de7aa5, 0x268eb7da, 0xa4bfad3f, - 0xe49d3a2c, 0xd927850, 0x9bcc5f6a, 0x62467e54, 0xc2138df6, 0xe8b8d890, - 0x5ef7392e, 0xf5afc382, 0xbe805d9f, 0x7c93d069, 0xa92dd56f, 0xb31225cf, - 0x3b99acc8, 0xa77d1810, 0x6e639ce8, 0x7bbb3bdb, 0x97826cd, 0xf418596e, - 0x1b79aec, 0xa89a4f83, 0x656e95e6, 0x7ee6ffaa, 0x8cfbc21, 0xe6e815ef, - 0xd99be7ba, 0xce366f4a, 0xd4099fea, 0xd67cb029, 0xafb2a431, 0x31233f2a, - 0x3094a5c6, 0xc066a235, 0x37bc4e74, 0xa6ca82fc, 0xb0d090e0, 0x15d8a733, - 0x4a9804f1, 0xf7daec41, 0xe50cd7f, 0x2ff69117, 0x8dd64d76, 0x4db0ef43, - 0x544daacc, 0xdf0496e4, 0xe3b5d19e, 0x1b886a4c, 0xb81f2cc1, 0x7f516546, - 0x4ea5e9d, 0x5d358c01, 0x737487fa, 0x2e410bfb, 0x5a1d67b3, 0x52d2db92, - 0x335610e9, 0x1347d66d, 0x8c61d79a, 0x7a0ca137, 0x8e14f859, 0x893c13eb, - 0xee27a9ce, 0x35c961b7, 0xede51ce1, 0x3cb1477a, 0x59dfd29c, 0x3f73f255, - 0x79ce1418, 0xbf37c773, 0xeacdf753, 0x5baafd5f, 0x146f3ddf, 0x86db4478, - 0x81f3afca, 0x3ec468b9, 0x2c342438, 0x5f40a3c2, 0x72c31d16, 0xc25e2bc, - 0x8b493c28, 0x41950dff, 0x7101a839, 0xdeb30c08, 0x9ce4b4d8, 0x90c15664, - 0x6184cb7b, 0x70b632d5, 0x745c6c48, 0x4257b8d0 -}; - -#define MR_TOBYTE(x) ((uchar)((x))) - -static unsign32 pack(const uchar *b) -{ - /* pack bytes into a 32-bit Word */ - return ((unsign32)b[3] << 24) | ((unsign32)b[2] << 16) | ((unsign32)b[1] << 8) | (unsign32)b[0]; -} - -static void unpack(unsign32 a, uchar *b) -{ - /* unpack bytes from a word */ - b[0] = MR_TOBYTE(a); - b[1] = MR_TOBYTE(a >> 8); - b[2] = MR_TOBYTE(a >> 16); - b[3] = MR_TOBYTE(a >> 24); -} - -/* SU= 8 */ -static uchar bmul(uchar x, uchar y) -{ - /* x.y= AntiLog(Log(x) + Log(y)) */ - if (x && y) return ptab[(ltab[x] + ltab[y]) % 255]; - else return 0; -} - -static unsign32 SubByte(unsign32 a) -{ - uchar b[4]; - unpack(a, b); - b[0] = fbsub[b[0]]; - b[1] = fbsub[b[1]]; - b[2] = fbsub[b[2]]; - b[3] = fbsub[b[3]]; - return pack(b); -} - -/* SU= 16 */ -static uchar product(unsign32 x, unsign32 y) -{ - /* dot product of two 4-byte arrays */ - uchar xb[4], yb[4]; - unpack(x, xb); - unpack(y, yb); - return bmul(xb[0], yb[0])^bmul(xb[1], yb[1])^bmul(xb[2], yb[2])^bmul(xb[3], yb[3]); -} - -static unsign32 InvMixCol(unsign32 x) -{ - /* matrix Multiplication */ - unsign32 y, m; - uchar b[4]; - - m = pack(InCo); - b[3] = product(m, x); - m = ROTL24(m); - b[2] = product(m, x); - m = ROTL24(m); - b[1] = product(m, x); - m = ROTL24(m); - b[0] = product(m, x); - y = pack(b); - return y; -} - -/* SU= 8 */ -/* reset cipher */ -void AES_reset(core_aes *a, int mode, char *iv) -{ - /* reset mode, or reset iv */ - int i; - a->mode = mode; - for (i = 0; i < 4 * NB; i++) - a->f[i] = 0; - if (mode != ECB && iv != NULL) - { - for (i = 0; i < 4 * NB; i++) - a->f[i] = iv[i]; - } -} - -void AES_getreg(core_aes *a, char *ir) -{ - int i; - for (i = 0; i < 4 * NB; i++) ir[i] = a->f[i]; -} - -/* SU= 72 */ -/* Initialise cipher */ -int AES_init(core_aes* a, int mode, int nk, char *key, char *iv) -{ - /* Key length Nk=16, 24 or 32 bytes */ - /* Key Scheduler. Create expanded encryption key */ - int i, j, k, N, nr; - unsign32 CipherKey[8]; - - nk /= 4; - - if (nk != 4 && nk != 6 && nk != 8) return 0; - - nr = 6 + nk; - - a->Nk = nk; - a->Nr = nr; - - AES_reset(a, mode, iv); - - N = NB * (nr + 1); - - for (i = j = 0; i < nk; i++, j += 4) - { - CipherKey[i] = pack((uchar *)&key[j]); - } - for (i = 0; i < nk; i++) a->fkey[i] = CipherKey[i]; - - for (j = nk, k = 0; j < N; j += nk, k++) - { - a->fkey[j] = a->fkey[j - nk] ^ SubByte(ROTL24(a->fkey[j - 1]))^rco[k]; - if (nk <= 6) - { - for (i = 1; i < nk && (i + j) < N; i++) - a->fkey[i + j] = a->fkey[i + j - nk] ^ a->fkey[i + j - 1]; - } - else - { - for (i = 1; i < 4 && (i + j) < N; i++) - a->fkey[i + j] = a->fkey[i + j - nk] ^ a->fkey[i + j - 1]; - if ((j + 4) < N) a->fkey[j + 4] = a->fkey[j + 4 - nk] ^ SubByte(a->fkey[j + 3]); - for (i = 5; i < nk && (i + j) < N; i++) - a->fkey[i + j] = a->fkey[i + j - nk] ^ a->fkey[i + j - 1]; - } - - } - /* now for the expanded decrypt key in reverse order */ - - for (j = 0; j < NB; j++) a->rkey[j + N - NB] = a->fkey[j]; - for (i = NB; i < N - NB; i += NB) - { - k = N - NB - i; - for (j = 0; j < NB; j++) a->rkey[k + j] = InvMixCol(a->fkey[i + j]); - } - for (j = N - NB; j < N; j++) a->rkey[j - N + NB] = a->fkey[j]; - return 1; -} - -/* SU= 80 */ -/* Encrypt a single block */ -void AES_ecb_encrypt(core_aes *a, uchar *buff) -{ - int i, j, k; - unsign32 p[4], q[4], *x, *y, *t; - - for (i = j = 0; i < NB; i++, j += 4) - { - p[i] = pack((uchar *)&buff[j]); - p[i] ^= a->fkey[i]; - } - - k = NB; - x = p; - y = q; - - /* State alternates between x and y */ - for (i = 1; i < a->Nr; i++) - { - - y[0] = a->fkey[k] ^ ftable[MR_TOBYTE(x[0])] ^ - ROTL8(ftable[MR_TOBYTE(x[1] >> 8)])^ - ROTL16(ftable[MR_TOBYTE(x[2] >> 16)])^ - ROTL24(ftable[x[3] >> 24]); - y[1] = a->fkey[k + 1] ^ ftable[MR_TOBYTE(x[1])] ^ - ROTL8(ftable[MR_TOBYTE(x[2] >> 8)])^ - ROTL16(ftable[MR_TOBYTE(x[3] >> 16)])^ - ROTL24(ftable[x[0] >> 24]); - y[2] = a->fkey[k + 2] ^ ftable[MR_TOBYTE(x[2])] ^ - ROTL8(ftable[MR_TOBYTE(x[3] >> 8)])^ - ROTL16(ftable[MR_TOBYTE(x[0] >> 16)])^ - ROTL24(ftable[x[1] >> 24]); - y[3] = a->fkey[k + 3] ^ ftable[MR_TOBYTE(x[3])] ^ - ROTL8(ftable[MR_TOBYTE(x[0] >> 8)])^ - ROTL16(ftable[MR_TOBYTE(x[1] >> 16)])^ - ROTL24(ftable[x[2] >> 24]); - - k += 4; - t = x; - x = y; - y = t; /* swap pointers */ - } - - /* Last Round */ - - y[0] = a->fkey[k] ^ (unsign32)fbsub[MR_TOBYTE(x[0])] ^ - ROTL8((unsign32)fbsub[MR_TOBYTE(x[1] >> 8)])^ - ROTL16((unsign32)fbsub[MR_TOBYTE(x[2] >> 16)])^ - ROTL24((unsign32)fbsub[x[3] >> 24]); - y[1] = a->fkey[k + 1] ^ (unsign32)fbsub[MR_TOBYTE(x[1])] ^ - ROTL8((unsign32)fbsub[MR_TOBYTE(x[2] >> 8)])^ - ROTL16((unsign32)fbsub[MR_TOBYTE(x[3] >> 16)])^ - ROTL24((unsign32)fbsub[x[0] >> 24]); - y[2] = a->fkey[k + 2] ^ (unsign32)fbsub[MR_TOBYTE(x[2])] ^ - ROTL8((unsign32)fbsub[MR_TOBYTE(x[3] >> 8)])^ - ROTL16((unsign32)fbsub[MR_TOBYTE(x[0] >> 16)])^ - ROTL24((unsign32)fbsub[x[1] >> 24]); - y[3] = a->fkey[k + 3] ^ (unsign32)fbsub[MR_TOBYTE(x[3])] ^ - ROTL8((unsign32)fbsub[MR_TOBYTE(x[0] >> 8)])^ - ROTL16((unsign32)fbsub[MR_TOBYTE(x[1] >> 16)])^ - ROTL24((unsign32)fbsub[x[2] >> 24]); - - for (i = j = 0; i < NB; i++, j += 4) - { - unpack(y[i], (uchar *)&buff[j]); - x[i] = y[i] = 0; /* clean up stack */ - } -} - -/* SU= 80 */ -/* Decrypt a single block */ -void AES_ecb_decrypt(core_aes *a, uchar *buff) -{ - int i, j, k; - unsign32 p[4], q[4], *x, *y, *t; - - for (i = j = 0; i < NB; i++, j += 4) - { - p[i] = pack((uchar *)&buff[j]); - p[i] ^= a->rkey[i]; - } - - k = NB; - x = p; - y = q; - - /* State alternates between x and y */ - for (i = 1; i < a->Nr; i++) - { - /* Nr is number of rounds. May be odd. */ - - y[0] = a->rkey[k] ^ rtable[MR_TOBYTE(x[0])] ^ - ROTL8(rtable[MR_TOBYTE(x[3] >> 8)])^ - ROTL16(rtable[MR_TOBYTE(x[2] >> 16)])^ - ROTL24(rtable[x[1] >> 24]); - y[1] = a->rkey[k + 1] ^ rtable[MR_TOBYTE(x[1])] ^ - ROTL8(rtable[MR_TOBYTE(x[0] >> 8)])^ - ROTL16(rtable[MR_TOBYTE(x[3] >> 16)])^ - ROTL24(rtable[x[2] >> 24]); - y[2] = a->rkey[k + 2] ^ rtable[MR_TOBYTE(x[2])] ^ - ROTL8(rtable[MR_TOBYTE(x[1] >> 8)])^ - ROTL16(rtable[MR_TOBYTE(x[0] >> 16)])^ - ROTL24(rtable[x[3] >> 24]); - y[3] = a->rkey[k + 3] ^ rtable[MR_TOBYTE(x[3])] ^ - ROTL8(rtable[MR_TOBYTE(x[2] >> 8)])^ - ROTL16(rtable[MR_TOBYTE(x[1] >> 16)])^ - ROTL24(rtable[x[0] >> 24]); - - k += 4; - t = x; - x = y; - y = t; /* swap pointers */ - } - - - /* Last Round */ - y[0] = a->rkey[k] ^ (unsign32)rbsub[MR_TOBYTE(x[0])] ^ - ROTL8((unsign32)rbsub[MR_TOBYTE(x[3] >> 8)])^ - ROTL16((unsign32)rbsub[MR_TOBYTE(x[2] >> 16)])^ - ROTL24((unsign32)rbsub[x[1] >> 24]); - y[1] = a->rkey[k + 1] ^ (unsign32)rbsub[MR_TOBYTE(x[1])] ^ - ROTL8((unsign32)rbsub[MR_TOBYTE(x[0] >> 8)])^ - ROTL16((unsign32)rbsub[MR_TOBYTE(x[3] >> 16)])^ - ROTL24((unsign32)rbsub[x[2] >> 24]); - y[2] = a->rkey[k + 2] ^ (unsign32)rbsub[MR_TOBYTE(x[2])] ^ - ROTL8((unsign32)rbsub[MR_TOBYTE(x[1] >> 8)])^ - ROTL16((unsign32)rbsub[MR_TOBYTE(x[0] >> 16)])^ - ROTL24((unsign32)rbsub[x[3] >> 24]); - y[3] = a->rkey[k + 3] ^ (unsign32)rbsub[MR_TOBYTE(x[3])] ^ - ROTL8((unsign32)rbsub[MR_TOBYTE(x[2] >> 8)])^ - ROTL16((unsign32)rbsub[MR_TOBYTE(x[1] >> 16)])^ - ROTL24((unsign32)rbsub[x[0] >> 24]); - - for (i = j = 0; i < NB; i++, j += 4) - { - unpack(y[i], (uchar *)&buff[j]); - x[i] = y[i] = 0; /* clean up stack */ - } - -} - -/* simple default increment function */ -static void increment(char *f) -{ - int i; - for (i = 0; i < 16; i++) - { - f[i]++; - if (f[i] != 0) break; - } -} - -/* SU= 40 */ -/* Encrypt using selected mode of operation */ -unsign32 AES_encrypt(core_aes* a, char *buff) -{ - int j, bytes; - char st[16]; - unsign32 fell_off; - - /* Supported Modes of Operation */ - - fell_off = 0; - switch (a->mode) - { - case ECB: - AES_ecb_encrypt(a, (uchar *)buff); - return 0; - case CBC: - for (j = 0; j < 4 * NB; j++) buff[j] ^= a->f[j]; - AES_ecb_encrypt(a, (uchar *)buff); - for (j = 0; j < 4 * NB; j++) a->f[j] = buff[j]; - return 0; - - case CFB1: - case CFB2: - case CFB4: - bytes = a->mode - CFB1 + 1; - for (j = 0; j < bytes; j++) fell_off = (fell_off << 8) | a->f[j]; - for (j = 0; j < 4 * NB; j++) st[j] = a->f[j]; - for (j = bytes; j < 4 * NB; j++) a->f[j - bytes] = a->f[j]; - AES_ecb_encrypt(a, (uchar *)st); - for (j = 0; j < bytes; j++) - { - buff[j] ^= st[j]; - a->f[16 - bytes + j] = buff[j]; - } - return fell_off; - - case OFB1: - case OFB2: - case OFB4: - case OFB8: - case OFB16: - - bytes = a->mode - OFB1 + 1; - AES_ecb_encrypt(a, (uchar *)(a->f)); - for (j = 0; j < bytes; j++) buff[j] ^= a->f[j]; - return 0; - - case CTR1: - case CTR2: - case CTR4: - case CTR8: - case CTR16: - - bytes = a->mode - CTR1 + 1; - for (j = 0; j < 4 * NB; j++) st[j] = a->f[j]; - AES_ecb_encrypt(a, (uchar *)st); - for (j = 0; j < bytes; j++) buff[j] ^= st[j]; - increment(a->f); - return 0; - - default: - return 0; - } -} - -/* SU= 40 */ -/* Decrypt using selected mode of operation */ -unsign32 AES_decrypt(core_aes *a, char *buff) -{ - int j, bytes; - char st[16]; - unsign32 fell_off; - - /* Supported modes of operation */ - fell_off = 0; - switch (a->mode) - { - case ECB: - AES_ecb_decrypt(a, (uchar *)buff); - return 0; - case CBC: - for (j = 0; j < 4 * NB; j++) - { - st[j] = a->f[j]; - a->f[j] = buff[j]; - } - AES_ecb_decrypt(a, (uchar *)buff); - for (j = 0; j < 4 * NB; j++) - { - buff[j] ^= st[j]; - st[j] = 0; - } - return 0; - case CFB1: - case CFB2: - case CFB4: - bytes = a->mode - CFB1 + 1; - for (j = 0; j < bytes; j++) fell_off = (fell_off << 8) | a->f[j]; - for (j = 0; j < 4 * NB; j++) st[j] = a->f[j]; - for (j = bytes; j < 4 * NB; j++) a->f[j - bytes] = a->f[j]; - AES_ecb_encrypt(a, (uchar *)st); - for (j = 0; j < bytes; j++) - { - a->f[16 - bytes + j] = buff[j]; - buff[j] ^= st[j]; - } - return fell_off; - case OFB1: - case OFB2: - case OFB4: - case OFB8: - case OFB16: - bytes = a->mode - OFB1 + 1; - AES_ecb_encrypt(a, (uchar *)(a->f)); - for (j = 0; j < bytes; j++) buff[j] ^= a->f[j]; - return 0; - - case CTR1: - case CTR2: - case CTR4: - case CTR8: - case CTR16: - - bytes = a->mode - CTR1 + 1; - for (j = 0; j < 4 * NB; j++) st[j] = a->f[j]; - AES_ecb_encrypt(a, (uchar *)st); - for (j = 0; j < bytes; j++) buff[j] ^= st[j]; - increment(a->f); - return 0; - - default: - return 0; - } -} - -/* Clean up and delete left-overs */ -void AES_end(core_aes *a) -{ - /* clean up */ - int i; - for (i = 0; i < NB * (a->Nr + 1); i++) - a->fkey[i] = a->rkey[i] = 0; - for (i = 0; i < 4 * NB; i++) - a->f[i] = 0; -} - -/* AES encryption/decryption. Encrypt byte array M using key K and returns ciphertext */ -void AES_CBC_IV0_ENCRYPT(octet *k, octet *m, octet *c) -{ - /* AES CBC encryption, with Null IV and key k */ - /* Input is from an octet string m, output is to an octet string c */ - /* Input is padded as necessary to make up a full final block */ - core_aes a; - int fin; - int i, j, ipt, opt; - char buff[16]; - int padlen; - - OCT_clear(c); - if (m->len == 0) return; - AES_init(&a, CBC, k->len, k->val, NULL); - - ipt = opt = 0; - fin = 0; - for (;;) - { - for (i = 0; i < 16; i++) - { - if (ipt < m->len) buff[i] = m->val[ipt++]; - else - { - fin = 1; - break; - } - } - if (fin) break; - AES_encrypt(&a, buff); - for (i = 0; i < 16; i++) - if (opt < c->max) c->val[opt++] = buff[i]; - } - - /* last block, filled up to i-th index */ - - padlen = 16 - i; - for (j = i; j < 16; j++) buff[j] = padlen; - AES_encrypt(&a, buff); - for (i = 0; i < 16; i++) - if (opt < c->max) c->val[opt++] = buff[i]; - AES_end(&a); - c->len = opt; -} - -/* decrypts and returns TRUE if all consistent, else returns FALSE */ -int AES_CBC_IV0_DECRYPT(octet *k, octet *c, octet *m) -{ - /* padding is removed */ - core_aes a; - int i, ipt, opt, ch; - char buff[16]; - int fin, bad; - int padlen; - ipt = opt = 0; - - OCT_clear(m); - if (c->len == 0) return 1; - ch = c->val[ipt++]; - - AES_init(&a, CBC, k->len, k->val, NULL); - fin = 0; - - for (;;) - { - for (i = 0; i < 16; i++) - { - buff[i] = ch; - if (ipt >= c->len) - { - fin = 1; - break; - } - else ch = c->val[ipt++]; - } - AES_decrypt(&a, buff); - if (fin) break; - for (i = 0; i < 16; i++) - if (opt < m->max) m->val[opt++] = buff[i]; - } - AES_end(&a); - bad = 0; - padlen = buff[15]; - if (i != 15 || padlen < 1 || padlen > 16) bad = 1; - if (padlen >= 2 && padlen <= 16) - for (i = 16 - padlen; i < 16; i++) if (buff[i] != padlen) bad = 1; - - if (!bad) for (i = 0; i < 16 - padlen; i++) - if (opt < m->max) m->val[opt++] = buff[i]; - - m->len = opt; - if (bad) return 0; - return 1; -} - - -/* -#include - -#define KK 32 - -int main() -{ - int i; - core_aes a; - unsign32 t; - uchar x,y; - - char key[KK]; - char block[16]; - char iv[16]; - for (i=0;i= 199901L -/* C99 code */ -#define C99 -#else -/* Not C99 code */ -#endif - -#ifndef C99 /* You are on your own! These are for Microsoft C */ -#define byte unsigned char /**< 8-bit unsigned integer */ -#define sign32 __int32 /**< 32-bit signed integer */ -#define sign8 signed char /**< 8-bit signed integer */ -#define sign16 short int /**< 16-bit signed integer */ -#define sign64 long long /**< 64-bit signed integer */ -#define unsign32 unsigned __int32 /**< 32-bit unsigned integer */ -#define unsign64 unsigned long long /**< 64-bit unsigned integer */ -#else -#include -#define byte uint8_t /**< 8-bit unsigned integer */ -#define sign8 int8_t /**< 8-bit signed integer */ -#define sign16 int16_t /**< 16-bit signed integer */ -#define sign32 int32_t /**< 32-bit signed integer */ -#define sign64 int64_t /**< 64-bit signed integer */ -#define unsign32 uint32_t /**< 32-bit unsigned integer */ -#define unsign64 uint64_t /**< 64-bit unsigned integer */ -#endif - -#define uchar unsigned char /**< Unsigned char */ - -/* Don't mess with anything below this line unless you know what you are doing */ -/* This next is probably OK, but may need changing for non-C99-standard environments */ - -/* This next is probably OK, but may need changing for non-C99-standard environments */ - -#if CHUNK==16 -#ifndef C99 -#define chunk __int16 /**< C type corresponding to word length */ -#define dchunk __int32 /**< Always define double length chunk type if available */ -#else -#define chunk int16_t /**< C type corresponding to word length */ -#define dchunk int32_t /**< Always define double length chunk type if available */ -#endif -#endif - -#if CHUNK == 32 -#ifndef C99 -#define chunk __int32 /**< C type corresponding to word length */ -#define dchunk __int64 /**< Always define double length chunk type if available */ -#else -#define chunk int32_t /**< C type corresponding to word length */ -#define dchunk int64_t /**< Always define double length chunk type if available */ -#endif -#endif - -#if CHUNK == 64 - -#ifndef C99 -#define chunk __int64 /**< C type corresponding to word length */ -/**< Note - no 128-bit type available */ -#else -#define chunk int64_t /**< C type corresponding to word length */ -//#ifdef __GNUC__ -//#define dchunk __int128 /**< Always define double length chunk type if available - GCC supports 128 bit type ??? */ -//#endif - -//#ifdef __clang__ -//#define dchunk __int128 -#if defined(__SIZEOF_INT128__) && __SIZEOF_INT128__ == 16 -#define dchunk __int128 -#endif - -#endif -#endif - -#ifdef dchunk -#define COMBA /**< Use COMBA method for faster muls, sqrs and reductions */ -#endif - - -#endif diff --git a/impl/cbits/big_384_58.c b/impl/cbits/big_384_58.c deleted file mode 100644 index 023981394..000000000 --- a/impl/cbits/big_384_58.c +++ /dev/null @@ -1,1677 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/* CORE basic functions for BIG type */ -/* SU=m, SU is Stack Usage */ - -#include "big_384_58.h" - -/* test a=0? */ -int BIG_384_58_iszilch(BIG_384_58 a) -{ - int i; - chunk d=0; - for (i = 0; i < NLEN_384_58; i++) - d|=a[i]; - return (1 & ((d-1)>>BASEBITS_384_58)); -} - -/* test a=1? */ -int BIG_384_58_isunity(BIG_384_58 a) -{ - int i; - chunk d=0; - for (i = 1; i < NLEN_384_58; i++) - d|=a[i]; - return (1 & ((d-1)>>BASEBITS_384_58) & (((a[0]^1)-1)>>BASEBITS_384_58)); -} - -/* test a=0? */ -int BIG_384_58_diszilch(DBIG_384_58 a) -{ - int i; - chunk d=0; - for (i = 0; i < DNLEN_384_58; i++) - d|=a[i]; - return (1 & ((d-1)>>BASEBITS_384_58)); -} - -/* SU= 56 */ -/* output a */ -void BIG_384_58_output(BIG_384_58 a) -{ - BIG_384_58 b; - int i, len; - len = BIG_384_58_nbits(a); - if (len % 4 == 0) len /= 4; - else - { - len /= 4; - len++; - } - if (len < MODBYTES_384_58 * 2) len = MODBYTES_384_58 * 2; - - for (i = len - 1; i >= 0; i--) - { - BIG_384_58_copy(b, a); - BIG_384_58_shr(b, i * 4); - printf("%01x", (unsigned int) b[0] & 15); - } -} - -/* SU= 16 */ -void BIG_384_58_rawoutput(BIG_384_58 a) -{ - int i; - printf("("); - for (i = 0; i < NLEN_384_58 - 1; i++) -#if CHUNK==64 - printf("%"PRIxMAX",", (uintmax_t) a[i]); - printf("%"PRIxMAX")", (uintmax_t) a[NLEN_384_58 - 1]); -#else - printf("%x,", (unsigned int) a[i]); - printf("%x)", (unsigned int) a[NLEN_384_58 - 1]); -#endif -} - -/* Swap a and b if d=1 */ -void BIG_384_58_cswap(BIG_384_58 a, BIG_384_58 b, int d) -{ - int i; - chunk t, c = d; - c = ~(c - 1); -#ifdef DEBUG_NORM - for (i = 0; i < NLEN_384_58 + 2; i++) -#else - for (i = 0; i < NLEN_384_58; i++) -#endif - { - t = c & (a[i] ^ b[i]); - a[i] ^= t; - b[i] ^= t; - } -} - -/* Move b to a if d=1 */ -void BIG_384_58_cmove(BIG_384_58 f, BIG_384_58 g, int d) -{ - int i; - chunk b = (chunk) - d; -#ifdef DEBUG_NORM - for (i = 0; i < NLEN_384_58 + 2; i++) -#else - for (i = 0; i < NLEN_384_58; i++) -#endif - { - f[i] ^= (f[i] ^ g[i])&b; - } -} - -/* Move g to f if d=1 */ -void BIG_384_58_dcmove(DBIG_384_58 f, DBIG_384_58 g, int d) -{ - int i; - chunk b = (chunk) - d; -#ifdef DEBUG_NORM - for (i = 0; i < DNLEN_384_58 + 2; i++) -#else - for (i = 0; i < DNLEN_384_58; i++) -#endif - { - f[i] ^= (f[i] ^ g[i])&b; - } -} - -/* convert BIG to/from bytes */ -/* SU= 64 */ -void BIG_384_58_toBytes(char *b, BIG_384_58 a) -{ - int i; - BIG_384_58 c; - BIG_384_58_copy(c, a); - BIG_384_58_norm(c); - for (i = MODBYTES_384_58 - 1; i >= 0; i--) - { - b[i] = c[0] & 0xff; - BIG_384_58_fshr(c, 8); - } -} - -/* SU= 16 */ -void BIG_384_58_fromBytes(BIG_384_58 a, char *b) -{ - int i; - BIG_384_58_zero(a); - for (i = 0; i < MODBYTES_384_58; i++) - { - BIG_384_58_fshl(a, 8); - a[0] += (int)(unsigned char)b[i]; - } -#ifdef DEBUG_NORM - a[MPV_384_58] = 1; - a[MNV_384_58] = 0; -#endif -} - -void BIG_384_58_fromBytesLen(BIG_384_58 a, char *b, int s) -{ - int i, len = s; - BIG_384_58_zero(a); - - if (len > MODBYTES_384_58) len = MODBYTES_384_58; - for (i = 0; i < len; i++) - { - BIG_384_58_fshl(a, 8); - a[0] += (int)(unsigned char)b[i]; - } -#ifdef DEBUG_NORM - a[MPV_384_58] = 1; - a[MNV_384_58] = 0; -#endif -} - - - -/* SU= 88 */ -void BIG_384_58_doutput(DBIG_384_58 a) -{ - DBIG_384_58 b; - int i, len; - BIG_384_58_dnorm(a); - len = BIG_384_58_dnbits(a); - if (len % 4 == 0) len /= 4; - else - { - len /= 4; - len++; - } - - for (i = len - 1; i >= 0; i--) - { - BIG_384_58_dcopy(b, a); - BIG_384_58_dshr(b, i * 4); - printf("%01x", (unsigned int) b[0] & 15); - } -} - - -void BIG_384_58_drawoutput(DBIG_384_58 a) -{ - int i; - printf("("); - for (i = 0; i < DNLEN_384_58 - 1; i++) -#if CHUNK==64 - printf("%"PRIxMAX",", (uintmax_t) a[i]); - printf("%"PRIxMAX")", (uintmax_t) a[DNLEN_384_58 - 1]); -#else - printf("%x,", (unsigned int) a[i]); - printf("%x)", (unsigned int) a[DNLEN_384_58 - 1]); -#endif -} - -/* Copy b=a */ -void BIG_384_58_copy(BIG_384_58 b, BIG_384_58 a) -{ - int i; - for (i = 0; i < NLEN_384_58; i++) - b[i] = a[i]; -#ifdef DEBUG_NORM - b[MPV_384_58] = a[MPV_384_58]; - b[MNV_384_58] = a[MNV_384_58]; -#endif -} - -/* Copy from ROM b=a */ -void BIG_384_58_rcopy(BIG_384_58 b, const BIG_384_58 a) -{ - int i; - for (i = 0; i < NLEN_384_58; i++) - b[i] = a[i]; -#ifdef DEBUG_NORM - b[MPV_384_58] = 1; - b[MNV_384_58] = 0; -#endif -} - -/* double length DBIG copy b=a */ -void BIG_384_58_dcopy(DBIG_384_58 b, DBIG_384_58 a) -{ - int i; - for (i = 0; i < DNLEN_384_58; i++) - b[i] = a[i]; -#ifdef DEBUG_NORM - b[DMPV_384_58] = a[DMPV_384_58]; - b[DMNV_384_58] = a[DMNV_384_58]; -#endif -} - -/* Copy BIG to bottom half of DBIG */ -void BIG_384_58_dscopy(DBIG_384_58 b, BIG_384_58 a) -{ - int i; - for (i = 0; i < NLEN_384_58 - 1; i++) - b[i] = a[i]; - - b[NLEN_384_58 - 1] = a[NLEN_384_58 - 1] & BMASK_384_58; /* top word normalized */ - b[NLEN_384_58] = a[NLEN_384_58 - 1] >> BASEBITS_384_58; - - for (i = NLEN_384_58 + 1; i < DNLEN_384_58; i++) b[i] = 0; -#ifdef DEBUG_NORM - b[DMPV_384_58] = a[MPV_384_58]; - b[DMNV_384_58] = a[MNV_384_58]; -#endif -} - -/* Copy BIG to top half of DBIG */ -void BIG_384_58_dsucopy(DBIG_384_58 b, BIG_384_58 a) -{ - int i; - for (i = 0; i < NLEN_384_58; i++) - b[i] = 0; - for (i = NLEN_384_58; i < DNLEN_384_58; i++) - b[i] = a[i - NLEN_384_58]; -#ifdef DEBUG_NORM - b[DMPV_384_58] = a[MPV_384_58]; - b[DMNV_384_58] = a[MNV_384_58]; -#endif -} - -/* Copy bottom half of DBIG to BIG */ -void BIG_384_58_sdcopy(BIG_384_58 b, DBIG_384_58 a) -{ - int i; - for (i = 0; i < NLEN_384_58; i++) - b[i] = a[i]; -#ifdef DEBUG_NORM - b[MPV_384_58] = a[DMPV_384_58]; - b[MNV_384_58] = a[DMNV_384_58]; -#endif -} - -/* Copy top half of DBIG to BIG */ -void BIG_384_58_sducopy(BIG_384_58 b, DBIG_384_58 a) -{ - int i; - for (i = 0; i < NLEN_384_58; i++) - b[i] = a[i + NLEN_384_58]; -#ifdef DEBUG_NORM - b[MPV_384_58] = a[DMPV_384_58]; - b[MNV_384_58] = a[DMNV_384_58]; - -#endif -} - -/* Set a=0 */ -void BIG_384_58_zero(BIG_384_58 a) -{ - int i; - for (i = 0; i < NLEN_384_58; i++) - a[i] = 0; -#ifdef DEBUG_NORM - a[MPV_384_58] = a[MNV_384_58] = 0; -#endif -} - -void BIG_384_58_dzero(DBIG_384_58 a) -{ - int i; - for (i = 0; i < DNLEN_384_58; i++) - a[i] = 0; -#ifdef DEBUG_NORM - a[DMPV_384_58] = a[DMNV_384_58] = 0; -#endif -} - -/* set a=1 */ -void BIG_384_58_one(BIG_384_58 a) -{ - int i; - a[0] = 1; - for (i = 1; i < NLEN_384_58; i++) - a[i] = 0; -#ifdef DEBUG_NORM - a[MPV_384_58] = 1; - a[MNV_384_58] = 0; -#endif -} - - - -/* Set c=a+b */ -/* SU= 8 */ -void BIG_384_58_add(BIG_384_58 c, BIG_384_58 a, BIG_384_58 b) -{ - int i; - for (i = 0; i < NLEN_384_58; i++) - c[i] = a[i] + b[i]; -#ifdef DEBUG_NORM - c[MPV_384_58] = a[MPV_384_58] + b[MPV_384_58]; - c[MNV_384_58] = a[MNV_384_58] + b[MNV_384_58]; - if (c[MPV_384_58] > NEXCESS_384_58) printf("add problem - positive digit overflow %d\n", (int)c[MPV_384_58]); - if (c[MNV_384_58] > NEXCESS_384_58) printf("add problem - negative digit overflow %d\n", (int)c[MNV_384_58]); - -#endif -} - -/* Set c=a or b */ -void BIG_384_58_or(BIG_384_58 c, BIG_384_58 a, BIG_384_58 b) -{ - int i; - BIG_384_58_norm(a); - BIG_384_58_norm(b); - for (i = 0; i < NLEN_384_58; i++) - c[i] = a[i] | b[i]; -#ifdef DEBUG_NORM - c[MPV_384_58] = 1; - c[MNV_384_58] = 0; -#endif -} - - -/* Set c=c+d */ -void BIG_384_58_inc(BIG_384_58 c, int d) -{ - BIG_384_58_norm(c); - c[0] += (chunk)d; -#ifdef DEBUG_NORM - c[MPV_384_58] += 1; -#endif -} - -/* Set c=a-b */ -/* SU= 8 */ -void BIG_384_58_sub(BIG_384_58 c, BIG_384_58 a, BIG_384_58 b) -{ - int i; - for (i = 0; i < NLEN_384_58; i++) - c[i] = a[i] - b[i]; -#ifdef DEBUG_NORM - c[MPV_384_58] = a[MPV_384_58] + b[MNV_384_58]; - c[MNV_384_58] = a[MNV_384_58] + b[MPV_384_58]; - if (c[MPV_384_58] > NEXCESS_384_58) printf("sub problem - positive digit overflow %d\n", (int)c[MPV_384_58]); - if (c[MNV_384_58] > NEXCESS_384_58) printf("sub problem - negative digit overflow %d\n", (int)c[MNV_384_58]); - -#endif -} - -/* SU= 8 */ - -void BIG_384_58_dsub(DBIG_384_58 c, DBIG_384_58 a, DBIG_384_58 b) -{ - int i; - for (i = 0; i < DNLEN_384_58; i++) - c[i] = a[i] - b[i]; -#ifdef DEBUG_NORM - c[DMPV_384_58] = a[DMPV_384_58] + b[DMNV_384_58]; - c[DMNV_384_58] = a[DMNV_384_58] + b[DMPV_384_58]; - if (c[DMPV_384_58] > NEXCESS_384_58) printf("double sub problem - positive digit overflow %d\n", (int)c[DMPV_384_58]); - if (c[DMNV_384_58] > NEXCESS_384_58) printf("double sub problem - negative digit overflow %d\n", (int)c[DMNV_384_58]); -#endif -} - -void BIG_384_58_dadd(DBIG_384_58 c, DBIG_384_58 a, DBIG_384_58 b) -{ - int i; - for (i = 0; i < DNLEN_384_58; i++) - c[i] = a[i] + b[i]; -#ifdef DEBUG_NORM - c[DMPV_384_58] = a[DMPV_384_58] + b[DMNV_384_58]; - c[DMNV_384_58] = a[DMNV_384_58] + b[DMPV_384_58]; - if (c[DMPV_384_58] > NEXCESS_384_58) printf("double add problem - positive digit overflow %d\n", (int)c[DMPV_384_58]); - if (c[DMNV_384_58] > NEXCESS_384_58) printf("double add problem - negative digit overflow %d\n", (int)c[DMNV_384_58]); -#endif -} - -/* Set c=c-1 */ -void BIG_384_58_dec(BIG_384_58 c, int d) -{ - BIG_384_58_norm(c); - c[0] -= (chunk)d; -#ifdef DEBUG_NORM - c[MNV_384_58] += 1; -#endif -} - -/* multiplication r=a*c by c<=NEXCESS_384_58 */ -void BIG_384_58_imul(BIG_384_58 r, BIG_384_58 a, int c) -{ - int i; - for (i = 0; i < NLEN_384_58; i++) r[i] = a[i] * c; -#ifdef DEBUG_NORM - r[MPV_384_58] = a[MPV_384_58] * c; - r[MNV_384_58] = a[MNV_384_58] * c; - if (r[MPV_384_58] > NEXCESS_384_58) printf("int mul problem - positive digit overflow %d\n", (int)r[MPV_384_58]); - if (r[MNV_384_58] > NEXCESS_384_58) printf("int mul problem - negative digit overflow %d\n", (int)r[MNV_384_58]); - -#endif -} - -/* multiplication r=a*c by larger integer - c<=FEXCESS */ -/* SU= 24 */ -chunk BIG_384_58_pmul(BIG_384_58 r, BIG_384_58 a, int c) -{ - int i; - chunk ak, carry = 0; - for (i = 0; i < NLEN_384_58; i++) - { - ak = a[i]; - r[i] = 0; - carry = muladd_384_58(ak, (chunk)c, carry, &r[i]); - } -#ifdef DEBUG_NORM - r[MPV_384_58] = 1; - r[MNV_384_58] = 0; -#endif - return carry; -} - -/* r/=3 */ -/* SU= 16 */ -int BIG_384_58_div3(BIG_384_58 r) -{ - int i; - chunk ak, base, carry = 0; - BIG_384_58_norm(r); - base = ((chunk)1 << BASEBITS_384_58); - for (i = NLEN_384_58 - 1; i >= 0; i--) - { - ak = (carry * base + r[i]); - r[i] = ak / 3; - carry = ak % 3; - } - return (int)carry; -} - -/* multiplication c=a*b by even larger integer b>FEXCESS, resulting in DBIG */ -/* SU= 24 */ -void BIG_384_58_pxmul(DBIG_384_58 c, BIG_384_58 a, int b) -{ - int j; - chunk carry; - BIG_384_58_dzero(c); - carry = 0; - for (j = 0; j < NLEN_384_58; j++) - carry = muladd_384_58(a[j], (chunk)b, carry, &c[j]); - c[NLEN_384_58] = carry; -#ifdef DEBUG_NORM - c[DMPV_384_58] = 1; - c[DMNV_384_58] = 0; -#endif -} - -/* .. if you know the result will fit in a BIG, c must be distinct from a and b */ -/* SU= 40 */ -void BIG_384_58_smul(BIG_384_58 c, BIG_384_58 a, BIG_384_58 b) -{ - int i, j; - chunk carry; - - BIG_384_58_zero(c); - for (i = 0; i < NLEN_384_58; i++) - { - carry = 0; - for (j = 0; j < NLEN_384_58; j++) - { - if (i + j < NLEN_384_58) - carry = muladd_384_58(a[i], b[j], carry, &c[i + j]); - } - } -#ifdef DEBUG_NORM - c[MPV_384_58] = 1; - c[MNV_384_58] = 0; -#endif - -} - -/* Set c=a*b */ -/* SU= 72 */ -//void BIG_384_58_mul(chunk c[restrict DNLEN_384_58],chunk a[restrict NLEN_384_58],chunk b[restrict NLEN_384_58]) -void BIG_384_58_mul(DBIG_384_58 c, BIG_384_58 a, BIG_384_58 b) -{ - int i; -#ifdef dchunk - dchunk t, co; - dchunk s; - dchunk d[NLEN_384_58]; - int k; -#endif - -#ifdef DEBUG_NORM - if ((a[MPV_384_58] != 1 && a[MPV_384_58] != 0) || a[MNV_384_58] != 0) printf("First input to mul not normed\n"); - if ((b[MPV_384_58] != 1 && b[MPV_384_58] != 0) || b[MNV_384_58] != 0) printf("Second input to mul not normed\n"); -#endif - - /* Faster to Combafy it.. Let the compiler unroll the loops! */ - -#ifdef COMBA - - /* faster psuedo-Karatsuba method */ -#ifdef UNWOUND - -#ifdef USE_KARATSUBA - - d[0]=(dchunk)a[0]*b[0]; - d[1]=(dchunk)a[1]*b[1]; - d[2]=(dchunk)a[2]*b[2]; - d[3]=(dchunk)a[3]*b[3]; - d[4]=(dchunk)a[4]*b[4]; - d[5]=(dchunk)a[5]*b[5]; - d[6]=(dchunk)a[6]*b[6]; - - s=d[0]; - t = s; c[0]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - s+=d[1]; t=co+s +(dchunk)(a[1]-a[0])*(b[0]-b[1]); c[1]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - s+=d[2]; t=co+s +(dchunk)(a[2]-a[0])*(b[0]-b[2]); c[2]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - s+=d[3]; t=co+s +(dchunk)(a[3]-a[0])*(b[0]-b[3])+(dchunk)(a[2]-a[1])*(b[1]-b[2]); c[3]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - s+=d[4]; t=co+s +(dchunk)(a[4]-a[0])*(b[0]-b[4])+(dchunk)(a[3]-a[1])*(b[1]-b[3]); c[4]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - s+=d[5]; t=co+s +(dchunk)(a[5]-a[0])*(b[0]-b[5])+(dchunk)(a[4]-a[1])*(b[1]-b[4])+(dchunk)(a[3]-a[2])*(b[2]-b[3]); c[5]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - s+=d[6]; t=co+s +(dchunk)(a[6]-a[0])*(b[0]-b[6])+(dchunk)(a[5]-a[1])*(b[1]-b[5])+(dchunk)(a[4]-a[2])*(b[2]-b[4]); c[6]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - - s-=d[0]; t=co+s +(dchunk)(a[6]-a[1])*(b[1]-b[6])+(dchunk)(a[5]-a[2])*(b[2]-b[5])+(dchunk)(a[4]-a[3])*(b[3]-b[4]); c[7]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - s-=d[1]; t=co+s +(dchunk)(a[6]-a[2])*(b[2]-b[6])+(dchunk)(a[5]-a[3])*(b[3]-b[5]); c[8]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - s-=d[2]; t=co+s +(dchunk)(a[6]-a[3])*(b[3]-b[6])+(dchunk)(a[5]-a[4])*(b[4]-b[5]); c[9]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - s-=d[3]; t=co+s +(dchunk)(a[6]-a[4])*(b[4]-b[6]); c[10]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - s-=d[4]; t=co+s +(dchunk)(a[6]-a[5])*(b[5]-b[6]); c[11]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - s-=d[5]; t=co+s ; c[12]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - c[13]=(chunk)co; - - -#else - - t=(dchunk)a[0]*b[0]; c[0]=(chunk)t & BMASK_384_58; t=t>>BASEBITS_384_58; - t=t+(dchunk)a[0]*b[1]+(dchunk)a[1]*b[0]; c[1]=(chunk)t & BMASK_384_58; t=t>>BASEBITS_384_58; - t=t+(dchunk)a[0]*b[2]+(dchunk)a[1]*b[1]+(dchunk)a[2]*b[0]; c[2]=(chunk)t & BMASK_384_58; t=t>>BASEBITS_384_58; - t=t+(dchunk)a[0]*b[3]+(dchunk)a[1]*b[2]+(dchunk)a[2]*b[1]+(dchunk)a[3]*b[0]; c[3]=(chunk)t & BMASK_384_58; t=t>>BASEBITS_384_58; - t=t+(dchunk)a[0]*b[4]+(dchunk)a[1]*b[3]+(dchunk)a[2]*b[2]+(dchunk)a[3]*b[1]+(dchunk)a[4]*b[0]; c[4]=(chunk)t & BMASK_384_58; t=t>>BASEBITS_384_58; - t=t+(dchunk)a[0]*b[5]+(dchunk)a[1]*b[4]+(dchunk)a[2]*b[3]+(dchunk)a[3]*b[2]+(dchunk)a[4]*b[1]+(dchunk)a[5]*b[0]; c[5]=(chunk)t & BMASK_384_58; t=t>>BASEBITS_384_58; - t=t+(dchunk)a[0]*b[6]+(dchunk)a[1]*b[5]+(dchunk)a[2]*b[4]+(dchunk)a[3]*b[3]+(dchunk)a[4]*b[2]+(dchunk)a[5]*b[1]+(dchunk)a[6]*b[0]; c[6]=(chunk)t & BMASK_384_58; t=t>>BASEBITS_384_58; - t=t+(dchunk)a[1]*b[6]+(dchunk)a[2]*b[5]+(dchunk)a[3]*b[4]+(dchunk)a[4]*b[3]+(dchunk)a[5]*b[2]+(dchunk)a[6]*b[1]; c[7]=(chunk)t & BMASK_384_58; t=t>>BASEBITS_384_58; - t=t+(dchunk)a[2]*b[6]+(dchunk)a[3]*b[5]+(dchunk)a[4]*b[4]+(dchunk)a[5]*b[3]+(dchunk)a[6]*b[2]; c[8]=(chunk)t & BMASK_384_58; t=t>>BASEBITS_384_58; - t=t+(dchunk)a[3]*b[6]+(dchunk)a[4]*b[5]+(dchunk)a[5]*b[4]+(dchunk)a[6]*b[3]; c[9]=(chunk)t & BMASK_384_58; t=t>>BASEBITS_384_58; - t=t+(dchunk)a[4]*b[6]+(dchunk)a[5]*b[5]+(dchunk)a[6]*b[4]; c[10]=(chunk)t & BMASK_384_58; t=t>>BASEBITS_384_58; - t=t+(dchunk)a[5]*b[6]+(dchunk)a[6]*b[5]; c[11]=(chunk)t & BMASK_384_58; t=t>>BASEBITS_384_58; - t=t+(dchunk)a[6]*b[6]; c[12]=(chunk)t & BMASK_384_58; t=t>>BASEBITS_384_58; - c[13]=(chunk)t; - - -#endif - - -#else - -#ifndef USE_KARATSUBA - - t=(dchunk)a[0]*b[0]; - c[0]=(chunk)t & BMASK_384_58; - t = t >> BASEBITS_384_58; - for (i=1;i> BASEBITS_384_58; - } - - for (i=NLEN_384_58;i<2*NLEN_384_58-1;i++) - { - k=i-(NLEN_384_58-1); - while (k<=NLEN_384_58-1) {t+=(dchunk)a[k]*b[i-k]; k++;} - c[i]=(chunk)t & BMASK_384_58; - t = t >> BASEBITS_384_58; - } - - c[2 * NLEN_384_58 - 1] = (chunk)t; -#else - - for (i = 0; i < NLEN_384_58; i++) - d[i] = (dchunk)a[i] * b[i]; - - s = d[0]; - t = s; - c[0] = (chunk)t & BMASK_384_58; - co = t >> BASEBITS_384_58; - - for (k = 1; k < NLEN_384_58; k++) - { - s += d[k]; - t = co + s; - - /*for (i = k; i >= 1 + k / 2; i--) This causes a huge slow down! gcc/g++ optimizer problem (I think) */ - for (i=1+k/2;i<=k;i++) t += (dchunk)(a[i] - a[k - i]) * (b[k - i] - b[i]); - c[k] = (chunk)t & BMASK_384_58; - co = t >> BASEBITS_384_58; - } - for (k = NLEN_384_58; k < 2 * NLEN_384_58 - 1; k++) - { - s -= d[k - NLEN_384_58]; - t = co + s; - for (i=1+k/2;i> BASEBITS_384_58; - } - c[2 * NLEN_384_58 - 1] = (chunk)co; -#endif -#endif - -#else - int j; - chunk carry; - BIG_384_58_dzero(c); - for (i = 0; i < NLEN_384_58; i++) - { - carry = 0; - for (j = 0; j < NLEN_384_58; j++) - carry = muladd_384_58(a[i], b[j], carry, &c[i + j]); - - c[NLEN_384_58 + i] = carry; - } - -#endif - -#ifdef DEBUG_NORM - c[DMPV_384_58] = 1; - c[DMNV_384_58] = 0; -#endif -} - -/* Set c=a*a */ -/* SU= 80 */ - -//void BIG_384_58_sqr(chunk c[restrict DNLEN_384_58],chunk a[restrict NLEN_384_58]) -void BIG_384_58_sqr(DBIG_384_58 c, BIG_384_58 a) -{ - int i, j; -#ifdef dchunk - dchunk t, co; -#endif - -#ifdef DEBUG_NORM - if ((a[MPV_384_58] != 1 && a[MPV_384_58] != 0) || a[MNV_384_58] != 0) printf("Input to sqr not normed\n"); -#endif - /* Note 2*a[i] in loop below and extra addition */ - -#ifdef COMBA - -#ifdef UNWOUND - - - t=(dchunk)a[0]*a[0]; c[0]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - t= +(dchunk)a[1]*a[0]; t+=t; t+=co; c[1]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - t= +(dchunk)a[2]*a[0]; t+=t; t+=co; t+=(dchunk)a[1]*a[1]; c[2]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - t= +(dchunk)a[3]*a[0]+(dchunk)a[2]*a[1]; t+=t; t+=co; c[3]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - t= +(dchunk)a[4]*a[0]+(dchunk)a[3]*a[1]; t+=t; t+=co; t+=(dchunk)a[2]*a[2]; c[4]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - t= +(dchunk)a[5]*a[0]+(dchunk)a[4]*a[1]+(dchunk)a[3]*a[2]; t+=t; t+=co; c[5]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - t= +(dchunk)a[6]*a[0]+(dchunk)a[5]*a[1]+(dchunk)a[4]*a[2]; t+=t; t+=co; t+=(dchunk)a[3]*a[3]; c[6]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - - t= +(dchunk)a[6]*a[1]+(dchunk)a[5]*a[2]+(dchunk)a[4]*a[3]; t+=t; t+=co; c[7]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - t= +(dchunk)a[6]*a[2]+(dchunk)a[5]*a[3]; t+=t; t+=co; t+=(dchunk)a[4]*a[4]; c[8]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - t= +(dchunk)a[6]*a[3]+(dchunk)a[5]*a[4]; t+=t; t+=co; c[9]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - t= +(dchunk)a[6]*a[4]; t+=t; t+=co; t+=(dchunk)a[5]*a[5]; c[10]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - t= +(dchunk)a[6]*a[5]; t+=t; t+=co; c[11]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - t=co; t+=(dchunk)a[6]*a[6]; c[12]=(chunk)t&BMASK_384_58; co=t>>BASEBITS_384_58; - c[13]=(chunk)co; - - -#else - - - t = (dchunk)a[0] * a[0]; - c[0] = (chunk)t & BMASK_384_58; - co = t >> BASEBITS_384_58; - - for (j = 1; j < NLEN_384_58 - 1; ) - { - t = (dchunk)a[j] * a[0]; - for (i = 1; i < (j + 1) / 2; i++) - { - t += (dchunk)a[j - i] * a[i]; - } - t += t; - t += co; - c[j] = (chunk)t & BMASK_384_58; - co = t >> BASEBITS_384_58; - j++; - t = (dchunk)a[j] * a[0]; - for (i = 1; i < (j + 1) / 2; i++) - { - t += (dchunk)a[j - i] * a[i]; - } - t += t; - t += co; - t += (dchunk)a[j / 2] * a[j / 2]; - c[j] = (chunk)t & BMASK_384_58; - co = t >> BASEBITS_384_58; - j++; - } - - for (j = NLEN_384_58 - 1 + NLEN_384_58 % 2; j < DNLEN_384_58 - 3; ) - { - t = (dchunk)a[NLEN_384_58 - 1] * a[j - NLEN_384_58 + 1]; - for (i = j - NLEN_384_58 + 2; i < (j + 1) / 2; i++) - { - t += (dchunk)a[j - i] * a[i]; - } - t += t; - t += co; - c[j] = (chunk)t & BMASK_384_58; - co = t >> BASEBITS_384_58; - j++; - t = (dchunk)a[NLEN_384_58 - 1] * a[j - NLEN_384_58 + 1]; - for (i = j - NLEN_384_58 + 2; i < (j + 1) / 2; i++) - { - t += (dchunk)a[j - i] * a[i]; - } - t += t; - t += co; - t += (dchunk)a[j / 2] * a[j / 2]; - c[j] = (chunk)t & BMASK_384_58; - co = t >> BASEBITS_384_58; - j++; - } - - t = (dchunk)a[NLEN_384_58 - 2] * a[NLEN_384_58 - 1]; - t += t; - t += co; - c[DNLEN_384_58 - 3] = (chunk)t & BMASK_384_58; - co = t >> BASEBITS_384_58; - - t = (dchunk)a[NLEN_384_58 - 1] * a[NLEN_384_58 - 1] + co; - c[DNLEN_384_58 - 2] = (chunk)t & BMASK_384_58; - co = t >> BASEBITS_384_58; - c[DNLEN_384_58 - 1] = (chunk)co; - - -#endif - -#else - chunk carry; - BIG_384_58_dzero(c); - for (i = 0; i < NLEN_384_58; i++) - { - carry = 0; - for (j = i + 1; j < NLEN_384_58; j++) - carry = muladd_384_58(a[i], a[j], carry, &c[i + j]); - c[NLEN_384_58 + i] = carry; - } - - for (i = 0; i < DNLEN_384_58; i++) c[i] *= 2; - - for (i = 0; i < NLEN_384_58; i++) - c[2 * i + 1] += muladd_384_58(a[i], a[i], 0, &c[2 * i]); - - BIG_384_58_dnorm(c); -#endif - - -#ifdef DEBUG_NORM - c[DMPV_384_58] = 1; - c[DMNV_384_58] = 0; -#endif - -} - -/* Montgomery reduction */ -//void BIG_384_58_monty(chunk a[restrict NLEN_384_58], chunk md[restrict NLEN_384_58], chunk MC, chunk d[restrict DNLEN_384_58]) -void BIG_384_58_monty(BIG_384_58 a, BIG_384_58 md, chunk MC, DBIG_384_58 d) -{ - int i, k; - -#ifdef dchunk - dchunk t, c, s; - dchunk dd[NLEN_384_58]; - chunk v[NLEN_384_58]; -#endif - -#ifdef DEBUG_NORM - if ((d[DMPV_384_58] != 1 && d[DMPV_384_58] != 0) || d[DMNV_384_58] != 0) printf("Input to redc not normed\n"); -#endif - -#ifdef COMBA - -#ifdef UNWOUND - -#ifdef USE_KARATSUBA - - t=d[0]; v[0]=((chunk)t*MC)&BMASK_384_58; t+=(dchunk)v[0]*md[0]; s=0; c=(t>>BASEBITS_384_58); - - t=d[1]+c+s+(dchunk)v[0]*md[1]; v[1]=((chunk)t*MC)&BMASK_384_58; t+=(dchunk)v[1]*md[0]; dd[1]=(dchunk)v[1]*md[1]; s+=dd[1]; c=(t>>BASEBITS_384_58); - t=d[2]+c+s+(dchunk)v[0]*md[2]; v[2]=((chunk)t*MC)&BMASK_384_58; t+=(dchunk)v[2]*md[0]; dd[2]=(dchunk)v[2]*md[2]; s+=dd[2]; c=(t>>BASEBITS_384_58); - t=d[3]+c+s+(dchunk)v[0]*md[3]+(dchunk)(v[1]-v[2])*(md[2]-md[1]); v[3]=((chunk)t*MC)&BMASK_384_58; t+=(dchunk)v[3]*md[0]; dd[3]=(dchunk)v[3]*md[3]; s+=dd[3]; c=(t>>BASEBITS_384_58); - t=d[4]+c+s+(dchunk)v[0]*md[4]+(dchunk)(v[1]-v[3])*(md[3]-md[1]); v[4]=((chunk)t*MC)&BMASK_384_58; t+=(dchunk)v[4]*md[0]; dd[4]=(dchunk)v[4]*md[4]; s+=dd[4]; c=(t>>BASEBITS_384_58); - t=d[5]+c+s+(dchunk)v[0]*md[5]+(dchunk)(v[1]-v[4])*(md[4]-md[1])+(dchunk)(v[2]-v[3])*(md[3]-md[2]); v[5]=((chunk)t*MC)&BMASK_384_58; t+=(dchunk)v[5]*md[0]; dd[5]=(dchunk)v[5]*md[5]; s+=dd[5]; c=(t>>BASEBITS_384_58); - t=d[6]+c+s+(dchunk)v[0]*md[6]+(dchunk)(v[1]-v[5])*(md[5]-md[1])+(dchunk)(v[2]-v[4])*(md[4]-md[2]); v[6]=((chunk)t*MC)&BMASK_384_58; t+=(dchunk)v[6]*md[0]; dd[6]=(dchunk)v[6]*md[6]; s+=dd[6]; c=(t>>BASEBITS_384_58); - - t=d[7]+c+s+(dchunk)(v[1]-v[6])*(md[6]-md[1])+(dchunk)(v[2]-v[5])*(md[5]-md[2])+(dchunk)(v[3]-v[4])*(md[4]-md[3]); a[0]=(chunk)t&BMASK_384_58; s-=dd[1]; c=(t>>BASEBITS_384_58); - t=d[8]+c+s+(dchunk)(v[2]-v[6])*(md[6]-md[2])+(dchunk)(v[3]-v[5])*(md[5]-md[3]); a[1]=(chunk)t&BMASK_384_58; s-=dd[2]; c=(t>>BASEBITS_384_58); - t=d[9]+c+s+(dchunk)(v[3]-v[6])*(md[6]-md[3])+(dchunk)(v[4]-v[5])*(md[5]-md[4]); a[2]=(chunk)t&BMASK_384_58; s-=dd[3]; c=(t>>BASEBITS_384_58); - t=d[10]+c+s+(dchunk)(v[4]-v[6])*(md[6]-md[4]); a[3]=(chunk)t&BMASK_384_58; s-=dd[4]; c=(t>>BASEBITS_384_58); - t=d[11]+c+s+(dchunk)(v[5]-v[6])*(md[6]-md[5]); a[4]=(chunk)t&BMASK_384_58; s-=dd[5]; c=(t>>BASEBITS_384_58); - t=d[12]+c+s; a[5]=(chunk)t&BMASK_384_58; s-=dd[6]; c=(t>>BASEBITS_384_58); - a[6]=d[13]+((chunk)c&BMASK_384_58); - - -#else - - t = d[0]; - v[0] = ((chunk)t * MC)&BMASK_384_58; - t += (dchunk)v[0] * md[0]; - t = (t >> BASEBITS_384_58) + d[1]; - t += (dchunk)v[0] * md[1] ; v[1] = ((chunk)t * MC)&BMASK_384_58; t += (dchunk)v[1] * md[0]; t = (t >> BASEBITS_384_58) + d[2]; - t += (dchunk)v[0] * md[2] + (dchunk)v[1]*md[1]; v[2] = ((chunk)t * MC)&BMASK_384_58; t += (dchunk)v[2] * md[0]; t = (t >> BASEBITS_384_58) + d[3]; - t += (dchunk)v[0] * md[3] + (dchunk)v[1]*md[2]+ (dchunk)v[2]*md[1]; v[3] = ((chunk)t * MC)&BMASK_384_58; t += (dchunk)v[3] * md[0]; t = (t >> BASEBITS_384_58) + d[4]; - t += (dchunk)v[0] * md[4] + (dchunk)v[1]*md[3]+ (dchunk)v[2]*md[2]+ (dchunk)v[3]*md[1]; v[4] = ((chunk)t * MC)&BMASK_384_58; t += (dchunk)v[4] * md[0]; t = (t >> BASEBITS_384_58) + d[5]; - t += (dchunk)v[0] * md[5] + (dchunk)v[1]*md[4]+ (dchunk)v[2]*md[3]+ (dchunk)v[3]*md[2]+ (dchunk)v[4]*md[1]; v[5] = ((chunk)t * MC)&BMASK_384_58; t += (dchunk)v[5] * md[0]; t = (t >> BASEBITS_384_58) + d[6]; - t += (dchunk)v[0] * md[6] + (dchunk)v[1]*md[5]+ (dchunk)v[2]*md[4]+ (dchunk)v[3]*md[3]+ (dchunk)v[4]*md[2]+ (dchunk)v[5]*md[1]; v[6] = ((chunk)t * MC)&BMASK_384_58; t += (dchunk)v[6] * md[0]; t = (t >> BASEBITS_384_58) + d[7]; - t=t + (dchunk)v[1]*md[6] + (dchunk)v[2]*md[5] + (dchunk)v[3]*md[4] + (dchunk)v[4]*md[3] + (dchunk)v[5]*md[2] + (dchunk)v[6]*md[1] ; a[0] = (chunk)t & BMASK_384_58; t = (t >> BASEBITS_384_58) + d[8]; - t=t + (dchunk)v[2]*md[6] + (dchunk)v[3]*md[5] + (dchunk)v[4]*md[4] + (dchunk)v[5]*md[3] + (dchunk)v[6]*md[2] ; a[1] = (chunk)t & BMASK_384_58; t = (t >> BASEBITS_384_58) + d[9]; - t=t + (dchunk)v[3]*md[6] + (dchunk)v[4]*md[5] + (dchunk)v[5]*md[4] + (dchunk)v[6]*md[3] ; a[2] = (chunk)t & BMASK_384_58; t = (t >> BASEBITS_384_58) + d[10]; - t=t + (dchunk)v[4]*md[6] + (dchunk)v[5]*md[5] + (dchunk)v[6]*md[4] ; a[3] = (chunk)t & BMASK_384_58; t = (t >> BASEBITS_384_58) + d[11]; - t=t + (dchunk)v[5]*md[6] + (dchunk)v[6]*md[5] ; a[4] = (chunk)t & BMASK_384_58; t = (t >> BASEBITS_384_58) + d[12]; - t=t + (dchunk)v[6]*md[6] ; a[5] = (chunk)t & BMASK_384_58; t = (t >> BASEBITS_384_58) + d[13]; - a[6] = (chunk)t & BMASK_384_58; - - -#endif - - -#else -#ifndef USE_KARATSUBA - t = d[0]; - v[0] = ((chunk)t * MC)&BMASK_384_58; - t += (dchunk)v[0] * md[0]; - t = (t >> BASEBITS_384_58) + d[1]; - - for (i = 1; i < NLEN_384_58; i++) - { - k=1; - t += (dchunk)v[0] * md[i]; - while (k> BASEBITS_384_58) + d[i + 1]; - } - for (i = NLEN_384_58; i < 2 * NLEN_384_58 - 1; i++) - { - k=i-(NLEN_384_58-1); - while (k<=NLEN_384_58-1) {t += (dchunk)v[k]*md[i-k]; k++;} - a[i - NLEN_384_58] = (chunk)t & BMASK_384_58; - t = (t >> BASEBITS_384_58) + d[i + 1]; - } - a[NLEN_384_58 - 1] = (chunk)t & BMASK_384_58; -#else - t = d[0]; - v[0] = ((chunk)t * MC)&BMASK_384_58; - t += (dchunk)v[0] * md[0]; - c = (t >> BASEBITS_384_58) + d[1]; - s = 0; - - for (k = 1; k < NLEN_384_58; k++) - { - t = c + s + (dchunk)v[0] * md[k]; - for (i=1+k/2;i> BASEBITS_384_58) + d[k + 1]; - dd[k] = (dchunk)v[k] * md[k]; - s += dd[k]; - } - for (k = NLEN_384_58; k < 2 * NLEN_384_58 - 1; k++) - { - t = c + s; - for (i=1+k/2;i> BASEBITS_384_58) + d[k + 1]; - s -= dd[k - NLEN_384_58 + 1]; - } - a[NLEN_384_58 - 1] = (chunk)c & BMASK_384_58; - -#endif -#endif - - -#else - int j; - chunk m, carry; - for (i = 0; i < NLEN_384_58; i++) - { - if (MC == -1) m = (-d[i])&BMASK_384_58; - else - { - if (MC == 1) m = d[i]; - else m = (MC * d[i])&BMASK_384_58; - } - carry = 0; - for (j = 0; j < NLEN_384_58; j++) - carry = muladd_384_58(m, md[j], carry, &d[i + j]); - d[NLEN_384_58 + i] += carry; - } - BIG_384_58_sducopy(a, d); - BIG_384_58_norm(a); - -#endif - -#ifdef DEBUG_NORM - a[MPV_384_58] = 1; - a[MNV_384_58] = 0; -#endif -} - -/* General shift left of a by n bits */ -/* a MUST be normalised */ -/* SU= 32 */ -void BIG_384_58_shl(BIG_384_58 a, int k) -{ - int i; - int n = k % BASEBITS_384_58; - int m = k / BASEBITS_384_58; - - a[NLEN_384_58 - 1] = ((a[NLEN_384_58 - 1 - m] << n)); - if (NLEN_384_58 >= m + 2) a[NLEN_384_58 - 1] |= (a[NLEN_384_58 - m - 2] >> (BASEBITS_384_58 - n)); - - for (i = NLEN_384_58 - 2; i > m; i--) - a[i] = ((a[i - m] << n)&BMASK_384_58) | (a[i - m - 1] >> (BASEBITS_384_58 - n)); - a[m] = (a[0] << n)&BMASK_384_58; - for (i = 0; i < m; i++) a[i] = 0; - -} - -/* Fast shift left of a by n bits, where n less than a word, Return excess (but store it as well) */ -/* a MUST be normalised */ -/* SU= 16 */ -int BIG_384_58_fshl(BIG_384_58 a, int n) -{ - int i; - - a[NLEN_384_58 - 1] = ((a[NLEN_384_58 - 1] << n)) | (a[NLEN_384_58 - 2] >> (BASEBITS_384_58 - n)); /* top word not masked */ - for (i = NLEN_384_58 - 2; i > 0; i--) - a[i] = ((a[i] << n)&BMASK_384_58) | (a[i - 1] >> (BASEBITS_384_58 - n)); - a[0] = (a[0] << n)&BMASK_384_58; - - return (int)(a[NLEN_384_58 - 1] >> ((8 * MODBYTES_384_58) % BASEBITS_384_58)); /* return excess - only used in ff.c */ -} - -/* double length left shift of a by k bits - k can be > BASEBITS , a MUST be normalised */ -/* SU= 32 */ -void BIG_384_58_dshl(DBIG_384_58 a, int k) -{ - int i; - int n = k % BASEBITS_384_58; - int m = k / BASEBITS_384_58; - - a[DNLEN_384_58 - 1] = ((a[DNLEN_384_58 - 1 - m] << n)) | (a[DNLEN_384_58 - m - 2] >> (BASEBITS_384_58 - n)); - - for (i = DNLEN_384_58 - 2; i > m; i--) - a[i] = ((a[i - m] << n)&BMASK_384_58) | (a[i - m - 1] >> (BASEBITS_384_58 - n)); - a[m] = (a[0] << n)&BMASK_384_58; - for (i = 0; i < m; i++) a[i] = 0; - -} - -/* General shift right of a by k bits */ -/* a MUST be normalised */ -/* SU= 32 */ -void BIG_384_58_shr(BIG_384_58 a, int k) -{ - int i; - int n = k % BASEBITS_384_58; - int m = k / BASEBITS_384_58; - for (i = 0; i < NLEN_384_58 - m - 1; i++) - a[i] = (a[m + i] >> n) | ((a[m + i + 1] << (BASEBITS_384_58 - n))&BMASK_384_58); - if (NLEN_384_58 > m) a[NLEN_384_58 - m - 1] = a[NLEN_384_58 - 1] >> n; - for (i = NLEN_384_58 - m; i < NLEN_384_58; i++) a[i] = 0; - -} - -/* Fast combined shift, subtract and norm. Return sign of result */ -int BIG_384_58_ssn(BIG_384_58 r, BIG_384_58 a, BIG_384_58 m) -{ - int i, n = NLEN_384_58 - 1; - chunk carry; - m[0] = (m[0] >> 1) | ((m[1] << (BASEBITS_384_58 - 1))&BMASK_384_58); - r[0] = a[0] - m[0]; - carry = r[0] >> BASEBITS_384_58; - r[0] &= BMASK_384_58; - - for (i = 1; i < n; i++) - { - m[i] = (m[i] >> 1) | ((m[i + 1] << (BASEBITS_384_58 - 1))&BMASK_384_58); - r[i] = a[i] - m[i] + carry; - carry = r[i] >> BASEBITS_384_58; - r[i] &= BMASK_384_58; - } - - m[n] >>= 1; - r[n] = a[n] - m[n] + carry; -#ifdef DEBUG_NORM - r[MPV_384_58] = 1; - r[MNV_384_58] = 0; -#endif - return ((r[n] >> (CHUNK - 1)) & 1); -} - -/* Faster shift right of a by k bits. Return shifted out part */ -/* a MUST be normalised */ -/* SU= 16 */ -int BIG_384_58_fshr(BIG_384_58 a, int k) -{ - int i; - chunk r = a[0] & (((chunk)1 << k) - 1); /* shifted out part */ - for (i = 0; i < NLEN_384_58 - 1; i++) - a[i] = (a[i] >> k) | ((a[i + 1] << (BASEBITS_384_58 - k))&BMASK_384_58); - a[NLEN_384_58 - 1] = a[NLEN_384_58 - 1] >> k; - return (int)r; -} - -/* double length right shift of a by k bits - can be > BASEBITS */ -/* SU= 32 */ -void BIG_384_58_dshr(DBIG_384_58 a, int k) -{ - int i; - int n = k % BASEBITS_384_58; - int m = k / BASEBITS_384_58; - for (i = 0; i < DNLEN_384_58 - m - 1; i++) - a[i] = (a[m + i] >> n) | ((a[m + i + 1] << (BASEBITS_384_58 - n))&BMASK_384_58); - a[DNLEN_384_58 - m - 1] = a[DNLEN_384_58 - 1] >> n; - for (i = DNLEN_384_58 - m; i < DNLEN_384_58; i++ ) a[i] = 0; -} - -/* Split DBIG d into two BIGs t|b. Split happens at n bits, where n falls into NLEN word */ -/* d MUST be normalised */ -/* SU= 24 */ -chunk BIG_384_58_split(BIG_384_58 t, BIG_384_58 b, DBIG_384_58 d, int n) -{ - int i; - chunk nw, carry = 0; - int m = n % BASEBITS_384_58; - - if (m == 0) - { - for (i = 0; i < NLEN_384_58; i++) b[i] = d[i]; - if (t != b) - { - for (i = NLEN_384_58; i < 2 * NLEN_384_58; i++) t[i - NLEN_384_58] = d[i]; - carry = t[NLEN_384_58 - 1] >> BASEBITS_384_58; - t[NLEN_384_58 - 1] = t[NLEN_384_58 - 1] & BMASK_384_58; /* top word normalized */ - } - return carry; - } - - for (i = 0; i < NLEN_384_58 - 1; i++) b[i] = d[i]; - - b[NLEN_384_58 - 1] = d[NLEN_384_58 - 1] & (((chunk)1 << m) - 1); - - if (t != b) - { - carry = (d[DNLEN_384_58 - 1] << (BASEBITS_384_58 - m)); - for (i = DNLEN_384_58 - 2; i >= NLEN_384_58 - 1; i--) - { - nw = (d[i] >> m) | carry; - carry = (d[i] << (BASEBITS_384_58 - m))&BMASK_384_58; - t[i - NLEN_384_58 + 1] = nw; - } - } -#ifdef DEBUG_NORM - t[MPV_384_58] = 1; - t[MNV_384_58] = 0; - b[MPV_384_58] = 1; - b[MNV_384_58] = 0; -#endif - return carry; -} - -/* you gotta keep the sign of carry! Look - no branching! */ -/* Note that sign bit is needed to disambiguate between +ve and -ve values */ -/* normalise BIG - force all digits < 2^BASEBITS */ -chunk BIG_384_58_norm(BIG_384_58 a) -{ - int i; - chunk d, carry; - - carry=a[0]>>BASEBITS_384_58; - a[0]&=BMASK_384_58; - - for (i = 1; i < NLEN_384_58 - 1; i++) - { - d = a[i] + carry; - a[i] = d & BMASK_384_58; - carry = d >> BASEBITS_384_58; - } - a[NLEN_384_58 - 1] = (a[NLEN_384_58 - 1] + carry); - -#ifdef DEBUG_NORM - a[MPV_384_58] = 1; - a[MNV_384_58] = 0; -#endif - return (a[NLEN_384_58 - 1] >> ((8 * MODBYTES_384_58) % BASEBITS_384_58)); /* only used in ff.c */ -} - -void BIG_384_58_dnorm(DBIG_384_58 a) -{ - int i; - chunk d, carry; - - carry=a[0]>>BASEBITS_384_58; - a[0]&=BMASK_384_58; - - for (i = 1; i < DNLEN_384_58 - 1; i++) - { - d = a[i] + carry; - a[i] = d & BMASK_384_58; - carry = d >> BASEBITS_384_58; - } - a[DNLEN_384_58 - 1] = (a[DNLEN_384_58 - 1] + carry); -#ifdef DEBUG_NORM - a[DMPV_384_58] = 1; - a[DMNV_384_58] = 0; -#endif -} - -/* Compare a and b. Return 1 for a>b, -1 for a=0; i--) - { - gt |= ((b[i]-a[i]) >> BASEBITS_384_58) & eq; - eq &= ((b[i]^a[i])-1) >> BASEBITS_384_58; - } - return (int)(gt+gt+eq-1); -} - -int BIG_384_58_dcomp(DBIG_384_58 a, DBIG_384_58 b) -{ - int i; - chunk gt=0; chunk eq=1; - for (i = DNLEN_384_58-1; i>=0; i--) - { - gt |= ((b[i]-a[i]) >> BASEBITS_384_58) & eq; - eq &= ((b[i]^a[i])-1) >> BASEBITS_384_58; - } - return (int)(gt+gt+eq-1); -} - -/* return number of bits in a */ -/* SU= 8 */ -int BIG_384_58_nbits(BIG_384_58 a) -{ - int bts, k = NLEN_384_58 - 1; - BIG_384_58 t; - chunk c; - BIG_384_58_copy(t, a); - BIG_384_58_norm(t); - while (k >= 0 && t[k] == 0) k--; - if (k < 0) return 0; - bts = BASEBITS_384_58 * k; - c = t[k]; - while (c != 0) - { - c /= 2; - bts++; - } - return bts; -} - -/* SU= 8, Calculate number of bits in a DBIG - output normalised */ -int BIG_384_58_dnbits(DBIG_384_58 a) -{ - int bts, k = DNLEN_384_58 - 1; - DBIG_384_58 t; - chunk c; - BIG_384_58_dcopy(t, a); - BIG_384_58_dnorm(t); - while (k >= 0 && t[k] == 0) k--; - if (k < 0) return 0; - bts = BASEBITS_384_58 * k; - c = t[k]; - while (c != 0) - { - c /= 2; - bts++; - } - return bts; -} - - -/* Set b=b mod c */ -/* SU= 16 */ -void BIG_384_58_mod(BIG_384_58 b, BIG_384_58 c1) -{ - int k = 0; - BIG_384_58 r; /**/ - BIG_384_58 c; - BIG_384_58_copy(c, c1); - - BIG_384_58_norm(b); - if (BIG_384_58_comp(b, c) < 0) - return; - do - { - BIG_384_58_fshl(c, 1); - k++; - } - while (BIG_384_58_comp(b, c) >= 0); - - while (k > 0) - { - BIG_384_58_fshr(c, 1); - -// constant time... - BIG_384_58_sub(r, b, c); - BIG_384_58_norm(r); - BIG_384_58_cmove(b, r, 1 - ((r[NLEN_384_58 - 1] >> (CHUNK - 1)) & 1)); - k--; - } -} - -/* Set a=b mod c, b is destroyed. Slow but rarely used. */ -/* SU= 96 */ -void BIG_384_58_dmod(BIG_384_58 a, DBIG_384_58 b, BIG_384_58 c) -{ - int k = 0; - DBIG_384_58 m, r; - BIG_384_58_dnorm(b); - BIG_384_58_dscopy(m, c); - - if (BIG_384_58_dcomp(b, m) < 0) - { - BIG_384_58_sdcopy(a, b); - return; - } - - do - { - BIG_384_58_dshl(m, 1); - k++; - } - while (BIG_384_58_dcomp(b, m) >= 0); - - while (k > 0) - { - BIG_384_58_dshr(m, 1); -// constant time... - BIG_384_58_dsub(r, b, m); - BIG_384_58_dnorm(r); - BIG_384_58_dcmove(b, r, 1 - ((r[DNLEN_384_58 - 1] >> (CHUNK - 1)) & 1)); - - k--; - } - BIG_384_58_sdcopy(a, b); -} - -/* Set a=b/c, b is destroyed. Slow but rarely used. */ -/* SU= 136 */ - -void BIG_384_58_ddiv(BIG_384_58 a, DBIG_384_58 b, BIG_384_58 c) -{ - int d, k = 0; - DBIG_384_58 m, dr; - BIG_384_58 e, r; - BIG_384_58_dnorm(b); - BIG_384_58_dscopy(m, c); - - BIG_384_58_zero(a); - BIG_384_58_zero(e); - BIG_384_58_inc(e, 1); - - while (BIG_384_58_dcomp(b, m) >= 0) - { - BIG_384_58_fshl(e, 1); - BIG_384_58_dshl(m, 1); - k++; - } - - while (k > 0) - { - BIG_384_58_dshr(m, 1); - BIG_384_58_fshr(e, 1); - - BIG_384_58_dsub(dr, b, m); - BIG_384_58_dnorm(dr); - d = 1 - ((dr[DNLEN_384_58 - 1] >> (CHUNK - 1)) & 1); - BIG_384_58_dcmove(b, dr, d); - - BIG_384_58_add(r, a, e); - BIG_384_58_norm(r); - BIG_384_58_cmove(a, r, d); - - k--; - } -} - -/* SU= 136 */ - -void BIG_384_58_sdiv(BIG_384_58 a, BIG_384_58 c) -{ - int d, k = 0; - BIG_384_58 m, e, b, r; - BIG_384_58_norm(a); - BIG_384_58_copy(b, a); - BIG_384_58_copy(m, c); - - BIG_384_58_zero(a); - BIG_384_58_zero(e); - BIG_384_58_inc(e, 1); - - while (BIG_384_58_comp(b, m) >= 0) - { - BIG_384_58_fshl(e, 1); - BIG_384_58_fshl(m, 1); - k++; - } - - while (k > 0) - { - BIG_384_58_fshr(m, 1); - BIG_384_58_fshr(e, 1); - - BIG_384_58_sub(r, b, m); - BIG_384_58_norm(r); - d = 1 - ((r[NLEN_384_58 - 1] >> (CHUNK - 1)) & 1); - BIG_384_58_cmove(b, r, d); - - BIG_384_58_add(r, a, e); - BIG_384_58_norm(r); - BIG_384_58_cmove(a, r, d); - k--; - } -} - -/* return LSB of a */ -int BIG_384_58_parity(BIG_384_58 a) -{ - return a[0] % 2; -} - -/* return n-th bit of a */ -/* SU= 16 */ -int BIG_384_58_bit(BIG_384_58 a, int n) -{ - if (a[n / BASEBITS_384_58] & ((chunk)1 << (n % BASEBITS_384_58))) return 1; - else return 0; -} - -/* return last n bits of a, where n is small < BASEBITS */ -/* SU= 16 */ -int BIG_384_58_lastbits(BIG_384_58 a, int n) -{ - int msk = (1 << n) - 1; - BIG_384_58_norm(a); - return ((int)a[0])&msk; -} - -/* get 8*MODBYTES size random number */ -void BIG_384_58_random(BIG_384_58 m, csprng *rng) -{ - int i, b, j = 0, r = 0; - int len = 8 * MODBYTES_384_58; - - BIG_384_58_zero(m); - /* generate random BIG */ - for (i = 0; i < len; i++) - { - if (j == 0) r = RAND_byte(rng); - else r >>= 1; - b = r & 1; - BIG_384_58_shl(m, 1); - m[0] += b; - j++; - j &= 7; - } - -#ifdef DEBUG_NORM - m[MPV_384_58] = 1; - m[MNV_384_58] = 0; -#endif -} - -/* get random BIG from rng, modulo q. Done one bit at a time, so its portable */ - -void BIG_384_58_randomnum(BIG_384_58 m, BIG_384_58 q, csprng *rng) -{ - int i, b, j = 0, r = 0; - DBIG_384_58 d; - BIG_384_58_dzero(d); - /* generate random DBIG */ - for (i = 0; i < 2 * BIG_384_58_nbits(q); i++) - { - if (j == 0) r = RAND_byte(rng); - else r >>= 1; - b = r & 1; - BIG_384_58_dshl(d, 1); - d[0] += b; - j++; - j &= 7; - } - /* reduce modulo a BIG. Removes bias */ - BIG_384_58_dmod(m, d, q); -#ifdef DEBUG_NORM - m[MPV_384_58] = 1; - m[MNV_384_58] = 0; -#endif -} - -/* create randum BIG less than r and less than trunc bits */ -void BIG_384_58_randtrunc(BIG_384_58 s, BIG_384_58 r, int trunc, csprng *rng) -{ - BIG_384_58_randomnum(s, r, rng); - if (BIG_384_58_nbits(r) > trunc) - BIG_384_58_mod2m(s, trunc); -} - - -/* Set r=a*b mod m */ -/* SU= 96 */ -void BIG_384_58_modmul(BIG_384_58 r, BIG_384_58 a1, BIG_384_58 b1, BIG_384_58 m) -{ - DBIG_384_58 d; - BIG_384_58 a, b; - BIG_384_58_copy(a, a1); - BIG_384_58_copy(b, b1); - BIG_384_58_mod(a, m); - BIG_384_58_mod(b, m); - - BIG_384_58_mul(d, a, b); - BIG_384_58_dmod(r, d, m); -} - -/* Set a=a*a mod m */ -/* SU= 88 */ -void BIG_384_58_modsqr(BIG_384_58 r, BIG_384_58 a1, BIG_384_58 m) -{ - DBIG_384_58 d; - BIG_384_58 a; - BIG_384_58_copy(a, a1); - BIG_384_58_mod(a, m); - BIG_384_58_sqr(d, a); - BIG_384_58_dmod(r, d, m); -} - -/* Set r=-a mod m */ -/* SU= 16 */ -void BIG_384_58_modneg(BIG_384_58 r, BIG_384_58 a1, BIG_384_58 m) -{ - BIG_384_58 a; - BIG_384_58_copy(a, a1); - BIG_384_58_mod(a, m); - BIG_384_58_sub(r, m, a); - BIG_384_58_mod(r, m); -} - -/* Set r=a+b mod m */ -void BIG_384_58_modadd(BIG_384_58 r, BIG_384_58 a1, BIG_384_58 b1, BIG_384_58 m) -{ - BIG_384_58 a, b; - BIG_384_58_copy(a, a1); - BIG_384_58_copy(b, b1); - BIG_384_58_mod(a, m); - BIG_384_58_mod(b, m); - BIG_384_58_add(r,a,b); BIG_384_58_norm(r); - BIG_384_58_mod(r,m); -} - -/* Set a=a/b mod m */ -/* SU= 136 */ -void BIG_384_58_moddiv(BIG_384_58 r, BIG_384_58 a1, BIG_384_58 b1, BIG_384_58 m) -{ - DBIG_384_58 d; - BIG_384_58 z; - BIG_384_58 a, b; - BIG_384_58_copy(a, a1); - BIG_384_58_copy(b, b1); - - BIG_384_58_mod(a, m); - BIG_384_58_invmodp(z, b, m); - - BIG_384_58_mul(d, a, z); - BIG_384_58_dmod(r, d, m); -} - -/* Get jacobi Symbol (a/p). Returns 0, 1 or -1 */ -/* SU= 216 */ -int BIG_384_58_jacobi(BIG_384_58 a, BIG_384_58 p) -{ - int n8, k, m = 0; - BIG_384_58 t, x, n, zilch, one; - BIG_384_58_one(one); - BIG_384_58_zero(zilch); - if (BIG_384_58_parity(p) == 0 || BIG_384_58_comp(a, zilch) == 0 || BIG_384_58_comp(p, one) <= 0) return 0; - BIG_384_58_norm(a); - BIG_384_58_copy(x, a); - BIG_384_58_copy(n, p); - BIG_384_58_mod(x, p); - - while (BIG_384_58_comp(n, one) > 0) - { - if (BIG_384_58_comp(x, zilch) == 0) return 0; - n8 = BIG_384_58_lastbits(n, 3); - k = 0; - while (BIG_384_58_parity(x) == 0) - { - k++; - BIG_384_58_shr(x, 1); - } - if (k % 2 == 1) m += (n8 * n8 - 1) / 8; - m += (n8 - 1) * (BIG_384_58_lastbits(x, 2) - 1) / 4; - BIG_384_58_copy(t, n); - - BIG_384_58_mod(t, x); - BIG_384_58_copy(n, x); - BIG_384_58_copy(x, t); - m %= 2; - - } - if (m == 0) return 1; - else return -1; -} - -/* Set r=1/a mod p. Binary method */ -/* SU= 240 */ -void BIG_384_58_invmodp(BIG_384_58 r, BIG_384_58 a, BIG_384_58 p) -{ - BIG_384_58 u, v, x1, x2, t, one; - - BIG_384_58_mod(a, p); - if (BIG_384_58_iszilch(a)) - { - BIG_384_58_zero(r); - return; - } - - BIG_384_58_copy(u, a); - BIG_384_58_copy(v, p); - BIG_384_58_one(one); - BIG_384_58_copy(x1, one); - BIG_384_58_zero(x2); - - while (BIG_384_58_comp(u, one) != 0 && BIG_384_58_comp(v, one) != 0) - { - while (BIG_384_58_parity(u) == 0) - { - BIG_384_58_fshr(u, 1); - BIG_384_58_add(t,x1,p); - BIG_384_58_cmove(x1,t,BIG_384_58_parity(x1)); - BIG_384_58_norm(x1); - BIG_384_58_fshr(x1,1); - } - while (BIG_384_58_parity(v) == 0) - { - BIG_384_58_fshr(v, 1); - BIG_384_58_add(t,x2,p); - BIG_384_58_cmove(x2,t,BIG_384_58_parity(x2)); - BIG_384_58_norm(x2); - BIG_384_58_fshr(x2,1); - } - if (BIG_384_58_comp(u, v) >= 0) - { - BIG_384_58_sub(u, u, v); - BIG_384_58_norm(u); - BIG_384_58_add(t,x1,p); - BIG_384_58_cmove(x1,t,(BIG_384_58_comp(x1,x2)>>1)&1); /* move if x1>1)&1); /* move if x2 -#include -#include -#include "arch.h" -#include "core.h" -#include "config_big_384_58.h" - -/* could comment this out if code size is a major issue */ -#define UNWOUND /**< Default to unwound code */ -/* Normally recommended, but may not be optimal for some architectures, for example 32-bit ARM M4 */ -#define USE_KARATSUBA /**< Default to use Karatsuba method */ - -#define BIGBITS_384_58 (8*MODBYTES_384_58) /**< Length in bits */ -#define NLEN_384_58 (1+((8*MODBYTES_384_58-1)/BASEBITS_384_58)) /**< length in bytes */ -#define DNLEN_384_58 2*NLEN_384_58 /**< Double length in bytes */ -#define BMASK_384_58 (((chunk)1<y - */ -extern int BIG_384_58_comp(BIG_384_58 x, BIG_384_58 y); -/** @brief Compares two DBIG numbers. Inputs must be normalised externally (Constant Time) - * - @param x first DBIG number to be compared - @param y second DBIG number to be compared - @return -1 is xy - */ -extern int BIG_384_58_dcomp(DBIG_384_58 x, DBIG_384_58 y); -/** @brief Calculate number of bits in a BIG - output normalised (Variable Time) - * - @param x BIG number - @return Number of bits in x - */ -extern int BIG_384_58_nbits(BIG_384_58 x); -/** @brief Calculate number of bits in a DBIG - output normalised (Variable Time) - * - @param x DBIG number - @return Number of bits in x - */ -extern int BIG_384_58_dnbits(DBIG_384_58 x); -/** @brief Reduce x mod n - input and output normalised (Variable Time) - * - Slow but rarely used - @param x BIG number to be reduced mod n - @param n The modulus - */ -extern void BIG_384_58_mod(BIG_384_58 x, BIG_384_58 n); -/** @brief Divide x by n - output normalised (Variable Time) - * - Slow but rarely used - @param x BIG number to be divided by n - @param n The Divisor - */ -extern void BIG_384_58_sdiv(BIG_384_58 x, BIG_384_58 n); -/** @brief x=y mod n - output normalised (Variable Time) - * - Slow but rarely used. y is destroyed. - @param x BIG number, on exit = y mod n - @param y DBIG number - @param n Modulus - */ -extern void BIG_384_58_dmod(BIG_384_58 x, DBIG_384_58 y, BIG_384_58 n); -/** @brief x=y/n - output normalised (Variable Time) - * - Slow but rarely used. y is destroyed. - @param x BIG number, on exit = y/n - @param y DBIG number - @param n Modulus - */ -extern void BIG_384_58_ddiv(BIG_384_58 x, DBIG_384_58 y, BIG_384_58 n); -/** @brief return parity of BIG, that is the least significant bit (Constant Time) - * - @param x BIG number - @return 0 or 1 - */ -extern int BIG_384_58_parity(BIG_384_58 x); -/** @brief return i-th of BIG (Constant Time) - * - @param x BIG number - @param i the bit of x to be returned - @return 0 or 1 - */ -extern int BIG_384_58_bit(BIG_384_58 x, int i); -/** @brief return least significant bits of a BIG (Constant Time) - * - @param x BIG number - @param n number of bits to return. Assumed to be less than BASEBITS. - @return least significant n bits as an integer - */ -extern int BIG_384_58_lastbits(BIG_384_58 x, int n); -/** @brief Create a random BIG from a random number generator (Constant Time) - * - Assumes that the random number generator has been suitably initialised - @param x BIG number, on exit a random number - @param r A pointer to a Cryptographically Secure Random Number Generator - */ -extern void BIG_384_58_random(BIG_384_58 x, csprng *r); -/** @brief Create an unbiased random BIG from a random number generator, reduced with respect to a modulus (Constant Time as used) - * - Assumes that the random number generator has been suitably initialised - @param x BIG number, on exit a random number - @param n The modulus - @param r A pointer to a Cryptographically Secure Random Number Generator - */ -extern void BIG_384_58_randomnum(BIG_384_58 x, BIG_384_58 n, csprng *r); - -/** @brief Create an unbiased random BIG from a random number generator, reduced with respect to a modulus and truncated to max bit length (Constant Time as used) - * - Assumes that the random number generator has been suitably initialised - @param x BIG number, on exit a random number - @param n The modulus - @param t Maximum bit length - @param r A pointer to a Cryptographically Secure Random Number Generator - */ -extern void BIG_384_58_randtrunc(BIG_384_58 x, BIG_384_58 n, int t, csprng *r); - -/** brief return NAF (Non-Adjacent-Form) value as +/- 1, 3 or 5, inputs must be normalised - * - Given x and 3*x extracts NAF value from given bit position, and returns number of bits processed, and number of trailing zeros detected if any - param x BIG number - param x3 BIG number, three times x - param i bit position - param nbs pointer to integer returning number of bits processed - param nzs pointer to integer returning number of trailing 0s - return + or - 1, 3 or 5 -*/ - -/** @brief Calculate x=y*z mod n (Variable Time) - * - Slow method for modular multiplication - @param x BIG number, on exit = y*z mod n - @param y BIG number - @param z BIG number - @param n The BIG Modulus - */ -extern void BIG_384_58_modmul(BIG_384_58 x, BIG_384_58 y, BIG_384_58 z, BIG_384_58 n); -/** @brief Calculate x=y/z mod n (Variable Time) - * - Slow method for modular division - @param x BIG number, on exit = y/z mod n - @param y BIG number - @param z BIG number - @param n The BIG Modulus - */ -extern void BIG_384_58_moddiv(BIG_384_58 x, BIG_384_58 y, BIG_384_58 z, BIG_384_58 n); -/** @brief Calculate x=y^2 mod n (Variable Time) - * - Slow method for modular squaring - @param x BIG number, on exit = y^2 mod n - @param y BIG number - @param n The BIG Modulus - */ -extern void BIG_384_58_modsqr(BIG_384_58 x, BIG_384_58 y, BIG_384_58 n); -/** @brief Calculate x=-y mod n (Variable Time) - * - Modular negation - @param x BIG number, on exit = -y mod n - @param y BIG number - @param n The BIG Modulus - */ -extern void BIG_384_58_modneg(BIG_384_58 x, BIG_384_58 y, BIG_384_58 n); - -/** @brief Calculate x=y+z mod n (Variable Time) - * - Slow method for modular addition - @param x BIG number, on exit = y+z mod n - @param y BIG number - @param z BIG number - @param n The BIG Modulus - */ -extern void BIG_384_58_modadd(BIG_384_58 x, BIG_384_58 y, BIG_384_58 z, BIG_384_58 n); - -/** @brief Calculate jacobi Symbol (x/y) (Variable Time) - * - @param x BIG number - @param y BIG number - @return Jacobi symbol, -1,0 or 1 - */ -extern int BIG_384_58_jacobi(BIG_384_58 x, BIG_384_58 y); -/** @brief Calculate x=1/y mod n (Variable Time) - * - Modular Inversion - This is slow. Uses binary method. - @param x BIG number, on exit = 1/y mod n - @param y BIG number - @param n The BIG Modulus - */ -extern void BIG_384_58_invmodp(BIG_384_58 x, BIG_384_58 y, BIG_384_58 n); -/** @brief Calculate x=x mod 2^m (Variable Time) - * - Truncation - @param x BIG number, on reduced mod 2^m - @param m new truncated size -*/ -extern void BIG_384_58_mod2m(BIG_384_58 x, int m); - -/** @brief Calculates a*b+c+*d (Constant Time) - * - Calculate partial product of a.b, add in carry c, and add total to d - @param x multiplier - @param y multiplicand - @param c carry - @param r pointer to accumulated bottom half of result - @return top half of result - */ - -#ifdef dchunk - -/* Method required to calculate x*y+c+r, bottom half in r, top half returned */ -static inline chunk muladd_384_58(chunk x, chunk y, chunk c, chunk *r) -{ - dchunk prod = (dchunk)x * y + c + *r; - *r = (chunk)prod & BMASK_384_58; - return (chunk)(prod >> BASEBITS_384_58); -} - -#else - -/* No integer type available that can store double the wordlength */ -/* accumulate partial products */ - -static inline chunk muladd_384_58(chunk x, chunk y, chunk c, chunk *r) -{ - chunk x0, x1, y0, y1; - chunk bot, top, mid, carry; - x0 = x & HMASK_384_58; - x1 = (x >> HBITS_384_58); - y0 = y & HMASK_384_58; - y1 = (y >> HBITS_384_58); - bot = x0 * y0; - top = x1 * y1; - mid = x0 * y1 + x1 * y0; - x0 = mid & HMASK_384_58; - x1 = (mid >> HBITS_384_58); - bot += x0 << HBITS_384_58; - bot += *r; - bot += c; - - top += x1; - carry = bot >> BASEBITS_384_58; - bot &= BMASK_384_58; - top += carry; - - *r = bot; - return top; -} - -#endif - - -#endif diff --git a/impl/cbits/bls_BLS12381.c b/impl/cbits/bls_BLS12381.c deleted file mode 100644 index 418a8dba7..000000000 --- a/impl/cbits/bls_BLS12381.c +++ /dev/null @@ -1,176 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/* Boneh-Lynn-Shacham signature 128-bit API */ - -/* Loosely (for now) following https://datatracker.ietf.org/doc/html/draft-irtf-cfrg-bls-signature-02 */ - -// Minimal-signature-size variant - -#include -#include -#include -#include "bls_BLS12381.h" - - -static FP4_BLS12381 G2_TAB[G2_TABLE_BLS12381]; // space for precomputation on fixed G2 parameter - -#define CEIL(a,b) (((a)-1)/(b)+1) - -/* output u[i] \in F_p */ -/* https://datatracker.ietf.org/doc/draft-irtf-cfrg-hash-to-curve/ */ -static void hash_to_field(int hash,int hlen,FP_BLS12381 *u,octet *DST,octet *M, int ctr) -{ - int i,j,L; - BIG_384_58 q,w; - DBIG_384_58 dx; - char okm[256],fd[128]; - octet OKM = {0,sizeof(okm),okm}; - - BIG_384_58_rcopy(q, Modulus_BLS12381); - L=CEIL(BIG_384_58_nbits(q)+CURVE_SECURITY_BLS12381,8); - - XMD_Expand(hash,hlen,&OKM,L*ctr,DST,M); - for (i=0;ival, s); - S->len = MODBYTES_384_58; - -// SkToPk - - PAIR_BLS12381_G2mul(&G, s); - //ECP2_BLS12381_toOctet(W, &G, true); - ECP2_BLS12381_toOctet_ZCash(W, &G); - return BLS_OK; -} - -/* Sign message M using private key S to produce signature SIG */ -int BLS_BLS12381_CORE_SIGN(octet *SIG, octet *M, octet *S) -{ - BIG_384_58 s; - ECP_BLS12381 D; - BLS_HASH_TO_POINT(&D, M); - BIG_384_58_fromBytes(s, S->val); - PAIR_BLS12381_G1mul(&D, s); - //ECP_BLS12381_toOctet(SIG, &D, true); /* compress output */ - ECP_BLS12381_toOctet_ZCash(SIG, &D); /* compress output */ - return BLS_OK; -} - -/* Verify signature of message M, the signature SIG, and the public key W */ -int BLS_BLS12381_CORE_VERIFY(octet *SIG, octet *M, octet *W) -{ - FP12_BLS12381 v; - ECP2_BLS12381 G, PK; - ECP_BLS12381 D, HM; - BLS_HASH_TO_POINT(&HM, M); - - //if (!ECP_BLS12381_fromOctet(&D, SIG)) return BLS_FAIL; - if (!ECP_BLS12381_fromOctet_ZCash(&D, SIG)) return BLS_FAIL; - if (!PAIR_BLS12381_G1member(&D)) return BLS_FAIL; - ECP_BLS12381_neg(&D); - - //if (!ECP2_BLS12381_fromOctet(&PK, W)) return BLS_FAIL; - if (!ECP2_BLS12381_fromOctet_ZCash(&PK, W)) return BLS_FAIL; - if (!PAIR_BLS12381_G2member(&PK)) return BLS_FAIL; - -// Use new multi-pairing mechanism - -/* - FP12_BLS12381 r[ATE_BITS_BLS12381]; - PAIR_BLS12381_initmp(r); - PAIR_BLS12381_another_pc(r, G2_TAB, &D); - PAIR_BLS12381_another(r, &PK, &HM); - PAIR_BLS12381_miller(&v, r); -*/ - -//.. or alternatively - if (!ECP2_BLS12381_generator(&G)) return BLS_FAIL; - PAIR_BLS12381_double_ate(&v,&G,&D,&PK,&HM); - - PAIR_BLS12381_fexp(&v); - if (FP12_BLS12381_isunity(&v)) return BLS_OK; - return BLS_FAIL; -} - diff --git a/impl/cbits/bls_BLS12381.h b/impl/cbits/bls_BLS12381.h deleted file mode 100644 index 854c9e5e3..000000000 --- a/impl/cbits/bls_BLS12381.h +++ /dev/null @@ -1,81 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file bls.h - * @author Mike Scott - * @date 28th Novemebr 2018 - * @brief BLS Header file - * - * Allows some user configuration - * defines structures - * declares functions - * - */ - -#ifndef BLS_BLS12381_H -#define BLS_BLS12381_H - -#include "pair_BLS12381.h" - -/* Field size is assumed to be greater than or equal to group size */ - -#define BGS_BLS12381 MODBYTES_384_58 /**< BLS Group Size */ -#define BFS_BLS12381 MODBYTES_384_58 /**< BLS Field Size */ - -#define BLS_OK 0 /**< Function completed without error */ -#define BLS_FAIL -1 /**< Point is NOT on the curve */ - -/* BLS API functions */ - -/** @brief Initialise BLS - * - @return BLS_OK if worked, otherwise BLS_FAIL - */ -int BLS_BLS12381_INIT(); - -/** @brief Generate Key Pair - * - @param IKM is an octet containing random Initial Keying Material - @param S on output a private key - @param W on output a public key = S*G, where G is fixed generator - @return BLS_OK - */ -int BLS_BLS12381_KEY_PAIR_GENERATE(octet *IKM, octet* S, octet *W); - -/** @brief Calculate a signature - * - @param SIG the ouput signature - @param M is the message to be signed - @param S an input private key - @return BLS_OK - */ -int BLS_BLS12381_CORE_SIGN(octet *SIG, octet *M, octet *S); - -/** @brief Verify a signature - * - @param SIG an input signature - @param M is the message whose signature is to be verified. - @param W an public key - @return BLS_OK if verified, otherwise BLS_FAIL - */ -int BLS_BLS12381_CORE_VERIFY(octet *SIG, octet *M, octet *W); - -#endif - diff --git a/impl/cbits/config_big_384_58.h b/impl/cbits/config_big_384_58.h deleted file mode 100644 index 84829c1b6..000000000 --- a/impl/cbits/config_big_384_58.h +++ /dev/null @@ -1,38 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file config_big.h - * @author Mike Scott - * @brief Config BIG Header File - * - */ - -#ifndef CONFIG_BIG_384_58_H -#define CONFIG_BIG_384_58_H - -#include "core.h" - -// BIG stuff - -#define MODBYTES_384_58 48 /**< Number of bytes in Modulus */ -#define BASEBITS_384_58 58 /**< Numbers represented to base 2*BASEBITS */ - - -#endif diff --git a/impl/cbits/config_curve_BLS12381.h b/impl/cbits/config_curve_BLS12381.h deleted file mode 100644 index 737777b4c..000000000 --- a/impl/cbits/config_curve_BLS12381.h +++ /dev/null @@ -1,76 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file config_curve.h - * @author Mike Scott - * @brief Config Curve Header File - * - */ - -#ifndef CONFIG_CURVE_BLS12381_H -#define CONFIG_CURVE_BLS12381_H - -#include"core.h" -#include"config_field_BLS12381.h" - -// ECP stuff - -#define CURVETYPE_BLS12381 WEIERSTRASS /**< Define Curve Type */ -#define CURVE_A_BLS12381 0 /**< Curve A parameter */ -#define PAIRING_FRIENDLY_BLS12381 BLS12_CURVE /**< Is curve pairing-friendly */ -#define CURVE_SECURITY_BLS12381 128 /**< Curve security level in AES bits */ -#define HTC_ISO_BLS12381 11 /**< Use Isogenies for Hash to Curve */ - -#if PAIRING_FRIENDLY_BLS12381 != NOT_PF - -#define HTC_ISO_G2_BLS12381 3 /**< Use Isogenies for G2 Hash to Curve */ -//#define USE_GLV_BLS12381 /**< Note this method is patented (GLV), so maybe you want to comment this out */ -//#define USE_GS_G2_BLS12381 /**< Well we didn't patent it :) But may be covered by GLV patent :( */ -#define USE_GS_GT_BLS12381 /**< Not patented, so probably OK to always use this */ - -#define POSITIVEX 0 -#define NEGATIVEX 1 - -#define SEXTIC_TWIST_BLS12381 M_TYPE /**< Sextic Twist M or D type */ -#define SIGN_OF_X_BLS12381 NEGATIVEX /**< Sign of curve parameter */ - -#define ATE_BITS_BLS12381 65 /**< Number of Bits in curve parameter */ -#define G2_TABLE_BLS12381 69 /**< Size of table for pairing precomputation for fixed G2 */ - -#endif - -#if CURVE_SECURITY_BLS12381 == 128 -#define AESKEY_BLS12381 16 /**< Symmetric Key size - 128 bits */ -#define HASH_TYPE_BLS12381 SHA256 /**< Hash type */ -#endif - -#if CURVE_SECURITY_BLS12381 == 192 -#define AESKEY_BLS12381 24 /**< Symmetric Key size - 192 bits */ -#define HASH_TYPE_BLS12381 SHA384 /**< Hash type */ -#endif - -#if CURVE_SECURITY_BLS12381 == 256 -#define AESKEY_BLS12381 32 /**< Symmetric Key size - 256 bits */ -#define HASH_TYPE_BLS12381 SHA512 /**< Hash type */ -#endif - - - -#endif diff --git a/impl/cbits/config_field_BLS12381.h b/impl/cbits/config_field_BLS12381.h deleted file mode 100644 index 90f06a352..000000000 --- a/impl/cbits/config_field_BLS12381.h +++ /dev/null @@ -1,45 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file config_field.h - * @author Mike Scott - * @brief Config Curve Header File - * - */ - -#ifndef CONFIG_FIELD_BLS12381_H -#define CONFIG_FIELD_BLS12381_H - -#include"core.h" -#include "config_big_384_58.h" - -// FP stuff - -#define MBITS_BLS12381 381 /**< Modulus bits */ -#define PM1D2_BLS12381 1 /**< Largest m such that 2^m|(p-1) */ -#define MODTYPE_BLS12381 NOT_SPECIAL /**< Modulus type */ -#define MAXXES_BLS12381 25 /**< Maximum excess for lazy reduction */ -#define QNRI_BLS12381 0 /**< Small Quadratic Non-Residue */ -#define RIADZ_BLS12381 11 /**< Z for hash to Curve */ -#define RIADZG2A_BLS12381 -2 /**< real part of Z in G2 for Hash to Curve */ -#define RIADZG2B_BLS12381 -1 /**< imaginary part of Z in G2 for Hash to Curve */ -#define TOWER_BLS12381 NEGATOWER /**< Postive or Negative towering */ - -#endif diff --git a/impl/cbits/core.h b/impl/cbits/core.h deleted file mode 100644 index 631a141fd..000000000 --- a/impl/cbits/core.h +++ /dev/null @@ -1,789 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file core.h - * @author Mike Scott - * @brief Main Header File - * - */ - -#ifndef CORE_H -#define CORE_H - -#include -#include -#include -#include -#include "arch.h" - -#ifdef CMAKE -#define CORE_VERSION_MAJOR @CORE_VERSION_MAJOR@ /**< Major version of the library */ -#define CORE_VERSION_MINOR @CORE_VERSION_MINOR@ /**< Minor version of the library */ -#define CORE_VERSION_PATCH @CORE_VERSION_PATCH@ /**< Patch version of the library */ -#define OS "@OS@" /**< Build OS */ -#endif - -/* modulus types */ - -#define NOT_SPECIAL 0 /**< Modulus of no exploitable form */ -#define PSEUDO_MERSENNE 1 /**< Pseudo-mersenne modulus of form $2^n-c$ */ -#define MONTGOMERY_FRIENDLY 3 /**< Montgomery Friendly modulus of form $2^a(2^b-c)-1$ */ -#define GENERALISED_MERSENNE 2 /**< Generalised-mersenne modulus of form $2^n-2^m-1$, GOLDILOCKS only */ - - -/* Curve types */ - -#define WEIERSTRASS 0 /**< Short Weierstrass form curve */ -#define EDWARDS 1 /**< Edwards or Twisted Edwards curve */ -#define MONTGOMERY 2 /**< Montgomery form curve */ - -/* Pairing-Friendly types */ - -#define NOT_PF 0 /**< Not a pairing friendly curve */ -#define BN_CURVE 1 /**< BN pairing-friendy curve */ -#define BLS12_CURVE 2 /**< BLS12 pairing-friendy curve */ -#define BLS24_CURVE 3 /**< BLS24 pairing-friendy curve */ -#define BLS48_CURVE 4 /**< BLS48 pairing-friendy curve */ - - -#define D_TYPE 0 /**< D-Type pairing-friendy curve */ -#define M_TYPE 1 /**< M-Type pairing-friendy curve */ - -#define FP_ZILCH 0 /**< FP extension is zero */ -#define FP_UNITY 1 /**< FP extension is one */ -#define FP_SPARSEST 2 /**< FP extension is sparsest */ -#define FP_SPARSER 3 /**< FP extension is sparser */ -#define FP_SPARSE 4 /**< FP extension is sparse */ -#define FP_DENSE 5 /**< FP extension is dense */ - -#define NEGATOWER 0 /**< Negative towering */ -#define POSITOWER 1 /**< Positive towering */ - -/** - * @brief SHA256 hash function instance */ -typedef struct -{ - unsign32 length[2]; /**< 64-bit input length */ - unsign32 h[8]; /**< Internal state */ - unsign32 w[80]; /**< Internal state */ - int hlen; /**< Hash length in bytes */ -} hash256; - -/** - * @brief SHA384-512 hash function instance */ -typedef struct -{ - unsign64 length[2]; /**< 64-bit input length */ - unsign64 h[8]; /**< Internal state */ - unsign64 w[80]; /**< Internal state */ - int hlen; /**< Hash length in bytes */ -} hash512; - -/** - * @brief SHA384 hash function instance */ -typedef hash512 hash384; - -/** - * @brief SHA3 hash function instance */ -typedef struct -{ - unsign64 length; /**< 64-bit input length */ - unsign64 S[5][5]; /**< Internal state */ - int rate; /**< TODO */ - int len; /**< Hash length in bytes */ -} sha3; - -#define MC_SHA2 2 /**< SHA2 family member */ -#define MC_SHA3 3 /**< SHA3 family member */ - -#define SHA256 32 /**< SHA-256 hashing */ -#define SHA384 48 /**< SHA-384 hashing */ -#define SHA512 64 /**< SHA-512 hashing */ - -#define SHA3_HASH224 28 /**< SHA3 224 bit hash */ -#define SHA3_HASH256 32 /**< SHA3 256 bit hash */ -#define SHA3_HASH384 48 /**< SHA3 384 bit hash */ -#define SHA3_HASH512 64 /**< SHA3 512 bit hash */ - -#define SHAKE128 16 /**< SHAKE128 hash */ -#define SHAKE256 32 /**< SHAKE256 hash */ - - -/* NewHope parameters */ - -//q= 12289 - -#define RLWE_PRIME 0x3001 /**< q in Hex */ -#define RLWE_LGN 10 /**< Degree n=2^LGN */ -#define RLWE_ND 0xF7002FFF /**< 1/(R-q) mod R */ -#define RLWE_ONE 0x2AC8 /**< R mod q */ -#define RLWE_R2MODP 0x1620 /**< R^2 mod q */ - -/* Symmetric Encryption AES structure */ - -#define ECB 0 /**< Electronic Code Book */ -#define CBC 1 /**< Cipher Block Chaining */ -#define CFB1 2 /**< Cipher Feedback - 1 byte */ -#define CFB2 3 /**< Cipher Feedback - 2 bytes */ -#define CFB4 5 /**< Cipher Feedback - 4 bytes */ -#define OFB1 14 /**< Output Feedback - 1 byte */ -#define OFB2 15 /**< Output Feedback - 2 bytes */ -#define OFB4 17 /**< Output Feedback - 4 bytes */ -#define OFB8 21 /**< Output Feedback - 8 bytes */ -#define OFB16 29 /**< Output Feedback - 16 bytes */ -#define CTR1 30 /**< Counter Mode - 1 byte */ -#define CTR2 31 /**< Counter Mode - 2 bytes */ -#define CTR4 33 /**< Counter Mode - 4 bytes */ -#define CTR8 37 /**< Counter Mode - 8 bytes */ -#define CTR16 45 /**< Counter Mode - 16 bytes */ - -#define uchar unsigned char /**< Unsigned char */ - -/** - @brief AES instance -*/ - - -typedef struct -{ - int Nk; /**< AES Key Length */ - int Nr; /**< AES Number of rounds */ - int mode; /**< AES mode of operation */ - unsign32 fkey[60]; /**< subkeys for encrypton */ - unsign32 rkey[60]; /**< subkeys for decrypton */ - char f[16]; /**< buffer for chaining vector */ -} core_aes; - -/* AES-GCM suppport. */ - -#define GCM_ACCEPTING_HEADER 0 /**< GCM status */ -#define GCM_ACCEPTING_CIPHER 1 /**< GCM status */ -#define GCM_NOT_ACCEPTING_MORE 2 /**< GCM status */ -#define GCM_FINISHED 3 /**< GCM status */ -#define GCM_ENCRYPTING 0 /**< GCM mode */ -#define GCM_DECRYPTING 1 /**< GCM mode */ - - -/** - @brief GCM mode instance, using AES internally -*/ - -typedef struct -{ - unsign32 table[128][4]; /**< 2k byte table */ - uchar stateX[16]; /**< GCM Internal State */ - uchar Y_0[16]; /**< GCM Internal State */ - unsign32 lenA[2]; /**< GCM 64-bit length of header */ - unsign32 lenC[2]; /**< GCM 64-bit length of ciphertext */ - int status; /**< GCM Status */ - core_aes a; /**< Internal Instance of CORE_AES cipher */ -} gcm; - -/* Marsaglia & Zaman Random number generator constants */ - -#define NK 21 /**< PRNG constant */ -#define NJ 6 /**< PRNG constant */ -#define NV 8 /**< PRNG constant */ - - -/** - @brief Cryptographically secure pseudo-random number generator instance -*/ - -typedef struct -{ - unsign32 ira[NK]; /**< random number array */ - int rndptr; /**< pointer into array */ - unsign32 borrow; /**< borrow as a result of subtraction */ - int pool_ptr; /**< pointer into random pool */ - char pool[32]; /**< random pool */ -} csprng; - - -/** - @brief Portable representation of a big positive number -*/ - -typedef struct -{ - int len; /**< length in bytes */ - int max; /**< max length allowed - enforce truncation */ - char *val; /**< byte array */ -} octet; - -/** - * @brief Share instance */ -typedef struct -{ - int id; /**< Unique Share ID */ - int nsr; /**< number of shares required */ - octet *B; /**< share as octet */ -} share; - -/* Octet string handlers */ -/** @brief Formats and outputs an octet to the console in hex - * - @param O Octet to be output - */ -extern void OCT_output(octet *O); -/** @brief Formats and outputs an octet to the console as a character string - * - @param O Octet to be output - */ -extern void OCT_output_string(octet *O); -/** @brief Wipe clean an octet - * - @param O Octet to be cleaned - */ -extern void OCT_clear(octet *O); - -/** @brief Reverse bytes in an octet - * - @param O Octet to be reversed - */ -extern void OCT_reverse(octet *O); - -/** @brief Compare two octets - * - @param O first Octet to be compared - @param P second Octet to be compared - @return 1 if equal, else 0 - */ -extern int OCT_comp(octet *O, octet *P); -/** @brief Compare first n bytes of two octets - * - @param O first Octet to be compared - @param P second Octet to be compared - @param n number of bytes to compare - @return 1 if equal, else 0 - */ -extern int OCT_ncomp(octet *O, octet *P, int n); -/** @brief Join from a C string to end of an octet - * - Truncates if there is no room - @param O Octet to be written to - @param s zero terminated string to be joined to octet - */ -extern void OCT_jstring(octet *O, char *s); -/** @brief Join bytes to end of an octet - * - Truncates if there is no room - @param O Octet to be written to - @param s bytes to be joined to end of octet - @param n number of bytes to join - */ -extern void OCT_jbytes(octet *O, char *s, int n); -/** @brief Join single byte to end of an octet, repeated n times - * - Truncates if there is no room - @param O Octet to be written to - @param b byte to be joined to end of octet - @param n number of times b is to be joined - */ -extern void OCT_jbyte(octet *O, int b, int n); -/** @brief Join one octet to the end of another - * - Truncates if there is no room - @param O Octet to be written to - @param P Octet to be joined to the end of O - */ -extern void OCT_joctet(octet *O, octet *P); -/** @brief XOR common bytes of a pair of Octets - * - @param O Octet - on exit = O xor P - @param P Octet to be xored into O - */ -extern void OCT_xor(octet *O, octet *P); -/** @brief reset Octet to zero length - * - @param O Octet to be emptied - */ -extern void OCT_empty(octet *O); -/** @brief Pad out an Octet to the given length - * - Padding is done by inserting leading zeros, so abcd becomes 00abcd - @param O Octet to be padded - @param n new length of Octet - */ -extern int OCT_pad(octet *O, int n); -/** @brief Convert an Octet to printable base64 number - * - @param b zero terminated byte array to take base64 conversion - @param O Octet to be converted - */ -extern void OCT_tobase64(char *b, octet *O); -/** @brief Populate an Octet from base64 number - * - @param O Octet to be populated - @param b zero terminated base64 string - - */ -extern void OCT_frombase64(octet *O, char *b); -/** @brief Copy one Octet into another - * - @param O Octet to be copied to - @param P Octet to be copied from - - */ -extern void OCT_copy(octet *O, octet *P); -/** @brief XOR every byte of an octet with input m - * - @param O Octet - @param m byte to be XORed with every byte of O - - */ -extern void OCT_xorbyte(octet *O, int m); -/** @brief Chops Octet into two, leaving first n bytes in O, moving the rest to P - * - @param O Octet to be chopped - @param P new Octet to be created - @param n number of bytes to chop off O - - */ -extern void OCT_chop(octet *O, octet *P, int n); -/** @brief Join n bytes of integer m to end of Octet O (big endian) - * - Typically n is 4 for a 32-bit integer - @param O Octet to be appended to - @param m integer to be appended to O - @param n number of bytes in m - - */ -extern void OCT_jint(octet *O, int m, int n); -/** @brief Create an Octet from bytes taken from a random number generator - * - Truncates if there is no room - @param O Octet to be populated - @param R an instance of a Cryptographically Secure Random Number Generator - @param n number of bytes to extracted from R - - */ -extern void OCT_rand(octet *O, csprng *R, int n); -/** @brief Shifts Octet left by n bytes - * - Leftmost bytes disappear - @param O Octet to be shifted - @param n number of bytes to shift - - */ -extern void OCT_shl(octet *O, int n); -/** @brief Convert a hex number to an Octet - * - @param dst Octet - @param src Hex string to be converted - */ -extern void OCT_fromHex(octet *dst, char *src); -/** @brief Convert an Octet to printable hex number - * - @param dst hex value - @param src Octet to be converted - */ -extern void OCT_toHex(octet *src, char *dst); -/** @brief Convert an Octet to string - * - @param dst string value - @param src Octet to be converted - */ -extern void OCT_toStr(octet *src, char *dst); - - - -/* Hash function */ -/** @brief Initialise an instance of SHA256 - * - @param H an instance SHA256 - */ -extern void HASH256_init(hash256 *H); -/** @brief Add a byte to the hash - * - @param H an instance SHA256 - @param b byte to be included in hash - */ -extern void HASH256_process(hash256 *H, int b); -/** @brief Generate 32-byte hash - * - @param H an instance SHA256 - @param h is the output 32-byte hash - */ -extern void HASH256_hash(hash256 *H, char *h); - - -/** @brief Initialise an instance of SHA384 - * - @param H an instance SHA384 - */ -extern void HASH384_init(hash384 *H); -/** @brief Add a byte to the hash - * - @param H an instance SHA384 - @param b byte to be included in hash - */ -extern void HASH384_process(hash384 *H, int b); -/** @brief Generate 48-byte hash - * - @param H an instance SHA384 - @param h is the output 48-byte hash - */ -extern void HASH384_hash(hash384 *H, char *h); - - -/** @brief Initialise an instance of SHA512 - * - @param H an instance SHA512 - */ -extern void HASH512_init(hash512 *H); -/** @brief Add a byte to the hash - * - @param H an instance SHA512 - @param b byte to be included in hash - */ -extern void HASH512_process(hash512 *H, int b); -/** @brief Generate 64-byte hash - * - @param H an instance SHA512 - @param h is the output 64-byte hash - */ -extern void HASH512_hash(hash512 *H, char *h); - - -/** @brief Initialise an instance of SHA3 - * - @param H an instance SHA3 - @param t the instance type - */ -extern void SHA3_init(sha3 *H, int t); -/** @brief process a byte for SHA3 - * - @param H an instance SHA3 - @param b a byte of date to be processed - */ -extern void SHA3_process(sha3 *H, int b); -/** @brief create fixed length hash output of SHA3 - * - @param H an instance SHA3 - @param h a byte array to take hash - */ -extern void SHA3_hash(sha3 *H, char *h); -/** @brief create variable length hash output of SHA3 - * - @param H an instance SHA3 - @param h a byte array to take hash - @param len is the length of the hash - */ -extern void SHA3_shake(sha3 *H, char *h, int len); -/** @brief generate further hash output of SHA3 - * - @param H an instance SHA3 - @param h a byte array to take hash - @param len is the length of the hash - */ -extern void SHA3_squeeze(sha3 *H, char *h, int len); - -/* MAC functions */ - -/** @brief General Purpose Hashing function - * - @param hash the hash family (SHA2 or SHA3) - @param hlen the hash function output length (32,48 or 64) - @param w an output octet - @param olen the output length - @param pad zero padding - @param p an input octet - @param n an input 32-bit integer - @param x an optional input octet - */ -extern void GPhash(int hash,int hlen,octet *w,int olen,int pad,octet *p,int n,octet *x); - -/** @brief Simple purpose Hashing function - * - @param hash the hash family (SHA2 or SHA3) - @param hlen the hash function output length (32,48 or 64) - @param w an output octet - @param p an input octet - */ -extern void SPhash(int hash, int hlen,octet *w, octet *p); - -/** @brief HMAC function - * - @param hash the hash family (SHA2 or SHA3) - @param hlen the hash function output length (32,48 or 64) - @param T an output tag - @param len the tag length - @param K an input key, or salt - @param M an input message - */ -extern void HMAC(int hash,int hlen,octet *T,int len,octet *K,octet *M); - - -/** @brief HKDF_Extract function - * - @param hash the hash family (SHA2 or SHA3) - @param hlen the hash function output length (32,48 or 64) - @param K an output Key - @param P public input salt - @param S raw secret keying material - */ -extern void HKDF_Extract(int hash,int hlen,octet *K,octet *P,octet *S); - -/** @brief HKDF_Extract function - * - @param hash the hash family (SHA2 or SHA3) - @param hlen the hash function output length (32,48 or 64) - @param E an expanded output Key - @param olen is the desired length of the expanded key - @param K is the fixed length input key - @param I is public context information - - */ -extern void HKDF_Expand(int hash,int hlen,octet *E,int olen,octet *K,octet *I); - -/** @brief XOF_Expand function - * - @param hlen the SHA3 output length (16 or 32) - @param E an expanded messsage - @param olen is the desired length of the expanded key - @param P is Domain Separator - @param S input message - - */ -extern void XOF_Expand(int hlen,octet *E,int olen,octet *P,octet *S); - -/** @brief XOF_Expand function - * - @param hash the hash family (SHA2 or SHA3) - @param hlen the SHA3 output length (16 or 32) - @param E an expanded messsage - @param olen is the desired length of the expanded key - @param P is Domain Separator - @param S input message - - */ -extern void XMD_Expand(int hash,int hlen,octet *E,int olen,octet *P,octet *S); - -/** @brief Key Derivation Function - generates key K from inputs Z and P - * - IEEE-1363 KDF2 Key Derivation Function. - @param hash is the hash family (SHA2 or SHA3) - @param hlen the hash function output length (32,48 or 64) - @param Z input octet - @param P input key derivation parameters - can be NULL - @param len is output desired length of key - @param K is the derived key - */ -extern void KDF2(int hash, int hlen, octet *K, int len, octet *Z, octet *P); -/** @brief Password Based Key Derivation Function - generates key K from password, salt and repeat counter - * - PBKDF2 Password Based Key Derivation Function. - @param hash is the hash family (SHA2 or SHA3) - @param hlen the hash function output length (32,48 or 64) - @param P input password - @param S input salt - @param rep Number of times to be iterated. - @param len is output desired length - @param K is the derived key - */ -extern void PBKDF2(int hash, int hlen, octet *K, int len, octet *P, octet *S, int rep); - - -/* AES functions */ -/** @brief Reset AES mode or IV - * - @param A an instance of the CORE_AES - @param m is the new active mode of operation (ECB, CBC, OFB, CFB etc) - @param iv the new Initialisation Vector - */ -extern void AES_reset(core_aes *A, int m, char *iv); -/** @brief Extract chaining vector from CORE_AES instance - * - @param A an instance of the CORE_AES - @param f the extracted chaining vector - */ -extern void AES_getreg(core_aes *A, char * f); -/** @brief Initialise an instance of CORE_AES and its mode of operation - * - @param A an instance CORE_AES - @param m is the active mode of operation (ECB, CBC, OFB, CFB etc) - @param n is the key length in bytes, 16, 24 or 32 - @param k the AES key as an array of 16 bytes - @param iv the Initialisation Vector - @return 0 for invalid n - */ -extern int AES_init(core_aes *A, int m, int n, char *k, char *iv); -/** @brief Encrypt a single 16 byte block in ECB mode - * - @param A an instance of the CORE_AES - @param b is an array of 16 plaintext bytes, on exit becomes ciphertext - */ -extern void AES_ecb_encrypt(core_aes *A, uchar * b); -/** @brief Decrypt a single 16 byte block in ECB mode - * - @param A an instance of the CORE_AES - @param b is an array of 16 cipherext bytes, on exit becomes plaintext - */ -extern void AES_ecb_decrypt(core_aes *A, uchar * b); -/** @brief Encrypt a single 16 byte block in active mode - * - @param A an instance of the CORE_AES - @param b is an array of 16 plaintext bytes, on exit becomes ciphertext - @return 0, or overflow bytes from CFB mode - */ -extern unsign32 AES_encrypt(core_aes *A, char *b ); -/** @brief Decrypt a single 16 byte block in active mode - * - @param A an instance of the CORE_AES - @param b is an array of 16 ciphertext bytes, on exit becomes plaintext - @return 0, or overflow bytes from CFB mode - */ -extern unsign32 AES_decrypt(core_aes *A, char *b); -/** @brief Clean up after application of AES - * - @param A an instance of the CORE_AES - */ -extern void AES_end(core_aes *A); -/** @brief AES encrypts a plaintext to a ciphtertext - * - IEEE-1363 AES_CBC_IV0_ENCRYPT function. Encrypts in CBC mode with a zero IV, padding as necessary to create a full final block. - @param K AES key - @param P input plaintext octet - @param C output ciphertext octet - */ -extern void AES_CBC_IV0_ENCRYPT(octet *K, octet *P, octet *C); -/** @brief AES encrypts a plaintext to a ciphtertext - * - IEEE-1363 AES_CBC_IV0_DECRYPT function. Decrypts in CBC mode with a zero IV. - @param K AES key - @param C input ciphertext octet - @param P output plaintext octet - @return 0 if bad input, else 1 - */ -extern int AES_CBC_IV0_DECRYPT(octet *K, octet *C, octet *P); - -/* AES-GCM functions */ -/** @brief Initialise an instance of AES-GCM mode - * - @param G an instance AES-GCM - @param nk is the key length in bytes, 16, 24 or 32 - @param k the AES key as an array of 16 bytes - @param n the number of bytes in the Initialisation Vector (IV) - @param iv the IV - */ -extern void GCM_init(gcm *G, int nk, char *k, int n, char *iv); -/** @brief Add header (material to be authenticated but not encrypted) - * - Note that this function can be called any number of times with n a multiple of 16, and then one last time with any value for n - @param G an instance AES-GCM - @param b is the header material to be added - @param n the number of bytes in the header - */ -extern int GCM_add_header(gcm *G, char *b, int n); -/** @brief Add plaintext and extract ciphertext - * - Note that this function can be called any number of times with n a multiple of 16, and then one last time with any value for n - @param G an instance AES-GCM - @param c is the ciphertext generated - @param p is the plaintext material to be added - @param n the number of bytes in the plaintext - */ -extern int GCM_add_plain(gcm *G, char *c, char *p, int n); -/** @brief Add ciphertext and extract plaintext - * - Note that this function can be called any number of times with n a multiple of 16, and then one last time with any value for n - @param G an instance AES-GCM - @param p is the plaintext generated - @param c is the ciphertext material to be added - @param n the number of bytes in the ciphertext - */ -extern int GCM_add_cipher(gcm *G, char *p, char *c, int n); -/** @brief Finish off and extract authentication tag (HMAC) - * - @param G is an active instance AES-GCM - @param t is the output 16 byte authentication tag - */ -extern void GCM_finish(gcm *G, char *t); - -/** @brief AES-GCM Encryption - * - @param K AES key - @param IV Initialization vector - @param H Header - @param P Plaintext - @param C Ciphertext - @param T Checksum - */ -void AES_GCM_ENCRYPT(octet *K, octet *IV, octet *H, octet *P, octet *C, octet *T); - -/** @brief AES-GCM Decryption - * - @param K AES key - @param IV Initialization vector - @param H Header - @param P Plaintext - @param C Ciphertext - @param T Checksum - */ -void AES_GCM_DECRYPT(octet *K, octet *IV, octet *H, octet *C, octet *P, octet *T); - -/* secret sharing */ - -/** @brief Get a share of a message - * - @param id unique share ID - @param nsr number of shares needed for message recovery - @param S the output share as an octet - @param M the Message octet to be shared - @param R an octet of random seed bytes - @return a share structure - - */ -extern share getshare(int id,int nsr,octet *S,octet *M,octet *R); -/** @brief Recover message from shares - * - @param M the recovered Message octet - @param S an array of sufficient shares - @return 0 on success else -1 - */ -extern int recover(octet *M,share *S); - - -/* random numbers */ -/** @brief Seed a random number generator from an array of bytes - * - The provided seed should be truly random - @param R an instance of a Cryptographically Secure Random Number Generator - @param n the number of seed bytes provided - @param b an array of seed bytes - - */ -extern void RAND_seed(csprng *R, int n, char *b); -/** @brief Delete all internal state of a random number generator - * - @param R an instance of a Cryptographically Secure Random Number Generator - */ -extern void RAND_clean(csprng *R); -/** @brief Return a random byte from a random number generator - * - @param R an instance of a Cryptographically Secure Random Number Generator - @return a random byte - */ -extern int RAND_byte(csprng *R); - - - -#endif diff --git a/impl/cbits/ecdh_BLS12381.h b/impl/cbits/ecdh_BLS12381.h deleted file mode 100644 index b147ecf10..000000000 --- a/impl/cbits/ecdh_BLS12381.h +++ /dev/null @@ -1,154 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file ecdh.h - * @author Mike Scott - * @brief ECDH Header file for implementation of standard EC protocols - * - * - */ - -#ifndef ECDH_BLS12381_H -#define ECDH_BLS12381_H - -#include "ecp_BLS12381.h" -//#include "ecdh_support.h" - - -/*** START OF USER CONFIGURABLE SECTION - ***/ - -//#define EAS_BLS12381 16 /**< Symmetric Key size - 128 bits */ -//#define HASH_TYPE_ECC_BLS12381 SHA512 /**< Hash type */ - -/*** END OF USER CONFIGURABLE SECTION ***/ - -#define EGS_BLS12381 MODBYTES_384_58 /**< ECC Group Size in bytes */ -#define EFS_BLS12381 MODBYTES_384_58 /**< ECC Field Size in bytes */ - -#define ECDH_OK 0 /**< Function completed without error */ -/*#define ECDH_DOMAIN_ERROR -1*/ -#define ECDH_INVALID_PUBLIC_KEY -2 /**< Public Key is Invalid */ -#define ECDH_ERROR -3 /**< ECDH Internal Error */ -//#define ECDH_INVALID -4 /**< ECDH Internal Error */ -/*#define ECDH_DOMAIN_NOT_FOUND -5 -#define ECDH_OUT_OF_MEMORY -6 -#define ECDH_DIV_BY_ZERO -7 -#define ECDH_BAD_ASSUMPTION -8*/ - -/* ECDH primitives */ - -/** @brief Test if group element in correct range - * - @param s is a random number - @return 1 if 0 -#include "ecp2_BLS12381.h" -#include "ecp_BLS12381.h" - -int ECP2_BLS12381_isinf(ECP2_BLS12381 *P) -{ - return (FP2_BLS12381_iszilch(&(P->x)) & FP2_BLS12381_iszilch(&(P->z))); -} - -/* Set P=Q */ -/* SU= 16 */ -void ECP2_BLS12381_copy(ECP2_BLS12381 *P, ECP2_BLS12381 *Q) -{ - FP2_BLS12381_copy(&(P->x), &(Q->x)); - FP2_BLS12381_copy(&(P->y), &(Q->y)); - FP2_BLS12381_copy(&(P->z), &(Q->z)); -} - -/* set P to Infinity */ -/* SU= 8 */ -void ECP2_BLS12381_inf(ECP2_BLS12381 *P) -{ - FP2_BLS12381_zero(&(P->x)); - FP2_BLS12381_one(&(P->y)); - FP2_BLS12381_zero(&(P->z)); -} - -/* Conditional move Q to P dependant on d */ -static void ECP2_BLS12381_cmove(ECP2_BLS12381 *P, ECP2_BLS12381 *Q, int d) -{ - FP2_BLS12381_cmove(&(P->x), &(Q->x), d); - FP2_BLS12381_cmove(&(P->y), &(Q->y), d); - FP2_BLS12381_cmove(&(P->z), &(Q->z), d); -} - -/* return 1 if b==c, no branching */ -static int teq(sign32 b, sign32 c) -{ - sign32 x = b ^ c; - x -= 1; // if x=0, x now -1 - return (int)((x >> 31) & 1); -} - -/* Constant time select from pre-computed table */ -static void ECP2_BLS12381_select(ECP2_BLS12381 *P, ECP2_BLS12381 W[], sign32 b) -{ - ECP2_BLS12381 MP; - sign32 m = b >> 31; - sign32 babs = (b ^ m) - m; - - babs = (babs - 1) / 2; - - ECP2_BLS12381_cmove(P, &W[0], teq(babs, 0)); // conditional move - ECP2_BLS12381_cmove(P, &W[1], teq(babs, 1)); - ECP2_BLS12381_cmove(P, &W[2], teq(babs, 2)); - ECP2_BLS12381_cmove(P, &W[3], teq(babs, 3)); - ECP2_BLS12381_cmove(P, &W[4], teq(babs, 4)); - ECP2_BLS12381_cmove(P, &W[5], teq(babs, 5)); - ECP2_BLS12381_cmove(P, &W[6], teq(babs, 6)); - ECP2_BLS12381_cmove(P, &W[7], teq(babs, 7)); - - ECP2_BLS12381_copy(&MP, P); - ECP2_BLS12381_neg(&MP); // minus P - ECP2_BLS12381_cmove(P, &MP, (int)(m & 1)); -} - -/* return 1 if P==Q, else 0 */ -/* SU= 312 */ -int ECP2_BLS12381_equals(ECP2_BLS12381 *P, ECP2_BLS12381 *Q) -{ - FP2_BLS12381 a, b; - - FP2_BLS12381_mul(&a, &(P->x), &(Q->z)); - FP2_BLS12381_mul(&b, &(Q->x), &(P->z)); - if (!FP2_BLS12381_equals(&a, &b)) return 0; - - FP2_BLS12381_mul(&a, &(P->y), &(Q->z)); - FP2_BLS12381_mul(&b, &(Q->y), &(P->z)); - if (!FP2_BLS12381_equals(&a, &b)) return 0; - return 1; -} - -/* Make P affine (so z=1) */ -/* SU= 232 */ -void ECP2_BLS12381_affine(ECP2_BLS12381 *P) -{ - FP2_BLS12381 one, iz; - if (ECP2_BLS12381_isinf(P)) return; - - FP2_BLS12381_one(&one); - if (FP2_BLS12381_isunity(&(P->z))) - { - FP2_BLS12381_reduce(&(P->x)); - FP2_BLS12381_reduce(&(P->y)); - return; - } - - FP2_BLS12381_inv(&iz, &(P->z)); - FP2_BLS12381_mul(&(P->x), &(P->x), &iz); - FP2_BLS12381_mul(&(P->y), &(P->y), &iz); - - FP2_BLS12381_reduce(&(P->x)); - FP2_BLS12381_reduce(&(P->y)); - FP2_BLS12381_copy(&(P->z), &one); -} - -/* extract x, y from point P */ -/* SU= 16 */ -int ECP2_BLS12381_get(FP2_BLS12381 *x, FP2_BLS12381 *y, ECP2_BLS12381 *P) -{ - ECP2_BLS12381 W; - ECP2_BLS12381_copy(&W, P); - ECP2_BLS12381_affine(&W); - if (ECP2_BLS12381_isinf(&W)) return -1; - FP2_BLS12381_copy(y, &(W.y)); - FP2_BLS12381_copy(x, &(W.x)); - return 0; -} - -/* SU= 152 */ -/* Output point P */ -void ECP2_BLS12381_output(ECP2_BLS12381 *P) -{ - FP2_BLS12381 x, y; - if (ECP2_BLS12381_isinf(P)) - { - printf("Infinity\n"); - return; - } - ECP2_BLS12381_get(&x, &y, P); - printf("("); - FP2_BLS12381_output(&x); - printf(","); - FP2_BLS12381_output(&y); - printf(")\n"); -} - -/* SU= 232 */ -void ECP2_BLS12381_outputxyz(ECP2_BLS12381 *P) -{ - ECP2_BLS12381 Q; - if (ECP2_BLS12381_isinf(P)) - { - printf("Infinity\n"); - return; - } - ECP2_BLS12381_copy(&Q, P); - printf("("); - FP2_BLS12381_output(&(Q.x)); - printf(","); - FP2_BLS12381_output(&(Q.y)); - printf(","); - FP2_BLS12381_output(&(Q.z)); - printf(")\n"); -} - -/* SU= 168 */ -/* Convert Q to octet string */ -void ECP2_BLS12381_toOctet(octet *W, ECP2_BLS12381 *Q, bool compress) -{ - BIG_384_58 b; - FP2_BLS12381 qx, qy; - ECP2_BLS12381_get(&qx, &qy, Q); - - FP_BLS12381_redc(b, &(qx.a)); - BIG_384_58_toBytes(&(W->val[1]), b); - FP_BLS12381_redc(b, &(qx.b)); - BIG_384_58_toBytes(&(W->val[MODBYTES_384_58+1]), b); - if (!compress) - { - W->val[0] = 0x04; - FP_BLS12381_redc(b, &(qy.a)); - BIG_384_58_toBytes(&(W->val[2 * MODBYTES_384_58+1]), b); - FP_BLS12381_redc(b, &(qy.b)); - BIG_384_58_toBytes(&(W->val[3 * MODBYTES_384_58+1]), b); - - W->len = 4 * MODBYTES_384_58+1; - } else { - W->val[0]=0x02; - if (FP2_BLS12381_sign(&qy)==1) W->val[0] = 0x03; - W->len = 2 * MODBYTES_384_58 + 1; - } - -} -void ECP2_BLS12381_toOctet_ZCash(octet *W, ECP2_BLS12381 *Q) -{ - BIG_384_58 b; - FP2_BLS12381 qx, qy, qy_neg; - ECP2_BLS12381_get(&qx, &qy, Q); - - FP_BLS12381_redc(b, &(qx.b)); - BIG_384_58_toBytes(&(W->val[0]), b); - FP_BLS12381_redc(b, &(qx.a)); - BIG_384_58_toBytes(&(W->val[MODBYTES_384_58]), b); - W->len = 2 * MODBYTES_384_58; - - W->val[0] |= 128; // 0x1000 0000: Compressed form - - // is this lexicographic larger, or the negation? - char t1[2*MODBYTES_384_58]; - octet T1 = {0,sizeof(t1),t1}; - FP_BLS12381_redc(b, &(qy.b)); - BIG_384_58_toBytes(&(T1.val[0]), b); - FP_BLS12381_redc(b, &(qy.a)); - BIG_384_58_toBytes(&(T1.val[MODBYTES_384_58]), b); - - ECP2_BLS12381 Q_neg; - ECP2_BLS12381_copy(&Q_neg, Q); - ECP2_BLS12381_neg(&Q_neg); - ECP2_BLS12381_get(&qx, &qy_neg, &Q_neg); - - char t2[2*MODBYTES_384_58]; - octet T2 = {0,sizeof(t2),t2}; - FP_BLS12381_redc(b, &(qy_neg.b)); - BIG_384_58_toBytes(&(T2.val[0]), b); - FP_BLS12381_redc(b, &(qy_neg.a)); - BIG_384_58_toBytes(&(T2.val[MODBYTES_384_58]), b); - - if (memcmp(T1.val, T2.val, 2*MODBYTES_384_58) > 0) { - W->val[0] |= 32; // 0b0010 0000: It is the larger of the two y - } else { - } -} - -/* SU= 176 */ -/* restore Q from octet string */ -int ECP2_BLS12381_fromOctet(ECP2_BLS12381 *Q, octet *W) -{ - BIG_384_58 b; - FP2_BLS12381 qx, qy; - int typ = W->val[0]; - - BIG_384_58_fromBytes(b, &(W->val[1])); - FP_BLS12381_nres(&(qx.a), b); - BIG_384_58_fromBytes(b, &(W->val[MODBYTES_384_58+1])); - FP_BLS12381_nres(&(qx.b), b); - if (typ == 0x04) - { - BIG_384_58_fromBytes(b, &(W->val[2 * MODBYTES_384_58+1])); - FP_BLS12381_nres(&(qy.a), b); - BIG_384_58_fromBytes(b, &(W->val[3 * MODBYTES_384_58+1])); - FP_BLS12381_nres(&(qy.b), b); - - if (ECP2_BLS12381_set(Q, &qx, &qy)) return 1; - } else { - if (ECP2_BLS12381_setx(Q, &qx, typ&1)) return 1; - } - return 0; -} - -int ECP2_BLS12381_fromOctet_ZCash(ECP2_BLS12381 *Q, octet *W) -{ - BIG_384_58 b; - FP2_BLS12381 qx; - if (!(W->val[0] & 128)) return 0; // only accept compressed - int large = (W->val[0] & 32); - W->val[0] &= 31; // mask high bits - - BIG_384_58_fromBytes(b, &(W->val[0])); - FP_BLS12381_nres(&(qx.b), b); - BIG_384_58_fromBytes(b, &(W->val[MODBYTES_384_58])); - FP_BLS12381_nres(&(qx.a), b); - if (!ECP2_BLS12381_setx(Q, &qx, 0)) return 0; - - // try to encode again to find out sign - // (expensive but easy to implement) - char t1[2*MODBYTES_384_58]; - octet T1 = {0,sizeof(t1),t1}; - ECP2_BLS12381_toOctet_ZCash(&T1, Q); - - if ((T1.val[0] & 32) != large) { - // need to negate second component of P - ECP2_BLS12381_neg(Q); - } - - return 1; -} - - -/* SU= 128 */ -/* Calculate RHS of twisted curve equation x^3+B/i or x^3+Bi*/ -void ECP2_BLS12381_rhs(FP2_BLS12381 *rhs, FP2_BLS12381 *x) -{ - /* calculate RHS of elliptic curve equation */ - FP2_BLS12381 t; - BIG_384_58 b; - FP2_BLS12381_sqr(&t, x); - - FP2_BLS12381_mul(rhs, &t, x); - - /* Assuming CURVE_A=0 */ - - BIG_384_58_rcopy(b, CURVE_B_BLS12381); - - FP2_BLS12381_from_BIG(&t, b); - -#if SEXTIC_TWIST_BLS12381 == D_TYPE - FP2_BLS12381_div_ip(&t); /* IMPORTANT - here we use the correct SEXTIC twist of the curve */ -#endif - -#if SEXTIC_TWIST_BLS12381 == M_TYPE - FP2_BLS12381_norm(&t); - FP2_BLS12381_mul_ip(&t); /* IMPORTANT - here we use the correct SEXTIC twist of the curve */ - FP2_BLS12381_norm(&t); - -#endif - - - FP2_BLS12381_add(rhs, &t, rhs); - FP2_BLS12381_reduce(rhs); -} - - -/* Set P=(x,y). Return 1 if (x,y) is on the curve, else return 0*/ -/* SU= 232 */ -int ECP2_BLS12381_set(ECP2_BLS12381 *P, FP2_BLS12381 *x, FP2_BLS12381 *y) -{ - FP2_BLS12381 rhs, y2; - - FP2_BLS12381_sqr(&y2, y); - ECP2_BLS12381_rhs(&rhs, x); - - if (!FP2_BLS12381_equals(&y2, &rhs)) - { - ECP2_BLS12381_inf(P); - return 0; - } - - FP2_BLS12381_copy(&(P->x), x); - FP2_BLS12381_copy(&(P->y), y); - - FP2_BLS12381_one(&(P->z)); - return 1; -} - -/* Set P=(x,y). Return 1 if (x,.) is on the curve, else return 0 */ -/* SU= 232 */ -int ECP2_BLS12381_setx(ECP2_BLS12381 *P, FP2_BLS12381 *x, int s) -{ - FP2_BLS12381 y; - ECP2_BLS12381_rhs(&y, x); - - if (!FP2_BLS12381_qr(&y)) - { - ECP2_BLS12381_inf(P); - return 0; - } - FP2_BLS12381_sqrt(&y, &y); - - FP2_BLS12381_copy(&(P->x), x); - FP2_BLS12381_copy(&(P->y), &y); - FP2_BLS12381_one(&(P->z)); - - if (FP2_BLS12381_sign(&(P->y)) != s) - FP2_BLS12381_neg(&(P->y),&(P->y)); - FP2_BLS12381_reduce(&(P->y)); - return 1; -} - -/* Set P=-P */ -/* SU= 8 */ -void ECP2_BLS12381_neg(ECP2_BLS12381 *P) -{ - FP2_BLS12381_norm(&(P->y)); - FP2_BLS12381_neg(&(P->y), &(P->y)); - FP2_BLS12381_norm(&(P->y)); -} - -/* R+=R */ -/* return -1 for Infinity, 0 for addition, 1 for doubling */ -/* SU= 448 */ -int ECP2_BLS12381_dbl(ECP2_BLS12381 *P) -{ - FP2_BLS12381 t0, t1, t2, iy, x3, y3; - - FP2_BLS12381_copy(&iy, &(P->y)); //FP2 iy=new FP2(y); -#if SEXTIC_TWIST_BLS12381==D_TYPE - FP2_BLS12381_mul_ip(&iy); //iy.mul_ip(); - FP2_BLS12381_norm(&iy); //iy.norm(); -#endif - FP2_BLS12381_sqr(&t0, &(P->y)); //t0.sqr(); -#if SEXTIC_TWIST_BLS12381==D_TYPE - FP2_BLS12381_mul_ip(&t0); //t0.mul_ip(); -#endif - FP2_BLS12381_mul(&t1, &iy, &(P->z)); //t1.mul(z); - FP2_BLS12381_sqr(&t2, &(P->z)); //t2.sqr(); - - FP2_BLS12381_add(&(P->z), &t0, &t0); //z.add(t0); - FP2_BLS12381_norm(&(P->z)); //z.norm(); - FP2_BLS12381_add(&(P->z), &(P->z), &(P->z)); //z.add(z); - FP2_BLS12381_add(&(P->z), &(P->z), &(P->z)); //z.add(z); - FP2_BLS12381_norm(&(P->z)); //z.norm(); - - FP2_BLS12381_imul(&t2, &t2, 3 * CURVE_B_I_BLS12381); //t2.imul(3*ROM.CURVE_B_I); -#if SEXTIC_TWIST_BLS12381==M_TYPE - FP2_BLS12381_mul_ip(&t2); - FP2_BLS12381_norm(&t2); -#endif - - FP2_BLS12381_mul(&x3, &t2, &(P->z)); //x3.mul(z); - - FP2_BLS12381_add(&y3, &t0, &t2); //y3.add(t2); - FP2_BLS12381_norm(&y3); //y3.norm(); - FP2_BLS12381_mul(&(P->z), &(P->z), &t1); //z.mul(t1); - - FP2_BLS12381_add(&t1, &t2, &t2); //t1.add(t2); - FP2_BLS12381_add(&t2, &t2, &t1); //t2.add(t1); - FP2_BLS12381_norm(&t2); //t2.norm(); - FP2_BLS12381_sub(&t0, &t0, &t2); //t0.sub(t2); - FP2_BLS12381_norm(&t0); //t0.norm(); //y^2-9bz^2 - FP2_BLS12381_mul(&y3, &y3, &t0); //y3.mul(t0); - FP2_BLS12381_add(&(P->y), &y3, &x3); //y3.add(x3); //(y^2+3z*2)(y^2-9z^2)+3b.z^2.8y^2 - FP2_BLS12381_mul(&t1, &(P->x), &iy); //t1.mul(iy); // - FP2_BLS12381_norm(&t0); //x.norm(); - FP2_BLS12381_mul(&(P->x), &t0, &t1); //x.mul(t1); - FP2_BLS12381_add(&(P->x), &(P->x), &(P->x)); //x.add(x); //(y^2-9bz^2)xy2 - - FP2_BLS12381_norm(&(P->x)); //x.norm(); - FP2_BLS12381_norm(&(P->y)); //y.norm(); - - return 1; -} - -/* Set P+=Q */ -/* SU= 400 */ -int ECP2_BLS12381_add(ECP2_BLS12381 *P, ECP2_BLS12381 *Q) -{ - FP2_BLS12381 t0, t1, t2, t3, t4, x3, y3, z3; - int b3 = 3 * CURVE_B_I_BLS12381; - - FP2_BLS12381_mul(&t0, &(P->x), &(Q->x)); //t0.mul(Q.x); // x.Q.x - FP2_BLS12381_mul(&t1, &(P->y), &(Q->y)); //t1.mul(Q.y); // y.Q.y - - FP2_BLS12381_mul(&t2, &(P->z), &(Q->z)); //t2.mul(Q.z); - FP2_BLS12381_add(&t3, &(P->x), &(P->y)); //t3.add(y); - FP2_BLS12381_norm(&t3); //t3.norm(); //t3=X1+Y1 - - FP2_BLS12381_add(&t4, &(Q->x), &(Q->y)); //t4.add(Q.y); - FP2_BLS12381_norm(&t4); //t4.norm(); //t4=X2+Y2 - FP2_BLS12381_mul(&t3, &t3, &t4); //t3.mul(t4); //t3=(X1+Y1)(X2+Y2) - FP2_BLS12381_add(&t4, &t0, &t1); //t4.add(t1); //t4=X1.X2+Y1.Y2 - - FP2_BLS12381_sub(&t3, &t3, &t4); //t3.sub(t4); - FP2_BLS12381_norm(&t3); //t3.norm(); -#if SEXTIC_TWIST_BLS12381==D_TYPE - FP2_BLS12381_mul_ip(&t3); //t3.mul_ip(); - FP2_BLS12381_norm(&t3); //t3.norm(); //t3=(X1+Y1)(X2+Y2)-(X1.X2+Y1.Y2) = X1.Y2+X2.Y1 -#endif - FP2_BLS12381_add(&t4, &(P->y), &(P->z)); //t4.add(z); - FP2_BLS12381_norm(&t4); //t4.norm(); //t4=Y1+Z1 - FP2_BLS12381_add(&x3, &(Q->y), &(Q->z)); //x3.add(Q.z); - FP2_BLS12381_norm(&x3); //x3.norm(); //x3=Y2+Z2 - - FP2_BLS12381_mul(&t4, &t4, &x3); //t4.mul(x3); //t4=(Y1+Z1)(Y2+Z2) - FP2_BLS12381_add(&x3, &t1, &t2); //x3.add(t2); //X3=Y1.Y2+Z1.Z2 - - FP2_BLS12381_sub(&t4, &t4, &x3); //t4.sub(x3); - FP2_BLS12381_norm(&t4); //t4.norm(); -#if SEXTIC_TWIST_BLS12381==D_TYPE - FP2_BLS12381_mul_ip(&t4); //t4.mul_ip(); - FP2_BLS12381_norm(&t4); //t4.norm(); //t4=(Y1+Z1)(Y2+Z2) - (Y1.Y2+Z1.Z2) = Y1.Z2+Y2.Z1 -#endif - FP2_BLS12381_add(&x3, &(P->x), &(P->z)); //x3.add(z); - FP2_BLS12381_norm(&x3); //x3.norm(); // x3=X1+Z1 - FP2_BLS12381_add(&y3, &(Q->x), &(Q->z)); //y3.add(Q.z); - FP2_BLS12381_norm(&y3); //y3.norm(); // y3=X2+Z2 - FP2_BLS12381_mul(&x3, &x3, &y3); //x3.mul(y3); // x3=(X1+Z1)(X2+Z2) - FP2_BLS12381_add(&y3, &t0, &t2); //y3.add(t2); // y3=X1.X2+Z1+Z2 - FP2_BLS12381_sub(&y3, &x3, &y3); //y3.rsub(x3); - FP2_BLS12381_norm(&y3); //y3.norm(); // y3=(X1+Z1)(X2+Z2) - (X1.X2+Z1.Z2) = X1.Z2+X2.Z1 -#if SEXTIC_TWIST_BLS12381==D_TYPE - FP2_BLS12381_mul_ip(&t0); //t0.mul_ip(); - FP2_BLS12381_norm(&t0); //t0.norm(); // x.Q.x - FP2_BLS12381_mul_ip(&t1); //t1.mul_ip(); - FP2_BLS12381_norm(&t1); //t1.norm(); // y.Q.y -#endif - FP2_BLS12381_add(&x3, &t0, &t0); //x3.add(t0); - FP2_BLS12381_add(&t0, &t0, &x3); //t0.add(x3); - FP2_BLS12381_norm(&t0); //t0.norm(); - FP2_BLS12381_imul(&t2, &t2, b3); //t2.imul(b); -#if SEXTIC_TWIST_BLS12381==M_TYPE - FP2_BLS12381_mul_ip(&t2); - FP2_BLS12381_norm(&t2); -#endif - FP2_BLS12381_add(&z3, &t1, &t2); //z3.add(t2); - FP2_BLS12381_norm(&z3); //z3.norm(); - FP2_BLS12381_sub(&t1, &t1, &t2); //t1.sub(t2); - FP2_BLS12381_norm(&t1); //t1.norm(); - FP2_BLS12381_imul(&y3, &y3, b3); //y3.imul(b); -#if SEXTIC_TWIST_BLS12381==M_TYPE - FP2_BLS12381_mul_ip(&y3); - FP2_BLS12381_norm(&y3); -#endif - FP2_BLS12381_mul(&x3, &y3, &t4); //x3.mul(t4); - FP2_BLS12381_mul(&t2, &t3, &t1); //t2.mul(t1); - FP2_BLS12381_sub(&(P->x), &t2, &x3); //x3.rsub(t2); - FP2_BLS12381_mul(&y3, &y3, &t0); //y3.mul(t0); - FP2_BLS12381_mul(&t1, &t1, &z3); //t1.mul(z3); - FP2_BLS12381_add(&(P->y), &y3, &t1); //y3.add(t1); - FP2_BLS12381_mul(&t0, &t0, &t3); //t0.mul(t3); - FP2_BLS12381_mul(&z3, &z3, &t4); //z3.mul(t4); - FP2_BLS12381_add(&(P->z), &z3, &t0); //z3.add(t0); - - FP2_BLS12381_norm(&(P->x)); //x.norm(); - FP2_BLS12381_norm(&(P->y)); //y.norm(); - FP2_BLS12381_norm(&(P->z)); //z.norm(); - - return 0; -} - -/* Set P-=Q */ -/* SU= 16 */ -void ECP2_BLS12381_sub(ECP2_BLS12381 *P, ECP2_BLS12381 *Q) -{ - ECP2_BLS12381 NQ; - ECP2_BLS12381_copy(&NQ, Q); - ECP2_BLS12381_neg(&NQ); - ECP2_BLS12381_add(P, &NQ); -} - -/* P*=e */ -/* SU= 280 */ -void ECP2_BLS12381_mul(ECP2_BLS12381 *P, BIG_384_58 e) -{ - /* fixed size windows */ - int i, nb, s, ns; - BIG_384_58 mt, t; - ECP2_BLS12381 Q, W[8], C; - sign8 w[1 + (NLEN_384_58 * BASEBITS_384_58 + 3) / 4]; - - if (ECP2_BLS12381_isinf(P)) return; - - /* precompute table */ - - ECP2_BLS12381_copy(&Q, P); - ECP2_BLS12381_dbl(&Q); - ECP2_BLS12381_copy(&W[0], P); - - for (i = 1; i < 8; i++) - { - ECP2_BLS12381_copy(&W[i], &W[i - 1]); - ECP2_BLS12381_add(&W[i], &Q); - } - - /* make exponent odd - add 2P if even, P if odd */ - BIG_384_58_copy(t, e); - s = BIG_384_58_parity(t); - BIG_384_58_inc(t, 1); - BIG_384_58_norm(t); - ns = BIG_384_58_parity(t); - BIG_384_58_copy(mt, t); - BIG_384_58_inc(mt, 1); - BIG_384_58_norm(mt); - BIG_384_58_cmove(t, mt, s); - ECP2_BLS12381_cmove(&Q, P, ns); - ECP2_BLS12381_copy(&C, &Q); - - nb = 1 + (BIG_384_58_nbits(t) + 3) / 4; - - /* convert exponent to signed 4-bit window */ - for (i = 0; i < nb; i++) - { - w[i] = BIG_384_58_lastbits(t, 5) - 16; - BIG_384_58_dec(t, w[i]); - BIG_384_58_norm(t); - BIG_384_58_fshr(t, 4); - } - w[nb] = BIG_384_58_lastbits(t, 5); - - ECP2_BLS12381_copy(P, &W[(w[nb] - 1) / 2]); - for (i = nb - 1; i >= 0; i--) - { - ECP2_BLS12381_select(&Q, W, w[i]); - ECP2_BLS12381_dbl(P); - ECP2_BLS12381_dbl(P); - ECP2_BLS12381_dbl(P); - ECP2_BLS12381_dbl(P); - ECP2_BLS12381_add(P, &Q); - } - ECP2_BLS12381_sub(P, &C); /* apply correction */ -} - -/* Calculates q.P using Frobenius constant X */ -/* SU= 96 */ -void ECP2_BLS12381_frob(ECP2_BLS12381 *P, FP2_BLS12381 *X) -{ - FP2_BLS12381 X2; - - FP2_BLS12381_sqr(&X2, X); - FP2_BLS12381_conj(&(P->x), &(P->x)); - FP2_BLS12381_conj(&(P->y), &(P->y)); - FP2_BLS12381_conj(&(P->z), &(P->z)); - FP2_BLS12381_reduce(&(P->z)); - - FP2_BLS12381_mul(&(P->x), &X2, &(P->x)); - FP2_BLS12381_mul(&(P->y), &X2, &(P->y)); - FP2_BLS12381_mul(&(P->y), X, &(P->y)); - -} - - -// Bos & Costello https://eprint.iacr.org/2013/458.pdf -// Faz-Hernandez & Longa & Sanchez https://eprint.iacr.org/2013/158.pdf -// Side channel attack secure - -void ECP2_BLS12381_mul4(ECP2_BLS12381 *P, ECP2_BLS12381 Q[4], BIG_384_58 u[4]) -{ - int i, j, k, nb, pb, bt; - ECP2_BLS12381 T[8], W; - BIG_384_58 t[4], mt; - sign8 w[NLEN_384_58 * BASEBITS_384_58 + 1]; - sign8 s[NLEN_384_58 * BASEBITS_384_58 + 1]; - - for (i = 0; i < 4; i++) - { - BIG_384_58_copy(t[i], u[i]); - } - -// Precomputed table - ECP2_BLS12381_copy(&T[0], &Q[0]); // Q[0] - ECP2_BLS12381_copy(&T[1], &T[0]); - ECP2_BLS12381_add(&T[1], &Q[1]); // Q[0]+Q[1] - ECP2_BLS12381_copy(&T[2], &T[0]); - ECP2_BLS12381_add(&T[2], &Q[2]); // Q[0]+Q[2] - ECP2_BLS12381_copy(&T[3], &T[1]); - ECP2_BLS12381_add(&T[3], &Q[2]); // Q[0]+Q[1]+Q[2] - ECP2_BLS12381_copy(&T[4], &T[0]); - ECP2_BLS12381_add(&T[4], &Q[3]); // Q[0]+Q[3] - ECP2_BLS12381_copy(&T[5], &T[1]); - ECP2_BLS12381_add(&T[5], &Q[3]); // Q[0]+Q[1]+Q[3] - ECP2_BLS12381_copy(&T[6], &T[2]); - ECP2_BLS12381_add(&T[6], &Q[3]); // Q[0]+Q[2]+Q[3] - ECP2_BLS12381_copy(&T[7], &T[3]); - ECP2_BLS12381_add(&T[7], &Q[3]); // Q[0]+Q[1]+Q[2]+Q[3] - -// Make it odd - pb = 1 - BIG_384_58_parity(t[0]); - BIG_384_58_inc(t[0], pb); - BIG_384_58_norm(t[0]); - -// Number of bits - BIG_384_58_zero(mt); - for (i = 0; i < 4; i++) - { - BIG_384_58_or(mt, mt, t[i]); - } - nb = 1 + BIG_384_58_nbits(mt); - -// Sign pivot - s[nb - 1] = 1; - for (i = 0; i < nb - 1; i++) - { - BIG_384_58_fshr(t[0], 1); - s[i] = 2 * BIG_384_58_parity(t[0]) - 1; - } - -// Recoded exponent - for (i = 0; i < nb; i++) - { - w[i] = 0; - k = 1; - for (j = 1; j < 4; j++) - { - bt = s[i] * BIG_384_58_parity(t[j]); - BIG_384_58_fshr(t[j], 1); - - BIG_384_58_dec(t[j], (bt >> 1)); - BIG_384_58_norm(t[j]); - w[i] += bt * k; - k *= 2; - } - } - -// Main loop - ECP2_BLS12381_select(P, T, 2 * w[nb - 1] + 1); - for (i = nb - 2; i >= 0; i--) - { - ECP2_BLS12381_select(&W, T, 2 * w[i] + s[i]); - ECP2_BLS12381_dbl(P); - ECP2_BLS12381_add(P, &W); - } - -// apply correction - ECP2_BLS12381_copy(&W, P); - ECP2_BLS12381_sub(&W, &Q[0]); - ECP2_BLS12381_cmove(P, &W, pb); -} - -/* Hunt and Peck a BIG to a curve point */ -/* -void ECP2_BLS12381_hap2point(ECP2_BLS12381 *Q,BIG_384_58 h) -{ - BIG_384_58 one,hv; - FP2_BLS12381 X; - BIG_384_58_one(one); - BIG_384_58_copy(hv,h); - for (;;) - { - FP2_BLS12381_from_BIGs(&X,one,hv); - if (ECP2_BLS12381_setx(Q,&X,0)) break; - BIG_384_58_inc(hv,1); - BIG_384_58_norm(hv); - } -} -*/ -/* Constant time Map to Point in G2 */ -void ECP2_BLS12381_map2point(ECP2_BLS12381 *Q,FP2_BLS12381 *H) -{ // SSWU plus isogenies method - - int i,k,sgn,ne,isox,isoy,iso=HTC_ISO_G2_BLS12381; - FP2_BLS12381 X1,X2,X3,W,Y,T,A,NY; - FP_BLS12381 Z,s; -#if HTC_ISO_G2_BLS12381 != 0 - FP2_BLS12381 ZZ,Ad,Bd; - FP2_BLS12381 xnum,xden,ynum,yden; - FP2_BLS12381_from_ints(&ZZ,RIADZG2A_BLS12381,RIADZG2B_BLS12381); - - FP2_BLS12381_rcopy(&Ad,CURVE_Adr_BLS12381,CURVE_Adi_BLS12381); - FP2_BLS12381_rcopy(&Bd,CURVE_Bdr_BLS12381,CURVE_Bdi_BLS12381); - - FP2_BLS12381_one(&NY); - FP2_BLS12381_copy(&T,H); - sgn=FP2_BLS12381_sign(&T); - - FP2_BLS12381_sqr(&T,&T); - FP2_BLS12381_mul(&T,&T,&ZZ); - FP2_BLS12381_add(&W,&T,&NY); - FP2_BLS12381_norm(&W); - - FP2_BLS12381_mul(&W,&W,&T); - FP2_BLS12381_mul(&A,&Ad,&W); - FP2_BLS12381_inv(&A,&A); - FP2_BLS12381_add(&W,&W,&NY); - FP2_BLS12381_norm(&W); - FP2_BLS12381_mul(&W,&W,&Bd); - FP2_BLS12381_neg(&W,&W); - FP2_BLS12381_norm(&W); - - FP2_BLS12381_mul(&X2,&W,&A); - FP2_BLS12381_mul(&X3,&T,&X2); - - FP2_BLS12381_sqr(&W,&X3); FP2_BLS12381_add(&W,&W,&Ad); FP2_BLS12381_norm(&W); FP2_BLS12381_mul(&W,&W,&X3); FP2_BLS12381_add(&W,&W,&Bd); FP2_BLS12381_norm(&W); // w=x^3+Ax+B - FP2_BLS12381_cmove(&X2,&X3,FP2_BLS12381_qr(&W)); - FP2_BLS12381_sqr(&W,&X2); FP2_BLS12381_add(&W,&W,&Ad); FP2_BLS12381_norm(&W); FP2_BLS12381_mul(&W,&W,&X2); FP2_BLS12381_add(&W,&W,&Bd); FP2_BLS12381_norm(&W); - FP2_BLS12381_sqrt(&Y,&W); - - ne=FP2_BLS12381_sign(&Y)^sgn; - FP2_BLS12381_neg(&NY,&Y); FP2_BLS12381_norm(&NY); - FP2_BLS12381_cmove(&Y,&NY,ne); - -// (X2,Y) is on isogenous curve - - k=0; - isox=iso; - isoy=3*(iso-1)/2; - -// xnum - FP2_BLS12381_rcopy(&xnum,PCR_BLS12381[k],PCI_BLS12381[k]); k++; - for (i=0;ix),&T); - - FP2_BLS12381_mul(&T,&ynum,&xden); - FP2_BLS12381_copy(&(Q->y),&T); - - FP2_BLS12381_mul(&T,&xden,&yden); - FP2_BLS12381_copy(&(Q->z),&T); - -#else - FP2_BLS12381_one(&NY); - FP2_BLS12381_copy(&T,H); - sgn=FP2_BLS12381_sign(&T); - - FP_BLS12381_from_int(&Z,RIADZG2A_BLS12381); - FP2_BLS12381_from_FP(&A,&Z); - ECP2_BLS12381_rhs(&A,&A); // A=g(Z) - - if (CURVE_B_I_BLS12381==4 && SEXTIC_TWIST_BLS12381==M_TYPE && RIADZG2A_BLS12381==-1 && RIADZG2B_BLS12381==0) - { // special case for BLS12381 - FP2_BLS12381_from_ints(&W,2,1); - } else { - FP2_BLS12381_sqrt(&W,&A); // sqrt(g(Z)) - } - FP2_BLS12381_sqrt(&W,&A); // sqrt(g(Z)) - - FP_BLS12381_rcopy(&s,SQRTm3_BLS12381); - - FP_BLS12381_mul(&Z,&Z,&s); // Z.sqrt(-3) - - FP2_BLS12381_sqr(&T,&T); - FP2_BLS12381_mul(&Y,&A,&T); // tv1=u^2*g(Z) - FP2_BLS12381_add(&T,&NY,&Y); FP2_BLS12381_norm(&T); // tv2=1+tv1 - FP2_BLS12381_sub(&Y,&NY,&Y); FP2_BLS12381_norm(&Y); // tv1=1-tv1 - FP2_BLS12381_mul(&NY,&T,&Y); - - FP2_BLS12381_pmul(&NY,&NY,&Z); - - FP2_BLS12381_inv(&NY,&NY); // tv3=inv0(tv1*tv2) - - FP2_BLS12381_pmul(&W,&W,&Z); // tv4=Z*sqrt(-3).sqrt(g(Z)) - if (FP2_BLS12381_sign(&W)==1) - { - FP2_BLS12381_neg(&W,&W); - FP2_BLS12381_norm(&W); - } - FP2_BLS12381_pmul(&W,&W,&Z); - FP2_BLS12381_mul(&W,&W,H); - FP2_BLS12381_mul(&W,&W,&Y); - FP2_BLS12381_mul(&W,&W,&NY); // tv5=u*tv1*tv3*tv4*Z*sqrt(-3) - - FP2_BLS12381_from_ints(&X1,RIADZG2A_BLS12381,RIADZG2B_BLS12381); - FP2_BLS12381_copy(&X3,&X1); - FP2_BLS12381_neg(&X1,&X1); FP2_BLS12381_norm(&X1); FP2_BLS12381_div2(&X1,&X1); // -Z/2 - FP2_BLS12381_copy(&X2,&X1); - FP2_BLS12381_sub(&X1,&X1,&W); FP2_BLS12381_norm(&X1); - FP2_BLS12381_add(&X2,&X2,&W); FP2_BLS12381_norm(&X2); - FP2_BLS12381_add(&A,&A,&A); - FP2_BLS12381_add(&A,&A,&A); - FP2_BLS12381_norm(&A); // 4*g(Z) - FP2_BLS12381_sqr(&T,&T); - FP2_BLS12381_mul(&T,&T,&NY); - FP2_BLS12381_sqr(&T,&T); // (tv2^2*tv3)^2 - FP2_BLS12381_mul(&A,&A,&T); // 4*g(Z)*(tv2^2*tv3)^2 - FP2_BLS12381_add(&X3,&X3,&A); FP2_BLS12381_norm(&X3); - - - ECP2_BLS12381_rhs(&W,&X2); - FP2_BLS12381_cmove(&X3,&X2,FP2_BLS12381_qr(&W)); - ECP2_BLS12381_rhs(&W,&X1); - FP2_BLS12381_cmove(&X3,&X1,FP2_BLS12381_qr(&W)); - ECP2_BLS12381_rhs(&W,&X3); - FP2_BLS12381_sqrt(&Y,&W); - - ne=FP2_BLS12381_sign(&Y)^sgn; - FP2_BLS12381_neg(&W,&Y); FP2_BLS12381_norm(&W); - FP2_BLS12381_cmove(&Y,&W,ne); - - ECP2_BLS12381_set(Q,&X3,&Y); -#endif -} - -/* Map octet to point */ -/* -void ECP2_BLS12381_mapit(ECP2_BLS12381 *Q, octet *W) -{ - BIG_384_58 q, x; - DBIG_384_58 dx; - BIG_384_58_rcopy(q, Modulus_BLS12381); - - BIG_384_58_dfromBytesLen(dx,W->val,W->len); - BIG_384_58_dmod(x,dx,q); - - - ECP2_BLS12381_hap2point(Q,x); - ECP2_BLS12381_cfp(Q); -} -*/ -/* cofactor product */ -void ECP2_BLS12381_cfp(ECP2_BLS12381 *Q) -{ - FP_BLS12381 Fx, Fy; - FP2_BLS12381 X; - BIG_384_58 x; -#if (PAIRING_FRIENDLY_BLS12381 == BN_CURVE) - ECP2_BLS12381 T, K; -#elif (PAIRING_FRIENDLY_BLS12381 > BN_CURVE) - ECP2_BLS12381 xQ, x2Q; -#endif - FP_BLS12381_rcopy(&Fx, Fra_BLS12381); - FP_BLS12381_rcopy(&Fy, Frb_BLS12381); - FP2_BLS12381_from_FPs(&X, &Fx, &Fy); - -#if SEXTIC_TWIST_BLS12381==M_TYPE - FP2_BLS12381_inv(&X, &X); - FP2_BLS12381_norm(&X); -#endif - - BIG_384_58_rcopy(x, CURVE_Bnx_BLS12381); - -#if (PAIRING_FRIENDLY_BLS12381 == BN_CURVE) - - // Faster Hashing to G2 - Fuentes-Castaneda, Knapp and Rodriguez-Henriquez - // Q -> xQ + F(3xQ) + F(F(xQ)) + F(F(F(Q))). - ECP2_BLS12381_copy(&T, Q); - ECP2_BLS12381_mul(&T, x); -#if SIGN_OF_X_BLS12381==NEGATIVEX - ECP2_BLS12381_neg(&T); // our x is negative -#endif - ECP2_BLS12381_copy(&K, &T); - ECP2_BLS12381_dbl(&K); - ECP2_BLS12381_add(&K, &T); - - ECP2_BLS12381_frob(&K, &X); - ECP2_BLS12381_frob(Q, &X); - ECP2_BLS12381_frob(Q, &X); - ECP2_BLS12381_frob(Q, &X); - ECP2_BLS12381_add(Q, &T); - ECP2_BLS12381_add(Q, &K); - ECP2_BLS12381_frob(&T, &X); - ECP2_BLS12381_frob(&T, &X); - ECP2_BLS12381_add(Q, &T); - -#elif (PAIRING_FRIENDLY_BLS12381 > BN_CURVE) - - // Efficient hash maps to G2 on BLS curves - Budroni, Pintore - // Q -> x2Q -xQ -Q +F(xQ -Q) +F(F(2Q)) - - ECP2_BLS12381_copy(&xQ, Q); - ECP2_BLS12381_mul(&xQ, x); - ECP2_BLS12381_copy(&x2Q, &xQ); - ECP2_BLS12381_mul(&x2Q, x); - -#if SIGN_OF_X_BLS12381==NEGATIVEX - ECP2_BLS12381_neg(&xQ); -#endif - - ECP2_BLS12381_sub(&x2Q, &xQ); - ECP2_BLS12381_sub(&x2Q, Q); - - ECP2_BLS12381_sub(&xQ, Q); - ECP2_BLS12381_frob(&xQ, &X); - - ECP2_BLS12381_dbl(Q); - ECP2_BLS12381_frob(Q, &X); - ECP2_BLS12381_frob(Q, &X); - - ECP2_BLS12381_add(Q, &x2Q); - ECP2_BLS12381_add(Q, &xQ); - -#endif -} - -int ECP2_BLS12381_generator(ECP2_BLS12381 *G) -{ - FP2_BLS12381 wx, wy; - - FP2_BLS12381_rcopy(&wx,CURVE_Pxa_BLS12381,CURVE_Pxb_BLS12381); - FP2_BLS12381_rcopy(&wy,CURVE_Pya_BLS12381,CURVE_Pyb_BLS12381); - - return ECP2_BLS12381_set(G, &wx, &wy); -} diff --git a/impl/cbits/ecp2_BLS12381.h b/impl/cbits/ecp2_BLS12381.h deleted file mode 100644 index 7750994f0..000000000 --- a/impl/cbits/ecp2_BLS12381.h +++ /dev/null @@ -1,248 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file ecp2.h - * @author Mike Scott - * @brief ECP2 Header File - * - */ - -#ifndef ECP2_BLS12381_H -#define ECP2_BLS12381_H - -#include "fp2_BLS12381.h" -#include "config_curve_BLS12381.h" - -/** - @brief ECP2 Structure - Elliptic Curve Point over quadratic extension field -*/ - -typedef struct -{ -// int inf; /**< Infinity Flag */ - FP2_BLS12381 x; /**< x-coordinate of point */ - FP2_BLS12381 y; /**< y-coordinate of point */ - FP2_BLS12381 z; /**< z-coordinate of point */ -} ECP2_BLS12381; - - -/* Curve Params - see rom_zzz.c */ - -extern const int CURVE_B_I_BLS12381; /**< Elliptic curve B parameter */ -extern const BIG_384_58 CURVE_B_BLS12381; /**< Elliptic curve B parameter */ -extern const BIG_384_58 CURVE_Order_BLS12381; /**< Elliptic curve group order */ -extern const BIG_384_58 CURVE_Cof_BLS12381; /**< Elliptic curve cofactor */ -extern const BIG_384_58 CURVE_Bnx_BLS12381; /**< Elliptic curve parameter */ -extern const BIG_384_58 CURVE_HTPC_BLS12381; /**< Hash to Point precomputation */ - -extern const BIG_384_58 Fra_BLS12381; /**< real part of BN curve Frobenius Constant */ -extern const BIG_384_58 Frb_BLS12381; /**< imaginary part of BN curve Frobenius Constant */ - - -/* Generator point on G1 */ -extern const BIG_384_58 CURVE_Gx_BLS12381; /**< x-coordinate of generator point in group G1 */ -extern const BIG_384_58 CURVE_Gy_BLS12381; /**< y-coordinate of generator point in group G1 */ - -/* For Pairings only */ - -/* Generator point on G2 */ -extern const BIG_384_58 CURVE_Pxa_BLS12381; /**< real part of x-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pxb_BLS12381; /**< imaginary part of x-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pya_BLS12381; /**< real part of y-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pyb_BLS12381; /**< imaginary part of y-coordinate of generator point in group G2 */ - -/* ECP2 E(Fp2) prototypes */ -/** @brief Tests for ECP2 point equal to infinity - * - @param P ECP2 point to be tested - @return 1 if infinity, else returns 0 - */ -extern int ECP2_BLS12381_isinf(ECP2_BLS12381 *P); -/** @brief Copy ECP2 point to another ECP2 point - * - @param P ECP2 instance, on exit = Q - @param Q ECP2 instance to be copied - */ -extern void ECP2_BLS12381_copy(ECP2_BLS12381 *P, ECP2_BLS12381 *Q); -/** @brief Set ECP2 to point-at-infinity - * - @param P ECP2 instance to be set to infinity - */ -extern void ECP2_BLS12381_inf(ECP2_BLS12381 *P); -/** @brief Tests for equality of two ECP2s - * - @param P ECP2 instance to be compared - @param Q ECP2 instance to be compared - @return 1 if P=Q, else returns 0 - */ -extern int ECP2_BLS12381_equals(ECP2_BLS12381 *P, ECP2_BLS12381 *Q); -/** @brief Converts an ECP2 point from Projective (x,y,z) coordinates to affine (x,y) coordinates - * - @param P ECP2 instance to be converted to affine form - */ -extern void ECP2_BLS12381_affine(ECP2_BLS12381 *P); -/** @brief Extract x and y coordinates of an ECP2 point P - * - If x=y, returns only x - @param x FP2 on exit = x coordinate of point - @param y FP2 on exit = y coordinate of point (unless x=y) - @param P ECP2 instance (x,y) - @return -1 if P is point-at-infinity, else 0 - */ -extern int ECP2_BLS12381_get(FP2_BLS12381 *x, FP2_BLS12381 *y, ECP2_BLS12381 *P); -/** @brief Formats and outputs an ECP2 point to the console, converted to affine coordinates - * - @param P ECP2 instance to be printed - */ -extern void ECP2_BLS12381_output(ECP2_BLS12381 *P); -/** @brief Formats and outputs an ECP2 point to the console, in projective coordinates - * - @param P ECP2 instance to be printed - */ -extern void ECP2_BLS12381_outputxyz(ECP2_BLS12381 *P); -/** @brief Formats and outputs an ECP2 point to an octet string - * - The octet string is created in the form x|y or just x if compressed - Convert the real and imaginary parts of the x and y coordinates to big-endian base 256 form. - @param S output octet string - @param P ECP2 instance to be converted to an octet string - @param c true for compression - */ -extern void ECP2_BLS12381_toOctet(octet *S, ECP2_BLS12381 *P, bool c); -extern void ECP2_BLS12381_toOctet_ZCash(octet *S, ECP2_BLS12381 *P); -/** @brief Creates an ECP2 point from an octet string - * - The octet string is in the form x|y - The real and imaginary parts of the x and y coordinates are in big-endian base 256 form. - If in compressed form only the x coordinate is provided as in 0x2|x if y is even, or 0x3|x if y is odd - @param P ECP2 instance to be created from the octet string - @param S input octet string - return 1 if octet string corresponds to a point on the curve, else 0 - */ -extern int ECP2_BLS12381_fromOctet(ECP2_BLS12381 *P, octet *S); -extern int ECP2_BLS12381_fromOctet_ZCash(ECP2_BLS12381 *P, octet *S); -/** @brief Calculate Right Hand Side of curve equation y^2=f(x) - * - Function f(x)=x^3+Ax+B - Used internally. - @param r FP2 value of f(x) - @param x FP2 instance - */ -extern void ECP2_BLS12381_rhs(FP2_BLS12381 *r, FP2_BLS12381 *x); -/** @brief Set ECP2 to point(x,y) given x and y - * - Point P set to infinity if no such point on the curve. - @param P ECP2 instance to be set (x,y) - @param x FP2 x coordinate of point - @param y FP2 y coordinate of point - @return 1 if point exists, else 0 - */ -extern int ECP2_BLS12381_set(ECP2_BLS12381 *P, FP2_BLS12381 *x, FP2_BLS12381 *y); -/** @brief Set ECP to point(x,[y]) given x and sign of y - * - Point P set to infinity if no such point on the curve. Otherwise y coordinate is calculated from x. - @param P ECP instance to be set (x,[y]) - @param x BIG x coordinate of point - @param s sign of y - @return 1 if point exists, else 0 - */ -extern int ECP2_BLS12381_setx(ECP2_BLS12381 *P, FP2_BLS12381 *x, int s); -/** @brief Negation of an ECP2 point - * - @param P ECP2 instance, on exit = -P - */ -extern void ECP2_BLS12381_neg(ECP2_BLS12381 *P); -/** @brief Doubles an ECP2 instance P - * - @param P ECP2 instance, on exit =2*P - */ -extern int ECP2_BLS12381_dbl(ECP2_BLS12381 *P); -/** @brief Adds ECP2 instance Q to ECP2 instance P - * - @param P ECP2 instance, on exit =P+Q - @param Q ECP2 instance to be added to P - */ -extern int ECP2_BLS12381_add(ECP2_BLS12381 *P, ECP2_BLS12381 *Q); -/** @brief Subtracts ECP instance Q from ECP2 instance P - * - @param P ECP2 instance, on exit =P-Q - @param Q ECP2 instance to be subtracted from P - */ -extern void ECP2_BLS12381_sub(ECP2_BLS12381 *P, ECP2_BLS12381 *Q); -/** @brief Multiplies an ECP2 instance P by a BIG, side-channel resistant - * - Uses fixed sized windows. - @param P ECP2 instance, on exit =b*P - @param b BIG number multiplier - - */ -extern void ECP2_BLS12381_mul(ECP2_BLS12381 *P, BIG_384_58 b); -/** @brief Multiplies an ECP2 instance P by the internal modulus p, using precalculated Frobenius constant f - * - Fast point multiplication using Frobenius - @param P ECP2 instance, on exit = p*P - @param f FP2 precalculated Frobenius constant - - */ -extern void ECP2_BLS12381_frob(ECP2_BLS12381 *P, FP2_BLS12381 *f); -/** @brief Calculates P=b[0]*Q[0]+b[1]*Q[1]+b[2]*Q[2]+b[3]*Q[3] - * - @param P ECP2 instance, on exit = b[0]*Q[0]+b[1]*Q[1]+b[2]*Q[2]+b[3]*Q[3] - @param Q ECP2 array of 4 points - @param b BIG array of 4 multipliers - */ -extern void ECP2_BLS12381_mul4(ECP2_BLS12381 *P, ECP2_BLS12381 *Q, BIG_384_58 *b); - -/** @brief Multiplies random point by co-factor - * - @param Q ECP2 multiplied by co-factor - */ -extern void ECP2_BLS12381_cfp(ECP2_BLS12381 *Q); - -/** @brief Maps random BIG to curve point in constant time - * - @param Q ECP2 instance - @param x FP2 derived from hash - */ -extern void ECP2_BLS12381_map2point(ECP2_BLS12381 *Q, FP2_BLS12381 *x); - - -/** @brief Maps random BIG to curve point using hunt-and-peck - * - @param Q ECP2 instance - @param x Fp derived from hash - */ -extern void ECP2_BLS12381_hap2point(ECP2_BLS12381 *Q, BIG_384_58 x); - -/** @brief Maps random BIG to curve point of correct order - * - @param P ECP2 instance of correct order - @param w OCTET byte array to be mapped - */ -extern void ECP2_BLS12381_mapit(ECP2_BLS12381 *P, octet *w); - -/** @brief Get Group Generator from ROM - * - @param G ECP2 instance - @return 1 if point exists, else 0 - */ -extern int ECP2_BLS12381_generator(ECP2_BLS12381 *G); - -#endif diff --git a/impl/cbits/ecp_BLS12381.c b/impl/cbits/ecp_BLS12381.c deleted file mode 100644 index c86d3e9ef..000000000 --- a/impl/cbits/ecp_BLS12381.c +++ /dev/null @@ -1,1762 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/* CORE Elliptic Curve Functions */ -/* SU=m, SU is Stack Usage (Weierstrass Curves) */ - -//#define HAS_MAIN - -#include -#include "ecp_BLS12381.h" - -/* test for P=O point-at-infinity */ -int ECP_BLS12381_isinf(ECP_BLS12381 *P) -{ - -#if CURVETYPE_BLS12381==EDWARDS - return (FP_BLS12381_iszilch(&(P->x)) && FP_BLS12381_equals(&(P->y), &(P->z))); -#endif -#if CURVETYPE_BLS12381==WEIERSTRASS - return (FP_BLS12381_iszilch(&(P->x)) && FP_BLS12381_iszilch(&(P->z))); -#endif -#if CURVETYPE_BLS12381==MONTGOMERY - return FP_BLS12381_iszilch(&(P->z)); -#endif - -} - -/* Conditional swap of P and Q dependant on d */ -static void ECP_BLS12381_cswap(ECP_BLS12381 *P, ECP_BLS12381 *Q, int d) -{ - FP_BLS12381_cswap(&(P->x), &(Q->x), d); -#if CURVETYPE_BLS12381!=MONTGOMERY - FP_BLS12381_cswap(&(P->y), &(Q->y), d); -#endif - FP_BLS12381_cswap(&(P->z), &(Q->z), d); -} - -#if CURVETYPE_BLS12381!=MONTGOMERY -/* Conditional move Q to P dependant on d */ -static void ECP_BLS12381_cmove(ECP_BLS12381 *P, ECP_BLS12381 *Q, int d) -{ - FP_BLS12381_cmove(&(P->x), &(Q->x), d); -#if CURVETYPE_BLS12381!=MONTGOMERY - FP_BLS12381_cmove(&(P->y), &(Q->y), d); -#endif - FP_BLS12381_cmove(&(P->z), &(Q->z), d); -} - -/* return 1 if b==c, no branching */ -static int teq(sign32 b, sign32 c) -{ - sign32 x = b ^ c; - x -= 1; // if x=0, x now -1 - return (int)((x >> 31) & 1); -} -#endif // CURVETYPE_BLS12381!=MONTGOMERY - -#if CURVETYPE_BLS12381!=MONTGOMERY -/* Constant time select from pre-computed table */ -static void ECP_BLS12381_select(ECP_BLS12381 *P, ECP_BLS12381 W[], sign32 b) -{ - ECP_BLS12381 MP; - sign32 m = b >> 31; - sign32 babs = (b ^ m) - m; - - babs = (babs - 1) / 2; - - ECP_BLS12381_cmove(P, &W[0], teq(babs, 0)); // conditional move - ECP_BLS12381_cmove(P, &W[1], teq(babs, 1)); - ECP_BLS12381_cmove(P, &W[2], teq(babs, 2)); - ECP_BLS12381_cmove(P, &W[3], teq(babs, 3)); - ECP_BLS12381_cmove(P, &W[4], teq(babs, 4)); - ECP_BLS12381_cmove(P, &W[5], teq(babs, 5)); - ECP_BLS12381_cmove(P, &W[6], teq(babs, 6)); - ECP_BLS12381_cmove(P, &W[7], teq(babs, 7)); - - ECP_BLS12381_copy(&MP, P); - ECP_BLS12381_neg(&MP); // minus P - ECP_BLS12381_cmove(P, &MP, (int)(m & 1)); -} -#endif - -/* Test P == Q */ -/* SU=168 */ -int ECP_BLS12381_equals(ECP_BLS12381 *P, ECP_BLS12381 *Q) -{ - FP_BLS12381 a, b; - - FP_BLS12381_mul(&a, &(P->x), &(Q->z)); - FP_BLS12381_mul(&b, &(Q->x), &(P->z)); - if (!FP_BLS12381_equals(&a, &b)) return 0; - -#if CURVETYPE_BLS12381!=MONTGOMERY - FP_BLS12381_mul(&a, &(P->y), &(Q->z)); - FP_BLS12381_mul(&b, &(Q->y), &(P->z)); - if (!FP_BLS12381_equals(&a, &b)) return 0; -#endif - - return 1; - -} - -/* Set P=Q */ -/* SU=16 */ -void ECP_BLS12381_copy(ECP_BLS12381 *P, ECP_BLS12381 *Q) -{ - FP_BLS12381_copy(&(P->x), &(Q->x)); -#if CURVETYPE_BLS12381!=MONTGOMERY - FP_BLS12381_copy(&(P->y), &(Q->y)); -#endif - FP_BLS12381_copy(&(P->z), &(Q->z)); -} - -/* Set P=-Q */ -#if CURVETYPE_BLS12381!=MONTGOMERY -/* SU=8 */ -void ECP_BLS12381_neg(ECP_BLS12381 *P) -{ -#if CURVETYPE_BLS12381==WEIERSTRASS - FP_BLS12381_neg(&(P->y), &(P->y)); - FP_BLS12381_norm(&(P->y)); -#else - FP_BLS12381_neg(&(P->x), &(P->x)); - FP_BLS12381_norm(&(P->x)); -#endif - -} -#endif - -/* Set P=O */ -void ECP_BLS12381_inf(ECP_BLS12381 *P) -{ - FP_BLS12381_zero(&(P->x)); -#if CURVETYPE_BLS12381!=MONTGOMERY - FP_BLS12381_one(&(P->y)); -#endif -#if CURVETYPE_BLS12381!=EDWARDS - FP_BLS12381_zero(&(P->z)); -#else - FP_BLS12381_one(&(P->z)); -#endif -} - -/* Calculate right Hand Side of curve equation y^2=RHS */ -/* SU=56 */ -void ECP_BLS12381_rhs(FP_BLS12381 *v, FP_BLS12381 *x) -{ -#if CURVETYPE_BLS12381==WEIERSTRASS - /* x^3+Ax+B */ - FP_BLS12381 t; - FP_BLS12381_sqr(&t, x); - FP_BLS12381_mul(&t, &t, x); - -#if CURVE_A_BLS12381 == -3 - - FP_BLS12381_neg(v, x); - FP_BLS12381_norm(v); - FP_BLS12381_imul(v, v, -CURVE_A_BLS12381); - FP_BLS12381_norm(v); - FP_BLS12381_add(v, &t, v); -#else - FP_BLS12381_copy(v, &t); -#endif - - FP_BLS12381_rcopy(&t, CURVE_B_BLS12381); - - FP_BLS12381_add(v, &t, v); - FP_BLS12381_reduce(v); -#endif - -#if CURVETYPE_BLS12381==EDWARDS - /* (Ax^2-1)/(Bx^2-1) */ - FP_BLS12381 t, one; - FP_BLS12381_sqr(v, x); - FP_BLS12381_one(&one); - FP_BLS12381_rcopy(&t, CURVE_B_BLS12381); - - FP_BLS12381_mul(&t, v, &t); - FP_BLS12381_sub(&t, &t, &one); - FP_BLS12381_norm(&t); -#if CURVE_A_BLS12381 == 1 - FP_BLS12381_sub(v, v, &one); -#endif - -#if CURVE_A_BLS12381 == -1 - FP_BLS12381_add(v, v, &one); - FP_BLS12381_norm(v); - FP_BLS12381_neg(v, v); -#endif - FP_BLS12381_norm(v); - FP_BLS12381_inv(&t, &t, NULL); - FP_BLS12381_mul(v, v, &t); - FP_BLS12381_reduce(v); -#endif - -#if CURVETYPE_BLS12381==MONTGOMERY - /* x^3+Ax^2+x */ - FP_BLS12381 x2, x3; - FP_BLS12381_sqr(&x2, x); - FP_BLS12381_mul(&x3, &x2, x); - FP_BLS12381_copy(v, x); - FP_BLS12381_imul(&x2, &x2, CURVE_A_BLS12381); - FP_BLS12381_add(v, v, &x2); - FP_BLS12381_add(v, v, &x3); - FP_BLS12381_reduce(v); -#endif -} - -#if CURVETYPE_BLS12381==MONTGOMERY - -/* Set P=(x,{y}) */ - -int ECP_BLS12381_set(ECP_BLS12381 *P, BIG_384_58 x) -{ - //BIG_384_58 m, b; - FP_BLS12381 rhs; - //BIG_384_58_rcopy(m, Modulus_BLS12381); - - FP_BLS12381_nres(&rhs, x); - - ECP_BLS12381_rhs(&rhs, &rhs); - - //FP_BLS12381_redc(b, &rhs); - //if (BIG_384_58_jacobi(b, m) != 1) - if (!FP_BLS12381_qr(&rhs,NULL)) - { - ECP_BLS12381_inf(P); - return 0; - } - - FP_BLS12381_nres(&(P->x), x); - FP_BLS12381_one(&(P->z)); - return 1; -} - -/* Extract x coordinate as BIG */ -int ECP_BLS12381_get(BIG_384_58 x, ECP_BLS12381 *P) -{ - ECP_BLS12381 W; - ECP_BLS12381_copy(&W, P); - ECP_BLS12381_affine(&W); - if (ECP_BLS12381_isinf(&W)) return -1; - FP_BLS12381_redc(x, &(W.x)); - return 0; -} - - -#else -/* Extract (x,y) and return sign of y. If x and y are the same return only x */ -/* SU=16 */ -int ECP_BLS12381_get(BIG_384_58 x, BIG_384_58 y, ECP_BLS12381 *P) -{ - ECP_BLS12381 W; - int s; - ECP_BLS12381_copy(&W, P); - ECP_BLS12381_affine(&W); - - if (ECP_BLS12381_isinf(&W)) return -1; - - FP_BLS12381_redc(y, &(W.y)); - s = BIG_384_58_parity(y); - - FP_BLS12381_redc(x, &(W.x)); - - return s; -} - -/* Set P=(x,{y}) */ -/* SU=96 */ -int ECP_BLS12381_set(ECP_BLS12381 *P, BIG_384_58 x, BIG_384_58 y) -{ - FP_BLS12381 rhs, y2; - - FP_BLS12381_nres(&y2, y); - FP_BLS12381_sqr(&y2, &y2); - FP_BLS12381_reduce(&y2); - - FP_BLS12381_nres(&rhs, x); - ECP_BLS12381_rhs(&rhs, &rhs); - - if (!FP_BLS12381_equals(&y2, &rhs)) - { - ECP_BLS12381_inf(P); - return 0; - } - - FP_BLS12381_nres(&(P->x), x); - FP_BLS12381_nres(&(P->y), y); - FP_BLS12381_one(&(P->z)); - return 1; -} - -/* Set P=(x,y), where y is calculated from x with sign s */ -/* SU=136 */ -int ECP_BLS12381_setx(ECP_BLS12381 *P, BIG_384_58 x, int s) -{ - FP_BLS12381 rhs,hint; - BIG_384_58 t;//, m; - //BIG_384_58_rcopy(m, Modulus_BLS12381); - - FP_BLS12381_nres(&rhs, x); - - ECP_BLS12381_rhs(&rhs, &rhs); - - //FP_BLS12381_redc(t, &rhs); - //if (BIG_384_58_jacobi(t, m) != 1) - if (!FP_BLS12381_qr(&rhs,&hint)) - { - ECP_BLS12381_inf(P); - return 0; - } - - FP_BLS12381_nres(&(P->x), x); - FP_BLS12381_sqrt(&(P->y), &rhs,&hint); - - FP_BLS12381_redc(t, &(P->y)); - - if (BIG_384_58_parity(t) != s) - FP_BLS12381_neg(&(P->y), &(P->y)); - FP_BLS12381_reduce(&(P->y)); - FP_BLS12381_one(&(P->z)); - return 1; -} - -#endif - -void ECP_BLS12381_cfp(ECP_BLS12381 *P) -{ /* multiply point by curves cofactor */ - BIG_384_58 c; - int cf = CURVE_Cof_I_BLS12381; - if (cf == 1) return; - if (cf == 4) - { - ECP_BLS12381_dbl(P); - ECP_BLS12381_dbl(P); - return; - } - if (cf == 8) - { - ECP_BLS12381_dbl(P); - ECP_BLS12381_dbl(P); - ECP_BLS12381_dbl(P); - return; - } - BIG_384_58_rcopy(c, CURVE_Cof_BLS12381); - ECP_BLS12381_mul(P, c); - return; -} - -/* Hunt and Peck a BIG to a curve point */ -/* -void ECP_BLS12381_hap2point(ECP_BLS12381 *P,BIG_384_58 h) -{ - BIG_384_58 x; - BIG_384_58_copy(x,h); - - for (;;) - { -#if CURVETYPE_BLS12381!=MONTGOMERY - ECP_BLS12381_setx(P,x,0); -#else - ECP_BLS12381_set(P,x); -#endif - BIG_384_58_inc(x,1); BIG_384_58_norm(x); - if (!ECP_BLS12381_isinf(P)) break; - } -} -*/ -/* Constant time Map to Point */ -void ECP_BLS12381_map2point(ECP_BLS12381 *P,FP_BLS12381 *h) -{ -#if CURVETYPE_BLS12381==MONTGOMERY -// Elligator 2 - int qres; - BIG_384_58 a; - FP_BLS12381 X1,X2,w,t,one,A,N,D,hint; - //BIG_384_58_zero(a); BIG_384_58_inc(a,CURVE_A_BLS12381); BIG_384_58_norm(a); FP_BLS12381_nres(&A,a); - FP_BLS12381_from_int(&A,CURVE_A_BLS12381); - FP_BLS12381_copy(&t,h); - FP_BLS12381_sqr(&t,&t); // t^2 - - if (PM1D2_BLS12381 == 2) - FP_BLS12381_add(&t,&t,&t); // 2t^2 - if (PM1D2_BLS12381 == 1) - FP_BLS12381_neg(&t,&t); // -t^2 - if (PM1D2_BLS12381 > 2) - FP_BLS12381_imul(&t,&t,QNRI_BLS12381); // precomputed QNR - FP_BLS12381_norm(&t); // z.t^2 - - FP_BLS12381_one(&one); - FP_BLS12381_add(&D,&t,&one); FP_BLS12381_norm(&D); // Denominator D=1+z.t^2 - - FP_BLS12381_copy(&X1,&A); - FP_BLS12381_neg(&X1,&X1); FP_BLS12381_norm(&X1); // X1 = -A/D - FP_BLS12381_copy(&X2,&X1); - FP_BLS12381_mul(&X2,&X2,&t); // X2 = -At/D - - FP_BLS12381_sqr(&w,&X1); FP_BLS12381_mul(&N,&w,&X1); // w=X1^2, N=X1^3 - FP_BLS12381_mul(&w,&w,&A); FP_BLS12381_mul(&w,&w,&D); FP_BLS12381_add(&N,&N,&w); // N = X1^3+ADX1^2 - FP_BLS12381_sqr(&t,&D); - FP_BLS12381_mul(&t,&t,&X1); - FP_BLS12381_add(&N,&N,&t); // N=X1^3+ADX1^2+D^2X1 // Numerator of gx = N/D^3 - FP_BLS12381_norm(&N); - - FP_BLS12381_mul(&t,&N,&D); // N.D - qres=FP_BLS12381_qr(&t,&hint); // *** exp - FP_BLS12381_inv(&w,&t,&hint); - FP_BLS12381_mul(&D,&w,&N); // 1/D - FP_BLS12381_mul(&X1,&X1,&D); // get X1 - FP_BLS12381_mul(&X2,&X2,&D); // get X2 - FP_BLS12381_cmove(&X1,&X2,1-qres); - FP_BLS12381_redc(a,&X1); - - ECP_BLS12381_set(P,a); - return; -#endif - -#if CURVETYPE_BLS12381==EDWARDS -// Elligator 2 - map to Montgomery, place point, map back - int qres,ne,rfc,qnr; - BIG_384_58 x,y; - FP_BLS12381 X1,X2,t,w,one,A,w1,w2,B,Y,K,D,hint,Y3; - FP_BLS12381_one(&one); - -#if MODTYPE_BLS12381 != GENERALISED_MERSENNE -// its NOT goldilocks! -// Figure out the Montgomery curve parameters - - FP_BLS12381_rcopy(&B,CURVE_B_BLS12381); -#if CURVE_A_BLS12381 == 1 - FP_BLS12381_add(&A,&B,&one); // A=B+1 - FP_BLS12381_sub(&B,&B,&one); // B=B-1 -#else - FP_BLS12381_sub(&A,&B,&one); // A=B-1 - FP_BLS12381_add(&B,&B,&one); // B=B+1 -#endif - FP_BLS12381_norm(&A); FP_BLS12381_norm(&B); - - FP_BLS12381_div2(&A,&A); // (A+B)/2 - FP_BLS12381_div2(&B,&B); // (B-A)/2 - FP_BLS12381_div2(&B,&B); // (B-A)/4 - - FP_BLS12381_neg(&K,&B); FP_BLS12381_norm(&K); - //FP_BLS12381_inv(&K,&K,NULL); // K - FP_BLS12381_invsqrt(&K,&w1,&K); - - rfc=RIADZ_BLS12381; - if (rfc) - { // RFC7748 method applies - FP_BLS12381_mul(&A,&A,&K); // = J - FP_BLS12381_mul(&K,&K,&w1); - //FP_BLS12381_sqrt(&K,&K,NULL); - } else { // generic method - FP_BLS12381_sqr(&B,&B); - } -#else - FP_BLS12381_from_int(&A,156326); - rfc=1; -#endif -// Map to this Montgomery curve X^2=X^3+AX^2+BX - - FP_BLS12381_copy(&t,h); - FP_BLS12381_sqr(&t,&t); // t^2 - - if (PM1D2_BLS12381 == 2) - { - FP_BLS12381_add(&t,&t,&t); // 2t^2 - qnr=2; - } - if (PM1D2_BLS12381 == 1) - { - FP_BLS12381_neg(&t,&t); // -t^2 - qnr=-1; - } - if (PM1D2_BLS12381 > 2) - { - FP_BLS12381_imul(&t,&t,QNRI_BLS12381); // precomputed QNR - qnr=QNRI_BLS12381; - } - FP_BLS12381_norm(&t); - FP_BLS12381_add(&D,&t,&one); FP_BLS12381_norm(&D); // Denominator=(1+z.u^2) - - FP_BLS12381_copy(&X1,&A); - FP_BLS12381_neg(&X1,&X1); FP_BLS12381_norm(&X1); // X1=-(J/K).inv(1+z.u^2) - FP_BLS12381_mul(&X2,&X1,&t); // X2=X1*z.u^2 - -// Figure out RHS of Montgomery curve in rational form gx1/d^3 - - FP_BLS12381_sqr(&w,&X1); FP_BLS12381_mul(&w1,&w,&X1); // w=X1^2, w1=X1^3 - FP_BLS12381_mul(&w,&w,&A); FP_BLS12381_mul(&w,&w,&D); FP_BLS12381_add(&w1,&w1,&w); // w1 = X1^3+ADX1^2 - FP_BLS12381_sqr(&w2,&D); - if (!rfc) - { - FP_BLS12381_mul(&w,&X1,&B); - FP_BLS12381_mul(&w2,&w2,&w); // - FP_BLS12381_add(&w1,&w1,&w2); // w1=X1^3+ADX1^2+BD^2X1 - } else { - FP_BLS12381_mul(&w2,&w2,&X1); - FP_BLS12381_add(&w1,&w1,&w2); // w1=X1^3+ADX1^2+D^2X1 // was &X1 - } - FP_BLS12381_norm(&w1); - - FP_BLS12381_mul(&B,&w1,&D); // gx1=num/den^3 - is_qr num*den (same as num/den, same as num/den^3) - qres=FP_BLS12381_qr(&B,&hint); // *** - FP_BLS12381_inv(&w,&B,&hint); - FP_BLS12381_mul(&D,&w,&w1); // 1/D - FP_BLS12381_mul(&X1,&X1,&D); // get X1 - FP_BLS12381_mul(&X2,&X2,&D); // get X2 - FP_BLS12381_sqr(&D,&D); - - FP_BLS12381_sqrt(&Y,&B,&hint); // sqrt(num*den) - FP_BLS12381_mul(&Y,&Y,&D); // sqrt(num/den^3) - - FP_BLS12381_imul(&B,&B,qnr); // now for gx2 = Z.u^2.gx1 - FP_BLS12381_rcopy(&w,CURVE_HTPC_BLS12381); // qnr^C3 - FP_BLS12381_mul(&hint,&hint,&w); // modify hint for gx2 - - FP_BLS12381_sqrt(&Y3,&B,&hint); // second candidate - FP_BLS12381_mul(&D,&D,h); - FP_BLS12381_mul(&Y3,&Y3,&D); - - FP_BLS12381_cmove(&X1,&X2,1-qres); // pick correct one - FP_BLS12381_cmove(&Y,&Y3,1-qres); - -// correct sign of Y - FP_BLS12381_neg(&w,&Y); FP_BLS12381_norm(&w); - FP_BLS12381_cmove(&Y,&w,qres^FP_BLS12381_sign(&Y)); - - if (!rfc) - { - FP_BLS12381_mul(&X1,&X1,&K); - FP_BLS12381_mul(&Y,&Y,&K); - } - -#if MODTYPE_BLS12381 == GENERALISED_MERSENNE -// GOLDILOCKS isogeny - FP_BLS12381_sqr(&t,&X1); // t=u^2 - FP_BLS12381_add(&w,&t,&one); // w=u^2+1 - FP_BLS12381_norm(&w); - FP_BLS12381_sub(&t,&t,&one); // t=u^2-1 - FP_BLS12381_norm(&t); - FP_BLS12381_mul(&w1,&t,&Y); // w1=v(u^2-1) - FP_BLS12381_add(&w1,&w1,&w1); - FP_BLS12381_add(&X2,&w1,&w1); - FP_BLS12381_norm(&X2); // w1=4v(u^2-1) - FP_BLS12381_sqr(&t,&t); // t=(u^2-1)^2 - FP_BLS12381_sqr(&Y,&Y); // v^2 - FP_BLS12381_add(&Y,&Y,&Y); - FP_BLS12381_add(&Y,&Y,&Y); - FP_BLS12381_norm(&Y); // 4v^2 - FP_BLS12381_add(&B,&t,&Y); // w2=(u^2-1)^2+4v^2 - FP_BLS12381_norm(&B); // X1=w1/w2 - X2=w1, B=w2 - - FP_BLS12381_sub(&w2,&Y,&t); // w2=4v^2-(u^2-1)^2 - FP_BLS12381_norm(&w2); - FP_BLS12381_mul(&w2,&w2,&X1); // w2=u(4v^2-(u^2-1)^2) - FP_BLS12381_mul(&t,&t,&X1); // t=u(u^2-1)^2 - FP_BLS12381_div2(&Y,&Y); // 2v^2 - FP_BLS12381_mul(&w1,&Y,&w); // w1=2v^2(u^2+1) - FP_BLS12381_sub(&w1,&t,&w1); // w1=u(u^2-1)^2 - 2v^2(u^2+1) - FP_BLS12381_norm(&w1); // Y=w2/w1 - - FP_BLS12381_mul(&t,&X2,&w1); // output in projective to avoid inversion - FP_BLS12381_copy(&(P->x),&t); - FP_BLS12381_mul(&t,&w2,&B); - FP_BLS12381_copy(&(P->y),&t); - FP_BLS12381_mul(&t,&w1,&B); - FP_BLS12381_copy(&(P->z),&t); - - return; - -#else - FP_BLS12381_add(&w1,&X1,&one); FP_BLS12381_norm(&w1); // (s+1) - FP_BLS12381_sub(&w2,&X1,&one); FP_BLS12381_norm(&w2); // (s-1) - FP_BLS12381_mul(&t,&w1,&Y); - FP_BLS12381_mul(&X1,&X1,&w1); - - if (rfc) - FP_BLS12381_mul(&X1,&X1,&K); - - FP_BLS12381_mul(&Y,&Y,&w2); // output in projective to avoid inversion - FP_BLS12381_copy(&(P->x),&X1); - FP_BLS12381_copy(&(P->y),&Y); - FP_BLS12381_copy(&(P->z),&t); - return; -#endif - -#endif - -#if CURVETYPE_BLS12381==WEIERSTRASS - int sgn,ne; - BIG_384_58 a,x,y; - FP_BLS12381 X1,X2,X3,t,w,one,A,B,Y,D; - FP_BLS12381 D2,hint,GX1,Y3; - -#if HTC_ISO_BLS12381 != 0 -// Map to point on isogenous curve - int i,k,isox,isoy,iso=HTC_ISO_BLS12381; - FP_BLS12381 xnum,xden,ynum,yden; - BIG_384_58 z; - FP_BLS12381_rcopy(&A,CURVE_Ad_BLS12381); - FP_BLS12381_rcopy(&B,CURVE_Bd_BLS12381); -#else - FP_BLS12381_from_int(&A,CURVE_A_BLS12381); - FP_BLS12381_rcopy(&B,CURVE_B_BLS12381); -#endif - - FP_BLS12381_one(&one); - FP_BLS12381_copy(&t,h); - sgn=FP_BLS12381_sign(&t); - -#if CURVE_A_BLS12381 != 0 || HTC_ISO_BLS12381 != 0 - - FP_BLS12381_sqr(&t,&t); - FP_BLS12381_imul(&t,&t,RIADZ_BLS12381); // Z from hash-to-point draft standard - FP_BLS12381_add(&w,&t,&one); // w=Zt^2+1 - FP_BLS12381_norm(&w); - - FP_BLS12381_mul(&w,&w,&t); // w=Z^2*t^4+Zt^2 - FP_BLS12381_mul(&D,&A,&w); // A=Aw - - FP_BLS12381_add(&w,&w,&one); FP_BLS12381_norm(&w); - FP_BLS12381_mul(&w,&w,&B); - FP_BLS12381_neg(&w,&w); // -B(w+1) - FP_BLS12381_norm(&w); - - FP_BLS12381_copy(&X2,&w); // Numerators - FP_BLS12381_mul(&X3,&t,&X2); - -// x^3+Ad^2x+Bd^3 - FP_BLS12381_sqr(&GX1,&X2); - FP_BLS12381_sqr(&D2,&D); FP_BLS12381_mul(&w,&A,&D2); FP_BLS12381_add(&GX1,&GX1,&w); FP_BLS12381_norm(&GX1); FP_BLS12381_mul(&GX1,&GX1,&X2); FP_BLS12381_mul(&D2,&D2,&D); FP_BLS12381_mul(&w,&B,&D2); FP_BLS12381_add(&GX1,&GX1,&w); FP_BLS12381_norm(&GX1); - - FP_BLS12381_mul(&w,&GX1,&D); - int qr=FP_BLS12381_qr(&w,&hint); // qr(ad) - only exp happens here - FP_BLS12381_inv(&D,&w,&hint); // d=1/(ad) - FP_BLS12381_mul(&D,&D,&GX1); // 1/d - FP_BLS12381_mul(&X2,&X2,&D); // X2/=D - FP_BLS12381_mul(&X3,&X3,&D); // X3/=D - FP_BLS12381_mul(&t,&t,h); // t=Z.u^3 - FP_BLS12381_sqr(&D2,&D); - - FP_BLS12381_sqrt(&Y,&w,&hint); // first candidate if X2 is correct - FP_BLS12381_mul(&Y,&Y,&D2); - - FP_BLS12381_mul(&D2,&D2,&t); // second candidate if X3 is correct - FP_BLS12381_imul(&w,&w,RIADZ_BLS12381); - - FP_BLS12381_rcopy(&X1,CURVE_HTPC_BLS12381); - FP_BLS12381_mul(&hint,&hint,&X1); // modify hint - - FP_BLS12381_sqrt(&Y3,&w,&hint); - FP_BLS12381_mul(&Y3,&Y3,&D2); - - FP_BLS12381_cmove(&X2,&X3,!qr); - FP_BLS12381_cmove(&Y,&Y3,!qr); - - ne=FP_BLS12381_sign(&Y)^sgn; - FP_BLS12381_neg(&w,&Y); FP_BLS12381_norm(&w); - FP_BLS12381_cmove(&Y,&w,ne); - -#if HTC_ISO_BLS12381 != 0 - -// (X2,Y) is on isogenous curve - k=0; - isox=iso; - isoy=3*(iso-1)/2; - -// xnum - FP_BLS12381_rcopy(&xnum,PC_BLS12381[k++]); - for (i=0;ix),&t); - - FP_BLS12381_mul(&t,&ynum,&xden); - FP_BLS12381_copy(&(P->y),&t); - - FP_BLS12381_mul(&t,&xden,&yden); - FP_BLS12381_copy(&(P->z),&t); - return; -#else - - FP_BLS12381_redc(x,&X2); - FP_BLS12381_redc(y,&Y); - ECP_BLS12381_set(P,x,y); - return; -#endif -#else -// SVDW - Shallue and van de Woestijne - FP_BLS12381_from_int(&Y,RIADZ_BLS12381); - ECP_BLS12381_rhs(&A,&Y); // A=g(Z) - FP_BLS12381_rcopy(&B,SQRTm3_BLS12381); - FP_BLS12381_imul(&B,&B,RIADZ_BLS12381); // Z*sqrt(-3) - - FP_BLS12381_sqr(&t,&t); - FP_BLS12381_mul(&Y,&A,&t); // tv1=u^2*g(Z) - FP_BLS12381_add(&t,&one,&Y); FP_BLS12381_norm(&t); // tv2=1+tv1 - FP_BLS12381_sub(&Y,&one,&Y); FP_BLS12381_norm(&Y); // tv1=1-tv1 - FP_BLS12381_mul(&D,&t,&Y); - FP_BLS12381_mul(&D,&D,&B); - - FP_BLS12381_copy(&w,&A); - FP_BLS12381_tpo(&D,&w); // tv3=inv0(tv1*tv2*z*sqrt(-3)) and sqrt(g(Z)) // *** - - FP_BLS12381_mul(&w,&w,&B); // tv4=Z.sqrt(-3).sqrt(g(Z)) - if (FP_BLS12381_sign(&w)==1) - { // depends only on sign of constant RIADZ - FP_BLS12381_neg(&w,&w); - FP_BLS12381_norm(&w); - } - FP_BLS12381_mul(&w,&w,&B); // Z.sqrt(-3) - FP_BLS12381_mul(&w,&w,h); // u - FP_BLS12381_mul(&w,&w,&Y); // tv1 - FP_BLS12381_mul(&w,&w,&D); // tv3 // tv5=u*tv1*tv3*tv4*Z*sqrt(-3) - - FP_BLS12381_from_int(&X1,RIADZ_BLS12381); - FP_BLS12381_copy(&X3,&X1); - FP_BLS12381_neg(&X1,&X1); FP_BLS12381_norm(&X1); FP_BLS12381_div2(&X1,&X1); // -Z/2 - FP_BLS12381_copy(&X2,&X1); - FP_BLS12381_sub(&X1,&X1,&w); FP_BLS12381_norm(&X1); - FP_BLS12381_add(&X2,&X2,&w); FP_BLS12381_norm(&X2); - FP_BLS12381_add(&A,&A,&A); - FP_BLS12381_add(&A,&A,&A); - FP_BLS12381_norm(&A); // 4*g(Z) - FP_BLS12381_sqr(&t,&t); - FP_BLS12381_mul(&t,&t,&D); - FP_BLS12381_sqr(&t,&t); // (tv2^2*tv3)^2 - FP_BLS12381_mul(&A,&A,&t); // 4*g(Z)*(tv2^2*tv3)^2 - - FP_BLS12381_add(&X3,&X3,&A); FP_BLS12381_norm(&X3); - - ECP_BLS12381_rhs(&w,&X2); - FP_BLS12381_cmove(&X3,&X2,FP_BLS12381_qr(&w,NULL)); // *** - ECP_BLS12381_rhs(&w,&X1); - FP_BLS12381_cmove(&X3,&X1,FP_BLS12381_qr(&w,NULL)); // *** - ECP_BLS12381_rhs(&w,&X3); - FP_BLS12381_sqrt(&Y,&w,NULL); // *** - FP_BLS12381_redc(x,&X3); - - ne=FP_BLS12381_sign(&Y)^sgn; - FP_BLS12381_neg(&w,&Y); FP_BLS12381_norm(&w); - FP_BLS12381_cmove(&Y,&w,ne); - - FP_BLS12381_redc(y,&Y); - ECP_BLS12381_set(P,x,y); - return; -#endif - -#endif -} - - -/* Map octet to point */ -/* -void ECP_BLS12381_mapit(ECP_BLS12381 *P, octet *W) -{ - BIG_384_58 q, x; - DBIG_384_58 dx; - BIG_384_58_rcopy(q, Modulus_BLS12381); - BIG_384_58_dfromBytesLen(dx, W->val,W->len); - BIG_384_58_dmod(x, dx, q); - ECP_BLS12381_hap2point(P,x); - ECP_BLS12381_cfp(P); -} -*/ -/* Convert P to Affine, from (x,y,z) to (x,y) */ -/* SU=160 */ -void ECP_BLS12381_affine(ECP_BLS12381 *P) -{ - FP_BLS12381 one, iz; - if (ECP_BLS12381_isinf(P)) return; - FP_BLS12381_one(&one); - if (FP_BLS12381_equals(&(P->z), &one)) return; - - FP_BLS12381_inv(&iz, &(P->z),NULL); - FP_BLS12381_mul(&(P->x), &(P->x), &iz); - -#if CURVETYPE_BLS12381==EDWARDS || CURVETYPE_BLS12381==WEIERSTRASS - - FP_BLS12381_mul(&(P->y), &(P->y), &iz); - FP_BLS12381_reduce(&(P->y)); - -#endif - - FP_BLS12381_reduce(&(P->x)); - FP_BLS12381_copy(&(P->z), &one); -} - -/* SU=120 */ -void ECP_BLS12381_outputxyz(ECP_BLS12381 *P) -{ - BIG_384_58 x, z; - if (ECP_BLS12381_isinf(P)) - { - printf("Infinity\n"); - return; - } - FP_BLS12381_reduce(&(P->x)); - FP_BLS12381_redc(x, &(P->x)); - FP_BLS12381_reduce(&(P->z)); - FP_BLS12381_redc(z, &(P->z)); - -#if CURVETYPE_BLS12381!=MONTGOMERY - BIG_384_58 y; - FP_BLS12381_reduce(&(P->y)); - FP_BLS12381_redc(y, &(P->y)); - printf("("); - BIG_384_58_output(x); - printf(","); - BIG_384_58_output(y); - printf(","); - BIG_384_58_output(z); - printf(")\n"); - -#else - printf("("); - BIG_384_58_output(x); - printf(","); - BIG_384_58_output(z); - printf(")\n"); -#endif -} - -/* SU=16 */ -/* Output point P */ -void ECP_BLS12381_output(ECP_BLS12381 *P) -{ - BIG_384_58 x; - if (ECP_BLS12381_isinf(P)) - { - printf("Infinity\n"); - return; - } - ECP_BLS12381_affine(P); -#if CURVETYPE_BLS12381!=MONTGOMERY - BIG_384_58 y; - FP_BLS12381_redc(x, &(P->x)); - FP_BLS12381_redc(y, &(P->y)); - printf("("); - BIG_384_58_output(x); - printf(","); - BIG_384_58_output(y); - printf(")\n"); -#else - FP_BLS12381_redc(x, &(P->x)); - printf("("); - BIG_384_58_output(x); - printf(")\n"); -#endif -} - -/* SU=16 */ -/* Output point P */ -void ECP_BLS12381_rawoutput(ECP_BLS12381 *P) -{ - BIG_384_58 x, z; - -#if CURVETYPE_BLS12381!=MONTGOMERY - BIG_384_58 y; - FP_BLS12381_redc(x, &(P->x)); - FP_BLS12381_redc(y, &(P->y)); - FP_BLS12381_redc(z, &(P->z)); - printf("("); - BIG_384_58_output(x); - printf(","); - BIG_384_58_output(y); - printf(","); - BIG_384_58_output(z); - printf(")\n"); -#else - FP_BLS12381_redc(x, &(P->x)); - FP_BLS12381_redc(z, &(P->z)); - printf("("); - BIG_384_58_output(x); - printf(","); - BIG_384_58_output(z); - printf(")\n"); -#endif -} - -/* SU=88 */ -/* Convert P to octet string */ -void ECP_BLS12381_toOctet(octet *W, ECP_BLS12381 *P, bool compress) -{ -#if CURVETYPE_BLS12381==MONTGOMERY - BIG_384_58 x; - ECP_BLS12381_get(x, P); - W->len = MODBYTES_384_58;// + 1; - //W->val[0] = 6; - BIG_384_58_toBytes(&(W->val[0]), x); -#else - BIG_384_58 x, y; - ECP_BLS12381_get(x, y, P); - if (compress) - { - W->val[0] = 0x02; - if (BIG_384_58_parity(y) == 1) W->val[0] = 0x03; - W->len = MODBYTES_384_58 + 1; - BIG_384_58_toBytes(&(W->val[1]), x); - } - else - { - W->val[0] = 0x04; - W->len = 2 * MODBYTES_384_58 + 1; - BIG_384_58_toBytes(&(W->val[1]), x); - BIG_384_58_toBytes(&(W->val[MODBYTES_384_58 + 1]), y); - } -#endif -} - -void ECP_BLS12381_toOctet_ZCash(octet *W, ECP_BLS12381 *P) -{ - BIG_384_58 x, y, x_neg, y_neg; - ECP_BLS12381_get(x, y, P); - - BIG_384_58_toBytes(&(W->val[0]), x); - W->len = MODBYTES_384_58; - - W->val[0] |= 128; // 0x1000 0000: Compressed form - - // is this lexicographic larger, or the negation? - char t1[MODBYTES_384_58]; - octet T1 = {0,sizeof(t1),t1}; - BIG_384_58_toBytes(&(T1.val[0]), y); - - ECP_BLS12381 P_neg; - ECP_BLS12381_copy(&P_neg, P); - ECP_BLS12381_neg(&P_neg); - ECP_BLS12381_get(x_neg, y_neg, &P_neg); - char t2[MODBYTES_384_58]; - octet T2 = {0,sizeof(t2),t2}; - BIG_384_58_toBytes(&(T2.val[0]), y_neg); - - if (memcmp(T1.val, T2.val, MODBYTES_384_58) > 0) { - W->val[0] |= 32; // 0b0010 0000: It is the larger of the two y - } else { - } -} - -/* SU=88 */ -/* Restore P from octet string */ -int ECP_BLS12381_fromOctet(ECP_BLS12381 *P, octet *W) -{ -#if CURVETYPE_BLS12381==MONTGOMERY - BIG_384_58 x; - BIG_384_58_fromBytes(x, &(W->val[0])); - if (ECP_BLS12381_set(P, x)) return 1; - return 0; -#else - BIG_384_58 x, y; - int typ = W->val[0]; - BIG_384_58_fromBytes(x, &(W->val[1])); - if (typ == 0x04) - { - BIG_384_58_fromBytes(y, &(W->val[MODBYTES_384_58 + 1])); - if (ECP_BLS12381_set(P, x, y)) return 1; - } - if (typ == 0x02 || typ == 0x03) - { - if (ECP_BLS12381_setx(P, x, typ & 1)) return 1; - } - return 0; -#endif -} - -int ECP_BLS12381_fromOctet_ZCash(ECP_BLS12381 *P, octet *W) -{ - BIG_384_58 x; - if (!(W->val[0] & 128)) return 0; // only accept compressed - int large = (W->val[0] & 32); - W->val[0] &= 31; // mask high bits - - BIG_384_58_fromBytes(x, &(W->val[0])); - if (!ECP_BLS12381_setx(P, x, 0)) return 0; - - // try to encode again to find out sign - // (expensive but easy to implement) - char t1[MODBYTES_384_58]; - octet T1 = {0,sizeof(t1),t1}; - ECP_BLS12381_toOctet_ZCash(&T1, P); - - if ((T1.val[0] & 32) != large) { - // need to negate second component of P - ECP_BLS12381_neg(P); - } - return 1; -} - - -/* Set P=2P */ -/* SU=272 */ -void ECP_BLS12381_dbl(ECP_BLS12381 *P) -{ -#if CURVETYPE_BLS12381==WEIERSTRASS - FP_BLS12381 t0, t1, t2, t3, x3, y3, z3, b; - -#if CURVE_A_BLS12381 == 0 - FP_BLS12381_sqr(&t0, &(P->y)); //t0.sqr(); - FP_BLS12381_mul(&t1, &(P->y), &(P->z)); //t1.mul(z); - - FP_BLS12381_sqr(&t2, &(P->z)); //t2.sqr(); - FP_BLS12381_add(&(P->z), &t0, &t0); //z.add(t0); - FP_BLS12381_norm(&(P->z)); //z.norm(); - FP_BLS12381_add(&(P->z), &(P->z), &(P->z)); //z.add(z); - FP_BLS12381_add(&(P->z), &(P->z), &(P->z)); //z.add(z); - FP_BLS12381_norm(&(P->z)); //z.norm(); - - FP_BLS12381_imul(&t2, &t2, 3 * CURVE_B_I_BLS12381); //t2.imul(3*ROM.CURVE_B_I); - FP_BLS12381_mul(&x3, &t2, &(P->z)); //x3.mul(z); - - FP_BLS12381_add(&y3, &t0, &t2); //y3.add(t2); - FP_BLS12381_norm(&y3); //y3.norm(); - FP_BLS12381_mul(&(P->z), &(P->z), &t1); //z.mul(t1); - - FP_BLS12381_add(&t1, &t2, &t2); //t1.add(t2); - FP_BLS12381_add(&t2, &t2, &t1); //t2.add(t1); - FP_BLS12381_sub(&t0, &t0, &t2); //t0.sub(t2); - FP_BLS12381_norm(&t0); //t0.norm(); - FP_BLS12381_mul(&y3, &y3, &t0); //y3.mul(t0); - FP_BLS12381_add(&y3, &y3, &x3); //y3.add(x3); - FP_BLS12381_mul(&t1, &(P->x), &(P->y)); //t1.mul(y); - - FP_BLS12381_norm(&t0); //x.norm(); - FP_BLS12381_mul(&(P->x), &t0, &t1); //x.mul(t1); - FP_BLS12381_add(&(P->x), &(P->x), &(P->x)); //x.add(x); - FP_BLS12381_norm(&(P->x)); //x.norm(); - FP_BLS12381_copy(&(P->y), &y3); //y.copy(y3); - FP_BLS12381_norm(&(P->y)); //y.norm(); -#else - - if (CURVE_B_I_BLS12381 == 0) //if (ROM.CURVE_B_I==0) - FP_BLS12381_rcopy(&b, CURVE_B_BLS12381); //b.copy(new FP(new BIG(ROM.CURVE_B))); - - FP_BLS12381_sqr(&t0, &(P->x)); //t0.sqr(); //1 x^2 - FP_BLS12381_sqr(&t1, &(P->y)); //t1.sqr(); //2 y^2 - FP_BLS12381_sqr(&t2, &(P->z)); //t2.sqr(); //3 - - FP_BLS12381_mul(&t3, &(P->x), &(P->y)); //t3.mul(y); //4 - FP_BLS12381_add(&t3, &t3, &t3); //t3.add(t3); - FP_BLS12381_norm(&t3); //t3.norm();//5 - - FP_BLS12381_mul(&z3, &(P->z), &(P->x)); //z3.mul(x); //6 - FP_BLS12381_add(&z3, &z3, &z3); //z3.add(z3); - FP_BLS12381_norm(&z3); //z3.norm();//7 - - if (CURVE_B_I_BLS12381 == 0) //if (ROM.CURVE_B_I==0) - FP_BLS12381_mul(&y3, &t2, &b); //y3.mul(b); //8 - else - FP_BLS12381_imul(&y3, &t2, CURVE_B_I_BLS12381); //y3.imul(ROM.CURVE_B_I); - - FP_BLS12381_sub(&y3, &y3, &z3); //y3.sub(z3); //y3.norm(); //9 *** - FP_BLS12381_add(&x3, &y3, &y3); //x3.add(y3); - FP_BLS12381_norm(&x3); //x3.norm();//10 - - FP_BLS12381_add(&y3, &y3, &x3); //y3.add(x3); //y3.norm();//11 - FP_BLS12381_sub(&x3, &t1, &y3); //x3.sub(y3); - FP_BLS12381_norm(&x3); //x3.norm();//12 - FP_BLS12381_add(&y3, &y3, &t1); //y3.add(t1); - FP_BLS12381_norm(&y3); //y3.norm();//13 - FP_BLS12381_mul(&y3, &y3, &x3); //y3.mul(x3); //14 - FP_BLS12381_mul(&x3, &x3, &t3); //x3.mul(t3); //15 - FP_BLS12381_add(&t3, &t2, &t2); //t3.add(t2); //16 - FP_BLS12381_add(&t2, &t2, &t3); //t2.add(t3); //17 - - if (CURVE_B_I_BLS12381 == 0) //if (ROM.CURVE_B_I==0) - FP_BLS12381_mul(&z3, &z3, &b); //z3.mul(b); //18 - else - FP_BLS12381_imul(&z3, &z3, CURVE_B_I_BLS12381); //z3.imul(ROM.CURVE_B_I); - - FP_BLS12381_sub(&z3, &z3, &t2); //z3.sub(t2); //z3.norm();//19 - FP_BLS12381_sub(&z3, &z3, &t0); //z3.sub(t0); - FP_BLS12381_norm(&z3); //z3.norm();//20 *** - FP_BLS12381_add(&t3, &z3, &z3); //t3.add(z3); //t3.norm();//21 - - FP_BLS12381_add(&z3, &z3, &t3); //z3.add(t3); - FP_BLS12381_norm(&z3); //z3.norm(); //22 - FP_BLS12381_add(&t3, &t0, &t0); //t3.add(t0); //t3.norm(); //23 - FP_BLS12381_add(&t0, &t0, &t3); //t0.add(t3); //t0.norm();//24 - FP_BLS12381_sub(&t0, &t0, &t2); //t0.sub(t2); - FP_BLS12381_norm(&t0); //t0.norm();//25 - - FP_BLS12381_mul(&t0, &t0, &z3); //t0.mul(z3);//26 - FP_BLS12381_add(&y3, &y3, &t0); //y3.add(t0); //y3.norm();//27 - FP_BLS12381_mul(&t0, &(P->y), &(P->z)); //t0.mul(z);//28 - FP_BLS12381_add(&t0, &t0, &t0); //t0.add(t0); - FP_BLS12381_norm(&t0); //t0.norm(); //29 - FP_BLS12381_mul(&z3, &z3, &t0); //z3.mul(t0);//30 - FP_BLS12381_sub(&(P->x), &x3, &z3); //x3.sub(z3); //x3.norm();//31 - FP_BLS12381_add(&t0, &t0, &t0); //t0.add(t0); - FP_BLS12381_norm(&t0); //t0.norm();//32 - FP_BLS12381_add(&t1, &t1, &t1); //t1.add(t1); - FP_BLS12381_norm(&t1); //t1.norm();//33 - FP_BLS12381_mul(&(P->z), &t0, &t1); //z3.mul(t1);//34 - - FP_BLS12381_norm(&(P->x)); //x.norm(); - FP_BLS12381_copy(&(P->y), &y3); //y.copy(y3); - FP_BLS12381_norm(&(P->y)); //y.norm(); - FP_BLS12381_norm(&(P->z)); //z.norm(); -#endif -#endif - -#if CURVETYPE_BLS12381==EDWARDS - /* Not using square for multiplication swap, as (1) it needs more adds, and (2) it triggers more reductions */ - - FP_BLS12381 C, D, H, J; - - FP_BLS12381_sqr(&C, &(P->x)); //C.sqr(); - - FP_BLS12381_mul(&(P->x), &(P->x), &(P->y)); //x.mul(y); - FP_BLS12381_add(&(P->x), &(P->x), &(P->x)); //x.add(x); - FP_BLS12381_norm(&(P->x)); //x.norm(); - - FP_BLS12381_sqr(&D, &(P->y)); //D.sqr(); - -#if CURVE_A_BLS12381 == -1 - FP_BLS12381_neg(&C, &C); // C.neg(); -#endif - FP_BLS12381_add(&(P->y), &C, &D); //y.add(D); - FP_BLS12381_norm(&(P->y)); //y.norm(); - FP_BLS12381_sqr(&H, &(P->z)); //H.sqr(); - FP_BLS12381_add(&H, &H, &H); //H.add(H); - - FP_BLS12381_sub(&J, &(P->y), &H); //J.sub(H); - FP_BLS12381_norm(&J); //J.norm(); - - FP_BLS12381_mul(&(P->x), &(P->x), &J); //x.mul(J); - FP_BLS12381_sub(&C, &C, &D); //C.sub(D); - FP_BLS12381_norm(&C); //C.norm(); - FP_BLS12381_mul(&(P->z), &(P->y), &J); //z.mul(J); - FP_BLS12381_mul(&(P->y), &(P->y), &C); //y.mul(C); - - -#endif - -#if CURVETYPE_BLS12381==MONTGOMERY - FP_BLS12381 A, B, AA, BB, C; - - FP_BLS12381_add(&A, &(P->x), &(P->z)); //A.add(z); - FP_BLS12381_norm(&A); //A.norm(); - FP_BLS12381_sqr(&AA, &A); //AA.sqr(); - FP_BLS12381_sub(&B, &(P->x), &(P->z)); //B.sub(z); - FP_BLS12381_norm(&B); //B.norm(); - - FP_BLS12381_sqr(&BB, &B); //BB.sqr(); - FP_BLS12381_sub(&C, &AA, &BB); //C.sub(BB); - FP_BLS12381_norm(&C); //C.norm(); - FP_BLS12381_mul(&(P->x), &AA, &BB); //x.mul(BB); - FP_BLS12381_imul(&A, &C, (CURVE_A_BLS12381 + 2) / 4); //A.imul((ROM.CURVE_A+2)/4); - - FP_BLS12381_add(&BB, &BB, &A); //BB.add(A); - FP_BLS12381_norm(&BB); //BB.norm(); - FP_BLS12381_mul(&(P->z), &BB, &C); //z.mul(C); - -#endif -} - -#if CURVETYPE_BLS12381==MONTGOMERY - -/* Set P+=Q. W is difference between P and Q and is affine */ -void ECP_BLS12381_add(ECP_BLS12381 *P, ECP_BLS12381 *Q, ECP_BLS12381 *W) -{ - FP_BLS12381 A, B, C, D, DA, CB; - - FP_BLS12381_add(&A, &(P->x), &(P->z)); //A.add(z); - FP_BLS12381_sub(&B, &(P->x), &(P->z)); //B.sub(z); - - FP_BLS12381_add(&C, &(Q->x), &(Q->z)); //C.add(Q.z); - - FP_BLS12381_sub(&D, &(Q->x), &(Q->z)); //D.sub(Q.z); - FP_BLS12381_norm(&A); //A.norm(); - - FP_BLS12381_norm(&D); //D.norm(); - FP_BLS12381_mul(&DA, &D, &A); //DA.mul(A); - - FP_BLS12381_norm(&C); //C.norm(); - FP_BLS12381_norm(&B); //B.norm(); - FP_BLS12381_mul(&CB, &C, &B); //CB.mul(B); - - FP_BLS12381_add(&A, &DA, &CB); //A.add(CB); - FP_BLS12381_norm(&A); //A.norm(); - FP_BLS12381_sqr(&(P->x), &A); //A.sqr(); - FP_BLS12381_sub(&B, &DA, &CB); //B.sub(CB); - FP_BLS12381_norm(&B); //B.norm(); - FP_BLS12381_sqr(&B, &B); //B.sqr(); - - FP_BLS12381_mul(&(P->z), &(W->x), &B); //z.mul(B); - -} - -#else - -/* Set P+=Q */ -/* SU=248 */ -void ECP_BLS12381_add(ECP_BLS12381 *P, ECP_BLS12381 *Q) -{ -#if CURVETYPE_BLS12381==WEIERSTRASS - - int b3; - FP_BLS12381 t0, t1, t2, t3, t4, x3, y3, z3, b; - -#if CURVE_A_BLS12381 == 0 - - b3 = 3 * CURVE_B_I_BLS12381; //int b=3*ROM.CURVE_B_I; - FP_BLS12381_mul(&t0, &(P->x), &(Q->x)); //t0.mul(Q.x); - FP_BLS12381_mul(&t1, &(P->y), &(Q->y)); //t1.mul(Q.y); - FP_BLS12381_mul(&t2, &(P->z), &(Q->z)); //t2.mul(Q.z); - FP_BLS12381_add(&t3, &(P->x), &(P->y)); //t3.add(y); - FP_BLS12381_norm(&t3); //t3.norm(); - - FP_BLS12381_add(&t4, &(Q->x), &(Q->y)); //t4.add(Q.y); - FP_BLS12381_norm(&t4); //t4.norm(); - FP_BLS12381_mul(&t3, &t3, &t4); //t3.mul(t4); - FP_BLS12381_add(&t4, &t0, &t1); //t4.add(t1); - - FP_BLS12381_sub(&t3, &t3, &t4); //t3.sub(t4); - FP_BLS12381_norm(&t3); //t3.norm(); - FP_BLS12381_add(&t4, &(P->y), &(P->z)); //t4.add(z); - FP_BLS12381_norm(&t4); //t4.norm(); - FP_BLS12381_add(&x3, &(Q->y), &(Q->z)); //x3.add(Q.z); - FP_BLS12381_norm(&x3); //x3.norm(); - - FP_BLS12381_mul(&t4, &t4, &x3); //t4.mul(x3); - FP_BLS12381_add(&x3, &t1, &t2); //x3.add(t2); - - FP_BLS12381_sub(&t4, &t4, &x3); //t4.sub(x3); - FP_BLS12381_norm(&t4); //t4.norm(); - FP_BLS12381_add(&x3, &(P->x), &(P->z)); //x3.add(z); - FP_BLS12381_norm(&x3); //x3.norm(); - FP_BLS12381_add(&y3, &(Q->x), &(Q->z)); //y3.add(Q.z); - FP_BLS12381_norm(&y3); //y3.norm(); - FP_BLS12381_mul(&x3, &x3, &y3); //x3.mul(y3); - - FP_BLS12381_add(&y3, &t0, &t2); //y3.add(t2); - FP_BLS12381_sub(&y3, &x3, &y3); //y3.rsub(x3); - FP_BLS12381_norm(&y3); //y3.norm(); - FP_BLS12381_add(&x3, &t0, &t0); //x3.add(t0); - FP_BLS12381_add(&t0, &t0, &x3); //t0.add(x3); - FP_BLS12381_norm(&t0); //t0.norm(); - FP_BLS12381_imul(&t2, &t2, b3); //t2.imul(b); - - FP_BLS12381_add(&z3, &t1, &t2); //z3.add(t2); - FP_BLS12381_norm(&z3); //z3.norm(); - FP_BLS12381_sub(&t1, &t1, &t2); //t1.sub(t2); - FP_BLS12381_norm(&t1); //t1.norm(); - FP_BLS12381_imul(&y3, &y3, b3); //y3.imul(b); - - FP_BLS12381_mul(&x3, &y3, &t4); //x3.mul(t4); - FP_BLS12381_mul(&t2, &t3, &t1); //t2.mul(t1); - FP_BLS12381_sub(&(P->x), &t2, &x3); //x3.rsub(t2); - FP_BLS12381_mul(&y3, &y3, &t0); //y3.mul(t0); - FP_BLS12381_mul(&t1, &t1, &z3); //t1.mul(z3); - FP_BLS12381_add(&(P->y), &y3, &t1); //y3.add(t1); - FP_BLS12381_mul(&t0, &t0, &t3); //t0.mul(t3); - FP_BLS12381_mul(&z3, &z3, &t4); //z3.mul(t4); - FP_BLS12381_add(&(P->z), &z3, &t0); //z3.add(t0); - - FP_BLS12381_norm(&(P->x)); //x.norm(); - FP_BLS12381_norm(&(P->y)); //y.norm(); - FP_BLS12381_norm(&(P->z)); //z.norm(); -#else - - if (CURVE_B_I_BLS12381 == 0) //if (ROM.CURVE_B_I==0) - FP_BLS12381_rcopy(&b, CURVE_B_BLS12381); //b.copy(new FP(new BIG(ROM.CURVE_B))); - - FP_BLS12381_mul(&t0, &(P->x), &(Q->x)); //t0.mul(Q.x); //1 - FP_BLS12381_mul(&t1, &(P->y), &(Q->y)); //t1.mul(Q.y); //2 - FP_BLS12381_mul(&t2, &(P->z), &(Q->z)); //t2.mul(Q.z); //3 - - FP_BLS12381_add(&t3, &(P->x), &(P->y)); //t3.add(y); - FP_BLS12381_norm(&t3); //t3.norm(); //4 - FP_BLS12381_add(&t4, &(Q->x), &(Q->y)); //t4.add(Q.y); - FP_BLS12381_norm(&t4); //t4.norm();//5 - FP_BLS12381_mul(&t3, &t3, &t4); //t3.mul(t4);//6 - - FP_BLS12381_add(&t4, &t0, &t1); //t4.add(t1); //t4.norm(); //7 - FP_BLS12381_sub(&t3, &t3, &t4); //t3.sub(t4); - FP_BLS12381_norm(&t3); //t3.norm(); //8 - FP_BLS12381_add(&t4, &(P->y), &(P->z)); //t4.add(z); - FP_BLS12381_norm(&t4); //t4.norm();//9 - FP_BLS12381_add(&x3, &(Q->y), &(Q->z)); //x3.add(Q.z); - FP_BLS12381_norm(&x3); //x3.norm();//10 - FP_BLS12381_mul(&t4, &t4, &x3); //t4.mul(x3); //11 - FP_BLS12381_add(&x3, &t1, &t2); //x3.add(t2); //x3.norm();//12 - - FP_BLS12381_sub(&t4, &t4, &x3); //t4.sub(x3); - FP_BLS12381_norm(&t4); //t4.norm();//13 - FP_BLS12381_add(&x3, &(P->x), &(P->z)); //x3.add(z); - FP_BLS12381_norm(&x3); //x3.norm(); //14 - FP_BLS12381_add(&y3, &(Q->x), &(Q->z)); //y3.add(Q.z); - FP_BLS12381_norm(&y3); //y3.norm();//15 - - FP_BLS12381_mul(&x3, &x3, &y3); //x3.mul(y3); //16 - FP_BLS12381_add(&y3, &t0, &t2); //y3.add(t2); //y3.norm();//17 - - FP_BLS12381_sub(&y3, &x3, &y3); //y3.rsub(x3); - FP_BLS12381_norm(&y3); //y3.norm(); //18 - - if (CURVE_B_I_BLS12381 == 0) //if (ROM.CURVE_B_I==0) - FP_BLS12381_mul(&z3, &t2, &b); //z3.mul(b); //18 - else - FP_BLS12381_imul(&z3, &t2, CURVE_B_I_BLS12381); //z3.imul(ROM.CURVE_B_I); - - FP_BLS12381_sub(&x3, &y3, &z3); //x3.sub(z3); - FP_BLS12381_norm(&x3); //x3.norm(); //20 - FP_BLS12381_add(&z3, &x3, &x3); //z3.add(x3); //z3.norm(); //21 - - FP_BLS12381_add(&x3, &x3, &z3); //x3.add(z3); //x3.norm(); //22 - FP_BLS12381_sub(&z3, &t1, &x3); //z3.sub(x3); - FP_BLS12381_norm(&z3); //z3.norm(); //23 - FP_BLS12381_add(&x3, &x3, &t1); //x3.add(t1); - FP_BLS12381_norm(&x3); //x3.norm(); //24 - - if (CURVE_B_I_BLS12381 == 0) //if (ROM.CURVE_B_I==0) - FP_BLS12381_mul(&y3, &y3, &b); //y3.mul(b); //18 - else - FP_BLS12381_imul(&y3, &y3, CURVE_B_I_BLS12381); //y3.imul(ROM.CURVE_B_I); - - FP_BLS12381_add(&t1, &t2, &t2); //t1.add(t2); //t1.norm();//26 - FP_BLS12381_add(&t2, &t2, &t1); //t2.add(t1); //t2.norm();//27 - - FP_BLS12381_sub(&y3, &y3, &t2); //y3.sub(t2); //y3.norm(); //28 - - FP_BLS12381_sub(&y3, &y3, &t0); //y3.sub(t0); - FP_BLS12381_norm(&y3); //y3.norm(); //29 - FP_BLS12381_add(&t1, &y3, &y3); //t1.add(y3); //t1.norm();//30 - FP_BLS12381_add(&y3, &y3, &t1); //y3.add(t1); - FP_BLS12381_norm(&y3); //y3.norm(); //31 - - FP_BLS12381_add(&t1, &t0, &t0); //t1.add(t0); //t1.norm(); //32 - FP_BLS12381_add(&t0, &t0, &t1); //t0.add(t1); //t0.norm();//33 - FP_BLS12381_sub(&t0, &t0, &t2); //t0.sub(t2); - FP_BLS12381_norm(&t0); //t0.norm();//34 - - FP_BLS12381_mul(&t1, &t4, &y3); //t1.mul(y3);//35 - FP_BLS12381_mul(&t2, &t0, &y3); //t2.mul(y3);//36 - FP_BLS12381_mul(&y3, &x3, &z3); //y3.mul(z3);//37 - FP_BLS12381_add(&(P->y), &y3, &t2); //y3.add(t2); //y3.norm();//38 - FP_BLS12381_mul(&x3, &x3, &t3); //x3.mul(t3);//39 - FP_BLS12381_sub(&(P->x), &x3, &t1); //x3.sub(t1);//40 - FP_BLS12381_mul(&z3, &z3, &t4); //z3.mul(t4);//41 - - FP_BLS12381_mul(&t1, &t3, &t0); //t1.mul(t0);//42 - FP_BLS12381_add(&(P->z), &z3, &t1); //z3.add(t1); - FP_BLS12381_norm(&(P->x)); //x.norm(); - - FP_BLS12381_norm(&(P->y)); //y.norm(); - FP_BLS12381_norm(&(P->z)); //z.norm(); -#endif - -#else - FP_BLS12381 A, B, C, D, E, F, G, b; - - FP_BLS12381_mul(&A, &(P->z), &(Q->z)); //A.mul(Q.z); - FP_BLS12381_sqr(&B, &A); //B.sqr(); - FP_BLS12381_mul(&C, &(P->x), &(Q->x)); //C.mul(Q.x); - FP_BLS12381_mul(&D, &(P->y), &(Q->y)); //D.mul(Q.y); - FP_BLS12381_mul(&E, &C, &D); //E.mul(D); - - if (CURVE_B_I_BLS12381 == 0) //if (ROM.CURVE_B_I==0) - { - FP_BLS12381_rcopy(&b, CURVE_B_BLS12381); //FP b=new FP(new BIG(ROM.CURVE_B)); - FP_BLS12381_mul(&E, &E, &b); //E.mul(b); - } - else - FP_BLS12381_imul(&E, &E, CURVE_B_I_BLS12381); //E.imul(ROM.CURVE_B_I); - - FP_BLS12381_sub(&F, &B, &E); //F.sub(E); - FP_BLS12381_add(&G, &B, &E); //G.add(E); - -#if CURVE_A_BLS12381 == 1 - FP_BLS12381_sub(&E, &D, &C); //E.sub(C); -#endif - FP_BLS12381_add(&C, &C, &D); //C.add(D); - FP_BLS12381_add(&B, &(P->x), &(P->y)); //B.add(y); - - FP_BLS12381_add(&D, &(Q->x), &(Q->y)); //D.add(Q.y); - FP_BLS12381_norm(&B); //B.norm(); - FP_BLS12381_norm(&D); //D.norm(); - FP_BLS12381_mul(&B, &B, &D); //B.mul(D); - FP_BLS12381_sub(&B, &B, &C); //B.sub(C); - FP_BLS12381_norm(&B); //B.norm(); - FP_BLS12381_norm(&F); //F.norm(); - FP_BLS12381_mul(&B, &B, &F); //B.mul(F); - FP_BLS12381_mul(&(P->x), &A, &B); //x.mul(B); - FP_BLS12381_norm(&G); //G.norm(); - -#if CURVE_A_BLS12381 == 1 - FP_BLS12381_norm(&E); //E.norm(); - FP_BLS12381_mul(&C, &E, &G); //C.mul(G); -#endif -#if CURVE_A_BLS12381 == -1 - FP_BLS12381_norm(&C); //C.norm(); - FP_BLS12381_mul(&C, &C, &G); //C.mul(G); -#endif - FP_BLS12381_mul(&(P->y), &A, &C); //y.mul(C); - FP_BLS12381_mul(&(P->z), &F, &G); //z.mul(G); - -#endif -} - -/* Set P-=Q */ -/* SU=16 */ -void ECP_BLS12381_sub(ECP_BLS12381 *P, ECP_BLS12381 *Q) -{ - ECP_BLS12381 NQ; - ECP_BLS12381_copy(&NQ, Q); - ECP_BLS12381_neg(&NQ); - ECP_BLS12381_add(P, &NQ); -} - -#endif - -#if CURVETYPE_BLS12381!=MONTGOMERY -/* constant time multiply by small integer of length bts - use ladder */ -void ECP_BLS12381_pinmul(ECP_BLS12381 *P, int e, int bts) -{ - int i, b; - ECP_BLS12381 R0, R1; - - ECP_BLS12381_affine(P); - ECP_BLS12381_inf(&R0); - ECP_BLS12381_copy(&R1, P); - - for (i = bts - 1; i >= 0; i--) - { - b = (e >> i) & 1; - ECP_BLS12381_copy(P, &R1); - ECP_BLS12381_add(P, &R0); - ECP_BLS12381_cswap(&R0, &R1, b); - ECP_BLS12381_copy(&R1, P); - ECP_BLS12381_dbl(&R0); - ECP_BLS12381_cswap(&R0, &R1, b); - } - ECP_BLS12381_copy(P, &R0); -} -#endif - -/* Set P=r*P */ -/* SU=424 */ -void ECP_BLS12381_mul(ECP_BLS12381 *P, BIG_384_58 e) -{ -#if CURVETYPE_BLS12381==MONTGOMERY - /* Montgomery ladder */ - int nb, i, b; - ECP_BLS12381 R0, R1, D; - if (ECP_BLS12381_isinf(P)) return; - if (BIG_384_58_iszilch(e)) - { - ECP_BLS12381_inf(P); - return; - } - - ECP_BLS12381_copy(&R0, P); - ECP_BLS12381_copy(&R1, P); - ECP_BLS12381_dbl(&R1); - - ECP_BLS12381_copy(&D, P); - ECP_BLS12381_affine(&D); - - nb = BIG_384_58_nbits(e); - for (i = nb - 2; i >= 0; i--) - { - b = BIG_384_58_bit(e, i); - ECP_BLS12381_copy(P, &R1); - ECP_BLS12381_add(P, &R0, &D); - ECP_BLS12381_cswap(&R0, &R1, b); - ECP_BLS12381_copy(&R1, P); - ECP_BLS12381_dbl(&R0); - - ECP_BLS12381_cswap(&R0, &R1, b); - } - - ECP_BLS12381_copy(P, &R0); - -#else - /* fixed size windows */ - int i, nb, s, ns; - BIG_384_58 mt, t; - ECP_BLS12381 Q, W[8], C; - sign8 w[1 + (NLEN_384_58 * BASEBITS_384_58 + 3) / 4]; - - if (ECP_BLS12381_isinf(P)) return; - if (BIG_384_58_iszilch(e)) - { - ECP_BLS12381_inf(P); - return; - } - - /* precompute table */ - - ECP_BLS12381_copy(&Q, P); - ECP_BLS12381_dbl(&Q); - - ECP_BLS12381_copy(&W[0], P); - - for (i = 1; i < 8; i++) - { - ECP_BLS12381_copy(&W[i], &W[i - 1]); - ECP_BLS12381_add(&W[i], &Q); - } - -//printf("W[1]= ");ECP_output(&W[1]); printf("\n"); - - /* make exponent odd - add 2P if even, P if odd */ - BIG_384_58_copy(t, e); - s = BIG_384_58_parity(t); - BIG_384_58_inc(t, 1); - BIG_384_58_norm(t); - ns = BIG_384_58_parity(t); - BIG_384_58_copy(mt, t); - BIG_384_58_inc(mt, 1); - BIG_384_58_norm(mt); - BIG_384_58_cmove(t, mt, s); - ECP_BLS12381_cmove(&Q, P, ns); - ECP_BLS12381_copy(&C, &Q); - - nb = 1 + (BIG_384_58_nbits(t) + 3) / 4; - - /* convert exponent to signed 4-bit window */ - for (i = 0; i < nb; i++) - { - w[i] = BIG_384_58_lastbits(t, 5) - 16; - BIG_384_58_dec(t, w[i]); - BIG_384_58_norm(t); - BIG_384_58_fshr(t, 4); - } - w[nb] = BIG_384_58_lastbits(t, 5); - - ECP_BLS12381_copy(P, &W[(w[nb] - 1) / 2]); - for (i = nb - 1; i >= 0; i--) - { - ECP_BLS12381_select(&Q, W, w[i]); - ECP_BLS12381_dbl(P); - ECP_BLS12381_dbl(P); - ECP_BLS12381_dbl(P); - ECP_BLS12381_dbl(P); - ECP_BLS12381_add(P, &Q); - } - ECP_BLS12381_sub(P, &C); /* apply correction */ -#endif -} - -#if CURVETYPE_BLS12381!=MONTGOMERY -/* Set P=eP+fQ double multiplication */ -/* constant time - as useful for GLV method in pairings */ -/* SU=456 */ - -void ECP_BLS12381_mul2(ECP_BLS12381 *P, ECP_BLS12381 *Q, BIG_384_58 e, BIG_384_58 f) -{ - BIG_384_58 te, tf, mt; - ECP_BLS12381 S, T, W[8], C; - sign8 w[1 + (NLEN_384_58 * BASEBITS_384_58 + 1) / 2]; - int i, a, b, s, ns, nb; - - BIG_384_58_copy(te, e); - BIG_384_58_copy(tf, f); - - /* precompute table */ - ECP_BLS12381_copy(&W[1], P); - ECP_BLS12381_sub(&W[1], Q); /* P+Q */ - ECP_BLS12381_copy(&W[2], P); - ECP_BLS12381_add(&W[2], Q); /* P-Q */ - ECP_BLS12381_copy(&S, Q); - ECP_BLS12381_dbl(&S); /* S=2Q */ - ECP_BLS12381_copy(&W[0], &W[1]); - ECP_BLS12381_sub(&W[0], &S); - ECP_BLS12381_copy(&W[3], &W[2]); - ECP_BLS12381_add(&W[3], &S); - ECP_BLS12381_copy(&T, P); - ECP_BLS12381_dbl(&T); /* T=2P */ - ECP_BLS12381_copy(&W[5], &W[1]); - ECP_BLS12381_add(&W[5], &T); - ECP_BLS12381_copy(&W[6], &W[2]); - ECP_BLS12381_add(&W[6], &T); - ECP_BLS12381_copy(&W[4], &W[5]); - ECP_BLS12381_sub(&W[4], &S); - ECP_BLS12381_copy(&W[7], &W[6]); - ECP_BLS12381_add(&W[7], &S); - - /* if multiplier is odd, add 2, else add 1 to multiplier, and add 2P or P to correction */ - - s = BIG_384_58_parity(te); - BIG_384_58_inc(te, 1); - BIG_384_58_norm(te); - ns = BIG_384_58_parity(te); - BIG_384_58_copy(mt, te); - BIG_384_58_inc(mt, 1); - BIG_384_58_norm(mt); - BIG_384_58_cmove(te, mt, s); - ECP_BLS12381_cmove(&T, P, ns); - ECP_BLS12381_copy(&C, &T); - - s = BIG_384_58_parity(tf); - BIG_384_58_inc(tf, 1); - BIG_384_58_norm(tf); - ns = BIG_384_58_parity(tf); - BIG_384_58_copy(mt, tf); - BIG_384_58_inc(mt, 1); - BIG_384_58_norm(mt); - BIG_384_58_cmove(tf, mt, s); - ECP_BLS12381_cmove(&S, Q, ns); - ECP_BLS12381_add(&C, &S); - - BIG_384_58_add(mt, te, tf); - BIG_384_58_norm(mt); - nb = 1 + (BIG_384_58_nbits(mt) + 1) / 2; - - /* convert exponent to signed 2-bit window */ - for (i = 0; i < nb; i++) - { - a = BIG_384_58_lastbits(te, 3) - 4; - BIG_384_58_dec(te, a); - BIG_384_58_norm(te); - BIG_384_58_fshr(te, 2); - b = BIG_384_58_lastbits(tf, 3) - 4; - BIG_384_58_dec(tf, b); - BIG_384_58_norm(tf); - BIG_384_58_fshr(tf, 2); - w[i] = 4 * a + b; - } - w[nb] = (4 * BIG_384_58_lastbits(te, 3) + BIG_384_58_lastbits(tf, 3)); - - ECP_BLS12381_copy(P, &W[(w[nb] - 1) / 2]); - for (i = nb - 1; i >= 0; i--) - { - ECP_BLS12381_select(&T, W, w[i]); - ECP_BLS12381_dbl(P); - ECP_BLS12381_dbl(P); - ECP_BLS12381_add(P, &T); - } - ECP_BLS12381_sub(P, &C); /* apply correction */ -} - -#endif - -int ECP_BLS12381_generator(ECP_BLS12381 *G) -{ - BIG_384_58 x, y; - BIG_384_58_rcopy(x, CURVE_Gx_BLS12381); -#if CURVETYPE_BLS12381!=MONTGOMERY - BIG_384_58_rcopy(y, CURVE_Gy_BLS12381); - return ECP_BLS12381_set(G, x, y); -#else - return ECP_BLS12381_set(G, x); -#endif -} - -#ifdef HAS_MAIN - -int main() -{ - int i; - ECP_BLS12381 G, P; - csprng RNG; - BIG_384_58 r, s, x, y, b, m, w, q; - BIG_384_58_rcopy(x, CURVE_Gx); -#if CURVETYPE_BLS12381!=MONTGOMERY - BIG_384_58_rcopy(y, CURVE_Gy); -#endif - BIG_384_58_rcopy(m, Modulus_BLS12381); - - printf("x= "); - BIG_384_58_output(x); - printf("\n"); -#if CURVETYPE_BLS12381!=MONTGOMERY - printf("y= "); - BIG_384_58_output(y); - printf("\n"); -#endif - RNG_seed(&RNG, 3, "abc"); - -#if CURVETYPE_BLS12381!=MONTGOMERY - ECP_BLS12381_set(&G, x, y); -#else - ECP_BLS12381_set(&G, x); -#endif - if (ECP_BLS12381_isinf(&G)) printf("Failed to set - point not on curve\n"); - else printf("set success\n"); - - ECP_BLS12381_output(&G); - - BIG_384_58_rcopy(r, CURVE_Order); //BIG_dec(r,7); - printf("r= "); - BIG_384_58_output(r); - printf("\n"); - - ECP_BLS12381_copy(&P, &G); - - ECP_BLS12381_mul(&P, r); - - ECP_BLS12381_output(&P); -//exit(0); - BIG_384_58_randomnum(w, &RNG); - BIG_384_58_mod(w, r); - - ECP_BLS12381_copy(&P, &G); - ECP_BLS12381_mul(&P, w); - - ECP_BLS12381_output(&P); - - return 0; -} - -#endif diff --git a/impl/cbits/ecp_BLS12381.h b/impl/cbits/ecp_BLS12381.h deleted file mode 100644 index 8ef3e0180..000000000 --- a/impl/cbits/ecp_BLS12381.h +++ /dev/null @@ -1,344 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file ecp.h - * @author Mike Scott - * @brief ECP Header File - * - */ - -#ifndef ECP_BLS12381_H -#define ECP_BLS12381_H - -#include "fp_BLS12381.h" -#include "config_curve_BLS12381.h" - -/* Curve Params - see rom_zzz.c */ -extern const int CURVE_Cof_I_BLS12381; /**< Elliptic curve cofactor */ -extern const int CURVE_B_I_BLS12381; /**< Elliptic curve B_i parameter */ -extern const BIG_384_58 CURVE_B_BLS12381; /**< Elliptic curve B parameter */ -extern const BIG_384_58 CURVE_Order_BLS12381; /**< Elliptic curve group order */ -extern const BIG_384_58 CURVE_Cof_BLS12381; /**< Elliptic curve cofactor */ -extern const BIG_384_58 CURVE_HTPC_BLS12381; /**< Hash to Point precomputation */ - -extern const BIG_384_58 CURVE_Ad_BLS12381; /**< A parameter of isogenous curve */ -extern const BIG_384_58 CURVE_Bd_BLS12381; /**< B parameter of isogenous curve */ -extern const BIG_384_58 PC_BLS12381[]; /**< Precomputed isogenies */ - -extern const BIG_384_58 CURVE_Adr_BLS12381; /**< Real part of A parameter of isogenous curve in G2 */ -extern const BIG_384_58 CURVE_Adi_BLS12381; /**< Imaginary part of A parameter of isogenous curve in G2 */ -extern const BIG_384_58 CURVE_Bdr_BLS12381; /**< Real part of B parameter of isogenous curve in G2 */ -extern const BIG_384_58 CURVE_Bdi_BLS12381; /**< Imaginary part of B parameter of isogenous curve in G2 */ -extern const BIG_384_58 PCR_BLS12381[]; /**< Real parts of precomputed isogenies */ -extern const BIG_384_58 PCI_BLS12381[]; /**< Imaginary parts of precomputed isogenies */ - -/* Generator point on G1 */ -extern const BIG_384_58 CURVE_Gx_BLS12381; /**< x-coordinate of generator point in group G1 */ -extern const BIG_384_58 CURVE_Gy_BLS12381; /**< y-coordinate of generator point in group G1 */ - - -/* For Pairings only */ - -/* Generator point on G2 */ -extern const BIG_384_58 CURVE_Pxa_BLS12381; /**< real part of x-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pxb_BLS12381; /**< imaginary part of x-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pya_BLS12381; /**< real part of y-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pyb_BLS12381; /**< imaginary part of y-coordinate of generator point in group G2 */ - - -/*** needed for BLS24 curves ***/ - -extern const BIG_384_58 CURVE_Pxaa_BLS12381; /**< real part of x-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pxab_BLS12381; /**< imaginary part of x-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pxba_BLS12381; /**< real part of x-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pxbb_BLS12381; /**< imaginary part of x-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pyaa_BLS12381; /**< real part of y-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pyab_BLS12381; /**< imaginary part of y-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pyba_BLS12381; /**< real part of y-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pybb_BLS12381; /**< imaginary part of y-coordinate of generator point in group G2 */ - -/*** needed for BLS48 curves ***/ - -extern const BIG_384_58 CURVE_Pxaaa_BLS12381; /**< real part of x-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pxaab_BLS12381; /**< imaginary part of x-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pxaba_BLS12381; /**< real part of x-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pxabb_BLS12381; /**< imaginary part of x-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pxbaa_BLS12381; /**< real part of x-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pxbab_BLS12381; /**< imaginary part of x-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pxbba_BLS12381; /**< real part of x-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pxbbb_BLS12381; /**< imaginary part of x-coordinate of generator point in group G2 */ - -extern const BIG_384_58 CURVE_Pyaaa_BLS12381; /**< real part of y-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pyaab_BLS12381; /**< imaginary part of y-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pyaba_BLS12381; /**< real part of y-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pyabb_BLS12381; /**< imaginary part of y-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pybaa_BLS12381; /**< real part of y-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pybab_BLS12381; /**< imaginary part of y-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pybba_BLS12381; /**< real part of y-coordinate of generator point in group G2 */ -extern const BIG_384_58 CURVE_Pybbb_BLS12381; /**< imaginary part of y-coordinate of generator point in group G2 */ - - -extern const BIG_384_58 CURVE_Bnx_BLS12381; /**< BN curve x parameter */ - - - -extern const BIG_384_58 Fra_BLS12381; /**< real part of BN curve Frobenius Constant */ -extern const BIG_384_58 Frb_BLS12381; /**< imaginary part of BN curve Frobenius Constant */ - - -extern const BIG_384_58 CURVE_W_BLS12381[2]; /**< BN curve constant for GLV decomposition */ -extern const BIG_384_58 CURVE_SB_BLS12381[2][2]; /**< BN curve constant for GLV decomposition */ -extern const BIG_384_58 CURVE_WB_BLS12381[4]; /**< BN curve constant for GS decomposition */ -extern const BIG_384_58 CURVE_BB_BLS12381[4][4]; /**< BN curve constant for GS decomposition */ - - -/** - @brief ECP structure - Elliptic Curve Point over base field -*/ - -typedef struct -{ -// int inf; /**< Infinity Flag - not needed for Edwards representation */ - - FP_BLS12381 x; /**< x-coordinate of point */ -#if CURVETYPE_BLS12381!=MONTGOMERY - FP_BLS12381 y; /**< y-coordinate of point. Not needed for Montgomery representation */ -#endif - FP_BLS12381 z;/**< z-coordinate of point */ -} ECP_BLS12381; - - -/* ECP E(Fp) prototypes */ -/** @brief Tests for ECP point equal to infinity - * - @param P ECP point to be tested - @return 1 if infinity, else returns 0 - */ -extern int ECP_BLS12381_isinf(ECP_BLS12381 *P); -/** @brief Tests for equality of two ECPs - * - @param P ECP instance to be compared - @param Q ECP instance to be compared - @return 1 if P=Q, else returns 0 - */ -extern int ECP_BLS12381_equals(ECP_BLS12381 *P, ECP_BLS12381 *Q); -/** @brief Copy ECP point to another ECP point - * - @param P ECP instance, on exit = Q - @param Q ECP instance to be copied - */ -extern void ECP_BLS12381_copy(ECP_BLS12381 *P, ECP_BLS12381 *Q); -/** @brief Negation of an ECP point - * - @param P ECP instance, on exit = -P - */ -extern void ECP_BLS12381_neg(ECP_BLS12381 *P); -/** @brief Set ECP to point-at-infinity - * - @param P ECP instance to be set to infinity - */ -extern void ECP_BLS12381_inf(ECP_BLS12381 *P); -/** @brief Calculate Right Hand Side of curve equation y^2=f(x) - * - Function f(x) depends on form of elliptic curve, Weierstrass, Edwards or Montgomery. - Used internally. - @param r BIG n-residue value of f(x) - @param x BIG n-residue x - */ -extern void ECP_BLS12381_rhs(FP_BLS12381 *r, FP_BLS12381 *x); - -#if CURVETYPE_BLS12381==MONTGOMERY -/** @brief Set ECP to point(x,[y]) given x - * - Point P set to infinity if no such point on the curve. Note that y coordinate is not needed. - @param P ECP instance to be set (x,[y]) - @param x BIG x coordinate of point - @return 1 if point exists, else 0 - */ -extern int ECP_BLS12381_set(ECP_BLS12381 *P, BIG_384_58 x); -/** @brief Extract x coordinate of an ECP point P - * - @param x BIG on exit = x coordinate of point - @param P ECP instance (x,[y]) - @return -1 if P is point-at-infinity, else 0 - */ -extern int ECP_BLS12381_get(BIG_384_58 x, ECP_BLS12381 *P); -/** @brief Adds ECP instance Q to ECP instance P, given difference D=P-Q - * - Differential addition of points on a Montgomery curve - @param P ECP instance, on exit =P+Q - @param Q ECP instance to be added to P - @param D Difference between P and Q - */ -extern void ECP_BLS12381_add(ECP_BLS12381 *P, ECP_BLS12381 *Q, ECP_BLS12381 *D); -#else -/** @brief Set ECP to point(x,y) given x and y - * - Point P set to infinity if no such point on the curve. - @param P ECP instance to be set (x,y) - @param x BIG x coordinate of point - @param y BIG y coordinate of point - @return 1 if point exists, else 0 - */ -extern int ECP_BLS12381_set(ECP_BLS12381 *P, BIG_384_58 x, BIG_384_58 y); -/** @brief Extract x and y coordinates of an ECP point P - * - If x=y, returns only x - @param x BIG on exit = x coordinate of point - @param y BIG on exit = y coordinate of point (unless x=y) - @param P ECP instance (x,y) - @return sign of y, or -1 if P is point-at-infinity - */ -extern int ECP_BLS12381_get(BIG_384_58 x, BIG_384_58 y, ECP_BLS12381 *P); -/** @brief Adds ECP instance Q to ECP instance P - * - @param P ECP instance, on exit =P+Q - @param Q ECP instance to be added to P - */ -extern void ECP_BLS12381_add(ECP_BLS12381 *P, ECP_BLS12381 *Q); -/** @brief Subtracts ECP instance Q from ECP instance P - * - @param P ECP instance, on exit =P-Q - @param Q ECP instance to be subtracted from P - */ -extern void ECP_BLS12381_sub(ECP_BLS12381 *P, ECP_BLS12381 *Q); -/** @brief Set ECP to point(x,y) given just x and sign of y - * - Point P set to infinity if no such point on the curve. If x is on the curve then y is calculated from the curve equation. - The correct y value (plus or minus) is selected given its sign s. - @param P ECP instance to be set (x,[y]) - @param x BIG x coordinate of point - @param s an integer representing the "sign" of y, in fact its least significant bit. - */ -extern int ECP_BLS12381_setx(ECP_BLS12381 *P, BIG_384_58 x, int s); - -#endif - -/** @brief Multiplies Point by curve co-factor - * - @param Q ECP instance - */ -extern void ECP_BLS12381_cfp(ECP_BLS12381 *Q); - - -/** @brief Maps random BIG to curve point in constant time - * - @param Q ECP instance - @param x FP derived from hash - */ -extern void ECP_BLS12381_map2point(ECP_BLS12381 *Q, FP_BLS12381 *x); - -/** @brief Maps random BIG to curve point using hunt-and-peck - * - @param Q ECP instance - @param x Fp derived from hash - */ -extern void ECP_BLS12381_hap2point(ECP_BLS12381 *Q, BIG_384_58 x); - - -/** @brief Maps random octet to curve point of correct order - * - @param Q ECP instance of correct order - @param w OCTET byte array to be mapped - */ -extern void ECP_BLS12381_mapit(ECP_BLS12381 *Q, octet *w); - -/** @brief Converts an ECP point from Projective (x,y,z) coordinates to affine (x,y) coordinates - * - @param P ECP instance to be converted to affine form - */ -extern void ECP_BLS12381_affine(ECP_BLS12381 *P); -/** @brief Formats and outputs an ECP point to the console, in projective coordinates - * - @param P ECP instance to be printed - */ -extern void ECP_BLS12381_outputxyz(ECP_BLS12381 *P); -/** @brief Formats and outputs an ECP point to the console, converted to affine coordinates - * - @param P ECP instance to be printed - */ -extern void ECP_BLS12381_output(ECP_BLS12381 * P); - -/** @brief Formats and outputs an ECP point to the console - * - @param P ECP instance to be printed - */ -extern void ECP_BLS12381_rawoutput(ECP_BLS12381 * P); - -/** @brief Formats and outputs an ECP point to an octet string - The octet string is normally in the standard form 0x04|x|y - Here x (and y) are the x and y coordinates in left justified big-endian base 256 form. - For Montgomery curve it is 0x06|x - If c is true, only the x coordinate is provided as in 0x2|x if y is even, or 0x3|x if y is odd - @param c compression required, true or false - @param S output octet string - @param P ECP instance to be converted to an octet string - */ -extern void ECP_BLS12381_toOctet(octet *S, ECP_BLS12381 *P, bool c); -extern void ECP_BLS12381_toOctet_ZCash(octet *S, ECP_BLS12381 *P); -/** @brief Creates an ECP point from an octet string - * - The octet string is normally in the standard form 0x04|x|y - Here x (and y) are the x and y coordinates in left justified big-endian base 256 form. - For Montgomery curve it is 0x06|x - If in compressed form only the x coordinate is provided as in 0x2|x if y is even, or 0x3|x if y is odd - @param P ECP instance to be created from the octet string - @param S input octet string - return 1 if octet string corresponds to a point on the curve, else 0 - */ -extern int ECP_BLS12381_fromOctet(ECP_BLS12381 *P, octet *S); -extern int ECP_BLS12381_fromOctet_ZCash(ECP_BLS12381 *P, octet *S); -/** @brief Doubles an ECP instance P - * - @param P ECP instance, on exit =2*P - */ -extern void ECP_BLS12381_dbl(ECP_BLS12381 *P); -/** @brief Multiplies an ECP instance P by a small integer, side-channel resistant - * - @param P ECP instance, on exit =i*P - @param i small integer multiplier - @param b maximum number of bits in multiplier - */ -extern void ECP_BLS12381_pinmul(ECP_BLS12381 *P, int i, int b); -/** @brief Multiplies an ECP instance P by a BIG, side-channel resistant - * - Uses Montgomery ladder for Montgomery curves, otherwise fixed sized windows. - @param P ECP instance, on exit =b*P - @param b BIG number multiplier - - */ -extern void ECP_BLS12381_mul(ECP_BLS12381 *P, BIG_384_58 b); -/** @brief Calculates double multiplication P=e*P+f*Q, side-channel resistant - * - @param P ECP instance, on exit =e*P+f*Q - @param Q ECP instance - @param e BIG number multiplier - @param f BIG number multiplier - */ -extern void ECP_BLS12381_mul2(ECP_BLS12381 *P, ECP_BLS12381 *Q, BIG_384_58 e, BIG_384_58 f); -/** @brief Get Group Generator from ROM - * - @param G ECP instance - @return success - */ -extern int ECP_BLS12381_generator(ECP_BLS12381 *G); - - -#endif diff --git a/impl/cbits/fp12_BLS12381.c b/impl/cbits/fp12_BLS12381.c deleted file mode 100644 index f3912747c..000000000 --- a/impl/cbits/fp12_BLS12381.c +++ /dev/null @@ -1,986 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/* CORE Fp^12 functions */ -/* SU=m, m is Stack Usage (no lazy )*/ -/* FP12 elements are of the form a+i.b+i^2.c */ - -#include "fp12_BLS12381.h" -#include "config_curve_BLS12381.h" - -/* return 1 if b==c, no branching */ -static int teq(sign32 b, sign32 c) -{ - sign32 x = b ^ c; - x -= 1; // if x=0, x now -1 - return (int)((x >> 31) & 1); -} - - -/* Constant time select from pre-computed table */ -static void FP12_BLS12381_select(FP12_BLS12381 *f, FP12_BLS12381 g[], sign32 b) -{ - FP12_BLS12381 invf; - sign32 m = b >> 31; - sign32 babs = (b ^ m) - m; - - babs = (babs - 1) / 2; - - FP12_BLS12381_cmove(f, &g[0], teq(babs, 0)); // conditional move - FP12_BLS12381_cmove(f, &g[1], teq(babs, 1)); - FP12_BLS12381_cmove(f, &g[2], teq(babs, 2)); - FP12_BLS12381_cmove(f, &g[3], teq(babs, 3)); - FP12_BLS12381_cmove(f, &g[4], teq(babs, 4)); - FP12_BLS12381_cmove(f, &g[5], teq(babs, 5)); - FP12_BLS12381_cmove(f, &g[6], teq(babs, 6)); - FP12_BLS12381_cmove(f, &g[7], teq(babs, 7)); - - FP12_BLS12381_copy(&invf, f); - FP12_BLS12381_conj(&invf, &invf); // 1/f - FP12_BLS12381_cmove(f, &invf, (int)(m & 1)); -} - - - -/* test x==0 ? */ -/* SU= 8 */ -int FP12_BLS12381_iszilch(FP12_BLS12381 *x) -{ - if (FP4_BLS12381_iszilch(&(x->a)) && FP4_BLS12381_iszilch(&(x->b)) && FP4_BLS12381_iszilch(&(x->c))) return 1; - return 0; -} - -/* test x==1 ? */ -/* SU= 8 */ -int FP12_BLS12381_isunity(FP12_BLS12381 *x) -{ - if (FP4_BLS12381_isunity(&(x->a)) && FP4_BLS12381_iszilch(&(x->b)) && FP4_BLS12381_iszilch(&(x->c))) return 1; - return 0; -} - -/* FP12 copy w=x */ -/* SU= 16 */ -void FP12_BLS12381_copy(FP12_BLS12381 *w, FP12_BLS12381 *x) -{ - if (x == w) return; - FP4_BLS12381_copy(&(w->a), &(x->a)); - FP4_BLS12381_copy(&(w->b), &(x->b)); - FP4_BLS12381_copy(&(w->c), &(x->c)); - w->type = x->type; -} - -/* FP12 w=1 */ -/* SU= 8 */ -void FP12_BLS12381_one(FP12_BLS12381 *w) -{ - FP4_BLS12381_one(&(w->a)); - FP4_BLS12381_zero(&(w->b)); - FP4_BLS12381_zero(&(w->c)); - w->type = FP_UNITY; -} - -void FP12_BLS12381_zero(FP12_BLS12381 *w) -{ - FP4_BLS12381_zero(&(w->a)); - FP4_BLS12381_zero(&(w->b)); - FP4_BLS12381_zero(&(w->c)); - w->type = FP_ZILCH; -} - -/* return 1 if x==y, else 0 */ -/* SU= 16 */ -int FP12_BLS12381_equals(FP12_BLS12381 *x, FP12_BLS12381 *y) -{ - if (FP4_BLS12381_equals(&(x->a), &(y->a)) && FP4_BLS12381_equals(&(x->b), &(y->b)) && FP4_BLS12381_equals(&(x->c), &(y->c))) - return 1; - return 0; -} - -/* Set w=conj(x) */ -/* SU= 8 */ -void FP12_BLS12381_conj(FP12_BLS12381 *w, FP12_BLS12381 *x) -{ - FP12_BLS12381_copy(w, x); - FP4_BLS12381_conj(&(w->a), &(w->a)); - FP4_BLS12381_nconj(&(w->b), &(w->b)); - FP4_BLS12381_conj(&(w->c), &(w->c)); -} - -/* Create FP12 from FP4 */ -/* SU= 8 */ -void FP12_BLS12381_from_FP4(FP12_BLS12381 *w, FP4_BLS12381 *a) -{ - FP4_BLS12381_copy(&(w->a), a); - FP4_BLS12381_zero(&(w->b)); - FP4_BLS12381_zero(&(w->c)); - w->type = FP_SPARSEST; -} - -/* Create FP12 from 3 FP4's */ -/* SU= 16 */ -void FP12_BLS12381_from_FP4s(FP12_BLS12381 *w, FP4_BLS12381 *a, FP4_BLS12381 *b, FP4_BLS12381 *c) -{ - FP4_BLS12381_copy(&(w->a), a); - FP4_BLS12381_copy(&(w->b), b); - FP4_BLS12381_copy(&(w->c), c); - w->type = FP_DENSE; -} - -/* Granger-Scott Unitary Squaring. This does not benefit from lazy reduction */ -/* SU= 600 */ -void FP12_BLS12381_usqr(FP12_BLS12381 *w, FP12_BLS12381 *x) -{ - FP4_BLS12381 A, B, C, D; - - FP4_BLS12381_copy(&A, &(x->a)); - - FP4_BLS12381_sqr(&(w->a), &(x->a)); - FP4_BLS12381_add(&D, &(w->a), &(w->a)); - FP4_BLS12381_add(&(w->a), &D, &(w->a)); - - FP4_BLS12381_norm(&(w->a)); - FP4_BLS12381_nconj(&A, &A); - - FP4_BLS12381_add(&A, &A, &A); - FP4_BLS12381_add(&(w->a), &(w->a), &A); - FP4_BLS12381_sqr(&B, &(x->c)); - FP4_BLS12381_times_i(&B); - - FP4_BLS12381_add(&D, &B, &B); - FP4_BLS12381_add(&B, &B, &D); - FP4_BLS12381_norm(&B); - - FP4_BLS12381_sqr(&C, &(x->b)); - - FP4_BLS12381_add(&D, &C, &C); - FP4_BLS12381_add(&C, &C, &D); - - FP4_BLS12381_norm(&C); - FP4_BLS12381_conj(&(w->b), &(x->b)); - FP4_BLS12381_add(&(w->b), &(w->b), &(w->b)); - FP4_BLS12381_nconj(&(w->c), &(x->c)); - - FP4_BLS12381_add(&(w->c), &(w->c), &(w->c)); - FP4_BLS12381_add(&(w->b), &B, &(w->b)); - FP4_BLS12381_add(&(w->c), &C, &(w->c)); - - w->type = FP_DENSE; - FP12_BLS12381_reduce(w); /* reduce here as in pow function repeated squarings would trigger multiple reductions */ -} - -/* FP12 squaring w=x^2 */ -/* SU= 600 */ -void FP12_BLS12381_sqr(FP12_BLS12381 *w, FP12_BLS12381 *x) -{ - /* Use Chung-Hasan SQR2 method from http://cacr.uwaterloo.ca/techreports/2006/cacr2006-24.pdf */ - - FP4_BLS12381 A, B, C, D; - - if (x->type <= FP_UNITY) - { - FP12_BLS12381_copy(w, x); - return; - } - - FP4_BLS12381_sqr(&A, &(x->a)); - FP4_BLS12381_mul(&B, &(x->b), &(x->c)); - FP4_BLS12381_add(&B, &B, &B); - FP4_BLS12381_norm(&B); - FP4_BLS12381_sqr(&C, &(x->c)); - - FP4_BLS12381_mul(&D, &(x->a), &(x->b)); - FP4_BLS12381_add(&D, &D, &D); - FP4_BLS12381_add(&(w->c), &(x->a), &(x->c)); - FP4_BLS12381_add(&(w->c), &(x->b), &(w->c)); - FP4_BLS12381_norm(&(w->c)); - - FP4_BLS12381_sqr(&(w->c), &(w->c)); - - FP4_BLS12381_copy(&(w->a), &A); - FP4_BLS12381_add(&A, &A, &B); - - FP4_BLS12381_norm(&A); - - FP4_BLS12381_add(&A, &A, &C); - FP4_BLS12381_add(&A, &A, &D); - - FP4_BLS12381_norm(&A); - FP4_BLS12381_neg(&A, &A); - FP4_BLS12381_times_i(&B); - FP4_BLS12381_times_i(&C); - - FP4_BLS12381_add(&(w->a), &(w->a), &B); - FP4_BLS12381_add(&(w->b), &C, &D); - FP4_BLS12381_add(&(w->c), &(w->c), &A); - - if (x->type == FP_SPARSER || x->type == FP_SPARSEST) - w->type = FP_SPARSE; - else - w->type = FP_DENSE; - - FP12_BLS12381_norm(w); -} - -/* FP12 full multiplication w=w*y */ - - -/* SU= 896 */ -/* FP12 full multiplication w=w*y */ -void FP12_BLS12381_mul(FP12_BLS12381 *w, FP12_BLS12381 *y) -{ - FP4_BLS12381 z0, z1, z2, z3, t0, t1; - - FP4_BLS12381_mul(&z0, &(w->a), &(y->a)); - FP4_BLS12381_mul(&z2, &(w->b), &(y->b)); // - - FP4_BLS12381_add(&t0, &(w->a), &(w->b)); - FP4_BLS12381_add(&t1, &(y->a), &(y->b)); // - - FP4_BLS12381_norm(&t0); - FP4_BLS12381_norm(&t1); - - FP4_BLS12381_mul(&z1, &t0, &t1); - FP4_BLS12381_add(&t0, &(w->b), &(w->c)); - FP4_BLS12381_add(&t1, &(y->b), &(y->c)); // - - FP4_BLS12381_norm(&t0); - FP4_BLS12381_norm(&t1); - - FP4_BLS12381_mul(&z3, &t0, &t1); - - FP4_BLS12381_neg(&t0, &z0); - FP4_BLS12381_neg(&t1, &z2); - - FP4_BLS12381_add(&z1, &z1, &t0); // z1=z1-z0 - FP4_BLS12381_add(&(w->b), &z1, &t1); // z1=z1-z2 - FP4_BLS12381_add(&z3, &z3, &t1); // z3=z3-z2 - FP4_BLS12381_add(&z2, &z2, &t0); // z2=z2-z0 - - FP4_BLS12381_add(&t0, &(w->a), &(w->c)); - FP4_BLS12381_add(&t1, &(y->a), &(y->c)); - - FP4_BLS12381_norm(&t0); - FP4_BLS12381_norm(&t1); - - FP4_BLS12381_mul(&t0, &t1, &t0); - FP4_BLS12381_add(&z2, &z2, &t0); - - FP4_BLS12381_mul(&t0, &(w->c), &(y->c)); - FP4_BLS12381_neg(&t1, &t0); - - FP4_BLS12381_add(&(w->c), &z2, &t1); - FP4_BLS12381_add(&z3, &z3, &t1); - FP4_BLS12381_times_i(&t0); - FP4_BLS12381_add(&(w->b), &(w->b), &t0); - FP4_BLS12381_norm(&z3); - FP4_BLS12381_times_i(&z3); - FP4_BLS12381_add(&(w->a), &z0, &z3); - - w->type = FP_DENSE; - FP12_BLS12381_norm(w); -} - -/* FP12 full multiplication w=w*y */ -/* Supports sparse multiplicands */ -/* Usually w is denser than y */ -void FP12_BLS12381_ssmul(FP12_BLS12381 *w, FP12_BLS12381 *y) -{ - FP4_BLS12381 z0, z1, z2, z3, t0, t1; - if (w->type == FP_UNITY) - { - FP12_BLS12381_copy(w, y); - return; - } - if (y->type == FP_UNITY) - return; - - if (y->type >= FP_SPARSE) - { - FP4_BLS12381_mul(&z0, &(w->a), &(y->a)); // xa.ya always 11x11 - -#if SEXTIC_TWIST_BLS12381 == M_TYPE - if (y->type == FP_SPARSE || w->type == FP_SPARSE) - { - FP2_BLS12381_mul(&z2.b, &(w->b).b, &(y->b).b); - FP2_BLS12381_zero(&z2.a); - if (y->type != FP_SPARSE) - FP2_BLS12381_mul(&z2.a, &(w->b).b, &(y->b).a); - if (w->type != FP_SPARSE) - FP2_BLS12381_mul(&z2.a, &(w->b).a, &(y->b).b); - FP4_BLS12381_times_i(&z2); - } - else -#endif - FP4_BLS12381_mul(&z2, &(w->b), &(y->b)); // xb.yb could be 00x00 or 01x01 or or 10x10 or 11x00 or 11x10 or 11x01 or 11x11 - - FP4_BLS12381_add(&t0, &(w->a), &(w->b)); // (xa+xb) - FP4_BLS12381_add(&t1, &(y->a), &(y->b)); // (ya+yb) - - FP4_BLS12381_norm(&t0); - FP4_BLS12381_norm(&t1); - - FP4_BLS12381_mul(&z1, &t0, &t1); // (xa+xb)(ya+yb) always 11x11 - FP4_BLS12381_add(&t0, &(w->b), &(w->c)); // (xb+xc) - FP4_BLS12381_add(&t1, &(y->b), &(y->c)); // (yb+yc) - - FP4_BLS12381_norm(&t0); - FP4_BLS12381_norm(&t1); - - FP4_BLS12381_mul(&z3, &t0, &t1); // (xb+xc)(yb+yc) could be anything... - FP4_BLS12381_neg(&t0, &z0); // -(xa.ya) - FP4_BLS12381_neg(&t1, &z2); // -(xb.yb) - - FP4_BLS12381_add(&z1, &z1, &t0); - FP4_BLS12381_add(&(w->b), &z1, &t1); // /wb = (xa+xb)(ya+yb) -(xa.ya) -(xb.yb) = xa.yb + xb.ya - - FP4_BLS12381_add(&z3, &z3, &t1); // (xb+xc)(yb+yc) -(xb.yb) - FP4_BLS12381_add(&z2, &z2, &t0); // (xb.yb) - (xa.ya) - - FP4_BLS12381_add(&t0, &(w->a), &(w->c)); // (xa+xc) - FP4_BLS12381_add(&t1, &(y->a), &(y->c)); // (ya+yc) - - FP4_BLS12381_norm(&t0); - FP4_BLS12381_norm(&t1); - - FP4_BLS12381_mul(&t0, &t1, &t0); // (xa+xc)(ya+yc) always 11x11 - FP4_BLS12381_add(&z2, &z2, &t0); // (xb.yb) - (xa.ya) + (xa+xc)(ya+yc) - -#if SEXTIC_TWIST_BLS12381 == D_TYPE - if (y->type == FP_SPARSE || w->type == FP_SPARSE) - { - FP2_BLS12381_mul(&t0.a, &(w->c).a, &(y->c).a); - FP2_BLS12381_zero(&t0.b); - if (y->type != FP_SPARSE) - FP2_BLS12381_mul(&t0.b, &(w->c).a, &(y->c).b); - if (w->type != FP_SPARSE) - FP2_BLS12381_mul(&t0.b, &(w->c).b, &(y->c).a); - } - else -#endif - FP4_BLS12381_mul(&t0, &(w->c), &(y->c)); // (xc.yc) could be anything - - FP4_BLS12381_neg(&t1, &t0); // -(xc.yc) - - FP4_BLS12381_add(&(w->c), &z2, &t1); // wc = (xb.yb) - (xa.ya) + (xa+xc)(ya+yc) - (xc.yc) = xb.yb + xc.ya + xa.yc - FP4_BLS12381_add(&z3, &z3, &t1); // (xb+xc)(yb+yc) -(xb.yb) - (xc.yc) = xb.yc + xc.yb - FP4_BLS12381_times_i(&t0); // i.(xc.yc) - FP4_BLS12381_add(&(w->b), &(w->b), &t0); // wb = (xa+xb)(ya+yb) -(xa.ya) -(xb.yb) +i(xc.yc) - FP4_BLS12381_norm(&z3); - FP4_BLS12381_times_i(&z3); // i[(xb+xc)(yb+yc) -(xb.yb) - (xc.yc)] = i(xb.yc + xc.yb) - FP4_BLS12381_add(&(w->a), &z0, &z3); // wa = xa.ya + i(xb.yc + xc.yb) - } else { - if (w->type == FP_SPARSER || w->type == FP_SPARSEST) - { - FP12_BLS12381_smul(w, y); - return; - } -// dense by sparser - 13m -#if SEXTIC_TWIST_BLS12381 == D_TYPE - FP4_BLS12381_copy(&z3, &(w->b)); - FP4_BLS12381_mul(&z0, &(w->a), &(y->a)); - - if (y->type == FP_SPARSEST) - FP4_BLS12381_qmul(&z2, &(w->b), &(y->b).a.a); - else - FP4_BLS12381_pmul(&z2, &(w->b), &(y->b).a); - - FP4_BLS12381_add(&(w->b), &(w->a), &(w->b)); - FP4_BLS12381_copy(&t1, &(y->a)); - FP2_BLS12381_add(&t1.a, &t1.a, &(y->b).a); - - FP4_BLS12381_norm(&t1); - FP4_BLS12381_norm(&(w->b)); - - FP4_BLS12381_mul(&(w->b), &(w->b), &t1); - FP4_BLS12381_add(&z3, &z3, &(w->c)); - FP4_BLS12381_norm(&z3); - if (y->type == FP_SPARSEST) - FP4_BLS12381_qmul(&z3, &z3, &(y->b).a.a); - else - FP4_BLS12381_pmul(&z3, &z3, &(y->b).a); - - FP4_BLS12381_neg(&t0, &z0); - FP4_BLS12381_neg(&t1, &z2); - - FP4_BLS12381_add(&(w->b), &(w->b), &t0); // z1=z1-z0 - FP4_BLS12381_add(&(w->b), &(w->b), &t1); // z1=z1-z2 - - FP4_BLS12381_add(&z3, &z3, &t1); // z3=z3-z2 - FP4_BLS12381_add(&z2, &z2, &t0); // z2=z2-z0 - - FP4_BLS12381_add(&t0, &(w->a), &(w->c)); - FP4_BLS12381_norm(&t0); - FP4_BLS12381_norm(&z3); - - FP4_BLS12381_mul(&t0, &(y->a), &t0); - FP4_BLS12381_add(&(w->c), &z2, &t0); - - FP4_BLS12381_times_i(&z3); - FP4_BLS12381_add(&(w->a), &z0, &z3); -#endif -#if SEXTIC_TWIST_BLS12381 == M_TYPE - FP4_BLS12381_mul(&z0, &(w->a), &(y->a)); - FP4_BLS12381_add(&t0, &(w->a), &(w->b)); - FP4_BLS12381_norm(&t0); - - FP4_BLS12381_mul(&z1, &t0, &(y->a)); - FP4_BLS12381_add(&t0, &(w->b), &(w->c)); - FP4_BLS12381_norm(&t0); - - if (y->type == FP_SPARSEST) - FP4_BLS12381_qmul(&z3, &t0, &(y->c).b.a); - else - FP4_BLS12381_pmul(&z3, &t0, &(y->c).b); - - FP4_BLS12381_times_i(&z3); - - FP4_BLS12381_neg(&t0, &z0); - FP4_BLS12381_add(&z1, &z1, &t0); // z1=z1-z0 - - FP4_BLS12381_copy(&(w->b), &z1); - FP4_BLS12381_copy(&z2, &t0); - - FP4_BLS12381_add(&t0, &(w->a), &(w->c)); - FP4_BLS12381_add(&t1, &(y->a), &(y->c)); - - FP4_BLS12381_norm(&t0); - FP4_BLS12381_norm(&t1); - - FP4_BLS12381_mul(&t0, &t1, &t0); - FP4_BLS12381_add(&z2, &z2, &t0); - - if (y->type == FP_SPARSEST) - FP4_BLS12381_qmul(&t0, &(w->c), &(y->c).b.a); - else - FP4_BLS12381_pmul(&t0, &(w->c), &(y->c).b); - - FP4_BLS12381_times_i(&t0); - FP4_BLS12381_neg(&t1, &t0); - FP4_BLS12381_times_i(&t0); - - FP4_BLS12381_add(&(w->c), &z2, &t1); - FP4_BLS12381_add(&z3, &z3, &t1); - - FP4_BLS12381_add(&(w->b), &(w->b), &t0); - FP4_BLS12381_norm(&z3); - FP4_BLS12381_times_i(&z3); - FP4_BLS12381_add(&(w->a), &z0, &z3); - -#endif - } - w->type = FP_DENSE; - FP12_BLS12381_norm(w); -} - -/* FP12 multiplication w=w*y */ -/* catering for special case that arises from special form of ATE pairing line function */ -/* w and y are both sparser line functions - cost = 6m */ -void FP12_BLS12381_smul(FP12_BLS12381 *w, FP12_BLS12381 *y) -{ - FP2_BLS12381 w1, w2, w3, ta, tb, tc, td, te, t; - -// if (type==D_TYPE) -// { -#if SEXTIC_TWIST_BLS12381 == D_TYPE - FP2_BLS12381_mul(&w1, &(w->a).a, &(y->a).a); // A1.A2 - FP2_BLS12381_mul(&w2, &(w->a).b, &(y->a).b); // B1.B2 - - if (y->type == FP_SPARSEST || w->type == FP_SPARSEST) - { - if (y->type == FP_SPARSEST && w->type == FP_SPARSEST) - { - FP_BLS12381_mul(&w3.a, &(w->b).a.a, &(y->b).a.a); - FP_BLS12381_zero(&w3.b); - } else { - if (y->type != FP_SPARSEST) - FP2_BLS12381_pmul(&w3, &(y->b).a, &(w->b).a.a); - if (w->type != FP_SPARSEST) - FP2_BLS12381_pmul(&w3, &(w->b).a, &(y->b).a.a); - } - } - else - FP2_BLS12381_mul(&w3, &(w->b).a, &(y->b).a); // C1.C2 - - FP2_BLS12381_add(&ta, &(w->a).a, &(w->a).b); // A1+B1 - FP2_BLS12381_add(&tb, &(y->a).a, &(y->a).b); // A2+B2 - FP2_BLS12381_norm(&ta); - FP2_BLS12381_norm(&tb); - FP2_BLS12381_mul(&tc, &ta, &tb); // (A1+B1)(A2+B2) - FP2_BLS12381_add(&t, &w1, &w2); - FP2_BLS12381_neg(&t, &t); - FP2_BLS12381_add(&tc, &tc, &t); // (A1+B1)(A2+B2)-A1.A2-B1*B2 = (A1.B2+A2.B1) - - FP2_BLS12381_add(&ta, &(w->a).a, &(w->b).a); // A1+C1 - FP2_BLS12381_add(&tb, &(y->a).a, &(y->b).a); // A2+C2 - FP2_BLS12381_norm(&ta); - FP2_BLS12381_norm(&tb); - FP2_BLS12381_mul(&td, &ta, &tb); // (A1+C1)(A2+C2) - FP2_BLS12381_add(&t, &w1, &w3); - FP2_BLS12381_neg(&t, &t); - FP2_BLS12381_add(&td, &td, &t); // (A1+C1)(A2+C2)-A1.A2-C1*C2 = (A1.C2+A2.C1) - - FP2_BLS12381_add(&ta, &(w->a).b, &(w->b).a); // B1+C1 - FP2_BLS12381_add(&tb, &(y->a).b, &(y->b).a); // B2+C2 - FP2_BLS12381_norm(&ta); - FP2_BLS12381_norm(&tb); - FP2_BLS12381_mul(&te, &ta, &tb); // (B1+C1)(B2+C2) - FP2_BLS12381_add(&t, &w2, &w3); - FP2_BLS12381_neg(&t, &t); - FP2_BLS12381_add(&te, &te, &t); // (B1+C1)(B2+C2)-B1.B2-C1*C2 = (B1.C2+B2.C1) - - FP2_BLS12381_mul_ip(&w2); - FP2_BLS12381_add(&w1, &w1, &w2); - FP4_BLS12381_from_FP2s(&(w->a), &w1, &tc); - FP4_BLS12381_from_FP2s(&(w->b), &td, &te); // only norm these 2 - FP4_BLS12381_from_FP2(&(w->c), &w3); - - FP4_BLS12381_norm(&(w->a)); - FP4_BLS12381_norm(&(w->b)); -#endif -// } else { -#if SEXTIC_TWIST_BLS12381 == M_TYPE - FP2_BLS12381_mul(&w1, &(w->a).a, &(y->a).a); // A1.A2 - FP2_BLS12381_mul(&w2, &(w->a).b, &(y->a).b); // B1.B2 - if (y->type == FP_SPARSEST || w->type == FP_SPARSEST) - { - if (y->type == FP_SPARSEST && w->type == FP_SPARSEST) - { - FP_BLS12381_mul(&w3.a, &(w->c).b.a, &(y->c).b.a); - FP_BLS12381_zero(&w3.b); - } else { - if (y->type != FP_SPARSEST) - FP2_BLS12381_pmul(&w3, &(y->c).b, &(w->c).b.a); - if (w->type != FP_SPARSEST) - FP2_BLS12381_pmul(&w3, &(w->c).b, &(y->c).b.a); - } - } - else - FP2_BLS12381_mul(&w3, &(w->c).b, &(y->c).b); // F1.F2 - - FP2_BLS12381_add(&ta, &(w->a).a, &(w->a).b); // A1+B1 - FP2_BLS12381_add(&tb, &(y->a).a, &(y->a).b); // A2+B2 - FP2_BLS12381_norm(&ta); - FP2_BLS12381_norm(&tb); - FP2_BLS12381_mul(&tc, &ta, &tb); // (A1+B1)(A2+B2) - FP2_BLS12381_add(&t, &w1, &w2); - FP2_BLS12381_neg(&t, &t); - FP2_BLS12381_add(&tc, &tc, &t); // (A1+B1)(A2+B2)-A1.A2-B1*B2 = (A1.B2+A2.B1) - - FP2_BLS12381_add(&ta, &(w->a).a, &(w->c).b); // A1+F1 - FP2_BLS12381_add(&tb, &(y->a).a, &(y->c).b); // A2+F2 - FP2_BLS12381_norm(&ta); - FP2_BLS12381_norm(&tb); - FP2_BLS12381_mul(&td, &ta, &tb); // (A1+F1)(A2+F2) - FP2_BLS12381_add(&t, &w1, &w3); - FP2_BLS12381_neg(&t, &t); - FP2_BLS12381_add(&td, &td, &t); // (A1+F1)(A2+F2)-A1.A2-F1*F2 = (A1.F2+A2.F1) - - FP2_BLS12381_add(&ta, &(w->a).b, &(w->c).b); // B1+F1 - FP2_BLS12381_add(&tb, &(y->a).b, &(y->c).b); // B2+F2 - FP2_BLS12381_norm(&ta); - FP2_BLS12381_norm(&tb); - FP2_BLS12381_mul(&te, &ta, &tb); // (B1+F1)(B2+F2) - FP2_BLS12381_add(&t, &w2, &w3); - FP2_BLS12381_neg(&t, &t); - FP2_BLS12381_add(&te, &te, &t); // (B1+F1)(B2+F2)-B1.B2-F1*F2 = (B1.F2+B2.F1) - - FP2_BLS12381_mul_ip(&w2); - FP2_BLS12381_add(&w1, &w1, &w2); - FP4_BLS12381_from_FP2s(&(w->a), &w1, &tc); - - FP2_BLS12381_mul_ip(&w3); - FP2_BLS12381_norm(&w3); - FP4_BLS12381_from_FP2H(&(w->b), &w3); - - FP2_BLS12381_norm(&te); - FP2_BLS12381_mul_ip(&te); - FP4_BLS12381_from_FP2s(&(w->c), &te, &td); - - FP4_BLS12381_norm(&(w->a)); - FP4_BLS12381_norm(&(w->c)); -#endif - -// } - w->type = FP_SPARSE; -} - -/* Set w=1/x */ -/* SU= 600 */ -void FP12_BLS12381_inv(FP12_BLS12381 *w, FP12_BLS12381 *x) -{ - FP4_BLS12381 f0, f1, f2, f3; - - FP4_BLS12381_sqr(&f0, &(x->a)); - FP4_BLS12381_mul(&f1, &(x->b), &(x->c)); - FP4_BLS12381_times_i(&f1); - FP4_BLS12381_sub(&f0, &f0, &f1); /* y.a */ - FP4_BLS12381_norm(&f0); - - FP4_BLS12381_sqr(&f1, &(x->c)); - FP4_BLS12381_times_i(&f1); - FP4_BLS12381_mul(&f2, &(x->a), &(x->b)); - FP4_BLS12381_sub(&f1, &f1, &f2); /* y.b */ - FP4_BLS12381_norm(&f1); - - FP4_BLS12381_sqr(&f2, &(x->b)); - FP4_BLS12381_mul(&f3, &(x->a), &(x->c)); - FP4_BLS12381_sub(&f2, &f2, &f3); /* y.c */ - FP4_BLS12381_norm(&f2); - - FP4_BLS12381_mul(&f3, &(x->b), &f2); - FP4_BLS12381_times_i(&f3); - FP4_BLS12381_mul(&(w->a), &f0, &(x->a)); - FP4_BLS12381_add(&f3, &(w->a), &f3); - FP4_BLS12381_mul(&(w->c), &f1, &(x->c)); - FP4_BLS12381_times_i(&(w->c)); - - FP4_BLS12381_add(&f3, &(w->c), &f3); - FP4_BLS12381_norm(&f3); - - FP4_BLS12381_inv(&f3, &f3); - - FP4_BLS12381_mul(&(w->a), &f0, &f3); - FP4_BLS12381_mul(&(w->b), &f1, &f3); - FP4_BLS12381_mul(&(w->c), &f2, &f3); - w->type = FP_DENSE; -} - -/* constant time powering by small integer of max length bts */ - -void FP12_BLS12381_pinpow(FP12_BLS12381 *r, int e, int bts) -{ - int i, b; - FP12_BLS12381 R[2]; - - FP12_BLS12381_one(&R[0]); - FP12_BLS12381_copy(&R[1], r); - - for (i = bts - 1; i >= 0; i--) - { - b = (e >> i) & 1; - FP12_BLS12381_mul(&R[1 - b], &R[b]); - FP12_BLS12381_usqr(&R[b], &R[b]); - } - FP12_BLS12381_copy(r, &R[0]); -} - -/* Compressed powering of unitary elements y=x^(e mod r) */ - -void FP12_BLS12381_compow(FP4_BLS12381 *c, FP12_BLS12381 *x, BIG_384_58 e, BIG_384_58 r) -{ - FP12_BLS12381 g1, g2; - FP4_BLS12381 cp, cpm1, cpm2; - FP2_BLS12381 f; - BIG_384_58 q, a, b, m; - - BIG_384_58_rcopy(a, Fra_BLS12381); - BIG_384_58_rcopy(b, Frb_BLS12381); - FP2_BLS12381_from_BIGs(&f, a, b); - - BIG_384_58_rcopy(q, Modulus_BLS12381); - - FP12_BLS12381_copy(&g1, x); - FP12_BLS12381_copy(&g2, x); - - BIG_384_58_copy(m, q); - BIG_384_58_mod(m, r); - - BIG_384_58_copy(a, e); - BIG_384_58_mod(a, m); - - BIG_384_58_copy(b, e); - BIG_384_58_sdiv(b, m); - - FP12_BLS12381_trace(c, &g1); - - if (BIG_384_58_iszilch(b)) - { - FP4_BLS12381_xtr_pow(c, c, e); - return; - } - - FP12_BLS12381_frob(&g2, &f); - FP12_BLS12381_trace(&cp, &g2); - - FP12_BLS12381_conj(&g1, &g1); - FP12_BLS12381_mul(&g2, &g1); - FP12_BLS12381_trace(&cpm1, &g2); - FP12_BLS12381_mul(&g2, &g1); - FP12_BLS12381_trace(&cpm2, &g2); - - FP4_BLS12381_xtr_pow2(c, &cp, c, &cpm1, &cpm2, a, b); -} - - -/* SU= 528 */ -/* set r=a^b */ -/* Note this is simple square and multiply, so not side-channel safe */ - -void FP12_BLS12381_pow(FP12_BLS12381 *r, FP12_BLS12381 *a, BIG_384_58 b) -{ - FP12_BLS12381 w, sf; - BIG_384_58 b1, b3; - int i, nb, bt; - BIG_384_58_copy(b1, b); - BIG_384_58_norm(b1); - BIG_384_58_pmul(b3, b1, 3); - BIG_384_58_norm(b3); - - FP12_BLS12381_copy(&sf, a); - FP12_BLS12381_norm(&sf); - FP12_BLS12381_copy(&w, &sf); - - - nb = BIG_384_58_nbits(b3); - for (i = nb - 2; i >= 1; i--) - { - FP12_BLS12381_usqr(&w, &w); - bt = BIG_384_58_bit(b3, i) - BIG_384_58_bit(b1, i); - if (bt == 1) - FP12_BLS12381_mul(&w, &sf); - if (bt == -1) - { - FP12_BLS12381_conj(&sf, &sf); - FP12_BLS12381_mul(&w, &sf); - FP12_BLS12381_conj(&sf, &sf); - } - } - - FP12_BLS12381_copy(r, &w); - FP12_BLS12381_reduce(r); -} - -/* p=q0^u0.q1^u1.q2^u2.q3^u3 */ -/* Side channel attack secure */ -// Bos & Costello https://eprint.iacr.org/2013/458.pdf -// Faz-Hernandez & Longa & Sanchez https://eprint.iacr.org/2013/158.pdf - -void FP12_BLS12381_pow4(FP12_BLS12381 *p, FP12_BLS12381 *q, BIG_384_58 u[4]) -{ - int i, j, k, nb, pb, bt; - FP12_BLS12381 g[8], r; - BIG_384_58 t[4], mt; - sign8 w[NLEN_384_58 * BASEBITS_384_58 + 1]; - sign8 s[NLEN_384_58 * BASEBITS_384_58 + 1]; - - for (i = 0; i < 4; i++) - BIG_384_58_copy(t[i], u[i]); - - -// Precomputed table - FP12_BLS12381_copy(&g[0], &q[0]); // q[0] - FP12_BLS12381_copy(&g[1], &g[0]); - FP12_BLS12381_mul(&g[1], &q[1]); // q[0].q[1] - FP12_BLS12381_copy(&g[2], &g[0]); - FP12_BLS12381_mul(&g[2], &q[2]); // q[0].q[2] - FP12_BLS12381_copy(&g[3], &g[1]); - FP12_BLS12381_mul(&g[3], &q[2]); // q[0].q[1].q[2] - FP12_BLS12381_copy(&g[4], &g[0]); - FP12_BLS12381_mul(&g[4], &q[3]); // q[0].q[3] - FP12_BLS12381_copy(&g[5], &g[1]); - FP12_BLS12381_mul(&g[5], &q[3]); // q[0].q[1].q[3] - FP12_BLS12381_copy(&g[6], &g[2]); - FP12_BLS12381_mul(&g[6], &q[3]); // q[0].q[2].q[3] - FP12_BLS12381_copy(&g[7], &g[3]); - FP12_BLS12381_mul(&g[7], &q[3]); // q[0].q[1].q[2].q[3] - -// Make it odd - pb = 1 - BIG_384_58_parity(t[0]); - BIG_384_58_inc(t[0], pb); - BIG_384_58_norm(t[0]); - -// Number of bits - BIG_384_58_zero(mt); - for (i = 0; i < 4; i++) - { - BIG_384_58_or(mt, mt, t[i]); - } - nb = 1 + BIG_384_58_nbits(mt); - -// Sign pivot - s[nb - 1] = 1; - for (i = 0; i < nb - 1; i++) - { - BIG_384_58_fshr(t[0], 1); - s[i] = 2 * BIG_384_58_parity(t[0]) - 1; - } - -// Recoded exponent - for (i = 0; i < nb; i++) - { - w[i] = 0; - k = 1; - for (j = 1; j < 4; j++) - { - bt = s[i] * BIG_384_58_parity(t[j]); - BIG_384_58_fshr(t[j], 1); - - BIG_384_58_dec(t[j], (bt >> 1)); - BIG_384_58_norm(t[j]); - w[i] += bt * k; - k *= 2; - } - } - -// Main loop - FP12_BLS12381_select(p, g, 2 * w[nb - 1] + 1); - for (i = nb - 2; i >= 0; i--) - { - FP12_BLS12381_select(&r, g, 2 * w[i] + s[i]); - FP12_BLS12381_usqr(p, p); - FP12_BLS12381_mul(p, &r); - } -// apply correction - FP12_BLS12381_conj(&r, &q[0]); - FP12_BLS12381_mul(&r, p); - FP12_BLS12381_cmove(p, &r, pb); - - FP12_BLS12381_reduce(p); -} - -/* Set w=w^p using Frobenius */ -/* SU= 160 */ -void FP12_BLS12381_frob(FP12_BLS12381 *w, FP2_BLS12381 *f) -{ - FP2_BLS12381 f2, f3; - FP2_BLS12381_sqr(&f2, f); /* f2=f^2 */ - FP2_BLS12381_mul(&f3, &f2, f); /* f3=f^3 */ - - FP4_BLS12381_frob(&(w->a), &f3); - FP4_BLS12381_frob(&(w->b), &f3); - FP4_BLS12381_frob(&(w->c), &f3); - - FP4_BLS12381_pmul(&(w->b), &(w->b), f); - FP4_BLS12381_pmul(&(w->c), &(w->c), &f2); - w->type = FP_DENSE; -} - -/* SU= 8 */ -/* normalise all components of w */ -void FP12_BLS12381_norm(FP12_BLS12381 *w) -{ - FP4_BLS12381_norm(&(w->a)); - FP4_BLS12381_norm(&(w->b)); - FP4_BLS12381_norm(&(w->c)); -} - -/* SU= 8 */ -/* reduce all components of w */ -void FP12_BLS12381_reduce(FP12_BLS12381 *w) -{ - FP4_BLS12381_reduce(&(w->a)); - FP4_BLS12381_reduce(&(w->b)); - FP4_BLS12381_reduce(&(w->c)); -} - -/* trace function w=trace(x) */ -/* SU= 8 */ -void FP12_BLS12381_trace(FP4_BLS12381 *w, FP12_BLS12381 *x) -{ - FP4_BLS12381_imul(w, &(x->a), 3); - FP4_BLS12381_reduce(w); -} - -/* SU= 8 */ -/* Output w in hex */ -void FP12_BLS12381_output(FP12_BLS12381 *w) -{ - printf("["); - FP4_BLS12381_output(&(w->a)); - printf(","); - FP4_BLS12381_output(&(w->b)); - printf(","); - FP4_BLS12381_output(&(w->c)); - printf("]"); -} - -/* SU= 64 */ -/* Convert g to octet string w */ -void FP12_BLS12381_toOctet(octet *W, FP12_BLS12381 *g) -{ - BIG_384_58 a; - W->len = 12 * MODBYTES_384_58; - - FP_BLS12381_redc(a, &(g->a.a.a)); - BIG_384_58_toBytes(&(W->val[0]), a); - FP_BLS12381_redc(a, &(g->a.a.b)); - BIG_384_58_toBytes(&(W->val[MODBYTES_384_58]), a); - FP_BLS12381_redc(a, &(g->a.b.a)); - BIG_384_58_toBytes(&(W->val[2 * MODBYTES_384_58]), a); - FP_BLS12381_redc(a, &(g->a.b.b)); - BIG_384_58_toBytes(&(W->val[3 * MODBYTES_384_58]), a); - FP_BLS12381_redc(a, &(g->b.a.a)); - BIG_384_58_toBytes(&(W->val[4 * MODBYTES_384_58]), a); - FP_BLS12381_redc(a, &(g->b.a.b)); - BIG_384_58_toBytes(&(W->val[5 * MODBYTES_384_58]), a); - FP_BLS12381_redc(a, &(g->b.b.a)); - BIG_384_58_toBytes(&(W->val[6 * MODBYTES_384_58]), a); - FP_BLS12381_redc(a, &(g->b.b.b)); - BIG_384_58_toBytes(&(W->val[7 * MODBYTES_384_58]), a); - FP_BLS12381_redc(a, &(g->c.a.a)); - BIG_384_58_toBytes(&(W->val[8 * MODBYTES_384_58]), a); - FP_BLS12381_redc(a, &(g->c.a.b)); - BIG_384_58_toBytes(&(W->val[9 * MODBYTES_384_58]), a); - FP_BLS12381_redc(a, &(g->c.b.a)); - BIG_384_58_toBytes(&(W->val[10 * MODBYTES_384_58]), a); - FP_BLS12381_redc(a, &(g->c.b.b)); - BIG_384_58_toBytes(&(W->val[11 * MODBYTES_384_58]), a); -} - -/* SU= 24 */ -/* Restore g from octet string w */ -void FP12_BLS12381_fromOctet(FP12_BLS12381 *g, octet *W) -{ - BIG_384_58 b; - BIG_384_58_fromBytes(b, &W->val[0]); - FP_BLS12381_nres(&(g->a.a.a), b); - BIG_384_58_fromBytes(b, &W->val[MODBYTES_384_58]); - FP_BLS12381_nres(&(g->a.a.b), b); - BIG_384_58_fromBytes(b, &W->val[2 * MODBYTES_384_58]); - FP_BLS12381_nres(&(g->a.b.a), b); - BIG_384_58_fromBytes(b, &W->val[3 * MODBYTES_384_58]); - FP_BLS12381_nres(&(g->a.b.b), b); - BIG_384_58_fromBytes(b, &W->val[4 * MODBYTES_384_58]); - FP_BLS12381_nres(&(g->b.a.a), b); - BIG_384_58_fromBytes(b, &W->val[5 * MODBYTES_384_58]); - FP_BLS12381_nres(&(g->b.a.b), b); - BIG_384_58_fromBytes(b, &W->val[6 * MODBYTES_384_58]); - FP_BLS12381_nres(&(g->b.b.a), b); - BIG_384_58_fromBytes(b, &W->val[7 * MODBYTES_384_58]); - FP_BLS12381_nres(&(g->b.b.b), b); - BIG_384_58_fromBytes(b, &W->val[8 * MODBYTES_384_58]); - FP_BLS12381_nres(&(g->c.a.a), b); - BIG_384_58_fromBytes(b, &W->val[9 * MODBYTES_384_58]); - FP_BLS12381_nres(&(g->c.a.b), b); - BIG_384_58_fromBytes(b, &W->val[10 * MODBYTES_384_58]); - FP_BLS12381_nres(&(g->c.b.a), b); - BIG_384_58_fromBytes(b, &W->val[11 * MODBYTES_384_58]); - FP_BLS12381_nres(&(g->c.b.b), b); -} - -/* Move b to a if d=1 */ -void FP12_BLS12381_cmove(FP12_BLS12381 *f, FP12_BLS12381 *g, int d) -{ - FP4_BLS12381_cmove(&(f->a), &(g->a), d); - FP4_BLS12381_cmove(&(f->b), &(g->b), d); - FP4_BLS12381_cmove(&(f->c), &(g->c), d); - d = ~(d - 1); - f->type ^= (f->type ^ g->type)&d; -} - diff --git a/impl/cbits/fp12_BLS12381.h b/impl/cbits/fp12_BLS12381.h deleted file mode 100644 index 40a6089f1..000000000 --- a/impl/cbits/fp12_BLS12381.h +++ /dev/null @@ -1,232 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file fp12.h - * @author Mike Scott - * @brief FP12 Header File - * - */ - -#ifndef FP12_BLS12381_H -#define FP12_BLS12381_H - -#include "fp4_BLS12381.h" - -/** - @brief FP12 Structure - towered over three FP4 -*/ - -typedef struct -{ - FP4_BLS12381 a; /**< first part of FP12 */ - FP4_BLS12381 b; /**< second part of FP12 */ - FP4_BLS12381 c; /**< third part of FP12 */ - int type; /**< record sparseness */ -} FP12_BLS12381; - -extern const BIG_384_58 Fra_BLS12381; /**< real part of BN curve Frobenius Constant */ -extern const BIG_384_58 Frb_BLS12381; /**< imaginary part of BN curve Frobenius Constant */ - -/* FP12 prototypes */ -/** @brief Tests for FP12 equal to zero - * - @param x FP12 number to be tested - @return 1 if zero, else returns 0 - */ -extern int FP12_BLS12381_iszilch(FP12_BLS12381 *x); -/** @brief Tests for FP12 equal to unity - * - @param x FP12 number to be tested - @return 1 if unity, else returns 0 - */ -extern int FP12_BLS12381_isunity(FP12_BLS12381 *x); -/** @brief Copy FP12 to another FP12 - * - @param x FP12 instance, on exit = y - @param y FP12 instance to be copied - */ -extern void FP12_BLS12381_copy(FP12_BLS12381 *x, FP12_BLS12381 *y); -/** @brief Set FP12 to unity - * - @param x FP12 instance to be set to one - */ -extern void FP12_BLS12381_one(FP12_BLS12381 *x); - -/** @brief Set FP12 to zero - * - @param x FP12 instance to be set to zero - */ -extern void FP12_BLS12381_zero(FP12_BLS12381 *x); - -/** @brief Tests for equality of two FP12s - * - @param x FP12 instance to be compared - @param y FP12 instance to be compared - @return 1 if x=y, else returns 0 - */ -extern int FP12_BLS12381_equals(FP12_BLS12381 *x, FP12_BLS12381 *y); -/** @brief Conjugation of FP12 - * - If y=(a,b,c) (where a,b,c are its three FP4 components) on exit x=(conj(a),-conj(b),conj(c)) - @param x FP12 instance, on exit = conj(y) - @param y FP12 instance - */ -extern void FP12_BLS12381_conj(FP12_BLS12381 *x, FP12_BLS12381 *y); -/** @brief Initialise FP12 from single FP4 - * - Sets first FP4 component of an FP12, other components set to zero - @param x FP12 instance to be initialised - @param a FP4 to form first part of FP4 - */ -extern void FP12_BLS12381_from_FP4(FP12_BLS12381 *x, FP4_BLS12381 *a); -/** @brief Initialise FP12 from three FP4s - * - @param x FP12 instance to be initialised - @param a FP4 to form first part of FP12 - @param b FP4 to form second part of FP12 - @param c FP4 to form third part of FP12 - */ -extern void FP12_BLS12381_from_FP4s(FP12_BLS12381 *x, FP4_BLS12381 *a, FP4_BLS12381* b, FP4_BLS12381 *c); -/** @brief Fast Squaring of an FP12 in "unitary" form - * - @param x FP12 instance, on exit = y^2 - @param y FP4 instance, must be unitary - */ -extern void FP12_BLS12381_usqr(FP12_BLS12381 *x, FP12_BLS12381 *y); -/** @brief Squaring an FP12 - * - @param x FP12 instance, on exit = y^2 - @param y FP12 instance - */ -extern void FP12_BLS12381_sqr(FP12_BLS12381 *x, FP12_BLS12381 *y); - -/** @brief Fast multiplication of two sparse FP12s that arises from ATE pairing line functions - * - @param x FP12 instance, on exit = x*y - @param y FP12 instance, of special form - */ -extern void FP12_BLS12381_smul(FP12_BLS12381 *x, FP12_BLS12381 *y); - -/** @brief Fast multiplication of what may be sparse multiplicands - * - @param x FP12 instance, on exit = x*y - @param y FP12 instance, of special form - */ -extern void FP12_BLS12381_ssmul(FP12_BLS12381 *x, FP12_BLS12381 *y); - - -/** @brief Full unconditional Multiplication of two FP12s - * - @param x FP12 instance, on exit = x*y - @param y FP12 instance, the multiplier - */ -extern void FP12_BLS12381_mul(FP12_BLS12381 *x, FP12_BLS12381 *y); -/** @brief Inverting an FP12 - * - @param x FP12 instance, on exit = 1/y - @param y FP12 instance - */ -extern void FP12_BLS12381_inv(FP12_BLS12381 *x, FP12_BLS12381 *y); -/** @brief Raises an FP12 to the power of a BIG - * - @param r FP12 instance, on exit = y^b - @param x FP12 instance - @param b BIG number - */ -extern void FP12_BLS12381_pow(FP12_BLS12381 *r, FP12_BLS12381 *x, BIG_384_58 b); -/** @brief Raises an FP12 instance x to a small integer power, side-channel resistant - * - @param x FP12 instance, on exit = x^i - @param i small integer exponent - @param b maximum number of bits in exponent - */ -extern void FP12_BLS12381_pinpow(FP12_BLS12381 *x, int i, int b); - -/** @brief Raises an FP12 instance x to a BIG power, compressed to FP4 - * - @param c FP4 instance, on exit = x^(e mod r) as FP4 - @param x FP12 input - @param e BIG exponent - @param r BIG group order - */ -extern void FP12_BLS12381_compow(FP4_BLS12381 *c, FP12_BLS12381 *x, BIG_384_58 e, BIG_384_58 r); - -/** @brief Calculate x[0]^b[0].x[1]^b[1].x[2]^b[2].x[3]^b[3], side-channel resistant - * - @param r FP12 instance, on exit = x[0]^b[0].x[1]^b[1].x[2]^b[2].x[3]^b[3] - @param x FP12 array with 4 FP12s - @param b BIG array of 4 exponents - */ -extern void FP12_BLS12381_pow4(FP12_BLS12381 *r, FP12_BLS12381 *x, BIG_384_58 *b); -/** @brief Raises an FP12 to the power of the internal modulus p, using the Frobenius - * - @param x FP12 instance, on exit = x^p - @param f FP2 precalculated Frobenius constant - */ -extern void FP12_BLS12381_frob(FP12_BLS12381 *x, FP2_BLS12381 *f); -/** @brief Reduces all components of possibly unreduced FP12 mod Modulus - * - @param x FP12 instance, on exit reduced mod Modulus - */ -extern void FP12_BLS12381_reduce(FP12_BLS12381 *x); -/** @brief Normalises the components of an FP12 - * - @param x FP12 instance to be normalised - */ -extern void FP12_BLS12381_norm(FP12_BLS12381 *x); -/** @brief Formats and outputs an FP12 to the console - * - @param x FP12 instance to be printed - */ -extern void FP12_BLS12381_output(FP12_BLS12381 *x); -/** @brief Formats and outputs an FP12 instance to an octet string - * - Serializes the components of an FP12 to big-endian base 256 form. - @param S output octet string - @param x FP12 instance to be converted to an octet string - */ -extern void FP12_BLS12381_toOctet(octet *S, FP12_BLS12381 *x); -/** @brief Creates an FP12 instance from an octet string - * - De-serializes the components of an FP12 to create an FP12 from big-endian base 256 components. - @param x FP12 instance to be created from an octet string - @param S input octet string - - */ -extern void FP12_BLS12381_fromOctet(FP12_BLS12381 *x, octet *S); -/** @brief Calculate the trace of an FP12 - * - @param t FP4 trace of x, on exit = tr(x) - @param x FP12 instance - - */ -extern void FP12_BLS12381_trace(FP4_BLS12381 *t, FP12_BLS12381 *x); - -/** @brief Conditional copy of FP12 number - * - Conditionally copies second parameter to the first (without branching) - @param x FP12 instance, set to y if s!=0 - @param y another FP12 instance - @param s copy only takes place if not equal to 0 - */ -extern void FP12_BLS12381_cmove(FP12_BLS12381 *x, FP12_BLS12381 *y, int s); - - -#endif diff --git a/impl/cbits/fp2_BLS12381.c b/impl/cbits/fp2_BLS12381.c deleted file mode 100644 index 24126822d..000000000 --- a/impl/cbits/fp2_BLS12381.c +++ /dev/null @@ -1,488 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/* CORE Fp^2 functions */ -/* SU=m, m is Stack Usage (no lazy )*/ - -/* FP2 elements are of the form a+ib, where i is sqrt(-1) */ - -#include "fp2_BLS12381.h" - -/* test x==0 ? */ -/* SU= 8 */ -int FP2_BLS12381_iszilch(FP2_BLS12381 *x) -{ - return (FP_BLS12381_iszilch(&(x->a)) & FP_BLS12381_iszilch(&(x->b))); -} - -/* Move b to a if d=1 */ -void FP2_BLS12381_cmove(FP2_BLS12381 *f, FP2_BLS12381 *g, int d) -{ - FP_BLS12381_cmove(&(f->a), &(g->a), d); - FP_BLS12381_cmove(&(f->b), &(g->b), d); -} - -/* test x==1 ? */ -/* SU= 48 */ -int FP2_BLS12381_isunity(FP2_BLS12381 *x) -{ - FP_BLS12381 one; - FP_BLS12381_one(&one); - return (FP_BLS12381_equals(&(x->a), &one) & FP_BLS12381_iszilch(&(x->b))); -} - -/* SU= 8 */ -/* Fully reduce a and b mod Modulus */ -void FP2_BLS12381_reduce(FP2_BLS12381 *w) -{ - FP_BLS12381_reduce(&(w->a)); - FP_BLS12381_reduce(&(w->b)); -} - -/* return 1 if x==y, else 0 */ -/* SU= 16 */ -int FP2_BLS12381_equals(FP2_BLS12381 *x, FP2_BLS12381 *y) -{ - return (FP_BLS12381_equals(&(x->a), &(y->a)) & FP_BLS12381_equals(&(x->b), &(y->b))); -} - -/* Create FP2 from two FPs */ -/* SU= 16 */ -void FP2_BLS12381_from_FPs(FP2_BLS12381 *w, FP_BLS12381 *x, FP_BLS12381 *y) -{ - FP_BLS12381_copy(&(w->a), x); - FP_BLS12381_copy(&(w->b), y); -} - -/* Create FP2 from two BIGS */ -/* SU= 16 */ -void FP2_BLS12381_from_BIGs(FP2_BLS12381 *w, BIG_384_58 x, BIG_384_58 y) -{ - FP_BLS12381_nres(&(w->a), x); - FP_BLS12381_nres(&(w->b), y); -} - -/* Create FP2 from two ints */ -void FP2_BLS12381_from_ints(FP2_BLS12381 *w, int xa, int xb) -{ - FP_BLS12381 a,b; - FP_BLS12381_from_int(&a,xa); - FP_BLS12381_from_int(&b,xb); - FP2_BLS12381_from_FPs(w,&a,&b); -// BIG_384_58 a, b; -// BIG_384_58_zero(a); BIG_384_58_inc(a, xa); BIG_384_58_norm(a); -// BIG_384_58_zero(b); BIG_384_58_inc(b, xb); BIG_384_58_norm(b); -// FP2_BLS12381_from_BIGs(w, a, b); -} - -/* Create FP2 from FP */ -/* SU= 8 */ -void FP2_BLS12381_from_FP(FP2_BLS12381 *w, FP_BLS12381 *x) -{ - FP_BLS12381_copy(&(w->a), x); - FP_BLS12381_zero(&(w->b)); -} - -/* Create FP2 from BIG */ -/* SU= 8 */ -void FP2_BLS12381_from_BIG(FP2_BLS12381 *w, BIG_384_58 x) -{ - FP_BLS12381_nres(&(w->a), x); - FP_BLS12381_zero(&(w->b)); -} - -/* FP2 copy w=x */ -/* SU= 16 */ -void FP2_BLS12381_copy(FP2_BLS12381 *w, FP2_BLS12381 *x) -{ - if (w == x) return; - FP_BLS12381_copy(&(w->a), &(x->a)); - FP_BLS12381_copy(&(w->b), &(x->b)); -} - -/* FP2 set w=0 */ -/* SU= 8 */ -void FP2_BLS12381_zero(FP2_BLS12381 *w) -{ - FP_BLS12381_zero(&(w->a)); - FP_BLS12381_zero(&(w->b)); -} - -/* FP2 set w=1 */ -/* SU= 48 */ -void FP2_BLS12381_one(FP2_BLS12381 *w) -{ - FP_BLS12381 one; - FP_BLS12381_one(&one); - FP2_BLS12381_from_FP(w, &one); -} - -void FP2_BLS12381_rcopy(FP2_BLS12381 *w,const BIG_384_58 a,const BIG_384_58 b) -{ - FP_BLS12381_rcopy(&(w->a),a); - FP_BLS12381_rcopy(&(w->b),b); -} - -int FP2_BLS12381_sign(FP2_BLS12381 *w) -{ - int p1,p2; - p1=FP_BLS12381_sign(&(w->a)); - p2=FP_BLS12381_sign(&(w->b)); -#ifdef BIG_ENDIAN_SIGN_BLS12381 - p2 ^= (p1 ^ p2)&FP_BLS12381_iszilch(&(w->b)); - return p2; -#else - p1 ^= (p1 ^ p2)&FP_BLS12381_iszilch(&(w->a)); - return p1; -#endif -} - -/* Set w=-x */ -/* SU= 88 */ -void FP2_BLS12381_neg(FP2_BLS12381 *w, FP2_BLS12381 *x) -{ - /* Just one neg! */ - FP_BLS12381 m, t; - FP_BLS12381_add(&m, &(x->a), &(x->b)); - FP_BLS12381_neg(&m, &m); - FP_BLS12381_add(&t, &m, &(x->b)); - FP_BLS12381_add(&(w->b), &m, &(x->a)); - FP_BLS12381_copy(&(w->a), &t); - -} - -/* Set w=conj(x) */ -/* SU= 16 */ -void FP2_BLS12381_conj(FP2_BLS12381 *w, FP2_BLS12381 *x) -{ - FP_BLS12381_copy(&(w->a), &(x->a)); - FP_BLS12381_neg(&(w->b), &(x->b)); - FP_BLS12381_norm(&(w->b)); -} - -/* Set w=x+y */ -/* SU= 16 */ -void FP2_BLS12381_add(FP2_BLS12381 *w, FP2_BLS12381 *x, FP2_BLS12381 *y) -{ - FP_BLS12381_add(&(w->a), &(x->a), &(y->a)); - FP_BLS12381_add(&(w->b), &(x->b), &(y->b)); -} - -/* Set w=x-y */ -/* Input y MUST be normed */ -void FP2_BLS12381_sub(FP2_BLS12381 *w, FP2_BLS12381 *x, FP2_BLS12381 *y) -{ - FP2_BLS12381 m; - FP2_BLS12381_neg(&m, y); - FP2_BLS12381_add(w, x, &m); -} - -/* Set w=s*x, where s is FP */ -/* SU= 16 */ -void FP2_BLS12381_pmul(FP2_BLS12381 *w, FP2_BLS12381 *x, FP_BLS12381 *s) -{ - FP_BLS12381_mul(&(w->a), &(x->a), s); - FP_BLS12381_mul(&(w->b), &(x->b), s); -} - -/* SU= 16 */ -/* Set w=s*x, where s is int */ -void FP2_BLS12381_imul(FP2_BLS12381 *w, FP2_BLS12381 *x, int s) -{ - FP_BLS12381_imul(&(w->a), &(x->a), s); - FP_BLS12381_imul(&(w->b), &(x->b), s); -} - -/* Set w=x^2 */ -/* SU= 128 */ -void FP2_BLS12381_sqr(FP2_BLS12381 *w, FP2_BLS12381 *x) -{ - FP_BLS12381 w1, w3, mb; - - FP_BLS12381_add(&w1, &(x->a), &(x->b)); - FP_BLS12381_neg(&mb, &(x->b)); - - FP_BLS12381_add(&w3, &(x->a), &(x->a)); - FP_BLS12381_norm(&w3); - FP_BLS12381_mul(&(w->b), &w3, &(x->b)); - - FP_BLS12381_add(&(w->a), &(x->a), &mb); - - FP_BLS12381_norm(&w1); - FP_BLS12381_norm(&(w->a)); - - FP_BLS12381_mul(&(w->a), &w1, &(w->a)); /* w->a#2 w->a=1 w1&w2=6 w1*w2=2 */ -} - -/* Set w=x*y */ -/* Inputs MUST be normed */ -/* Now uses Lazy reduction */ -void FP2_BLS12381_mul(FP2_BLS12381 *w, FP2_BLS12381 *x, FP2_BLS12381 *y) -{ - DBIG_384_58 A, B, E, F, pR; - BIG_384_58 C, D, p; - - BIG_384_58_rcopy(p, Modulus_BLS12381); - BIG_384_58_dsucopy(pR, p); - -// reduce excesses of a and b as required (so product < pR) - - if ((sign64)(x->a.XES + x->b.XES) * (y->a.XES + y->b.XES) > (sign64)FEXCESS_BLS12381) - { -#ifdef DEBUG_REDUCE - printf("FP2 Product too large - reducing it\n"); -#endif - if (x->a.XES > 1) FP_BLS12381_reduce(&(x->a)); - if (x->b.XES > 1) FP_BLS12381_reduce(&(x->b)); - } - - BIG_384_58_mul(A, x->a.g, y->a.g); - BIG_384_58_mul(B, x->b.g, y->b.g); - - BIG_384_58_add(C, x->a.g, x->b.g); - BIG_384_58_norm(C); - BIG_384_58_add(D, y->a.g, y->b.g); - BIG_384_58_norm(D); - - BIG_384_58_mul(E, C, D); - BIG_384_58_dadd(F, A, B); - BIG_384_58_dsub(B, pR, B); // - - BIG_384_58_dadd(A, A, B); // Aa.g, A); - w->a.XES = 3; // may drift above 2p... - BIG_384_58_dnorm(E); - FP_BLS12381_mod(w->b.g, E); - w->b.XES = 2; - -} - -/* output FP2 in hex format [a,b] */ -/* SU= 16 */ -void FP2_BLS12381_output(FP2_BLS12381 *w) -{ - BIG_384_58 bx, by; - FP2_BLS12381_reduce(w); - FP_BLS12381_redc(bx, &(w->a)); - FP_BLS12381_redc(by, &(w->b)); - printf("["); - BIG_384_58_output(bx); - printf(","); - BIG_384_58_output(by); - printf("]"); - FP_BLS12381_nres(&(w->a), bx); - FP_BLS12381_nres(&(w->b), by); -} - -/* SU= 8 */ -void FP2_BLS12381_rawoutput(FP2_BLS12381 *w) -{ - printf("["); - BIG_384_58_rawoutput(w->a.g); - printf(","); - BIG_384_58_rawoutput(w->b.g); - printf("]"); -} - - -/* Set w=1/x */ -/* SU= 128 */ -void FP2_BLS12381_inv(FP2_BLS12381 *w, FP2_BLS12381 *x) -{ - BIG_384_58 m, b; - FP_BLS12381 w1, w2; - - FP2_BLS12381_norm(x); - FP_BLS12381_sqr(&w1, &(x->a)); - FP_BLS12381_sqr(&w2, &(x->b)); - FP_BLS12381_add(&w1, &w1, &w2); - - FP_BLS12381_inv(&w1, &w1, NULL); - - FP_BLS12381_mul(&(w->a), &(x->a), &w1); - FP_BLS12381_neg(&w1, &w1); - FP_BLS12381_norm(&w1); - FP_BLS12381_mul(&(w->b), &(x->b), &w1); -} - - -/* Set w=x/2 */ -/* SU= 16 */ -void FP2_BLS12381_div2(FP2_BLS12381 *w, FP2_BLS12381 *x) -{ - FP_BLS12381_div2(&(w->a), &(x->a)); - FP_BLS12381_div2(&(w->b), &(x->b)); -} - -/* Set w*=(1+sqrt(-1)) */ -/* where X^2-(1+sqrt(-1)) is irreducible for FP4, assumes p=3 mod 8 */ - -/* Input MUST be normed */ -void FP2_BLS12381_mul_ip(FP2_BLS12381 *w) -{ - FP2_BLS12381 t; - - int i = QNRI_BLS12381; - - FP2_BLS12381_copy(&t, w); - FP2_BLS12381_times_i(w); - -// add 2^i.t - while (i > 0) - { - FP2_BLS12381_add(&t, &t, &t); - FP2_BLS12381_norm(&t); - i--; - } - FP2_BLS12381_add(w, &t, w); - -#if TOWER_BLS12381 == POSITOWER - FP2_BLS12381_norm(w); - FP2_BLS12381_neg(w, w); // *** -#endif -// Output NOT normed, so use with care -} - -/* Set w/=(1+sqrt(-1)) */ -/* SU= 88 */ -void FP2_BLS12381_div_ip(FP2_BLS12381 *w) -{ - FP2_BLS12381 z; - FP2_BLS12381_norm(w); - FP2_BLS12381_from_ints(&z, (1 << QNRI_BLS12381), 1); - FP2_BLS12381_inv(&z, &z); - FP2_BLS12381_mul(w, &z, w); -#if TOWER_BLS12381 == POSITOWER - FP2_BLS12381_neg(w, w); // *** -#endif -} - -/* SU= 8 */ -/* normalise a and b components of w */ -void FP2_BLS12381_norm(FP2_BLS12381 *w) -{ - FP_BLS12381_norm(&(w->a)); - FP_BLS12381_norm(&(w->b)); -} - -/* Set w=a^b mod m */ -/* SU= 208 */ -/* -void FP2_BLS12381_pow(FP2_BLS12381 *r, FP2_BLS12381* a, BIG_384_58 b) -{ - FP2_BLS12381 w; - FP_BLS12381 one; - BIG_384_58 z, zilch; - int bt; - - BIG_384_58_norm(b); - BIG_384_58_copy(z, b); - FP2_BLS12381_copy(&w, a); - FP_BLS12381_one(&one); - BIG_384_58_zero(zilch); - FP2_BLS12381_from_FP(r, &one); - while (1) - { - bt = BIG_384_58_parity(z); - BIG_384_58_shr(z, 1); - if (bt) FP2_BLS12381_mul(r, r, &w); - if (BIG_384_58_comp(z, zilch) == 0) break; - FP2_BLS12381_sqr(&w, &w); - } - FP2_BLS12381_reduce(r); -} -*/ -/* test for x a QR */ - -int FP2_BLS12381_qr(FP2_BLS12381 *x) -{ /* test x^(p^2-1)/2 = 1 */ - FP2_BLS12381 c; - FP2_BLS12381_conj(&c,x); - FP2_BLS12381_mul(&c,&c,x); - - return FP_BLS12381_qr(&(c.a),NULL); -} - -/* sqrt(a+ib) = sqrt(a+sqrt(a*a-n*b*b)/2)+ib/(2*sqrt(a+sqrt(a*a-n*b*b)/2)) */ - -void FP2_BLS12381_sqrt(FP2_BLS12381 *w, FP2_BLS12381 *u) -{ - FP_BLS12381 w1, w2, w3; - FP2_BLS12381 nw; - int sgn; - - FP2_BLS12381_copy(w, u); - if (FP2_BLS12381_iszilch(w)) return; - - FP_BLS12381_sqr(&w1, &(w->b)); - FP_BLS12381_sqr(&w2, &(w->a)); - FP_BLS12381_add(&w1, &w1, &w2); - FP_BLS12381_norm(&w1); - FP_BLS12381_sqrt(&w1, &w1,NULL); - - FP_BLS12381_add(&w2, &(w->a), &w1); - FP_BLS12381_norm(&w2); - FP_BLS12381_div2(&w2, &w2); - - FP_BLS12381_sub(&w3, &(w->a), &w1); - FP_BLS12381_norm(&w3); - FP_BLS12381_div2(&w3, &w3); - - FP_BLS12381_cmove(&w2,&w3,FP_BLS12381_qr(&w3,NULL)); // one or the other will be a QR - - FP_BLS12381_invsqrt(&w3,&(w->a),&w2); - FP_BLS12381_mul(&w3,&w3,&(w->a)); - FP_BLS12381_div2(&w2,&w3); -/* - - FP_BLS12381_sqrt(&w2, &w2,NULL); - FP_BLS12381_copy(&(w->a), &w2); - FP_BLS12381_add(&w2, &w2, &w2); - FP_BLS12381_norm(&w2); - FP_BLS12381_inv(&w2, &w2, NULL); -*/ - FP_BLS12381_mul(&(w->b), &(w->b), &w2); - - sgn=FP2_BLS12381_sign(w); - FP2_BLS12381_neg(&nw,w); FP2_BLS12381_norm(&nw); - FP2_BLS12381_cmove(w,&nw,sgn); -} - -/* New stuff for ECp4 support */ - -/* Input MUST be normed */ -void FP2_BLS12381_times_i(FP2_BLS12381 *w) -{ - FP_BLS12381 z; - FP_BLS12381_copy(&z, &(w->a)); - FP_BLS12381_neg(&(w->a), &(w->b)); - FP_BLS12381_copy(&(w->b), &z); - -// Output NOT normed, so use with care -} - -void FP2_BLS12381_rand(FP2_BLS12381 *x,csprng *rng) -{ - FP_BLS12381_rand(&(x->a),rng); - FP_BLS12381_rand(&(x->b),rng); -} - diff --git a/impl/cbits/fp2_BLS12381.h b/impl/cbits/fp2_BLS12381.h deleted file mode 100644 index 8664d3373..000000000 --- a/impl/cbits/fp2_BLS12381.h +++ /dev/null @@ -1,281 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file fp2.h - * @author Mike Scott - * @brief FP2 Header File - * - */ - -#ifndef FP2_BLS12381_H -#define FP2_BLS12381_H - -#include "fp_BLS12381.h" - -/** - @brief FP2 Structure - quadratic extension field -*/ - -typedef struct -{ - FP_BLS12381 a; /**< real part of FP2 */ - FP_BLS12381 b; /**< imaginary part of FP2 */ -} FP2_BLS12381; - -/* FP2 prototypes */ - -/** @brief Tests for FP2 equal to zero - * - @param x FP2 number to be tested - @return 1 if zero, else returns 0 - */ -extern int FP2_BLS12381_iszilch(FP2_BLS12381 *x); -/** @brief Conditional copy of FP2 number - * - Conditionally copies second parameter to the first (without branching) - @param x FP2 instance, set to y if s!=0 - @param y another FP2 instance - @param s copy only takes place if not equal to 0 - */ -extern void FP2_BLS12381_cmove(FP2_BLS12381 *x, FP2_BLS12381 *y, int s); -/** @brief Tests for FP2 equal to one - * - @param x FP2 instance to be tested - @return 1 if x=1, else returns 0 - */ -extern int FP2_BLS12381_isunity(FP2_BLS12381 *x); -/** @brief Tests for equality of two FP2s - * - @param x FP2 instance to be compared - @param y FP2 instance to be compared - @return 1 if x=y, else returns 0 - */ -extern int FP2_BLS12381_equals(FP2_BLS12381 *x, FP2_BLS12381 *y); -/** @brief Initialise FP2 from two FP numbers - * - @param x FP2 instance to be initialised - @param a FP to form real part of FP2 - @param b FP to form imaginary part of FP2 - */ -extern void FP2_BLS12381_from_FPs(FP2_BLS12381 *x, FP_BLS12381 *a, FP_BLS12381 *b); -/** @brief Initialise FP2 from two BIG integers - * - @param x FP2 instance to be initialised - @param a BIG to form real part of FP2 - @param b BIG to form imaginary part of FP2 - */ -extern void FP2_BLS12381_from_BIGs(FP2_BLS12381 *x, BIG_384_58 a, BIG_384_58 b); - - -/** @brief Initialise FP2 from two integers - * - @param x FP2 instance to be initialised - @param a int to form real part of FP2 - @param b int to form imaginary part of FP2 - */ -extern void FP2_BLS12381_from_ints(FP2_BLS12381 *x, int a, int b); - - - -/** @brief Initialise FP2 from single FP - * - Imaginary part is set to zero - @param x FP2 instance to be initialised - @param a FP to form real part of FP2 - */ -extern void FP2_BLS12381_from_FP(FP2_BLS12381 *x, FP_BLS12381 *a); -/** @brief Initialise FP2 from single BIG - * - Imaginary part is set to zero - @param x FP2 instance to be initialised - @param a BIG to form real part of FP2 - */ -extern void FP2_BLS12381_from_BIG(FP2_BLS12381 *x, BIG_384_58 a); -/** @brief Copy FP2 to another FP2 - * - @param x FP2 instance, on exit = y - @param y FP2 instance to be copied - */ -extern void FP2_BLS12381_copy(FP2_BLS12381 *x, FP2_BLS12381 *y); -/** @brief Set FP2 to zero - * - @param x FP2 instance to be set to zero - */ -extern void FP2_BLS12381_zero(FP2_BLS12381 *x); -/** @brief Set FP2 to unity - * - @param x FP2 instance to be set to one - */ -extern void FP2_BLS12381_one(FP2_BLS12381 *x); - -/** @brief Copy from ROM to an FP2 - * - @param w FP2 number to be copied to - @param a BIG real part to be copied from ROM - @param b BIG imag part to be copied from ROM - */ -extern void FP2_BLS12381_rcopy(FP2_BLS12381 *w,const BIG_384_58 a,const BIG_384_58 b); - -/** @brief Sign of FP2 - * - @param x FP2 instance - @return "sign" of FP2 - */ -extern int FP2_BLS12381_sign(FP2_BLS12381 *x); - -/** @brief Negation of FP2 - * - @param x FP2 instance, on exit = -y - @param y FP2 instance - */ -extern void FP2_BLS12381_neg(FP2_BLS12381 *x, FP2_BLS12381 *y); -/** @brief Conjugation of FP2 - * - If y=(a,b) on exit x=(a,-b) - @param x FP2 instance, on exit = conj(y) - @param y FP2 instance - */ -extern void FP2_BLS12381_conj(FP2_BLS12381 *x, FP2_BLS12381 *y); -/** @brief addition of two FP2s - * - @param x FP2 instance, on exit = y+z - @param y FP2 instance - @param z FP2 instance - */ -extern void FP2_BLS12381_add(FP2_BLS12381 *x, FP2_BLS12381 *y, FP2_BLS12381 *z); -/** @brief subtraction of two FP2s - * - @param x FP2 instance, on exit = y-z - @param y FP2 instance - @param z FP2 instance - */ -extern void FP2_BLS12381_sub(FP2_BLS12381 *x, FP2_BLS12381 *y, FP2_BLS12381 *z); -/** @brief Multiplication of an FP2 by an FP - * - @param x FP2 instance, on exit = y*b - @param y FP2 instance - @param b FP residue - */ -extern void FP2_BLS12381_pmul(FP2_BLS12381 *x, FP2_BLS12381 *y, FP_BLS12381 *b); -/** @brief Multiplication of an FP2 by a small integer - * - @param x FP2 instance, on exit = y*i - @param y FP2 instance - @param i an integer - */ -extern void FP2_BLS12381_imul(FP2_BLS12381 *x, FP2_BLS12381 *y, int i); -/** @brief Squaring an FP2 - * - @param x FP2 instance, on exit = y^2 - @param y FP2 instance - */ -extern void FP2_BLS12381_sqr(FP2_BLS12381 *x, FP2_BLS12381 *y); -/** @brief Multiplication of two FP2s - * - @param x FP2 instance, on exit = y*z - @param y FP2 instance - @param z FP2 instance - */ -extern void FP2_BLS12381_mul(FP2_BLS12381 *x, FP2_BLS12381 *y, FP2_BLS12381 *z); -/** @brief Formats and outputs an FP2 to the console - * - @param x FP2 instance - */ -extern void FP2_BLS12381_output(FP2_BLS12381 *x); -/** @brief Formats and outputs an FP2 to the console in raw form (for debugging) - * - @param x FP2 instance - */ -extern void FP2_BLS12381_rawoutput(FP2_BLS12381 *x); -/** @brief Inverting an FP2 - * - @param x FP2 instance, on exit = 1/y - @param y FP2 instance - */ -extern void FP2_BLS12381_inv(FP2_BLS12381 *x, FP2_BLS12381 *y); -/** @brief Divide an FP2 by 2 - * - @param x FP2 instance, on exit = y/2 - @param y FP2 instance - */ -extern void FP2_BLS12381_div2(FP2_BLS12381 *x, FP2_BLS12381 *y); -/** @brief Multiply an FP2 by (1+sqrt(-1)) - * - Note that (1+sqrt(-1)) is irreducible for FP4 - @param x FP2 instance, on exit = x*(1+sqrt(-1)) - */ -extern void FP2_BLS12381_mul_ip(FP2_BLS12381 *x); -/** @brief Divide an FP2 by (1+sqrt(-1))/2 - - * - Note that (1+sqrt(-1)) is irreducible for FP4 - @param x FP2 instance, on exit = 2x/(1+sqrt(-1)) - */ -extern void FP2_BLS12381_div_ip2(FP2_BLS12381 *x); -/** @brief Divide an FP2 by (1+sqrt(-1)) - * - Note that (1+sqrt(-1)) is irreducible for FP4 - @param x FP2 instance, on exit = x/(1+sqrt(-1)) - */ -extern void FP2_BLS12381_div_ip(FP2_BLS12381 *x); -/** @brief Normalises the components of an FP2 - * - @param x FP2 instance to be normalised - */ -extern void FP2_BLS12381_norm(FP2_BLS12381 *x); -/** @brief Reduces all components of possibly unreduced FP2 mod Modulus - * - @param x FP2 instance, on exit reduced mod Modulus - */ -extern void FP2_BLS12381_reduce(FP2_BLS12381 *x); -/** @brief Raises an FP2 to the power of a BIG - * - @param x FP2 instance, on exit = y^b - @param y FP2 instance - @param b BIG number - */ -extern void FP2_BLS12381_pow(FP2_BLS12381 *x, FP2_BLS12381 *y, BIG_384_58 b); - -/** @brief Test FP2 for QR - * - @param x FP2 instance - @return true or false - */ -extern int FP2_BLS12381_qr(FP2_BLS12381 *x); - -/** @brief Square root of an FP2 - * - @param x FP2 instance, on exit = sqrt(y) - @param y FP2 instance - */ -extern void FP2_BLS12381_sqrt(FP2_BLS12381 *x, FP2_BLS12381 *y); - -/** @brief Multiply an FP2 by sqrt(-1) - * - Note that -1 is QNR - @param x FP2 instance, on exit = x*sqrt(-1) - */ -extern void FP2_BLS12381_times_i(FP2_BLS12381 *x); -/** @brief Generate random FP2 - * - @param x random FP2 number - @param rng random number generator - */ -extern void FP2_BLS12381_rand(FP2_BLS12381 *x, csprng *rng); -#endif diff --git a/impl/cbits/fp4_BLS12381.c b/impl/cbits/fp4_BLS12381.c deleted file mode 100644 index ec06546c2..000000000 --- a/impl/cbits/fp4_BLS12381.c +++ /dev/null @@ -1,698 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/* CORE Fp^4 functions */ -/* SU=m, m is Stack Usage (no lazy )*/ - -/* FP4 elements are of the form a+ib, where i is sqrt(-1+sqrt(-1)) */ - -#include "fp4_BLS12381.h" - -/* test x==0 ? */ -/* SU= 8 */ -int FP4_BLS12381_iszilch(FP4_BLS12381 *x) -{ - return (FP2_BLS12381_iszilch(&(x->a)) & FP2_BLS12381_iszilch(&(x->b))); -} - -/* test x==1 ? */ -/* SU= 8 */ -int FP4_BLS12381_isunity(FP4_BLS12381 *x) -{ - return (FP2_BLS12381_isunity(&(x->a)) & FP2_BLS12381_iszilch(&(x->b))); -} - -/* test is w real? That is in a+ib test b is zero */ -int FP4_BLS12381_isreal(FP4_BLS12381 *w) -{ - return FP2_BLS12381_iszilch(&(w->b)); -} - -/* return 1 if x==y, else 0 */ -/* SU= 16 */ -int FP4_BLS12381_equals(FP4_BLS12381 *x, FP4_BLS12381 *y) -{ - return (FP2_BLS12381_equals(&(x->a), &(y->a)) & FP2_BLS12381_equals(&(x->b), &(y->b))); -} - -/* set FP4 from two FP2s */ -/* SU= 16 */ -void FP4_BLS12381_from_FP2s(FP4_BLS12381 *w, FP2_BLS12381 * x, FP2_BLS12381* y) -{ - FP2_BLS12381_copy(&(w->a), x); - FP2_BLS12381_copy(&(w->b), y); -} - -/* set FP4 from FP2 */ -/* SU= 8 */ -void FP4_BLS12381_from_FP2(FP4_BLS12381 *w, FP2_BLS12381 *x) -{ - FP2_BLS12381_copy(&(w->a), x); - FP2_BLS12381_zero(&(w->b)); -} - -/* set high part of FP4 from FP2 */ -/* SU= 8 */ -void FP4_BLS12381_from_FP2H(FP4_BLS12381 *w, FP2_BLS12381 *x) -{ - FP2_BLS12381_copy(&(w->b), x); - FP2_BLS12381_zero(&(w->a)); -} - -/* set FP4 from FP */ -void FP4_BLS12381_from_FP(FP4_BLS12381 *w, FP_BLS12381 *x) -{ - FP2_BLS12381 t; - FP2_BLS12381_from_FP(&t, x); - FP4_BLS12381_from_FP2(w, &t); -} - -/* FP4 copy w=x */ -/* SU= 16 */ -void FP4_BLS12381_copy(FP4_BLS12381 *w, FP4_BLS12381 *x) -{ - if (w == x) return; - FP2_BLS12381_copy(&(w->a), &(x->a)); - FP2_BLS12381_copy(&(w->b), &(x->b)); -} - -/* FP4 w=0 */ -/* SU= 8 */ -void FP4_BLS12381_zero(FP4_BLS12381 *w) -{ - FP2_BLS12381_zero(&(w->a)); - FP2_BLS12381_zero(&(w->b)); -} - -/* FP4 w=1 */ -/* SU= 8 */ -void FP4_BLS12381_one(FP4_BLS12381 *w) -{ - FP2_BLS12381_one(&(w->a)); - FP2_BLS12381_zero(&(w->b)); -} - -int FP4_BLS12381_sign(FP4_BLS12381 *w) -{ - int p1,p2; - p1=FP2_BLS12381_sign(&(w->a)); - p2=FP2_BLS12381_sign(&(w->b)); -#ifdef BIG_ENDIAN_SIGN_BLS12381 - p2 ^= (p1 ^ p2)&FP2_BLS12381_iszilch(&(w->b)); - return p2; -#else - p1 ^= (p1 ^ p2)&FP2_BLS12381_iszilch(&(w->a)); - return p1; -#endif -} - -/* Set w=-x */ -/* SU= 160 */ -void FP4_BLS12381_neg(FP4_BLS12381 *w, FP4_BLS12381 *x) -{ - /* Just one field neg */ - FP2_BLS12381 m, t; - FP4_BLS12381_norm(x); - FP2_BLS12381_add(&m, &(x->a), &(x->b)); - FP2_BLS12381_neg(&m, &m); - FP2_BLS12381_add(&t, &m, &(x->b)); - FP2_BLS12381_add(&(w->b), &m, &(x->a)); - FP2_BLS12381_copy(&(w->a), &t); - FP4_BLS12381_norm(w); -} - -/* Set w=conj(x) */ -/* SU= 16 */ -void FP4_BLS12381_conj(FP4_BLS12381 *w, FP4_BLS12381 *x) -{ - FP2_BLS12381_copy(&(w->a), &(x->a)); - FP2_BLS12381_neg(&(w->b), &(x->b)); - FP4_BLS12381_norm(w); -} - -/* Set w=-conj(x) */ -/* SU= 16 */ -void FP4_BLS12381_nconj(FP4_BLS12381 *w, FP4_BLS12381 *x) -{ - FP2_BLS12381_copy(&(w->b), &(x->b)); - FP2_BLS12381_neg(&(w->a), &(x->a)); - FP4_BLS12381_norm(w); -} - -/* Set w=x+y */ -/* SU= 16 */ -void FP4_BLS12381_add(FP4_BLS12381 *w, FP4_BLS12381 *x, FP4_BLS12381 *y) -{ - FP2_BLS12381_add(&(w->a), &(x->a), &(y->a)); - FP2_BLS12381_add(&(w->b), &(x->b), &(y->b)); -} - -/* Set w=x-y */ -/* Input y MUST be normed */ -void FP4_BLS12381_sub(FP4_BLS12381 *w, FP4_BLS12381 *x, FP4_BLS12381 *y) -{ - FP4_BLS12381 my; - FP4_BLS12381_neg(&my, y); - FP4_BLS12381_add(w, x, &my); -} -/* SU= 8 */ -/* reduce all components of w mod Modulus */ -void FP4_BLS12381_reduce(FP4_BLS12381 *w) -{ - FP2_BLS12381_reduce(&(w->a)); - FP2_BLS12381_reduce(&(w->b)); -} - -/* SU= 8 */ -/* normalise all elements of w */ -void FP4_BLS12381_norm(FP4_BLS12381 *w) -{ - FP2_BLS12381_norm(&(w->a)); - FP2_BLS12381_norm(&(w->b)); -} - -/* Set w=s*x, where s is FP2 */ -/* SU= 16 */ -void FP4_BLS12381_pmul(FP4_BLS12381 *w, FP4_BLS12381 *x, FP2_BLS12381 *s) -{ - FP2_BLS12381_mul(&(w->a), &(x->a), s); - FP2_BLS12381_mul(&(w->b), &(x->b), s); -} - -/* Set w=s*x, where s is FP */ -void FP4_BLS12381_qmul(FP4_BLS12381 *w, FP4_BLS12381 *x, FP_BLS12381 *s) -{ - FP2_BLS12381_pmul(&(w->a), &(x->a), s); - FP2_BLS12381_pmul(&(w->b), &(x->b), s); -} - -/* SU= 16 */ -/* Set w=s*x, where s is int */ -void FP4_BLS12381_imul(FP4_BLS12381 *w, FP4_BLS12381 *x, int s) -{ - FP2_BLS12381_imul(&(w->a), &(x->a), s); - FP2_BLS12381_imul(&(w->b), &(x->b), s); -} - -/* Set w=x^2 */ -/* Input MUST be normed */ -void FP4_BLS12381_sqr(FP4_BLS12381 *w, FP4_BLS12381 *x) -{ - FP2_BLS12381 t1, t2, t3; - - FP2_BLS12381_mul(&t3, &(x->a), &(x->b)); /* norms x */ - FP2_BLS12381_copy(&t2, &(x->b)); - FP2_BLS12381_add(&t1, &(x->a), &(x->b)); - FP2_BLS12381_mul_ip(&t2); - - FP2_BLS12381_add(&t2, &(x->a), &t2); - - FP2_BLS12381_norm(&t1); // 2 - FP2_BLS12381_norm(&t2); // 2 - - FP2_BLS12381_mul(&(w->a), &t1, &t2); - - FP2_BLS12381_copy(&t2, &t3); - FP2_BLS12381_mul_ip(&t2); - - FP2_BLS12381_add(&t2, &t2, &t3); - - FP2_BLS12381_norm(&t2); // 2 - FP2_BLS12381_neg(&t2, &t2); - FP2_BLS12381_add(&(w->a), &(w->a), &t2); /* a=(a+b)(a+i^2.b)-i^2.ab-ab = a*a+ib*ib */ - FP2_BLS12381_add(&(w->b), &t3, &t3); /* b=2ab */ - - FP4_BLS12381_norm(w); -} - -/* Set w=x*y */ -/* Inputs MUST be normed */ -void FP4_BLS12381_mul(FP4_BLS12381 *w, FP4_BLS12381 *x, FP4_BLS12381 *y) -{ - - FP2_BLS12381 t1, t2, t3, t4; - FP2_BLS12381_mul(&t1, &(x->a), &(y->a)); - FP2_BLS12381_mul(&t2, &(x->b), &(y->b)); - - FP2_BLS12381_add(&t3, &(y->b), &(y->a)); - FP2_BLS12381_add(&t4, &(x->b), &(x->a)); - - FP2_BLS12381_norm(&t4); // 2 - FP2_BLS12381_norm(&t3); // 2 - - FP2_BLS12381_mul(&t4, &t4, &t3); /* (xa+xb)(ya+yb) */ - - FP2_BLS12381_neg(&t3, &t1); // 1 - FP2_BLS12381_add(&t4, &t4, &t3); //t4E=3 - FP2_BLS12381_norm(&t4); - - FP2_BLS12381_neg(&t3, &t2); // 1 - FP2_BLS12381_add(&(w->b), &t4, &t3); //wbE=3 - - FP2_BLS12381_mul_ip(&t2); - FP2_BLS12381_add(&(w->a), &t2, &t1); - - FP4_BLS12381_norm(w); -} - -/* output FP4 in format [a,b] */ -/* SU= 8 */ -void FP4_BLS12381_output(FP4_BLS12381 *w) -{ - printf("["); - FP2_BLS12381_output(&(w->a)); - printf(","); - FP2_BLS12381_output(&(w->b)); - printf("]"); -} - -/* SU= 8 */ -void FP4_BLS12381_rawoutput(FP4_BLS12381 *w) -{ - printf("["); - FP2_BLS12381_rawoutput(&(w->a)); - printf(","); - FP2_BLS12381_rawoutput(&(w->b)); - printf("]"); -} - -/* Set w=1/x */ -/* SU= 160 */ -void FP4_BLS12381_inv(FP4_BLS12381 *w, FP4_BLS12381 *x) -{ - FP2_BLS12381 t1, t2; - FP2_BLS12381_sqr(&t1, &(x->a)); - FP2_BLS12381_sqr(&t2, &(x->b)); - FP2_BLS12381_mul_ip(&t2); - FP2_BLS12381_norm(&t2); - FP2_BLS12381_sub(&t1, &t1, &t2); - FP2_BLS12381_inv(&t1, &t1); - FP2_BLS12381_mul(&(w->a), &t1, &(x->a)); - FP2_BLS12381_neg(&t1, &t1); - FP2_BLS12381_norm(&t1); - FP2_BLS12381_mul(&(w->b), &t1, &(x->b)); -} - -/* w*=i where i = sqrt(1+sqrt(-1)) */ -/* SU= 200 */ -void FP4_BLS12381_times_i(FP4_BLS12381 *w) -{ - FP2_BLS12381 t; - FP2_BLS12381_copy(&t, &(w->b)); - FP2_BLS12381_copy(&(w->b), &(w->a)); - FP2_BLS12381_mul_ip(&t); - FP2_BLS12381_copy(&(w->a), &t); - FP4_BLS12381_norm(w); -#if TOWER_BLS12381 == POSITOWER - FP4_BLS12381_neg(w, w); // *** - FP4_BLS12381_norm(w); -#endif -} - -/* Set w=w^p using Frobenius */ -/* SU= 16 */ -void FP4_BLS12381_frob(FP4_BLS12381 *w, FP2_BLS12381 *f) -{ - FP2_BLS12381_conj(&(w->a), &(w->a)); - FP2_BLS12381_conj(&(w->b), &(w->b)); - FP2_BLS12381_mul( &(w->b), f, &(w->b)); -} - -/* Set r=a^b mod m */ -/* SU= 240 */ -/* -void FP4_BLS12381_pow(FP4_BLS12381 *r, FP4_BLS12381* a, BIG_384_58 b) -{ - FP4_BLS12381 w; - BIG_384_58 z, zilch; - int bt; - - BIG_384_58_zero(zilch); - - BIG_384_58_copy(z, b); - BIG_384_58_norm(z); - FP4_BLS12381_copy(&w, a); - FP4_BLS12381_norm(&w); - FP4_BLS12381_one(r); - - while (1) - { - bt = BIG_384_58_parity(z); - BIG_384_58_shr(z, 1); - if (bt) FP4_BLS12381_mul(r, r, &w); - if (BIG_384_58_comp(z, zilch) == 0) break; - FP4_BLS12381_sqr(&w, &w); - } - FP4_BLS12381_reduce(r); -} */ - -/* SU= 304 */ -/* XTR xtr_a function */ -void FP4_BLS12381_xtr_A(FP4_BLS12381 *r, FP4_BLS12381 *w, FP4_BLS12381 *x, FP4_BLS12381 *y, FP4_BLS12381 *z) -{ - FP4_BLS12381 t1, t2; - FP4_BLS12381_copy(r, x); - FP4_BLS12381_sub(&t1, w, y); - FP4_BLS12381_norm(&t1); - FP4_BLS12381_pmul(&t1, &t1, &(r->a)); - FP4_BLS12381_add(&t2, w, y); - FP4_BLS12381_norm(&t2); - FP4_BLS12381_pmul(&t2, &t2, &(r->b)); - FP4_BLS12381_times_i(&t2); - - FP4_BLS12381_add(r, &t1, &t2); - FP4_BLS12381_add(r, r, z); - - FP4_BLS12381_reduce(r); -} - -/* SU= 152 */ -/* XTR xtr_d function */ -void FP4_BLS12381_xtr_D(FP4_BLS12381 *r, FP4_BLS12381 *x) -{ - FP4_BLS12381 w; - FP4_BLS12381_copy(r, x); - FP4_BLS12381_conj(&w, r); - FP4_BLS12381_add(&w, &w, &w); - FP4_BLS12381_sqr(r, r); - FP4_BLS12381_norm(&w); - FP4_BLS12381_sub(r, r, &w); - FP4_BLS12381_reduce(r); /* reduce here as multiple calls trigger automatic reductions */ -} - -/* SU= 728 */ -/* r=x^n using XTR method on traces of FP12s */ -void FP4_BLS12381_xtr_pow(FP4_BLS12381 *r, FP4_BLS12381 *x, BIG_384_58 n) -{ - int i, par, nb; - BIG_384_58 v; - FP2_BLS12381 w; - FP4_BLS12381 t, a, b, c, sf; - - BIG_384_58_zero(v); - BIG_384_58_inc(v, 3); - BIG_384_58_norm(v); - FP2_BLS12381_from_BIG(&w, v); - FP4_BLS12381_from_FP2(&a, &w); - - FP4_BLS12381_copy(&sf, x); - FP4_BLS12381_norm(&sf); - FP4_BLS12381_copy(&b, &sf); - FP4_BLS12381_xtr_D(&c, &sf); - - par = BIG_384_58_parity(n); - BIG_384_58_copy(v, n); - BIG_384_58_norm(v); - BIG_384_58_shr(v, 1); - if (par == 0) - { - BIG_384_58_dec(v, 1); - BIG_384_58_norm(v); - } - - nb = BIG_384_58_nbits(v); - for (i = nb - 1; i >= 0; i--) - { - if (!BIG_384_58_bit(v, i)) - { - FP4_BLS12381_copy(&t, &b); - FP4_BLS12381_conj(&sf, &sf); - FP4_BLS12381_conj(&c, &c); - FP4_BLS12381_xtr_A(&b, &a, &b, &sf, &c); - FP4_BLS12381_conj(&sf, &sf); - FP4_BLS12381_xtr_D(&c, &t); - FP4_BLS12381_xtr_D(&a, &a); - } - else - { - FP4_BLS12381_conj(&t, &a); - FP4_BLS12381_xtr_D(&a, &b); - FP4_BLS12381_xtr_A(&b, &c, &b, &sf, &t); - FP4_BLS12381_xtr_D(&c, &c); - } - } - - if (par == 0) FP4_BLS12381_copy(r, &c); - else FP4_BLS12381_copy(r, &b); - FP4_BLS12381_reduce(r); -} - -/* SU= 872 */ -/* r=ck^a.cl^n using XTR double exponentiation method on traces of FP12s. See Stam thesis. */ -void FP4_BLS12381_xtr_pow2(FP4_BLS12381 *r, FP4_BLS12381 *ck, FP4_BLS12381 *cl, FP4_BLS12381 *ckml, FP4_BLS12381 *ckm2l, BIG_384_58 a, BIG_384_58 b) -{ - int i, f2; - BIG_384_58 d, e, w; - FP4_BLS12381 t, cu, cv, cumv, cum2v; - - - BIG_384_58_copy(e, a); - BIG_384_58_copy(d, b); - BIG_384_58_norm(e); - BIG_384_58_norm(d); - FP4_BLS12381_copy(&cu, ck); - FP4_BLS12381_copy(&cv, cl); - FP4_BLS12381_copy(&cumv, ckml); - FP4_BLS12381_copy(&cum2v, ckm2l); - - f2 = 0; - while (BIG_384_58_parity(d) == 0 && BIG_384_58_parity(e) == 0) - { - BIG_384_58_shr(d, 1); - BIG_384_58_shr(e, 1); - f2++; - } - while (BIG_384_58_comp(d, e) != 0) - { - if (BIG_384_58_comp(d, e) > 0) - { - BIG_384_58_imul(w, e, 4); - BIG_384_58_norm(w); - if (BIG_384_58_comp(d, w) <= 0) - { - BIG_384_58_copy(w, d); - BIG_384_58_copy(d, e); - BIG_384_58_sub(e, w, e); - BIG_384_58_norm(e); - FP4_BLS12381_xtr_A(&t, &cu, &cv, &cumv, &cum2v); - FP4_BLS12381_conj(&cum2v, &cumv); - FP4_BLS12381_copy(&cumv, &cv); - FP4_BLS12381_copy(&cv, &cu); - FP4_BLS12381_copy(&cu, &t); - } - else if (BIG_384_58_parity(d) == 0) - { - BIG_384_58_shr(d, 1); - FP4_BLS12381_conj(r, &cum2v); - FP4_BLS12381_xtr_A(&t, &cu, &cumv, &cv, r); - FP4_BLS12381_xtr_D(&cum2v, &cumv); - FP4_BLS12381_copy(&cumv, &t); - FP4_BLS12381_xtr_D(&cu, &cu); - } - else if (BIG_384_58_parity(e) == 1) - { - BIG_384_58_sub(d, d, e); - BIG_384_58_norm(d); - BIG_384_58_shr(d, 1); - FP4_BLS12381_xtr_A(&t, &cu, &cv, &cumv, &cum2v); - FP4_BLS12381_xtr_D(&cu, &cu); - FP4_BLS12381_xtr_D(&cum2v, &cv); - FP4_BLS12381_conj(&cum2v, &cum2v); - FP4_BLS12381_copy(&cv, &t); - } - else - { - BIG_384_58_copy(w, d); - BIG_384_58_copy(d, e); - BIG_384_58_shr(d, 1); - BIG_384_58_copy(e, w); - FP4_BLS12381_xtr_D(&t, &cumv); - FP4_BLS12381_conj(&cumv, &cum2v); - FP4_BLS12381_conj(&cum2v, &t); - FP4_BLS12381_xtr_D(&t, &cv); - FP4_BLS12381_copy(&cv, &cu); - FP4_BLS12381_copy(&cu, &t); - } - } - if (BIG_384_58_comp(d, e) < 0) - { - BIG_384_58_imul(w, d, 4); - BIG_384_58_norm(w); - if (BIG_384_58_comp(e, w) <= 0) - { - BIG_384_58_sub(e, e, d); - BIG_384_58_norm(e); - FP4_BLS12381_xtr_A(&t, &cu, &cv, &cumv, &cum2v); - FP4_BLS12381_copy(&cum2v, &cumv); - FP4_BLS12381_copy(&cumv, &cu); - FP4_BLS12381_copy(&cu, &t); - } - else if (BIG_384_58_parity(e) == 0) - { - BIG_384_58_copy(w, d); - BIG_384_58_copy(d, e); - BIG_384_58_shr(d, 1); - BIG_384_58_copy(e, w); - FP4_BLS12381_xtr_D(&t, &cumv); - FP4_BLS12381_conj(&cumv, &cum2v); - FP4_BLS12381_conj(&cum2v, &t); - FP4_BLS12381_xtr_D(&t, &cv); - FP4_BLS12381_copy(&cv, &cu); - FP4_BLS12381_copy(&cu, &t); - } - else if (BIG_384_58_parity(d) == 1) - { - BIG_384_58_copy(w, e); - BIG_384_58_copy(e, d); - BIG_384_58_sub(w, w, d); - BIG_384_58_norm(w); - BIG_384_58_copy(d, w); - BIG_384_58_shr(d, 1); - FP4_BLS12381_xtr_A(&t, &cu, &cv, &cumv, &cum2v); - FP4_BLS12381_conj(&cumv, &cumv); - FP4_BLS12381_xtr_D(&cum2v, &cu); - FP4_BLS12381_conj(&cum2v, &cum2v); - FP4_BLS12381_xtr_D(&cu, &cv); - FP4_BLS12381_copy(&cv, &t); - } - else - { - BIG_384_58_shr(d, 1); - FP4_BLS12381_conj(r, &cum2v); - FP4_BLS12381_xtr_A(&t, &cu, &cumv, &cv, r); - FP4_BLS12381_xtr_D(&cum2v, &cumv); - FP4_BLS12381_copy(&cumv, &t); - FP4_BLS12381_xtr_D(&cu, &cu); - } - } - } - FP4_BLS12381_xtr_A(r, &cu, &cv, &cumv, &cum2v); - for (i = 0; i < f2; i++) FP4_BLS12381_xtr_D(r, r); - FP4_BLS12381_xtr_pow(r, r, d); -} - -/* Move b to a if d=1 */ -void FP4_BLS12381_cmove(FP4_BLS12381 *f, FP4_BLS12381 *g, int d) -{ - FP2_BLS12381_cmove(&(f->a), &(g->a), d); - FP2_BLS12381_cmove(&(f->b), &(g->b), d); -} - -/* New stuff for ECp4 support */ - -/* Set w=x/2 */ -void FP4_BLS12381_div2(FP4_BLS12381 *w, FP4_BLS12381 *x) -{ - FP2_BLS12381_div2(&(w->a), &(x->a)); - FP2_BLS12381_div2(&(w->b), &(x->b)); -} - -void FP4_BLS12381_rand(FP4_BLS12381 *x,csprng *rng) -{ - FP2_BLS12381_rand(&(x->a),rng); - FP2_BLS12381_rand(&(x->b),rng); -} - -#if PAIRING_FRIENDLY_BLS12381 >= BLS24_CURVE - -/* test for x a QR */ -int FP4_BLS12381_qr(FP4_BLS12381 *x) -{ /* test x^(p^4-1)/2 = 1 */ - FP4_BLS12381 c; - FP4_BLS12381_conj(&c,x); - FP4_BLS12381_mul(&c,&c,x); - - return FP2_BLS12381_qr(&(c.a)); -} - -/* sqrt(a+xb) = sqrt((a+sqrt(a*a-n*b*b))/2)+x.b/(2*sqrt((a+sqrt(a*a-n*b*b))/2)) */ - -void FP4_BLS12381_sqrt(FP4_BLS12381 *r, FP4_BLS12381* x) -{ - FP2_BLS12381 a, b, s, t; - FP4_BLS12381 nr; - int sgn; - - FP4_BLS12381_copy(r, x); - if (FP4_BLS12381_iszilch(x)) return; - - FP2_BLS12381_copy(&a, &(x->a)); - FP2_BLS12381_copy(&s, &(x->b)); - - FP2_BLS12381_sqr(&s, &s); // s*=s - FP2_BLS12381_sqr(&a, &a); // a*=a - FP2_BLS12381_mul_ip(&s); - FP2_BLS12381_norm(&s); - FP2_BLS12381_sub(&a, &a, &s); // a-=txx(s) - FP2_BLS12381_norm(&a); - FP2_BLS12381_sqrt(&s, &a); - - FP2_BLS12381_copy(&t, &(x->a)); - - FP2_BLS12381_add(&a, &t, &s); - FP2_BLS12381_norm(&a); - FP2_BLS12381_div2(&a, &a); - - FP2_BLS12381_sub(&b, &t, &s); - FP2_BLS12381_norm(&b); - FP2_BLS12381_div2(&b, &b); - - FP2_BLS12381_cmove(&a,&b,FP2_BLS12381_qr(&b)); // one or the other will be a QR - - FP2_BLS12381_sqrt(&a, &a); - FP2_BLS12381_copy(&t, &(x->b)); - FP2_BLS12381_add(&s, &a, &a); - FP2_BLS12381_norm(&s); - FP2_BLS12381_inv(&s, &s); - - FP2_BLS12381_mul(&t, &t, &s); - FP4_BLS12381_from_FP2s(r, &a, &t); - - sgn=FP4_BLS12381_sign(r); - FP4_BLS12381_neg(&nr,r); FP4_BLS12381_norm(&nr); - FP4_BLS12381_cmove(r,&nr,sgn); -} - -void FP4_BLS12381_div_i(FP4_BLS12381 *f) -{ - FP2_BLS12381 u, v; - FP2_BLS12381_copy(&u, &(f->a)); - FP2_BLS12381_copy(&v, &(f->b)); - FP2_BLS12381_div_ip(&u); - FP2_BLS12381_copy(&(f->a), &v); - FP2_BLS12381_copy(&(f->b), &u); -#if TOWER_BLS12381 == POSITOWER - FP4_BLS12381_neg(f, f); // *** - FP4_BLS12381_norm(f); -#endif -} -/* -void FP4_BLS12381_div_2i(FP4_BLS12381 *f) -{ - FP2_BLS12381 u,v; - FP2_BLS12381_copy(&u,&(f->a)); - FP2_BLS12381_copy(&v,&(f->b)); - FP2_BLS12381_div_ip2(&u); - FP2_BLS12381_add(&v,&v,&v); - FP2_BLS12381_norm(&v); - FP2_BLS12381_copy(&(f->a),&v); - FP2_BLS12381_copy(&(f->b),&u); -} -*/ -#endif diff --git a/impl/cbits/fp4_BLS12381.h b/impl/cbits/fp4_BLS12381.h deleted file mode 100644 index 8b20935f2..000000000 --- a/impl/cbits/fp4_BLS12381.h +++ /dev/null @@ -1,332 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file fp4.h - * @author Mike Scott - * @brief FP4 Header File - * - */ - -#ifndef FP4_BLS12381_H -#define FP4_BLS12381_H - -#include "fp2_BLS12381.h" -#include "config_curve_BLS12381.h" - -/** - @brief FP4 Structure - towered over two FP2 -*/ - -typedef struct -{ - FP2_BLS12381 a; /**< real part of FP4 */ - FP2_BLS12381 b; /**< imaginary part of FP4 */ -} FP4_BLS12381; - - -/* FP4 prototypes */ -/** @brief Tests for FP4 equal to zero - * - @param x FP4 number to be tested - @return 1 if zero, else returns 0 - */ -extern int FP4_BLS12381_iszilch(FP4_BLS12381 *x); -/** @brief Tests for FP4 equal to unity - * - @param x FP4 number to be tested - @return 1 if unity, else returns 0 - */ -extern int FP4_BLS12381_isunity(FP4_BLS12381 *x); -/** @brief Tests for equality of two FP4s - * - @param x FP4 instance to be compared - @param y FP4 instance to be compared - @return 1 if x=y, else returns 0 - */ -extern int FP4_BLS12381_equals(FP4_BLS12381 *x, FP4_BLS12381 *y); -/** @brief Tests for FP4 having only a real part and no imaginary part - * - @param x FP4 number to be tested - @return 1 if real, else returns 0 - */ -extern int FP4_BLS12381_isreal(FP4_BLS12381 *x); -/** @brief Initialise FP4 from two FP2s - * - @param x FP4 instance to be initialised - @param a FP2 to form real part of FP4 - @param b FP2 to form imaginary part of FP4 - */ -extern void FP4_BLS12381_from_FP2s(FP4_BLS12381 *x, FP2_BLS12381 *a, FP2_BLS12381 *b); -/** @brief Initialise FP4 from single FP2 - * - Imaginary part is set to zero - @param x FP4 instance to be initialised - @param a FP2 to form real part of FP4 - */ -extern void FP4_BLS12381_from_FP2(FP4_BLS12381 *x, FP2_BLS12381 *a); - -/** @brief Initialise FP4 from single FP2 - * - real part is set to zero - @param x FP4 instance to be initialised - @param a FP2 to form imaginary part of FP4 - */ -extern void FP4_BLS12381_from_FP2H(FP4_BLS12381 *x, FP2_BLS12381 *a); - -/** @brief Initialise FP4 from single FP - * - @param x FP4 instance to be initialised - @param a FP to form real part of FP4 - */ -extern void FP4_BLS12381_from_FP(FP4_BLS12381 *x, FP_BLS12381 *a); - -/** @brief Copy FP4 to another FP4 - * - @param x FP4 instance, on exit = y - @param y FP4 instance to be copied - */ -extern void FP4_BLS12381_copy(FP4_BLS12381 *x, FP4_BLS12381 *y); -/** @brief Set FP4 to zero - * - @param x FP4 instance to be set to zero - */ -extern void FP4_BLS12381_zero(FP4_BLS12381 *x); -/** @brief Set FP4 to unity - * - @param x FP4 instance to be set to one - */ -extern void FP4_BLS12381_one(FP4_BLS12381 *x); - -/** @brief Sign of FP4 - * - @param x FP4 instance - @return "sign" of FP4 - */ -extern int FP4_BLS12381_sign(FP4_BLS12381 *x); - -/** @brief Negation of FP4 - * - @param x FP4 instance, on exit = -y - @param y FP4 instance - */ -extern void FP4_BLS12381_neg(FP4_BLS12381 *x, FP4_BLS12381 *y); -/** @brief Conjugation of FP4 - * - If y=(a,b) on exit x=(a,-b) - @param x FP4 instance, on exit = conj(y) - @param y FP4 instance - */ -extern void FP4_BLS12381_conj(FP4_BLS12381 *x, FP4_BLS12381 *y); -/** @brief Negative conjugation of FP4 - * - If y=(a,b) on exit x=(-a,b) - @param x FP4 instance, on exit = -conj(y) - @param y FP4 instance - */ -extern void FP4_BLS12381_nconj(FP4_BLS12381 *x, FP4_BLS12381 *y); -/** @brief addition of two FP4s - * - @param x FP4 instance, on exit = y+z - @param y FP4 instance - @param z FP4 instance - */ -extern void FP4_BLS12381_add(FP4_BLS12381 *x, FP4_BLS12381 *y, FP4_BLS12381 *z); -/** @brief subtraction of two FP4s - * - @param x FP4 instance, on exit = y-z - @param y FP4 instance - @param z FP4 instance - */ -extern void FP4_BLS12381_sub(FP4_BLS12381 *x, FP4_BLS12381 *y, FP4_BLS12381 *z); -/** @brief Multiplication of an FP4 by an FP2 - * - @param x FP4 instance, on exit = y*a - @param y FP4 instance - @param a FP2 multiplier - */ -extern void FP4_BLS12381_pmul(FP4_BLS12381 *x, FP4_BLS12381 *y, FP2_BLS12381 *a); - -/** @brief Multiplication of an FP4 by an FP - * - @param x FP4 instance, on exit = y*a - @param y FP4 instance - @param a FP multiplier - */ -extern void FP4_BLS12381_qmul(FP4_BLS12381 *x, FP4_BLS12381 *y, FP_BLS12381 *a); - -/** @brief Multiplication of an FP4 by a small integer - * - @param x FP4 instance, on exit = y*i - @param y FP4 instance - @param i an integer - */ -extern void FP4_BLS12381_imul(FP4_BLS12381 *x, FP4_BLS12381 *y, int i); -/** @brief Squaring an FP4 - * - @param x FP4 instance, on exit = y^2 - @param y FP4 instance - */ -extern void FP4_BLS12381_sqr(FP4_BLS12381 *x, FP4_BLS12381 *y); -/** @brief Multiplication of two FP4s - * - @param x FP4 instance, on exit = y*z - @param y FP4 instance - @param z FP4 instance - */ -extern void FP4_BLS12381_mul(FP4_BLS12381 *x, FP4_BLS12381 *y, FP4_BLS12381 *z); -/** @brief Inverting an FP4 - * - @param x FP4 instance, on exit = 1/y - @param y FP4 instance - */ -extern void FP4_BLS12381_inv(FP4_BLS12381 *x, FP4_BLS12381 *y); -/** @brief Formats and outputs an FP4 to the console - * - @param x FP4 instance to be printed - */ -extern void FP4_BLS12381_output(FP4_BLS12381 *x); -/** @brief Formats and outputs an FP4 to the console in raw form (for debugging) - * - @param x FP4 instance to be printed - */ -extern void FP4_BLS12381_rawoutput(FP4_BLS12381 *x); -/** @brief multiplies an FP4 instance by irreducible polynomial sqrt(1+sqrt(-1)) - * - @param x FP4 instance, on exit = sqrt(1+sqrt(-1)*x - */ -extern void FP4_BLS12381_times_i(FP4_BLS12381 *x); -/** @brief Normalises the components of an FP4 - * - @param x FP4 instance to be normalised - */ -extern void FP4_BLS12381_norm(FP4_BLS12381 *x); -/** @brief Reduces all components of possibly unreduced FP4 mod Modulus - * - @param x FP4 instance, on exit reduced mod Modulus - */ -extern void FP4_BLS12381_reduce(FP4_BLS12381 *x); -/** @brief Raises an FP4 to the power of a BIG - * - @param x FP4 instance, on exit = y^b - @param y FP4 instance - @param b BIG number - */ -extern void FP4_BLS12381_pow(FP4_BLS12381 *x, FP4_BLS12381 *y, BIG_384_58 b); -/** @brief Raises an FP4 to the power of the internal modulus p, using the Frobenius - * - @param x FP4 instance, on exit = x^p - @param f FP2 precalculated Frobenius constant - */ -extern void FP4_BLS12381_frob(FP4_BLS12381 *x, FP2_BLS12381 *f); -/** @brief Calculates the XTR addition function r=w*x-conj(x)*y+z - * - @param r FP4 instance, on exit = w*x-conj(x)*y+z - @param w FP4 instance - @param x FP4 instance - @param y FP4 instance - @param z FP4 instance - */ -extern void FP4_BLS12381_xtr_A(FP4_BLS12381 *r, FP4_BLS12381 *w, FP4_BLS12381 *x, FP4_BLS12381 *y, FP4_BLS12381 *z); -/** @brief Calculates the XTR doubling function r=x^2-2*conj(x) - * - @param r FP4 instance, on exit = x^2-2*conj(x) - @param x FP4 instance - */ -extern void FP4_BLS12381_xtr_D(FP4_BLS12381 *r, FP4_BLS12381 *x); -/** @brief Calculates FP4 trace of an FP12 raised to the power of a BIG number - * - XTR single exponentiation - @param r FP4 instance, on exit = trace(w^b) - @param x FP4 instance, trace of an FP12 w - @param b BIG number - */ -extern void FP4_BLS12381_xtr_pow(FP4_BLS12381 *r, FP4_BLS12381 *x, BIG_384_58 b); -/** @brief Calculates FP4 trace of c^a.d^b, where c and d are derived from FP4 traces of FP12s - * - XTR double exponentiation - Assumes c=tr(x^m), d=tr(x^n), e=tr(x^(m-n)), f=tr(x^(m-2n)) - @param r FP4 instance, on exit = trace(c^a.d^b) - @param c FP4 instance, trace of an FP12 - @param d FP4 instance, trace of an FP12 - @param e FP4 instance, trace of an FP12 - @param f FP4 instance, trace of an FP12 - @param a BIG number - @param b BIG number - */ -extern void FP4_BLS12381_xtr_pow2(FP4_BLS12381 *r, FP4_BLS12381 *c, FP4_BLS12381 *d, FP4_BLS12381 *e, FP4_BLS12381 *f, BIG_384_58 a, BIG_384_58 b); - -/** @brief Conditional copy of FP4 number - * - Conditionally copies second parameter to the first (without branching) - @param x FP4 instance, set to y if s!=0 - @param y another FP4 instance - @param s copy only takes place if not equal to 0 - */ -extern void FP4_BLS12381_cmove(FP4_BLS12381 *x, FP4_BLS12381 *y, int s); - -/** @brief Test FP4 for QR - * - @param r FP4 instance - @return 1 x is a QR, otherwise 0 - */ -extern int FP4_BLS12381_qr(FP4_BLS12381 *r); - -/** @brief Calculate square root of an FP4 - * - Square root - @param r FP4 instance, on exit = sqrt(x) - @param x FP4 instance - @return 1 x is a QR, otherwise 0 - */ -extern void FP4_BLS12381_sqrt(FP4_BLS12381 *r, FP4_BLS12381 *x); - - -/** @brief Divide FP4 number by QNR - * - Divide FP4 by the QNR - @param x FP4 instance - */ -extern void FP4_BLS12381_div_i(FP4_BLS12381 *x); - -/** @brief Divide an FP4 by QNR/2 - * - Divide FP4 by the QNR/2 - @param x FP4 instance - */ -extern void FP4_BLS12381_div_2i(FP4_BLS12381 *x); - - - -/** @brief Divide an FP4 by 2 - * - @param x FP4 instance, on exit = y/2 - @param y FP4 instance - */ -extern void FP4_BLS12381_div2(FP4_BLS12381 *x, FP4_BLS12381 *y); - -/** @brief Generate random FP4 - * - @param x random FP4 number - @param rng random number generator - */ -extern void FP4_BLS12381_rand(FP4_BLS12381 *x, csprng *rng); - -#endif - diff --git a/impl/cbits/fp_BLS12381.c b/impl/cbits/fp_BLS12381.c deleted file mode 100644 index cb7417f46..000000000 --- a/impl/cbits/fp_BLS12381.c +++ /dev/null @@ -1,886 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/* CORE mod p functions */ -/* Small Finite Field arithmetic */ -/* SU=m, SU is Stack Usage (NOT_SPECIAL Modulus) */ - -#include "fp_BLS12381.h" - -/* Fast Modular Reduction Methods */ - -/* r=d mod m */ -/* d MUST be normalised */ -/* Products must be less than pR in all cases !!! */ -/* So when multiplying two numbers, their product *must* be less than MODBITS+BASEBITS*NLEN */ -/* Results *may* be one bit bigger than MODBITS */ - -#if MODTYPE_BLS12381 == PSEUDO_MERSENNE -/* r=d mod m */ - -/* Converts from BIG integer to residue form mod Modulus */ -void FP_BLS12381_nres(FP_BLS12381 *y, BIG_384_58 x) -{ - BIG_384_58 mdls; - BIG_384_58_rcopy(mdls, Modulus_BLS12381); - BIG_384_58_copy(y->g, x); - BIG_384_58_mod(y->g,mdls); - y->XES = 1; -} - -/* Converts from residue form back to BIG integer form */ -void FP_BLS12381_redc(BIG_384_58 x, FP_BLS12381 *y) -{ - BIG_384_58_copy(x, y->g); -} - -/* reduce a DBIG to a BIG exploiting the special form of the modulus */ -void FP_BLS12381_mod(BIG_384_58 r, DBIG_384_58 d) -{ - BIG_384_58 t, b; - chunk v, tw; - BIG_384_58_split(t, b, d, MODBITS_BLS12381); - - /* Note that all of the excess gets pushed into t. So if squaring a value with a 4-bit excess, this results in - t getting all 8 bits of the excess product! So products must be less than pR which is Montgomery compatible */ - - if (MConst_BLS12381 < NEXCESS_384_58) - { - BIG_384_58_imul(t, t, MConst_BLS12381); - BIG_384_58_norm(t); - BIG_384_58_add(r, t, b); - BIG_384_58_norm(r); - tw = r[NLEN_384_58 - 1]; - r[NLEN_384_58 - 1] &= TMASK_BLS12381; - r[0] += MConst_BLS12381 * ((tw >> TBITS_BLS12381)); - } - else - { - v = BIG_384_58_pmul(t, t, MConst_BLS12381); - BIG_384_58_add(r, t, b); - BIG_384_58_norm(r); - tw = r[NLEN_384_58 - 1]; - r[NLEN_384_58 - 1] &= TMASK_BLS12381; -#if CHUNK == 16 - r[1] += muladd_384_58(MConst_BLS12381, ((tw >> TBITS_BLS12381) + (v << (BASEBITS_384_58 - TBITS_BLS12381))), 0, &r[0]); -#else - r[0] += MConst_BLS12381 * ((tw >> TBITS_BLS12381) + (v << (BASEBITS_384_58 - TBITS_BLS12381))); -#endif - } - BIG_384_58_norm(r); -} -#endif - -/* This only applies to Curve C448, so specialised (for now) */ -#if MODTYPE_BLS12381 == GENERALISED_MERSENNE - -void FP_BLS12381_nres(FP_BLS12381 *y, BIG_384_58 x) -{ - BIG_384_58 mdls; - BIG_384_58_rcopy(mdls, Modulus_BLS12381); - BIG_384_58_copy(y->g, x); - BIG_384_58_mod(y->g,mdls); - y->XES = 1; -} - -/* Converts from residue form back to BIG integer form */ -void FP_BLS12381_redc(BIG_384_58 x, FP_BLS12381 *y) -{ - BIG_384_58_copy(x, y->g); -} - -/* reduce a DBIG to a BIG exploiting the special form of the modulus */ -void FP_BLS12381_mod(BIG_384_58 r, DBIG_384_58 d) -{ - BIG_384_58 t, b; - chunk carry; - BIG_384_58_split(t, b, d, MBITS_BLS12381); - - BIG_384_58_add(r, t, b); - - BIG_384_58_dscopy(d, t); - BIG_384_58_dshl(d, MBITS_BLS12381 / 2); - - BIG_384_58_split(t, b, d, MBITS_BLS12381); - - BIG_384_58_add(r, r, t); - BIG_384_58_add(r, r, b); - BIG_384_58_norm(r); - BIG_384_58_shl(t, MBITS_BLS12381 / 2); - - BIG_384_58_add(r, r, t); - - carry = r[NLEN_384_58 - 1] >> TBITS_BLS12381; - - r[NLEN_384_58 - 1] &= TMASK_BLS12381; - r[0] += carry; - - r[224 / BASEBITS_384_58] += carry << (224 % BASEBITS_384_58); /* need to check that this falls mid-word */ - BIG_384_58_norm(r); -} - -#endif - -#if MODTYPE_BLS12381 == MONTGOMERY_FRIENDLY - -/* convert to Montgomery n-residue form */ -void FP_BLS12381_nres(FP_BLS12381 *y, BIG_384_58 x) -{ - DBIG_384_58 d; - BIG_384_58 r; - BIG_384_58_rcopy(r, R2modp_BLS12381); - BIG_384_58_mul(d, x, r); - FP_BLS12381_mod(y->g, d); - y->XES = 2; -} - -/* convert back to regular form */ -void FP_BLS12381_redc(BIG_384_58 x, FP_BLS12381 *y) -{ - DBIG_384_58 d; - BIG_384_58_dzero(d); - BIG_384_58_dscopy(d, y->g); - FP_BLS12381_mod(x, d); -} - -/* fast modular reduction from DBIG to BIG exploiting special form of the modulus */ -void FP_BLS12381_mod(BIG_384_58 a, DBIG_384_58 d) -{ - int i; - - for (i = 0; i < NLEN_384_58; i++) - d[NLEN_384_58 + i] += muladd_384_58(d[i], MConst_BLS12381 - 1, d[i], &d[NLEN_384_58 + i - 1]); - - BIG_384_58_sducopy(a, d); - BIG_384_58_norm(a); -} - -#endif - -#if MODTYPE_BLS12381 == NOT_SPECIAL - -/* convert to Montgomery n-residue form */ -void FP_BLS12381_nres(FP_BLS12381 *y, BIG_384_58 x) -{ - DBIG_384_58 d; - BIG_384_58 r; - BIG_384_58_rcopy(r, R2modp_BLS12381); - BIG_384_58_mul(d, x, r); - FP_BLS12381_mod(y->g, d); - y->XES = 2; -} - -/* convert back to regular form */ -void FP_BLS12381_redc(BIG_384_58 x, FP_BLS12381 *y) -{ - DBIG_384_58 d; - BIG_384_58_dzero(d); - BIG_384_58_dscopy(d, y->g); - FP_BLS12381_mod(x, d); -} - - -/* reduce a DBIG to a BIG using Montgomery's no trial division method */ -/* d is expected to be dnormed before entry */ -/* SU= 112 */ -void FP_BLS12381_mod(BIG_384_58 a, DBIG_384_58 d) -{ - BIG_384_58 mdls; - BIG_384_58_rcopy(mdls, Modulus_BLS12381); - BIG_384_58_monty(a, mdls, MConst_BLS12381, d); -} - -#endif - -void FP_BLS12381_from_int(FP_BLS12381 *x,int a) -{ - BIG_384_58 w; - if (a<0) BIG_384_58_rcopy(w, Modulus_BLS12381); - else BIG_384_58_zero(w); - BIG_384_58_inc(w,a); BIG_384_58_norm(w); - FP_BLS12381_nres(x,w); -} - -/* test x==0 ? */ -/* SU= 48 */ -int FP_BLS12381_iszilch(FP_BLS12381 *x) -{ - BIG_384_58 m; - FP_BLS12381 y; - FP_BLS12381_copy(&y,x); - FP_BLS12381_reduce(&y); - FP_BLS12381_redc(m,&y); - return BIG_384_58_iszilch(m); -} - -int FP_BLS12381_isunity(FP_BLS12381 *x) -{ - BIG_384_58 m; - FP_BLS12381 y; - FP_BLS12381_copy(&y,x); - FP_BLS12381_reduce(&y); - FP_BLS12381_redc(m,&y); - return BIG_384_58_isunity(m); -} - - -void FP_BLS12381_copy(FP_BLS12381 *y, FP_BLS12381 *x) -{ - BIG_384_58_copy(y->g, x->g); - y->XES = x->XES; -} - -void FP_BLS12381_rcopy(FP_BLS12381 *y, const BIG_384_58 c) -{ - BIG_384_58 b; - BIG_384_58_rcopy(b, c); - FP_BLS12381_nres(y, b); -} - -/* Swap a and b if d=1 */ -void FP_BLS12381_cswap(FP_BLS12381 *a, FP_BLS12381 *b, int d) -{ - sign32 t, c = d; - BIG_384_58_cswap(a->g, b->g, d); - - c = ~(c - 1); - t = c & ((a->XES) ^ (b->XES)); - a->XES ^= t; - b->XES ^= t; - -} - -/* Move b to a if d=1 */ -void FP_BLS12381_cmove(FP_BLS12381 *a, FP_BLS12381 *b, int d) -{ - sign32 c = -d; - - BIG_384_58_cmove(a->g, b->g, d); - a->XES ^= (a->XES ^ b->XES)&c; -} - -void FP_BLS12381_zero(FP_BLS12381 *x) -{ - BIG_384_58_zero(x->g); - x->XES = 1; -} - -int FP_BLS12381_equals(FP_BLS12381 *x, FP_BLS12381 *y) -{ - FP_BLS12381 xg, yg; - FP_BLS12381_copy(&xg, x); - FP_BLS12381_copy(&yg, y); - FP_BLS12381_reduce(&xg); - FP_BLS12381_reduce(&yg); - if (BIG_384_58_comp(xg.g, yg.g) == 0) return 1; - return 0; -} - -/* output FP */ -/* SU= 48 */ -void FP_BLS12381_output(FP_BLS12381 *r) -{ - BIG_384_58 c; - FP_BLS12381_reduce(r); - FP_BLS12381_redc(c, r); - BIG_384_58_output(c); -} - -void FP_BLS12381_rawoutput(FP_BLS12381 *r) -{ - BIG_384_58_rawoutput(r->g); -} - -#ifdef GET_STATS -int tsqr = 0, rsqr = 0, tmul = 0, rmul = 0; -int tadd = 0, radd = 0, tneg = 0, rneg = 0; -int tdadd = 0, rdadd = 0, tdneg = 0, rdneg = 0; -#endif - -#ifdef FUSED_MODMUL - -/* Insert fastest code here */ - -#endif - -/* r=a*b mod Modulus */ -/* product must be less that p.R - and we need to know this in advance! */ -/* SU= 88 */ -void FP_BLS12381_mul(FP_BLS12381 *r, FP_BLS12381 *a, FP_BLS12381 *b) -{ - DBIG_384_58 d; - - if ((sign64)a->XES * b->XES > (sign64)FEXCESS_BLS12381) - { -#ifdef DEBUG_REDUCE - printf("Product too large - reducing it\n"); -#endif - FP_BLS12381_reduce(a); /* it is sufficient to fully reduce just one of them < p */ - } - -#ifdef FUSED_MODMUL - FP_BLS12381_modmul(r->g, a->g, b->g); -#else - BIG_384_58_mul(d, a->g, b->g); - FP_BLS12381_mod(r->g, d); -#endif - r->XES = 2; -} - - -/* multiplication by an integer, r=a*c */ -/* SU= 136 */ -void FP_BLS12381_imul(FP_BLS12381 *r, FP_BLS12381 *a, int c) -{ - int s = 0; - - if (c < 0) - { - c = -c; - s = 1; - } - -#if MODTYPE_BLS12381==PSEUDO_MERSENNE || MODTYPE_BLS12381==GENERALISED_MERSENNE - DBIG_384_58 d; - BIG_384_58_pxmul(d, a->g, c); - FP_BLS12381_mod(r->g, d); - r->XES = 2; - -#else - //Montgomery - BIG_384_58 k; - FP_BLS12381 f; - if (a->XES * c <= FEXCESS_BLS12381) - { - BIG_384_58_pmul(r->g, a->g, c); - r->XES = a->XES * c; // careful here - XES jumps! - } - else - { - // don't want to do this - only a problem for Montgomery modulus and larger constants - BIG_384_58_zero(k); - BIG_384_58_inc(k, c); - BIG_384_58_norm(k); - FP_BLS12381_nres(&f, k); - FP_BLS12381_mul(r, a, &f); - } -#endif - - if (s) - { - FP_BLS12381_neg(r, r); - FP_BLS12381_norm(r); - } -} - -/* Set r=a^2 mod m */ -/* SU= 88 */ -void FP_BLS12381_sqr(FP_BLS12381 *r, FP_BLS12381 *a) -{ - DBIG_384_58 d; - - if ((sign64)a->XES * a->XES > (sign64)FEXCESS_BLS12381) - { -#ifdef DEBUG_REDUCE - printf("Product too large - reducing it\n"); -#endif - FP_BLS12381_reduce(a); - } - - BIG_384_58_sqr(d, a->g); - FP_BLS12381_mod(r->g, d); - r->XES = 2; -} - -/* SU= 16 */ -/* Set r=a+b */ -void FP_BLS12381_add(FP_BLS12381 *r, FP_BLS12381 *a, FP_BLS12381 *b) -{ - BIG_384_58_add(r->g, a->g, b->g); - r->XES = a->XES + b->XES; - if (r->XES > FEXCESS_BLS12381) - { -#ifdef DEBUG_REDUCE - printf("Sum too large - reducing it \n"); -#endif - FP_BLS12381_reduce(r); - } -} - -/* Set r=a-b mod m */ -/* SU= 56 */ -void FP_BLS12381_sub(FP_BLS12381 *r, FP_BLS12381 *a, FP_BLS12381 *b) -{ - FP_BLS12381 n; - FP_BLS12381_neg(&n, b); - FP_BLS12381_add(r, a, &n); -} - -// https://graphics.stanford.edu/~seander/bithacks.html -// constant time log to base 2 (or number of bits in) - -static int logb2(unsign32 v) -{ - int r; - v |= v >> 1; - v |= v >> 2; - v |= v >> 4; - v |= v >> 8; - v |= v >> 16; - - v = v - ((v >> 1) & 0x55555555); - v = (v & 0x33333333) + ((v >> 2) & 0x33333333); - r = (((v + (v >> 4)) & 0xF0F0F0F) * 0x1010101) >> 24; - return r; -} - -// find appoximation to quotient of a/m -// Out by at most 2. -// Note that MAXXES is bounded to be 2-bits less than half a word -static int quo(BIG_384_58 n, BIG_384_58 m) -{ - int sh; - chunk num, den; - int hb = CHUNK / 2; - if (TBITS_BLS12381 < hb) - { - sh = hb - TBITS_BLS12381; - num = (n[NLEN_384_58 - 1] << sh) | (n[NLEN_384_58 - 2] >> (BASEBITS_384_58 - sh)); - den = (m[NLEN_384_58 - 1] << sh) | (m[NLEN_384_58 - 2] >> (BASEBITS_384_58 - sh)); - } - else - { - num = n[NLEN_384_58 - 1]; - den = m[NLEN_384_58 - 1]; - } - return (int)(num / (den + 1)); -} - -/* SU= 48 */ -/* Fully reduce a mod Modulus */ -void FP_BLS12381_reduce(FP_BLS12381 *a) -{ - BIG_384_58 m, r; - int sr, sb, q; - chunk carry; - - BIG_384_58_rcopy(m, Modulus_BLS12381); - - BIG_384_58_norm(a->g); - - if (a->XES > 16) - { - q = quo(a->g, m); - carry = BIG_384_58_pmul(r, m, q); - r[NLEN_384_58 - 1] += (carry << BASEBITS_384_58); // correction - put any carry out back in again - BIG_384_58_sub(a->g, a->g, r); - BIG_384_58_norm(a->g); - sb = 2; - } - else sb = logb2(a->XES - 1); // sb does not depend on the actual data - - BIG_384_58_fshl(m, sb); - - while (sb > 0) - { -// constant time... - sr = BIG_384_58_ssn(r, a->g, m); // optimized combined shift, subtract and norm - BIG_384_58_cmove(a->g, r, 1 - sr); - sb--; - } - - //BIG_384_58_mod(a->g,m); - a->XES = 1; -} - -void FP_BLS12381_norm(FP_BLS12381 *x) -{ - BIG_384_58_norm(x->g); -} - -/* Set r=-a mod Modulus */ -/* SU= 64 */ -void FP_BLS12381_neg(FP_BLS12381 *r, FP_BLS12381 *a) -{ - int sb; - BIG_384_58 m; - - BIG_384_58_rcopy(m, Modulus_BLS12381); - - sb = logb2(a->XES - 1); - BIG_384_58_fshl(m, sb); - BIG_384_58_sub(r->g, m, a->g); - r->XES = ((sign32)1 << sb) + 1; - - if (r->XES > FEXCESS_BLS12381) - { -#ifdef DEBUG_REDUCE - printf("Negation too large - reducing it \n"); -#endif - FP_BLS12381_reduce(r); - } - -} - -/* Set r=a/2. */ -/* SU= 56 */ -void FP_BLS12381_div2(FP_BLS12381 *r, FP_BLS12381 *a) -{ - BIG_384_58 m; - BIG_384_58 w; - BIG_384_58_rcopy(m, Modulus_BLS12381); - int pr=BIG_384_58_parity(a->g); - - FP_BLS12381_copy(r, a); - BIG_384_58_copy(w,r->g); - BIG_384_58_fshr(r->g,1); - BIG_384_58_add(w, w, m); - BIG_384_58_norm(w); - BIG_384_58_fshr(w, 1); - - BIG_384_58_cmove(r->g,w,pr); - -} - - -void FP_BLS12381_pow(FP_BLS12381 *r, FP_BLS12381 *a, BIG_384_58 b) -{ - sign8 w[1 + (NLEN_384_58 * BASEBITS_384_58 + 3) / 4]; - FP_BLS12381 tb[16]; - BIG_384_58 t; - int i, nb; - - FP_BLS12381_copy(r,a); - FP_BLS12381_norm(r); - BIG_384_58_copy(t, b); - BIG_384_58_norm(t); - nb = 1 + (BIG_384_58_nbits(t) + 3) / 4; - /* convert exponent to 4-bit window */ - for (i = 0; i < nb; i++) - { - w[i] = BIG_384_58_lastbits(t, 4); - BIG_384_58_dec(t, w[i]); - BIG_384_58_norm(t); - BIG_384_58_fshr(t, 4); - } - - FP_BLS12381_one(&tb[0]); - FP_BLS12381_copy(&tb[1], r); - for (i = 2; i < 16; i++) - FP_BLS12381_mul(&tb[i], &tb[i - 1], r); - - FP_BLS12381_copy(r, &tb[w[nb - 1]]); - for (i = nb - 2; i >= 0; i--) - { - FP_BLS12381_sqr(r, r); - FP_BLS12381_sqr(r, r); - FP_BLS12381_sqr(r, r); - FP_BLS12381_sqr(r, r); - FP_BLS12381_mul(r, r, &tb[w[i]]); - } - FP_BLS12381_reduce(r); -} - - -#if MODTYPE_BLS12381 == PSEUDO_MERSENNE || MODTYPE_BLS12381==GENERALISED_MERSENNE - -// See eprint paper https://eprint.iacr.org/2018/1038 -// If p=3 mod 4 r= x^{(p-3)/4}, if p=5 mod 8 r=x^{(p-5)/8} - -static void FP_BLS12381_fpow(FP_BLS12381 *r, FP_BLS12381 *x) -{ - int i, j, k, bw, w, c, nw, lo, m, n, nd, e=PM1D2_BLS12381; - FP_BLS12381 xp[11], t, key; - const int ac[] = {1, 2, 3, 6, 12, 15, 30, 60, 120, 240, 255}; -// phase 1 - FP_BLS12381_copy(&xp[0], x); // 1 - FP_BLS12381_sqr(&xp[1], x); // 2 - FP_BLS12381_mul(&xp[2], &xp[1], x); //3 - FP_BLS12381_sqr(&xp[3], &xp[2]); // 6 - FP_BLS12381_sqr(&xp[4], &xp[3]); // 12 - FP_BLS12381_mul(&xp[5], &xp[4], &xp[2]); // 15 - FP_BLS12381_sqr(&xp[6], &xp[5]); // 30 - FP_BLS12381_sqr(&xp[7], &xp[6]); // 60 - FP_BLS12381_sqr(&xp[8], &xp[7]); // 120 - FP_BLS12381_sqr(&xp[9], &xp[8]); // 240 - FP_BLS12381_mul(&xp[10], &xp[9], &xp[5]); // 255 - -#if MODTYPE_BLS12381==PSEUDO_MERSENNE - n = MODBITS_BLS12381; -#endif -#if MODTYPE_BLS12381==GENERALISED_MERSENNE // Goldilocks ONLY - n = MODBITS_BLS12381 / 2; -#endif - - n-=(e+1); - c=(MConst_BLS12381+(1< k) i--; - FP_BLS12381_copy(&key, &xp[i]); - k -= ac[i]; - } - while (k != 0) - { - i--; - if (ac[i] > k) continue; - FP_BLS12381_mul(&key, &key, &xp[i]); - k -= ac[i]; - } - -// phase 2 - FP_BLS12381_copy(&xp[1], &xp[2]); - FP_BLS12381_copy(&xp[2], &xp[5]); - FP_BLS12381_copy(&xp[3], &xp[10]); - - j = 3; m = 8; - nw = n - bw; - while (2 * m < nw) - { - FP_BLS12381_copy(&t, &xp[j++]); - for (i = 0; i < m; i++) - FP_BLS12381_sqr(&t, &t); - FP_BLS12381_mul(&xp[j], &xp[j - 1], &t); - m *= 2; - } - - lo = nw - m; - FP_BLS12381_copy(r, &xp[j]); - - while (lo != 0) - { - m /= 2; j--; - if (lo < m) continue; - lo -= m; - FP_BLS12381_copy(&t, r); - for (i = 0; i < m; i++) - FP_BLS12381_sqr(&t, &t); - FP_BLS12381_mul(r, &t, &xp[j]); - } -// phase 3 - - if (bw != 0) - { - for (i = 0; i < bw; i++ ) - FP_BLS12381_sqr(r, r); - FP_BLS12381_mul(r, r, &key); - } -#if MODTYPE_BLS12381==GENERALISED_MERSENNE // Goldilocks ONLY - FP_BLS12381_copy(&key, r); - FP_BLS12381_sqr(&t, &key); - FP_BLS12381_mul(r, &t, &xp[0]); - for (i = 0; i < n + 1; i++) - FP_BLS12381_sqr(r, r); - FP_BLS12381_mul(r, r, &key); -#endif - - for (i=0;i1;k--) - { - for (j=1;j>1; - -#else - BIG_384_58 m; - FP_BLS12381 y; - FP_BLS12381_copy(&y,x); - FP_BLS12381_reduce(&y); - FP_BLS12381_redc(m,&y); - return BIG_384_58_parity(m); -#endif -} - -void FP_BLS12381_rand(FP_BLS12381 *x,csprng *rng) -{ - BIG_384_58 w,m; - BIG_384_58_rcopy(m,Modulus_BLS12381); - BIG_384_58_randomnum(w,m,rng); - FP_BLS12381_nres(x,w); -} - - diff --git a/impl/cbits/fp_BLS12381.h b/impl/cbits/fp_BLS12381.h deleted file mode 100644 index 27d20bbed..000000000 --- a/impl/cbits/fp_BLS12381.h +++ /dev/null @@ -1,313 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file fp.h - * @author Mike Scott - * @brief FP Header File - * - */ - -#ifndef FP_BLS12381_H -#define FP_BLS12381_H - -#include "big_384_58.h" -#include "config_field_BLS12381.h" - - -/** - @brief FP Structure - quadratic extension field -*/ - -typedef struct -{ - BIG_384_58 g; /**< Big representation of field element */ - sign32 XES; /**< Excess */ -} FP_BLS12381; - - -/* Field Params - see rom.c */ -extern const BIG_384_58 Modulus_BLS12381; /**< Actual Modulus set in rom_field_yyy.c */ -extern const BIG_384_58 ROI_BLS12381; /**< Root of unity set in rom_field_yyy.c */ -extern const BIG_384_58 R2modp_BLS12381; /**< Montgomery constant */ -extern const BIG_384_58 CRu_BLS12381; /**< Cube Root of Unity */ -extern const BIG_384_58 SQRTm3_BLS12381; /**< Square root of -3 */ -extern const chunk MConst_BLS12381; /**< Constant associated with Modulus - for Montgomery = 1/p mod 2^BASEBITS */ - - -#define MODBITS_BLS12381 MBITS_BLS12381 /**< Number of bits in Modulus for selected curve */ -#define TBITS_BLS12381 (MBITS_BLS12381%BASEBITS_384_58) /**< Number of active bits in top word */ -#define TMASK_BLS12381 (((chunk)1< -#include -#include "arch.h" -#include "core.h" - -#define NB 4 -#define MR_TOBYTE(x) ((uchar)((x))) - -static unsign32 pack(const uchar *b) -{ - /* pack bytes into a 32-bit Word */ - return ((unsign32)b[0] << 24) | ((unsign32)b[1] << 16) | ((unsign32)b[2] << 8) | (unsign32)b[3]; -} - -static void unpack(unsign32 a, uchar *b) -{ - /* unpack bytes from a word */ - b[3] = MR_TOBYTE(a); - b[2] = MR_TOBYTE(a >> 8); - b[1] = MR_TOBYTE(a >> 16); - b[0] = MR_TOBYTE(a >> 24); -} - -static void precompute(gcm *g, uchar *H) -{ - /* precompute small 2k bytes gf2m table of x^n.H */ - int i, j; - unsign32 *last, *next, b; - - for (i = j = 0; i < NB; i++, j += 4) g->table[0][i] = pack((uchar *)&H[j]); - - for (i = 1; i < 128; i++) - { - next = g->table[i]; - last = g->table[i - 1]; - b = 0; - for (j = 0; j < NB; j++) - { - next[j] = b | (last[j]) >> 1; - b = last[j] << 31; - } - if (b) next[0] ^= 0xE1000000; /* irreducible polynomial */ - } -} - -/* SU= 32 */ -static void gf2mul(gcm *g) -{ - /* gf2m mul - Z=H*X mod 2^128 */ - int i, j, m, k; - unsign32 P[4]; - unsign32 b; - - P[0] = P[1] = P[2] = P[3] = 0; - j = 8; - m = 0; - for (i = 0; i < 128; i++) - { - b = (unsign32)(g->stateX[m] >> (--j)) & 1; - b = ~b + 1; - for (k = 0; k < NB; k++) P[k] ^= (g->table[i][k] & b); - if (j == 0) - { - j = 8; - m++; - if (m == 16) break; - } - } - for (i = j = 0; i < NB; i++, j += 4) unpack(P[i], (uchar *)&g->stateX[j]); -} - -/* SU= 32 */ -static void GCM_wrap(gcm *g) -{ - /* Finish off GHASH */ - int i, j; - unsign32 F[4]; - uchar L[16]; - - /* convert lengths from bytes to bits */ - F[0] = (g->lenA[0] << 3) | (g->lenA[1] & 0xE0000000) >> 29; - F[1] = g->lenA[1] << 3; - F[2] = (g->lenC[0] << 3) | (g->lenC[1] & 0xE0000000) >> 29; - F[3] = g->lenC[1] << 3; - for (i = j = 0; i < NB; i++, j += 4) unpack(F[i], (uchar *)&L[j]); - - for (i = 0; i < 16; i++) g->stateX[i] ^= L[i]; - gf2mul(g); -} - -static int GCM_ghash(gcm *g, char *plain, int len) -{ - int i, j = 0; - if (g->status == GCM_ACCEPTING_HEADER) g->status = GCM_ACCEPTING_CIPHER; - if (g->status != GCM_ACCEPTING_CIPHER) return 0; - - while (j < len) - { - for (i = 0; i < 16 && j < len; i++) - { - g->stateX[i] ^= plain[j++]; - g->lenC[1]++; - if (g->lenC[1] == 0) g->lenC[0]++; - } - gf2mul(g); - } - if (len % 16 != 0) g->status = GCM_NOT_ACCEPTING_MORE; - return 1; -} - -/* SU= 48 */ -/* Initialize GCM mode */ -void GCM_init(gcm* g, int nk, char *key, int niv, char *iv) -{ - /* iv size niv is usually 12 bytes (96 bits). AES key size nk can be 16,24 or 32 bytes */ - int i; - uchar H[16]; - for (i = 0; i < 16; i++) - { - H[i] = 0; - g->stateX[i] = 0; - } - - AES_init(&(g->a), ECB, nk, key, iv); - AES_ecb_encrypt(&(g->a), H); /* E(K,0) */ - precompute(g, H); - - g->lenA[0] = g->lenC[0] = g->lenA[1] = g->lenC[1] = 0; - if (niv == 12) - { - for (i = 0; i < 12; i++) g->a.f[i] = iv[i]; - unpack((unsign32)1, (uchar *) & (g->a.f[12])); /* initialise IV */ - for (i = 0; i < 16; i++) g->Y_0[i] = g->a.f[i]; - } - else - { - g->status = GCM_ACCEPTING_CIPHER; - GCM_ghash(g, iv, niv); /* GHASH(H,0,IV) */ - GCM_wrap(g); - for (i = 0; i < 16; i++) - { - g->a.f[i] = g->stateX[i]; - g->Y_0[i] = g->a.f[i]; - g->stateX[i] = 0; - } - g->lenA[0] = g->lenC[0] = g->lenA[1] = g->lenC[1] = 0; - } - g->status = GCM_ACCEPTING_HEADER; -} - -/* SU= 24 */ -/* Add Header data - included but not encrypted */ -int GCM_add_header(gcm* g, char *header, int len) -{ - /* Add some header. Won't be encrypted, but will be authenticated. len is length of header */ - int i, j = 0; - if (g->status != GCM_ACCEPTING_HEADER) return 0; - - while (j < len) - { - for (i = 0; i < 16 && j < len; i++) - { - g->stateX[i] ^= header[j++]; - g->lenA[1]++; - if (g->lenA[1] == 0) g->lenA[0]++; - } - gf2mul(g); - } - if (len % 16 != 0) g->status = GCM_ACCEPTING_CIPHER; - return 1; -} - -/* SU= 48 */ -/* Add Plaintext - included and encrypted */ -int GCM_add_plain(gcm *g, char *cipher, char *plain, int len) -{ - /* Add plaintext to extract ciphertext, len is length of plaintext. */ - int i, j = 0; - unsign32 counter; - uchar B[16]; - if (g->status == GCM_ACCEPTING_HEADER) g->status = GCM_ACCEPTING_CIPHER; - if (g->status != GCM_ACCEPTING_CIPHER) return 0; - - while (j < len) - { - counter = pack((uchar *) & (g->a.f[12])); - counter++; - unpack(counter, (uchar *) & (g->a.f[12])); /* increment counter */ - for (i = 0; i < 16; i++) B[i] = g->a.f[i]; - AES_ecb_encrypt(&(g->a), B); /* encrypt it */ - - for (i = 0; i < 16 && j < len; i++) - { - cipher[j] = plain[j] ^ B[i]; - g->stateX[i] ^= cipher[j++]; - g->lenC[1]++; - if (g->lenC[1] == 0) g->lenC[0]++; - } - gf2mul(g); - } - if (len % 16 != 0) g->status = GCM_NOT_ACCEPTING_MORE; - return 1; -} - -/* SU= 48 */ -/* Add Ciphertext - decrypts to plaintext */ -int GCM_add_cipher(gcm *g, char *plain, char *cipher, int len) -{ - /* Add ciphertext to extract plaintext, len is length of ciphertext. */ - int i, j = 0; - unsign32 counter; - char oc; - uchar B[16]; - if (g->status == GCM_ACCEPTING_HEADER) g->status = GCM_ACCEPTING_CIPHER; - if (g->status != GCM_ACCEPTING_CIPHER) return 0; - - while (j < len) - { - counter = pack((uchar *) & (g->a.f[12])); - counter++; - unpack(counter, (uchar *) & (g->a.f[12])); /* increment counter */ - for (i = 0; i < 16; i++) B[i] = g->a.f[i]; - AES_ecb_encrypt(&(g->a), B); /* encrypt it */ - for (i = 0; i < 16 && j < len; i++) - { - oc = cipher[j]; - plain[j] = cipher[j] ^ B[i]; - g->stateX[i] ^= oc; - j++; - g->lenC[1]++; - if (g->lenC[1] == 0) g->lenC[0]++; - } - gf2mul(g); - } - if (len % 16 != 0) g->status = GCM_NOT_ACCEPTING_MORE; - return 1; -} - -/* SU= 16 */ -/* Finish and extract Tag */ -void GCM_finish(gcm *g, char *tag) -{ - /* Finish off GHASH and extract tag (MAC) */ - int i; - - GCM_wrap(g); - - /* extract tag */ - if (tag != NULL) - { - AES_ecb_encrypt(&(g->a), g->Y_0); /* E(K,Y0) */ - for (i = 0; i < 16; i++) g->Y_0[i] ^= g->stateX[i]; - for (i = 0; i < 16; i++) - { - tag[i] = g->Y_0[i]; - g->Y_0[i] = g->stateX[i] = 0; - } - } - g->status = GCM_FINISHED; - AES_end(&(g->a)); -} - -/* AES-GCM Encryption of octets, K is key, H is header, - P is plaintext, C is ciphertext, T is authentication tag */ -void AES_GCM_ENCRYPT(octet *K, octet *IV, octet *H, octet *P, octet *C, octet *T) -{ - gcm g; - GCM_init(&g, K->len, K->val, IV->len, IV->val); - GCM_add_header(&g, H->val, H->len); - GCM_add_plain(&g, C->val, P->val, P->len); - C->len = P->len; - GCM_finish(&g, T->val); - T->len = 16; -} - -/* AES-GCM Decryption of octets, K is key, H is header, - P is plaintext, C is ciphertext, T is authentication tag */ -void AES_GCM_DECRYPT(octet *K, octet *IV, octet *H, octet *C, octet *P, octet *T) -{ - gcm g; - GCM_init(&g, K->len, K->val, IV->len, IV->val); - GCM_add_header(&g, H->val, H->len); - GCM_add_cipher(&g, P->val, C->val, C->len); - P->len = C->len; - GCM_finish(&g, T->val); - T->len = 16; -} - -// Compile with -// gcc -O2 gcm.c aes.c -o gcm.exe -/* SU= 16 -*/ - -/* static void hex2bytes(char *hex,char *bin) */ -/* { */ -/* int i; */ -/* char v; */ -/* int len=strlen(hex); */ -/* for (i = 0; i < len/2; i++) { */ -/* char c = hex[2*i]; */ -/* if (c >= '0' && c <= '9') { */ -/* v = c - '0'; */ -/* } else if (c >= 'A' && c <= 'F') { */ -/* v = c - 'A' + 10; */ -/* } else if (c >= 'a' && c <= 'f') { */ -/* v = c - 'a' + 10; */ -/* } else { */ -/* v = 0; */ -/* } */ -/* v <<= 4; */ -/* c = hex[2*i + 1]; */ -/* if (c >= '0' && c <= '9') { */ -/* v += c - '0'; */ -/* } else if (c >= 'A' && c <= 'F') { */ -/* v += c - 'A' + 10; */ -/* } else if (c >= 'a' && c <= 'f') { */ -/* v += c - 'a' + 10; */ -/* } else { */ -/* v = 0; */ -/* } */ -/* bin[i] = v; */ -/* } */ -/* } */ - -/* -int main() -{ - int i; - -// char* KT="feffe9928665731c6d6a8f9467308308"; -// char* MT="d9313225f88406e5a55909c5aff5269a86a7a9531534f7da2e4c303d8a318a721c3c0c95956809532fcf0e2449a6b525b16aedf5aa0de657ba637b39"; -// char* HT="feedfacedeadbeeffeedfacedeadbeefabaddad2"; -// char* NT="cafebabefacedbaddecaf888"; -// Tag should be 5bc94fbc3221a5db94fae95ae7121a47 -// char* NT="9313225df88406e555909c5aff5269aa6a7a9538534f7da1e4c303d2a318a728c3c0c95156809539fcf0e2429a6b525416aedbf5a0de6a57a637b39b"; -// Tag should be 619cc5aefffe0bfa462af43c1699d050 - - char* KT="6dfb5dc68af6ae2f3242e9184f100918"; - char* MT="47809d16c2c6ec685962c90e53fe1bba"; - char* HT="dd0fa6e494031139d71ee45f00d56fa4"; - char* NT="37d36f5c54d53479d4745dd1"; - - - int len=strlen(MT)/2; - int lenH=strlen(HT)/2; - int lenK=strlen(KT)/2; - int lenIV=strlen(NT)/2; - - char T[16]; // Tag - char K[16]; // AES Key - char H[64]; // Header - to be included in Authentication, but not encrypted - char N[100]; // IV - Initialisation vector - char M[100]; // Plaintext to be encrypted/authenticated - char C[100]; // Ciphertext - char P[100]; // Recovered Plaintext - - gcm g; - - hex2bytes(MT, M); - hex2bytes(HT, H); - hex2bytes(NT, N); - hex2bytes(KT, K); - - printf("lenK= %d\n",lenK); - - printf("Plaintext=\n"); - for (i=0;i>n) | ((x)<<(m-n))) -#define R(n,x) ((x)>>n) - -#define Ch(x,y,z) ((x&y)^(~(x)&z)) -#define Maj(x,y,z) ((x&y)^(x&z)^(y&z)) -#define Sig0_256(x) (S(32,2,x)^S(32,13,x)^S(32,22,x)) -#define Sig1_256(x) (S(32,6,x)^S(32,11,x)^S(32,25,x)) -#define theta0_256(x) (S(32,7,x)^S(32,18,x)^R(3,x)) -#define theta1_256(x) (S(32,17,x)^S(32,19,x)^R(10,x)) - -#define Sig0_512(x) (S(64,28,x)^S(64,34,x)^S(64,39,x)) -#define Sig1_512(x) (S(64,14,x)^S(64,18,x)^S(64,41,x)) -#define theta0_512(x) (S(64,1,x)^S(64,8,x)^R(7,x)) -#define theta1_512(x) (S(64,19,x)^S(64,61,x)^R(6,x)) - - -/* SU= 72 */ -static void HASH256_transform(hash256 *sh) -{ - /* basic transformation step */ - unsign32 a, b, c, d, e, f, g, h, t1, t2; - int j; - for (j = 16; j < 64; j++) - sh->w[j] = theta1_256(sh->w[j - 2]) + sh->w[j - 7] + theta0_256(sh->w[j - 15]) + sh->w[j - 16]; - - a = sh->h[0]; - b = sh->h[1]; - c = sh->h[2]; - d = sh->h[3]; - e = sh->h[4]; - f = sh->h[5]; - g = sh->h[6]; - h = sh->h[7]; - - for (j = 0; j < 64; j++) - { - /* 64 times - mush it up */ - t1 = h + Sig1_256(e) + Ch(e, f, g) + K_256[j] + sh->w[j]; - t2 = Sig0_256(a) + Maj(a, b, c); - h = g; - g = f; - f = e; - e = d + t1; - d = c; - c = b; - b = a; - a = t1 + t2; - } - - sh->h[0] += a; - sh->h[1] += b; - sh->h[2] += c; - sh->h[3] += d; - sh->h[4] += e; - sh->h[5] += f; - sh->h[6] += g; - sh->h[7] += h; -} - -/* Initialise Hash function */ -void HASH256_init(hash256 *sh) -{ - /* re-initialise */ - int i; - for (i = 0; i < 64; i++) sh->w[i] = 0L; - sh->length[0] = sh->length[1] = 0L; - sh->h[0] = H0_256; - sh->h[1] = H1_256; - sh->h[2] = H2_256; - sh->h[3] = H3_256; - sh->h[4] = H4_256; - sh->h[5] = H5_256; - sh->h[6] = H6_256; - sh->h[7] = H7_256; - - sh->hlen = 32; -} - -/* process a single byte */ -void HASH256_process(hash256 *sh, int byt) -{ - /* process the next message byte */ - int cnt; - cnt = (int)((sh->length[0] / 32) % 16); - - sh->w[cnt] <<= 8; - sh->w[cnt] |= (unsign32)(byt & 0xFF); - - sh->length[0] += 8; - if (sh->length[0] == 0L) - { - sh->length[1]++; - sh->length[0] = 0L; - } - if ((sh->length[0] % 512) == 0) HASH256_transform(sh); -} - -/* SU= 24 */ -/* Generate 32-byte Hash */ -void HASH256_hash(hash256 *sh, char *digest) -{ - /* pad message and finish - supply digest */ - int i; - unsign32 len0, len1; - len0 = sh->length[0]; - len1 = sh->length[1]; - HASH256_process(sh, PAD); - while ((sh->length[0] % 512) != 448) HASH256_process(sh, ZERO); - sh->w[14] = len1; - sh->w[15] = len0; - HASH256_transform(sh); - for (i = 0; i < sh->hlen; i++) - { - /* convert to bytes */ - digest[i] = (char)((sh->h[i / 4] >> (8 * (3 - i % 4))) & 0xffL); - } - HASH256_init(sh); -} - - -#define H0_512 0x6a09e667f3bcc908 -#define H1_512 0xbb67ae8584caa73b -#define H2_512 0x3c6ef372fe94f82b -#define H3_512 0xa54ff53a5f1d36f1 -#define H4_512 0x510e527fade682d1 -#define H5_512 0x9b05688c2b3e6c1f -#define H6_512 0x1f83d9abfb41bd6b -#define H7_512 0x5be0cd19137e2179 - -#define H8_512 0xcbbb9d5dc1059ed8 -#define H9_512 0x629a292a367cd507 -#define HA_512 0x9159015a3070dd17 -#define HB_512 0x152fecd8f70e5939 -#define HC_512 0x67332667ffc00b31 -#define HD_512 0x8eb44a8768581511 -#define HE_512 0xdb0c2e0d64f98fa7 -#define HF_512 0x47b5481dbefa4fa4 - -/* */ - -static const unsign64 K_512[80] = -{ - 0x428a2f98d728ae22, 0x7137449123ef65cd, 0xb5c0fbcfec4d3b2f, 0xe9b5dba58189dbbc, - 0x3956c25bf348b538, 0x59f111f1b605d019, 0x923f82a4af194f9b, 0xab1c5ed5da6d8118, - 0xd807aa98a3030242, 0x12835b0145706fbe, 0x243185be4ee4b28c, 0x550c7dc3d5ffb4e2, - 0x72be5d74f27b896f, 0x80deb1fe3b1696b1, 0x9bdc06a725c71235, 0xc19bf174cf692694, - 0xe49b69c19ef14ad2, 0xefbe4786384f25e3, 0x0fc19dc68b8cd5b5, 0x240ca1cc77ac9c65, - 0x2de92c6f592b0275, 0x4a7484aa6ea6e483, 0x5cb0a9dcbd41fbd4, 0x76f988da831153b5, - 0x983e5152ee66dfab, 0xa831c66d2db43210, 0xb00327c898fb213f, 0xbf597fc7beef0ee4, - 0xc6e00bf33da88fc2, 0xd5a79147930aa725, 0x06ca6351e003826f, 0x142929670a0e6e70, - 0x27b70a8546d22ffc, 0x2e1b21385c26c926, 0x4d2c6dfc5ac42aed, 0x53380d139d95b3df, - 0x650a73548baf63de, 0x766a0abb3c77b2a8, 0x81c2c92e47edaee6, 0x92722c851482353b, - 0xa2bfe8a14cf10364, 0xa81a664bbc423001, 0xc24b8b70d0f89791, 0xc76c51a30654be30, - 0xd192e819d6ef5218, 0xd69906245565a910, 0xf40e35855771202a, 0x106aa07032bbd1b8, - 0x19a4c116b8d2d0c8, 0x1e376c085141ab53, 0x2748774cdf8eeb99, 0x34b0bcb5e19b48a8, - 0x391c0cb3c5c95a63, 0x4ed8aa4ae3418acb, 0x5b9cca4f7763e373, 0x682e6ff3d6b2b8a3, - 0x748f82ee5defb2fc, 0x78a5636f43172f60, 0x84c87814a1f0ab72, 0x8cc702081a6439ec, - 0x90befffa23631e28, 0xa4506cebde82bde9, 0xbef9a3f7b2c67915, 0xc67178f2e372532b, - 0xca273eceea26619c, 0xd186b8c721c0c207, 0xeada7dd6cde0eb1e, 0xf57d4f7fee6ed178, - 0x06f067aa72176fba, 0x0a637dc5a2c898a6, 0x113f9804bef90dae, 0x1b710b35131c471b, - 0x28db77f523047d84, 0x32caab7b40c72493, 0x3c9ebe0a15c9bebc, 0x431d67c49c100d4c, - 0x4cc5d4becb3e42b6, 0x597f299cfc657e2a, 0x5fcb6fab3ad6faec, 0x6c44198c4a475817 -}; - - -static void HASH512_transform(hash512 *sh) -{ - /* basic transformation step */ - unsign64 a, b, c, d, e, f, g, h, t1, t2; - int j; - for (j = 16; j < 80; j++) - sh->w[j] = theta1_512(sh->w[j - 2]) + sh->w[j - 7] + theta0_512(sh->w[j - 15]) + sh->w[j - 16]; - - a = sh->h[0]; - b = sh->h[1]; - c = sh->h[2]; - d = sh->h[3]; - e = sh->h[4]; - f = sh->h[5]; - g = sh->h[6]; - h = sh->h[7]; - - for (j = 0; j < 80; j++) - { - /* 80 times - mush it up */ - t1 = h + Sig1_512(e) + Ch(e, f, g) + K_512[j] + sh->w[j]; - t2 = Sig0_512(a) + Maj(a, b, c); - h = g; - g = f; - f = e; - e = d + t1; - d = c; - c = b; - b = a; - a = t1 + t2; - } - sh->h[0] += a; - sh->h[1] += b; - sh->h[2] += c; - sh->h[3] += d; - sh->h[4] += e; - sh->h[5] += f; - sh->h[6] += g; - sh->h[7] += h; -} - -void HASH384_init(hash384 *sh) -{ - /* re-initialise */ - int i; - for (i = 0; i < 80; i++) sh->w[i] = 0; - sh->length[0] = sh->length[1] = 0; - sh->h[0] = H8_512; - sh->h[1] = H9_512; - sh->h[2] = HA_512; - sh->h[3] = HB_512; - sh->h[4] = HC_512; - sh->h[5] = HD_512; - sh->h[6] = HE_512; - sh->h[7] = HF_512; - - sh->hlen = 48; - -} - -void HASH384_process(hash384 *sh, int byt) -{ - /* process the next message byte */ - HASH512_process(sh, byt); -} - -void HASH384_hash(hash384 *sh, char *hash) -{ - /* pad message and finish - supply digest */ - HASH512_hash(sh, hash); -} - -void HASH512_init(hash512 *sh) -{ - /* re-initialise */ - int i; - - for (i = 0; i < 80; i++) sh->w[i] = 0; - sh->length[0] = sh->length[1] = 0; - sh->h[0] = H0_512; - sh->h[1] = H1_512; - sh->h[2] = H2_512; - sh->h[3] = H3_512; - sh->h[4] = H4_512; - sh->h[5] = H5_512; - sh->h[6] = H6_512; - sh->h[7] = H7_512; - - sh->hlen = 64; -} - -void HASH512_process(hash512 *sh, int byt) -{ - /* process the next message byte */ - int cnt; - - cnt = (int)((sh->length[0] / 64) % 16); - - sh->w[cnt] <<= 8; - sh->w[cnt] |= (unsign64)(byt & 0xFF); - - sh->length[0] += 8; - if (sh->length[0] == 0L) - { - sh->length[1]++; - sh->length[0] = 0L; - } - if ((sh->length[0] % 1024) == 0) HASH512_transform(sh); -} - -void HASH512_hash(hash512 *sh, char *hash) -{ - /* pad message and finish - supply digest */ - int i; - unsign64 len0, len1; - len0 = sh->length[0]; - len1 = sh->length[1]; - HASH512_process(sh, PAD); - while ((sh->length[0] % 1024) != 896) HASH512_process(sh, ZERO); - sh->w[14] = len1; - sh->w[15] = len0; - HASH512_transform(sh); - for (i = 0; i < sh->hlen; i++) - { - /* convert to bytes */ - hash[i] = (char)((sh->h[i / 8] >> (8 * (7 - i % 8))) & 0xffL); - } - HASH512_init(sh); -} - - - -/* SHA3 */ - -#define SHA3_ROUNDS 24 -#define rotl(x,n) (((x)<>(64-n))) - -/* round constants */ - -static const unsign64 RC[24] = -{ - 0x0000000000000001UL, 0x0000000000008082UL, 0x800000000000808AUL, 0x8000000080008000UL, - 0x000000000000808BUL, 0x0000000080000001UL, 0x8000000080008081UL, 0x8000000000008009UL, - 0x000000000000008AUL, 0x0000000000000088UL, 0x0000000080008009UL, 0x000000008000000AUL, - 0x000000008000808BUL, 0x800000000000008BUL, 0x8000000000008089UL, 0x8000000000008003UL, - 0x8000000000008002UL, 0x8000000000000080UL, 0x000000000000800AUL, 0x800000008000000AUL, - 0x8000000080008081UL, 0x8000000000008080UL, 0x0000000080000001UL, 0x8000000080008008UL -}; - -/* permutation */ - -static void SHA3_transform(sha3 *sh) -{ - int i, j, k; - unsign64 C[5], D[5], B[5][5]; - - for (k = 0; k < SHA3_ROUNDS; k++) - { - C[0] = sh->S[0][0] ^ sh->S[0][1] ^ sh->S[0][2] ^ sh->S[0][3] ^ sh->S[0][4]; - C[1] = sh->S[1][0] ^ sh->S[1][1] ^ sh->S[1][2] ^ sh->S[1][3] ^ sh->S[1][4]; - C[2] = sh->S[2][0] ^ sh->S[2][1] ^ sh->S[2][2] ^ sh->S[2][3] ^ sh->S[2][4]; - C[3] = sh->S[3][0] ^ sh->S[3][1] ^ sh->S[3][2] ^ sh->S[3][3] ^ sh->S[3][4]; - C[4] = sh->S[4][0] ^ sh->S[4][1] ^ sh->S[4][2] ^ sh->S[4][3] ^ sh->S[4][4]; - - D[0] = C[4] ^ rotl(C[1], 1); - D[1] = C[0] ^ rotl(C[2], 1); - D[2] = C[1] ^ rotl(C[3], 1); - D[3] = C[2] ^ rotl(C[4], 1); - D[4] = C[3] ^ rotl(C[0], 1); - - for (i = 0; i < 5; i++) - for (j = 0; j < 5; j++) - sh->S[i][j] ^= D[i]; /* let the compiler unroll it! */ - - B[0][0] = sh->S[0][0]; - B[1][3] = rotl(sh->S[0][1], 36); - B[2][1] = rotl(sh->S[0][2], 3); - B[3][4] = rotl(sh->S[0][3], 41); - B[4][2] = rotl(sh->S[0][4], 18); - - B[0][2] = rotl(sh->S[1][0], 1); - B[1][0] = rotl(sh->S[1][1], 44); - B[2][3] = rotl(sh->S[1][2], 10); - B[3][1] = rotl(sh->S[1][3], 45); - B[4][4] = rotl(sh->S[1][4], 2); - - B[0][4] = rotl(sh->S[2][0], 62); - B[1][2] = rotl(sh->S[2][1], 6); - B[2][0] = rotl(sh->S[2][2], 43); - B[3][3] = rotl(sh->S[2][3], 15); - B[4][1] = rotl(sh->S[2][4], 61); - - B[0][1] = rotl(sh->S[3][0], 28); - B[1][4] = rotl(sh->S[3][1], 55); - B[2][2] = rotl(sh->S[3][2], 25); - B[3][0] = rotl(sh->S[3][3], 21); - B[4][3] = rotl(sh->S[3][4], 56); - - B[0][3] = rotl(sh->S[4][0], 27); - B[1][1] = rotl(sh->S[4][1], 20); - B[2][4] = rotl(sh->S[4][2], 39); - B[3][2] = rotl(sh->S[4][3], 8); - B[4][0] = rotl(sh->S[4][4], 14); - - for (i = 0; i < 5; i++) - for (j = 0; j < 5; j++) - sh->S[i][j] = B[i][j] ^ (~B[(i + 1) % 5][j] & B[(i + 2) % 5][j]); - - sh->S[0][0] ^= RC[k]; - } -} - -/* Re-Initialize. olen is output length in bytes - - should be 28, 32, 48 or 64 (224, 256, 384, 512 bits resp.) */ - -void SHA3_init(sha3 *sh, int olen) -{ - int i, j; - for (i = 0; i < 5; i++) - for (j = 0; j < 5; j++) - sh->S[i][j] = 0; /* 5x5x8 bytes = 200 bytes of state */ - sh->length = 0; - sh->len = olen; - sh->rate = 200 - 2 * olen; /* number of bytes consumed in one gulp. Note that some bytes in the - state ("capacity") are not touched. Gulps are smaller for larger digests. - Important that olenlength % sh->rate); - int i, j, b = cnt % 8; - cnt /= 8; - i = cnt % 5; - j = cnt / 5; /* process by columns! */ - sh->S[i][j] ^= ((unsign64)byt << (8 * b)); - sh->length++; - if (sh->length % sh->rate == 0) SHA3_transform(sh); -} - -/* squeeze the sponge */ -void SHA3_squeeze(sha3 *sh, char *buff, int len) -{ - int done, i, j, k, m = 0; - unsign64 el; - /* extract by columns */ - done = 0; - for (;;) - { - for (j = 0; j < 5; j++) - { - for (i = 0; i < 5; i++) - { - el = sh->S[i][j]; - for (k = 0; k < 8; k++) - { - buff[m++] = (el & 0xff); - if (m >= len || m % sh->rate == 0) - { - done = 1; - break; - } - el >>= 8; - } - if (done) break; - } - if (done) break; - } - if (m >= len) break; - done = 0; - SHA3_transform(sh); - } -} - -void SHA3_hash(sha3 *sh, char *hash) -{ - /* generate a SHA3 hash of appropriate size */ - int q = sh->rate - (sh->length % sh->rate); - if (q == 1) SHA3_process(sh, 0x86); - else - { - SHA3_process(sh, 0x06); /* 0x06 for SHA-3 */ - while ((int)sh->length % sh->rate != sh->rate - 1) SHA3_process(sh, 0x00); - SHA3_process(sh, 0x80); /* this will force a final transform */ - } - SHA3_squeeze(sh, hash, sh->len); -} - -void SHA3_shake(sha3 *sh, char *buff, int len) -{ - /* SHAKE out a buffer of variable length len */ - int q = sh->rate - (sh->length % sh->rate); - if (q == 1) SHA3_process(sh, 0x9f); - else - { - SHA3_process(sh, 0x1f); // 0x06 for SHA-3 !!!! - while ((int) sh->length % sh->rate != sh->rate - 1) SHA3_process(sh, 0x00); - SHA3_process(sh, 0x80); /* this will force a final transform */ - } - SHA3_squeeze(sh, buff, len); -} - - -/* test program: should produce digest - -160 bit - -84983e44 1c3bd26e baae4aa1 f95129e5 e54670f1 - -256 bit - -248d6a61 d20638b8 e5c02693 0c3e6039 a33ce459 64ff2167 f6ecedd4 19db06c1 - -512 bit - -8e959b75dae313da 8cf4f72814fc143f 8f7779c6eb9f7fa1 7299aeadb6889018 -501d289e4900f7e4 331b99dec4b5433a c7d329eeb6dd2654 5e96e55b874be909 - -384 bit - -09330c33f71147e8 3d192fc782cd1b47 53111b173b3b05d2 2fa08086e3b0f712 -fcc7c71a557e2db9 66c3e9fa91746039 -*/ -/* -#include - -char test160[]="abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"; -char test256[]="abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"; -char test512[]="abcdefghbcdefghicdefghijdefghijkefghijklfghijklmghijklmnhijklmnoijklmnopjklmnopqklmnopqrlmnopqrsmnopqrstnopqrstu"; - -int main() -{ - char digest[100]; - int i; - - hash256 sh256; - hash384 sh384; - hash512 sh512; - sha3 SHA3; - - HASH256_init(&sh256); - for (i=0;test256[i]!=0;i++) HASH256_process(&sh256,test256[i]); - HASH256_hash(&sh256,digest); - for (i=0;i<32;i++) printf("%02x",(unsigned char)digest[i]); - printf("\n"); - - HASH384_init(&sh384); - for (i=0;test512[i]!=0;i++) HASH384_process(&sh384,test512[i]); - HASH384_hash(&sh384,digest); - for (i=0;i<48;i++) printf("%02x",(unsigned char)digest[i]); - printf("\n"); - - HASH512_init(&sh512); - for (i=0;test512[i]!=0;i++) HASH512_process(&sh512,test512[i]); - HASH512_hash(&sh512,digest); - for (i=0;i<64;i++) printf("%02x",(unsigned char)digest[i]); - printf("\n"); - - SHA3_init(&SHA3,SHA3_HASH256); - for (i=0;test512[i]!=0;i++) SHA3_process(&SHA3,test512[i]); - SHA3_hash(&SHA3,digest); - for (i=0;i<32;i++) printf("%02x",(unsigned char)digest[i]); - printf("\n"); - - SHA3_init(&SHA3,SHA3_HASH512); - for (i=0;test512[i]!=0;i++) SHA3_process(&SHA3,test512[i]); - SHA3_hash(&SHA3,digest); - for (i=0;i<64;i++) printf("%02x",(unsigned char)digest[i]); - printf("\n"); - - SHA3_init(&SHA3,SHAKE256); - for (i=0;test512[i]!=0;i++) SHA3_process(&SHA3,test512[i]); - SHA3_shake(&SHA3,digest,72); - for (i=0;i<72;i++) printf("%02x",(unsigned char)digest[i]); - printf("\n"); - - - return 0; -} - -*/ diff --git a/impl/cbits/hmac.c b/impl/cbits/hmac.c deleted file mode 100644 index 7a790c35c..000000000 --- a/impl/cbits/hmac.c +++ /dev/null @@ -1,359 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/* - HMAC functions -*/ - -#include "arch.h" -#include "core.h" - -#define ROUNDUP(a,b) ((a)-1)/(b)+1 -#define CEIL(a,b) (((a)-1)/(b)+1) - -/* General Purpose hash function, padding with zeros, optional input octets p and x, optional integer n,hash to octet w of length olen */ -/* hash is the Hash family, either MC_SHA2 or MC_SHA3 */ -/* hlen should be 32,48 or 64 for MC_SHA2 (that is SHA256/384/512) */ -/* hlen should be 24,32,48,64 for MC_SHA3 */ -/* olen=0 - output = hlen bytes */ -/* olen<=hlen - output = olen bytes */ -/* olen>hlen - output is padded with leading zeros and then hlen bytes */ - - -void GPhash(int hash,int hlen,octet *w,int olen,int pad,octet *p,int n,octet *x) -{ - hash256 sh256; - hash384 sh384; - hash512 sh512; - sha3 sh3; - int i,c[4]; - char hh[64]; - - if (n>=0) - { - c[0] = (n >> 24) & 0xff; - c[1] = (n >> 16) & 0xff; - c[2] = (n >> 8) & 0xff; - c[3] = (n) & 0xff; - } - - switch (hash) - { - case MC_SHA2 : - switch (hlen) - { - case SHA256 : - HASH256_init(&sh256); - for (i=0;ilen;i++) HASH256_process(&sh256,p->val[i]); - if (n>=0) - for (i=0;i<4;i++) HASH256_process(&sh256,c[i]); - if (x!=NULL) - for (i=0;ilen;i++) HASH256_process(&sh256,x->val[i]); - HASH256_hash(&sh256,hh); - break; - case SHA384 : - HASH384_init(&sh384); - for (i=0;ilen;i++) HASH384_process(&sh384,p->val[i]); - if (n>=0) - for (i=0;i<4;i++) HASH384_process(&sh384,c[i]); - if (x!=NULL) - for (i=0;ilen;i++) HASH384_process(&sh384,x->val[i]); - HASH384_hash(&sh384,hh); - break; - case SHA512 : - HASH512_init(&sh512); - for (i=0;ilen;i++) HASH512_process(&sh512,p->val[i]); - if (n>=0) - for (i=0;i<4;i++) HASH512_process(&sh512,c[i]); - if (x!=NULL) - for (i=0;ilen;i++) HASH512_process(&sh512,x->val[i]); - HASH512_hash(&sh512,hh); - break; - } - break; - case MC_SHA3 : - SHA3_init(&sh3,hlen); - for (i=0;ilen;i++) SHA3_process(&sh3,p->val[i]); - if (n>=0) - for (i=0;i<4;i++) SHA3_process(&sh3,c[i]); - if (x!=NULL) - for (i=0;x->len;i++) SHA3_process(&sh3,x->val[i]); - SHA3_hash(&sh3,hh); - break; - default: return; - } - OCT_empty(w); - if (!olen) - OCT_jbytes(w,hh,hlen); - else - { - if (olen<=hlen) - { - OCT_jbytes(w,hh,olen); - } else { - OCT_jbyte(w, 0, olen - hlen); - OCT_jbytes(w, hh, hlen); - } - } -} - -/* Simple hash octet p to octet w of length hlen */ -void SPhash(int hash, int hlen,octet *w, octet *p) -{ - GPhash(hash, hlen, w, 0, 0, p, -1, NULL); -} - - -static int blksize(int hash,int hlen) -{ - int blk=0; - switch (hash) - { - case MC_SHA2 : - blk=64; - if (hlen>32) blk=128; - break; - case MC_SHA3 : - blk=200-2*hlen; - break; - default: break; - } - return blk; -} - -/* RFC 2104 */ -void HMAC(int hash,int hlen,octet *TAG,int olen,octet *K,octet *M) -{ - int blk; - char h[128],k0[200]; // assumes max block sizes - octet K0 = {0, sizeof(k0), k0}; - octet H={0,sizeof(h),h}; - - blk=blksize(hash,hlen); - if (blk==0) return; - - if (K->len > blk) SPhash(hash,hlen,&K0,K); - else OCT_copy(&K0,K); - OCT_jbyte(&K0,0,blk-K0.len); - OCT_xorbyte(&K0,0x36); - - GPhash(hash,hlen,&H,0,0,&K0,-1,M); - - OCT_xorbyte(&K0,0x6a); /* 0x6a = 0x36 ^ 0x5c */ - GPhash(hash,hlen,&H,0,0,&K0,-1,&H); - - OCT_empty(TAG); - OCT_jbytes(TAG,H.val,olen); - - OCT_clear(&H); - OCT_clear(&K0); -} - -/* RFC 5869 */ - -void HKDF_Extract(int hash,int hlen,octet *PRK,octet *SALT,octet *IKM) -{ - char h[64]; - octet H={0,sizeof(h),h}; - if (SALT==NULL) { - OCT_jbyte(&H,0,hlen); - HMAC(hash,hlen,PRK,hlen,&H,IKM); - } else { - HMAC(hash,hlen,PRK,hlen,SALT,IKM); - } -} - -void HKDF_Expand(int hash,int hlen,octet *OKM,int olen,octet *PRK,octet *INFO) -{ - int i; - char t[1024]; // >= info.length+hlen+1 - octet T={0,sizeof(t),t}; - int n=olen/hlen; - int flen=olen%hlen; - - OCT_empty(OKM); - for (i=1;i<=n;i++) - { - OCT_joctet(&T,INFO); - OCT_jbyte(&T,i,1); - HMAC(hash,hlen,&T,hlen,PRK,&T); - OCT_joctet(OKM,&T); - } - if (flen>0) - { - OCT_joctet(&T,INFO); - OCT_jbyte(&T,n+1,1); - HMAC(hash,hlen,&T,flen,PRK,&T); - OCT_joctet(OKM,&T); - } -} - -/* https://datatracker.ietf.org/doc/draft-irtf-cfrg-hash-to-curve/ */ - -void XOF_Expand(int hlen,octet *OKM,int olen,octet *DST,octet *M) -{ - int i; - sha3 SHA3; - SHA3_init(&SHA3,hlen); - for (i=0;ilen;i++) SHA3_process(&SHA3,M->val[i]); - SHA3_process(&SHA3,olen/256); - SHA3_process(&SHA3,olen%256); - - for (i=0;ilen;i++) - SHA3_process(&SHA3,DST->val[i]); - SHA3_process(&SHA3,DST->len); - - - SHA3_shake(&SHA3,OKM->val,olen); - OKM->len=olen; -} - -void XMD_Expand(int hash, int hlen,octet *OKM,int olen,octet *DST,octet *M) -{ - int i,blk; - int ell=CEIL(olen,hlen); - char tmp[260]; - octet TMP={0,sizeof(tmp),tmp}; - char h0[64]; - octet H0 = {0, sizeof(h0), h0}; - char h1[64]; - octet H1 = {0, sizeof(h1), h1}; - - blk=blksize(hash,hlen); - OCT_jint(&TMP,olen,2); - OCT_jint(&TMP,0,1); - OCT_joctet(&TMP,DST); - OCT_jint(&TMP,DST->len,1); - - GPhash(hash,hlen,&H0,0,blk,M,-1,&TMP); - OCT_empty(&TMP); - OCT_jint(&TMP,1,1); - OCT_joctet(&TMP,DST); - OCT_jint(&TMP,DST->len,1); - - GPhash(hash,hlen,&H1,0,0,&H0,-1,&TMP); - OCT_empty(OKM); - OCT_joctet(OKM,&H1); - for (i=2;i<=ell;i++) - { - OCT_xor(&H1,&H0); - OCT_empty(&TMP); - OCT_jint(&TMP,i,1); - OCT_joctet(&TMP,DST); - OCT_jint(&TMP,DST->len,1); - GPhash(hash,hlen,&H1,0,0,&H1,-1,&TMP); - OCT_joctet(OKM,&H1); - } - OKM->len=olen; -} - -/* Key Derivation Function */ - -void KDF2(int hash, int hlen, octet *key, int olen, octet *z, octet *p) -{ - /* NOTE: the parameter olen is the length of the output k in bytes */ - char h[64]; - octet H = {0, sizeof(h), h}; - int counter, cthreshold; - - OCT_empty(key); - - cthreshold = ROUNDUP(olen, hlen); - - for (counter = 1; counter <= cthreshold; counter++) - { - GPhash(hash,hlen, &H, 0, 0, z, counter, p); - if (key->len + hlen > olen) OCT_jbytes(key, H.val, olen % hlen); - else OCT_joctet(key, &H); - } - -} - -/* Password based Key Derivation Function */ -/* Input password p, salt s, and repeat count */ -/* Output key of length olen */ -void PBKDF2(int hash, int hlen, octet *key, int olen, octet *p, octet *s, int rep) -{ - int i, j, len, d = ROUNDUP(olen, hlen); - char f[64], u[64]; - octet F = {0, sizeof(f), f}; - octet U = {0, sizeof(u), u}; - OCT_empty(key); - - for (i = 1; i <= d; i++) - { - len = s->len; - OCT_jint(s, i, 4); - - HMAC(hash, hlen, &F, hlen, s, p); - - s->len = len; - OCT_copy(&U, &F); - for (j = 2; j <= rep; j++) - { - HMAC(hash, hlen, &U, hlen, &U, p); - OCT_xor(&F, &U); - } - - OCT_joctet(key, &F); - } - - OCT_chop(key, NULL, olen); -} - -/* gcc -O2 hmac.c oct.c hash.c rand.c -o hmac - -int main() -{ - char ikm[22],salt[13],prk[32],info[10],okm[50]; - octet IKM = {0, sizeof(ikm), ikm}; - octet SALT={0,sizeof(salt),salt}; - octet PRK={0,sizeof(prk),prk}; - octet OKM={0,sizeof(okm),okm}; - octet INFO={0,sizeof(info),info}; - int i; - for (i=0;i<22;i++) IKM.val[i]=0x0b; - for (i=0;i<13;i++) SALT.val[i]=i; - for (i=0;i<10;i++) INFO.val[i]=0xf0+i; - - IKM.len=22; SALT.len=13; INFO.len=10; - - printf("IKM= "); OCT_output(&IKM); - printf("SALT= "); OCT_output(&SALT); - - HKDF_Extract(MC_SHA2,32,&PRK,&SALT,&IKM); - - //HMAC(&PRK,32,&SALT,&IKM,SHA2,32); - - printf("PRK= "); OCT_output(&PRK); - - HKDF_Expand(MC_SHA2,32,&OKM,42,&PRK,&INFO); - - printf("OKM= %d ",OKM.len); OCT_output(&OKM); -} -*/ - diff --git a/impl/cbits/hpke_BLS12381.h b/impl/cbits/hpke_BLS12381.h deleted file mode 100644 index addf65652..000000000 --- a/impl/cbits/hpke_BLS12381.h +++ /dev/null @@ -1,114 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file hpke.h - * @author Mike Scott - * @date 2nd December 2019 - * @brief HPKE Header file - * - * declares functions - * - */ - -#ifndef HPKE_BLS12381_H -#define HPKE_BLS12381_H - -#include "ecdh_BLS12381.h" - -//#define CONFIG_ID 0x2A // 01|01|010 = 1, 1, 2 -//#define KEM_ID 2 // Curve X25519 -//#define KEM_ID 3 // Curve X448 -//#define KDF_ID 1 // HKDF-SHA256 -//#define AEAD_ID 1 // AES-GCM-128 - -#define HPKE_OK 0 /**< Function completed without error */ -#define HPKE_INVALID_PUBLIC_KEY -2 /**< Public Key is Invalid */ -#define HPKE_ERROR -3 /**< HPKE Internal Error */ - -/* HPKE DHKEM primitives */ - -/** @brief Derive a Key Pair from a seed - * - @param config_id is the configuration KEM/KDF/AEAD - @param SK is the output secret key - @param PK is the output public key - @param SEED is the input random seed - @return 1 if OK, 0 if failed - */ -extern int DeriveKeyPair_BLS12381(int config_id,octet *SK,octet *PK,octet *SEED); - -/** @brief Encapsulate function - * - @param config_id is the configuration KEM/KDF/AEAD - @param SK is the input ephemeral secret - @param Z is a pointer to a shared secret DH(skE,pkR) - @param pkE the ephemeral public key, which is skE.G, where G is a fixed generator - @param pkR the respondents public key - */ -extern void HPKE_BLS12381_Encap(int config_id,octet *SK,octet *Z,octet *pkE,octet *pkR); - -/** @brief Decapsulate function - * - @param config_id is the configuration KEM/KDF/AEAD - @param skR the respondents private key - @param Z is a pointer to a shared secret DH(skR,pkE) - @param pkE the ephemeral public key - @param pkR the respondents private key - */ -extern void HPKE_BLS12381_Decap(int config_id,octet *skR,octet *Z,octet *pkE,octet *pkR); - -/** @brief Encapsulate/Authenticate function - * - @param config_id is the configuration KEM/KDF/AEAD - @param skE is the input ephemeral secret - @param skS is the Initiators private key - @param Z is a pointer to a shared secret DH(skE,pkR) - @param pkE the ephemeral public key, which is skE.G, where G is a fixed generator - @param pkR the Respondents public key - @param pkS the Initiators public key - */ -extern void HPKE_BLS12381_AuthEncap(int config_id,octet *skE,octet *skS,octet *Z,octet *pkE,octet *pkR,octet *pkS); - -/** @brief Decapsulate function - * - @param config_id is the configuration KEM/KDF/AEAD - @param skR is the Respondents private key - @param Z is a pointer to a shared secret DH(skR,pkE) - @param pkE the ephemeral public key - @param pkR the Respondents public key - @param pkS the Initiators public key - */ -extern void HPKE_BLS12381_AuthDecap(int config_id,octet *skR,octet *Z,octet *pkE,octet *pkR,octet *pkS); - -/** @brief KeyScheduler function - * - @param config_id is the configuration KEM/KDF/AEAD - @param key the output key for aead encryption - @param nonce the output nonce for aead encryption - @param exp_secret the exporter secret - @param mode the mode of operation - @param Z the shared key - @param info application dependent info - @param psk pre-shared key - @param pskID identifier for the psk - */ -extern void HPKE_BLS12381_KeySchedule(int config_id,octet *key,octet *nonce,octet *exp_secret,int mode,octet *Z,octet *info,octet *psk,octet *pskID); - -#endif diff --git a/impl/cbits/mpin_BLS12381.h b/impl/cbits/mpin_BLS12381.h deleted file mode 100644 index fdaf604ae..000000000 --- a/impl/cbits/mpin_BLS12381.h +++ /dev/null @@ -1,130 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file mpin.h - * @author Mike Scott and Kealan McCusker - * @date 2nd June 2015 - * @brief M-Pin Header file - * - * Allows some user configuration - * defines structures - * declares functions - * - */ - -#ifndef MPIN_BLS12381_H -#define MPIN_BLS12381_H - -#include "pair_BLS12381.h" - -/* Field size is assumed to be greater than or equal to group size */ - -#define PGS_BLS12381 MODBYTES_384_58 /**< MPIN Group Size */ -#define PFS_BLS12381 MODBYTES_384_58 /**< MPIN Field Size */ - -#define MPIN_OK 0 /**< Function completed without error */ -#define MPIN_INVALID_POINT -14 /**< Point is NOT on the curve */ -#define MPIN_BAD_PIN -19 /**< Bad PIN number entered */ - -#define MAXPIN 10000 /**< max PIN */ -#define PBLEN 14 /**< max length of PIN in bits */ - -//#define PAS_BLS12381 16 /**< MPIN Symmetric Key Size 128 bits */ -//#define HASH_TYPE_MPIN_BLS12381 SHA256 /**< Choose Hash function */ - -/* MPIN support functions */ - -/* MPIN primitives */ - -/** @brief Encode a string to a curve point (in constant time) - * - @param DST is the Domain Separation Tag - @param ID is the input string - @param HID is the output point in G1 -*/ -void MPIN_BLS12381_ENCODE_TO_CURVE(octet *DST,octet *ID,octet *HID); - -/** @brief Extract a PIN number from a client secret - * - @param HID is the hashed-to-curve input client identity - @param pin is an input PIN number - @param CS is the client secret from which the PIN is to be extracted - @return 0 or an error code - */ -int MPIN_BLS12381_EXTRACT_PIN(octet *HID, int pin, octet *CS); - -/** @brief Perform first pass of the client side of the 3-pass version of the M-Pin protocol - * - @param HID is the hashed-to-curve input client identity - @param R is a pointer to a cryptographically secure random number generator - @param x an output internally randomly generated if R!=NULL, otherwise must be provided as an input - @param pin is the input PIN number - @param T is the input M-Pin token (the client secret with PIN portion removed) - @param S is the reconstructed client secret - @param U is output = x.H(ID) - @return 0 or an error code - */ -int MPIN_BLS12381_CLIENT_1(octet *HID, csprng *R, octet *x, int pin, octet *T, octet *S, octet *U); -/** @brief Generate a random group element - * - @param R is a pointer to a cryptographically secure random number generator - @param S is the output random octet - @return 0 or an error code - */ -int MPIN_BLS12381_RANDOM_GENERATE(csprng *R, octet *S); -/** @brief Perform second pass of the client side of the 3-pass version of the M-Pin protocol - * - @param x an input, a locally generated random number - @param y an input random challenge from the server - @param V on output = -(x+y).V - @return 0 or an error code - */ -int MPIN_BLS12381_CLIENT_2(octet *x, octet *y, octet *V); - -/** @brief Perform final pass on the server side of the M-Pin protocol - - @param HID is input H(ID), a hash of the client ID - @param y is the input server's randomly generated challenge - @param SS is the input server secret - @param U is input from the client = x.H(ID) - @param V is an input from the client - @return 0 or an error code - */ -int MPIN_BLS12381_SERVER(octet *HID, octet *y, octet *SS, octet *U, octet *V); - -/** @brief Create a client secret in G1 from a master secret and the client ID - * - @param S is an input master secret - @param HID is the input client identity hashed to curve - @param CS is the full client secret = s.H(ID) - @return 0 or an error code - */ -int MPIN_BLS12381_GET_CLIENT_SECRET(octet *S, octet *HID, octet *CS); - -/** @brief Create a server secret in G2 from a master secret - * - @param S is an input master secret - @param SS is the server secret = s.Q where Q is a fixed generator of G2 - @return 0 or an error code - */ -int MPIN_BLS12381_GET_SERVER_SECRET(octet *S, octet *SS); - -#endif - diff --git a/impl/cbits/newhope.c b/impl/cbits/newhope.c deleted file mode 100644 index aa9a0ac56..000000000 --- a/impl/cbits/newhope.c +++ /dev/null @@ -1,510 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/* NewHope API implementation. Constant time. - - LOOK - no if statements! - - M.Scott 21/07/2017 -*/ - -#include "newhope.h" - -const sign16 roots[] = {0x2ac8, 0x2baf, 0x299b, 0x685, 0x2f04, 0x158d, 0x2d49, 0x24b5, 0x1edc, 0xab3, 0x2a95, 0x24d, 0x3cb, 0x6a8, 0x12f9, 0x15ba, 0x1861, 0x2a89, 0x1c5c, 0xbe6, 0xc1e, 0x2024, 0x207, 0x19ce, 0x2710, 0x1744, 0x18bc, 0x2cd7, 0x396, 0x18d5, 0x1c45, 0xc4, 0x21a6, 0xe03, 0x2b3c, 0x2d91, 0xc5d, 0x432, 0x1fbc, 0xcae, 0x2512, 0x2979, 0x3b2, 0x714, 0xb2e, 0x1a97, 0x1a03, 0x1bcd, 0x2216, 0x2701, 0xa, 0x263c, 0x1179, 0x200c, 0x2d08, 0x1c34, 0x291, 0x2c99, 0x2a5a, 0x723, 0xb1d, 0x1ccc, 0x1fb6, 0x2f58, 0x2bfe, 0x1cda, 0x2a0, 0x5f1, 0x2de, 0x1fc7, 0x1ea8, 0x1719, 0x2fa7, 0x27ec, 0x20ff, 0x12c0, 0x1ac1, 0x2232, 0x2f9b, 0xd3e, 0x2aed, 0x15f0, 0x11e8, 0xed0, 0x26a, 0x1de5, 0xa3f, 0xf43, 0xebf, 0x204e, 0xac7, 0x2d9c, 0x5ea, 0x25d1, 0xb6, 0x49c, 0x995, 0x2555, 0x26e2, 0x100, 0x1878, 0x5aa, 0x2e10, 0x271c, 0xcb, 0x1b4c, 0x2fb8, 0x25b7, 0x1543, 0x2c7b, 0x241a, 0x2223, 0x20ca, 0x24ed, 0x137, 0x1b65, 0x1dc2, 0x7c7, 0x2ec3, 0xd0c, 0x1169, 0x1c7a, 0x1ea1, 0xf89, 0x2199, 0x291d, 0x1088, 0x2046, 0x256d, 0x2bc7, 0x2e9b, 0x41f, 0x1b55, 0x2b38, 0xd0, 0x2e6a, 0x1755, 0x6bc, 0x2724, 0x3ba, 0x222e, 0x2c5c, 0x2da5, 0x213c, 0x10fe, 0x169a, 0x1552, 0x5d3, 0x300, 0x1b5d, 0x1342, 0x2004, 0x256f, 0x2039, 0x667, 0x23b5, 0x1123, 0xdb, 0x2da0, 0xe1e, 0x2f54, 0x2767, 0x154a, 0x40a, 0x11d3, 0x2821, 0xc09, 0x974, 0x694, 0xfbf, 0x27ba, 0x132, 0x83f, 0x2d06, 0x10e, 0x183f, 0x29ae, 0x28c3, 0x2dc9, 0x1144, 0x2c70, 0x2a4a, 0xf3c, 0x1e32, 0x1171, 0x1e43, 0xdd4, 0x2ddf, 0x28d2, 0xfac, 0x3c4, 0x2f19, 0x10a6, 0x2f7, 0xe1d, 0x828, 0x138f, 0x1332, 0xfab, 0xcf6, 0x13f8, 0x24a0, 0x112d, 0x2717, 0x6e7, 0x1044, 0x36e, 0xfe8, 0x6a, 0xba7, 0x1d69, 0x29ec, 0x23b2, 0xaee, 0x16df, 0x1068, 0x1a7e, 0x253f, 0x24c, 0xb33, 0x2683, 0x15ce, 0x1ad3, 0x1a36, 0xc96, 0xaea, 0x260a, 0xce, 0x28b1, 0xe4f, 0x2b11, 0x5f8, 0x1fc4, 0xe77, 0x2366, 0x11f9, 0x153c, 0x24eb, 0x20cd, 0x1398, 0x22, 0x2b97, 0x249b, 0x8eb, 0x12b2, 0x2fe3, 0x29c1, 0x1b00, 0x2663, 0xeaa, 0x2e06, 0xe0, 0x1569, 0x10f5, 0x284e, 0xa38, 0x201d, 0x1c53, 0x1681, 0x1f6f, 0x2f95, 0x2fe8, 0xacb, 0x1680, 0x17fd, 0x2c39, 0x165a, 0x10bb, 0x29d8, 0x2622, 0x1196, 0x884, 0x2a79, 0x140e, 0x2d80, 0x6fa, 0x11b2, 0x26c4, 0x355, 0x1054, 0x29e9, 0x23ed, 0xbe3, 0x24fa, 0x1fb3, 0x10ac, 0x2919, 0x2584, 0x10a4, 0xe85, 0x650, 0x1893, 0x1dc1, 0xd8e, 0x12dc, 0x2d42, 0x284d, 0xfff, 0x250f, 0xacd, 0x13c3, 0x6cc, 0x1a79, 0x1221, 0x2614, 0x270a, 0x1ea, 0x155, 0x2818, 0x222c, 0x2e5b, 0x25d8, 0x1dbf, 0x191c, 0xb0f, 0xdac, 0x1082, 0x12ef, 0x11b6, 0xfa8, 0x2b72, 0x159d, 0x209e, 0x31b, 0x2c7c, 0x14f7, 0xe09, 0x1bb2, 0x1ec7, 0x2404, 0x20ae, 0x6ad, 0xed6, 0x2b70, 0x1c7b, 0x18d1, 0x2732, 0x12da, 0xd56, 0x5c1, 0x1648, 0x18b7, 0x1605, 0x1bc4, 0x280, 0x2ece, 0xc, 0x1aae, 0x1c4, 0x1cdb, 0x22d6, 0x21d8, 0x257c, 0x51f, 0x211b, 0xff, 0x2ee0, 0x2585, 0xe1, 0x2c35, 0x26db, 0x2971, 0x2208, 0x17e1, 0x21be, 0x135e, 0x28d6, 0x2891, 0x1689, 0x2138, 0xb86, 0x2e3a, 0x1204, 0x2d10, 0x2324, 0xf3f, 0x2508, 0x33d, 0xcb2, 0x292a, 0xe27, 0x2e64, 0x29f8, 0x2d46, 0x9b7, 0x20eb, 0x1b7c, 0x9eb, 0x2b2a, 0x58c, 0x27d0, 0x121b, 0x272e, 0x29f6, 0x2dbd, 0x2697, 0x2aac, 0xd6f, 0x1c67, 0x2c5b, 0x108d, 0x363, 0x249d, 0x2d5e, 0x2fd, 0x2cb2, 0x1f8f, 0x20a4, 0xa19, 0x2ac9, 0x19b1, 0x1581, 0x17a2, 0x29eb, 0x1b72, 0x13b0, 0xee4, 0xa8f, 0x2315, 0x5e6, 0x951, 0x2e29, 0xdad, 0x1f2b, 0x224e, 0x37f, 0x1a72, 0xa91, 0x1407, 0x2df9, 0x3ad, 0x23f7, 0x1a24, 0x1d2a, 0x234b, 0x1df3, 0x1143, 0x7ff, 0x1a6d, 0x2774, 0x2690, 0x2ab5, 0x586, 0x2781, 0x2009, 0x2fdd, 0x2881, 0x399, 0x2fb6, 0x144, 0x137f, 0xfa0, 0x2e4c, 0x1c7f, 0x2fac, 0xb09, 0x1264, 0x127b, 0x198c, 0x2b40, 0x230, 0x1cf4, 0x180b, 0xb58, 0x144a, 0x2aec, 0xfb, 0x2602, 0x14ee, 0x783, 0x1098, 0x23d8, 0x203, 0xe9, 0x108a, 0x14b8, 0xeec, 0xc58, 0x1248, 0x243c, 0x28aa, 0x6bf, 0x27c4, 0x276e, 0x19b8, 0x1d11, 0x2e16, 0x472, 0x1464, 0x24b9, 0x662, 0x1097, 0x2067, 0x20d6, 0x171c, 0x4, 0x682, 0x17bb, 0x1186, 0x4f2, 0x3ff, 0x2a43, 0x1dc7, 0x1ae5, 0x8cc, 0x2e7c, 0x2ef8, 0x2ae0, 0x2904, 0xed4, 0x6c5, 0x14ae, 0xb72, 0x11c3, 0x337, 0x2da3, 0x2916, 0x6d8, 0x1cf9, 0x10ee, 0x1800, 0x1ae4, 0xa0d, 0x101b, 0x1a8d, 0x2e98, 0x24cd, 0x813, 0x1aa4, 0x9b9, 0x680, 0x2349, 0x24d1, 0x20f8, 0xe31, 0x249f, 0x216b, 0x12d9, 0x1d21, 0x19db, 0x191a, 0x1dd0, 0x5df, 0x55c, 0x2b86, 0x213, 0xe9e, 0x1ef1, 0x268a, 0x1d5e, 0x1e20, 0x28c1, 0x1379, 0x249, 0x19de, 0x18b, 0x1e41, 0x2a1e, 0x2612, 0x297, 0x2e96, 0x2102, 0x46, 0x1b9f, 0x1a4d, 0x2050, 0x1b32, 0x568, 0x11f7, 0x1829, 0x870, 0x1f4, 0x1dca, 0x990, 0x1df6, 0x2b62, 0x13ec, 0x9f2, 0x1260, 0x2997, 0x1412, 0x1e6d, 0x1694, 0x11ac, 0x2d8b, 0x276f, 0x26f5, 0x233e, 0x2b44, 0x2f5a, 0x2d37, 0x2cb1, 0xc75, 0x98d, 0x1d56, 0x7ae, 0x10e6, 0x113f, 0x17b8, 0xad3, 0x737, 0x221e, 0x1b70, 0x1f3e, 0x2966, 0x18b2, 0x4fa, 0x2044, 0x1312, 0x154e, 0x2029, 0x700, 0x1b45, 0x27a6, 0x226a, 0x21bf, 0x58d, 0x2f11, 0x2e02, 0x17fc, 0x4d2, 0x1757, 0xcb1, 0x2ef1, 0x2582, 0x1276, 0x881, 0x2fc0, 0x104a, 0x670, 0x274f, 0x2b53, 0x19dd, 0x752, 0x1663, 0xcbd, 0x2b2b, 0x2fc6, 0x13b6, 0x21e6, 0x15f6, 0x126b, 0x2637, 0x1cd9, 0x2f50, 0xe82, 0x5b0, 0x24e0, 0x1350, 0x2f24, 0x21f7, 0x1a16, 0x2f3e, 0x167e, 0x1f7d, 0x28a0, 0x16f0, 0xe33, 0x53b, 0x28c5, 0x1500, 0x2f88, 0x26cc, 0x2018, 0x1604, 0x218b, 0x2cd1, 0x9ee, 0x17f3, 0x5fd, 0x1f5a, 0x2d0, 0x2b46, 0x23cc, 0x503, 0x1c46, 0x1cc3, 0x28e2, 0x243e, 0x122b, 0x2e0c, 0xe37, 0x2611, 0x85e, 0x9b8, 0x1b24, 0x762, 0x19b6, 0x3bc, 0x2d50, 0x2079, 0x18da, 0x170a, 0x800, 0xaa2, 0x135a, 0x1a15, 0x13d1, 0xca, 0x2113, 0x2db9, 0xdb2, 0x1a5c, 0x29a9, 0x1488, 0x14c1, 0x2c9, 0x917, 0x28e7, 0x265c, 0xdab, 0x2ab9, 0x2bc6, 0x105b, 0x1839, 0x219c, 0x50, 0x11da, 0x1802, 0xf56, 0x2e6, 0x2190, 0xddb, 0x56e, 0x9d9, 0x1c81, 0x1016, 0x12d6, 0x296f, 0x14b4, 0x1014, 0x1e64, 0x1d90, 0x89f, 0x2bc2, 0x2777, 0x2819, 0x1c65, 0x1a41, 0x5a2, 0x2cd2, 0x427, 0xd71, 0x29c8, 0x1e58, 0x53f, 0x7c5, 0x1dcd, 0x4a1, 0x1268, 0x2597, 0x2926, 0xee, 0x111b, 0x1038, 0xe6c, 0x22dc, 0x2f2f, 0x441, 0x2cfd, 0x1cb0, 0x6a4, 0x2224, 0x620, 0x5dc, 0x16b1, 0x2a1d, 0x1787, 0x20c7, 0x641, 0xd84, 0x1c05, 0x2d0d, 0x2f52, 0x1b8c, 0xd7d, 0x17e8, 0x1589, 0xc73, 0x151b, 0x4e2, 0x1ae9, 0x1b18, 0xb9b, 0x949, 0x2c60, 0x1e7a, 0xd5, 0x1bdc, 0x1f57, 0x1753, 0x124a, 0x559, 0xb76, 0x2334, 0x12d1, 0x1de1, 0x14b2, 0x2faa, 0x1697, 0x147a, 0x5a1, 0x2c30, 0x1c02, 0x1043, 0x2ee1, 0x2402, 0x1cc8, 0x2a16, 0xff7, 0x1364, 0x1b9a, 0x2a53, 0x2f94, 0x294c, 0x1ee5, 0x1a87, 0x2141, 0xd66, 0x953, 0x28a3, 0x2f30, 0x2477, 0x18e3, 0x1035, 0x1fc1, 0x1d68, 0x2fb3, 0x138c, 0x2487, 0x1bf8, 0xd96, 0x1018, 0x748, 0x244e, 0x15bd, 0x175e, 0x2be, 0x23d, 0x1da, 0x176d, 0xc17, 0x24be, 0x2ebb, 0x7d8, 0x100a, 0x759, 0x1db4, 0x2259, 0x23f4, 0x2d59, 0x2847, 0xbf5, 0x1cfe, 0xa20, 0x258, 0x1180, 0x279c, 0x54, 0x2abf, 0xc5c, 0x9f9, 0x3d5, 0x2ce4, 0x165f, 0x23d9, 0x27b9, 0x6f9, 0x281a, 0x169e, 0x627, 0x156d, 0x1ff8, 0x211, 0x2e34, 0x1724, 0x2c2e, 0x2790, 0x2dd5, 0x2bf2, 0xdbc, 0x2884, 0x20a9, 0x2390, 0x1e1a, 0x1b6a, 0x5f7, 0xab7, 0x1333, 0x16ab, 0x28dd, 0x20, 0x30f, 0x24b6, 0x5c2, 0x1ce4, 0x1400, 0x2669, 0x60, 0x156c, 0xe20, 0x26d4, 0x26ab, 0x1ebb, 0x223d, 0x5b4, 0x2025, 0x1e1c, 0xaae, 0x2e08, 0x6cd, 0x1677, 0x13d9, 0x17b5, 0x1046, 0x1d8c, 0x14eb, 0x18d8, 0x1ce5, 0x2478, 0x16ae, 0xb79, 0x23d4, 0x684, 0x156b, 0x567, 0x1a, 0x29ce, 0x83a, 0x19e8, 0x58e, 0x294a, 0x1136, 0x2319, 0x2fba, 0x1a29, 0x1d, 0x1879, 0x291b, 0x19f6, 0x2c2f, 0x21c9, 0x19bb, 0xbbc, 0x26f9, 0xc22, 0x708, 0x11a1, 0x18d3, 0x7f8, 0x28f8, 0x2427, 0x1deb, 0xaed, 0x26aa, 0x2482, 0x203b, 0x2f05, 0x2b82, 0x192f, 0x2df4, 0x8dc, 0x2877, 0xd5e, 0x240e, 0x775, 0x2dae, 0x1d3e, 0x20ba, 0x215b, 0x22d1, 0xeba, 0xf50, 0xaa8, 0x184a, 0x1f67, 0x2e04, 0xc6e, 0x6dd, 0x1a09, 0x27f, 0x494, 0x1426, 0xae3, 0xe15, 0x65f, 0x13c4, 0x105, 0x872, 0x2667, 0x1ff6, 0xd9f, 0x2ca1, 0x2f39, 0x2657, 0x23fd, 0x2405, 0xb73, 0x2294, 0x1f1e, 0x2eba, 0x110a, 0x2cae, 0x141f, 0x22cd, 0x25d6, 0x11c1, 0x1c, 0x2d8e, 0x161a, 0x1aa8, 0x229e, 0x1bf9, 0x7cf, 0x106d, 0x2c40, 0xd93, 0x255e, 0x28c2, 0xc1a, 0x2f17, 0x7ca, 0x2f63, 0xbf}; -const sign16 iroots[] = {0x2ac8, 0x452, 0x297c, 0x666, 0xb4c, 0x2b8, 0x1a74, 0xfd, 0x1a47, 0x1d08, 0x2959, 0x2c36, 0x2db4, 0x56c, 0x254e, 0x1125, 0x2f3d, 0x13bc, 0x172c, 0x2c6b, 0x32a, 0x1745, 0x18bd, 0x8f1, 0x1633, 0x2dfa, 0xfdd, 0x23e3, 0x241b, 0x13a5, 0x578, 0x17a0, 0xa9, 0x104b, 0x1335, 0x24e4, 0x28de, 0x5a7, 0x368, 0x2d70, 0x13cd, 0x2f9, 0xff5, 0x1e88, 0x9c5, 0x2ff7, 0x900, 0xdeb, 0x1434, 0x15fe, 0x156a, 0x24d3, 0x28ed, 0x2c4f, 0x688, 0xaef, 0x2353, 0x1045, 0x2bcf, 0x23a4, 0x270, 0x4c5, 0x21fe, 0xe5b, 0xfbb, 0x1f79, 0x6e4, 0xe68, 0x2078, 0x1160, 0x1387, 0x1e98, 0x22f5, 0x13e, 0x283a, 0x123f, 0x149c, 0x2eca, 0xb14, 0xf37, 0xdde, 0xbe7, 0x386, 0x1abe, 0xa4a, 0x49, 0x14b5, 0x2f36, 0x8e5, 0x1f1, 0x2a57, 0x1789, 0x2f01, 0x91f, 0xaac, 0x266c, 0x2b65, 0x2f4b, 0xa30, 0x2a17, 0x265, 0x253a, 0xfb3, 0x2142, 0x20be, 0x25c2, 0x121c, 0x2d97, 0x2131, 0x1e19, 0x1a11, 0x514, 0x22c3, 0x66, 0xdcf, 0x1540, 0x1d41, 0xf02, 0x815, 0x5a, 0x18e8, 0x1159, 0x103a, 0x2d23, 0x2a10, 0x2d61, 0x1327, 0x403, 0x25c9, 0x7b3, 0x1f0c, 0x1a98, 0x2f21, 0x1fb, 0x2157, 0x99e, 0x1501, 0x640, 0x1e, 0x1d4f, 0x2716, 0xb66, 0x46a, 0x2fdf, 0x1c69, 0xf34, 0xb16, 0x1ac5, 0x1e08, 0xc9b, 0x218a, 0x103d, 0x2a09, 0x4f0, 0x21b2, 0x750, 0x2f33, 0x9f7, 0x2517, 0x236b, 0x15cb, 0x152e, 0x1a33, 0x97e, 0x24ce, 0x2db5, 0xac2, 0x1583, 0x1f99, 0x1922, 0x2513, 0xc4f, 0x615, 0x1298, 0x245a, 0x2f97, 0x2019, 0x2c93, 0x1fbd, 0x291a, 0x8ea, 0x1ed4, 0xb61, 0x1c09, 0x230b, 0x2056, 0x1ccf, 0x1c72, 0x27d9, 0x21e4, 0x2d0a, 0x1f5b, 0xe8, 0x2c3d, 0x2055, 0x72f, 0x222, 0x222d, 0x11be, 0x1e90, 0x11cf, 0x20c5, 0x5b7, 0x391, 0x1ebd, 0x238, 0x73e, 0x653, 0x17c2, 0x2ef3, 0x2fb, 0x27c2, 0x2ecf, 0x847, 0x2042, 0x296d, 0x268d, 0x23f8, 0x7e0, 0x1e2e, 0x2bf7, 0x1ab7, 0x89a, 0xad, 0x21e3, 0x261, 0x2f26, 0x1ede, 0xc4c, 0x299a, 0xfc8, 0xa92, 0xffd, 0x1cbf, 0x14a4, 0x2d01, 0x2a2e, 0x1aaf, 0x1967, 0x1f03, 0xec5, 0x25c, 0x3a5, 0xdd3, 0x2c47, 0x8dd, 0x2945, 0x18ac, 0x197, 0x2f31, 0x4c9, 0x14ac, 0x2be2, 0x166, 0x43a, 0xa94, 0x1b53, 0x293c, 0x212d, 0x6fd, 0x521, 0x109, 0x185, 0x2735, 0x151c, 0x123a, 0x5be, 0x2c02, 0x2b0f, 0x1e7b, 0x1846, 0x297f, 0x2ffd, 0x18e5, 0xf2b, 0xf9a, 0x1f6a, 0x299f, 0xb48, 0x1b9d, 0x2b8f, 0x1eb, 0x12f0, 0x1649, 0x893, 0x83d, 0x2942, 0x757, 0xbc5, 0x1db9, 0x23a9, 0x2115, 0x1b49, 0x1f77, 0x2f18, 0x2dfe, 0xc29, 0x1f69, 0x287e, 0x1b13, 0x9ff, 0x2f06, 0x515, 0x1bb7, 0x24a9, 0x17f6, 0x130d, 0x2dd1, 0x4c1, 0x1675, 0x1d86, 0x1d9d, 0x24f8, 0x55, 0x1382, 0x1b5, 0x2061, 0x1c82, 0x2ebd, 0x4b, 0x2c68, 0x780, 0x24, 0xff8, 0x880, 0x2a7b, 0x54c, 0x971, 0x88d, 0x1594, 0x2802, 0x1ebe, 0x120e, 0xcb6, 0x12d7, 0x15dd, 0xc0a, 0x2c54, 0x208, 0x1bfa, 0x2570, 0x158f, 0x2c82, 0xdb3, 0x10d6, 0x2254, 0x1d8, 0x26b0, 0x2a1b, 0xcec, 0x2572, 0x211d, 0x1c51, 0x148f, 0x616, 0x185f, 0x1a80, 0x1650, 0x538, 0x25e8, 0xf5d, 0x1072, 0x34f, 0x2d04, 0x2a3, 0xb64, 0x2c9e, 0x1f74, 0x3a6, 0x139a, 0x2292, 0x555, 0x96a, 0x244, 0x60b, 0x8d3, 0x1de6, 0x831, 0x2a75, 0x4d7, 0x2616, 0x1485, 0xf16, 0x264a, 0x2bb, 0x609, 0x19d, 0x21da, 0x6d7, 0x234f, 0x2cc4, 0xaf9, 0x20c2, 0xcdd, 0x2f1, 0x1dfd, 0x1c7, 0x247b, 0xec9, 0x1978, 0x770, 0x72b, 0x1ca3, 0xe43, 0x1820, 0xdf9, 0x690, 0x926, 0x3cc, 0x2f20, 0xa7c, 0x121, 0x2f02, 0xee6, 0x2ae2, 0xa85, 0xe29, 0xd2b, 0x1326, 0x2e3d, 0x1553, 0x2ff5, 0x133, 0x2d81, 0x143d, 0x19fc, 0x174a, 0x19b9, 0x2a40, 0x22ab, 0x1d27, 0x8cf, 0x1730, 0x1386, 0x491, 0x212b, 0x2954, 0xf53, 0xbfd, 0x113a, 0x144f, 0x21f8, 0x1b0a, 0x385, 0x2ce6, 0xf63, 0x1a64, 0x48f, 0x2059, 0x1e4b, 0x1d12, 0x1f7f, 0x2255, 0x24f2, 0x16e5, 0x1242, 0xa29, 0x1a6, 0xdd5, 0x7e9, 0x2eac, 0x2e17, 0x8f7, 0x9ed, 0x1de0, 0x1588, 0x2935, 0x1c3e, 0x2534, 0xaf2, 0x2002, 0x7b4, 0x2bf, 0x1d25, 0x2273, 0x1240, 0x176e, 0x29b1, 0x217c, 0x1f5d, 0xa7d, 0x6e8, 0x1f55, 0x104e, 0xb07, 0x241e, 0xc14, 0x618, 0x1fad, 0x2cac, 0x93d, 0x1e4f, 0x2907, 0x281, 0x1bf3, 0x588, 0x277d, 0x1e6b, 0x9df, 0x629, 0x1f46, 0x19a7, 0x3c8, 0x1804, 0x1981, 0x2536, 0x19, 0x6c, 0x1092, 0x1980, 0x13ae, 0xfe4, 0x2f42, 0x9e, 0x2837, 0xea, 0x23e7, 0x73f, 0xaa3, 0x226e, 0x3c1, 0x1f94, 0x2832, 0x1408, 0xd63, 0x1559, 0x19e7, 0x273, 0x2fe5, 0x1e40, 0xa2b, 0xd34, 0x1be2, 0x353, 0x1ef7, 0x147, 0x10e3, 0xd6d, 0x248e, 0xbfc, 0xc04, 0x9aa, 0xc8, 0x360, 0x2262, 0x100b, 0x99a, 0x278f, 0x2efc, 0x1c3d, 0x29a2, 0x21ec, 0x251e, 0x1bdb, 0x2b6d, 0x2d82, 0x15f8, 0x2924, 0x2393, 0x1fd, 0x109a, 0x17b7, 0x2559, 0x20b1, 0x2147, 0xd30, 0xea6, 0xf47, 0x12c3, 0x253, 0x288c, 0xbf3, 0x22a3, 0x78a, 0x2725, 0x20d, 0x16d2, 0x47f, 0xfc, 0xfc6, 0xb7f, 0x957, 0x2514, 0x1216, 0xbda, 0x709, 0x2809, 0x172e, 0x1e60, 0x28f9, 0x23df, 0x908, 0x2445, 0x1646, 0xe38, 0x3d2, 0x160b, 0x6e6, 0x1788, 0x2fe4, 0x15d8, 0x47, 0xce8, 0x1ecb, 0x6b7, 0x2a73, 0x1619, 0x27c7, 0x633, 0x2fe7, 0x2a9a, 0x1a96, 0x297d, 0xc2d, 0x2488, 0x1953, 0xb89, 0x131c, 0x1729, 0x1b16, 0x1275, 0x1fbb, 0x184c, 0x1c28, 0x198a, 0x2934, 0x1f9, 0x2553, 0x11e5, 0xfdc, 0x2a4d, 0xdc4, 0x1146, 0x956, 0x92d, 0x21e1, 0x1a95, 0x2fa1, 0x998, 0x1c01, 0x131d, 0x2a3f, 0xb4b, 0x2cf2, 0x2fe1, 0x724, 0x1956, 0x1cce, 0x254a, 0x2a0a, 0x1497, 0x11e7, 0xc71, 0xf58, 0x77d, 0x2245, 0x40f, 0x22c, 0x871, 0x3d3, 0x18dd, 0x1cd, 0x2df0, 0x1009, 0x1a94, 0x29da, 0x1963, 0x7e7, 0x2908, 0x848, 0xc28, 0x19a2, 0x31d, 0x2c2c, 0x2608, 0x23a5, 0x542, 0x2fad, 0x865, 0x1e81, 0x2da9, 0x25e1, 0x1303, 0x240c, 0x7ba, 0x2a8, 0xc0d, 0xda8, 0x124d, 0x28a8, 0x1ff7, 0x2829, 0x146, 0xb43, 0x23ea, 0x1894, 0x2e27, 0x2dc4, 0x2d43, 0x18a3, 0x1a44, 0xbb3, 0x28b9, 0x1fe9, 0x226b, 0x1409, 0xb7a, 0x1c75, 0x4e, 0x1299, 0x1040, 0x1fcc, 0x171e, 0xb8a, 0xd1, 0x75e, 0x26ae, 0x229b, 0xec0, 0x157a, 0x111c, 0x6b5, 0x6d, 0x5ae, 0x1467, 0x1c9d, 0x200a, 0x5eb, 0x1339, 0xbff, 0x120, 0x1fbe, 0x13ff, 0x3d1, 0x2a60, 0x1b87, 0x196a, 0x57, 0x1b4f, 0x1220, 0x1d30, 0xccd, 0x248b, 0x2aa8, 0x1db7, 0x18ae, 0x10aa, 0x1425, 0x2f2c, 0x1187, 0x3a1, 0x26b8, 0x2466, 0x14e9, 0x1518, 0x2b1f, 0x1ae6, 0x238e, 0x1a78, 0x1819, 0x2284, 0x1475, 0xaf, 0x2f4, 0x13fc, 0x227d, 0x29c0, 0xf3a, 0x187a, 0x5e4, 0x1950, 0x2a25, 0x29e1, 0xddd, 0x295d, 0x1351, 0x304, 0x2bc0, 0xd2, 0xd25, 0x2195, 0x1fc9, 0x1ee6, 0x2f13, 0x6db, 0xa6a, 0x1d99, 0x2b60, 0x1234, 0x283c, 0x2ac2, 0x11a9, 0x639, 0x2290, 0x2bda, 0x32f, 0x2a5f, 0x15c0, 0x139c, 0x7e8, 0x88a, 0x43f, 0x2762, 0x1271, 0x119d, 0x1fed, 0x1b4d, 0x692, 0x1d2b, 0x1feb, 0x1380, 0x2628, 0x2a93, 0x2226, 0xe71, 0x2d1b, 0x20ab, 0x17ff, 0x1e27, 0x2fb1, 0xe65, 0x17c8, 0x1fa6, 0x43b, 0x548, 0x2256, 0x9a5, 0x71a, 0x26ea, 0x2d38, 0x1b40, 0x1b79, 0x658, 0x15a5, 0x224f, 0x248, 0xeee, 0x2f37, 0x1c30, 0x15ec, 0x1ca7, 0x255f, 0x2801, 0x18f7, 0x1727, 0xf88, 0x2b1, 0x2c45, 0x164b, 0x289f, 0x14dd, 0x2649, 0x27a3, 0x9f0, 0x21ca, 0x1f5, 0x1dd6, 0xbc3, 0x71f, 0x133e, 0x13bb, 0x2afe, 0xc35, 0x4bb, 0x2d31, 0x10a7, 0x2a04, 0x180e, 0x2613, 0x330, 0xe76, 0x19fd, 0xfe9, 0x935, 0x79, 0x1b01, 0x73c, 0x2ac6, 0x21ce, 0x1911, 0x761, 0x1084, 0x1983, 0xc3, 0x15eb, 0xe0a, 0xdd, 0x1cb1, 0xb21, 0x2a51, 0x217f, 0xb1, 0x1328, 0x9ca, 0x1d96, 0x1a0b, 0xe1b, 0x1c4b, 0x3b, 0x4d6, 0x2344, 0x199e, 0x28af, 0x1624, 0x4ae, 0x8b2, 0x2991, 0x1fb7, 0x41, 0x2780, 0x1d8b, 0xa7f, 0x110, 0x2350, 0x18aa, 0x2b2f, 0x1805, 0x1ff, 0xf0, 0x2a74, 0xe42, 0xd97, 0x85b, 0x14bc, 0x2901, 0xfd8, 0x1ab3, 0x1cef, 0xfbd, 0x2b07, 0x174f, 0x69b, 0x10c3, 0x1491, 0xde3, 0x28ca, 0x252e, 0x1849, 0x1ec2, 0x1f1b, 0x2853, 0x12ab, 0x2674, 0x238c, 0x350, 0x2ca, 0xa7, 0x4bd, 0xcc3, 0x90c, 0x892, 0x276, 0x1e55, 0x196d, 0x1194, 0x1bef, 0x66a, 0x1da1, 0x260f, 0x1c15, 0x49f, 0x120b, 0x2671, 0x1237, 0x2e0d, 0x2791, 0x17d8, 0x1e0a, 0x2a99, 0x14cf, 0xfb1, 0x15b4, 0x1462, 0x2fbb, 0xeff, 0x16b, 0x2d6a, 0x9ef, 0x5e3, 0x11c0, 0x2e76, 0x1623, 0x2db8, 0x1c88, 0x740, 0x11e1, 0x12a3, 0x977, 0x1110, 0x2163, 0x2dee, 0x47b, 0x2aa5, 0x2a22, 0x1231, 0x16e7, 0x1626, 0x12e0, 0x1d28, 0xe96, 0xb62, 0x21d0, 0xf09, 0xb30, 0xcb8, 0x2981, 0x2648, 0x155d, 0x27ee, 0xb34, 0x169, 0x1574, 0x1fe6, 0x25f4, 0x151d, 0x1801, 0x1f13, 0x1308, 0x2929, 0x6eb, 0x25e, 0x2cca, 0x1e3e, 0x248f}; -const sign16 inv = 0xeab; -const sign16 invpr = 0x2c2a; - -#define DEGREE (1<> 31); - return (x + mask)^mask; -} - -/* Montgomery stuff */ - -static sign32 redc(unsign64 T) -{ - unsign32 m = (unsign32)T * (unsign32)RLWE_ND; - return ((unsign64)m * RLWE_PRIME + T) >> WL; -} - -static sign32 nres(unsign32 x) -{ - return redc((unsign64)x * RLWE_R2MODP); -} - -static sign32 modmul(unsign32 a, unsign32 b) -{ - return redc((unsign64)a * b); -} - -/* NTT code */ -/* Cooley-Tukey NTT */ - -static void ntt(sign32 *x) -{ - int m, i, j, k, t = DEGREE / 2; - sign32 S, U, V, W, q = RLWE_PRIME; - - /* Convert to Montgomery form */ - for (j = 0; j < DEGREE; j++) - x[j] = nres(x[j]); - - m = 1; - while (m < DEGREE) - { - k = 0; - for (i = 0; i < m; i++) - { - S = roots[m + i]; - for (j = k; j < k + t; j++) - { - U = x[j]; - V = modmul(x[j + t], S); - x[j] = U + V; - x[j + t] = U + 2 * q - V; - } - k += 2 * t; - } - t /= 2; - m *= 2; - } -} - -/* Gentleman-Sande INTT */ - -static void intt(sign32 *x) -{ - int m, i, j, k, t = 1; - sign32 S, U, V, W, q = RLWE_PRIME; - - m = DEGREE / 2; - while (m > 1) - { - k = 0; - for (i = 0; i < m; i++) - { - S = iroots[m + i]; - for (j = k; j < k + t; j++) - { - U = x[j]; - V = x[j + t]; - x[j] = U + V; - W = U + DEGREE * q - V; - x[j + t] = modmul(W, S); - } - k += 2 * t; - } - t *= 2; - m /= 2; - } - - /* Last iteration merged with n^-1 */ - - t = DEGREE / 2; - for (j = 0; j < t; j++) - { - U = x[j]; - V = x[j + t]; - W = U + DEGREE * q - V; - x[j + t] = modmul(W, (sign32)invpr); - x[j] = modmul(U + V, (sign32)inv); - } - /* convert back from Montgomery to "normal" form */ - for (j = 0; j < DEGREE; j++) - { - x[j] = redc(x[j]); - x[j] -= q; - x[j] += (x[j] >> (WL - 1))&q; - } -} - -/* See https://eprint.iacr.org/2016/1157.pdf */ - -static void NHSEncode(byte *key, sign32 *poly) -{ - int i, j, b, k, kj, q2; - - q2 = RLWE_PRIME / 2; - for (i = j = 0; i < 256;) - { - kj = key[j++]; - for (k = 0; k < 8; k++) - { - b = kj & 1; - poly[i] = b * q2; - poly[i + 256] = b * q2; - poly[i + 512] = b * q2; - poly[i + 768] = b * q2; - kj >>= 1; - i++; - } - } -} - -static void NHSDecode(sign32 *poly, byte *key) -{ - int i, j, k; - sign32 b, t, q2; - q2 = RLWE_PRIME / 2; - for (i = 0; i < 32; i++) - key[i] = 0; - - for (i = j = 0; i < 256;) - { - for (k = 0; k < 8; k++) - { - t = nabs(poly[i] - q2) + nabs(poly[i + 256] - q2) + nabs(poly[i + 512] - q2) + nabs(poly[i + 768] - q2); - - b = t - RLWE_PRIME; - b = (b >> 31) & 1; - key[j] = (key[j] >> 1) + (b << 7); - i++; - } - j++; - } -} - -/* convert 32-byte seed to random polynomial */ - -static void parse(byte *seed, sign32 *poly) -{ - int i, j; - sign32 n; - byte hash[4 * DEGREE]; - sha3 sh; - - SHA3_init(&sh, SHAKE128); - for (i = 0; i < 32; i++) - SHA3_process(&sh, seed[i]); - SHA3_shake(&sh, (char *)hash, 4 * DEGREE); - - for (i = j = 0; i < DEGREE; i++) - { - - n = hash[j] & 0x7f; n <<= 8; - n += hash[j + 1]; n <<= 8; - n += hash[j + 2]; n <<= 8; - n += hash[j + 3]; j += 4; - poly[i] = nres(n); - } -} - -/* Compress 14 bits polynomial coefficients into byte array */ -/* 7 bytes is 3x14 */ - -static void NHSpack(sign32 *poly, byte *array) -{ - int i, j; - sign32 a, b, c, d; - - for (i = j = 0; i < DEGREE; ) - { - a = poly[i++]; b = poly[i++]; c = poly[i++]; d = poly[i++]; - array[j++] = (byte)(a & 0xff); - array[j++] = (byte)(((a >> 8) | (b << 6)) & 0xff); - array[j++] = (byte)((b >> 2) & 0xff); - array[j++] = (byte)(((b >> 10) | (c << 4)) & 0xff); - array[j++] = (byte)((c >> 4) & 0xff); - array[j++] = (byte)(((c >> 12) | (d << 2)) & 0xff); - array[j++] = (byte)(d >> 6); - } -} - -static void NHSunpack(byte *array, sign32 *poly) -{ - int i, j; - sign32 a, b, c, d, e, f, g; - - for (i = j = 0; i < DEGREE; ) - { - a = ((sign32)array[j++]) & 0xff; b = ((sign32)array[j++]) & 0xff; c = ((sign32)array[j++]) & 0xff; d = ((sign32)array[j++]) & 0xff; e = ((sign32)array[j++]) & 0xff; f = ((sign32)array[j++]) & 0xff; g = ((sign32)array[j++]) & 0xff; - poly[i++] = a | ((b & 0x3f) << 8); - poly[i++] = (b >> 6) | (c << 2) | ((d & 0xf) << 10); - poly[i++] = (d >> 4) | (e << 4) | ((f & 3) << 12); - poly[i++] = (f >> 2) | (g << 6); - } -} - -/* See https://eprint.iacr.org/2016/1157.pdf */ - -static void NHSCompress(sign32 *poly, byte *array) -{ - int i, j, k, b; - unsign32 col = 0; - - for (i = j = 0; i < DEGREE;) - { - for (k = 0; k < 8; k++) - { - b = round((poly[i] * 8), RLWE_PRIME) & 7; - col = (col << 3) + b; - i++; - } - array[j] = col & 0xff; - array[j + 1] = (col >> 8) & 0xff; - array[j + 2] = (col >> 16) & 0xff; - j += 3; col = 0; - } -} - -static void NHSDecompress(byte *array, sign32 *poly) -{ - int i, j, k, b; - unsign32 col = 0; - - for (i = j = 0; i < DEGREE;) - { - col = array[j + 2]; - col = (col << 8) + array[j + 1]; - col = (col << 8) + array[j]; - j += 3; - for (k = 0; k < 8; k++) - { - b = (col & 0xe00000) >> 21; col <<= 3; - poly[i] = round((b * RLWE_PRIME), 8); - i++; - } - } -} - -/* generate centered binomial distribution */ - -static void NHSError(csprng *RNG, sign32 *poly) -{ - int i, j; - sign32 n1, n2, r; - for (i = 0; i < DEGREE; i++) - { - n1 = RAND_byte(RNG) + (RAND_byte(RNG) << 8); - n2 = RAND_byte(RNG) + (RAND_byte(RNG) << 8); - r = 0; - for (j = 0; j < 16; j++) - { - r += (n1 & 1) - (n2 & 1); - n1 >>= 1; n2 >>= 1; - } - poly[i] = (r + RLWE_PRIME); - } -} - -static void redc_it(sign32 *p) -{ - int i; - for (i = 0; i < DEGREE; i++) - p[i] = redc(p[i]); -} - -static void nres_it(sign32 *p) -{ - int i; - for (i = 0; i < DEGREE; i++) - p[i] = nres(p[i]); -} - -static void poly_mul(sign32 *p1, sign32 *p2, sign32 *p3) -{ - int i; - for (i = 0; i < DEGREE; i++) - p1[i] = modmul(p2[i], p3[i]); -} - -static void poly_add(sign32 *p1, sign32 *p2, sign32 *p3) -{ - int i; - for (i = 0; i < DEGREE; i++) - p1[i] = (p2[i] + p3[i]); -} - -static void poly_sub(sign32 *p1, sign32 *p2, sign32 *p3) -{ - int i; - for (i = 0; i < DEGREE; i++) - p1[i] = (p2[i] + RLWE_PRIME - p3[i]); -} - -/* reduces inputs < 2q */ -static void poly_soft_reduce(sign32 *poly) -{ - int i; - sign32 e; - for (i = 0; i < DEGREE; i++) - { - e = poly[i] - RLWE_PRIME; - poly[i] = e + ((e >> (WL - 1))&RLWE_PRIME); - } -} - -/* fully reduces modulo q */ -static void poly_hard_reduce(sign32 *poly) -{ - int i; - sign32 e; - for (i = 0; i < DEGREE; i++) - { - e = modmul(poly[i], RLWE_ONE); - e = e - RLWE_PRIME; - poly[i] = e + ((e >> (WL - 1))&RLWE_PRIME); - } -} - -/* API files */ - -void NHS_SERVER_1(csprng *RNG, octet *SB, octet *S) -{ - int i; - byte seed[32], array[1792]; - sign32 s[DEGREE], e[DEGREE], b[DEGREE]; - - for (i = 0; i < 32; i++) - seed[i] = RAND_byte(RNG); - - parse(seed, b); - - NHSError(RNG, e); - NHSError(RNG, s); - - ntt(s); - ntt(e); - poly_mul(b, b, s); - poly_add(b, b, e); - poly_hard_reduce(b); - - redc_it(b); - NHSpack(b, array); - - OCT_empty(SB); - OCT_jbytes(SB, (char *)seed, 32); - OCT_jbytes(SB, (char *)array, 1792); - - poly_hard_reduce(s); - - NHSpack(s, array); - OCT_empty(S); - OCT_jbytes(S, (char *)array, 1792); - -} - -void NHS_CLIENT(csprng *RNG, octet *SB, octet *UC, octet *KEY) -{ - int i; - sha3 sh; - byte seed[32], array[1792], key[32], cc[384]; - sign32 sd[DEGREE], ed[DEGREE], u[DEGREE], k[DEGREE], c[DEGREE]; - NHSError(RNG, sd); - NHSError(RNG, ed); - - ntt(sd); - ntt(ed); - - for (i = 0; i < 32; i++) - seed[i] = SB->val[i]; - - for (i = 0; i < 1792; i++) - array[i] = SB->val[i + 32]; - - parse(seed, u); - - poly_mul(u, u, sd); - poly_add(u, u, ed); - poly_hard_reduce(u); - - for (i = 0; i < 32; i++) - key[i] = RAND_byte(RNG); - - SHA3_init(&sh, SHA3_HASH256); - for (i = 0; i < 32; i++) - SHA3_process(&sh, key[i]); - SHA3_hash(&sh, (char *)key); - - NHSEncode(key, k); - - NHSunpack(array, c); - nres_it(c); - - poly_mul(c, c, sd); - intt(c); - NHSError(RNG, ed); - poly_add(c, c, ed); - poly_add(c, c, k); - - NHSCompress(c, cc); - - SHA3_init(&sh, SHA3_HASH256); - for (i = 0; i < 32; i++) - SHA3_process(&sh, key[i]); - SHA3_hash(&sh, (char *)key); - - OCT_empty(KEY); - OCT_jbytes(KEY, (char *)key, 32); - - redc_it(u); - NHSpack(u, array); - - OCT_empty(UC); - OCT_jbytes(UC, (char *)array, 1792); - OCT_jbytes(UC, (char *)cc, 384); -} - -void NHS_SERVER_2(octet *S, octet *UC, octet *KEY) -{ - int i; - sha3 sh; - sign32 c[DEGREE], s[DEGREE], k[DEGREE]; - byte array[1792], key[32], cc[384]; - - for (i = 0; i < 1792; i++) - array[i] = UC->val[i]; - - NHSunpack(array, k); - nres_it(k); - - for (i = 0; i < 384; i++) - cc[i] = UC->val[i + 1792]; - - NHSDecompress(cc, c); - - for (i = 0; i < 1792; i++) - array[i] = S->val[i]; - - NHSunpack(array, s); - - poly_mul(k, k, s); - intt(k); - poly_sub(k, c, k); - poly_soft_reduce(k); - - NHSDecode(k, key); - - SHA3_init(&sh, SHA3_HASH256); - for (i = 0; i < 32; i++) - SHA3_process(&sh, key[i]); - SHA3_hash(&sh, (char *)key); - - OCT_empty(KEY); - OCT_jbytes(KEY, (char *)key, 32); -} - diff --git a/impl/cbits/newhope.h b/impl/cbits/newhope.h deleted file mode 100644 index 2fde4ec8a..000000000 --- a/impl/cbits/newhope.h +++ /dev/null @@ -1,58 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file newhope.h - * @author Mike Scott - * @brief Newhope Header File - * - */ - -/* NewHope Simple API */ - -#ifndef NHS_H -#define NHS_H - -#include "core.h" - -/** @brief NHS server first pass - * - @param RNG Random Number Generator handle - @param SB seed and polynomial B concatenated - output - @param S server secret - output - - */ -extern void NHS_SERVER_1(csprng *RNG, octet *SB, octet *S); -/** @brief NHS client pass - * - @param RNG Random Number Generator handle - @param SB seed and polynomial B concatenated - input - @param UC polynomial U and compressed polynomial c - output - @param KEY client key - */ -extern void NHS_CLIENT(csprng *RNG, octet *SB, octet *UC, octet *KEY); -/** @brief NHS server second pass - * - @param S server secret - input - @param UC polynomial U and compressed polynomial c - input - @param KEY server key - */ -extern void NHS_SERVER_2(octet *S, octet *UC, octet *KEY); - -#endif diff --git a/impl/cbits/oct.c b/impl/cbits/oct.c deleted file mode 100644 index 674dd75d1..000000000 --- a/impl/cbits/oct.c +++ /dev/null @@ -1,413 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/*** Basic Octet string maintainance routines ***/ -/* SU=m, m is Stack Usage */ - -#include -#include "core.h" - -/* Output an octet string (Debug Only) */ - -/* SU= 16 */ -/* output octet */ -void OCT_output(octet *w) -{ - int i; - unsigned char ch; - for (i = 0; i < w->len; i++) - { - ch = w->val[i]; - printf("%02x", ch); - } - printf("\n"); -} - -/* SU= 16 */ -void OCT_output_string(octet *w) -{ - int i; - unsigned char ch; - for (i = 0; i < w->len; i++) - { - ch = w->val[i]; - printf("%c", ch); - } - /* printf("\n"); */ -} - -/* reverse bytes. Useful if dealing with those little-endian bastards */ -void OCT_reverse(octet *w) -{ - int i; - unsigned char ch; - for (i = 0; i < w->len/2; i++) { - ch = w->val[i]; - w->val[i] = w->val[w->len - i - 1]; - w->val[w->len - i - 1] = ch; - } -} - -/* Convert C string to octet format - truncates if no room */ -void OCT_jstring(octet *y, char *s) -{ - int i, j; - i = y->len; - j = 0; - while (s[j] != 0 && i < y->max) - { - y->val[i] = s[j]; - y->len++; - i++; - j++; - } -} - -/* compare 2 octet strings. - * If x==y return TRUE, else return FALSE */ -/* SU= 8 */ -int OCT_comp(octet *x, octet *y) -{ - int i; - if (x->len > y->len) return 0; - if (x->len < y->len) return 0; - for (i = 0; i < x->len; i++) - { - if (x->val[i] != y->val[i]) return 0; - } - return 1; -} - -/* check are first n bytes the same (in constant time) */ - -int OCT_ncomp(octet *x, octet *y, int n) -{ - int i, res = 0; - if (n > y->len || n > x->len) return 0; - for (i = 0; i < n; i++) - { - res |= (int)(x->val[i] ^ y->val[i]); - } - if (res == 0) return 1; - return 0; -} - -/* Shift octet to the left by n bytes. Leftmost bytes disappear */ -void OCT_shl(octet *x, int n) -{ - int i; - if (n >= x->len) - { - x->len = 0; - return; - } - x->len -= n; - for (i = 0; i < x->len; i++) - x->val[i] = x->val[i + n]; -} - -/* Append binary string to octet - truncates if no room */ -/* SU= 12 */ -void OCT_jbytes(octet *y, char *b, int len) -{ - int i, j; - i = y->len; - for (j = 0; j < len && i < y->max; j++) - { - y->val[i] = b[j]; - y->len++; - i++; - } -} - -/* Concatenates two octet strings */ -/* SU= 8 */ -void OCT_joctet(octet *y, octet *x) -{ - /* y=y || x */ - int i, j; - if (x == NULL) return; - - for (i = 0; i < x->len; i++) - { - j = y->len + i; - if (j >= y->max) - { - y->len = y->max; - return; - } - y->val[j] = x->val[i]; - } - y->len += x->len; -} - -/* Append byte to octet rep times */ -/* SU= 8 */ -void OCT_jbyte(octet *y, int ch, int rep) -{ - int i, j; - i = y->len; - for (j = 0; j < rep && i < y->max; j++) - { - y->val[i] = ch; - y->len++; - i++; - } -} - -/* XOR common bytes of x with y */ -/* SU= 8 */ -void OCT_xor(octet *y, octet *x) -{ - /* xor first x->len bytes of y */ - - int i; - for (i = 0; i < x->len && i < y->len; i++) - { - y->val[i] ^= x->val[i]; - } -} - -/* clear an octet */ -void OCT_empty(octet *w) -{ - w->len = 0; -} - -/* Kill an octet string - Zeroise it for security */ -void OCT_clear(octet *w) -{ - int i; - for (i = 0; i < w->max; i++) w->val[i] = 0; - w->len = 0; -} - -/* appends int x of length len bytes to OCTET string */ -/* SU= 8 */ -void OCT_jint(octet *y, int x, int len) -{ - int i, n; - n = y->len + len; - if (n > y->max || len <= 0) return; - for (i = y->len; i < n; i++) y->val[i] = 0; - y->len = n; - - i = y->len; - while (x > 0 && i > 0) - { - i--; - y->val[i] = x % 256; - x /= 256; - } -} - -/* Pad an octet to a given length */ -/* SU= 8 */ -int OCT_pad(octet *w, int n) -{ - int i, d; - if (w->len > n || n > w->max) return 0; - if (n == w->len) return 1; - d = n - w->len; - for (i = n - 1; i >= d; i--) - w->val[i] = w->val[i - d]; - for (i = d - 1; i >= 0; i--) - w->val[i] = 0; - w->len = n; - return 1; -} - - -/* Convert an octet string to base64 string */ -/* SU= 56 */ -void OCT_tobase64(char *b, octet *w) -{ - int i, j, k, rem, last; - int c, ch[4]; - unsigned char ptr[3]; - rem = w->len % 3; - j = k = 0; - last = 4; - while (j < w->len) - { - for (i = 0; i < 3; i++) - { - if (j < w->len) ptr[i] = w->val[j++]; - else - { - ptr[i] = 0; - last--; - } - } - ch[0] = (ptr[0] >> 2) & 0x3f; - ch[1] = ((ptr[0] << 4) | (ptr[1] >> 4)) & 0x3f; - ch[2] = ((ptr[1] << 2) | (ptr[2] >> 6)) & 0x3f; - ch[3] = ptr[2] & 0x3f; - for (i = 0; i < last; i++) - { - c = ch[i]; - if (c < 26) c += 65; - if (c >= 26 && c < 52) c += 71; - if (c >= 52 && c < 62) c -= 4; - if (c == 62) c = '+'; - if (c == 63) c = '/'; - b[k++] = c; - } - } - if (rem > 0) for (i = rem; i < 3; i++) b[k++] = '='; - b[k] = '\0'; /* dangerous! */ -} - -/* SU= 56 */ -void OCT_frombase64(octet *w, char *b) -{ - int i, j, k, pads, len = (int)strlen(b); - int c, ch[4], ptr[3]; - j = k = 0; - while (j < len && k < w->max) - { - pads = 0; - for (i = 0; i < 4; i++) - { - c = 80 + b[j++]; - if (c <= 112) continue; /* ignore white space */ - if (c > 144 && c < 171) c -= 145; - if (c > 176 && c < 203) c -= 151; - if (c > 127 && c < 138) c -= 76; - if (c == 123) c = 62; - if (c == 127) c = 63; - if (c == 141) - { - pads++; /* ignore pads '=' */ - continue; - } - ch[i] = c; - } - ptr[0] = (ch[0] << 2) | (ch[1] >> 4); - ptr[1] = (ch[1] << 4) | (ch[2] >> 2); - ptr[2] = (ch[2] << 6) | ch[3]; - for (i = 0; i < 3 - pads && k < w->max; i++) - { - /* don't put in leading zeros */ - w->val[k++] = ptr[i]; - } - } - w->len = k; -} - -/* copy an octet string - truncates if no room */ -/* SU= 16 */ -void OCT_copy(octet *y, octet *x) -{ - int i; - OCT_clear(y); - y->len = x->len; - if (y->len > y->max) y->len = y->max; - - for (i = 0; i < y->len; i++) - y->val[i] = x->val[i]; -} - -/* XOR m with all of x */ -void OCT_xorbyte(octet *x, int m) -{ - int i; - for (i = 0; i < x->len; i++) x->val[i] ^= m; -} - -/* truncates x to n bytes and places the rest in y (if y is not NULL) */ -/* SU= 8 */ -void OCT_chop(octet *x, octet *y, int n) -{ - int i; - if (n >= x->len) - { - if (y != NULL) y->len = 0; - return; - } - if (y != NULL) y->len = x->len - n; - x->len = n; - - if (y != NULL) - { - for (i = 0; i < y->len && i < y->max; i++) y->val[i] = x->val[i + n]; - } -} - -/* set x to len random bytes */ -void OCT_rand(octet *x, csprng *RNG, int len) -{ - int i; - if (len > x->max) len = x->max; - x->len = len; - - for (i = 0; i < len; i++) x->val[i] = RAND_byte(RNG); -} - -/* Convert an octet to a hex string */ -void OCT_toHex(octet *src, char *dst) -{ - int i,len=src->len; - unsigned char ch; - for (i = 0; i < len; i++) - { - ch = src->val[i]; - sprintf(&dst[i * 2], "%02x", ch); - } - dst[2*len]='\0'; -} - -static int char2int(char input) -{ - if (input >= '0' && input <= '9') - return input - '0'; - if (input >= 'A' && input <= 'F') - return input - 'A' + 10; - if (input >= 'a' && input <= 'f') - return input - 'a' + 10; - return 0; -} - -/* Convert from a hex string */ -void OCT_fromHex(octet *dst, char *src) -{ - int i = 0; - int j = 0; - OCT_clear(dst); - while (src[j] != 0) - { - dst->val[i++] = char2int(src[j]) * 16 + char2int(src[j + 1]); - j += 2; - } - dst->len = i; -} - - -/* Convert an octet to a string */ -void OCT_toStr(octet *src, char *dst) -{ - int i; - unsigned char ch; - for (i = 0; i < src->len; i++) - { - ch = src->val[i]; - sprintf(&dst[i], "%c", ch); - } -} - diff --git a/impl/cbits/pair_BLS12381.c b/impl/cbits/pair_BLS12381.c deleted file mode 100644 index bae48a052..000000000 --- a/impl/cbits/pair_BLS12381.c +++ /dev/null @@ -1,1165 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/* CORE BN Curve pairing functions */ - -//#define HAS_MAIN - -#include "pair_BLS12381.h" - -// Point doubling for pairings -static void PAIR_BLS12381_double(ECP2_BLS12381 *A, FP2_BLS12381 *AA, FP2_BLS12381 *BB, FP2_BLS12381 *CC) -{ - FP2_BLS12381 YY; - FP2_BLS12381_copy(CC, &(A->x)); //FP2 XX=new FP2(A.getx()); //X - FP2_BLS12381_copy(&YY, &(A->y)); //FP2 YY=new FP2(A.gety()); //Y - FP2_BLS12381_copy(BB, &(A->z)); //FP2 ZZ=new FP2(A.getz()); //Z - - FP2_BLS12381_copy(AA, &YY); //FP2 YZ=new FP2(YY); //Y - FP2_BLS12381_mul(AA, AA, BB); //YZ.mul(ZZ); //YZ - FP2_BLS12381_sqr(CC, CC); //XX.sqr(); //X^2 - FP2_BLS12381_sqr(&YY, &YY); //YY.sqr(); //Y^2 - FP2_BLS12381_sqr(BB, BB); //ZZ.sqr(); //Z^2 - - FP2_BLS12381_add(AA, AA, AA); - FP2_BLS12381_neg(AA, AA); - FP2_BLS12381_norm(AA); // -2YZ - FP2_BLS12381_mul_ip(AA); - FP2_BLS12381_norm(AA); // -2YZi - - FP2_BLS12381_imul(BB, BB, 3 * CURVE_B_I_BLS12381); //3Bz^2 - FP2_BLS12381_imul(CC, CC, 3); //3X^2 - -#if SEXTIC_TWIST_BLS12381==D_TYPE - FP2_BLS12381_mul_ip(&YY); // Y^2.i - FP2_BLS12381_norm(&YY); - FP2_BLS12381_mul_ip(CC); // 3X^2.i - FP2_BLS12381_norm(CC); -#endif - -#if SEXTIC_TWIST_BLS12381==M_TYPE - FP2_BLS12381_mul_ip(BB); // 3Bz^2.i - FP2_BLS12381_norm(BB); -#endif - - FP2_BLS12381_sub(BB, BB, &YY); - FP2_BLS12381_norm(BB); - - ECP2_BLS12381_dbl(A); //A.dbl(); -} - -// Point addition for pairings -static void PAIR_BLS12381_add(ECP2_BLS12381 *A, ECP2_BLS12381 *B, FP2_BLS12381 *AA, FP2_BLS12381 *BB, FP2_BLS12381 *CC) -{ - FP2_BLS12381 T1; - FP2_BLS12381_copy(AA, &(A->x)); //FP2 X1=new FP2(A.getx()); // X1 - FP2_BLS12381_copy(CC, &(A->y)); //FP2 Y1=new FP2(A.gety()); // Y1 - FP2_BLS12381_copy(&T1, &(A->z)); //FP2 T1=new FP2(A.getz()); // Z1 - - FP2_BLS12381_copy(BB, &T1); //FP2 T2=new FP2(A.getz()); // Z1 - - FP2_BLS12381_mul(&T1, &T1, &(B->y)); //T1.mul(B.gety()); // T1=Z1.Y2 - FP2_BLS12381_mul(BB, BB, &(B->x)); //T2.mul(B.getx()); // T2=Z1.X2 - - FP2_BLS12381_sub(AA, AA, BB); //X1.sub(T2); - FP2_BLS12381_norm(AA); //X1.norm(); // X1=X1-Z1.X2 - FP2_BLS12381_sub(CC, CC, &T1); //Y1.sub(T1); - FP2_BLS12381_norm(CC); //Y1.norm(); // Y1=Y1-Z1.Y2 - - FP2_BLS12381_copy(&T1, AA); //T1.copy(X1); // T1=X1-Z1.X2 - -#if SEXTIC_TWIST_BLS12381==M_TYPE - FP2_BLS12381_mul_ip(AA); - FP2_BLS12381_norm(AA); -#endif - - FP2_BLS12381_mul(&T1, &T1, &(B->y)); //T1.mul(B.gety()); // T1=(X1-Z1.X2).Y2 - - FP2_BLS12381_copy(BB, CC); //T2.copy(Y1); // T2=Y1-Z1.Y2 - FP2_BLS12381_mul(BB, BB, &(B->x)); //T2.mul(B.getx()); // T2=(Y1-Z1.Y2).X2 - FP2_BLS12381_sub(BB, BB, &T1); //T2.sub(T1); - FP2_BLS12381_norm(BB); //T2.norm(); // T2=(Y1-Z1.Y2).X2 - (X1-Z1.X2).Y2 - - FP2_BLS12381_neg(CC, CC); //Y1.neg(); - FP2_BLS12381_norm(CC); //Y1.norm(); // Y1=-(Y1-Z1.Y2).Xs - *** - - ECP2_BLS12381_add(A, B); //A.add(B); -} - -/* Line function */ -static void PAIR_BLS12381_line(FP12_BLS12381 *v, ECP2_BLS12381 *A, ECP2_BLS12381 *B, FP_BLS12381 *Qx, FP_BLS12381 *Qy) -{ - FP2_BLS12381 AA, BB, CC; - FP4_BLS12381 a, b, c; - - if (A == B) - PAIR_BLS12381_double(A, &AA, &BB, &CC); - else - PAIR_BLS12381_add(A, B, &AA, &BB, &CC); - - FP2_BLS12381_pmul(&CC, &CC, Qx); - FP2_BLS12381_pmul(&AA, &AA, Qy); - - FP4_BLS12381_from_FP2s(&a, &AA, &BB); -#if SEXTIC_TWIST_BLS12381==D_TYPE - FP4_BLS12381_from_FP2(&b, &CC); - FP4_BLS12381_zero(&c); -#endif -#if SEXTIC_TWIST_BLS12381==M_TYPE - FP4_BLS12381_zero(&b); - FP4_BLS12381_from_FP2H(&c, &CC); -#endif - - FP12_BLS12381_from_FP4s(v, &a, &b, &c); - v->type = FP_SPARSER; -} - -/* prepare ate parameter, n=6u+2 (BN) or n=u (BLS), n3=3*n */ -int PAIR_BLS12381_nbits(BIG_384_58 n3, BIG_384_58 n) -{ - BIG_384_58 x; - BIG_384_58_rcopy(x, CURVE_Bnx_BLS12381); - -#if PAIRING_FRIENDLY_BLS12381==BN_CURVE - BIG_384_58_pmul(n, x, 6); -#if SIGN_OF_X_BLS12381==POSITIVEX - BIG_384_58_inc(n, 2); -#else - BIG_384_58_dec(n, 2); -#endif - -#else - BIG_384_58_copy(n, x); -#endif - - BIG_384_58_norm(n); - BIG_384_58_pmul(n3, n, 3); - BIG_384_58_norm(n3); - - return BIG_384_58_nbits(n3); -} - -/* - For multi-pairing, product of n pairings - 1. Declare FP12 array of length number of bits in Ate parameter - 2. Initialise this array by calling PAIR_initmp() - 3. Accumulate each pairing by calling PAIR_another() n times - 4. Call PAIR_miller() - 5. Call final exponentiation PAIR_fexp() -*/ - -/* prepare for multi-pairing */ -void PAIR_BLS12381_initmp(FP12_BLS12381 r[]) -{ - int i; - for (i = ATE_BITS_BLS12381 - 1; i >= 0; i--) - FP12_BLS12381_one(&r[i]); - return; -} - -// Store precomputed line details in an FP4 -static void PAIR_BLS12381_pack(FP4_BLS12381 *T, FP2_BLS12381* AA, FP2_BLS12381* BB, FP2_BLS12381 *CC) -{ - FP2_BLS12381 I, A, B; - FP2_BLS12381_inv(&I, CC); - FP2_BLS12381_mul(&A, AA, &I); - FP2_BLS12381_mul(&B, BB, &I); - FP4_BLS12381_from_FP2s(T, &A, &B); -} - -// Unpack G2 line function details and include G1 -static void PAIR_BLS12381_unpack(FP12_BLS12381 *v, FP4_BLS12381* T, FP_BLS12381 *Qx, FP_BLS12381 *Qy) -{ - FP4_BLS12381 a, b, c; - FP2_BLS12381 t; - FP4_BLS12381_copy(&a, T); - FP2_BLS12381_pmul(&a.a, &a.a, Qy); - FP2_BLS12381_from_FP(&t, Qx); - -#if SEXTIC_TWIST_BLS12381==D_TYPE - FP4_BLS12381_from_FP2(&b, &t); - FP4_BLS12381_zero(&c); -#endif -#if SEXTIC_TWIST_BLS12381==M_TYPE - FP4_BLS12381_zero(&b); - FP4_BLS12381_from_FP2H(&c, &t); -#endif - - FP12_BLS12381_from_FP4s(v, &a, &b, &c); - v->type = FP_SPARSEST; -} - - -/* basic Miller loop */ -void PAIR_BLS12381_miller(FP12_BLS12381 *res, FP12_BLS12381 r[]) -{ - int i; - FP12_BLS12381_one(res); - for (i = ATE_BITS_BLS12381 - 1; i >= 1; i--) - { - FP12_BLS12381_sqr(res, res); - FP12_BLS12381_ssmul(res, &r[i]); - FP12_BLS12381_zero(&r[i]); - } - -#if SIGN_OF_X_BLS12381==NEGATIVEX - FP12_BLS12381_conj(res, res); -#endif - FP12_BLS12381_ssmul(res, &r[0]); - FP12_BLS12381_zero(&r[0]); - return; -} - - -// Precompute table of line functions for fixed G2 value -void PAIR_BLS12381_precomp(FP4_BLS12381 T[], ECP2_BLS12381* GV) -{ - int i, j, nb, bt; - BIG_384_58 n, n3; - FP2_BLS12381 AA, BB, CC; - ECP2_BLS12381 A, G, NG; -#if PAIRING_FRIENDLY_BLS12381==BN_CURVE - ECP2_BLS12381 K; - FP2_BLS12381 X; - FP_BLS12381 Qx, Qy; - FP_BLS12381_rcopy(&Qx, Fra_BLS12381); - FP_BLS12381_rcopy(&Qy, Frb_BLS12381); - FP2_BLS12381_from_FPs(&X, &Qx, &Qy); -#if SEXTIC_TWIST_BLS12381==M_TYPE - FP2_BLS12381_inv(&X, &X); - FP2_BLS12381_norm(&X); -#endif -#endif - - ECP2_BLS12381_copy(&A, GV); - ECP2_BLS12381_copy(&G, GV); - ECP2_BLS12381_copy(&NG, GV); - ECP2_BLS12381_neg(&NG); - - nb = PAIR_BLS12381_nbits(n3, n); - j = 0; - - for (i = nb - 2; i >= 1; i--) - { - PAIR_BLS12381_double(&A, &AA, &BB, &CC); - PAIR_BLS12381_pack(&T[j++], &AA, &BB, &CC); - - bt = BIG_384_58_bit(n3, i) - BIG_384_58_bit(n, i); // bt=BIG_bit(n,i); - if (bt == 1) - { - PAIR_BLS12381_add(&A, &G, &AA, &BB, &CC); - PAIR_BLS12381_pack(&T[j++], &AA, &BB, &CC); - } - if (bt == -1) - { - PAIR_BLS12381_add(&A, &NG, &AA, &BB, &CC); - PAIR_BLS12381_pack(&T[j++], &AA, &BB, &CC); - } - } -#if PAIRING_FRIENDLY_BLS12381==BN_CURVE - -#if SIGN_OF_X_BLS12381==NEGATIVEX - ECP2_BLS12381_neg(&A); -#endif - - ECP2_BLS12381_copy(&K, &G); - ECP2_BLS12381_frob(&K, &X); - PAIR_BLS12381_add(&A, &K, &AA, &BB, &CC); - PAIR_BLS12381_pack(&T[j++], &AA, &BB, &CC); - ECP2_BLS12381_frob(&K, &X); - ECP2_BLS12381_neg(&K); - PAIR_BLS12381_add(&A, &K, &AA, &BB, &CC); - PAIR_BLS12381_pack(&T[j++], &AA, &BB, &CC); - -#endif -} - -/* Accumulate another set of line functions for n-pairing, assuming precomputation on G2 */ -void PAIR_BLS12381_another_pc(FP12_BLS12381 r[], FP4_BLS12381 T[], ECP_BLS12381 *QV) -{ - int i, j, nb, bt; - BIG_384_58 x, n, n3; - FP12_BLS12381 lv, lv2; - ECP_BLS12381 Q; - FP_BLS12381 Qx, Qy; - - if (ECP_BLS12381_isinf(QV)) return; - - nb = PAIR_BLS12381_nbits(n3, n); - - ECP_BLS12381_copy(&Q, QV); - ECP_BLS12381_affine(&Q); - - FP_BLS12381_copy(&Qx, &(Q.x)); - FP_BLS12381_copy(&Qy, &(Q.y)); - - j = 0; - for (i = nb - 2; i >= 1; i--) - { - PAIR_BLS12381_unpack(&lv, &T[j++], &Qx, &Qy); - - bt = BIG_384_58_bit(n3, i) - BIG_384_58_bit(n, i); // bt=BIG_bit(n,i); - if (bt == 1) - { - PAIR_BLS12381_unpack(&lv2, &T[j++], &Qx, &Qy); - FP12_BLS12381_smul(&lv, &lv2); - } - if (bt == -1) - { - PAIR_BLS12381_unpack(&lv2, &T[j++], &Qx, &Qy); - FP12_BLS12381_smul(&lv, &lv2); - } - FP12_BLS12381_ssmul(&r[i], &lv); - } - -#if PAIRING_FRIENDLY_BLS12381==BN_CURVE - PAIR_BLS12381_unpack(&lv, &T[j++], &Qx, &Qy); - PAIR_BLS12381_unpack(&lv2, &T[j++], &Qx, &Qy); - FP12_BLS12381_smul(&lv, &lv2); - FP12_BLS12381_ssmul(&r[0], &lv); -#endif -} - -/* Accumulate another set of line functions for n-pairing */ -void PAIR_BLS12381_another(FP12_BLS12381 r[], ECP2_BLS12381* PV, ECP_BLS12381* QV) -{ - int i, j, nb, bt; - BIG_384_58 x, n, n3; - FP12_BLS12381 lv, lv2; - ECP2_BLS12381 A, NP, P; - ECP_BLS12381 Q; - FP_BLS12381 Qx, Qy; -#if PAIRING_FRIENDLY_BLS12381==BN_CURVE - ECP2_BLS12381 K; - FP2_BLS12381 X; - FP_BLS12381_rcopy(&Qx, Fra_BLS12381); - FP_BLS12381_rcopy(&Qy, Frb_BLS12381); - FP2_BLS12381_from_FPs(&X, &Qx, &Qy); -#if SEXTIC_TWIST_BLS12381==M_TYPE - FP2_BLS12381_inv(&X, &X); - FP2_BLS12381_norm(&X); -#endif -#endif - - if (ECP_BLS12381_isinf(QV)) return; - - nb = PAIR_BLS12381_nbits(n3, n); - - ECP2_BLS12381_copy(&P, PV); - ECP_BLS12381_copy(&Q, QV); - - ECP2_BLS12381_affine(&P); - ECP_BLS12381_affine(&Q); - - FP_BLS12381_copy(&Qx, &(Q.x)); - FP_BLS12381_copy(&Qy, &(Q.y)); - - ECP2_BLS12381_copy(&A, &P); - ECP2_BLS12381_copy(&NP, &P); ECP2_BLS12381_neg(&NP); - - for (i = nb - 2; i >= 1; i--) - { - PAIR_BLS12381_line(&lv, &A, &A, &Qx, &Qy); - - bt = BIG_384_58_bit(n3, i) - BIG_384_58_bit(n, i); // bt=BIG_bit(n,i); - if (bt == 1) - { - PAIR_BLS12381_line(&lv2, &A, &P, &Qx, &Qy); - FP12_BLS12381_smul(&lv, &lv2); - } - if (bt == -1) - { - PAIR_BLS12381_line(&lv2, &A, &NP, &Qx, &Qy); - FP12_BLS12381_smul(&lv, &lv2); - } - FP12_BLS12381_ssmul(&r[i], &lv); - } - -#if PAIRING_FRIENDLY_BLS12381==BN_CURVE - -#if SIGN_OF_X_BLS12381==NEGATIVEX - ECP2_BLS12381_neg(&A); -#endif - - ECP2_BLS12381_copy(&K, &P); - ECP2_BLS12381_frob(&K, &X); - PAIR_BLS12381_line(&lv, &A, &K, &Qx, &Qy); - ECP2_BLS12381_frob(&K, &X); - ECP2_BLS12381_neg(&K); - PAIR_BLS12381_line(&lv2, &A, &K, &Qx, &Qy); - FP12_BLS12381_smul(&lv, &lv2); - FP12_BLS12381_ssmul(&r[0], &lv); - -#endif -} - -/* Optimal R-ate pairing r=e(P,Q) */ -void PAIR_BLS12381_ate(FP12_BLS12381 *r, ECP2_BLS12381 *P1, ECP_BLS12381 *Q1) -{ - - BIG_384_58 x, n, n3; - FP_BLS12381 Qx, Qy; - int i, nb, bt; - ECP2_BLS12381 A, NP, P; - ECP_BLS12381 Q; - FP12_BLS12381 lv, lv2; -#if PAIRING_FRIENDLY_BLS12381==BN_CURVE - ECP2_BLS12381 KA; - FP2_BLS12381 X; - - FP_BLS12381_rcopy(&Qx, Fra_BLS12381); - FP_BLS12381_rcopy(&Qy, Frb_BLS12381); - FP2_BLS12381_from_FPs(&X, &Qx, &Qy); - -#if SEXTIC_TWIST_BLS12381==M_TYPE - FP2_BLS12381_inv(&X, &X); - FP2_BLS12381_norm(&X); -#endif -#endif - - FP12_BLS12381_one(r); - if (ECP_BLS12381_isinf(Q1)) return; - - nb = PAIR_BLS12381_nbits(n3, n); - - ECP2_BLS12381_copy(&P, P1); - ECP_BLS12381_copy(&Q, Q1); - - ECP2_BLS12381_affine(&P); - ECP_BLS12381_affine(&Q); - - FP_BLS12381_copy(&Qx, &(Q.x)); - FP_BLS12381_copy(&Qy, &(Q.y)); - - ECP2_BLS12381_copy(&A, &P); - ECP2_BLS12381_copy(&NP, &P); ECP2_BLS12381_neg(&NP); - - - - /* Main Miller Loop */ - for (i = nb - 2; i >= 1; i--) //0 - { - FP12_BLS12381_sqr(r, r); - PAIR_BLS12381_line(&lv, &A, &A, &Qx, &Qy); - - bt = BIG_384_58_bit(n3, i) - BIG_384_58_bit(n, i); // bt=BIG_bit(n,i); - if (bt == 1) - { - PAIR_BLS12381_line(&lv2, &A, &P, &Qx, &Qy); - FP12_BLS12381_smul(&lv, &lv2); - } - if (bt == -1) - { - PAIR_BLS12381_line(&lv2, &A, &NP, &Qx, &Qy); - FP12_BLS12381_smul(&lv, &lv2); - } - FP12_BLS12381_ssmul(r, &lv); - - } - - -#if SIGN_OF_X_BLS12381==NEGATIVEX - FP12_BLS12381_conj(r, r); -#endif - - /* R-ate fixup required for BN curves */ -#if PAIRING_FRIENDLY_BLS12381==BN_CURVE - -#if SIGN_OF_X_BLS12381==NEGATIVEX - ECP2_BLS12381_neg(&A); -#endif - - ECP2_BLS12381_copy(&KA, &P); - ECP2_BLS12381_frob(&KA, &X); - PAIR_BLS12381_line(&lv, &A, &KA, &Qx, &Qy); - ECP2_BLS12381_frob(&KA, &X); - ECP2_BLS12381_neg(&KA); - PAIR_BLS12381_line(&lv2, &A, &KA, &Qx, &Qy); - FP12_BLS12381_smul(&lv, &lv2); - FP12_BLS12381_ssmul(r, &lv); -#endif -} - -/* Optimal R-ate double pairing e(P,Q).e(R,S) */ -void PAIR_BLS12381_double_ate(FP12_BLS12381 *r, ECP2_BLS12381 *P1, ECP_BLS12381 *Q1, ECP2_BLS12381 *R1, ECP_BLS12381 *S1) -{ - BIG_384_58 x, n, n3; - FP_BLS12381 Qx, Qy, Sx, Sy; - int i, nb, bt; - ECP2_BLS12381 A, B, NP, NR, P, R; - ECP_BLS12381 Q, S; - FP12_BLS12381 lv, lv2; -#if PAIRING_FRIENDLY_BLS12381==BN_CURVE - FP2_BLS12381 X; - ECP2_BLS12381 K; - - FP_BLS12381_rcopy(&Qx, Fra_BLS12381); - FP_BLS12381_rcopy(&Qy, Frb_BLS12381); - FP2_BLS12381_from_FPs(&X, &Qx, &Qy); - -#if SEXTIC_TWIST_BLS12381==M_TYPE - FP2_BLS12381_inv(&X, &X); - FP2_BLS12381_norm(&X); -#endif -#endif - - if (ECP_BLS12381_isinf(Q1)) - { - PAIR_BLS12381_ate(r, R1, S1); - return; - } - if (ECP_BLS12381_isinf(S1)) - { - PAIR_BLS12381_ate(r, P1, Q1); - return; - } - nb = PAIR_BLS12381_nbits(n3, n); - - ECP2_BLS12381_copy(&P, P1); - ECP_BLS12381_copy(&Q, Q1); - - ECP2_BLS12381_affine(&P); - ECP_BLS12381_affine(&Q); - - ECP2_BLS12381_copy(&R, R1); - ECP_BLS12381_copy(&S, S1); - - ECP2_BLS12381_affine(&R); - ECP_BLS12381_affine(&S); - - FP_BLS12381_copy(&Qx, &(Q.x)); - FP_BLS12381_copy(&Qy, &(Q.y)); - - FP_BLS12381_copy(&Sx, &(S.x)); - FP_BLS12381_copy(&Sy, &(S.y)); - - ECP2_BLS12381_copy(&A, &P); - ECP2_BLS12381_copy(&B, &R); - - ECP2_BLS12381_copy(&NP, &P); ECP2_BLS12381_neg(&NP); - ECP2_BLS12381_copy(&NR, &R); ECP2_BLS12381_neg(&NR); - - FP12_BLS12381_one(r); - - /* Main Miller Loop */ - for (i = nb - 2; i >= 1; i--) - { - FP12_BLS12381_sqr(r, r); - PAIR_BLS12381_line(&lv, &A, &A, &Qx, &Qy); - PAIR_BLS12381_line(&lv2, &B, &B, &Sx, &Sy); - FP12_BLS12381_smul(&lv, &lv2); - FP12_BLS12381_ssmul(r, &lv); - - bt = BIG_384_58_bit(n3, i) - BIG_384_58_bit(n, i); // bt=BIG_bit(n,i); - if (bt == 1) - { - PAIR_BLS12381_line(&lv, &A, &P, &Qx, &Qy); - PAIR_BLS12381_line(&lv2, &B, &R, &Sx, &Sy); - FP12_BLS12381_smul(&lv, &lv2); - FP12_BLS12381_ssmul(r, &lv); - } - if (bt == -1) - { - PAIR_BLS12381_line(&lv, &A, &NP, &Qx, &Qy); - PAIR_BLS12381_line(&lv2, &B, &NR, &Sx, &Sy); - FP12_BLS12381_smul(&lv, &lv2); - FP12_BLS12381_ssmul(r, &lv); - } - - } - - - /* R-ate fixup required for BN curves */ - -#if SIGN_OF_X_BLS12381==NEGATIVEX - FP12_BLS12381_conj(r, r); -#endif - -#if PAIRING_FRIENDLY_BLS12381==BN_CURVE - -#if SIGN_OF_X_BLS12381==NEGATIVEX - ECP2_BLS12381_neg(&A); - ECP2_BLS12381_neg(&B); -#endif - - ECP2_BLS12381_copy(&K, &P); - ECP2_BLS12381_frob(&K, &X); - PAIR_BLS12381_line(&lv, &A, &K, &Qx, &Qy); - ECP2_BLS12381_frob(&K, &X); - ECP2_BLS12381_neg(&K); - PAIR_BLS12381_line(&lv2, &A, &K, &Qx, &Qy); - FP12_BLS12381_smul(&lv, &lv2); - FP12_BLS12381_ssmul(r, &lv); - - ECP2_BLS12381_copy(&K, &R); - ECP2_BLS12381_frob(&K, &X); - PAIR_BLS12381_line(&lv, &B, &K, &Sx, &Sy); - ECP2_BLS12381_frob(&K, &X); - ECP2_BLS12381_neg(&K); - PAIR_BLS12381_line(&lv2, &B, &K, &Sx, &Sy); - FP12_BLS12381_smul(&lv, &lv2); - FP12_BLS12381_ssmul(r, &lv); -#endif -} - -/* final exponentiation - keep separate for multi-pairings and to avoid thrashing stack */ -void PAIR_BLS12381_fexp(FP12_BLS12381 *r) -{ - FP2_BLS12381 X; - BIG_384_58 x; - FP_BLS12381 a, b; - FP12_BLS12381 t0, y0, y1; -#if PAIRING_FRIENDLY_BLS12381==BN_CURVE - FP12_BLS12381 y2, y3; -#endif - - BIG_384_58_rcopy(x, CURVE_Bnx_BLS12381); - FP_BLS12381_rcopy(&a, Fra_BLS12381); - FP_BLS12381_rcopy(&b, Frb_BLS12381); - FP2_BLS12381_from_FPs(&X, &a, &b); - - /* Easy part of final exp */ - - FP12_BLS12381_inv(&t0, r); - FP12_BLS12381_conj(r, r); - - FP12_BLS12381_mul(r, &t0); - FP12_BLS12381_copy(&t0, r); - - FP12_BLS12381_frob(r, &X); - FP12_BLS12381_frob(r, &X); - FP12_BLS12381_mul(r, &t0); - - /* Hard part of final exp - see Duquesne & Ghamman eprint 2015/192.pdf */ -#if PAIRING_FRIENDLY_BLS12381==BN_CURVE - FP12_BLS12381_pow(&t0, r, x); // t0=f^-u -#if SIGN_OF_X_BLS12381==POSITIVEX - FP12_BLS12381_conj(&t0, &t0); -#endif - FP12_BLS12381_usqr(&y3, &t0); // y3=t0^2 - FP12_BLS12381_copy(&y0, &t0); - FP12_BLS12381_mul(&y0, &y3); // y0=t0*y3 - FP12_BLS12381_copy(&y2, &y3); - FP12_BLS12381_frob(&y2, &X); // y2=y3^p - FP12_BLS12381_mul(&y2, &y3); //y2=y2*y3 - FP12_BLS12381_usqr(&y2, &y2); //y2=y2^2 - FP12_BLS12381_mul(&y2, &y3); // y2=y2*y3 - - FP12_BLS12381_pow(&t0, &y0, x); //t0=y0^-u -#if SIGN_OF_X_BLS12381==POSITIVEX - FP12_BLS12381_conj(&t0, &t0); -#endif - FP12_BLS12381_conj(&y0, r); //y0=~r - FP12_BLS12381_copy(&y1, &t0); - FP12_BLS12381_frob(&y1, &X); - FP12_BLS12381_frob(&y1, &X); //y1=t0^p^2 - FP12_BLS12381_mul(&y1, &y0); // y1=y0*y1 - FP12_BLS12381_conj(&t0, &t0); // t0=~t0 - FP12_BLS12381_copy(&y3, &t0); - FP12_BLS12381_frob(&y3, &X); //y3=t0^p - FP12_BLS12381_mul(&y3, &t0); // y3=t0*y3 - FP12_BLS12381_usqr(&t0, &t0); // t0=t0^2 - FP12_BLS12381_mul(&y1, &t0); // y1=t0*y1 - - FP12_BLS12381_pow(&t0, &y3, x); // t0=y3^-u -#if SIGN_OF_X_BLS12381==POSITIVEX - FP12_BLS12381_conj(&t0, &t0); -#endif - FP12_BLS12381_usqr(&t0, &t0); //t0=t0^2 - FP12_BLS12381_conj(&t0, &t0); //t0=~t0 - FP12_BLS12381_mul(&y3, &t0); // y3=t0*y3 - - FP12_BLS12381_frob(r, &X); - FP12_BLS12381_copy(&y0, r); - FP12_BLS12381_frob(r, &X); - FP12_BLS12381_mul(&y0, r); - FP12_BLS12381_frob(r, &X); - FP12_BLS12381_mul(&y0, r); - - FP12_BLS12381_usqr(r, &y3); //r=y3^2 - FP12_BLS12381_mul(r, &y2); //r=y2*r - FP12_BLS12381_copy(&y3, r); - FP12_BLS12381_mul(&y3, &y0); // y3=r*y0 - FP12_BLS12381_mul(r, &y1); // r=r*y1 - FP12_BLS12381_usqr(r, r); // r=r^2 - FP12_BLS12381_mul(r, &y3); // r=r*y3 - FP12_BLS12381_reduce(r); -#else - -// See https://eprint.iacr.org/2020/875.pdf - FP12_BLS12381_usqr(&y1,r); - FP12_BLS12381_mul(&y1,r); // y1=r^3 - - FP12_BLS12381_pow(&y0,r,x); // y0=r^x -#if SIGN_OF_X_BLS12381==NEGATIVEX - FP12_BLS12381_conj(&y0, &y0); -#endif - FP12_BLS12381_conj(&t0,r); // t0=r^-1 - FP12_BLS12381_copy(r,&y0); - FP12_BLS12381_mul(r,&t0); // r=r^(x-1) - - FP12_BLS12381_pow(&y0,r,x); // y0=r^x -#if SIGN_OF_X_BLS12381==NEGATIVEX - FP12_BLS12381_conj(&y0, &y0); -#endif - FP12_BLS12381_conj(&t0,r); // t0=r^-1 - FP12_BLS12381_copy(r,&y0); - FP12_BLS12381_mul(r,&t0); // r=r^(x-1) - -// ^(x+p) - FP12_BLS12381_pow(&y0,r,x); // y0=r^x -#if SIGN_OF_X_BLS12381==NEGATIVEX - FP12_BLS12381_conj(&y0, &y0); -#endif - FP12_BLS12381_copy(&t0,r); - FP12_BLS12381_frob(&t0,&X); // t0=r^p - FP12_BLS12381_copy(r,&y0); - FP12_BLS12381_mul(r,&t0); // r=r^x.r^p - -// ^(x^2+p^2-1) - FP12_BLS12381_pow(&y0,r,x); - FP12_BLS12381_pow(&y0,&y0,x); // y0=r^x^2 - FP12_BLS12381_copy(&t0,r); - FP12_BLS12381_frob(&t0,&X); - FP12_BLS12381_frob(&t0,&X); // t0=r^p^2 - FP12_BLS12381_mul(&y0,&t0); // y0=r^x^2.r^p^2 - FP12_BLS12381_conj(&t0,r); // t0=r^-1 - FP12_BLS12381_copy(r,&y0); // - FP12_BLS12381_mul(r,&t0); // r=r^x^2.r^p^2.r^-1 - - FP12_BLS12381_mul(r,&y1); - FP12_BLS12381_reduce(r); - -// Ghamman & Fouotsa Method -/* - FP12_BLS12381_usqr(&y0, r); - FP12_BLS12381_pow(&y1, &y0, x); -#if SIGN_OF_X_BLS12381==NEGATIVEX - FP12_BLS12381_conj(&y1, &y1); -#endif - - BIG_384_58_fshr(x, 1); - FP12_BLS12381_pow(&y2, &y1, x); -#if SIGN_OF_X_BLS12381==NEGATIVEX - FP12_BLS12381_conj(&y2, &y2); -#endif - - BIG_384_58_fshl(x, 1); // x must be even - FP12_BLS12381_conj(&y3, r); - FP12_BLS12381_mul(&y1, &y3); - - FP12_BLS12381_conj(&y1, &y1); - FP12_BLS12381_mul(&y1, &y2); - - FP12_BLS12381_pow(&y2, &y1, x); -#if SIGN_OF_X_BLS12381==NEGATIVEX - FP12_BLS12381_conj(&y2, &y2); -#endif - - FP12_BLS12381_pow(&y3, &y2, x); -#if SIGN_OF_X_BLS12381==NEGATIVEX - FP12_BLS12381_conj(&y3, &y3); -#endif - FP12_BLS12381_conj(&y1, &y1); - FP12_BLS12381_mul(&y3, &y1); - - FP12_BLS12381_conj(&y1, &y1); - FP12_BLS12381_frob(&y1, &X); - FP12_BLS12381_frob(&y1, &X); - FP12_BLS12381_frob(&y1, &X); - FP12_BLS12381_frob(&y2, &X); - FP12_BLS12381_frob(&y2, &X); - FP12_BLS12381_mul(&y1, &y2); - - FP12_BLS12381_pow(&y2, &y3, x); -#if SIGN_OF_X_BLS12381==NEGATIVEX - FP12_BLS12381_conj(&y2, &y2); -#endif - FP12_BLS12381_mul(&y2, &y0); - FP12_BLS12381_mul(&y2, r); - - FP12_BLS12381_mul(&y1, &y2); - FP12_BLS12381_copy(&y2, &y3); - FP12_BLS12381_frob(&y2, &X); - FP12_BLS12381_mul(&y1, &y2); - FP12_BLS12381_copy(r, &y1); - FP12_BLS12381_reduce(r); -*/ -#endif -} - -#ifdef USE_GLV_BLS12381 -/* GLV method */ -static void glv(BIG_384_58 u[2], BIG_384_58 e) -{ -#if PAIRING_FRIENDLY_BLS12381==BN_CURVE - int i, j; - BIG_384_58 v[2], t, q; - DBIG_384_58 d; - BIG_384_58_rcopy(q, CURVE_Order_BLS12381); - for (i = 0; i < 2; i++) - { - BIG_384_58_rcopy(t, CURVE_W_BLS12381[i]); - BIG_384_58_mul(d, t, e); - BIG_384_58_ddiv(v[i], d, q); - BIG_384_58_zero(u[i]); - } - BIG_384_58_copy(u[0], e); - for (i = 0; i < 2; i++) - for (j = 0; j < 2; j++) - { - BIG_384_58_rcopy(t, CURVE_SB_BLS12381[j][i]); - BIG_384_58_modmul(t, v[j], t, q); - BIG_384_58_add(u[i], u[i], q); - BIG_384_58_sub(u[i], u[i], t); - BIG_384_58_mod(u[i], q); - } - -#else -// -(x^2).P = (Beta.x,y) - - BIG_384_58 x, x2, q; - BIG_384_58_rcopy(x, CURVE_Bnx_BLS12381); - BIG_384_58_smul(x2, x, x); - BIG_384_58_copy(u[0], e); - BIG_384_58_mod(u[0], x2); - BIG_384_58_copy(u[1], e); - BIG_384_58_sdiv(u[1], x2); - - BIG_384_58_rcopy(q, CURVE_Order_BLS12381); - BIG_384_58_sub(u[1], q, u[1]); - -#endif - - return; -} -#endif // USE_GLV - -/* Galbraith & Scott Method */ -static void gs(BIG_384_58 u[4], BIG_384_58 e) -{ - int i; -#if PAIRING_FRIENDLY_BLS12381==BN_CURVE - int j; - BIG_384_58 v[4], t, q; - DBIG_384_58 d; - BIG_384_58_rcopy(q, CURVE_Order_BLS12381); - for (i = 0; i < 4; i++) - { - BIG_384_58_rcopy(t, CURVE_WB_BLS12381[i]); - BIG_384_58_mul(d, t, e); - BIG_384_58_ddiv(v[i], d, q); - BIG_384_58_zero(u[i]); - } - - BIG_384_58_copy(u[0], e); - for (i = 0; i < 4; i++) - for (j = 0; j < 4; j++) - { - BIG_384_58_rcopy(t, CURVE_BB_BLS12381[j][i]); - BIG_384_58_modmul(t, v[j], t, q); - BIG_384_58_add(u[i], u[i], q); - BIG_384_58_sub(u[i], u[i], t); - BIG_384_58_mod(u[i], q); - } - -#else - - BIG_384_58 x, w, q; - BIG_384_58_rcopy(q, CURVE_Order_BLS12381); - BIG_384_58_rcopy(x, CURVE_Bnx_BLS12381); - BIG_384_58_copy(w, e); - - for (i = 0; i < 3; i++) - { - BIG_384_58_copy(u[i], w); - BIG_384_58_mod(u[i], x); - BIG_384_58_sdiv(w, x); - } - BIG_384_58_copy(u[3], w); - - /* */ -#if SIGN_OF_X_BLS12381==NEGATIVEX - BIG_384_58_modneg(u[1], u[1], q); - BIG_384_58_modneg(u[3], u[3], q); -#endif - -#endif - - - - return; -} - -/* Multiply P by e in group G1 */ -void PAIR_BLS12381_G1mul(ECP_BLS12381 *P, BIG_384_58 e) -{ -#ifdef USE_GLV_BLS12381 /* Note this method is patented */ - int np, nn; - ECP_BLS12381 Q; - FP_BLS12381 cru; - BIG_384_58 t, q; - BIG_384_58 u[2]; - - BIG_384_58_rcopy(q, CURVE_Order_BLS12381); - glv(u, e); - - ECP_BLS12381_copy(&Q, P); ECP_BLS12381_affine(&Q); - FP_BLS12381_rcopy(&cru, CRu_BLS12381); - FP_BLS12381_mul(&(Q.x), &(Q.x), &cru); - - /* note that -a.B = a.(-B). Use a or -a depending on which is smaller */ - - np = BIG_384_58_nbits(u[0]); - BIG_384_58_modneg(t, u[0], q); - nn = BIG_384_58_nbits(t); - if (nn < np) - { - BIG_384_58_copy(u[0], t); - ECP_BLS12381_neg(P); - } - - np = BIG_384_58_nbits(u[1]); - BIG_384_58_modneg(t, u[1], q); - nn = BIG_384_58_nbits(t); - if (nn < np) - { - BIG_384_58_copy(u[1], t); - ECP_BLS12381_neg(&Q); - } - BIG_384_58_norm(u[0]); - BIG_384_58_norm(u[1]); - ECP_BLS12381_mul2(P, &Q, u[0], u[1]); - -#else - ECP_BLS12381_mul(P, e); -#endif -} - -/* Multiply P by e in group G2 */ -void PAIR_BLS12381_G2mul(ECP2_BLS12381 *P, BIG_384_58 e) -{ -#ifdef USE_GS_G2_BLS12381 /* Well I didn't patent it :) */ - int i, np, nn; - ECP2_BLS12381 Q[4]; - FP2_BLS12381 X; - FP_BLS12381 fx, fy; - BIG_384_58 x, y, u[4]; - - FP_BLS12381_rcopy(&fx, Fra_BLS12381); - FP_BLS12381_rcopy(&fy, Frb_BLS12381); - FP2_BLS12381_from_FPs(&X, &fx, &fy); - -#if SEXTIC_TWIST_BLS12381==M_TYPE - FP2_BLS12381_inv(&X, &X); - FP2_BLS12381_norm(&X); -#endif - - BIG_384_58_rcopy(y, CURVE_Order_BLS12381); - gs(u, e); - - ECP2_BLS12381_copy(&Q[0], P); - for (i = 1; i < 4; i++) - { - ECP2_BLS12381_copy(&Q[i], &Q[i - 1]); - ECP2_BLS12381_frob(&Q[i], &X); - } - - for (i = 0; i < 4; i++) - { - np = BIG_384_58_nbits(u[i]); - BIG_384_58_modneg(x, u[i], y); - nn = BIG_384_58_nbits(x); - if (nn < np) - { - BIG_384_58_copy(u[i], x); - ECP2_BLS12381_neg(&Q[i]); - } - BIG_384_58_norm(u[i]); - } - - ECP2_BLS12381_mul4(P, Q, u); - -#else - ECP2_BLS12381_mul(P, e); -#endif -} - -/* f=f^e */ -void PAIR_BLS12381_GTpow(FP12_BLS12381 *f, BIG_384_58 e) -{ -#ifdef USE_GS_GT_BLS12381 /* Note that this option requires a lot of RAM! Maybe better to use compressed XTR method, see fp4.c */ - int i, np, nn; - FP12_BLS12381 g[4]; - FP2_BLS12381 X; - BIG_384_58 t, q; - FP_BLS12381 fx, fy; - BIG_384_58 u[4]; - - FP_BLS12381_rcopy(&fx, Fra_BLS12381); - FP_BLS12381_rcopy(&fy, Frb_BLS12381); - FP2_BLS12381_from_FPs(&X, &fx, &fy); - - BIG_384_58_rcopy(q, CURVE_Order_BLS12381); - gs(u, e); - - FP12_BLS12381_copy(&g[0], f); - for (i = 1; i < 4; i++) - { - FP12_BLS12381_copy(&g[i], &g[i - 1]); - FP12_BLS12381_frob(&g[i], &X); - } - - for (i = 0; i < 4; i++) - { - np = BIG_384_58_nbits(u[i]); - BIG_384_58_modneg(t, u[i], q); - nn = BIG_384_58_nbits(t); - if (nn < np) - { - BIG_384_58_copy(u[i], t); - FP12_BLS12381_conj(&g[i], &g[i]); - } - BIG_384_58_norm(u[i]); - } - FP12_BLS12381_pow4(f, g, u); - -#else - FP12_BLS12381_pow(f, f, e); -#endif -} - - -/* test G1 group membership */ - -int PAIR_BLS12381_G1member(ECP_BLS12381 *P) -{ - BIG_384_58 q; - ECP_BLS12381 W; - if (ECP_BLS12381_isinf(P)) return 0; - BIG_384_58_rcopy(q, CURVE_Order_BLS12381); - ECP_BLS12381_copy(&W,P); - PAIR_BLS12381_G1mul(&W,q); - if (!ECP_BLS12381_isinf(&W)) return 0; - return 1; -} - -/* test G2 group membership */ - -int PAIR_BLS12381_G2member(ECP2_BLS12381 *P) -{ - BIG_384_58 q; - ECP2_BLS12381 W; - if (ECP2_BLS12381_isinf(P)) return 0; - BIG_384_58_rcopy(q, CURVE_Order_BLS12381); - ECP2_BLS12381_copy(&W,P); - PAIR_BLS12381_G2mul(&W,q); - if (!ECP2_BLS12381_isinf(&W)) return 0; - return 1; -} - -/* test GT group membership */ -/* First check that m!=1, conj(m)*m==1, and m.m^{p^4}=m^{p^2} */ - -int PAIR_BLS12381_GTmember(FP12_BLS12381 *m) -{ - BIG_384_58 q; - FP_BLS12381 fx,fy; - FP2_BLS12381 X; - FP12_BLS12381 r,w; - if (FP12_BLS12381_isunity(m)) return 0; - FP12_BLS12381_conj(&r,m); - FP12_BLS12381_mul(&r,m); - if (!FP12_BLS12381_isunity(&r)) return 0; - - FP_BLS12381_rcopy(&fx,Fra_BLS12381); - FP_BLS12381_rcopy(&fy,Frb_BLS12381); - FP2_BLS12381_from_FPs(&X,&fx,&fy); - - FP12_BLS12381_copy(&r,m); FP12_BLS12381_frob(&r,&X); FP12_BLS12381_frob(&r,&X); - FP12_BLS12381_copy(&w,&r); FP12_BLS12381_frob(&w,&X); FP12_BLS12381_frob(&w,&X); - FP12_BLS12381_mul(&w,m); - - if (!FP12_BLS12381_equals(&w,&r)) return 0; - - BIG_384_58_rcopy(q, CURVE_Order_BLS12381); - FP12_BLS12381_copy(&r,m); - PAIR_BLS12381_GTpow(&r,q); - if (!FP12_BLS12381_isunity(&r)) return 0; - return 1; - -} - -#ifdef HAS_MAIN - -int main() -{ - int i; - char byt[32]; - csprng rng; - BIG_384_58 xa, xb, ya, yb, w, a, b, t1, q, u[2], v[4], m, r; - ECP2_BLS12381 P, G; - ECP_BLS12381 Q, R; - FP12_BLS12381 g, gp; - FP4_BLS12381 t, c, cp, cpm1, cpm2; - FP2_BLS12381 x, y, X; - - - BIG_384_58_rcopy(a, CURVE_Fra); - BIG_384_58_rcopy(b, CURVE_Frb); - FP2_BLS12381_from_BIGs(&X, a, b); - - BIG_384_58_rcopy(xa, CURVE_Gx); - BIG_384_58_rcopy(ya, CURVE_Gy); - - ECP_BLS12381_set(&Q, xa, ya); - if (Q.inf) printf("Failed to set - point not on curve\n"); - else printf("G1 set success\n"); - - printf("Q= "); - ECP_BLS12381_output(&Q); - printf("\n"); - - BIG_384_58_rcopy(xa, CURVE_Pxa); - BIG_384_58_rcopy(xb, CURVE_Pxb); - BIG_384_58_rcopy(ya, CURVE_Pya); - BIG_384_58_rcopy(yb, CURVE_Pyb); - - FP2_BLS12381_from_BIGs(&x, xa, xb); - FP2_BLS12381_from_BIGs(&y, ya, yb); - - ECP2_BLS12381_set(&P, &x, &y); - if (P.inf) printf("Failed to set - point not on curve\n"); - else printf("G2 set success\n"); - - printf("P= "); - ECP2_BLS12381_output(&P); - printf("\n"); - - for (i = 0; i < 1000; i++ ) - { - PAIR_BLS12381_ate(&g, &P, &Q); - PAIR_BLS12381_fexp(&g); - } - printf("g= "); - FP12_BLS12381_output(&g); - printf("\n"); -} - -#endif diff --git a/impl/cbits/pair_BLS12381.h b/impl/cbits/pair_BLS12381.h deleted file mode 100644 index 540ea05be..000000000 --- a/impl/cbits/pair_BLS12381.h +++ /dev/null @@ -1,172 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file pair.h - * @author Mike Scott - * @brief PAIR Header File - * - */ - -#ifndef PAIR_BLS12381_H -#define PAIR_BLS12381_H - -#include "fp12_BLS12381.h" -#include "ecp2_BLS12381.h" -#include "ecp_BLS12381.h" - -/* Pairing constants */ - -extern const BIG_384_58 CURVE_Bnx_BLS12381; /**< BN curve x parameter */ -extern const BIG_384_58 CURVE_Cru_BLS12381; /**< BN curve Cube Root of Unity */ - -extern const BIG_384_58 CURVE_W_BLS12381[2]; /**< BN curve constant for GLV decomposition */ -extern const BIG_384_58 CURVE_SB_BLS12381[2][2]; /**< BN curve constant for GLV decomposition */ -extern const BIG_384_58 CURVE_WB_BLS12381[4]; /**< BN curve constant for GS decomposition */ -extern const BIG_384_58 CURVE_BB_BLS12381[4][4]; /**< BN curve constant for GS decomposition */ - -/* Pairing function prototypes */ - -/** @brief Precompute line functions details for fixed G2 value - * - @param T array of precomputed FP4 partial line functions - @param GV a fixed ECP2 instance - */ -extern void PAIR_BLS12381_precomp(FP4_BLS12381 T[], ECP2_BLS12381* GV); - - - -/** @brief Precompute line functions for n-pairing - * - @param r array of precomputed FP12 products of line functions - @param PV ECP2 instance, an element of G2 - @param QV ECP instance, an element of G1 - - */ -extern void PAIR_BLS12381_another(FP12_BLS12381 r[], ECP2_BLS12381* PV, ECP_BLS12381* QV); - - -/** @brief Compute line functions for n-pairing, assuming precomputation on G2 - * - @param r array of precomputed FP12 products of line functions - @param T array contains precomputed partial line fucntions from G2 - @param QV ECP instance, an element of G1 - - */ -extern void PAIR_BLS12381_another_pc(FP12_BLS12381 r[], FP4_BLS12381 T[], ECP_BLS12381 *QV); - - -/** @brief Calculate Miller loop for Optimal ATE pairing e(P,Q) - * - @param r FP12 result of the pairing calculation e(P,Q) - @param P ECP2 instance, an element of G2 - @param Q ECP instance, an element of G1 - - */ -extern void PAIR_BLS12381_ate(FP12_BLS12381 *r, ECP2_BLS12381 *P, ECP_BLS12381 *Q); -/** @brief Calculate Miller loop for Optimal ATE double-pairing e(P,Q).e(R,S) - * - Faster than calculating two separate pairings - @param r FP12 result of the pairing calculation e(P,Q).e(R,S), an element of GT - @param P ECP2 instance, an element of G2 - @param Q ECP instance, an element of G1 - @param R ECP2 instance, an element of G2 - @param S ECP instance, an element of G1 - */ -extern void PAIR_BLS12381_double_ate(FP12_BLS12381 *r, ECP2_BLS12381 *P, ECP_BLS12381 *Q, ECP2_BLS12381 *R, ECP_BLS12381 *S); -/** @brief Final exponentiation of pairing, converts output of Miller loop to element in GT - * - Here p is the internal modulus, and r is the group order - @param x FP12, on exit = x^((p^12-1)/r) - */ -extern void PAIR_BLS12381_fexp(FP12_BLS12381 *x); -/** @brief Fast point multiplication of a member of the group G1 by a BIG number - * - May exploit endomorphism for speed. - @param Q ECP member of G1. - @param b BIG multiplier - - */ -extern void PAIR_BLS12381_G1mul(ECP_BLS12381 *Q, BIG_384_58 b); -/** @brief Fast point multiplication of a member of the group G2 by a BIG number - * - May exploit endomorphism for speed. - @param P ECP2 member of G1. - @param b BIG multiplier - - */ -extern void PAIR_BLS12381_G2mul(ECP2_BLS12381 *P, BIG_384_58 b); -/** @brief Fast raising of a member of GT to a BIG power - * - May exploit endomorphism for speed. - @param x FP12 member of GT. - @param b BIG exponent - - */ -extern void PAIR_BLS12381_GTpow(FP12_BLS12381 *x, BIG_384_58 b); - -/** @brief Tests ECP for membership of G1 - * - @param P ECP member of G1 - @return true or false - - */ -extern int PAIR_BLS12381_G1member(ECP_BLS12381 *P); - -/** @brief Tests ECP2 for membership of G2 - * - @param P ECP2 member of G2 - @return true or false - - */ -extern int PAIR_BLS12381_G2member(ECP2_BLS12381 *P); -/** @brief Tests FP12 for membership of GT - * - @param x FP12 instance - @return 1 if x is in GT, else return 0 - - */ -extern int PAIR_BLS12381_GTmember(FP12_BLS12381 *x); - -/** @brief Prepare Ate parameter - * - @param n BIG parameter - @param n3 BIG paramter = 3*n - @return number of nits in n3 - - */ -extern int PAIR_BLS12381_nbits(BIG_384_58 n3, BIG_384_58 n); - -/** @brief Initialise structure for multi-pairing - * - @param r FP12 array, to be initialised to 1 - - */ -extern void PAIR_BLS12381_initmp(FP12_BLS12381 r[]); - - -/** @brief Miller loop - * - @param res FP12 result - @param r FP12 precomputed array of accumulated line functions - - */ -extern void PAIR_BLS12381_miller(FP12_BLS12381 *res, FP12_BLS12381 r[]); - -#endif diff --git a/impl/cbits/rand.c b/impl/cbits/rand.c deleted file mode 100644 index a74532593..000000000 --- a/impl/cbits/rand.c +++ /dev/null @@ -1,172 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/* - * Cryptographic strong random number generator - * - * Unguessable seed -> SHA -> PRNG internal state -> SHA -> random numbers - * Slow - but secure - * - * See ftp://ftp.rsasecurity.com/pub/pdfs/bull-1.pdf for a justification - */ -/* SU=m, m is Stack Usage */ - -#include "core.h" - -/* SU= 20 */ -static unsign32 sbrand(csprng *rng) -{ - /* Marsaglia & Zaman random number generator */ - int i, k; - unsign32 pdiff, t; - rng->rndptr++; - if (rng->rndptr < NK) return rng->ira[rng->rndptr]; - rng->rndptr = 0; - for (i = 0, k = NK - NJ; i < NK; i++, k++) - { - /* calculate next NK values */ - if (k == NK) k = 0; - t = rng->ira[k]; - pdiff = t - rng->ira[i] - rng->borrow; - - if (pdiff < t) rng->borrow = 0; - if (pdiff > t) rng->borrow = 1; - rng->ira[i] = pdiff; - } - return rng->ira[0]; -} - -/* SU= 20 */ -static void sirand(csprng* rng, unsign32 seed) -{ - /* initialise random number system */ - /* modified so that a subsequent call "stirs" in another seed value */ - /* in this way as many seed bits as desired may be used */ - int i, in; - unsign32 t, m = 1; - rng->borrow = 0L; - rng->rndptr = 0; - rng->ira[0] ^= seed; - for (i = 1; i < NK; i++) - { - /* fill initialisation vector */ - in = (NV * i) % NK; - rng->ira[in] ^= m; /* note XOR */ - t = m; - m = seed - m; - seed = t; - } - for (i = 0; i < 10000; i++) sbrand(rng ); /* "warm-up" & stir the generator */ -} - -/* SU= 312 */ -static void fill_pool(csprng *rng) -{ - /* hash down output of RNG to re-fill the pool */ - int i; - hash256 sh; - HASH256_init(&sh); - for (i = 0; i < 128; i++) HASH256_process(&sh, sbrand(rng)); - HASH256_hash(&sh, rng->pool); - rng->pool_ptr = 0; -} - -static unsign32 pack(const uchar *b) -{ - /* pack bytes into a 32-bit Word */ - return ((unsign32)b[3] << 24) | ((unsign32)b[2] << 16) | ((unsign32)b[1] << 8) | (unsign32)b[0]; -} - -/* SU= 360 */ -/* Initialize RNG with some real entropy from some external source */ -void RAND_seed(csprng *rng, int rawlen, char *raw) -{ - /* initialise from at least 128 byte string of raw * - * random (keyboard?) input, and 32-bit time-of-day */ - int i; - char digest[32]; - uchar b[4]; - hash256 sh; - rng->pool_ptr = 0; - for (i = 0; i < NK; i++) rng->ira[i] = 0; - if (rawlen > 0) - { - HASH256_init(&sh); - for (i = 0; i < rawlen; i++) - HASH256_process(&sh, raw[i]); - HASH256_hash(&sh, digest); - - /* initialise PRNG from distilled randomness */ - - for (i = 0; i < 8; i++) - { - b[0] = digest[4 * i]; - b[1] = digest[4 * i + 1]; - b[2] = digest[4 * i + 2]; - b[3] = digest[4 * i + 3]; - // printf("%08x\n",pack(b)); - sirand(rng, pack(b)); - } - } - fill_pool(rng); -} - -/* Terminate and clean up */ -void RAND_clean(csprng *rng) -{ - /* kill internal state */ - int i; - rng->pool_ptr = rng->rndptr = 0; - for (i = 0; i < 32; i++) rng->pool[i] = 0; - for (i = 0; i < NK; i++) rng->ira[i] = 0; - rng->borrow = 0; -} - -/* get random byte */ -/* SU= 8 */ -int RAND_byte(csprng *rng) -{ - int r; - r = rng->pool[rng->pool_ptr++]; - if (rng->pool_ptr >= 32) fill_pool(rng); - return (r & 0xff); -} - -/* test main program */ -/* -#include -#include - -void main() -{ - int i; - char raw[256]; - csprng rng; - - RAND_clean(&rng); - - - for (i=0;i<256;i++) raw[i]=(char)i; - RAND_seed(&rng,256,raw); - - for (i=0;i<1000;i++) - printf("%02x ",(unsigned char)RAND_byte(&rng)); -} - -*/ diff --git a/impl/cbits/randapi.c b/impl/cbits/randapi.c deleted file mode 100644 index 7f0a17256..000000000 --- a/impl/cbits/randapi.c +++ /dev/null @@ -1,34 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#include "randapi.h" - -/* Initialise a Cryptographically Strong Random Number Generator from - an octet of raw random data */ - -void CREATE_CSPRNG(csprng *RNG, octet *RAW) -{ - RAND_seed(RNG, RAW->len, RAW->val); -} - -void KILL_CSPRNG(csprng *RNG) -{ - RAND_clean(RNG); -} - diff --git a/impl/cbits/randapi.h b/impl/cbits/randapi.h deleted file mode 100644 index 57e731b1b..000000000 --- a/impl/cbits/randapi.h +++ /dev/null @@ -1,46 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/** - * @file randapi.h - * @author Mike Scott - * @brief PRNG API File - * - */ - -#ifndef RANDOM_H -#define RANDOM_H - -#include "core.h" - -/** @brief Initialise a random number generator - * - @param R is a pointer to a cryptographically secure random number generator - @param S is an input truly random seed value - */ -extern void CREATE_CSPRNG(csprng *R, octet *S); -/** @brief Kill a random number generator - * - Deletes all internal state - @param R is a pointer to a cryptographically secure random number generator - */ -extern void KILL_CSPRNG(csprng *R); - -#endif - diff --git a/impl/cbits/rom_curve_BLS12381.c b/impl/cbits/rom_curve_BLS12381.c deleted file mode 100644 index 5d1092e4e..000000000 --- a/impl/cbits/rom_curve_BLS12381.c +++ /dev/null @@ -1,108 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -#include "arch.h" -#include "ecp_BLS12381.h" - -/* Curve BLS12381 - Pairing friendly BLS curve */ - -#if CHUNK==16 - -#error Not supported - -#endif - -#if CHUNK==32 - -const int CURVE_Cof_I_BLS12381= 0; - -const int CURVE_B_I_BLS12381= 4; -const BIG_384_29 CURVE_B_BLS12381= {0x4,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0}; -const BIG_384_29 CURVE_Order_BLS12381= {0x1,0x1FFFFFF8,0x1F96FFBF,0x1B4805FF,0x1D80553B,0xC0404D0,0x1520CCE7,0xA6533AF,0x73EDA7,0x0,0x0,0x0,0x0,0x0}; -const BIG_384_29 CURVE_Gx_BLS12381= {0x1B22C6BB,0x19D78056,0x1E86BBFE,0xBD07FF2,0x1AC586C5,0x1D1F8B8D,0x4168538,0x9F2EE97,0xFC3688C,0x27D4D60,0x9A558E3,0x32FAF28,0x1F1D3A73,0xB}; -const BIG_384_29 CURVE_Gy_BLS12381= {0x6C5E7E1,0x551194A,0x222B903,0x198E8945,0xB3EDD03,0xC659602,0xBD8036C,0x12BABA01,0x4FCF5E0,0xBA0EC57,0x8278C3B,0x75541E3,0xB3F481E,0x4}; -const BIG_384_29 CURVE_HTPC_BLS12381= {0x1DE821B8,0x6288315,0x1715FEDF,0xD2A41DC,0x1C31088B,0xDEEA01F,0x7FC11BB,0x1E9291A1,0x1A12F01D,0xD1EB8DE,0x16CE3D2A,0x16D97EE9,0x1F7462C8,0x0}; - -const BIG_384_29 CURVE_Bnx_BLS12381= {0x10000,0x10080000,0x34,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0}; -const BIG_384_29 CURVE_Cof_BLS12381= {0x10001,0x10080000,0x34,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0}; -//const BIG_384_29 CURVE_Cof_BLS12381= {0xAAAB,0x55558,0x157855A3,0x191800AA,0x396,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0}; - - -const BIG_384_29 CURVE_Pxa_BLS12381= {0x121BDB8,0x402B646,0x16EFBF5,0x18064D50,0x1D1770BA,0x5B23D71,0xC0AD144,0x1A9F4807,0x11C6E47A,0x196E2882,0x9820149,0x11E1522,0x4AA2B2F,0x1}; -const BIG_384_29 CURVE_Pxb_BLS12381= {0x1D042B7E,0xD63E82A,0x51755F9,0x19E22427,0x15049334,0x10DDEE3F,0x186AD769,0x1A132416,0x5596BD0,0x4413A7B,0x1F6B34E8,0x4E33EC0,0x1E02B605,0x9}; -const BIG_384_29 CURVE_Pya_BLS12381= {0x8B82801,0xC9AA430,0xB28A278,0x15939877,0xD12C923,0xD34A8B0,0xE9DB50A,0x155197BA,0x1AADFD9B,0x16D171A8,0x3327371,0x4FADC23,0xE5D5277,0x6}; -const BIG_384_29 CURVE_Pyb_BLS12381= {0x105F79BE,0x15483AFF,0x1B07686A,0xE1A4EB9,0x99AB3F3,0x955AB97,0xEBC99D2,0xFD0B4EC,0x19CB3E28,0x15E145C,0xCAB34AC,0x1D4E6998,0x6C4A02,0x3}; - - -#if HTC_ISO_BLS12381 != 0 - -const BIG_384_29 CURVE_Ad_BLS12381= {0xD584C1D,0x7A14041,0x183E5FD7,0x6DF1B41,0x81AC989,0xC0D77EC,0x1AA363A2,0xA707DCC,0x2B0EA98,0x164B6A4C,0xF5A4E80,0x771D286,0x144698A,0x0}; -const BIG_384_29 CURVE_Bd_BLS12381= {0xE172BE0,0xE62474C,0x1B3AA974,0x642B462,0x15EF55A2,0xA7E779,0x1C282E7,0x1E1E49E8,0x1B2016C1,0x3A9F771,0x62C4BA,0x2D10060,0xE2908D1,0x9}; -const BIG_384_29 PC_BLS12381[53]= {{0xBA2D229,0xE45D174,0x134E47EA,0x1637016C,0x6B68C24,0x1F8DE126,0x1EF08F02,0xFC45906,0x1D31D79D,0x1C0F6F71,0xF47A588,0x1C4C1CE1,0xE08C248,0x3},{0x1605FB7B,0x133EF9F8,0xA177B32,0x16EE3F18,0x14866F69,0x19B001D8,0x1E5B542B,0x1BBCCF0F,0xDFA7DCC,0xE92B2D8,0x1CB63B02,0x139C0FC4,0x321DA07,0x8},{0x1E390C9E,0x1920833D,0xC9DE5F,0x12165DB8,0x11B7FA31,0xA5D7A5D,0x12659D8C,0x1007418B,0x2DD2ECB,0xAE89C79,0xB830DD4,0x179F4F88,0x9B1F8E1,0xB},{0x497E317,0xB8CC354,0xDD3A55B,0x52BE52D,0x1D1DE4FA,0xB649462,0x15D28B16,0xD9CF3EA,0xDC43B75,0xB1DF4C8,0x1EE42CCD,0x134F1F88,0xD3CF1F,0x4},{0x3F0C88E,0x65AB0C7,0x1D1D6BE7,0xF91F191,0x753339B,0x3177879,0x16C69A0B,0x1564EB69,0x13356DE5,0x6888BF2,0x1A1D0E21,0x357B7C5,0x1B81E770,0xB},{0x139ED84,0xEBF912D,0x14BB2B7,0x4A25182,0x6B2A8DA,0x110C7CE4,0x13864023,0x4C9E1F1,0x1FB11586,0x1C573295,0x1A8DC9B0,0x1FC89A52,0x16ED6553,0x6},{0xF652983,0x89E0E33,0x19CF4673,0xE1A5B95,0x8F90A08,0x15C84BF3,0x66E7B4E,0xFBB2A4F,0x15DB3CB1,0x1FBD3A55,0x744806,0x1AE627FE,0x30C3250,0xB},{0xC8895D9,0x8AA674D,0x79DF114,0x1450DE60,0x1AC18985,0x15B2CC17,0xCFC21BB,0xB424AFF,0x1499DB99,0x1F208C72,0x1990AD2C,0x333E886,0x99726A3,0x7},{0x1D9B6861,0xD9C4320,0x41C64F1,0xDC4B9C6,0x13083533,0x1944F8D9,0x1C97C6CC,0xCAD51B7,0x12D7F5E4,0x183F2AA0,0x13818274,0x1F98DB6E,0x178E7166,0xB},{0xC9EDCB0,0xBCFCED,0x25CA7F8,0x187C7A54,0xE25C958,0x1280F634,0xF95A1E3,0xE652B30,0x1BCE0324,0xE8854D0,0x7441231,0x12ECF1D8,0x154005DB,0x6},{0x13CB83BB,0x1A7778D,0x630D5BA,0x11E54DE6,0x1E86B483,0x119E3868,0x105FD597,0xB65ED50,0x1C7C17E7,0x110A3D40,0x1622EAC,0x1287565E,0x1294ED3E,0xB},{0x134649B7,0x1560B313,0x198B5BAB,0x185ABE5,0xE2C8561,0x1DAB66DA,0x17FC989,0x11145AE0,0x56B303E,0xECCC0AC,0xE024407,0x1D066681,0x1A05F2B1,0x8},{0x8ECDD0A,0xB1C268B,0x1E19400B,0xE9C9696,0x11C15931,0x99CBC79,0xDDDB7D,0x1DD2DEFA,0xF682B4,0x159D2B34,0x11DB5B8F,0x13D255A8,0x15FC13AB,0x4},{0x19A1D641,0x1BB761D3,0xE90DC11,0x4CD2557,0x18835038,0x6D33F9C,0x19ADD040,0x3AE2C26,0xCE07F8D,0xD7E3D1E,0x17A482CF,0x1B4A9F04,0x10ECF6A,0x5},{0x1DCC5A5E,0xFBECCDD,0x478B4C4,0xB72913A,0x2C580FA,0x10E6FCC1,0x2A0665B,0x1843794D,0x196E7F63,0x3A6780C,0xC2CFD6C,0x1AC95164,0xA7AC2A9,0xA},{0xEE84A3A,0x12BA24B,0x3781B3B,0x766A71E,0xDE9CEA7,0x3983157,0x62538B8,0x1335EA74,0x1570F57,0x1F02CB39,0x3CF8318,0x2D26C32,0x172CAACF,0x3},{0x1F6304A5,0x16FCD14,0x8A3C470,0x1A49788,0x982F740,0x1E77925C,0x1534290E,0x1D39D395,0x9395735,0x18283637,0x154E43DF,0x9CCCF72,0x7355F8E,0x7},{0x1532A21E,0x1CE9CAD9,0xD5E0754,0x537503E,0x106DA9BD,0x27419D9,0xAEE35AD,0xB34240C,0x1DFFDFC7,0x1A1F3D03,0x29BC757,0x4522950,0x1A8E1620,0x9},{0xDC62CD8,0x186F449C,0x1B3D7104,0xDAA487D,0x16FD0497,0x1455E146,0x15455332,0x7E2D62C,0x145B0824,0x1BE2075A,0x120EABFB,0xB15C5FD,0x1425581A,0x1},{0x1CB83E19,0x611CDD2,0x53FB73F,0x7A12CF9,0xCEACD6A,0x700588D,0x1347F299,0xDEB4E31,0x1F6F8941,0xDFF94C8,0x4DF98A,0xF4644BD,0x12962FE5,0x5},{0x82B3BFF,0xE413B76,0xC09BA79,0x155108D9,0xBF5713D,0x12C4624,0x30049B,0x19419E10,0x167041E8,0x14C729B1,0x122D1C44,0x16AB3886,0x561A5DE,0x9},{0xD21B1C,0x9E7CFD2,0xD0F7E26,0x11AD037C,0xAC62B55,0x430BFE4,0x2EA7256,0x9746B69,0xF01D5EF,0x1A5E9FD3,0x62CB98B,0x19FE335C,0xCA8D548,0x4},{0x9C8B604,0x5A2B5F3,0x10071DC1,0xA04FDFD,0x101B2B66,0xA7D4AD7,0x8E55EB7,0x11F092CB,0x15CB181D,0x1A16F975,0x13A942CE,0x121E079C,0x1E6BE4E9,0xA},{0x1475224B,0x1358F38A,0x1E6BEDE1,0x20936CA,0x7CE46BA,0x7AE9CB5,0x15A366AC,0x103AFD0C,0x1C5E673D,0x1A46251F,0xA8567D,0x1C899E22,0x1C129645,0x2},{0x1B980133,0x16CE9FAE,0x8CA9910,0x1F215A38,0x659CC6C,0x11969E20,0x16004F99,0x101A982,0x1C757B3B,0x13DF18AE,0x1CBF002B,0x1A3D9536,0x45A394A,0x1},{0xB971EF8,0xA602780,0x4847C83,0x10A38323,0x633F06C,0x87403DA,0x23B009C,0x54684D6,0x47AA7B1,0x27A9FA,0x14554258,0x372733,0x1182CAC1,0x5},{0x10074D8E,0x103E4526,0x113581B3,0x139BE836,0x1643249D,0x1F3FC88F,0x918B9AF,0x17155E18,0xC523559,0x1FF6976E,0xE463050,0x1E6DEDBD,0xB46A908,0xC},{0x1011C132,0x9B88D6,0xFEEBF3A,0x1E74B99C,0x1E61031B,0x1F20B1C4,0x4FF4460,0x196D95E9,0x13CD2FCB,0x18EA1FDC,0x37F42E3,0x6F9A37C,0x1713E479,0xC},{0xA731C30,0x1D7D575E,0x13AE9BCA,0x1EE0ABBA,0xD43B9B3,0xF3F68F2,0x1BF81A61,0x14F22B5E,0x3C42A0C,0x1D6D0A51,0x88EAF79,0x30D7B6A,0x1BBA7A1,0x7},{0x1BDBA587,0x1B872BB,0x181E8D8,0xCA4038F,0xCABE69D,0x17350F90,0x9B07A2D,0x2CCF3B8,0x1B8F3ABD,0x10F26D0D,0x1A232788,0x1B2CD097,0x1FC4018B,0x4},{0x1870FB29,0xAF26518,0x17FA4D68,0xC8AA1FD,0x842642F,0x6D36136,0x7FF40E,0x17FC77BB,0x14170A05,0x9653633,0x17A649AF,0x67570DF,0x187C8D53,0x4},{0x1FE9D6F2,0xB0FC42A,0x3D057B2,0x10F5848C,0x14F3747A,0x9E26B1,0x132D48C5,0x19457C30,0x1CE75BB8,0x13BCB59,0xCB25DF4,0x1F583779,0xAB0B9BC,0x2},{0x1633A5F0,0xD91D589,0x16A01CA6,0x1EC64D92,0x1544E203,0xE1E9D6A,0x1EF5D941,0x1A95F5B6,0x74A7D0,0xDC78535,0x8847847,0xC696D4,0x603FCA4,0xB},{0x12E8FEDB,0xDB6D767,0x4102A10,0xFF1B813,0x11ADC2EE,0x1FE9109A,0x2E1E60C,0x1F7C79CA,0x4195536,0x1510A94E,0x172BD3F8,0x1FC1FE26,0xCC03FDE,0x4},{0x10E5F4CB,0x11AAE3BD,0x11877B29,0xB5753D,0x11CF9DE4,0x11F60192,0x4702792,0x1721DD6F,0x17D42AA7,0x16C3A33A,0x1E261D46,0x11303842,0x1F86376E,0x0},{0x72DE1F6,0x6FF1206,0xC0148EE,0x1AA42C51,0xDA7D26,0x1F25C8A0,0x138B0D12,0x1ACB1463,0x142552E2,0x351DA4C,0x1D28E132,0x152CDCCD,0xCC786BA,0x0},{0xE41C696,0x4BF3AD1,0xBEA2FF8,0xACE232C,0x1AD34D6C,0x11A1F5B3,0xF43E41,0xD84A9E7,0x31223E9,0x1BB7DA34,0x15440DB5,0x9DCB023,0x14996A10,0x9},{0x1707BB33,0x14C22B8C,0xEE8F0AF,0x18F5DD36,0x143D3CD0,0x17B64AB2,0x548AD4A,0x11C9150D,0x1A11AD13,0xA4C06E7,0x96747C2,0x17449DC0,0x10D97C81,0x4},{0x1D634B8F,0xAA39D0,0xD25E011,0x5EAE1E2,0xAA205CA,0x1E6B1AB6,0x14CC93B,0xCBC4E77,0x171C40F,0x106BC0CE,0x1AC90957,0xDBB807C,0xFA1D81,0x7},{0x6ED06F7,0xFD6E099,0x5332034,0xA2F7B0E,0x480E420,0x6F93CA1,0x1F072DD2,0x129CE524,0x12BF565B,0xA9E6BB7,0x18A2F743,0x165C9E76,0x660400E,0x1},{0x173345CC,0x14CD89C2,0xE42B047,0xEC7C7,0x19B86930,0x177CD006,0x899F573,0x1B315BE0,0x16543346,0x5A2F8A4,0x10D84C51,0x18ECFFC7,0xD6B9514,0x5},{0x2561092,0x1425A94F,0x1FAEFAA5,0x12D130DE,0x1913516F,0xD446753,0xB4A303E,0x115DF9C8,0x77F94FF,0x12462862,0x1D614B07,0x103A067F,0xCCBB674,0x5},{0x1A8F6AA8,0x7C5A4E5,0xC18100,0xB853E9F,0xA5C871A,0xD9B731B,0x18A43964,0x7376C34,0x1D9C6DD0,0xD69488,0x123C0428,0x1D480B7A,0xD2F259E,0x2},{0x18913F55,0x377A45D,0xA6CD78D,0x10BD47AA,0x1D4FBC73,0xC973F53,0x1EED4C21,0xC7C27B0,0x103216F7,0x1ECA5424,0x1AA08165,0xE14DC39,0x7A55CDA,0xB},{0x15535D4A,0x1919ECEA,0x49220DA,0x1FC5EF77,0x19B4852C,0x1A8625F9,0x482AF15,0x1C98D5EB,0x4F9FB0C,0x1E8EBA66,0x686F953,0x6D8C246,0x66C8ED3,0xC},{0x15812ED9,0x7720AD0,0x77B918,0x1EB6010,0x17132B92,0x7E9031A,0x1F5FFACD,0xBDF43E9,0xEE5A437,0x15DD37FB,0xEF377E,0x1C7D4FD4,0xA3EF08B,0xB},{0x126A775C,0x8D09CC8,0x2C7EE4F,0x1538034B,0x51D5F,0x12DE2005,0x3BD774D,0x1F51A19F,0xB5EECFD,0x5674C12,0x10EEA1CD,0x1533B65F,0x6007C08,0xB},{0xAF9B7AC,0x16323BFD,0xA733880,0x71B73BF,0x15A6449F,0xC3DB787,0x20717B3,0x18CAAA1B,0x2B70152,0x1563C18C,0x7EC99BA,0x30DB65B,0xD9E5297,0x4},{0x11A5001D,0x11C8A118,0x14BB7B76,0x162BB81F,0xC916A20,0xD07E4EF,0xEC150BB,0x13E1ED37,0x1CC6D19C,0x17C1146E,0xC033244,0x8BE87C9,0x1E0E0795,0x5},{0x45F5416,0x6936CC2,0xA5EB6A,0x6C9E585,0xAF41727,0x1244F393,0xC3848F6,0x1B7BB79A,0x11D115C5,0x1C4F6DA6,0x1C8348EF,0x131CA72B,0xB7D2887,0xB},{0x1DBF67F2,0x1129C5A9,0x1E5BE247,0xAF9AC6D,0xD2ECA67,0x12EE93CE,0x1CC430D6,0xAAA35CF,0x1778C485,0xB74758A,0x1BEAAB9F,0xC81B44E,0x18DF3306,0x2},{0xE49A03D,0x17B08161,0x14A78D4C,0x84C0EC6,0x1E01F78A,0x1AB7A29,0x16729284,0x1EE6389A,0x1885C84F,0x21E1A45,0x6832F5B,0x702403C,0x162D75C2,0xC},{0x103663C1,0xA3C929D,0x3081B40,0x6D11DEC,0x12E7A07F,0x1195ADF3,0xF9BBB0C,0x1CAF1301,0x9601A6D,0x7D68757,0x14860450,0x15393164,0x112C4C3,0xB}}; - -#endif - -#if HTC_ISO_G2_BLS12381 != 0 -const BIG_384_29 CURVE_Adr_BLS12381= {0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0}; -const BIG_384_29 CURVE_Adi_BLS12381= {0xF0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0}; -const BIG_384_29 CURVE_Bdr_BLS12381= {0x3F4,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0}; -const BIG_384_29 CURVE_Bdi_BLS12381= {0x3F4,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0}; -const BIG_384_29 PCR_BLS12381[13]= {{0xAAA5ED1,0x7155555,0x19C71C62,0x11C71A1E,0x18575709,0x8478A15,0x2A88B58,0x1CFE9D02,0x14CB14B4,0x8FAFDB0,0x1B5B7A9A,0x147199F5,0x11D6541F,0xB},{0x1FFFC71E,0x154FFFFF,0x3555549,0x5555397,0xA418147,0x635A790,0x11FE6882,0x15BEF5C1,0xF984F87,0x16BC3E44,0xC849BF3,0x17553378,0x1560BF17,0x8},{0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0},{0xAAA97D6,0x11C55555,0x1671C718,0xC71C687,0xE15D5C2,0x211E285,0x10AA22D6,0x73FA740,0x532C52D,0x123EBF6C,0xED6DEA6,0x1D1C667D,0x1C759507,0x2},{0xC,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0},{0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0},{0x1C718B10,0xD9B8E38,0x1712F678,0x1212F4AD,0x74524E7,0x1BE34D51,0xA1AC3A5,0x6F43C4C,0x10761B0F,0xF1C08D6,0x1EFDC10F,0x16D9EF37,0x4C9AD43,0x9},{0x1FFFC71C,0x154FFFFF,0x3555549,0x5555397,0xA418147,0x635A790,0x11FE6882,0x15BEF5C1,0xF984F87,0x16BC3E44,0xC849BF3,0x17553378,0x1560BF17,0x8},{0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0},{0x11C6D706,0x167E38E3,0x124BDA04,0x184BD7F1,0x1E500FC8,0x1CEC3E93,0x126FD510,0x1A940FEC,0x130F7DA5,0x183B688C,0x16693062,0x15682276,0x130477C7,0xA},{0x12,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0},{0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0},{0x1FFFA8FB,0xFF7FFFF,0x14FFFFEE,0x17FFFD62,0xF6241EA,0x9507B58,0xAFD9CC3,0x109E70A2,0x1764774B,0x121A5D66,0x12C6E9ED,0x12FFCD34,0x111EA3,0xD}}; -const BIG_384_29 PCI_BLS12381[13]= {{0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0},{0x1FFFE38D,0x1AA7FFFF,0x11AAAAA4,0x12AAA9CB,0x520C0A3,0x31AD3C8,0x18FF3441,0x1ADF7AE0,0x7CC27C3,0x1B5E1F22,0x6424DF9,0x1BAA99BC,0xAB05F8B,0x4},{0x1FFFC71A,0x154FFFFF,0x3555549,0x5555397,0xA418147,0x635A790,0x11FE6882,0x15BEF5C1,0xF984F87,0x16BC3E44,0xC849BF3,0x17553378,0x1560BF17,0x8},{0xAAA97D6,0x11C55555,0x1671C718,0xC71C687,0xE15D5C2,0x211E285,0x10AA22D6,0x73FA740,0x532C52D,0x123EBF6C,0xED6DEA6,0x1D1C667D,0x1C759507,0x2},{0x1FFFAA9F,0xFF7FFFF,0x14FFFFEE,0x17FFFD62,0xF6241EA,0x9507B58,0xAFD9CC3,0x109E70A2,0x1764774B,0x121A5D66,0x12C6E9ED,0x12FFCD34,0x111EA3,0xD},{0x1FFFAA63,0xFF7FFFF,0x14FFFFEE,0x17FFFD62,0xF6241EA,0x9507B58,0xAFD9CC3,0x109E70A2,0x1764774B,0x121A5D66,0x12C6E9ED,0x12FFCD34,0x111EA3,0xD},{0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0,0x0},{0x1FFFE38F,0x1AA7FFFF,0x11AAAAA4,0x12AAA9CB,0x520C0A3,0x31AD3C8,0x18FF3441,0x1ADF7AE0,0x7CC27C3,0x1B5E1F22,0x6424DF9,0x1BAA99BC,0xAB05F8B,0x4},{0xAAA97BE,0x11C55555,0x1671C718,0xC71C687,0xE15D5C2,0x211E285,0x10AA22D6,0x73FA740,0x532C52D,0x123EBF6C,0xED6DEA6,0x1D1C667D,0x1C759507,0x2},{0x11C6D706,0x167E38E3,0x124BDA04,0x184BD7F1,0x1E500FC8,0x1CEC3E93,0x126FD510,0x1A940FEC,0x130F7DA5,0x183B688C,0x16693062,0x15682276,0x130477C7,0xA},{0x1FFFAA99,0xFF7FFFF,0x14FFFFEE,0x17FFFD62,0xF6241EA,0x9507B58,0xAFD9CC3,0x109E70A2,0x1764774B,0x121A5D66,0x12C6E9ED,0x12FFCD34,0x111EA3,0xD},{0x1FFFA9D3,0xFF7FFFF,0x14FFFFEE,0x17FFFD62,0xF6241EA,0x9507B58,0xAFD9CC3,0x109E70A2,0x1764774B,0x121A5D66,0x12C6E9ED,0x12FFCD34,0x111EA3,0xD},{0x1FFFA8FB,0xFF7FFFF,0x14FFFFEE,0x17FFFD62,0xF6241EA,0x9507B58,0xAFD9CC3,0x109E70A2,0x1764774B,0x121A5D66,0x12C6E9ED,0x12FFCD34,0x111EA3,0xD}}; - -#endif - -#endif - -#if CHUNK==64 - -const int CURVE_Cof_I_BLS12381= 0; - -const int CURVE_B_I_BLS12381= 4; -const BIG_384_58 CURVE_B_BLS12381= {0x4L,0x0L,0x0L,0x0L,0x0L,0x0L,0x0L}; -const BIG_384_58 CURVE_Order_BLS12381= {0x3FFFFFF00000001L,0x36900BFFF96FFBFL,0x180809A1D80553BL,0x14CA675F520CCE7L,0x73EDA7L,0x0L,0x0L}; -const BIG_384_58 CURVE_Gx_BLS12381= {0x33AF00ADB22C6BBL,0x17A0FFE5E86BBFEL,0x3A3F171BAC586C5L,0x13E5DD2E4168538L,0x4FA9AC0FC3688CL,0x65F5E509A558E3L,0x17F1D3A73L}; -const BIG_384_58 CURVE_Gy_BLS12381= {0xAA232946C5E7E1L,0x331D128A222B903L,0x18CB2C04B3EDD03L,0x25757402BD8036CL,0x1741D8AE4FCF5E0L,0xEAA83C68278C3BL,0x8B3F481EL}; -const BIG_384_58 CURVE_HTPC_BLS12381= {0xC51062BDE821B8L,0x1A5483B9715FEDFL,0x1BDD403FC31088BL,0x3D2523427FC11BBL,0x1A3D71BDA12F01DL,0x2DB2FDD36CE3D2AL,0x1F7462C8L}; - -const BIG_384_58 CURVE_Bnx_BLS12381= {0x201000000010000L,0x34L,0x0L,0x0L,0x0L,0x0L,0x0L}; -const BIG_384_58 CURVE_Cof_BLS12381= {0x201000000010001L,0x34L,0x0L,0x0L,0x0L,0x0L,0x0L}; -//const BIG_384_58 CURVE_Cof_BLS12381= {0xAAAB0000AAABL,0x3230015557855A3L,0x396L,0x0L,0x0L,0x0L,0x0L}; - - -const BIG_384_58 CURVE_Pxa_BLS12381= {0x8056C8C121BDB8L,0x300C9AA016EFBF5L,0xB647AE3D1770BAL,0x353E900EC0AD144L,0x32DC51051C6E47AL,0x23C2A449820149L,0x24AA2B2FL}; -const BIG_384_58 CURVE_Pxb_BLS12381= {0x1AC7D055D042B7EL,0x33C4484E51755F9L,0x21BBDC7F5049334L,0x3426482D86AD769L,0x88274F65596BD0L,0x9C67D81F6B34E8L,0x13E02B605L}; -const BIG_384_58 CURVE_Pya_BLS12381= {0x193548608B82801L,0x2B2730EEB28A278L,0x1A695160D12C923L,0x2AA32F74E9DB50AL,0x2DA2E351AADFD9BL,0x9F5B8463327371L,0xCE5D5277L}; -const BIG_384_58 CURVE_Pyb_BLS12381= {0x2A9075FF05F79BEL,0x1C349D73B07686AL,0x12AB572E99AB3F3L,0x1FA169D8EBC99D2L,0x2BC28B99CB3E28L,0x3A9CD330CAB34ACL,0x606C4A02L}; - -#if HTC_ISO_BLS12381 != 0 -const BIG_384_58 CURVE_Ad_BLS12381= {0xF428082D584C1DL,0xDBE368383E5FD7L,0x181AEFD881AC989L,0x14E0FB99AA363A2L,0x2C96D4982B0EA98L,0xEE3A50CF5A4E80L,0x144698AL}; -const BIG_384_58 CURVE_Bd_BLS12381= {0x1CC48E98E172BE0L,0xC8568C5B3AA974L,0x14FCEF35EF55A2L,0x3C3C93D01C282E7L,0x753EEE3B2016C1L,0x5A200C0062C4BAL,0x12E2908D1L}; -const BIG_384_58 PC_BLS12381[53]= {{0x1C8BA2E8BA2D229L,0x2C6E02D934E47EAL,0x3F1BC24C6B68C24L,0x1F88B20DEF08F02L,0x381EDEE3D31D79DL,0x389839C2F47A588L,0x6E08C248L},{0x267DF3F1605FB7BL,0x2DDC7E30A177B32L,0x336003B14866F69L,0x37799E1FE5B542BL,0x1D2565B0DFA7DCCL,0x27381F89CB63B02L,0x10321DA07L},{0x3241067BE390C9EL,0x242CBB700C9DE5FL,0x14BAF4BB1B7FA31L,0x200E83172659D8CL,0x15D138F22DD2ECBL,0x2F3E9F10B830DD4L,0x169B1F8E1L},{0x171986A8497E317L,0xA57CA5ADD3A55BL,0x16C928C5D1DE4FAL,0x1B39E7D55D28B16L,0x163BE990DC43B75L,0x269E3F11EE42CCDL,0x80D3CF1FL},{0xCB5618E3F0C88EL,0x1F23E323D1D6BE7L,0x62EF0F2753339BL,0x2AC9D6D36C69A0BL,0xD1117E53356DE5L,0x6AF6F8BA1D0E21L,0x17B81E770L},{0x1D7F225A139ED84L,0x944A30414BB2B7L,0x2218F9C86B2A8DAL,0x993C3E33864023L,0x38AE652BFB11586L,0x3F9134A5A8DC9B0L,0xD6ED6553L},{0x113C1C66F652983L,0x1C34B72B9CF4673L,0x2B9097E68F90A08L,0x1F76549E66E7B4EL,0x3F7A74AB5DB3CB1L,0x35CC4FFC0744806L,0x1630C3250L},{0x1154CE9AC8895D9L,0x28A1BCC079DF114L,0x2B65982FAC18985L,0x168495FECFC21BBL,0x3E4118E5499DB99L,0x667D10D990AD2CL,0xE99726A3L},{0x1B388641D9B6861L,0x1B89738C41C64F1L,0x3289F1B33083533L,0x195AA36FC97C6CCL,0x307E55412D7F5E4L,0x3F31B6DD3818274L,0x1778E7166L},{0x179F9DAC9EDCB0L,0x30F8F4A825CA7F8L,0x2501EC68E25C958L,0x1CCA5660F95A1E3L,0x1D10A9A1BCE0324L,0x25D9E3B07441231L,0xD54005DBL},{0x34EEF1B3CB83BBL,0x23CA9BCC630D5BAL,0x233C70D1E86B483L,0x16CBDAA105FD597L,0x22147A81C7C17E7L,0x250EACBC1622EACL,0x17294ED3EL},{0x2AC1662734649B7L,0x30B57CB98B5BABL,0x3B56CDB4E2C8561L,0x2228B5C017FC989L,0x1D99815856B303EL,0x3A0CCD02E024407L,0x11A05F2B1L},{0x16384D168ECDD0AL,0x1D392D2DE19400BL,0x133978F31C15931L,0x3BA5BDF40DDDB7DL,0x2B3A56680F682B4L,0x27A4AB511DB5B8FL,0x95FC13ABL},{0x376EC3A79A1D641L,0x99A4AAEE90DC11L,0xDA67F398835038L,0x75C584D9ADD040L,0x1AFC7A3CCE07F8DL,0x36953E097A482CFL,0xA10ECF6AL},{0x1F7D99BBDCC5A5EL,0x16E52274478B4C4L,0x21CDF9822C580FAL,0x3086F29A2A0665BL,0x74CF01996E7F63L,0x3592A2C8C2CFD6CL,0x14A7AC2A9L},{0x2574496EE84A3AL,0xECD4E3C3781B3BL,0x73062AEDE9CEA7L,0x266BD4E862538B8L,0x3E0596721570F57L,0x5A4D8643CF8318L,0x772CAACFL},{0x2DF9A29F6304A5L,0x3492F108A3C470L,0x3CEF24B8982F740L,0x3A73A72B534290EL,0x30506C6E9395735L,0x13999EE554E43DFL,0xE7355F8EL},{0x39D395B3532A21EL,0xA6EA07CD5E0754L,0x4E833B306DA9BDL,0x16684818AEE35ADL,0x343E7A07DFFDFC7L,0x8A452A029BC757L,0x13A8E1620L},{0x30DE8938DC62CD8L,0x1B5490FBB3D7104L,0x28ABC28D6FD0497L,0xFC5AC595455332L,0x37C40EB545B0824L,0x162B8BFB20EABFBL,0x3425581AL},{0xC239BA5CB83E19L,0xF4259F253FB73FL,0xE00B11ACEACD6AL,0x1BD69C63347F299L,0x1BFF2991F6F8941L,0x1E8C897A04DF98AL,0xB2962FE5L},{0x1C8276EC82B3BFFL,0x2AA211B2C09BA79L,0x2588C48BF5713DL,0x32833C20030049BL,0x298E536367041E8L,0x2D56710D22D1C44L,0x12561A5DEL},{0x13CF9FA40D21B1CL,0x235A06F8D0F7E26L,0x8617FC8AC62B55L,0x12E8D6D22EA7256L,0x34BD3FA6F01D5EFL,0x33FC66B862CB98BL,0x8CA8D548L},{0xB456BE69C8B604L,0x1409FBFB0071DC1L,0x14FA95AF01B2B66L,0x23E125968E55EB7L,0x342DF2EB5CB181DL,0x243C0F393A942CEL,0x15E6BE4E9L},{0x26B1E715475224BL,0x4126D95E6BEDE1L,0xF5D396A7CE46BAL,0x2075FA195A366ACL,0x348C4A3FC5E673DL,0x39133C440A8567DL,0x5C129645L},{0x2D9D3F5DB980133L,0x3E42B4708CA9910L,0x232D3C40659CC6CL,0x20353056004F99L,0x27BE315DC757B3BL,0x347B2A6DCBF002BL,0x245A394AL},{0x14C04F00B971EF8L,0x214706464847C83L,0x10E807B4633F06CL,0xA8D09AC23B009CL,0x4F53F447AA7B1L,0x6E4E674554258L,0xB182CAC1L},{0x207C8A4D0074D8EL,0x2737D06D13581B3L,0x3E7F911F643249DL,0x2E2ABC30918B9AFL,0x3FED2EDCC523559L,0x3CDBDB7AE463050L,0x18B46A908L},{0x13711AD011C132L,0x3CE97338FEEBF3AL,0x3E416389E61031BL,0x32DB2BD24FF4460L,0x31D43FB93CD2FCBL,0xDF346F837F42E3L,0x19713E479L},{0x3AFAAEBCA731C30L,0x3DC157753AE9BCAL,0x1E7ED1E4D43B9B3L,0x29E456BDBF81A61L,0x3ADA14A23C42A0CL,0x61AF6D488EAF79L,0xE1BBA7A1L},{0x370E577BDBA587L,0x1948071E181E8D8L,0x2E6A1F20CABE69DL,0x599E7709B07A2DL,0x21E4DA1BB8F3ABDL,0x3659A12FA232788L,0x9FC4018BL},{0x15E4CA31870FB29L,0x191543FB7FA4D68L,0xDA6C26C842642FL,0x2FF8EF7607FF40EL,0x12CA6C674170A05L,0xCEAE1BF7A649AFL,0x987C8D53L},{0x161F8855FE9D6F2L,0x21EB09183D057B2L,0x13C4D634F3747AL,0x328AF86132D48C5L,0x27796B3CE75BB8L,0x3EB06EF2CB25DF4L,0x4AB0B9BCL},{0x1B23AB13633A5F0L,0x3D8C9B256A01CA6L,0x1C3D3AD5544E203L,0x352BEB6DEF5D941L,0x1B8F0A6A074A7D0L,0x18D2DA88847847L,0x16603FCA4L},{0x1B6DAECF2E8FEDBL,0x1FE370264102A10L,0x3FD221351ADC2EEL,0x3EF8F3942E1E60CL,0x2A21529C4195536L,0x3F83FC4D72BD3F8L,0x8CC03FDEL},{0x2355C77B0E5F4CBL,0x16AEA7B1877B29L,0x23EC03251CF9DE4L,0x2E43BADE4702792L,0x2D8746757D42AA7L,0x22607085E261D46L,0x1F86376EL},{0xDFE240C72DE1F6L,0x354858A2C0148EEL,0x3E4B91400DA7D26L,0x359628C738B0D12L,0x6A3B49942552E2L,0x2A59B99BD28E132L,0xCC786BAL},{0x97E75A2E41C696L,0x159C4658BEA2FF8L,0x2343EB67AD34D6CL,0x1B0953CE0F43E41L,0x376FB46831223E9L,0x13B960475440DB5L,0x134996A10L},{0x29845719707BB33L,0x31EBBA6CEE8F0AFL,0x2F6C956543D3CD0L,0x23922A1A548AD4AL,0x14980DCFA11AD13L,0x2E893B8096747C2L,0x90D97C81L},{0x15473A1D634B8FL,0xBD5C3C4D25E011L,0x3CD6356CAA205CAL,0x19789CEE14CC93BL,0x20D7819C171C40FL,0x1B7700F9AC90957L,0xE0FA1D81L},{0x1FADC1326ED06F7L,0x145EF61C5332034L,0xDF27942480E420L,0x2539CA49F072DD2L,0x153CD76F2BF565BL,0x2CB93CED8A2F743L,0x2660400EL},{0x299B138573345CCL,0x1D8F8EE42B047L,0x2EF9A00D9B86930L,0x3662B7C0899F573L,0xB45F1496543346L,0x31D9FF8F0D84C51L,0xAD6B9514L},{0x284B529E2561092L,0x25A261BDFAEFAA5L,0x1A88CEA7913516FL,0x22BBF390B4A303EL,0x248C50C477F94FFL,0x20740CFFD614B07L,0xACCBB674L},{0xF8B49CBA8F6AA8L,0x170A7D3E0C18100L,0x1B36E636A5C871AL,0xE6ED8698A43964L,0x1AD2911D9C6DD0L,0x3A9016F523C0428L,0x4D2F259EL},{0x6EF48BB8913F55L,0x217A8F54A6CD78DL,0x192E7EA7D4FBC73L,0x18F84F61EED4C21L,0x3D94A84903216F7L,0x1C29B873AA08165L,0x167A55CDAL},{0x3233D9D55535D4AL,0x3F8BDEEE49220DAL,0x350C4BF39B4852CL,0x3931ABD6482AF15L,0x3D1D74CC4F9FB0CL,0xDB1848C686F953L,0x1866C8ED3L},{0xEE415A15812ED9L,0x3D6C020077B918L,0xFD206357132B92L,0x17BE87D3F5FFACDL,0x2BBA6FF6EE5A437L,0x38FA9FA80EF377EL,0x16A3EF08BL},{0x11A1399126A775CL,0x2A7006962C7EE4FL,0x25BC400A0051D5FL,0x3EA3433E3BD774DL,0xACE9824B5EECFDL,0x2A676CBF0EEA1CDL,0x166007C08L},{0x2C6477FAAF9B7ACL,0xE36E77EA733880L,0x187B6F0F5A6449FL,0x3195543620717B3L,0x2AC783182B70152L,0x61B6CB67EC99BAL,0x8D9E5297L},{0x239142311A5001DL,0x2C57703F4BB7B76L,0x1A0FC9DEC916A20L,0x27C3DA6EEC150BBL,0x2F8228DDCC6D19CL,0x117D0F92C033244L,0xBE0E0795L},{0xD26D98445F5416L,0xD93CB0A0A5EB6AL,0x2489E726AF41727L,0x36F76F34C3848F6L,0x389EDB4D1D115C5L,0x26394E57C8348EFL,0x16B7D2887L},{0x22538B53DBF67F2L,0x15F358DBE5BE247L,0x25DD279CD2ECA67L,0x15546B9FCC430D6L,0x16E8EB15778C485L,0x1903689DBEAAB9FL,0x58DF3306L},{0x2F6102C2E49A03DL,0x10981D8D4A78D4CL,0x356F453E01F78AL,0x3DCC71356729284L,0x43C348B885C84FL,0xE0480786832F5BL,0x1962D75C2L},{0x1479253B03663C1L,0xDA23BD83081B40L,0x232B5BE72E7A07FL,0x395E2602F9BBB0CL,0xFAD0EAE9601A6DL,0x2A7262C94860450L,0x16112C4C3L}}; -#endif - -#if HTC_ISO_G2_BLS12381 != 0 -const BIG_384_58 CURVE_Adr_BLS12381= {0x0L,0x0L,0x0L,0x0L,0x0L,0x0L,0x0L}; -const BIG_384_58 CURVE_Adi_BLS12381= {0xF0L,0x0L,0x0L,0x0L,0x0L,0x0L,0x0L}; -const BIG_384_58 CURVE_Bdr_BLS12381= {0x3F4L,0x0L,0x0L,0x0L,0x0L,0x0L,0x0L}; -const BIG_384_58 CURVE_Bdi_BLS12381= {0x3F4L,0x0L,0x0L,0x0L,0x0L,0x0L,0x0L}; -const BIG_384_58 PCR_BLS12381[13]= {{0xE2AAAAAAAA5ED1L,0x238E343D9C71C62L,0x108F142B8575709L,0x39FD3A042A88B58L,0x11F5FB614CB14B4L,0x28E333EBB5B7A9AL,0x171D6541FL},{0x2A9FFFFFFFFC71EL,0xAAAA72E3555549L,0xC6B4F20A418147L,0x2B7DEB831FE6882L,0x2D787C88F984F87L,0x2EAA66F0C849BF3L,0x11560BF17L},{0x0L,0x0L,0x0L,0x0L,0x0L,0x0L,0x0L},{0x238AAAAAAAA97D6L,0x18E38D0F671C718L,0x423C50AE15D5C2L,0xE7F4E810AA22D6L,0x247D7ED8532C52DL,0x3A38CCFAED6DEA6L,0x5C759507L},{0xCL,0x0L,0x0L,0x0L,0x0L,0x0L,0x0L},{0x0L,0x0L,0x0L,0x0L,0x0L,0x0L,0x0L},{0x1B371C71C718B10L,0x2425E95B712F678L,0x37C69AA274524E7L,0xDE87898A1AC3A5L,0x1E3811AD0761B0FL,0x2DB3DE6FEFDC10FL,0x124C9AD43L},{0x2A9FFFFFFFFC71CL,0xAAAA72E3555549L,0xC6B4F20A418147L,0x2B7DEB831FE6882L,0x2D787C88F984F87L,0x2EAA66F0C849BF3L,0x11560BF17L},{0x0L,0x0L,0x0L,0x0L,0x0L,0x0L,0x0L},{0x2CFC71C71C6D706L,0x3097AFE324BDA04L,0x39D87D27E500FC8L,0x35281FD926FD510L,0x3076D11930F7DA5L,0x2AD044ED6693062L,0x1530477C7L},{0x12L,0x0L,0x0L,0x0L,0x0L,0x0L,0x0L},{0x0L,0x0L,0x0L,0x0L,0x0L,0x0L,0x0L},{0x1FEFFFFFFFFA8FBL,0x2FFFFAC54FFFFEEL,0x12A0F6B0F6241EAL,0x213CE144AFD9CC3L,0x2434BACD764774BL,0x25FF9A692C6E9EDL,0x1A0111EA3L}}; -const BIG_384_58 PCI_BLS12381[13]= {{0x0L,0x0L,0x0L,0x0L,0x0L,0x0L,0x0L},{0x354FFFFFFFFE38DL,0x255553971AAAAA4L,0x635A790520C0A3L,0x35BEF5C18FF3441L,0x36BC3E447CC27C3L,0x375533786424DF9L,0x8AB05F8BL},{0x2A9FFFFFFFFC71AL,0xAAAA72E3555549L,0xC6B4F20A418147L,0x2B7DEB831FE6882L,0x2D787C88F984F87L,0x2EAA66F0C849BF3L,0x11560BF17L},{0x238AAAAAAAA97D6L,0x18E38D0F671C718L,0x423C50AE15D5C2L,0xE7F4E810AA22D6L,0x247D7ED8532C52DL,0x3A38CCFAED6DEA6L,0x5C759507L},{0x1FEFFFFFFFFAA9FL,0x2FFFFAC54FFFFEEL,0x12A0F6B0F6241EAL,0x213CE144AFD9CC3L,0x2434BACD764774BL,0x25FF9A692C6E9EDL,0x1A0111EA3L},{0x1FEFFFFFFFFAA63L,0x2FFFFAC54FFFFEEL,0x12A0F6B0F6241EAL,0x213CE144AFD9CC3L,0x2434BACD764774BL,0x25FF9A692C6E9EDL,0x1A0111EA3L},{0x0L,0x0L,0x0L,0x0L,0x0L,0x0L,0x0L},{0x354FFFFFFFFE38FL,0x255553971AAAAA4L,0x635A790520C0A3L,0x35BEF5C18FF3441L,0x36BC3E447CC27C3L,0x375533786424DF9L,0x8AB05F8BL},{0x238AAAAAAAA97BEL,0x18E38D0F671C718L,0x423C50AE15D5C2L,0xE7F4E810AA22D6L,0x247D7ED8532C52DL,0x3A38CCFAED6DEA6L,0x5C759507L},{0x2CFC71C71C6D706L,0x3097AFE324BDA04L,0x39D87D27E500FC8L,0x35281FD926FD510L,0x3076D11930F7DA5L,0x2AD044ED6693062L,0x1530477C7L},{0x1FEFFFFFFFFAA99L,0x2FFFFAC54FFFFEEL,0x12A0F6B0F6241EAL,0x213CE144AFD9CC3L,0x2434BACD764774BL,0x25FF9A692C6E9EDL,0x1A0111EA3L},{0x1FEFFFFFFFFA9D3L,0x2FFFFAC54FFFFEEL,0x12A0F6B0F6241EAL,0x213CE144AFD9CC3L,0x2434BACD764774BL,0x25FF9A692C6E9EDL,0x1A0111EA3L},{0x1FEFFFFFFFFA8FBL,0x2FFFFAC54FFFFEEL,0x12A0F6B0F6241EAL,0x213CE144AFD9CC3L,0x2434BACD764774BL,0x25FF9A692C6E9EDL,0x1A0111EA3L}}; -#endif - -#endif diff --git a/impl/cbits/rom_field_BLS12381.c b/impl/cbits/rom_field_BLS12381.c deleted file mode 100644 index 79ffc7859..000000000 --- a/impl/cbits/rom_field_BLS12381.c +++ /dev/null @@ -1,52 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -#include "arch.h" -#include "fp_BLS12381.h" - -/* Curve BLS12381 - Pairing friendly BLS curve */ - -#if CHUNK==16 - -#error Not supported - -#endif - -#if CHUNK==32 -// Base Bits= 29 -const BIG_384_29 Modulus_BLS12381= {0x1FFFAAAB,0xFF7FFFF,0x14FFFFEE,0x17FFFD62,0xF6241EA,0x9507B58,0xAFD9CC3,0x109E70A2,0x1764774B,0x121A5D66,0x12C6E9ED,0x12FFCD34,0x111EA3,0xD}; -const BIG_384_29 ROI_BLS12381= {0x1FFFAAAA,0xFF7FFFF,0x14FFFFEE,0x17FFFD62,0xF6241EA,0x9507B58,0xAFD9CC3,0x109E70A2,0x1764774B,0x121A5D66,0x12C6E9ED,0x12FFCD34,0x111EA3,0xD}; -const BIG_384_29 R2modp_BLS12381= {0x15BEF7AE,0x1031CD0E,0x2DD93E8,0x9226323,0xE6E2CD2,0x11684DAA,0x1170E5DB,0x88E25B1,0x1B366399,0x1C536F47,0xD1F9CBC,0x278B67F,0x1EA66A2B,0xC}; -const chunk MConst_BLS12381= 0x1FFCFFFD; -const BIG_384_29 CRu_BLS12381= {0x1FFEFFFE,0x100FFFFF,0x280008B,0xFB026C4,0x9688DE1,0x149DF37C,0x1FAB76CE,0xED41EE,0x11BA69C6,0x1EFBB672,0x17C659CB,0x0,0x0,0x0}; -const BIG_384_29 Fra_BLS12381= {0x12235FB8,0x83BAF6C,0x19E04F63,0x1D4A7AC7,0xB9C4F67,0x1EBC25D,0x1D3DEC91,0x1FA797AB,0x1F0FD603,0x1016068,0x108C6FAD,0x5760CCF,0x104D3BF0,0xC}; -const BIG_384_29 Frb_BLS12381= {0xDDC4AF3,0x7BC5093,0x1B1FB08B,0x1AB5829A,0x3C5F282,0x764B8FB,0xDBFB032,0x10F6D8F6,0x1854A147,0x1118FCFD,0x23A7A40,0xD89C065,0xFC3E2B3,0x0}; -const BIG_384_29 SQRTm3_BLS12381= {0x1AAAE,0xFD80000,0xFFFFED7,0x189FAFDA,0x1C912627,0x14945F,0xBA6AF26,0xEC3ECC4,0x13EFA3BF,0x1422F081,0x33A3655,0x12FFCD33,0x111EA3,0xD}; -#endif - -#if CHUNK==64 -// Base Bits= 58 -const BIG_384_58 Modulus_BLS12381= {0x1FEFFFFFFFFAAABL,0x2FFFFAC54FFFFEEL,0x12A0F6B0F6241EAL,0x213CE144AFD9CC3L,0x2434BACD764774BL,0x25FF9A692C6E9EDL,0x1A0111EA3L}; -const BIG_384_58 ROI_BLS12381= {0x1FEFFFFFFFFAAAAL,0x2FFFFAC54FFFFEEL,0x12A0F6B0F6241EAL,0x213CE144AFD9CC3L,0x2434BACD764774BL,0x25FF9A692C6E9EDL,0x1A0111EA3L}; -const BIG_384_58 R2modp_BLS12381= {0x20639A1D5BEF7AEL,0x1244C6462DD93E8L,0x22D09B54E6E2CD2L,0x111C4B63170E5DBL,0x38A6DE8FB366399L,0x4F16CFED1F9CBCL,0x19EA66A2BL}; -const chunk MConst_BLS12381= 0x1F3FFFCFFFCFFFDL; -const BIG_384_58 CRu_BLS12381= {0x201FFFFFFFEFFFEL,0x1F604D88280008BL,0x293BE6F89688DE1L,0x1DA83DDFAB76CEL,0x3DF76CE51BA69C6L,0x17C659CBL,0x0L}; -const BIG_384_58 Fra_BLS12381= {0x10775ED92235FB8L,0x3A94F58F9E04F63L,0x3D784BAB9C4F67L,0x3F4F2F57D3DEC91L,0x202C0D1F0FD603L,0xAEC199F08C6FADL,0x1904D3BF0L}; -const BIG_384_58 Frb_BLS12381= {0xF78A126DDC4AF3L,0x356B0535B1FB08BL,0xEC971F63C5F282L,0x21EDB1ECDBFB032L,0x2231F9FB854A147L,0x1B1380CA23A7A40L,0xFC3E2B3L}; -const BIG_384_58 SQRTm3_BLS12381= {0x1FB00000001AAAEL,0x313F5FB4FFFFED7L,0x2928BFC912627L,0x1D87D988BA6AF26L,0x2845E1033EFA3BFL,0x25FF9A6633A3655L,0x1A0111EA3L}; -#endif diff --git a/impl/cbits/share.c b/impl/cbits/share.c deleted file mode 100644 index 534668334..000000000 --- a/impl/cbits/share.c +++ /dev/null @@ -1,230 +0,0 @@ -/* - * Copyright (c) 2012-2020 MIRACL UK Ltd. - * - * This file is part of MIRACL Core - * (see https://github.com/miracl/core). - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -/* Shamir threshold secret sharing module */ -/* Split any octet into number of shares <256 */ -/* Specify number of shares required for recovery - nsr */ - -/* See testmpin.c for an example of use */ - -#include "arch.h" -#include "core.h" - -/* Field GF(2^8) precalculated tables */ - -static const uchar ptab[] = -{ - 1, 3, 5, 15, 17, 51, 85, 255, 26, 46, 114, 150, 161, 248, 19, 53, - 95, 225, 56, 72, 216, 115, 149, 164, 247, 2, 6, 10, 30, 34, 102, 170, - 229, 52, 92, 228, 55, 89, 235, 38, 106, 190, 217, 112, 144, 171, 230, 49, - 83, 245, 4, 12, 20, 60, 68, 204, 79, 209, 104, 184, 211, 110, 178, 205, - 76, 212, 103, 169, 224, 59, 77, 215, 98, 166, 241, 8, 24, 40, 120, 136, - 131, 158, 185, 208, 107, 189, 220, 127, 129, 152, 179, 206, 73, 219, 118, 154, - 181, 196, 87, 249, 16, 48, 80, 240, 11, 29, 39, 105, 187, 214, 97, 163, - 254, 25, 43, 125, 135, 146, 173, 236, 47, 113, 147, 174, 233, 32, 96, 160, - 251, 22, 58, 78, 210, 109, 183, 194, 93, 231, 50, 86, 250, 21, 63, 65, - 195, 94, 226, 61, 71, 201, 64, 192, 91, 237, 44, 116, 156, 191, 218, 117, - 159, 186, 213, 100, 172, 239, 42, 126, 130, 157, 188, 223, 122, 142, 137, 128, - 155, 182, 193, 88, 232, 35, 101, 175, 234, 37, 111, 177, 200, 67, 197, 84, - 252, 31, 33, 99, 165, 244, 7, 9, 27, 45, 119, 153, 176, 203, 70, 202, - 69, 207, 74, 222, 121, 139, 134, 145, 168, 227, 62, 66, 198, 81, 243, 14, - 18, 54, 90, 238, 41, 123, 141, 140, 143, 138, 133, 148, 167, 242, 13, 23, - 57, 75, 221, 124, 132, 151, 162, 253, 28, 36, 108, 180, 199, 82, 246, 1 -}; - -static const uchar ltab[] = -{ - 0, 255, 25, 1, 50, 2, 26, 198, 75, 199, 27, 104, 51, 238, 223, 3, - 100, 4, 224, 14, 52, 141, 129, 239, 76, 113, 8, 200, 248, 105, 28, 193, - 125, 194, 29, 181, 249, 185, 39, 106, 77, 228, 166, 114, 154, 201, 9, 120, - 101, 47, 138, 5, 33, 15, 225, 36, 18, 240, 130, 69, 53, 147, 218, 142, - 150, 143, 219, 189, 54, 208, 206, 148, 19, 92, 210, 241, 64, 70, 131, 56, - 102, 221, 253, 48, 191, 6, 139, 98, 179, 37, 226, 152, 34, 136, 145, 16, - 126, 110, 72, 195, 163, 182, 30, 66, 58, 107, 40, 84, 250, 133, 61, 186, - 43, 121, 10, 21, 155, 159, 94, 202, 78, 212, 172, 229, 243, 115, 167, 87, - 175, 88, 168, 80, 244, 234, 214, 116, 79, 174, 233, 213, 231, 230, 173, 232, - 44, 215, 117, 122, 235, 22, 11, 245, 89, 203, 95, 176, 156, 169, 81, 160, - 127, 12, 246, 111, 23, 196, 73, 236, 216, 67, 31, 45, 164, 118, 123, 183, - 204, 187, 62, 90, 251, 96, 177, 134, 59, 82, 161, 108, 170, 85, 41, 157, - 151, 178, 135, 144, 97, 190, 220, 252, 188, 149, 207, 205, 55, 63, 91, 209, - 83, 57, 132, 60, 65, 162, 109, 71, 20, 42, 158, 93, 86, 242, 211, 171, - 68, 17, 146, 217, 35, 32, 46, 137, 180, 124, 184, 38, 119, 153, 227, 165, - 103, 74, 237, 222, 197, 49, 254, 24, 13, 99, 140, 128, 192, 247, 112, 7 -}; - -/* field addition */ -static uchar add(uchar x,uchar y) -{ - return (x^y); -} - -/* x.y= AntiLog(Log(x) + Log(y)) */ -static uchar mul(uchar x,uchar y) -{ - if (x && y) return ptab[(ltab[x]+ltab[y])%255]; - else return 0; -} - -/* multiplicative inverse */ -static uchar inv(uchar x) -{ - return ptab[255-ltab[x]]; -} - -/* Lagrange interpolation */ -static int interpolate(int n, uchar x[], uchar y[]) -{ - int i,j; - uchar p,yp=0; - for(i=0;i=256 || nsr<2 || nsr>=256) - { - Sh.id=0; - Sh.nsr=0; - Sh.B=NULL; - return Sh; - } - RAND_clean(&rng); - RAND_seed(&rng,R->len,R->val); - Sh.id=id; - Sh.nsr=nsr; - Sh.B=S; - m=M->len; - S->len=m; - for (j=0;jval[j]=M->val[j]; - for (n=1;nval[j]=add(S->val[j],mul(RAND_byte(&rng),x)); - x=mul(x,(uchar)id); - } - } - return Sh; -} - -/* Recover message from shares */ - -int recover(octet *M,share S[]) -{ - uchar x[256],y[256]; - int i,j,res=0; - int len=S[0].B->len; - int nsr=S[0].nsr; - for (i=1;ilen!=len) - { - res=-1; - break; - } - } - if (res) return res; - - for (j=0;jval[j]; - } - M->val[j]=interpolate(nsr,x,y); - } - M->len=len; - return res; -} - -/* -#include -#include - -int main() -{ // test driver - int ii,j,k,l,m,n,nsr; - char mc[10],b1c[10],b2c[10],b3c[10],b4c[10],r[30]; - octet M={0,sizeof(mc),mc}; - octet B1={0,sizeof(b1c),b1c}; - octet B2={0,sizeof(b2c),b2c}; - octet B3={0,sizeof(b3c),b3c}; - octet B4={0,sizeof(b4c),b4c}; - octet R={0,sizeof(r),r}; - share S[3]; - - nsr=3; - - srand(time(NULL)); - - M.len=5; - M.val[0]=rand()%128; M.val[1]=rand()%128; M.val[2]=rand()%128; M.val[3]=rand()%128; M.val[4]=rand()%128; - - printf("Message= "); OCT_output(&M); - - R.len=(nsr-1)*M.len; - for (j=0;jnc~3!1}vAdb5lz)@>AmT5_3~i WGg5PM@{<#b(%oG=U3?h+Y5@Sb92wRC diff --git a/impl/ic-ref.cabal b/impl/ic-ref.cabal deleted file mode 100644 index 3206d9174..000000000 --- a/impl/ic-ref.cabal +++ /dev/null @@ -1,253 +0,0 @@ -cabal-version: 2.2 -name: ic-ref -version: 0.0.1 -author: Joachim Breitner -build-type: Simple -extra-source-files: cbits/*.h ic.did - -flag release - default: False - description: Release build, warnings are errors - --- NB: It might look odd that we are using common stanzas instead of --- internal library. The main reason is that otherwise, --- --- > ghcid -c "cabal new-repl" --- --- cannot be used on all files, due to --- --- > cabal: Cannot open a repl for multiple components at once. - -common version - hs-source-dirs: src - other-modules: IC.Version - other-modules: SourceId - build-depends: text - build-depends: template-haskell - build-depends: process - build-depends: split - -common certificates - hs-source-dirs: src - other-modules: IC.CBOR.Patterns - other-modules: IC.CBOR.Parser - other-modules: IC.HashTree.CBOR - other-modules: IC.HashTree - other-modules: IC.Certificate - other-modules: IC.Certificate.CBOR - other-modules: IC.Certificate.Value - other-modules: IC.Certificate.Validate - build-depends: leb128-cereal - build-depends: cborg - -common crypto - other-modules: IC.Crypto - other-modules: IC.Crypto.DER.Decode - other-modules: IC.Crypto.DER - other-modules: IC.Crypto.Ed25519 - other-modules: IC.Crypto.ECDSA - other-modules: IC.Crypto.Secp256k1 - other-modules: IC.Crypto.BLS - other-modules: IC.Crypto.DER_BLS - other-modules: IC.Crypto.WebAuthn - other-modules: IC.Crypto.CanisterSig - build-depends: ed25519 - build-depends: cryptonite - build-depends: cereal - build-depends: memory - build-depends: bindings-DSL - build-depends: asn1-types - build-depends: asn1-encoding - build-depends: hashable - build-depends: parallel - build-depends: base64-bytestring >= 1.1 - build-depends: aeson - include-dirs: cbits - c-sources: cbits/aes.c - c-sources: cbits/rom_curve_BLS12381.c - c-sources: cbits/rom_field_BLS12381.c - c-sources: cbits/big_384_58.c - c-sources: cbits/bls_BLS12381.c - c-sources: cbits/ecp2_BLS12381.c - c-sources: cbits/ecp_BLS12381.c - c-sources: cbits/fp12_BLS12381.c - c-sources: cbits/fp2_BLS12381.c - c-sources: cbits/fp4_BLS12381.c - c-sources: cbits/fp_BLS12381.c - c-sources: cbits/gcm.c - c-sources: cbits/hash.c - c-sources: cbits/hmac.c - c-sources: cbits/newhope.c - c-sources: cbits/oct.c - c-sources: cbits/pair_BLS12381.c - c-sources: cbits/randapi.c - c-sources: cbits/rand.c - c-sources: cbits/share.c - -common cbor - hs-source-dirs: src - other-modules: IC.CBOR.Utils - other-modules: IC.HTTP.GenR - other-modules: IC.HTTP.CBOR - other-modules: IC.HTTP.GenR.Parse - other-modules: IC.HTTP.RequestId - other-modules: IC.Hash - other-modules: IC.Id.Forms - other-modules: IC.Types - build-depends: base >=4.12 && <5 - build-depends: base32 - build-depends: text - build-depends: bytestring - build-depends: containers - build-depends: crc - build-depends: unordered-containers - build-depends: mtl - build-depends: cborg - build-depends: cryptonite - build-depends: memory - build-depends: leb128-cereal - -common logic - hs-source-dirs: src - other-modules: IC.Ref - other-modules: IC.Types - other-modules: IC.Constants - other-modules: IC.Canister - other-modules: IC.Id.Fresh - other-modules: IC.Id.Forms - other-modules: IC.Utils - other-modules: IC.Hash - other-modules: IC.Canister.Imp - other-modules: IC.Canister.Snapshot - other-modules: IC.Purify - other-modules: IC.Management - other-modules: IC.Wasm.Winter - other-modules: IC.Wasm.WinterMemory - other-modules: IC.Wasm.Winter.Persist - other-modules: IC.Wasm.Imports - build-depends: base >=4.12 && <5 - build-depends: filepath - build-depends: split - build-depends: hex-text - build-depends: crc - build-depends: text - build-depends: bytestring - build-depends: containers - build-depends: winter - build-depends: binary - build-depends: mtl - build-depends: transformers - build-depends: data-default-class - build-depends: vector - build-depends: primitive - build-depends: utf8-string - build-depends: hex-text - build-depends: cryptonite - build-depends: memory - build-depends: row-types - build-depends: candid - build-depends: base32 - build-depends: time - build-depends: MonadRandom - build-depends: ed25519 - -common serialise - hs-source-dirs: src - other-modules: IC.Serialise - other-modules: IC.StateFile - build-depends: serialise - build-depends: random >= 1.2 - build-depends: splitmix - build-depends: directory - build-depends: atomic-write - build-depends: zlib - -common ghc-flags - default-language: Haskell2010 - ghc-options: -rtsopts - ghc-options: -Wall -Wno-name-shadowing - if flag(release) - ghc-options: -Werror - -executable ic-ref-run - import: version, logic, serialise, crypto, cbor, certificates, ghc-flags - main-is: ic-ref-run.hs - hs-source-dirs: src - other-modules: IC.DRun.Parse - build-depends: optparse-applicative - build-depends: prettyprinter - -executable ic-ref - import: version, logic, serialise, crypto, cbor, certificates, ghc-flags - main-is: ic-ref.hs - other-modules: IC.HTTP - other-modules: IC.HTTP.Status - other-modules: IC.HTTP.Request - other-modules: IC.Debug.JSON - build-depends: optparse-applicative - build-depends: cborg - build-depends: aeson - build-depends: warp - build-depends: wai-extra - build-depends: wai - build-depends: http-types - build-depends: unordered-containers - -executable ic-request-id - import: version, cbor, ghc-flags - main-is: ic-request-id.hs - build-depends: hex-text - build-depends: optparse-applicative - -executable ic-ref-test - import: version, crypto, cbor, certificates, ghc-flags - main-is: ic-ref-test.hs - other-modules: IC.Test.Spec - other-modules: IC.Test.Options - other-modules: IC.Test.Universal - other-modules: IC.Management - other-modules: IC.Types - build-depends: binary - build-depends: tasty - build-depends: tasty-hunit - build-depends: tasty-html - build-depends: tasty-ant-xml - build-depends: tasty-rerun - build-depends: optparse-applicative - build-depends: http-client - build-depends: http-client-tls - build-depends: http-types - build-depends: random - build-depends: filepath - build-depends: directory - build-depends: row-types - build-depends: candid - build-depends: hex-text - build-depends: crc - build-depends: transformers - build-depends: base32 - build-depends: time - build-depends: vector - -test-suite unit-test - import: logic, certificates, serialise, cbor, crypto, ghc-flags - type: exitcode-stdio-1.0 - main-is: unit-tests.hs - other-modules: IC.CBOR.Patterns - other-modules: IC.CBOR.Parser - other-modules: IC.HashTree.CBOR - other-modules: IC.HashTree - other-modules: IC.Test.HashTree - other-modules: IC.Test.BLS - other-modules: IC.Test.WebAuthn - other-modules: IC.Test.Secp256k1 - other-modules: IC.Test.ECDSA - build-depends: base >= 4 && < 5 - build-depends: tasty >= 0.7 - build-depends: tasty-hunit - build-depends: tasty-quickcheck - build-depends: QuickCheck - build-depends: quickcheck-io - build-depends: hex-text - build-depends: temporary - build-depends: directory diff --git a/impl/ic.did b/impl/ic.did deleted file mode 100644 index 5b64725d1..000000000 --- a/impl/ic.did +++ /dev/null @@ -1,54 +0,0 @@ -type canister_id = principal; -type user_id = principal; -type wasm_module = blob; - -type canister_settings = record { - controllers : opt vec principal; - compute_allocation : opt nat; - memory_allocation : opt nat; - freezing_threshold : opt nat; -}; - -type definite_canister_settings = record { - controllers : vec principal; - compute_allocation : nat; - memory_allocation : nat; - freezing_threshold : nat; -}; - -service ic : { - create_canister : (record { - settings : opt canister_settings - }) -> (record {canister_id : canister_id}); - update_settings : (record { - canister_id : principal; - settings : canister_settings - }) -> (); - install_code : (record { - mode : variant {install; reinstall; upgrade}; - canister_id : canister_id; - wasm_module : wasm_module; - arg : blob; - }) -> (); - uninstall_code : (record {canister_id : canister_id}) -> (); - start_canister : (record {canister_id : canister_id}) -> (); - stop_canister : (record {canister_id : canister_id}) -> (); - canister_status : (record {canister_id : canister_id}) -> (record { - status : variant { running; stopping; stopped }; - settings: definite_canister_settings; - module_hash: opt blob; - memory_size: nat; - cycles: nat; - }); - delete_canister : (record {canister_id : canister_id}) -> (); - deposit_cycles : (record {canister_id : canister_id}) -> (); - raw_rand : () -> (blob); - - // provisional interfaces for the pre-ledger world - provisional_create_canister_with_cycles : (record { - amount: opt nat; - settings : opt canister_settings - }) -> (record {canister_id : canister_id}); - provisional_top_up_canister : - (record { canister_id: canister_id; amount: nat }) -> (); -} diff --git a/impl/shell.nix b/impl/shell.nix deleted file mode 100644 index e215b9dfb..000000000 --- a/impl/shell.nix +++ /dev/null @@ -1,2 +0,0 @@ -{ system ? builtins.currentSystem }: -(import ../default.nix {inherit system;}).ic-ref-shell diff --git a/impl/src/IC/CBOR/Parser.hs b/impl/src/IC/CBOR/Parser.hs deleted file mode 100644 index 65d72424d..000000000 --- a/impl/src/IC/CBOR/Parser.hs +++ /dev/null @@ -1,45 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -module IC.CBOR.Parser where - -import Data.Text (Text) -import qualified Data.Text as T -import qualified Data.ByteString.Lazy as BS -import Data.Bifunctor -import Codec.CBOR.Term -import Codec.CBOR.Read - -import IC.CBOR.Patterns - -decodeWithoutTag :: BS.ByteString -> Either Text Term -decodeWithoutTag s = - first (\(DeserialiseFailure _ s) -> "CBOR decoding failure: " <> T.pack s) - (deserialiseFromBytes decodeTerm s) >>= begin - where - begin (leftOver, _) | not (BS.null leftOver) = Left "Left-over bytes" - begin (_, TTagged 55799 _) = Left "Did not expect semantic tag 55799 here" - begin (_, t) = return t - -decodeWithTag :: BS.ByteString -> Either Text Term -decodeWithTag s = - first (\(DeserialiseFailure _ s) -> "CBOR decoding failure: " <> T.pack s) - (deserialiseFromBytes decodeTerm s) >>= begin - where - begin (leftOver, _) | not (BS.null leftOver) = Left "Left-over bytes" - begin (_, TTagged 55799 t) = return t - begin (_, t) = Left $ "Expected certificate to begin with tag 55799, got " <> T.pack (show t) <> " in " <> T.pack (show s) - -parseMap :: Text -> Term -> Either Text [(Term, Term)] -parseMap _ (TMap_ kv) = return kv -parseMap what t = Left $ "expected " <> what <> ", found " <> T.pack (show t) - -parseBlob :: Text -> Term -> Either Text BS.ByteString -parseBlob _ (TBlob s) = return s -parseBlob what t = Left $ "expected " <> what <> ", found " <> T.pack (show t) - -parseField :: Text -> [(Term, a)] -> Either Text a -parseField f kv = case lookup (TString f) kv of - Just t -> return t - Nothing -> Left $ "Missing expected field " <> f - -optionalField :: Text -> [(Term, a)] -> Either Text (Maybe a) -optionalField f kv = return $ lookup (TString f) kv diff --git a/impl/src/IC/CBOR/Patterns.hs b/impl/src/IC/CBOR/Patterns.hs deleted file mode 100644 index 264e570a9..000000000 --- a/impl/src/IC/CBOR/Patterns.hs +++ /dev/null @@ -1,32 +0,0 @@ -{-# LANGUAGE RecordWildCards #-} -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE PatternSynonyms #-} -{-# LANGUAGE ViewPatterns #-} -{-# LANGUAGE LambdaCase #-} - -module IC.CBOR.Patterns where - -import qualified Data.ByteString.Lazy as BS -import Numeric.Natural -import Codec.CBOR.Term - -pattern TMap_ :: [(Term, Term)] -> Term -pattern TMap_ m <- (\case {TMapI m -> Just m; TMap m -> Just m; _ -> Nothing} -> Just m) - -pattern TList_ :: [Term] -> Term -pattern TList_ m <- (\case {TListI m -> Just m; TList m -> Just m; _ -> Nothing} -> Just m) - -pattern TNat :: Natural -> Term -pattern TNat m <- (\case - TInt m | m >= 0 -> Just (fromIntegral m) - TInteger m | m >= 0 -> Just (fromIntegral m) - _ -> Nothing - -> Just m) - -pattern TBlob :: BS.ByteString -> Term -pattern TBlob m <- TBytes (BS.fromStrict -> m) - where TBlob m = TBytes (BS.toStrict m) - - - - diff --git a/impl/src/IC/CBOR/Utils.hs b/impl/src/IC/CBOR/Utils.hs deleted file mode 100644 index d4ef96daa..000000000 --- a/impl/src/IC/CBOR/Utils.hs +++ /dev/null @@ -1,9 +0,0 @@ -module IC.CBOR.Utils where - -import qualified Data.ByteString.Builder as BS -import IC.HTTP.CBOR -import IC.HTTP.GenR -import IC.Types - -encodePrincipalList :: [EntityId] -> Blob -encodePrincipalList entities = BS.toLazyByteString $ encode $ GList $ map (GBlob . rawEntityId) entities diff --git a/impl/src/IC/Canister.hs b/impl/src/IC/Canister.hs deleted file mode 100644 index 08ffb5ccd..000000000 --- a/impl/src/IC/Canister.hs +++ /dev/null @@ -1,110 +0,0 @@ -{-# LANGUAGE TypeOperators #-} -{-# LANGUAGE RankNTypes #-} -{-# LANGUAGE LambdaCase #-} - -module IC.Canister - ( WasmState - , parseCanister - , CanisterModule(..) - , InitFunc, UpdateFunc, QueryFunc - , asUpdate - ) - where - -import qualified Data.Map as M -import Data.List - -import IC.Types -import IC.Wasm.Winter (parseModule, exportedFunctions, Module) - -import IC.Purify -import IC.Canister.Snapshot -import IC.Canister.Imp -import IC.Hash - --- Here we can swap out the purification machinery -type WasmState = CanisterSnapshot --- type WasmState = Replay ImpState - -type InitFunc = EntityId -> Env -> Blob -> TrapOr (WasmState, CanisterActions) -type UpdateFunc = WasmState -> TrapOr (WasmState, UpdateResult) -type QueryFunc = WasmState -> TrapOr Response - -data CanisterModule = CanisterModule - { raw_wasm :: Blob - , raw_wasm_hash :: Blob -- just caching, it’s worth it - , init_method :: InitFunc - , update_methods :: MethodName ↦ (EntityId -> Env -> Responded -> Cycles -> Blob -> UpdateFunc) - , query_methods :: MethodName ↦ (EntityId -> Env -> Blob -> QueryFunc) - , callbacks :: Callback -> Env -> Responded -> Cycles -> Response -> Cycles -> UpdateFunc - , cleanup :: WasmClosure -> Env -> WasmState -> TrapOr (WasmState, ()) - , pre_upgrade_method :: WasmState -> EntityId -> Env -> TrapOr (CanisterActions, Blob) - , post_upgrade_method :: EntityId -> Env -> Blob -> Blob -> TrapOr (WasmState, CanisterActions) - , inspect_message :: MethodName -> EntityId -> Env -> Blob -> WasmState -> TrapOr () - } - -instance Show CanisterModule where - show _ = "CanisterModule{...}" - -parseCanister :: Blob -> Either String CanisterModule -parseCanister bytes = - case parseModule bytes of - Left err -> Left err - Right wasm_mod -> Right $ CanisterModule - { raw_wasm = bytes - , raw_wasm_hash = sha256 bytes - , init_method = \caller env dat -> - case instantiate wasm_mod of - Trap err -> Trap err - Return wasm_state0 -> - invoke wasm_state0 (rawInitialize caller env dat) - , update_methods = M.fromList - [ (m, - \caller env responded cycles_available dat wasm_state -> - invoke wasm_state (rawUpdate m caller env responded cycles_available dat)) - | n <- exportedFunctions wasm_mod - , Just m <- return $ stripPrefix "canister_update " n - ] - , query_methods = M.fromList - [ (m, \caller env arg wasm_state -> - snd <$> invoke wasm_state (rawQuery m caller env arg)) - | n <- exportedFunctions wasm_mod - , Just m <- return $ stripPrefix "canister_query " n - ] - , callbacks = \cb env responded cycles_available res refund wasm_state -> - invoke wasm_state (rawCallback cb env responded cycles_available res refund) - , cleanup = \cb env wasm_state -> - invoke wasm_state (rawCleanup cb env) - , pre_upgrade_method = \wasm_state caller env -> - snd <$> invoke wasm_state (rawPreUpgrade caller env) - , post_upgrade_method = \caller env mem dat -> - case instantiate wasm_mod of - Trap err -> Trap err - Return wasm_state0 -> - invoke wasm_state0 (rawPostUpgrade caller env mem dat) - , inspect_message = \method_name caller env arg wasm_state -> - snd <$> invoke wasm_state (rawInspectMessage method_name caller env arg) - } - -instantiate :: Module -> TrapOr WasmState -instantiate wasm_mod = - either Trap Return $ snd $ createMaybe $ do - rawInstantiate wasm_mod >>= \case - Trap err -> return ((), Left err) - Return rs -> return ((), Right rs) - -invoke :: WasmState -> CanisterEntryPoint (TrapOr r) -> TrapOr (WasmState, r) -invoke s f = - case perform f s of - (_, Trap msg) -> Trap msg - (s', Return r) -> Return (s', r) - --- | Turns a query function into an update function -asUpdate :: - (EntityId -> Env -> Blob -> QueryFunc) -> - (EntityId -> Env -> Responded -> Cycles -> Blob -> UpdateFunc) -asUpdate f caller env (Responded responded) _cycles_available dat wasm_state - | responded = error "asUpdate: responded == True" - | otherwise = - (\res -> (wasm_state, (noCallActions { ca_response = Just res }, noCanisterActions))) <$> - f caller env dat wasm_state diff --git a/impl/src/IC/Canister/Imp.hs b/impl/src/IC/Canister/Imp.hs deleted file mode 100644 index cdf7898b2..000000000 --- a/impl/src/IC/Canister/Imp.hs +++ /dev/null @@ -1,722 +0,0 @@ -{-# LANGUAGE RankNTypes #-} -{-# LANGUAGE LambdaCase #-} -{-# LANGUAGE NamedFieldPuns #-} -{-# LANGUAGE ScopedTypeVariables #-} -{-# LANGUAGE GADTs #-} -{-# LANGUAGE FlexibleContexts #-} - -{-| -The canister interface, presented imperatively (or impurely), i.e. without rollback --} -module IC.Canister.Imp - ( CanisterEntryPoint - , ImpState(..) - , rawInstantiate - , rawInitialize - , rawQuery - , rawUpdate - , rawCallback - , rawCleanup - , rawPreUpgrade - , rawPostUpgrade - , rawInspectMessage - ) -where - -import qualified Data.Text as T -import qualified Data.Text.Encoding as T -import qualified Data.ByteString.Lazy as BS -import qualified Data.ByteString.Lazy.Char8 as BSC -import qualified Data.ByteString.Lazy.UTF8 as BSU -import Control.Monad.Primitive -import Control.Monad.ST -import Control.Monad.Except -import Data.STRef -import Data.Maybe -import Data.Int -- TODO: Should be Word32 in most cases -import Data.Word -import Data.Functor -import Numeric.Natural - -import IC.Types -import IC.Constants -import IC.Wasm.Winter -import IC.Wasm.WinterMemory as Mem -import IC.Wasm.Imports - --- Parameters are the data that come from the caller - -data Params = Params - { param_dat :: Maybe Blob - , param_caller :: Maybe EntityId - , reject_code :: Maybe Natural - , reject_message :: Maybe String - , cycles_refunded :: Maybe Cycles - } - --- The execution state is all information available to the --- canister. Some of it is immutable (could be separated here) - -data ExecutionState s = ExecutionState - { inst :: Instance s - , stableMem :: Memory s - , params :: Params - , method_name :: Maybe MethodName - , env :: Env - -- now the mutable parts - , cycles_available :: Maybe Cycles - , cycles_accepted :: Cycles - , balance :: Cycles - , responded :: Responded - , response :: Maybe Response - , reply_data :: Blob - , pending_call :: Maybe MethodCall - , calls :: [MethodCall] - , new_certified_data :: Maybe Blob - , accepted :: Bool -- for canister_inspect_message - } - - -initialExecutionState :: Instance s -> Memory s -> Env -> Responded -> ExecutionState s -initialExecutionState inst stableMem env responded = ExecutionState - { inst - , stableMem - , params = Params Nothing Nothing Nothing Nothing Nothing - , method_name = Nothing - , env - , cycles_available = Nothing - , balance = env_balance env - , cycles_accepted = 0 - , responded - , response = Nothing - , reply_data = mempty - , pending_call = Nothing - , calls = mempty - , new_certified_data = Nothing - , accepted = False - } - --- Some bookkeeping to access the ExecutionState --- --- We “always” have the 'STRef', but only within 'withES' is it actually --- present. - -type ESRef s = STRef s (Maybe (ExecutionState s)) - -newESRef :: ST s (ESRef s) -newESRef = newSTRef Nothing - --- | runs a computation with the given initial execution state --- and returns the final execution state with it. -withES :: PrimMonad m => - ESRef (PrimState m) -> - ExecutionState (PrimState m) -> - m a -> m (a, ExecutionState (PrimState m)) -withES esref es f = do - before <- stToPrim $ readSTRef esref - unless (isNothing before) $ error "withES with non-empty es" - stToPrim $ writeSTRef esref $ Just es - x <- f - es' <- stToPrim $ readSTRef esref - case es' of - Nothing -> error "withES: ExecutionState lost" - Just es' -> do - stToPrim $ writeSTRef esref Nothing - return (x, es') - -getsES :: ESRef s -> (ExecutionState s -> b) -> HostM s b -getsES esref f = lift (readSTRef esref) >>= \case - Nothing -> throwError "System API not available yet" - Just es -> return (f es) - -modES :: ESRef s -> (ExecutionState s -> ExecutionState s) -> HostM s () -modES esref f = lift $ modifySTRef esref (fmap f) - -appendReplyData :: ESRef s -> Blob -> HostM s () -appendReplyData esref dat = modES esref $ \es -> - es { reply_data = reply_data es <> dat } - -setResponse :: ESRef s -> Response -> HostM s () -setResponse esref r = modES esref $ \es -> - es { response = Just r } - -appendCall :: ESRef s -> MethodCall -> HostM s () -appendCall esref c = modES esref $ \es -> - es { calls = calls es ++ [c] } - -getAvailable :: ESRef s -> HostM s Cycles -getAvailable esref = - getsES esref cycles_available >>= - maybe (throwError "no cycles available") return - -getRefunded :: ESRef s -> HostM s Cycles -getRefunded esref = - getsES esref (cycles_refunded . params) >>= - maybe (throwError "no cycles refunded") return - -addBalance :: ESRef s -> Cycles -> HostM s () -addBalance esref f = modES esref $ \es -> - es { balance = min (balance es + f) cMAX_CANISTER_BALANCE } - -addAccepted :: ESRef s -> Cycles -> HostM s () -addAccepted esref f = modES esref $ \es -> - es { cycles_accepted = cycles_accepted es + f } - -subtractBalance :: ESRef s -> Cycles -> HostM s () -subtractBalance esref f = do - current_balance <- getsES esref balance - if f <= current_balance - then modES esref $ \es -> es { balance = current_balance - f } - else throwError "insufficient cycles to put on call" - -subtractAvailable :: ESRef s -> Cycles -> HostM s () -subtractAvailable esref f = do - current <- getAvailable esref - when (f > current) $ error "internal error: insufficient cycles to accept" - modES esref $ \es -> es { cycles_available = Just (current - f) } - --- The System API, with all imports - --- The code is defined in the where clause to scope over the 'ESRef' - -systemAPI :: forall s. ESRef s -> Imports s -systemAPI esref = - [ toImport "ic0" "msg_arg_data_size" msg_arg_data_size - , toImport "ic0" "msg_arg_data_copy" msg_arg_data_copy - , toImport "ic0" "msg_caller_size" msg_caller_size - , toImport "ic0" "msg_caller_copy" msg_caller_copy - , toImport "ic0" "msg_reject_code" msg_reject_code - , toImport "ic0" "msg_reject_msg_size" msg_reject_msg_size - , toImport "ic0" "msg_reject_msg_copy" msg_reject_msg_copy - - , toImport "ic0" "msg_reply_data_append" msg_reply_data_append - , toImport "ic0" "msg_reply" msg_reply - , toImport "ic0" "msg_reject" msg_reject - - , toImport "ic0" "canister_self_copy" canister_self_copy - , toImport "ic0" "canister_self_size" canister_self_size - , toImport "ic0" "canister_status" canister_status - - , toImport "ic0" "msg_cycles_available" msg_cycles_available - , toImport "ic0" "msg_cycles_refunded" msg_cycles_refunded - , toImport "ic0" "msg_cycles_accept" msg_cycles_accept - , toImport "ic0" "canister_cycle_balance" canister_cycle_balance - - , toImport "ic0" "call_new" call_new - , toImport "ic0" "call_on_cleanup" call_on_cleanup - , toImport "ic0" "call_data_append" call_data_append - , toImport "ic0" "call_cycles_add" call_cycles_add - , toImport "ic0" "call_perform" call_perform - - , toImport "ic0" "stable_size" stable_size - , toImport "ic0" "stable_grow" stable_grow - , toImport "ic0" "stable_write" stable_write - , toImport "ic0" "stable_read" stable_read - - , toImport "ic0" "certified_data_set" certified_data_set - , toImport "ic0" "data_certificate_present" data_certificate_present - , toImport "ic0" "data_certificate_size" data_certificate_size - , toImport "ic0" "data_certificate_copy" data_certificate_copy - - , toImport "ic0" "msg_method_name_size" msg_method_name_size - , toImport "ic0" "msg_method_name_copy" msg_method_name_copy - , toImport "ic0" "accept_message" accept_message - - , toImport "ic0" "time" get_time - - , toImport "ic0" "debug_print" debug_print - , toImport "ic0" "trap" explicit_trap - ] - where - -- Utilities - gets :: (ExecutionState s -> b) -> HostM s b - gets = getsES esref - - copy_to_canister :: Int32 -> Int32 -> Int32 -> Blob -> HostM s () - copy_to_canister dst offset size blob = do - unless (offset == 0) $ - throwError "offset /= 0 not supported" - unless (size == fromIntegral (BS.length blob)) $ - throwError "copying less than the full blob is not supported" - i <- getsES esref inst - -- TODO Bounds checking - setBytes i (fromIntegral dst) blob - - copy_from_canister :: String -> Int32 -> Int32 -> HostM s Blob - copy_from_canister _name src size = do - i <- gets inst - getBytes i (fromIntegral src) size - - size_and_copy :: HostM s Blob -> - ( () -> HostM s Int32 - , (Int32, Int32, Int32) -> HostM s () - ) - size_and_copy get_blob = - ( \() -> - get_blob >>= \blob -> return $ fromIntegral (BS.length blob) - , \(dst, offset, size) -> - get_blob >>= \blob -> copy_to_canister dst offset size blob - ) - - -- Unsafely print - putBytes :: BS.ByteString -> HostM s () - putBytes bytes = - unsafeIOToPrim $ BSC.putStrLn $ BSC.pack "debug.print: " <> bytes - - -- The system calls (in the order of the public spec) - -- https://docs.dfinity.systems/spec/public/#_system_imports - - msg_arg_data_size :: () -> HostM s Int32 - msg_arg_data_copy :: (Int32, Int32, Int32) -> HostM s () - (msg_arg_data_size, msg_arg_data_copy) = size_and_copy $ - gets (param_dat . params) >>= maybe (throwError "No argument") return - - msg_caller_size :: () -> HostM s Int32 - msg_caller_copy :: (Int32, Int32, Int32) -> HostM s () - (msg_caller_size, msg_caller_copy) = size_and_copy $ - gets (param_caller . params) - >>= maybe (throwError "No argument") (return . rawEntityId) - - msg_reject_code :: () -> HostM s Int32 - msg_reject_code () = - gets (reject_code . params) - >>= maybe (throwError "No reject code") (return . fromIntegral) - - msg_reject_msg_size :: () -> HostM s Int32 - msg_reject_msg_copy :: (Int32, Int32, Int32) -> HostM s () - (msg_reject_msg_size, msg_reject_msg_copy) = size_and_copy $ do - gets (reject_message . params) - >>= maybe (throwError "No reject code") (return . BSU.fromString) - - assert_not_responded :: HostM s () - assert_not_responded = do - gets responded >>= \case - Responded False -> return () - Responded True -> throwError "This call has already been responded to earlier" - gets response >>= \case - Nothing -> return () - Just _ -> throwError "This call has already been responded to in this function" - - msg_reply_data_append :: (Int32, Int32) -> HostM s () - msg_reply_data_append (src, size) = do - assert_not_responded - bytes <- copy_from_canister "msg_reply_data_append" src size - appendReplyData esref bytes - - msg_reply :: () -> HostM s () - msg_reply () = do - assert_not_responded - bytes <- gets reply_data - setResponse esref (Reply bytes) - - msg_reject :: (Int32, Int32) -> HostM s () - msg_reject (src, size) = do - assert_not_responded - bytes <- copy_from_canister "msg_reject" src size - let msg = BSU.toString bytes - setResponse esref $ Reject (RC_CANISTER_REJECT, msg) - - canister_self_size :: () -> HostM s Int32 - canister_self_copy :: (Int32, Int32, Int32) -> HostM s () - (canister_self_size, canister_self_copy) = size_and_copy $ - rawEntityId <$> gets (env_self . env) - - canister_status :: () -> HostM s Int32 - canister_status () = gets (env_status . env) <&> \case - Running -> 1 - Stopping -> 2 - Stopped -> 3 - - msg_cycles_refunded :: () -> HostM s Word64 - msg_cycles_refunded () = fromIntegral <$> getRefunded esref - - msg_cycles_available :: () -> HostM s Word64 - msg_cycles_available () = fromIntegral <$> getAvailable esref - - msg_cycles_accept :: Word64 -> HostM s Word64 - msg_cycles_accept max_amount = do - available <- fromIntegral <$> getAvailable esref - balance <- gets balance - let amount = minimum - [ fromIntegral max_amount - , available - , cMAX_CANISTER_BALANCE - balance] - subtractAvailable esref amount - addBalance esref amount - addAccepted esref amount - return (fromIntegral amount) - - canister_cycle_balance :: () -> HostM s Word64 - canister_cycle_balance () = fromIntegral <$> gets balance - - call_new :: ( Int32, Int32, Int32, Int32, Int32, Int32, Int32, Int32 ) -> HostM s () - call_new ( callee_src, callee_size, name_src, name_size - , reply_fun, reply_env, reject_fun, reject_env ) = do - discard_pending_call - callee <- copy_from_canister "call_simple" callee_src callee_size - method_name <- copy_from_canister "call_simple" name_src name_size - let reply_closure = WasmClosure reply_fun reply_env - let reject_closure = WasmClosure reject_fun reject_env - setPendingCall $ MethodCall - { call_callee = EntityId callee - , call_method_name = BSU.toString method_name -- TODO: check for valid UTF8 - , call_arg = mempty - , call_callback = Callback reply_closure reject_closure Nothing - , call_transferred_cycles = 0 - } - - call_on_cleanup :: (Int32, Int32) -> HostM s () - call_on_cleanup (fun, env) = do - let cleanup_closure = WasmClosure fun env - changePendingCall $ \pc -> do - let callback = call_callback pc - when (isJust (cleanup_callback callback)) $ - throwError "call_on_cleanup invoked twice" - return $ pc { call_callback = callback { cleanup_callback = Just cleanup_closure } } - - call_data_append :: (Int32, Int32) -> HostM s () - call_data_append (src, size) = do - arg <- copy_from_canister "call_data_append" src size - changePendingCall $ \pc -> return $ pc { call_arg = call_arg pc <> arg } - - call_cycles_add :: Word64 -> HostM s () - call_cycles_add amount = do - let cycles = fromIntegral amount - changePendingCall $ \pc -> do - subtractBalance esref cycles - return $ pc { call_transferred_cycles = call_transferred_cycles pc + cycles } - - call_perform :: () -> HostM s Int32 - call_perform () = do - pc <- getPendingCall - - appendCall esref pc - modES esref $ \es -> es { pending_call = Nothing } - return 0 - - -- utilities for the pending call - - setPendingCall :: MethodCall -> HostM s () - setPendingCall pc = - modES esref $ \es -> es { pending_call = Just pc } - - getPendingCall :: HostM s MethodCall - getPendingCall = - gets pending_call >>= \case - Nothing -> throwError "No call in process" - Just pc -> return pc - - changePendingCall :: (MethodCall -> HostM s MethodCall) -> HostM s () - changePendingCall act = - getPendingCall >>= act >>= setPendingCall - - discard_pending_call = do - mpc <- gets pending_call - forM_ mpc $ \pc -> addBalance esref (call_transferred_cycles pc) - modES esref $ \es -> es { pending_call = Nothing } - - stable_size :: () -> HostM s Int32 - stable_size () = do - m <- gets stableMem - Mem.size m - - stable_grow :: Int32 -> HostM s Int32 - stable_grow delta = do - m <- gets stableMem - Mem.grow m delta - - stable_write :: (Int32, Int32, Int32) -> HostM s () - stable_write (dst, src, size) = do - m <- gets stableMem - i <- getsES esref inst - blob <- getBytes i (fromIntegral src) size - Mem.write m (fromIntegral dst) blob - - stable_read :: (Int32, Int32, Int32) -> HostM s () - stable_read (dst, src, size) = do - m <- gets stableMem - i <- getsES esref inst - blob <- Mem.read m (fromIntegral src) size - setBytes i (fromIntegral dst) blob - - certified_data_set :: (Int32, Int32) -> HostM s () - certified_data_set (src, size) = do - when (size > 32) $ throwError "certified_data_set: too large" - blob <- copy_from_canister "certified_data_set" src size - modES esref $ \es -> es { new_certified_data = Just blob } - - data_certificate_present :: () -> HostM s Int32 - data_certificate_present () = - gets (env_certificate . env) >>= \case - Just _ -> return 1 - Nothing -> return 0 - - data_certificate_size :: () -> HostM s Int32 - data_certificate_copy :: (Int32, Int32, Int32) -> HostM s () - (data_certificate_size, data_certificate_copy) = size_and_copy $ - gets (env_certificate . env) >>= \case - Just b -> return b - Nothing -> throwError "no certificate available" - - msg_method_name_size :: () -> HostM s Int32 - msg_method_name_copy :: (Int32, Int32, Int32) -> HostM s () - (msg_method_name_size, msg_method_name_copy) = size_and_copy $ - gets method_name >>= - maybe (throwError "Cannot query method name here") - (return . BS.fromStrict . T.encodeUtf8 . T.pack) - - accept_message :: () -> HostM s () - accept_message () = do - a <- gets accepted - when a $ throwError "Message already accepted" - modES esref $ \es -> es { accepted = True } - - get_time :: () -> HostM s Word64 - get_time () = do - Timestamp ns <- gets (env_time . env) - return (fromIntegral ns) - - debug_print :: (Int32, Int32) -> HostM s () - debug_print (src, size) = do - -- TODO: This should be a non-trapping copy - bytes <- copy_from_canister "debug_print" src size - putBytes bytes - - explicit_trap :: (Int32, Int32) -> HostM s () - explicit_trap (src, size) = do - -- TODO: This should be a non-trapping copy - bytes <- copy_from_canister "trap" src size - let msg = BSU.toString bytes - throwError $ "canister trapped explicitly: " ++ msg - --- The state of an instance, consistig of --- * the underlying Wasm state, --- * additional remembered information like the CanisterId --- * the 'ESRef' that the system api functions are accessing --- * the original module (so that this ImpState can be snapshotted) - -data ImpState s = ImpState - { isESRef :: ESRef s - , isInstance :: Instance s - , isStableMem :: Memory s - , isModule :: Module - } - -rawInstantiate :: Module -> ST s (TrapOr (ImpState s)) -rawInstantiate wasm_mod = do - esref <- newESRef - result <- runExceptT $ (,) - <$> initialize wasm_mod (systemAPI esref) - <*> Mem.new - case result of - Left err -> return $ Trap err - Right (inst, sm) -> return $ Return $ ImpState esref inst sm wasm_mod - -cantRespond :: Responded -cantRespond = Responded True - -canRespond :: Responded -canRespond = Responded False - -canisterActions :: ExecutionState s -> CanisterActions -canisterActions es = CanisterActions - { set_certified_data = new_certified_data es - } - -type CanisterEntryPoint r = forall s. (ImpState s -> ST s r) - -rawInitialize :: EntityId -> Env -> Blob -> ImpState s -> ST s (TrapOr CanisterActions) -rawInitialize caller env dat (ImpState esref inst sm wasm_mod) = do - result <- runExceptT $ do - let es = (initialExecutionState inst sm env cantRespond) - { params = Params - { param_dat = Just dat - , param_caller = Just caller - , reject_code = Nothing - , reject_message = Nothing - , cycles_refunded = Nothing - } - } - - -- invoke canister_init - if "canister_init" `elem` exportedFunctions wasm_mod - then withES esref es $ void $ invokeExport inst "canister_init" [] - else return ((), es) - - case result of - Left err -> return $ Trap err - Right (_, es') - | accepted es' -> return $ Trap "cannot accept_message here" - | not (null (calls es')) -> return $ Trap "cannot call from init" - | otherwise -> return $ Return $ canisterActions es' - -rawPreUpgrade :: EntityId -> Env -> ImpState s -> ST s (TrapOr (CanisterActions, Blob)) -rawPreUpgrade caller env (ImpState esref inst sm wasm_mod) = do - result <- runExceptT $ do - let es = (initialExecutionState inst sm env cantRespond) - { params = Params - { param_dat = Nothing - , param_caller = Just caller - , reject_code = Nothing - , reject_message = Nothing - , cycles_refunded = Nothing - } - } - - if "canister_pre_upgrade" `elem` exportedFunctions wasm_mod - then withES esref es $ void $ invokeExport inst "canister_pre_upgrade" [] - else return ((), es) - - case result of - Left err -> return $ Trap err - Right (_, es') - | accepted es' -> return $ Trap "cannot accept_message here" - | not (null (calls es')) -> return $ Trap "cannot call from pre_upgrade" - | otherwise -> do - stable_mem <- Mem.export (stableMem es') - return $ Return (canisterActions es', stable_mem) - -rawPostUpgrade :: EntityId -> Env -> Blob -> Blob -> ImpState s -> ST s (TrapOr CanisterActions) -rawPostUpgrade caller env mem dat (ImpState esref inst sm wasm_mod) = do - result <- runExceptT $ do - let es = (initialExecutionState inst sm env cantRespond) - { params = Params - { param_dat = Just dat - , param_caller = Just caller - , reject_code = Nothing - , reject_message = Nothing - , cycles_refunded = Nothing - } - } - lift $ Mem.imp (stableMem es) mem - - if "canister_post_upgrade" `elem` exportedFunctions wasm_mod - then withES esref es $ void $ invokeExport inst "canister_post_upgrade" [] - else return ((), es) - - case result of - Left err -> return $ Trap err - Right (_, es') - | accepted es' -> return $ Trap "cannot accept_message here" - | not (null (calls es')) -> return $ Trap "cannot call from post_upgrade" - | otherwise -> return $ Return (canisterActions es') - -rawQuery :: MethodName -> EntityId -> Env -> Blob -> ImpState s -> ST s (TrapOr Response) -rawQuery method caller env dat (ImpState esref inst sm _) = do - let es = (initialExecutionState inst sm env canRespond) - { params = Params - { param_dat = Just dat - , param_caller = Just caller - , reject_code = Nothing - , reject_message = Nothing - , cycles_refunded = Nothing - } - } - result <- runExceptT $ withES esref es $ - invokeExport inst ("canister_query " ++ method) [] - - case result of - Left err -> return $ Trap err - Right (_, es') - | Just _ <- new_certified_data es' - -> return $ Trap "Cannot set certified data from a query method" - | not (null (calls es')) -> return $ Trap "cannot call from query" - | accepted es' -> return $ Trap "cannot accept_message here" - | Just r <- response es' -> return $ Return r - | otherwise -> return $ Trap "No response" - -rawUpdate :: MethodName -> EntityId -> Env -> Responded -> Cycles -> Blob -> ImpState s -> ST s (TrapOr UpdateResult) -rawUpdate method caller env responded cycles_available dat (ImpState esref inst sm _) = do - let es = (initialExecutionState inst sm env responded) - { params = Params - { param_dat = Just dat - , param_caller = Just caller - , reject_code = Nothing - , reject_message = Nothing - , cycles_refunded = Nothing - } - , cycles_available = Just cycles_available - } - - result <- runExceptT $ withES esref es $ - invokeExport inst ("canister_update " ++ method) [] - case result of - Left err -> return $ Trap err - Right (_, es') - | accepted es' -> return $ Trap "cannot accept_message here" - | otherwise -> return $ Return - ( CallActions (calls es') (cycles_accepted es') (response es') - , canisterActions es' - ) - -rawCallback :: Callback -> Env -> Responded -> Cycles -> Response -> Cycles -> ImpState s -> ST s (TrapOr UpdateResult) -rawCallback callback env responded cycles_available res refund (ImpState esref inst sm _) = do - let params = case res of - Reply dat -> - Params { param_dat = Just dat, param_caller = Nothing, reject_code = Just 0, reject_message = Nothing, cycles_refunded = Just refund } - Reject (rc, reject_message) -> - Params { param_dat = Nothing, param_caller = Nothing, reject_code = Just (rejectCode rc), reject_message = Just reject_message, cycles_refunded = Just refund } - let es = (initialExecutionState inst sm env responded) - { params - , cycles_available = Just cycles_available - } - - let WasmClosure fun_idx env = case res of - Reply {} -> reply_callback callback - Reject {} -> reject_callback callback - - result <- runExceptT $ withES esref es $ - invokeTable inst fun_idx [I32 env] - case result of - Left err -> return $ Trap err - Right (_, es') - | accepted es' -> return $ Trap "cannot accept_message here" - | otherwise -> return $ Return - ( CallActions (calls es') (cycles_accepted es') (response es') - , canisterActions es' - ) - --- Needs to be separate from rawCallback, as it is its own transaction -rawCleanup :: WasmClosure -> Env -> ImpState s -> ST s (TrapOr ()) -rawCleanup (WasmClosure fun_idx cb_env) env (ImpState esref inst sm _) = do - let es = initialExecutionState inst sm env cantRespond - - result <- runExceptT $ withES esref es $ - invokeTable inst fun_idx [I32 cb_env] - case result of - Left err -> return $ Trap err - Right (_, es') - | Just _ <- new_certified_data es' - -> return $ Trap "Cannot set certified data from inspect_message" - | not (null (calls es')) -> return $ Trap "cannot call from inspect_message" - | isJust (response es') -> return $ Trap "cannot respond from inspect_message" - | accepted es' -> return $ Trap "cannot accept_message here" - | otherwise -> return $ Return () - -rawInspectMessage :: MethodName -> EntityId -> Env -> Blob -> ImpState s -> ST s (TrapOr ()) -rawInspectMessage method caller env dat (ImpState esref inst sm wasm_mod) = do - result <- runExceptT $ do - let es = (initialExecutionState inst sm env cantRespond) - { params = Params - { param_dat = Just dat - , param_caller = Just caller - , reject_code = Nothing - , reject_message = Nothing - , cycles_refunded = Nothing - } - , method_name = Just method - } - - if "canister_inspect_message" `elem` exportedFunctions wasm_mod - then withES esref es $ void $ invokeExport inst "canister_inspect_message" [] - else return ((), es { accepted = True } ) - - case result of - Left err -> return $ Trap err - Right (_, es') - | Just _ <- new_certified_data es' - -> return $ Trap "Cannot set certified data from inspect_message" - | not (null (calls es')) -> return $ Trap "cannot call from inspect_message" - | isJust (response es') -> return $ Trap "cannot respond from inspect_message" - | not (accepted es') -> return $ Trap "message not accepted by inspect_message" - | otherwise -> return $ Return () diff --git a/impl/src/IC/Canister/Snapshot.hs b/impl/src/IC/Canister/Snapshot.hs deleted file mode 100644 index 7fc9b1900..000000000 --- a/impl/src/IC/Canister/Snapshot.hs +++ /dev/null @@ -1,38 +0,0 @@ -{-# LANGUAGE TypeFamilies #-} -{-# LANGUAGE StandaloneDeriving #-} -{-# LANGUAGE DerivingVia #-} -{-# LANGUAGE MultiParamTypeClasses #-} - -{-# OPTIONS_GHC -Wno-orphans #-} -module IC.Canister.Snapshot ( CanisterSnapshot(..) ) where - -import Data.ByteString.Lazy (ByteString) - -import IC.Types -import IC.Wasm.Winter (Module) -import IC.Wasm.Winter.Persist -import IC.Canister.Imp -import IC.Purify - -data CanisterSnapshot = CanisterSnapshot - { wsModule :: Module - , wsInstances :: PInstance - , wsStableMem :: ByteString - } deriving Show - -instance SnapshotAble ImpState where - type SnapshotOf ImpState = CanisterSnapshot - persist (ImpState _ inst sm mod) = do - CanisterSnapshot mod <$> persistInstance inst <*> persistMemory sm - recreate (CanisterSnapshot wasm_mod pinst pmem) = do - rs <- rawInstantiate wasm_mod >>= trapToFail - resumeInstance (isInstance rs) pinst - resumeMemory (isStableMem rs) pmem - return rs - where - trapToFail (Trap err) = fail $ "replay failed: " ++ show err - trapToFail (Return x) = return x - -deriving - via (Snapshot CanisterSnapshot) - instance Purify ImpState CanisterSnapshot diff --git a/impl/src/IC/Certificate.hs b/impl/src/IC/Certificate.hs deleted file mode 100644 index 760cfb8ab..000000000 --- a/impl/src/IC/Certificate.hs +++ /dev/null @@ -1,17 +0,0 @@ -module IC.Certificate where - -import IC.HashTree - -data Certificate = Certificate - { cert_tree :: HashTree - , cert_sig :: Blob - , cert_delegation :: Maybe Delegation - } - deriving (Show) - -data Delegation = Delegation - { del_subnet_id :: Blob - , del_certificate :: Blob - } - deriving (Show) - diff --git a/impl/src/IC/Certificate/CBOR.hs b/impl/src/IC/Certificate/CBOR.hs deleted file mode 100644 index f86b413ab..000000000 --- a/impl/src/IC/Certificate/CBOR.hs +++ /dev/null @@ -1,44 +0,0 @@ -{-# LANGUAGE RecordWildCards #-} -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE PatternSynonyms #-} -{-# LANGUAGE ViewPatterns #-} -{-# LANGUAGE LambdaCase #-} - -module IC.Certificate.CBOR (encodeCert, decodeCert) where - -import qualified Data.Text as T -import Codec.CBOR.Term -import Codec.CBOR.Write - -import IC.Certificate -import IC.CBOR.Parser -import IC.CBOR.Patterns -import IC.HashTree -import IC.HashTree.CBOR - -encodeCert :: Certificate -> Blob -encodeCert Certificate{..} = toLazyByteString $ encodeTerm $ TTagged 55799 $ TMap $ - [ (TString "tree", encodeHashTree cert_tree) - , (TString "signature", TBlob cert_sig) - ] ++ - [ (TString "delegation", TMap - [ (TString "subnet_id", TBlob del_subnet_id) - , (TString "certificate", TBlob del_certificate) - ]) - | Just Delegation{..} <- pure cert_delegation - ] - -decodeCert :: Blob -> Either T.Text Certificate -decodeCert s = do - kv <- decodeWithTag s >>= parseMap "certificate" - cert_tree <- parseField "tree" kv >>= parseHashTree - cert_sig <- parseField "signature" kv >>= parseBlob "signature" - cert_delegation <- optionalField "delegation" kv >>= mapM parseDelegation - return $ Certificate{..} - -parseDelegation :: Term -> Either T.Text Delegation -parseDelegation t = do - kv <- parseMap "delegation" t - del_subnet_id <- parseField "subnet_id" kv >>= parseBlob "subnet_id" - del_certificate <- parseField "certificate" kv >>= parseBlob "certificate" - return $ Delegation{..} diff --git a/impl/src/IC/Certificate/Validate.hs b/impl/src/IC/Certificate/Validate.hs deleted file mode 100644 index 4384e4c57..000000000 --- a/impl/src/IC/Certificate/Validate.hs +++ /dev/null @@ -1,52 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -module IC.Certificate.Validate (validateCertificate) where - -import qualified Data.Text as T -import qualified Data.ByteString.Lazy as BS -import qualified Text.Hex as H -import Control.Monad.Error.Class - -import IC.Crypto.DER_BLS -import IC.Types -import IC.Certificate -import IC.Certificate.CBOR -import IC.HashTree hiding (Blob) - -validateCertificate :: Blob -> Certificate -> Either T.Text () -validateCertificate = validate' "certificate" - -validate' :: T.Text -> Blob -> Certificate -> Either T.Text () -validate' what root_key cert = do - pk <- validateDelegation root_key (cert_delegation cert) - verboseVerify what "ic-state-root" pk (reconstruct (cert_tree cert)) (cert_sig cert) - -validateDelegation :: Blob -> Maybe Delegation -> Either T.Text Blob -validateDelegation root_key Nothing = return root_key -validateDelegation root_key (Just del) = do - cert <- decodeCert (del_certificate del) - case wellFormed (cert_tree cert) of - Left err -> throwError $ "Hash tree not well formed: " <> T.pack err - Right () -> return () - validate' "certificate delegation" root_key cert - - case lookupPath (cert_tree cert) ["subnet", del_subnet_id del, "public_key"] of - Found b -> return b - x -> throwError $ "Expected to find subnet public key in certificate, " <> - "but got " <> T.pack (show x) - -verboseVerify :: T.Text -> Blob -> Blob -> Blob -> Blob -> Either T.Text () -verboseVerify what domain_sep pk msg sig = - case verify domain_sep pk msg sig of - Left err -> throwError $ T.unlines - [ "Signature verification failed on " <> what - , err - , "Domain separator: " <> T.pack (prettyBlob domain_sep) - , "Public key (DER): " <> T.pack (asHex pk) - , "Signature: " <> T.pack (asHex sig) - , "Checked message: " <> T.pack (prettyBlob msg) - ] - Right () -> return () - -asHex :: Blob -> String -asHex = T.unpack . H.encodeHex . BS.toStrict - diff --git a/impl/src/IC/Certificate/Value.hs b/impl/src/IC/Certificate/Value.hs deleted file mode 100644 index 124f56d4a..000000000 --- a/impl/src/IC/Certificate/Value.hs +++ /dev/null @@ -1,31 +0,0 @@ --- Encoding and decoding of state tree values - -{-# LANGUAGE TypeSynonymInstances #-} -module IC.Certificate.Value (CertVal(..)) where - -import qualified Data.Text as T -import qualified Data.Text.Encoding as T -import qualified Data.ByteString.Lazy as BS -import Data.Serialize.LEB128 -import Numeric.Natural - -import IC.Types - -class CertVal a where - toCertVal :: a -> Blob - fromCertVal :: Blob -> Maybe a - -instance CertVal Blob where - toCertVal = id - fromCertVal = Just - -instance CertVal T.Text where - toCertVal = BS.fromStrict . T.encodeUtf8 - fromCertVal = forgetLeft . T.decodeUtf8' . BS.toStrict - -instance CertVal Natural where - toCertVal = BS.fromStrict . toLEB128 - fromCertVal = forgetLeft . fromLEB128 . BS.toStrict - -forgetLeft :: Either a b -> Maybe b -forgetLeft = either (const Nothing) Just diff --git a/impl/src/IC/Constants.hs b/impl/src/IC/Constants.hs deleted file mode 100644 index 5c7748111..000000000 --- a/impl/src/IC/Constants.hs +++ /dev/null @@ -1,6 +0,0 @@ -module IC.Constants where - -import Numeric.Natural - -cMAX_CANISTER_BALANCE :: Natural -cMAX_CANISTER_BALANCE = 2^(60::Int) diff --git a/impl/src/IC/Crypto.hs b/impl/src/IC/Crypto.hs deleted file mode 100644 index 0f6a028e7..000000000 --- a/impl/src/IC/Crypto.hs +++ /dev/null @@ -1,123 +0,0 @@ -{- | -Everything related to signature creation and checking --} -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE LambdaCase #-} -module IC.Crypto - ( SecretKey(..) - , createSecretKeyEd25519 - , createSecretKeyWebAuthn - , createSecretKeyECDSA - , createSecretKeySecp256k1 - , createSecretKeyBLS - , toPublicKey - , signPure - , sign - , verify - ) - where - -import qualified Data.Text as T -import qualified Data.ByteString.Lazy as BS -import qualified IC.Crypto.Ed25519 as Ed25519 -import qualified IC.Crypto.DER as DER -import qualified IC.Crypto.WebAuthn as WebAuthn -import qualified IC.Crypto.ECDSA as ECDSA -import qualified IC.Crypto.Secp256k1 as Secp256k1 -import qualified IC.Crypto.BLS as BLS -import qualified IC.Crypto.CanisterSig as CanisterSig -import Data.Int -import Control.Monad.Except - -data SecretKey - = Ed25519 Ed25519.SecretKey - | ECDSA ECDSA.SecretKey - | Secp256k1 Secp256k1.SecretKey - | WebAuthn WebAuthn.SecretKey - | BLS BLS.SecretKey - deriving (Show) - -createSecretKeyEd25519 :: BS.ByteString -> SecretKey -createSecretKeyEd25519 = Ed25519 . Ed25519.createKey - -createSecretKeyWebAuthn :: BS.ByteString -> SecretKey -createSecretKeyWebAuthn = WebAuthn . WebAuthn.createKey - -createSecretKeyECDSA :: BS.ByteString -> SecretKey -createSecretKeyECDSA = ECDSA . ECDSA.createKey - -createSecretKeySecp256k1 :: BS.ByteString -> SecretKey -createSecretKeySecp256k1 = Secp256k1 . Secp256k1.createKey - -createSecretKeyBLS :: BS.ByteString -> SecretKey -createSecretKeyBLS = BLS . BLS.createKey - -toPublicKey :: SecretKey -> BS.ByteString -toPublicKey (Ed25519 sk) = DER.encode DER.Ed25519 $ Ed25519.toPublicKey sk -toPublicKey (WebAuthn sk) = DER.encode DER.WebAuthn $ WebAuthn.toPublicKey sk -toPublicKey (ECDSA sk) = DER.encode DER.ECDSA $ ECDSA.toPublicKey sk -toPublicKey (Secp256k1 sk) = DER.encode DER.Secp256k1 $ Secp256k1.toPublicKey sk -toPublicKey (BLS sk) = DER.encode DER.BLS $ BLS.toPublicKey sk - -signPure :: BS.ByteString -> SecretKey -> BS.ByteString -> BS.ByteString -signPure domain_sep sk payload = case sk of - Ed25519 sk -> Ed25519.sign sk msg - WebAuthn _ -> error "WebAuthn not a pure signature" - ECDSA _ -> error "ECDSA not a pure signature" - Secp256k1 _ -> error "Secp256k1 is not a pure signature" - BLS sk -> BLS.sign sk msg - where - msg | BS.null domain_sep = payload - | otherwise = BS.singleton (fromIntegral (BS.length domain_sep)) <> domain_sep <> payload - -sign :: BS.ByteString -> SecretKey -> BS.ByteString -> IO BS.ByteString -sign domain_sep sk payload = case sk of - Ed25519 sk -> return $ Ed25519.sign sk msg - WebAuthn sk -> WebAuthn.sign sk msg - ECDSA sk -> ECDSA.sign sk msg - Secp256k1 sk -> Secp256k1.sign sk msg - BLS sk -> return $ BLS.sign sk msg - where - msg | BS.null domain_sep = payload - | otherwise = BS.singleton (fromIntegral (BS.length domain_sep)) <> domain_sep <> payload - -verify :: BS.ByteString -> BS.ByteString -> BS.ByteString -> BS.ByteString -> BS.ByteString -> Either T.Text () -verify root_key domain_sep der_pk payload sig = DER.decode der_pk >>= \case - (DER.WebAuthn, pk) -> WebAuthn.verify pk msg sig - - (DER.Ed25519, pk) -> do - assertLen "Ed25519 public key" 32 pk - assertLen "Ed25519 signature" 64 sig - - unless (Ed25519.verify pk msg sig) $ do - when (Ed25519.verify pk payload sig) $ - throwError $ "domain separator " <> T.pack (show domain_sep) <> " missing" - throwError "signature verification failed" - - (DER.ECDSA, pk) -> do - unless (ECDSA.verify pk msg sig) $ do - when (ECDSA.verify pk payload sig) $ - throwError $ "domain separator " <> T.pack (show domain_sep) <> " missing" - throwError "signature verification failed" - - (DER.Secp256k1, pk) -> Secp256k1.verify pk msg sig - - (DER.BLS, pk) -> do - assertLen "BLS public key" 96 pk - assertLen "BLS signature" 48 sig - - unless (BLS.verify pk msg sig) $ do - when (BLS.verify pk payload sig) $ - throwError $ "domain separator " <> T.pack (show domain_sep) <> " missing" - throwError "signature verification failed" - - (DER.CanisterSig, pk) -> CanisterSig.verify root_key pk msg sig - - where - msg = BS.singleton (fromIntegral (BS.length domain_sep)) <> domain_sep <> payload - - -assertLen :: T.Text -> Int64 -> BS.ByteString -> Either T.Text () -assertLen what len bs - | BS.length bs == len = return () - | otherwise = throwError $ what <> " has wrong length " <> T.pack (show (BS.length bs)) <> ", expected " <> T.pack (show len) diff --git a/impl/src/IC/Crypto/BLS.hsc b/impl/src/IC/Crypto/BLS.hsc deleted file mode 100644 index d8e0a4c46..000000000 --- a/impl/src/IC/Crypto/BLS.hsc +++ /dev/null @@ -1,98 +0,0 @@ -{-# OPTIONS_GHC -fno-warn-unused-imports -Wno-unused-top-binds #-} -{-# LANGUAGE DeriveGeneric #-} -#include -#include -module IC.Crypto.BLS - ( init - , SecretKey - , createKey - , toPublicKey - , sign - , verify - ) where - -import Prelude hiding (init) -import qualified Data.ByteString.Lazy as BS -import qualified Data.ByteString as BSS -import Control.Monad -import Foreign.Ptr -import Foreign.Marshal.Alloc -import System.IO.Unsafe -import GHC.Generics - -#strict_import - - -{- typedef struct { - int len; int max; char * val; - } octet; -} -#starttype octet -#field len , CInt -#field max , CInt -#field val , CString -#stoptype - - -#ccall BLS_BLS12381_INIT , IO CInt -#ccall BLS_BLS12381_KEY_PAIR_GENERATE , Ptr -> Ptr -> Ptr -> IO CInt -#ccall BLS_BLS12381_CORE_SIGN , Ptr -> Ptr -> Ptr -> IO CInt -#ccall BLS_BLS12381_CORE_VERIFY , Ptr -> Ptr -> Ptr -> IO CInt - -init :: IO () -init = do - r <- c'BLS_BLS12381_INIT - unless (r == 0) $ fail "Could not initialize BLS" - - --- Cache the public key as well -data SecretKey = SecretKey BS.ByteString BS.ByteString - deriving (Show, Generic) - -toPublicKey :: SecretKey -> BS.ByteString -toPublicKey (SecretKey _ pk) = pk - -useAsOctet :: BS.ByteString -> (Ptr C'octet -> IO a) -> IO a -useAsOctet bs a = - BSS.useAsCStringLen (BS.toStrict bs) $ \(cstr, len) -> - alloca $ \oct_ptr -> do - poke oct_ptr (C'octet (fromIntegral len) (fromIntegral len) cstr) - a oct_ptr - -allocOctet :: Int -> (Ptr C'octet -> IO a) -> IO a -allocOctet size a = - allocaBytes size $ \cstr -> - alloca $ \oct_ptr -> do - poke oct_ptr (C'octet 0 (fromIntegral size) cstr) - a oct_ptr - -packOctet :: Ptr C'octet -> IO BS.ByteString -packOctet oct_ptr = do - C'octet len _ cstr' <- peek oct_ptr - bs <- BSS.packCStringLen (cstr', fromIntegral len) - return (BS.fromStrict bs) - -createKey :: BS.ByteString -> SecretKey -createKey seed = unsafePerformIO $ - useAsOctet seed $ \seed_ptr -> - allocOctet 48 $ \sk_ptr -> - allocOctet (4*48+1) $ \pk_ptr -> do - r <- c'BLS_BLS12381_KEY_PAIR_GENERATE seed_ptr sk_ptr pk_ptr - unless (r == 0) $ fail "Could not create BLS keys" - SecretKey <$> packOctet sk_ptr <*> packOctet pk_ptr - -sign :: SecretKey -> BS.ByteString -> BS.ByteString -sign (SecretKey sk _) msg = unsafePerformIO $ - useAsOctet sk $ \sk_ptr -> - useAsOctet msg $ \msg_ptr -> - allocOctet (48+1) $ \sig_ptr -> do - r <- c'BLS_BLS12381_CORE_SIGN sig_ptr msg_ptr sk_ptr - unless (r == 0) $ fail "Could not create BLS keys" - packOctet sig_ptr - -verify :: BS.ByteString -> BS.ByteString -> BS.ByteString -> Bool -verify pk msg sig = unsafePerformIO $ - useAsOctet pk $ \pk_ptr -> - useAsOctet sig $ \sig_ptr -> - useAsOctet msg $ \msg_ptr -> do - r <- c'BLS_BLS12381_CORE_VERIFY sig_ptr msg_ptr pk_ptr - return (r == 0) diff --git a/impl/src/IC/Crypto/CanisterSig.hs b/impl/src/IC/Crypto/CanisterSig.hs deleted file mode 100644 index 6146ad001..000000000 --- a/impl/src/IC/Crypto/CanisterSig.hs +++ /dev/null @@ -1,86 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE FlexibleContexts #-} -{-# LANGUAGE BlockArguments #-} -module IC.Crypto.CanisterSig - ( genPublicKey - , genSig - , verify - ) where - -import qualified Data.Text as T -import qualified Data.ByteString.Lazy as BS -import qualified Data.ByteString.Builder as BS -import Data.Serialize.Get -import Data.Bifunctor -import Control.Monad -import Codec.CBOR.Term -import Codec.CBOR.Write - -import IC.CBOR.Patterns -import IC.CBOR.Parser -import IC.Types -import IC.Certificate -import IC.Certificate.CBOR -import IC.Certificate.Validate -import IC.Hash -import IC.HashTree -import IC.HashTree.CBOR - --- | Produces a public key, without the DER wrapping -genPublicKey :: EntityId -> BS.ByteString -> BS.ByteString -genPublicKey (EntityId cid) seed = BS.toLazyByteString $ - BS.word8 (fromIntegral (BS.length cid)) <> - BS.lazyByteString cid <> - BS.lazyByteString seed - --- | Parses the public key into a canister id and a seed -parsePublicKey :: BS.ByteString -> Either T.Text (EntityId, BS.ByteString) -parsePublicKey = first T.pack . runGetLazy do - t <- getWord8 - id <- BS.fromStrict <$> getByteString (fromIntegral t) - seed <- BS.fromStrict <$> (remaining >>= getByteString) - return (EntityId id, seed) - -genSig :: Certificate -> HashTree -> BS.ByteString -genSig cert tree = toLazyByteString $ encodeTerm $ TTagged 55799 $ TMap - [ (TString "certificate", TBlob (encodeCert cert)) - , (TString "tree", encodeHashTree tree) - ] - -parseSig :: BS.ByteString -> Either T.Text (Certificate, HashTree) -parseSig s = do - kv <- decodeWithTag s >>= parseMap "canister signature" - certificate <- parseField "certificate" kv >>= - parseBlob "certificate" >>= decodeCert - tree <- parseField "tree" kv >>= parseHashTree - return (certificate, tree) - -verify :: BS.ByteString -> BS.ByteString -> BS.ByteString -> BS.ByteString -> Either T.Text () -verify root_key pk msg sig = do - (id, seed) <- parsePublicKey pk - (certificate, tree) <- parseSig sig - - validateCertificate root_key certificate - - expected_tree_hash <- case lookupPath (cert_tree certificate) - ["canister", rawEntityId id, "certified_data"] of - Found h -> return h - r -> Left $ "Did not find certified_data data for canister id " <> - T.pack (prettyID id) <> " in certificate (got " <> T.pack (show r) <> ")\n" <> - T.pack (show (cert_tree certificate)) - - let actual_tree_hash = reconstruct tree - - unless (expected_tree_hash == actual_tree_hash) $ do - Left $ "Tree hashes did not match.\n" <> - "Certified tree hash: " <> T.pack (prettyBlob expected_tree_hash) <> "\n" <> - "Actual tree hash: " <> T.pack (prettyBlob actual_tree_hash) - - case lookupPath tree ["sig", sha256 seed, sha256 msg] of - Found "" -> return () - Found b -> Left $ "Signature found, but value not \"\", but " <> T.pack (prettyBlob b) - _ -> Left $ "Did not find signature in tree\n" <> - "Seed: " <> T.pack (prettyBlob seed) <> "\n" <> - "Msg: " <> T.pack (prettyBlob msg) <> "\n" <> - "Tree: " <> T.pack (show tree) - diff --git a/impl/src/IC/Crypto/DER.hs b/impl/src/IC/Crypto/DER.hs deleted file mode 100644 index d6d77a347..000000000 --- a/impl/src/IC/Crypto/DER.hs +++ /dev/null @@ -1,91 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -module IC.Crypto.DER (Suite(..), encode, decode) where - -import qualified Data.Text as T -import qualified Data.ByteString.Lazy as BS -import Data.ASN1.Types -import Data.ASN1.Encoding -import Data.ASN1.BinaryEncoding -import Data.ASN1.BitArray - -import IC.Crypto.DER.Decode - -data Suite = Ed25519 | WebAuthn | ECDSA | Secp256k1 | BLS | CanisterSig deriving Show - -webAuthnOID :: OID -webAuthnOID = [1,3,6,1,4,1,56387,1,1] - -canisterSigOID :: OID -canisterSigOID = [1,3,6,1,4,1,56387,1,2] - -ed25519OID :: OID -ed25519OID = [1,3,101,112] - -ecPublicKeyOID :: OID -ecPublicKeyOID =[1,2,840,10045,2,1] - -secp256r1OID :: OID -secp256r1OID = [1,2,840,10045,3,1,7] -secp256k1OID :: OID -secp256k1OID = [1,3,132,0,10] - -blsAlgoOID :: OID -blsAlgoOID = [1,3,6,1,4,1,44668,5,3,1,2,1] -blsCurveOID :: OID -blsCurveOID = [1,3,6,1,4,1,44668,5,3,2,1] - -encode :: Suite -> BS.ByteString -> BS.ByteString -encode Ed25519 = encodeDER [ed25519OID] -encode WebAuthn = encodeDER [webAuthnOID] -encode ECDSA = encodeDER [ecPublicKeyOID, secp256r1OID] -encode Secp256k1 = encodeDER [ecPublicKeyOID, secp256k1OID] -encode BLS = encodeDER [blsAlgoOID, blsCurveOID] -encode CanisterSig = encodeDER [canisterSigOID] - -encodeDER :: [OID] -> BS.ByteString -> BS.ByteString -encodeDER oids pk = encodeASN1 DER $ - [ Start Sequence - , Start Sequence - ] ++ - [ OID oid | oid <- oids ] ++ - [ End Sequence - , BitString (toBitArray (BS.toStrict pk) 0) - , End Sequence - ] - -decode :: BS.ByteString -> Either T.Text (Suite, BS.ByteString) -decode bs = case safeDecode bs of - Left err -> Left $ "Could not decode DER: " <> T.pack err - Right asn -> case asn of - [ Start Sequence - , Start Sequence - , OID algo - , End Sequence - , BitString ba - , End Sequence - ] - | algo == webAuthnOID - -> Right (WebAuthn, BS.fromStrict (bitArrayGetData ba)) - | algo == ed25519OID - -> Right (Ed25519, BS.fromStrict (bitArrayGetData ba)) - | algo == canisterSigOID - -> Right (CanisterSig, BS.fromStrict (bitArrayGetData ba)) - | otherwise - -> Left $ "Unexpected cipher: algo = " <> T.pack (show algo) - [ Start Sequence - , Start Sequence - , OID algo - , OID curve - , End Sequence - , BitString ba - , End Sequence - ] - | algo == ecPublicKeyOID && curve == secp256r1OID - -> Right (ECDSA, BS.fromStrict (bitArrayGetData ba)) - | algo == ecPublicKeyOID && curve == secp256k1OID - -> Right (Secp256k1, BS.fromStrict (bitArrayGetData ba)) - | algo == blsAlgoOID && curve == blsCurveOID - -> Right (BLS, BS.fromStrict (bitArrayGetData ba)) - | otherwise - -> Left $ "Unexpected cipher: algo = " <> T.pack (show algo) <> " curve = " <> T.pack (show curve) - _ -> Left $ "Unexpected DER shape: " <> T.pack (show asn) diff --git a/impl/src/IC/Crypto/DER/Decode.hs b/impl/src/IC/Crypto/DER/Decode.hs deleted file mode 100644 index bdd49c662..000000000 --- a/impl/src/IC/Crypto/DER/Decode.hs +++ /dev/null @@ -1,20 +0,0 @@ -{-# LANGUAGE TypeApplications #-} -module IC.Crypto.DER.Decode (safeDecode) where - -import qualified Data.ByteString.Lazy as BS -import Data.ASN1.Types -import Data.ASN1.Encoding -import Data.ASN1.BinaryEncoding -import Data.Bifunctor - -import Control.Exception -import System.IO.Unsafe -import Control.Monad -import Control.Seq - --- Works around https://github.com/vincenthz/hs-asn1/issues/41 -safeDecode :: BS.ByteString -> Either String [ASN1] -safeDecode bs = unsafePerformIO $ do - let r = first show $ decodeASN1 DER bs - join . first show <$> - try @SomeException (evaluate (r `using` seqFoldable (seqList rseq))) diff --git a/impl/src/IC/Crypto/DER_BLS.hs b/impl/src/IC/Crypto/DER_BLS.hs deleted file mode 100644 index 239eeea3e..000000000 --- a/impl/src/IC/Crypto/DER_BLS.hs +++ /dev/null @@ -1,61 +0,0 @@ --- This module is a bit like IC.Crypto.DER, but only handles BLS signature --- checking --- --- This is used in IC.Crypto.CanisterSig to check signatures (which are only --- BLS), and breaks the module cycle - -{-# LANGUAGE OverloadedStrings #-} -module IC.Crypto.DER_BLS (verify) where - -import qualified Data.Text as T -import qualified Data.ByteString.Lazy as BS -import Data.ASN1.Types -import Data.ASN1.BitArray -import Data.Int -import Control.Monad -import Control.Monad.Error.Class - -import IC.Crypto.DER.Decode -import qualified IC.Crypto.BLS as BLS - -blsAlgoOID :: OID -blsAlgoOID = [1,3,6,1,4,1,44668,5,3,1,2,1] -blsCurveOID :: OID -blsCurveOID = [1,3,6,1,4,1,44668,5,3,2,1] - -decode :: BS.ByteString -> Either T.Text BS.ByteString -decode bs = case safeDecode bs of - Left err -> Left $ "Could not decode DER: " <> T.pack err - Right asn -> case asn of - [ Start Sequence - , Start Sequence - , OID algo - , OID curve - , End Sequence - , BitString ba - , End Sequence - ] - | algo == blsAlgoOID && curve == blsCurveOID - -> Right (BS.fromStrict (bitArrayGetData ba)) - | otherwise - -> Left $ "Unexpected cipher: algo = " <> T.pack (show algo) <> " curve = " <> T.pack (show curve) - _ -> Left $ "Unexpected DER shape: " <> T.pack (show asn) - -verify :: BS.ByteString -> BS.ByteString -> BS.ByteString -> BS.ByteString -> Either T.Text () -verify domain_sep der_pk payload sig = do - pk <- decode der_pk - assertLen "BLS public key" 96 pk - assertLen "BLS signature" 48 sig - - unless (BLS.verify pk msg sig) $ do - when (BLS.verify pk payload sig) $ - throwError $ "domain separator " <> T.pack (show domain_sep) <> " missing" - throwError "signature verification failed" - where - msg = BS.singleton (fromIntegral (BS.length domain_sep)) <> domain_sep <> payload - - -assertLen :: T.Text -> Int64 -> BS.ByteString -> Either T.Text () -assertLen what len bs - | BS.length bs == len = return () - | otherwise = throwError $ what <> " has wrong length " <> T.pack (show (BS.length bs)) <> ", expected " <> T.pack (show len) diff --git a/impl/src/IC/Crypto/ECDSA.hs b/impl/src/IC/Crypto/ECDSA.hs deleted file mode 100644 index 9ff966d94..000000000 --- a/impl/src/IC/Crypto/ECDSA.hs +++ /dev/null @@ -1,60 +0,0 @@ -{-# LANGUAGE TypeApplications #-} -{-# LANGUAGE StandaloneDeriving #-} -{-# LANGUAGE FlexibleInstances #-} -{-# OPTIONS_GHC -Wno-orphans #-} -module IC.Crypto.ECDSA - ( SecretKey - , createKey - , toPublicKey - , IC.Crypto.ECDSA.sign - , IC.Crypto.ECDSA.verify - ) where - -import qualified Data.ByteString.Lazy as BS -import Crypto.ECC -import Crypto.Error -import Crypto.Random -import Crypto.PubKey.ECDSA -import Crypto.Hash.Algorithms -import Crypto.Number.Serialize -import Data.Proxy -import Data.Hashable - -newtype SecretKey = SecretKey (KeyPair Curve_P256R1) - deriving Show - -deriving instance Show (KeyPair Curve_P256R1) - - -createKey :: BS.ByteString -> SecretKey -createKey seed = - SecretKey $ fst $ withDRG drg (curveGenerateKeyPair Proxy) - where - drg = drgNewSeed $ seedFromInteger $ fromIntegral $ hash seed - -toPublicKey :: SecretKey -> BS.ByteString -toPublicKey (SecretKey kp) = - BS.fromStrict $ encodePublic (Proxy @Curve_P256R1) $ keypairGetPublic kp - -sign :: SecretKey -> BS.ByteString -> IO BS.ByteString -sign (SecretKey kp) msg = do - (r,s) <- signatureToIntegers Proxy <$> - Crypto.PubKey.ECDSA.sign (Proxy @Curve_P256R1) (keypairGetPrivate kp) SHA256 (BS.toStrict msg) - return $ BS.fromStrict $ i2ospOf_ 32 r <> i2ospOf_ 32 s - -verify :: BS.ByteString -> BS.ByteString -> BS.ByteString -> Bool -verify pk msg sig - | CryptoPassed pk <- decodePublic (Proxy @Curve_P256R1) (BS.toStrict pk) - , BS.length sig == 64 - , (rb,sb) <- BS.splitAt 32 sig - , let r = os2ip $ BS.toStrict rb - , let s = os2ip $ BS.toStrict sb - , CryptoPassed sig <- signatureFromIntegers (Proxy @Curve_P256R1) (r, s) - = Crypto.PubKey.ECDSA.verify - (Proxy @Curve_P256R1) - SHA256 - pk - sig - (BS.toStrict msg) - | otherwise = False - diff --git a/impl/src/IC/Crypto/Ed25519.hs b/impl/src/IC/Crypto/Ed25519.hs deleted file mode 100644 index 95156d075..000000000 --- a/impl/src/IC/Crypto/Ed25519.hs +++ /dev/null @@ -1,32 +0,0 @@ -module IC.Crypto.Ed25519 - ( SecretKey - , createKey - , toPublicKey - , sign - , verify - ) where - -import qualified Data.ByteString.Lazy as BS -import qualified Crypto.Sign.Ed25519 as Ed25519 - -type SecretKey = Ed25519.SecretKey - -createKey :: BS.ByteString -> SecretKey -createKey seed | BS.length seed > 32 = error "Seed too long" -createKey seed = sk - where - seed' = seed <> BS.replicate (32 - BS.length seed) 0x00 - Just (_, sk) = Ed25519.createKeypairFromSeed_ (BS.toStrict seed') - -toPublicKey :: SecretKey -> BS.ByteString -toPublicKey = BS.fromStrict . Ed25519.unPublicKey . Ed25519.toPublicKey - -sign :: SecretKey -> BS.ByteString -> BS.ByteString -sign sk msg = BS.fromStrict $ Ed25519.unSignature $ Ed25519.dsign sk $ BS.toStrict msg - -verify :: BS.ByteString -> BS.ByteString -> BS.ByteString -> Bool -verify pk msg sig = Ed25519.dverify pk' msg' sig' - where - sig' = Ed25519.Signature (BS.toStrict sig) - pk' = Ed25519.PublicKey (BS.toStrict pk) - msg' = BS.toStrict msg diff --git a/impl/src/IC/Crypto/Secp256k1.hs b/impl/src/IC/Crypto/Secp256k1.hs deleted file mode 100644 index 5e480f603..000000000 --- a/impl/src/IC/Crypto/Secp256k1.hs +++ /dev/null @@ -1,78 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE FlexibleContexts #-} -{-# LANGUAGE BlockArguments #-} -module IC.Crypto.Secp256k1 - ( init - , SecretKey - , createKey - , toPublicKey - , sign - , verify - ) where - -import qualified Data.Text as T -import qualified Data.ByteString.Lazy as BS -import Data.Serialize.Get -import Data.Bifunctor -import Control.Monad -import Data.Hashable -import Control.Monad.Except -import qualified Crypto.PubKey.ECC.ECDSA as EC -import qualified Crypto.PubKey.ECC.Generate as EC -import qualified Crypto.PubKey.ECC.Types as EC -import qualified Crypto.PubKey.ECC.Prim as EC -import Crypto.Number.Serialize -import Crypto.Hash.Algorithms (SHA256(..)) - -data SecretKey = SecretKey EC.PrivateKey EC.PublicKey - deriving Show - -toPublicKey :: SecretKey -> BS.ByteString -toPublicKey (SecretKey _ (EC.PublicKey _ (EC.Point x y))) = - BS.singleton 0x04 <> BS.fromStrict (i2ospOf_ 32 x <> i2ospOf_ 32 y) -toPublicKey (SecretKey _ (EC.PublicKey _ EC.PointO)) = error "toPublicKey: Point at infinity" - -curve :: EC.Curve -curve = EC.getCurveByName EC.SEC_p256k1 - -createKey :: BS.ByteString -> SecretKey -createKey seed = - SecretKey (EC.PrivateKey curve d) (EC.PublicKey curve q) - where - n = EC.ecc_n $ EC.common_curve curve - d = fromIntegral (hash seed) `mod` (n-2) + 1 - q = EC.generateQ curve d - -sign :: SecretKey -> BS.ByteString -> IO BS.ByteString -sign (SecretKey sk _) msg = do - EC.Signature r s <- EC.sign sk SHA256 (BS.toStrict msg) - return $ BS.fromStrict $ i2ospOf_ 32 r <> i2ospOf_ 32 s - --- Parsing SEC keys. Unfortunately not supported directly in cryptonite --- https://github.com/haskell-crypto/cryptonite/issues/302 -parsePublicKey :: BS.ByteString -> Either T.Text EC.PublicKey -parsePublicKey = first T.pack . runGetLazy do - t <- getWord8 - when (t == 0x03) $ do - fail "compressed secp256k1 public keys not supported" - when (t /= 0x04) $ - fail "unexpected public key byte t" - x <- os2ip <$> getByteString 32 - y <- os2ip <$> getByteString 32 - let p = EC.Point x y - unless (EC.isPointValid curve p) $ do - fail "point not vaild" - return $ EC.PublicKey curve p - -parseSig :: BS.ByteString -> Either T.Text EC.Signature -parseSig = first T.pack . runGetLazy do - r <- os2ip <$> getByteString 32 - s <- os2ip <$> getByteString 32 - return $ EC.Signature r s - -verify :: BS.ByteString -> BS.ByteString -> BS.ByteString -> Either T.Text () -verify pk msg sig = do - pk <- parsePublicKey pk - sig <- parseSig sig - unless (EC.verify SHA256 pk sig (BS.toStrict msg)) $ - throwError "secp256k1 signature did not validate" diff --git a/impl/src/IC/Crypto/WebAuthn.hs b/impl/src/IC/Crypto/WebAuthn.hs deleted file mode 100644 index 57795ce00..000000000 --- a/impl/src/IC/Crypto/WebAuthn.hs +++ /dev/null @@ -1,166 +0,0 @@ -{-| -This module implements WebAuthN crypto. WebauthN is a big mess, involving -nesting of CBOR, DER and JSON… --} - -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE LambdaCase #-} -{-# LANGUAGE FlexibleContexts #-} -{-# LANGUAGE BlockArguments #-} -module IC.Crypto.WebAuthn - ( init - , SecretKey - , createKey - , toPublicKey - , sign - , verify - ) where - -import qualified Data.Text as T -import qualified Data.Text.Encoding as T -import qualified Data.ByteString.Lazy.Char8 as BS -import Data.Bifunctor -import Control.Monad -import Data.Hashable -import Data.ByteString.Base64.URL.Lazy as Base64 -import qualified Data.Aeson as JSON -import qualified Data.Aeson.Types as JSON -import qualified IC.HTTP.CBOR as CBOR -import Codec.CBOR.Term -import Codec.CBOR.Write (toLazyByteString) -import IC.CBOR.Parser -import qualified Data.Map as M -import IC.HTTP.GenR.Parse -import IC.Hash -import Control.Monad.Except -import qualified Crypto.PubKey.ECC.ECDSA as EC -import qualified Crypto.PubKey.ECC.Generate as EC -import qualified Crypto.PubKey.ECC.Types as EC -import qualified Crypto.Number.Serialize as EC -import Crypto.Hash.Algorithms (SHA256(..)) -import Data.ASN1.Types -import Data.ASN1.Encoding -import Data.ASN1.BinaryEncoding -import IC.Crypto.DER.Decode - -parseSig :: BS.ByteString -> Either T.Text (BS.ByteString, BS.ByteString, BS.ByteString) -parseSig = CBOR.decode >=> record do - ad <- field blob "authenticator_data" - cdj <- BS.fromStrict . T.encodeUtf8 <$> field text "client_data_json" - sig <- field blob "signature" - return (ad, cdj, sig) - -genSig :: (BS.ByteString, BS.ByteString, BS.ByteString) -> BS.ByteString -genSig (ad, cdj, sig) = - toLazyByteString $ encodeTerm $ TTagged 55799 $ TMap - [ (TString "authenticator_data", TBytes (BS.toStrict ad)) - , (TString "client_data_json", TString (T.decodeUtf8 (BS.toStrict cdj))) - , (TString "signature", TBytes (BS.toStrict sig)) - ] - -parseClientDataJson :: BS.ByteString -> Either T.Text BS.ByteString -parseClientDataJson blob = first T.pack $ - JSON.eitherDecode blob >>= JSON.parseEither p - where - p = JSON.withObject "clientData" $ \o -> do - x <- o JSON..: "challenge" - either JSON.parseFail return $ Base64.decodeUnpadded (BS.pack x) - -genClientDataJson :: BS.ByteString -> BS.ByteString -genClientDataJson challenge = JSON.encode $ JSON.Object $ - "challenge" JSON..= BS.unpack (Base64.encodeUnpadded challenge) - <> "type" JSON..= ("webauthn.get" :: T.Text) - <> "origin" JSON..= ("ic-ref-test" :: T.Text) - -parseCOSEKey :: BS.ByteString -> Either T.Text EC.PublicKey -parseCOSEKey s = do - kv <- decodeWithoutTag s >>= parseMap "COSE key" - m <- M.fromList <$> mapM keyVal kv - let field n = case M.lookup n m of - Just x -> return x - Nothing -> throwError $ "COSE: missing entry " <> T.pack (show n) - let intField n = field n >>= \case - TInt i -> pure i - _ -> throwError $ "COSE field " <> T.pack (show n) <> " not an int" - let bytesField n = field n >>= \case - TBytes b -> pure b - _ -> throwError $ "COSE field " <> T.pack (show n) <> " not bytes" - - ty <- intField 1 - unless (ty == 2) $ - throwError "COSE: Only key type 2 (EC2) supported" - ty <- intField 3 - unless (ty == -7) $ - throwError "COSE: Only type -7 (ECDSA) supported" - crv <- intField (-1) - unless (crv == 1) $ - throwError $ "parsePublicKey: unknown curve: " <> T.pack (show crv) - xb <- bytesField (-2) - yb <- bytesField (-3) - let x = EC.os2ip xb - let y = EC.os2ip yb - return $ EC.PublicKey curve (EC.Point x y) - where - keyVal (TInt k,v) = pure (fromIntegral k,v) - keyVal (TInteger k,v) = pure (k,v) - keyVal _ = throwError "Non-integer key in CBOR map" - -genCOSEKey :: EC.PublicKey -> BS.ByteString -genCOSEKey (EC.PublicKey _curve (EC.Point x y)) = - toLazyByteString $ encodeTerm $ TMap - [ (TInt 1, TInt 2) - , (TInt 3, TInt (-7)) - , (TInt (-1), TInt 1) - , (TInt (-2), TBytes (EC.i2ospOf_ 32 x)) - , (TInt (-3), TBytes (EC.i2ospOf_ 32 y)) - ] -genCOSEKey (EC.PublicKey _ EC.PointO) = error "genCOSEKey: Point at infinity" - - - -parseCOSESig :: BS.ByteString -> Either T.Text EC.Signature -parseCOSESig s = - first T.pack (safeDecode s) >>= \case - [Start Sequence,IntVal r,IntVal s,End Sequence] -> pure $ EC.Signature r s - a -> throwError $ "Unexpected DER encoding for COSE sig: " <> T.pack (show a) - -genCOSESig :: EC.Signature -> BS.ByteString -genCOSESig (EC.Signature r s) = encodeASN1 DER - [Start Sequence,IntVal r,IntVal s,End Sequence] - -data SecretKey = SecretKey EC.PrivateKey EC.PublicKey - deriving Show - -curve :: EC.Curve -curve = EC.getCurveByName EC.SEC_p256r1 - -createKey :: BS.ByteString -> SecretKey -createKey seed = - SecretKey (EC.PrivateKey curve d) (EC.PublicKey curve q) - where - n = EC.ecc_n $ EC.common_curve curve - d = fromIntegral (hash seed) `mod` (n-2) + 1 - q = EC.generateQ curve d - -toPublicKey :: SecretKey -> BS.ByteString -toPublicKey (SecretKey _ pk) = genCOSEKey pk - - -sign :: SecretKey -> BS.ByteString -> IO BS.ByteString -sign (SecretKey sk _) msg = do - let cdj = genClientDataJson msg - let ad = "arbitrary?" - sig <- EC.sign sk SHA256 (BS.toStrict (ad <> sha256 cdj)) - return $ genSig (ad, cdj, genCOSESig sig) - -verify :: BS.ByteString -> BS.ByteString -> BS.ByteString -> Either T.Text () -verify pk msg sig = do - (ad, cdj, sig) <- parseSig sig - pk <- parseCOSEKey pk - sig <- parseCOSESig sig - unless (EC.verify SHA256 pk sig (BS.toStrict $ ad <> sha256 cdj)) $ - throwError "WebAuthn signature verification failed" - challenge <- parseClientDataJson cdj - unless (challenge == msg) $ - throwError $ "Wrong challenge. Expected " <> T.pack (show msg) <> - " got " <> T.pack (show challenge) diff --git a/impl/src/IC/DRun/Parse.hs b/impl/src/IC/DRun/Parse.hs deleted file mode 100644 index c81a7f681..000000000 --- a/impl/src/IC/DRun/Parse.hs +++ /dev/null @@ -1,65 +0,0 @@ -{-# LANGUAGE ScopedTypeVariables #-} -module IC.DRun.Parse where - -import qualified Data.ByteString.Lazy.Char8 as B -import qualified Text.Hex as H -import qualified Data.Text as T -import Data.ByteString.Base32 -import Control.Exception - -type MethodName = String -type Payload = B.ByteString -type Id = B.ByteString - -data Ingress - = Create - | Install Id FilePath Payload - | Reinstall Id FilePath Payload - | Upgrade Id FilePath Payload - | Update Id MethodName Payload - | Query Id MethodName Payload - deriving Show - -parseFile :: FilePath -> IO [Ingress] -parseFile input = do - x <- parse <$> readFile input - _ <- evaluate (show x) -- hack to evaluate until we have a proper parser - return x - -parse :: String -> [Ingress] -parse = map parseLine . lines - -parseLine :: String -> Ingress -parseLine l = case words l of - ["create"] -> Create - ["install", i, f, a] -> Install (parseId i) f (parseArg a) - ["reinstall", i, f, a] -> Reinstall (parseId i) f (parseArg a) - ["upgrade", i, f, a] -> Upgrade (parseId i) f (parseArg a) - ["ingress", i, m, a] -> Update (parseId i) m (parseArg a) - ["query", i, m, a] -> Query (parseId i) m (parseArg a) - _ -> error $ "Cannot parse: " ++ show l - --- TODO: Implement proper and extract in own module -parseId :: String -> Id -parseId s = case B.fromStrict <$> decodeBase32Unpadded (B.toStrict (B.pack (filter (/= '-') s))) of - Right bytes -> - if B.length bytes >= 4 - then B.drop 4 bytes - else error "Too short id" - Left err -> error $ "Invalid canister id: " ++ T.unpack err - -parseArg :: String -> Payload -parseArg ('0':'x':xs) - | Just x <- B.fromStrict <$> H.decodeHex (T.pack xs) = x -parseArg ('"':xs) - = B.pack $ go xs - where - go "" = error "Missing terminating \"" - go "\"" = [] - go ('\\':'x':a:b:ys) - | Just h <- H.decodeHex (T.pack [a,b]) - = B.unpack (B.fromStrict h) ++ go ys - go (c:ys) = c : go ys -parseArg x = error $ "Invalid argument " ++ x - - diff --git a/impl/src/IC/Debug/JSON.hs b/impl/src/IC/Debug/JSON.hs deleted file mode 100644 index 3a409b168..000000000 --- a/impl/src/IC/Debug/JSON.hs +++ /dev/null @@ -1,188 +0,0 @@ -{-# LANGUAGE StandaloneDeriving #-} -{-# LANGUAGE ConstraintKinds #-} -{-# LANGUAGE DeriveGeneric #-} -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE TypeSynonymInstances #-} -{-# OPTIONS_GHC -Wno-orphans #-} -{- | -This module defines ToJSON instances of the IC state. - -We put them into their own module, despite the usual advise against orphan -instances, to emphasize that these are there just for debugging purposes using -`ic-ref`. - -(Why JSON? Because Browsers render them nicely in a interactive display where -you can open and collapse subcomponents – much easier to get this feature this -way, compared to writing custom HTML output.) --} - -module IC.Debug.JSON () where - -import GHC.Generics -import Data.Aeson -import Data.Aeson.Types - -import qualified Data.ByteString.Lazy as BS -import qualified Wasm.Syntax.Values as W -import qualified Wasm.Syntax.AST as W -import qualified Text.Hex as H -import qualified Data.Text as T -import Control.Monad.Random.Lazy - -import IC.Types -import IC.Wasm.Winter.Persist -import IC.Purify -import IC.Canister.Snapshot -import IC.Canister -import IC.Ref -import IC.Crypto - -customOptions :: Options -customOptions = defaultOptions - { sumEncoding = ObjectWithSingleField - } - -instance ToJSON W.Value where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -instance ToJSON BS.ByteString where - toJSON = String . H.encodeHex . BS.toStrict - -instance ToJSONKey BS.ByteString where - toJSONKey = toJSONKeyText (H.encodeHex . BS.toStrict) - -instance ToJSON (W.Module f) where - toJSON = placeholder "(module)" - -instance ToJSON (Replay i) where - toJSON = placeholder "(replay)" - -placeholder :: String -> a -> Value -placeholder s = const (String (T.pack s)) - -deriving instance Generic Timestamp -instance ToJSON Timestamp where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic Responded -instance ToJSON Responded where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic RejectCode -instance ToJSON RejectCode where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic Response -instance ToJSON Response where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic WasmClosure -instance ToJSON WasmClosure where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic Callback -instance ToJSON Callback where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic MethodCall -instance ToJSON MethodCall where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic PInstance -instance ToJSON PInstance where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic PModuleInst -instance ToJSON PModuleInst where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic (Snapshot a) -instance ToJSON a => ToJSON (Snapshot a) where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic CanisterSnapshot -instance ToJSON CanisterSnapshot where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic IC -instance ToJSON IC where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic CallContext -instance ToJSON CallContext where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic Message -instance ToJSON Message where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic RequestStatus -instance ToJSON RequestStatus where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic CallResponse -instance ToJSON CallResponse where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - - -deriving instance Generic CallRequest -instance ToJSON CallRequest where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic RunStatus -instance ToJSON RunStatus where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic CanState -instance ToJSON CanState where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic CanisterContent -instance ToJSON CanisterContent where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic CallOrigin -instance ToJSON CallOrigin where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -deriving instance Generic EntryPoint -instance ToJSON EntryPoint where - toJSON = genericToJSON customOptions - toEncoding = genericToEncoding customOptions - -instance ToJSON CanisterModule where - toJSON = placeholder "(CanisterModule)" - -instance ToJSON EntityId where - toJSON = toJSON . prettyID - -instance ToJSONKey EntityId where - toJSONKey = contramapToJSONKeyFunction prettyID toJSONKey - -instance ToJSON StdGen where - toJSON = toJSON . show - -instance ToJSON SecretKey where - toJSON = placeholder "(secret key)" diff --git a/impl/src/IC/HTTP.hs b/impl/src/IC/HTTP.hs deleted file mode 100644 index 4f30210ae..000000000 --- a/impl/src/IC/HTTP.hs +++ /dev/null @@ -1,139 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE LambdaCase #-} -module IC.HTTP where - -import Network.Wai -import Control.Concurrent (forkIO) -import Network.HTTP.Types -import qualified Data.Text as T -import qualified Data.Text.Encoding as T -import Data.ByteString.Builder (stringUtf8) -import Control.Monad.State -import Control.Monad.Except -import Data.Aeson as JSON -import Codec.Candid (Principal(..), parsePrincipal) - -import IC.Types -import IC.Ref -import IC.HTTP.Status -import IC.HTTP.CBOR -import IC.HTTP.GenR -import IC.HTTP.Request -import IC.HTTP.RequestId -import IC.Debug.JSON () -import IC.Serialise () -import IC.StateFile -import IC.Crypto - -withApp :: Maybe FilePath -> (Application -> IO a) -> IO a -withApp backingFile action = - withStore initialIC backingFile (action . handle) - -handle :: Store IC -> Application -handle store req respond = case (requestMethod req, pathInfo req) of - ("GET", []) -> peekStore store >>= json status200 - ("GET", ["api","v1",_]) -> noV1 req - ("GET", ["api","v2","status"]) -> do - r <- peekIC $ gets IC.HTTP.Status.r - cbor status200 r - ("POST", ["api","v2","canister",textual_ecid,verb]) -> - case parsePrincipal textual_ecid of - Left err -> invalidRequest $ "cannot parse effective canister id: " <> T.pack err - Right (Principal ecid) -> do - root_key <- peekIC $ gets $ toPublicKey . secretRootKey - case verb of - "call" -> withSignedCBOR root_key $ \(gr, ev) -> case callRequest gr of - Left err -> invalidRequest err - Right cr -> runIC $ do - t <- lift getTimestamp - runExceptT (authCallRequest t (EntityId ecid) ev cr) >>= \case - Left err -> - lift $ invalidRequest err - Right () -> do - submitRequest (requestId gr) cr - lift $ empty status202 - "query" -> withSignedCBOR root_key $ \(gr, ev) -> case queryRequest gr of - Left err -> invalidRequest err - Right qr -> peekIC $ do - t <- lift getTimestamp - runExceptT (authQueryRequest t (EntityId ecid) ev qr) >>= \case - Left err -> - lift $ invalidRequest err - Right () -> do - t <- lift getTimestamp - r <- handleQuery t qr - lift $ cbor status200 (IC.HTTP.Request.response r) - "read_state" -> withSignedCBOR root_key $ \(gr, ev) -> case readStateRequest gr of - Left err -> invalidRequest err - Right rsr -> peekIC $ do - t <- lift getTimestamp - runExceptT (authReadStateRequest t (EntityId ecid) ev rsr) >>= \case - Left err -> - lift $ invalidRequest err - Right () -> do - t <- lift getTimestamp - r <- handleReadState t rsr - lift $ cbor status200 (IC.HTTP.Request.response r) - _ -> notFound req - _ -> notFound req - where - runIC :: StateT IC IO a -> IO a - runIC a = do - x <- modifyStore store $ do - -- Here we make IC.Ref use “real time” - lift getTimestamp >>= setAllTimesTo - a - -- begin processing in the background (it is important that - -- this thread returns, else warp is blocked somehow) - void $ forkIO loopIC - return x - - -- Not atomic, reads most recent state - peekIC :: StateT IC IO a -> IO a - peekIC a = peekStore store >>= evalStateT a - - loopIC :: IO () - loopIC = modifyStore store runStep >>= \case - True -> loopIC - False -> return () - - cbor status gr = respond $ responseBuilder - status - [ (hContentType, "application/cbor") ] - (IC.HTTP.CBOR.encode gr) - - json status x = respond $ responseBuilder - status - [ (hContentType, "application/json") ] - (JSON.fromEncoding $ JSON.toEncoding x) - - plain status x = respond $ responseBuilder - status - [ (hContentType, "text/plain") ] - x - - empty status = plain status mempty - - invalidRequest msg = do - when False $ print (T.unpack msg) - -- ^ When testing against dfx, and until it prints error messages - -- this can be enabled - plain status400 (T.encodeUtf8Builder msg) - - notFound req = plain status404 $ stringUtf8 $ - "ic-ref does not know how to handle a " ++ show (requestMethod req) ++ - " request to " ++ show (rawPathInfo req) - - noV1 req = plain status404 $ stringUtf8 $ - "ic-ref no longer supports the v1 HTTP API at " ++ show (rawPathInfo req) - - withCBOR k = case lookup hContentType (requestHeaders req) of - Just "application/cbor" -> do - body <- strictRequestBody req - case IC.HTTP.CBOR.decode body of - Left err -> invalidRequest err - Right gr -> k gr - _ -> invalidRequest "Expected application/cbor request" - - withSignedCBOR :: Blob -> ((GenR, EnvValidity) -> IO ResponseReceived) -> IO ResponseReceived - withSignedCBOR root_key k = withCBOR $ either invalidRequest k . stripEnvelope root_key diff --git a/impl/src/IC/HTTP/CBOR.hs b/impl/src/IC/HTTP/CBOR.hs deleted file mode 100644 index dd6777f9f..000000000 --- a/impl/src/IC/HTTP/CBOR.hs +++ /dev/null @@ -1,63 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE TupleSections #-} -{- | -Encoding from generic requests/responses to/from CBOR --} -module IC.HTTP.CBOR where - -import IC.HTTP.GenR -import qualified Data.HashMap.Lazy as HM -import Codec.CBOR.Term -import Codec.CBOR.Write -import Codec.CBOR.Read -import Data.ByteString.Builder (Builder) -import qualified Data.ByteString.Lazy as BS -import Data.ByteString.Lazy (ByteString) -import Data.Bifunctor -import qualified Data.Text as T -import Control.Monad - -encode :: GenR -> Builder -encode r = toBuilder $ encodeTerm $ TTagged 55799 $ go r - where - go (GNat n) = TInteger (fromIntegral n) - go (GText t) = TString t - go (GBlob b) = TBytes (BS.toStrict b) - go (GRec m) = TMap [ (TString k, go v) | (k,v) <- HM.toList m ] - go (GList xs) = TList (map go xs) - -decode :: ByteString -> Either T.Text GenR -decode s = - first (\(DeserialiseFailure _ s) -> "CBOR decoding failure: " <> T.pack s) - (deserialiseFromBytes decodeTerm s) - >>= begin - where - begin (leftOver, _) - | not (BS.null leftOver) = Left $ "Left-over bytes: " <> shorten 20 (T.pack (show leftOver)) - begin (_, TTagged 55799 t) = go t - begin _ = Left "Expected CBOR request to begin with tag 55799" - - shorten :: Int -> T.Text -> T.Text - shorten n s = a <> (if T.null b then "" else "…") - where (a,b) = T.splitAt n s - - go (TInt n) | n < 0 = Left "Negative integer" - go (TInt n) = return $ GNat (fromIntegral n) - go (TInteger n) | n < 0 = Left "Negative integer" - go (TInteger n) = return $ GNat (fromIntegral n) - go (TBytes b) = return $ GBlob $ BS.fromStrict b - go (TString t) = return $ GText t - go (TMap kv) = goMap kv - go (TMapI kv) = goMap kv - go (TList vs) = GList <$> mapM go vs - go (TListI vs) = GList <$> mapM go vs - go t = Left $ "Unexpected term: " <> T.pack (show t) - - goMap kv = do - tv <- mapM keyVal kv - let hm = HM.fromList tv - when (HM.size hm < length tv) $ Left "Duplicate keys in CBOR map" - return (GRec hm) - - keyVal (TString k,v) = (k,) <$> go v - keyVal _ = Left "Non-string key in CBOR map" diff --git a/impl/src/IC/HTTP/GenR.hs b/impl/src/IC/HTTP/GenR.hs deleted file mode 100644 index 15afb5caa..000000000 --- a/impl/src/IC/HTTP/GenR.hs +++ /dev/null @@ -1,34 +0,0 @@ -{-| -This module describe a type for our “generic request (or response)” format. It -can be seen as a simplified (and more abstract) AST for CBOR data. - -The following operations can be done on generic requests - * Parsing from CBOR - * Encoding to CBOR - * Request ID calculation - * Thus: Signing and signature checking --} -module IC.HTTP.GenR where - -import Numeric.Natural -import Data.Text -import Data.ByteString.Lazy -import Data.HashMap.Lazy - -data GenR - = GNat Natural - | GText Text - | GBlob ByteString - | GRec (HashMap Text GenR) - | GList [GenR] - deriving Show - -emptyR :: GenR -emptyR = GRec Data.HashMap.Lazy.empty - --- For assembling generic records -(=:) :: Text -> v -> HashMap Text v -(=:) = Data.HashMap.Lazy.singleton -rec :: [HashMap Text GenR] -> GenR -rec = GRec . mconcat - diff --git a/impl/src/IC/HTTP/GenR/Parse.hs b/impl/src/IC/HTTP/GenR/Parse.hs deleted file mode 100644 index d96dcfb96..000000000 --- a/impl/src/IC/HTTP/GenR/Parse.hs +++ /dev/null @@ -1,84 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE RankNTypes #-} -{-# LANGUAGE ConstraintKinds #-} -{-# LANGUAGE FlexibleContexts #-} -{-# LANGUAGE FlexibleInstances #-} - -{- | -Utilities to deconstruct a generic record. --} -module IC.HTTP.GenR.Parse where - -import Numeric.Natural -import qualified Data.Text as T -import qualified Data.ByteString.Lazy as BS -import Control.Monad.State -import Control.Monad.Writer -import qualified Data.HashMap.Lazy as HM -import GHC.Stack - -import IC.HTTP.GenR - --- A monad to parse a record --- (reading each field once, checking for left-over fields in the end) -type RecordM m = StateT (HM.HashMap T.Text GenR) m -type Field a = forall m. HasCallStack => Parse m => GenR -> m a -class Monad m => Parse m where parseError :: HasCallStack => T.Text -> m a - -instance Parse (Either T.Text) where parseError = Left -instance (Monoid a, Parse m) => Parse (WriterT a m) where parseError = lift . parseError - -record :: HasCallStack => Parse m => RecordM m a -> GenR -> m a -record m (GRec hm) = (`evalStateT` hm) $ do - x <- m - -- Check for left-over fields - hm <- get - unless (HM.null hm) $ lift $ - parseError $ "Unexpected fields: " <> T.intercalate ", " (HM.keys hm) - return x -record _ _ = parseError "Expected CBOR record" - -field :: HasCallStack => Parse m => Field a -> T.Text -> RecordM m a -field parse name = do - hm <- get - put (HM.delete name hm) - lift $ case HM.lookup name hm of - Nothing -> parseError $ "Missing expected field \"" <> name <> "\"" - Just gr -> parse gr - -optionalField :: HasCallStack => Parse m => Field a -> T.Text -> RecordM m (Maybe a) -optionalField parse name = do - hm <- get - put (HM.delete name hm) - case HM.lookup name hm of - Nothing -> return Nothing - Just gr -> lift $ Just <$> parse gr - -swallowAllFields :: Monad m => RecordM m () -swallowAllFields = put HM.empty - -anyType :: Field GenR -anyType = return - -text :: Field T.Text -text (GText t) = return t -text _ = parseError "Expected text value" - -blob :: Field BS.ByteString -blob (GBlob b) = return b -blob _ = parseError "Expected blob" - -nat :: Field Natural -nat (GNat n) = return n -nat _ = parseError "Expected natural number" - -percentage :: Field Natural -percentage gr = do - n <- nat gr - unless (0 <= n && n <= 100) $ - parseError "Expected a percentage (0..100)" - return n - -listOf :: Field a -> Field [a] -listOf f (GList xs) = mapM f xs -listOf _ _ = parseError "Expected a list" diff --git a/impl/src/IC/HTTP/Request.hs b/impl/src/IC/HTTP/Request.hs deleted file mode 100644 index 743b28c9e..000000000 --- a/impl/src/IC/HTTP/Request.hs +++ /dev/null @@ -1,159 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE NumericUnderscores #-} -{-# LANGUAGE FlexibleContexts #-} -{- | Parses/produces generic requests -} -module IC.HTTP.Request where - -import qualified Data.Text as T -import qualified Data.ByteString.Lazy as BS -import qualified Data.HashMap.Lazy as HM -import Control.Monad.Except -import Control.Monad.Writer -import Data.Foldable -import Data.Time.Clock.POSIX -import Data.Maybe -import Data.Bifunctor - -import IC.Types -import IC.Crypto -import IC.Ref (CallRequest(..), QueryRequest(..), ReadStateRequest(..), - ReqResponse(..), CallResponse(..)) -import IC.Id.Forms hiding (Blob) -import IC.Certificate.CBOR -import IC.HTTP.RequestId -import IC.HTTP.GenR -import IC.HTTP.GenR.Parse - -dummyUserId :: EntityId -dummyUserId = EntityId $ BS.pack [0xCA, 0xFF, 0xEE] - -stripEnvelope :: Blob -> GenR -> Either T.Text (GenR, EnvValidity) -stripEnvelope root_key gr = runWriterT $ flip record gr $ do - content <- field anyType "content" - pk <- optionalField blob "sender_pubkey" - sig <- optionalField blob "sender_sig" - checkExpiry content - case (pk, sig) of - (Just pk, Just sig) -> do - tell $ validFor $ \(EntityId id) -> - unless (isSelfAuthenticatingId pk id) $ - throwError "Public key not authorized to sign for user" - - delegations <- optionalField (listOf delegationField) "sender_delegation" - pk' <- checkDelegations pk (fromMaybe [] delegations) - - let rid = requestId content - lift $ lift $ - first (<> "\nExpected request id: " <> T.pack (prettyBlob rid) - <> "\nPublic Key: " <> T.pack (prettyBlob pk') - <> "\nSignature: " <> T.pack (prettyBlob sig)) $ - verify root_key "ic-request" pk' rid sig - (Nothing, Nothing) -> - tell $ validFor $ \(EntityId id) -> - unless (isAnonymousId id) $ - throwError "Missing signature, but user is not the anonymous user" - _ -> throwError "Need to set either both or none of sender_pubkey and sender_sig" - return content - - where - checkDelegations pk [] = return pk - checkDelegations pk ((pk', Timestamp expiry, targets, hash, sig):ds) = do - lift $ lift $ verify root_key "ic-request-auth-delegation" pk hash sig - tell $ validWhen $ \(Timestamp t) -> - unless (expiry > t) $ - throwError $ "Delegation expiry is " <> T.pack (show ((t - expiry)`div`1000_000_000)) <> " seconds in the past" - for_ targets $ \ts -> - tell $ validWhere $ \c -> - unless (c `elem` ts) $ - throwError "Delegation does not apply to this canister" - checkDelegations pk' ds - - - checkExpiry :: GenR -> RecordM (WriterT EnvValidity (Either T.Text)) () - checkExpiry (GRec hm) - | Just (GNat expiry) <- HM.lookup "ingress_expiry" hm = - tell $ validWhen $ \(Timestamp t) -> do - -- Here we check that the expiry field is not in the past and not - -- too far in the future - unless (expiry > t) $ - throwError $ "Expiry is " <> T.pack (show ((t - expiry)`div`1000_000_000)) <> " seconds in the past" - unless (expiry < t + max_future) $ - throwError $ "Expiry is " <> T.pack (show ((expiry - t)`div`1000_000_000)) <> " seconds in the future" - where - -- max expiry time is 5 minutes - max_future = 5*60*1000_000_000 - checkExpiry _ = throwError "No ingress_expiry field found" - - -type DelegationHash = Blob -delegationField :: Field (PublicKey, Timestamp, Maybe [EntityId], DelegationHash, Blob) -delegationField = record $ do - delegation <- field anyType "delegation" - (pk, ts, targets) <- lift $ flip record delegation $ do - pk <- field blob "pubkey" - ts <- Timestamp <$> field nat "expiration" - targets <- optionalField (listOf entitiyId) "targets" - return (pk, ts, targets) - sig <- field blob "signature" - return (pk, ts, targets, requestId delegation, sig) - -getTimestamp :: IO Timestamp -getTimestamp = do - t <- getPOSIXTime - return $ Timestamp $ round (t * 1000_000_000) - --- Parsing requests -callRequest :: GenR -> Either T.Text CallRequest -callRequest = record $ do - t <- field text "request_type" - unless (t == "call") $ - throwError $ "Expected request_type to be \"call\", got \"" <> t <> "\"" - _ <- optionalField blob "nonce" - _ <- field nat "ingress_expiry" - cid <- field entitiyId "canister_id" - sender <- field entitiyId "sender" - method_name <- field text "method_name" - arg <- field blob "arg" - return $ CallRequest cid sender (T.unpack method_name) arg - -queryRequest :: GenR -> Either T.Text QueryRequest -queryRequest = record $ do - t <- field text "request_type" - unless (t == "query") $ - throwError $ "Expected request_type to be \"query\", got \"" <> t <> "\"" - _ <- optionalField blob "nonce" - _ <- field nat "ingress_expiry" - cid <- field entitiyId "canister_id" - sender <- field entitiyId "sender" - method_name <- field text "method_name" - arg <- field blob "arg" - return $ QueryRequest cid sender (T.unpack method_name) arg - -readStateRequest :: GenR -> Either T.Text ReadStateRequest -readStateRequest = record $ do - t <- field text "request_type" - unless (t == "read_state") $ - throwError $ "Expected request_type to be \"read_state\", got \"" <> t <> "\"" - _ <- optionalField blob "nonce" - _ <- field nat "ingress_expiry" - sender <- field entitiyId "sender" - paths <- field (listOf (listOf blob)) "paths" - return $ ReadStateRequest sender paths - -entitiyId :: Field EntityId -entitiyId = fmap EntityId <$> blob - --- Printing responses -response :: ReqResponse -> GenR -response (QueryResponse (Rejected (c, s))) = rec - [ "status" =: GText "rejected" - , "reject_code" =: GNat (fromIntegral (rejectCode c)) - , "reject_message" =: GText (T.pack s) - ] -response (QueryResponse (Replied blob)) = rec - [ "status" =: GText "replied" - , "reply" =: rec [ "arg" =: GBlob blob ] - ] -response (ReadStateResponse cert) = rec - [ "certificate" =: GBlob (encodeCert cert) - ] diff --git a/impl/src/IC/HTTP/RequestId.hs b/impl/src/IC/HTTP/RequestId.hs deleted file mode 100644 index 4de454d10..000000000 --- a/impl/src/IC/HTTP/RequestId.hs +++ /dev/null @@ -1,33 +0,0 @@ -module IC.HTTP.RequestId (requestId) where - -import Numeric.Natural -import IC.HTTP.GenR -import qualified Data.ByteString.Lazy as BS -import qualified Data.HashMap.Lazy as HM -import qualified Data.Text as T -import qualified Data.Text.Encoding as T -import Data.List (sort) -import Data.Serialize.LEB128 -import IC.Hash - -type RequestId = BS.ByteString - -requestId :: GenR -> RequestId -requestId (GRec hm) = sha256 $ BS.concat $ sort $ map encodeKV $ HM.toList hm -requestId _ = error "requestID: expected a record" - -encodeKV :: (T.Text, GenR) -> BS.ByteString -encodeKV (k,v) = sha256 (encodeText k) <> sha256 (encodeVal v) - -encodeVal :: GenR -> BS.ByteString -encodeVal (GBlob b) = b -encodeVal (GText t) = encodeText t -encodeVal (GNat n) = encodeNat n -encodeVal (GRec _) = error "requestID: Nested record" -encodeVal (GList vs) = BS.concat $ map (sha256 . encodeVal) vs - -encodeText :: T.Text -> BS.ByteString -encodeText = BS.fromStrict . T.encodeUtf8 - -encodeNat :: Natural -> BS.ByteString -encodeNat = BS.fromStrict . toLEB128 diff --git a/impl/src/IC/HTTP/Status.hs b/impl/src/IC/HTTP/Status.hs deleted file mode 100644 index 02ee6308a..000000000 --- a/impl/src/IC/HTTP/Status.hs +++ /dev/null @@ -1,20 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} --- | the response to the status request -module IC.HTTP.Status where - -import IC.HTTP.GenR -import IC.Version -import IC.Ref -import IC.Crypto -import Data.HashMap.Lazy - -r :: IC -> GenR -r ic = GRec $ mconcat - [ "ic_api_version" =: GText specVersion - , "impl_version" =: GText implVersion - , "impl_source" =: GText "https://github.com/dfinity-lab/ic-ref" - , "root_key" =: GBlob (toPublicKey (secretRootKey ic)) - ] - where - -- Convenient syntax - (=:) = Data.HashMap.Lazy.singleton diff --git a/impl/src/IC/Hash.hs b/impl/src/IC/Hash.hs deleted file mode 100644 index 8927b6bb4..000000000 --- a/impl/src/IC/Hash.hs +++ /dev/null @@ -1,12 +0,0 @@ -{-# LANGUAGE TypeApplications #-} -module IC.Hash where - -import qualified Data.ByteString.Lazy as BS -import Crypto.Hash (hashlazy, SHA256, SHA224) -import Data.ByteArray (convert) - -sha256 :: BS.ByteString -> BS.ByteString -sha256 = BS.fromStrict . convert . hashlazy @SHA256 - -sha224 :: BS.ByteString -> BS.ByteString -sha224 = BS.fromStrict . convert . hashlazy @SHA224 diff --git a/impl/src/IC/HashTree.hs b/impl/src/IC/HashTree.hs deleted file mode 100644 index bbeba8099..000000000 --- a/impl/src/IC/HashTree.hs +++ /dev/null @@ -1,148 +0,0 @@ --- | This module implements the (possible pruned) merkle trees used in the --- Internet Computer, in particular --- * Conversion from a labeled tree (with blobs) --- * Root hash reconstruction --- * Lookup --- * Pruning --- * Checking well-formedness - -{-# LANGUAGE TypeApplications #-} -{-# LANGUAGE OverloadedStrings #-} -module IC.HashTree where - -import qualified Data.Map.Lazy as M -import qualified Data.Set as S -import qualified Data.ByteString.Lazy as BS -import Crypto.Hash (hashlazy, SHA256) -import Data.ByteArray (convert) - -type Blob = BS.ByteString -type Path = [Label] -type Label = Blob -type Value = Blob -type Hash = Blob - -data LabeledTree - = Value Value - | SubTrees (M.Map Blob LabeledTree) - deriving Show - -data HashTree - = EmptyTree - | Fork HashTree HashTree - | Labeled Blob HashTree - | Leaf Value - | Pruned Hash - deriving Show - -construct :: LabeledTree -> HashTree -construct (Value v) = Leaf v -construct (SubTrees m) = - foldBinary EmptyTree Fork - [ Labeled k (construct v) | (k,v) <- M.toAscList m ] - -foldBinary :: a -> (a -> a -> a) -> [a] -> a -foldBinary e (⋔) = go - where - go [] = e - go [x] = x - go xs = go xs1 ⋔ go xs2 - where (xs1, xs2) = splitAt (length xs `div` 2) xs - -reconstruct :: HashTree -> Hash -reconstruct = go - where - go EmptyTree = h $ domSep "ic-hashtree-empty" - go (Fork t1 t2) = h $ domSep "ic-hashtree-fork" <> go t1 <> go t2 - go (Labeled l t) = h $ domSep "ic-hashtree-labeled" <> l <> go t - go (Leaf v) = h $ domSep "ic-hashtree-leaf" <> v - go (Pruned h) = h - - -h :: BS.ByteString -> BS.ByteString -h = BS.fromStrict . convert . hashlazy @SHA256 - -domSep :: Blob -> Blob -domSep s = BS.singleton (fromIntegral (BS.length s)) <> s - -data Res = Absent | Unknown | Error String | Found Value - deriving (Eq, Show) - --- See lookupL in IC.Test.HashTree for a high-level spec -lookupPath :: HashTree -> Path -> Res -lookupPath tree (l:ls) = find Absent (flatten tree) - where - find r [] = r - find r (Labeled l' t : ts) | l < l' = r - | l == l' = lookupPath t ls - | otherwise = find Absent ts - find _ (Pruned _ : ts) = find Unknown ts - find _ (EmptyTree : _) = error "Empty in flattened list" - find _ (Fork _ _ : _) = error "Fork in flattened list" - find _ (Leaf _ : _) = Error "Found leaf when expecting subtree" - -lookupPath (Leaf v) [] = Found v -lookupPath (Pruned _) [] = Unknown -lookupPath (Labeled _ _) [] = Error "Found forest when expecting leaf" -lookupPath (Fork _ _) [] = Error "Found forest when expecting leaf" -lookupPath _ [] = Error "Found forest when expecting leaf" - -flatten :: HashTree -> [HashTree] -flatten t = go t [] -- using difference lists - where - go EmptyTree = id - go (Fork t1 t2) = go t1 . go t2 - go t = (t:) - -prune :: HashTree -> [Path] -> HashTree -prune tree [] = Pruned (reconstruct tree) -prune tree paths | [] `elem` paths = tree -prune tree paths = go tree - where - -- These labels are availbale - present :: S.Set Label - present = S.fromList [ l | Labeled l _ <- flatten tree] - - -- We need all requested labels, and if not present, the immediate neighbors - -- This maps labels to paths at that label that we need - wanted :: M.Map Label (S.Set Path) - wanted = M.fromListWith S.union $ concat - [ if l `S.member` present - then [ (l, S.singleton p) ] - else - [ (l', S.empty) | Just l' <- pure $ l `S.lookupLT` present ] ++ - [ (l', S.empty) | Just l' <- pure $ l `S.lookupGT` present ] - | l:p <- paths ] - - -- Smart constructor to avoid unnecessary forks - fork t1 t2 - | prunedOrEmpty t1, prunedOrEmpty t2 = Pruned (reconstruct (Fork t1 t2)) - | otherwise = Fork t1 t2 - where - prunedOrEmpty (Pruned _) = True - prunedOrEmpty EmptyTree = True - prunedOrEmpty _ = False - - go EmptyTree = EmptyTree - go (Labeled l subtree) - | Just path_tails <- M.lookup l wanted = Labeled l (prune subtree (S.toList path_tails)) - go (Fork t1 t2) = fork (go t1) (go t2) - go tree = Pruned (reconstruct tree) - - -wellFormed :: HashTree -> Either String () -wellFormed (Leaf _) = return () -wellFormed tree = wellFormedForest $ flatten tree - -wellFormedForest :: [HashTree] -> Either String () -wellFormedForest trees = do - isInOrder [ l | Labeled l _ <- trees ] - sequence_ [ wellFormed t | Labeled _ t <- trees ] - sequence_ [ Left "Value in forest" | Leaf _ <- trees ] - -isInOrder :: [Label] -> Either String () -isInOrder [] = return () -isInOrder [_] = return () -isInOrder (x:y:zs) - | x < y = isInOrder (y:zs) - | otherwise = Left $ "Tree values out of order: " ++ show x ++ " " ++ show y diff --git a/impl/src/IC/HashTree/CBOR.hs b/impl/src/IC/HashTree/CBOR.hs deleted file mode 100644 index a31317449..000000000 --- a/impl/src/IC/HashTree/CBOR.hs +++ /dev/null @@ -1,27 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -module IC.HashTree.CBOR where - -import Codec.CBOR.Term -import qualified Data.Text as T - -import IC.CBOR.Patterns -import IC.HashTree - -encodeHashTree :: HashTree -> Term -encodeHashTree = go - where - go EmptyTree = TList [ TInteger 0 ] - go (Fork t1 t2) = TList [ TInteger 1, go t1, go t2 ] - go (Labeled l t) = TList [ TInteger 2, TBlob l, go t ] - go (Leaf v) = TList [ TInteger 3, TBlob v ] - go (Pruned h) = TList [ TInteger 4, TBlob h ] - -parseHashTree :: Term -> Either T.Text HashTree -parseHashTree = go - where - go (TList_ [ TNat 0 ]) = return EmptyTree - go (TList_ [ TNat 1, t1, t2 ]) = Fork <$> parseHashTree t1 <*> parseHashTree t2 - go (TList_ [ TNat 2, TBlob l, t ]) = Labeled l <$> parseHashTree t - go (TList_ [ TNat 3, TBlob v ]) = return $ Leaf v - go (TList_ [ TNat 4, TBlob h ]) = return $ Pruned h - go t = Left $ "Cannot parse as a Hash Tree: " <> T.pack (show t) diff --git a/impl/src/IC/Id/Forms.hs b/impl/src/IC/Id/Forms.hs deleted file mode 100644 index 78acb9b69..000000000 --- a/impl/src/IC/Id/Forms.hs +++ /dev/null @@ -1,42 +0,0 @@ -{- | -Implements the special forms of ids (https://docs.dfinity.systems/public/#id-classes) --} -{-# LANGUAGE TypeApplications #-} -{-# LANGUAGE OverloadedStrings #-} -module IC.Id.Forms where - -import qualified Data.ByteString.Lazy as BS -import IC.Hash - -type Blob = BS.ByteString - -mkOpaqueId :: Blob -> Blob -mkOpaqueId b = - b <> BS.singleton 1 - -isOpaqueId :: Blob -> Bool -isOpaqueId b = BS.drop (BS.length b - 1) b == BS.singleton 1 - -mkSelfAuthenticatingId :: Blob -> Blob -mkSelfAuthenticatingId pubkey = - sha224 pubkey <> BS.singleton 2 - -isSelfAuthenticatingId :: Blob -> Blob -> Bool -isSelfAuthenticatingId pubkey id = - mkSelfAuthenticatingId pubkey == id - -mkDerivedId :: Blob -> Blob -> Blob -mkDerivedId registering bytes = - sha224 (len_prefixed registering <> bytes) <> BS.singleton 3 - -isDerivedId :: Blob -> Blob -> Bool -isDerivedId registering blob = - BS.length blob == 256`div`8 + 8 + 1 && - BS.last blob == 3 && - BS.take (256`div`8) blob == sha224 registering - -isAnonymousId :: Blob -> Bool -isAnonymousId blob = blob == "\x04" - -len_prefixed :: BS.ByteString -> BS.ByteString -len_prefixed s = BS.singleton (fromIntegral (BS.length s)) <> s diff --git a/impl/src/IC/Id/Fresh.hs b/impl/src/IC/Id/Fresh.hs deleted file mode 100644 index 7ae79f5c1..000000000 --- a/impl/src/IC/Id/Fresh.hs +++ /dev/null @@ -1,15 +0,0 @@ -module IC.Id.Fresh where - -import IC.Types -import IC.Id.Forms - -import Data.ByteString.Builder -import Data.Word - --- Not particulary efficent, but this is a reference implementation, right? -freshId :: [EntityId] -> EntityId -freshId ids = - head $ - filter (`notElem` ids) $ - map (EntityId . mkOpaqueId . toLazyByteString . word64LE) - [1024::Word64 ..] diff --git a/impl/src/IC/Management.hs b/impl/src/IC/Management.hs deleted file mode 100644 index 310ae1ec0..000000000 --- a/impl/src/IC/Management.hs +++ /dev/null @@ -1,45 +0,0 @@ -{- -Plumbing related to Candid and the management canister. --} -{-# LANGUAGE TypeFamilies #-} -{-# LANGUAGE TypeApplications #-} -{-# LANGUAGE DataKinds #-} -{-# LANGUAGE OverloadedLabels #-} -{-# LANGUAGE TypeOperators #-} -{-# LANGUAGE UndecidableInstances #-} -{-# LANGUAGE QuasiQuotes #-} -{-# OPTIONS_GHC -Wno-orphans #-} -module IC.Management where - -import Codec.Candid -import IC.Types -import qualified Data.Row.Internal as R - --- This needs cleaning up -principalToEntityId :: Principal -> EntityId -principalToEntityId = EntityId . rawPrincipal - -entityIdToPrincipal :: EntityId -> Principal -entityIdToPrincipal = Principal . rawEntityId - -type InstallMode = [candidType| - variant {install : null; reinstall : null; upgrade : null} - |] - -type RunState = [candidType| - variant { running; stopping; stopped } - |] - -type Settings = [candidType| - record { - controllers : opt vec principal; - compute_allocation : opt nat; - memory_allocation : opt nat; - freezing_threshold : opt nat; - } - |] - -type ICManagement m = [candidFile|ic.did|] - -managementMethods :: [String] -managementMethods = R.labels @(ICManagement IO) @R.Unconstrained1 diff --git a/impl/src/IC/Purify.hs b/impl/src/IC/Purify.hs deleted file mode 100644 index 1789b61e9..000000000 --- a/impl/src/IC/Purify.hs +++ /dev/null @@ -1,60 +0,0 @@ -{-# LANGUAGE RankNTypes #-} -{-# LANGUAGE TypeFamilies #-} -{-# LANGUAGE LambdaCase #-} -{-# LANGUAGE MultiParamTypeClasses #-} -{-# LANGUAGE FlexibleInstances #-} -module IC.Purify where - -import Control.Monad.ST -import Data.Functor -import Data.Either -import Data.Bifunctor - -class SnapshotAble i where - type SnapshotOf i :: * - persist :: i s -> ST s (SnapshotOf i) - recreate :: SnapshotOf i -> ST s (i s) - -class Purify i a where - create :: (forall s. ST s (i s)) -> a - createMaybe :: (forall s. ST s (b, Either c (i s))) -> (b, Either c a) - perform :: (forall s. i s -> ST s b) -> a -> (a, b) - -newtype Snapshot a = Snapshot a - deriving (Show) - -instance (SnapshotAble i, SnapshotOf i ~ a) => Purify i (Snapshot a) where - create act = Snapshot $ runST $ act >>= persist - - createMaybe act = runST $ do - act >>= \case - (x, Left e) -> return (x, Left e) - (x, Right i) -> do - s' <- persist i - return (x, Right (Snapshot s')) - - perform act (Snapshot s) = runST $ do - i <- recreate s - x <- act i - s' <- persist i - return (Snapshot s', x) - - -newtype Replay i = Replay (forall s. ST s (i s)) - -instance Show (Replay i) where show _ = "Replay …" - -instance Purify a (Replay a) where - create = Replay - - createMaybe act = runST $ second ($> replay') <$> act - where - replay' = Replay $ fromRight err . snd <$> act - err = error "createMaybe: ST action was not deterministic?" - - perform act (Replay replay) = runST $ do - x <- replay >>= act - return (replay', x) - where - replay' = Replay $ do x <- replay; void (act x); return x - diff --git a/impl/src/IC/Ref.hs b/impl/src/IC/Ref.hs deleted file mode 100644 index b723b5a29..000000000 --- a/impl/src/IC/Ref.hs +++ /dev/null @@ -1,1192 +0,0 @@ -{-# LANGUAGE TypeOperators #-} -{-# LANGUAGE TypeApplications #-} -{-# LANGUAGE ConstraintKinds #-} -{-# LANGUAGE FlexibleContexts #-} -{-# LANGUAGE LambdaCase #-} -{-# LANGUAGE NamedFieldPuns #-} -{-# LANGUAGE TypeFamilies #-} -{-# LANGUAGE ScopedTypeVariables #-} -{-# LANGUAGE RankNTypes #-} -{-# LANGUAGE OverloadedLabels #-} -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE DataKinds #-} -{-# LANGUAGE AllowAmbiguousTypes #-} -{-# LANGUAGE MultiWayIf #-} - -{-| -This module implements the main abstract logic of the Internet Computer. It -assumes a pure and abstracted view on Canisters (provided by "IC.Canister"), -and deals with abstract requests ('CallRequest', 'QueryRequest', ...), so HTTP -and CBOR-level processing has already happened. --} -module IC.Ref - ( IC(..) - , CallRequest(..) - , callerOfCallRequest - , QueryRequest(..) - , ReadStateRequest(..) - , RequestStatus(..) - , ReqResponse(..) - , CallResponse(..) - , initialIC - , authCallRequest - , authQueryRequest - , authReadStateRequest - , submitRequest - , handleQuery - , handleReadState - , runStep - , runToCompletion - , setAllTimesTo - -- $ Exported merely for debug introspection - , CallContext(..) - , Message(..) - , CanState(..) - , CallOrigin(..) - , EntryPoint(..) - , RunStatus(..) - , CanisterContent(..) - ) -where - -import qualified Data.Map as M -import qualified Data.Row as R -import qualified Data.Row.Variants as V -import qualified Data.ByteString.Lazy as BS -import qualified Data.Text as T -import qualified Data.Vector as Vec -import qualified Data.Set as S -import Data.List -import Data.Maybe -import Numeric.Natural -import Data.Functor -import Control.Monad.State.Class -import Control.Monad.Except -import Control.Monad.Random.Lazy -import Data.Sequence (Seq(..)) -import Data.Foldable (toList) -import Codec.Candid -import Data.Row ((.==), (.+), (.!), type (.!)) -import GHC.Stack - -import IC.Types -import IC.Constants -import IC.Canister -import IC.CBOR.Utils -import IC.Id.Fresh -import IC.Utils -import IC.Management -import IC.HashTree hiding (Blob) -import IC.Certificate -import IC.Certificate.Value -import IC.Certificate.CBOR -import IC.Crypto - --- Abstract HTTP Interface - -data CallRequest - = CallRequest CanisterId UserId MethodName Blob - deriving (Eq, Ord, Show) - -data QueryRequest = QueryRequest CanisterId UserId MethodName Blob -data ReadStateRequest = ReadStateRequest UserId [Path] - -data RequestStatus - = Received - | Processing - | CallResponse CallResponse - deriving (Show) - -data CallResponse - = Rejected (RejectCode, String) - | Replied Blob - deriving (Show) - -data ReqResponse - = QueryResponse CallResponse - | ReadStateResponse Certificate - deriving (Show) - --- IC state - --- The canister state - -data RunStatus - = IsRunning - | IsStopping [CallId] - | IsStopped - | IsDeleted -- not actually a run state, but convenient in this code - deriving (Show) - -data CanisterContent = CanisterContent - { can_mod :: CanisterModule - , wasm_state :: WasmState - } - deriving (Show) - -data CanState = CanState - { content :: Maybe CanisterContent -- absent when empty - , run_status :: RunStatus - , controllers :: S.Set EntityId - , memory_allocation :: Natural - , compute_allocation :: Natural - , freezing_threshold :: Natural - , time :: Timestamp - , cycle_balance :: Natural - , certified_data :: Blob - } - deriving (Show) - --- A canister entry point is either a publicly named function, or a closure --- (callback + environment) -data EntryPoint - = Public MethodName Blob - | Closure Callback Response Cycles - deriving (Show) - -type CallId = Int -data CallContext = CallContext - { canister :: CanisterId - , origin :: CallOrigin - , responded :: Responded - , deleted :: Bool - , available_cycles :: Cycles - , last_trap :: Maybe String - -- ^ non-normative, but yields better reject messages - } - deriving (Show) - -data CallOrigin - = FromUser RequestID - | FromCanister CallId Callback - deriving (Show) - -data Message - = CallMessage - { call_context :: CallId - , entry :: EntryPoint - } - | ResponseMessage - { call_context :: CallId - , response :: Response - , refunded_cycles :: Cycles - } - deriving (Show) - --- Finally, the full IC state: - -data IC = IC - { canisters :: CanisterId ↦ CanState - , requests :: RequestID ↦ (CallRequest, RequestStatus) - , messages :: Seq Message - , call_contexts :: CallId ↦ CallContext - , rng :: StdGen - , secretRootKey :: SecretKey - , secretSubnetKey :: SecretKey - } - deriving (Show) - --- The functions below want stateful access to a value of type 'IC' -type ICM m = (MonadState IC m, HasCallStack) - -initialIC :: IO IC -initialIC = do - let sk1 = createSecretKeyBLS "ic-ref's very secure secret key" - let sk2 = createSecretKeyBLS "ic-ref's very secure subnet key" - IC mempty mempty mempty mempty <$> newStdGen <*> pure sk1 <*> pure sk2 - --- Request handling - -findRequest :: RequestID -> IC -> Maybe (CallRequest, RequestStatus) -findRequest rid ic = M.lookup rid (requests ic) - -setReqStatus :: ICM m => RequestID -> RequestStatus -> m () -setReqStatus rid s = modify $ \ic -> - ic { requests = M.adjust (\(r,_) -> (r,s)) rid (requests ic) } - -calleeOfCallRequest :: CallRequest -> EntityId -calleeOfCallRequest = \case - CallRequest canister_id _ _ _ -> canister_id - -callerOfCallRequest :: CallRequest -> EntityId -callerOfCallRequest = \case - CallRequest _ user_id _ _ -> user_id - -callerOfRequest :: ICM m => RequestID -> m EntityId -callerOfRequest rid = gets (M.lookup rid . requests) >>= \case - Just (ar,_) -> return (callerOfCallRequest ar) - Nothing -> error "callerOfRequest" - - --- Canister handling - -createEmptyCanister :: ICM m => CanisterId -> S.Set EntityId -> Timestamp -> m () -createEmptyCanister cid controllers time = modify $ \ic -> - ic { canisters = M.insert cid can (canisters ic) } - where - can = CanState - { content = Nothing - , run_status = IsRunning - , controllers = controllers - , memory_allocation = 0 - , compute_allocation = 0 - , freezing_threshold = 2592000 - , time = time - , cycle_balance = 0 - , certified_data = "" - } - -canisterMustExist :: (CanReject m, ICM m) => CanisterId -> m () -canisterMustExist cid = - gets (M.lookup cid . canisters) >>= \case - Nothing -> - reject RC_DESTINATION_INVALID ("canister does not exist: " ++ prettyID cid) - Just CanState{ run_status = IsDeleted } -> - reject RC_DESTINATION_INVALID ("canister no longer exists: " ++ prettyID cid) - _ -> return () - -isCanisterEmpty :: ICM m => CanisterId -> m Bool -isCanisterEmpty cid = isNothing . content <$> getCanister cid - - --- the following functions assume the canister does exist; --- it would be an internal error if they dont - -getCanister :: ICM m => CanisterId -> m CanState -getCanister cid = - gets (M.lookup cid . canisters) - `orElse` error ("canister does not exist: " ++ prettyID cid) - -modCanister :: ICM m => CanisterId -> (CanState -> CanState) -> m () -modCanister cid f = do - void $ getCanister cid - modify $ \ic -> ic { canisters = M.adjust f cid (canisters ic) } - -setCanisterContent :: ICM m => CanisterId -> CanisterContent -> m () -setCanisterContent cid content = modCanister cid $ - \cs -> cs { content = Just content } - -modCanisterContent :: ICM m => CanisterId -> (CanisterContent -> CanisterContent) -> m () -modCanisterContent cid f = do - modCanister cid $ \c -> c { content = Just (f (fromMaybe err (content c))) } - where err = error ("canister is empty: " ++ prettyID cid) - -setCanisterState :: ICM m => CanisterId -> WasmState -> m () -setCanisterState cid wasm_state = modCanisterContent cid $ - \cs -> cs { wasm_state = wasm_state } - -getControllers :: ICM m => CanisterId -> m (S.Set EntityId) -getControllers cid = controllers <$> getCanister cid - -setControllers :: ICM m => CanisterId -> (S.Set EntityId) -> m () -setControllers cid controllers = modCanister cid $ - \cs -> cs { controllers = controllers } - -setComputeAllocation :: ICM m => CanisterId -> Natural -> m () -setComputeAllocation cid n = modCanister cid $ - \cs -> cs { compute_allocation = n } - -setMemoryAllocation :: ICM m => CanisterId -> Natural -> m () -setMemoryAllocation cid n = modCanister cid $ - \cs -> cs { memory_allocation = n } - -setFreezingThreshold :: ICM m => CanisterId -> Natural -> m () -setFreezingThreshold cid n = modCanister cid $ - \cs -> cs { freezing_threshold = n } - -getBalance :: ICM m => CanisterId -> m Natural -getBalance cid = cycle_balance <$> getCanister cid - -setBalance :: ICM m => CanisterId -> Natural -> m () -setBalance cid balance = modCanister cid $ - \cs -> cs { cycle_balance = min cMAX_CANISTER_BALANCE balance } - -setCertifiedData :: ICM m => CanisterId -> Blob -> m () -setCertifiedData cid b = modCanister cid $ - \cs -> cs { certified_data = b } - -getRunStatus :: ICM m => CanisterId -> m RunStatus -getRunStatus cid = run_status <$> getCanister cid - -setRunStatus :: ICM m => CanisterId -> RunStatus -> m () -setRunStatus cid run_status = modCanister cid $ - \cs -> cs { run_status = run_status } - -getCanisterState :: ICM m => CanisterId -> m WasmState -getCanisterState cid = wasm_state . fromJust . content <$> getCanister cid - -getCanisterMod :: ICM m => CanisterId -> m CanisterModule -getCanisterMod cid = can_mod . fromJust . content <$> getCanister cid - -getCanisterTime :: ICM m => CanisterId -> m Timestamp -getCanisterTime cid = time <$> getCanister cid - - -module_hash :: CanState -> Maybe Blob -module_hash = fmap (raw_wasm_hash . can_mod) . content - --- Authentication and authorization of requests --- --- The envelope has already been validated. So this includes --- * Comparing the envelope validity with the contents --- * Authorization of the sender --- * ingress message inspection --- * checking the correct effective id - -type RequestValidation m = (MonadError T.Text m, ICM m) - -authCallRequest :: RequestValidation m => Timestamp -> CanisterId -> EnvValidity -> CallRequest -> m () -authCallRequest t ecid ev r@(CallRequest canister_id user_id meth arg) = do - checkEffectiveCanisterID ecid canister_id meth arg - valid_when ev t - valid_for ev user_id - valid_where ev canister_id - inspectIngress r - -authQueryRequest :: RequestValidation m => Timestamp -> CanisterId -> EnvValidity -> QueryRequest -> m () -authQueryRequest t ecid ev (QueryRequest canister_id user_id meth arg) = do - checkEffectiveCanisterID ecid canister_id meth arg - valid_when ev t - valid_for ev user_id - valid_where ev canister_id - -authReadStateRequest :: RequestValidation m => Timestamp -> CanisterId -> EnvValidity -> ReadStateRequest -> m () -authReadStateRequest t ecid ev (ReadStateRequest user_id paths) = do - valid_when ev t - valid_for ev user_id - -- Implement ACL for read requests here - forM_ paths $ \case - ["time"] -> return () - ("subnet":_) -> return () - ("canister":cid:"module_hash":_) -> - assertEffectiveCanisterId ecid (EntityId cid) - ("canister":cid:"controllers":_) -> - assertEffectiveCanisterId ecid (EntityId cid) - ("request_status" :rid: _) -> - gets (findRequest rid) >>= \case - Just (ar@(CallRequest cid _ meth arg),_) -> do - checkEffectiveCanisterID ecid cid meth arg - unless (user_id == callerOfCallRequest ar) $ - throwError "User is not authorized to read this request status" - valid_where ev (calleeOfCallRequest ar) - Nothing -> return () - _ -> throwError "User is not authorized to read unspecified state paths" - -canisterEnv :: ICM m => CanisterId -> m Env -canisterEnv canister_id = do - env_time <- getCanisterTime canister_id - env_balance <- getBalance canister_id - env_status <- getRunStatus canister_id <&> \case - IsRunning -> Running - IsStopping _pending -> Stopping - IsStopped -> Stopped - IsDeleted -> error "deleted canister encountered" - return $ Env - { env_self = canister_id - , env_time - , env_balance - , env_status - , env_certificate = Nothing - } - --- Synchronous requests - -handleQuery :: ICM m => Timestamp -> QueryRequest -> m ReqResponse -handleQuery time (QueryRequest canister_id user_id method arg) = - fmap QueryResponse $ onReject (return . Rejected) $ do - canisterMustExist canister_id - getRunStatus canister_id >>= \case - IsRunning -> return () - _ -> reject RC_CANISTER_ERROR "canister is stopped" - empty <- isCanisterEmpty canister_id - when empty $ reject RC_DESTINATION_INVALID "canister is empty" - wasm_state <- getCanisterState canister_id - can_mod <- getCanisterMod canister_id - certificate <- getDataCertificate time canister_id - env0 <- canisterEnv canister_id - let env = env0 { env_certificate = Just certificate } - - f <- return (M.lookup method (query_methods can_mod)) - `orElse` reject RC_DESTINATION_INVALID "query method does not exist" - - case f user_id env arg wasm_state of - Trap msg -> reject RC_CANISTER_ERROR $ "canister trapped: " ++ msg - Return (Reject (rc,rm)) -> reject rc rm - Return (Reply res) -> return $ Replied res - -handleReadState :: ICM m => Timestamp -> ReadStateRequest -> m ReqResponse -handleReadState time (ReadStateRequest _sender paths) = do - -- NB: Already authorized in authSyncRequest - cert <- getPrunedCertificate time (["time"] : paths) - return $ ReadStateResponse cert - -checkEffectiveCanisterID :: RequestValidation m => CanisterId -> CanisterId -> MethodName -> Blob -> m () -checkEffectiveCanisterID ecid cid method arg - | cid == managementCanisterId = case method of - "provisional_create_canister_with_cycles" -> pure () - "raw_rand" -> throwError "raw_rand() cannot be invoked via ingress calls" - _ -> case Codec.Candid.decode @(R.Rec ("canister_id" R..== Principal)) arg of - Left err -> - throwError $ "call to management canister is not valid candid: " <> T.pack err - Right r -> - assertEffectiveCanisterId ecid (principalToEntityId (r .! #canister_id)) - | otherwise = assertEffectiveCanisterId ecid cid - -assertEffectiveCanisterId :: RequestValidation m => CanisterId -> CanisterId -> m () -assertEffectiveCanisterId ecid cid = do - unless (ecid == cid) $ do - throwError $ "expected effective canister_id " <> T.pack (prettyID cid) <> ", got " <> T.pack (prettyID ecid) - -inspectIngress :: RequestValidation m => CallRequest -> m () -inspectIngress (CallRequest canister_id user_id method arg) - | canister_id == managementCanisterId = - if| method `elem` ["provisional_create_canister_with_cycles", "provisional_top_up_canister"] - -> return () - | method `elem` [ "raw_rand", "deposit_cycles" ] - -> throwError $ "Management method " <> T.pack method <> " cannot be invoked via an ingress call" - | method `elem` managementMethods - -> case decode @(R.Rec ("canister_id" R..== Principal)) arg of - Left msg -> throwError $ "Candid failed to decode: " <> T.pack msg - Right r -> do - let canister_id = principalToEntityId $ r .! #canister_id - onReject (throwError . T.pack . snd) $ - canisterMustExist canister_id - controllers <- getControllers canister_id - unless (user_id `S.member` controllers) $ - throwError "Wrong sender" - | otherwise - -> throwError $ "Unknown management method " <> T.pack method - | otherwise = do - onReject (throwError . T.pack . snd) $ - canisterMustExist canister_id - getRunStatus canister_id >>= \case - IsRunning -> return () - _ -> throwError "canister is stopped" - empty <- isCanisterEmpty canister_id - when empty $ throwError "canister is empty" - wasm_state <- getCanisterState canister_id - can_mod <- getCanisterMod canister_id - env <- canisterEnv canister_id - - case inspect_message can_mod method user_id env arg wasm_state of - Trap msg -> throwError $ "canister trapped in inspect_message: " <> T.pack msg - Return () -> return () - --- The state tree - -stateTree :: Timestamp -> IC -> LabeledTree -stateTree (Timestamp t) ic = node - [ "time" =: val t - , "request_status" =: node - [ rid =: case rs of - Received -> node - [ "status" =: str "received" ] - Processing -> node - [ "status" =: str "processing" ] - CallResponse (Replied r) -> node - [ "status" =: str "replied" - , "reply" =: val r - ] - CallResponse (Rejected (c,msg)) -> node - [ "status" =: str "rejected" - , "reject_code" =: val (rejectCode c) - , "reject_message" =: val (T.pack msg) - ] - | (rid, (_, rs)) <- M.toList (requests ic) - ] - , "canister" =: node - [ cid =: node ( - [ "certified_data" =: val (certified_data cs) - , "controllers" =: val (encodePrincipalList (S.toList (controllers cs))) - ] ++ - [ "module_hash" =: val h | Just h <- pure $ module_hash cs ] - ) - | (EntityId cid, cs) <- M.toList (canisters ic) - ] - ] - where - node = SubTrees . mconcat - val :: CertVal a => a -> LabeledTree - val = Value . toCertVal - str = val @T.Text - (=:) = M.singleton - -delegationTree :: Timestamp -> SubnetId -> Blob -> LabeledTree -delegationTree (Timestamp t) (EntityId subnet_id) subnet_pub_key = node - [ "time" =: val t - , "subnet" =: node - [ subnet_id =: node - [ "public_key" =: val subnet_pub_key ] - ] - ] - where - node = SubTrees . mconcat - val :: CertVal a => a -> LabeledTree - val = Value . toCertVal - (=:) = M.singleton - -getPrunedCertificate :: ICM m => Timestamp -> [Path] -> m Certificate -getPrunedCertificate time paths = do - full_tree <- gets (construct . stateTree time) - let cert_tree = prune full_tree (["time"] : paths) - sk1 <- gets secretRootKey - sk2 <- gets secretSubnetKey - return $ signCertificate time sk1 (Just (fake_subnet_id, sk2)) cert_tree - where - fake_subnet_id = EntityId "\x01" - -signCertificate :: Timestamp -> SecretKey -> Maybe (SubnetId, SecretKey) -> HashTree -> Certificate -signCertificate time rootKey (Just (subnet_id, subnet_key)) cert_tree = - Certificate { cert_tree, cert_sig, cert_delegation } - where - cert_sig = signPure "ic-state-root" subnet_key (reconstruct cert_tree) - cert_delegation = Just $ Delegation { del_subnet_id, del_certificate } - del_subnet_id = rawEntityId subnet_id - del_certificate = - encodeCert $ - signCertificate time rootKey Nothing $ - construct $ - delegationTree time subnet_id (toPublicKey subnet_key) - -signCertificate _time rootKey Nothing cert_tree = - Certificate { cert_tree, cert_sig, cert_delegation = Nothing } - where - cert_sig = signPure "ic-state-root" rootKey (reconstruct cert_tree) - --- If `stateTree` ever becomes a bottleneck: --- Since ic-ref creates a fresh state tree everytime it is used, we _could_ --- construct one with just the required data, e.g. only of the canister in --- question. That would not be secure, but `ic-ref` doesn’t have to be. -getDataCertificate :: ICM m => Timestamp -> CanisterId -> m Blob -getDataCertificate t cid = do - encodeCert <$> getPrunedCertificate t - [["time"], ["canister", rawEntityId cid, "certified_data"]] - --- Asynchronous requests - --- | Submission simply enqueues requests - -submitRequest :: ICM m => RequestID -> CallRequest -> m () -submitRequest rid r = modify $ \ic -> - if M.member rid (requests ic) - then ic - else ic { requests = M.insert rid (r, Received) (requests ic) } - - --- | Eventually, they are processed - -processRequest :: ICM m => (RequestID, CallRequest) -> m () -processRequest (rid, req) = onReject (setReqStatus rid . CallResponse . Rejected) $ - case req of - CallRequest canister_id _user_id method arg -> do - ctxt_id <- newCallContext $ CallContext - { canister = canister_id - , origin = FromUser rid - , responded = Responded False - , deleted = False - , last_trap = Nothing - , available_cycles = 0 - } - enqueueMessage $ CallMessage - { call_context = ctxt_id - , entry = Public method arg - } - setReqStatus rid Processing - --- Call context handling - -newCallContext :: ICM m => CallContext -> m CallId -newCallContext cc = state $ \ic -> - let i = freshKey (call_contexts ic) - in (i, ic { call_contexts = M.insert i cc (call_contexts ic)}) - -getCallContext :: ICM m => CallId -> m CallContext -getCallContext ctxt_id = gets ((M.! ctxt_id) . call_contexts) - -modifyCallContext :: ICM m => CallId -> (CallContext -> CallContext) -> m () -modifyCallContext ctxt_id f = modify $ \ic -> - ic { call_contexts = M.adjust f ctxt_id (call_contexts ic) } - -getCallContextCycles :: ICM m => CallId -> m Cycles -getCallContextCycles ctxt_id = available_cycles <$> getCallContext ctxt_id - -setCallContextCycles :: ICM m => CallId -> Cycles -> m () -setCallContextCycles ctxt_id cycles = modifyCallContext ctxt_id $ \ctxt -> - ctxt { available_cycles = cycles } - -respondCallContext :: ICM m => CallId -> Response -> m () -respondCallContext ctxt_id response = do - ctxt <- getCallContext ctxt_id - when (deleted ctxt) $ - error "Internal error: response to deleted call context" - when (responded ctxt == Responded True) $ - error "Internal error: Double response" - modifyCallContext ctxt_id $ \ctxt -> ctxt - { responded = Responded True - , available_cycles = 0 - } - enqueueMessage $ ResponseMessage { - call_context = ctxt_id, - response, - refunded_cycles = available_cycles ctxt - } - -replyCallContext :: ICM m => CallId -> Blob -> m () -replyCallContext ctxt_id blob = - respondCallContext ctxt_id (Reply blob) - -rejectCallContext :: ICM m => CallId -> (RejectCode, String) -> m () -rejectCallContext ctxt_id r = - respondCallContext ctxt_id (Reject r) - -deleteCallContext :: ICM m => CallId -> m () -deleteCallContext ctxt_id = - modifyCallContext ctxt_id $ \ctxt -> - if responded ctxt == Responded False - then error "Internal error: deleteCallContext on non-responded call context" - else if deleted ctxt - then error "Internal error: deleteCallContext on deleted call context" - else ctxt { deleted = True } - -rememberTrap :: ICM m => CallId -> String -> m () -rememberTrap ctxt_id msg = - modifyCallContext ctxt_id $ \ctxt -> ctxt { last_trap = Just msg } - -callerOfCallID :: ICM m => CallId -> m EntityId -callerOfCallID ctxt_id = do - ctxt <- getCallContext ctxt_id - case origin ctxt of - FromUser rid -> callerOfRequest rid - FromCanister other_ctxt_id _callback -> calleeOfCallID other_ctxt_id - -calleeOfCallID :: ICM m => CallId -> m EntityId -calleeOfCallID ctxt_id = canister <$> getCallContext ctxt_id - -respondedCallID :: ICM m => CallId -> m Responded -respondedCallID ctxt_id = responded <$> getCallContext ctxt_id - -deletedCallID :: ICM m => CallId -> m Bool -deletedCallID ctxt_id = deleted <$> getCallContext ctxt_id - -starveCallContext :: ICM m => CallId -> m () -starveCallContext ctxt_id = do - ctxt <- getCallContext ctxt_id - let msg | Just t <- last_trap ctxt = "canister trapped: " ++ t - | otherwise = "canister did not respond" - rejectCallContext ctxt_id (RC_CANISTER_ERROR, msg) - --- Message handling - -enqueueMessage :: ICM m => Message -> m () -enqueueMessage m = modify $ \ic -> ic { messages = messages ic :|> m } - -processMessage :: ICM m => Message -> m () -processMessage m = case m of - CallMessage ctxt_id entry -> onReject (rejectCallContext ctxt_id) $ do - callee <- calleeOfCallID ctxt_id - if callee == managementCanisterId - then do - caller <- callerOfCallID ctxt_id - rejectAsCanister $ - invokeManagementCanister caller ctxt_id entry - else do - canisterMustExist callee - getRunStatus callee >>= \case - IsRunning -> return () - _ -> reject RC_CANISTER_ERROR "canister is stopped" - empty <- isCanisterEmpty callee - when empty $ reject RC_DESTINATION_INVALID "canister is empty" - wasm_state <- getCanisterState callee - can_mod <- getCanisterMod callee - env <- canisterEnv callee - invokeEntry ctxt_id wasm_state can_mod env entry >>= \case - Trap msg -> do - -- Eventually update cycle balance here - rememberTrap ctxt_id msg - Return (new_state, (call_actions, canister_actions)) -> do - performCallActions ctxt_id call_actions - performCanisterActions callee canister_actions - setCanisterState callee new_state - - ResponseMessage ctxt_id response refunded_cycles -> do - ctxt <- getCallContext ctxt_id - case origin ctxt of - FromUser rid -> setReqStatus rid $ CallResponse $ - -- NB: Here cycles disappear - case response of - Reject (rc, msg) -> Rejected (rc, msg) - Reply blob -> Replied blob - FromCanister other_ctxt_id callback -> do - -- Add refund to balance - cid <- calleeOfCallID other_ctxt_id - prev_balance <- getBalance cid - setBalance cid $ prev_balance + refunded_cycles - -- Unless deleted - d <- deletedCallID other_ctxt_id - unless d $ - -- Enqueue execution - enqueueMessage $ CallMessage - { call_context = other_ctxt_id - , entry = Closure callback response refunded_cycles - } - -performCallActions :: ICM m => CallId -> CallActions -> m () -performCallActions ctxt_id ca = do - updateBalances ctxt_id (ca_new_calls ca) (ca_accept ca) - mapM_ (newCall ctxt_id) (ca_new_calls ca) - mapM_ (respondCallContext ctxt_id) (ca_response ca) - - -performCanisterActions :: ICM m => CanisterId -> CanisterActions -> m () -performCanisterActions cid ca = do - mapM_ (setCertifiedData cid) (set_certified_data ca) - -updateBalances :: ICM m => CallId -> [MethodCall] -> Cycles -> m () -updateBalances ctxt_id new_calls accepted = do - cid <- calleeOfCallID ctxt_id - - -- Eventually update when we track cycle consumption - let max_cycles = 0 - let cycles_consumed = 0 - - prev_balance <- getBalance cid - available <- getCallContextCycles ctxt_id - if accepted <= available - then do - let to_spend = prev_balance + accepted - max_cycles - let transferred = sum [ call_transferred_cycles c | c <- new_calls] - if transferred <= to_spend - then do - setBalance cid $ prev_balance - + accepted - - cycles_consumed - - transferred - setCallContextCycles ctxt_id $ available - accepted - else error "Internal error: More cycles transferred than available" - else error "Internal error: More cycles accepted than available" - - -managementCanisterId :: EntityId -managementCanisterId = EntityId mempty - - -invokeManagementCanister :: - forall m. (CanReject m, ICM m) => EntityId -> CallId -> EntryPoint -> m () -invokeManagementCanister caller ctxt_id (Public method_name arg) = - case method_name of - "create_canister" -> atomic $ icCreateCanister caller ctxt_id - "install_code" -> atomic $ onlyController caller $ icInstallCode caller - "uninstall_code" -> atomic $ onlyController caller $ icUninstallCode - "update_settings" -> atomic $ onlyController caller icUpdateCanisterSettings - "start_canister" -> atomic $ onlyController caller icStartCanister - "stop_canister" -> deferred $ onlyController caller $ icStopCanister ctxt_id - "canister_status" -> atomic $ onlyController caller icCanisterStatus - "delete_canister" -> atomic $ onlyController caller icDeleteCanister - "deposit_cycles" -> atomic $ icDepositCycles ctxt_id - "provisional_create_canister_with_cycles" -> atomic $ icCreateCanisterWithCycles caller - "provisional_top_up_canister" -> atomic icTopUpCanister - "raw_rand" -> atomic icRawRand - _ -> reject RC_DESTINATION_INVALID $ "Unsupported management function " ++ method_name - where - -- always responds - atomic :: forall a b. (CandidArg a, CandidArg b) => (a -> m b) -> m () - atomic meth = wrap (\k x -> meth x >>= k) (replyCallContext ctxt_id) arg - - -- no implict reply - deferred :: forall a. CandidArg a => (a -> m ()) -> m () - deferred meth = wrap @a @() (\_k x -> meth x) (error "unused") arg - - wrap - :: forall a b. - (CandidArg a, CandidArg b) => - ((b -> m ()) -> a -> m ()) -> - ((Blob -> m ()) -> Blob -> m ()) - wrap method raw_reply blob = - case decode @a blob of - Left msg -> reject RC_CANISTER_ERROR $ "Candid failed to decode: " ++ msg - Right x -> method (raw_reply . encode @b) x - -invokeManagementCanister _ _ Closure{} = error "closure invoked on management function " - -icCreateCanister :: (ICM m, CanReject m) => EntityId -> CallId -> ICManagement m .! "create_canister" -icCreateCanister caller ctxt_id r = do - forM_ (r .! #settings) validateSettings - available <- getCallContextCycles ctxt_id - setCallContextCycles ctxt_id 0 - cid <- icCreateCanisterCommon caller available - forM_ (r .! #settings) $ applySettings cid - return (#canister_id .== entityIdToPrincipal cid) - -icCreateCanisterWithCycles :: (ICM m, CanReject m) => EntityId -> ICManagement m .! "provisional_create_canister_with_cycles" -icCreateCanisterWithCycles caller r = do - forM_ (r .! #settings) validateSettings - cid <- icCreateCanisterCommon caller (fromMaybe cMAX_CANISTER_BALANCE (r .! #amount)) - forM_ (r .! #settings) $ applySettings cid - return (#canister_id .== entityIdToPrincipal cid) - -icCreateCanisterCommon :: (ICM m, CanReject m) => EntityId -> Natural -> m EntityId -icCreateCanisterCommon controller amount = do - new_id <- gets (freshId . M.keys . canisters) - let currentTime = 0 -- ic-ref lives in the 70ies - createEmptyCanister new_id (S.singleton controller) currentTime - -- Here we fill up the canister with the cycles provided by the caller - setBalance new_id amount - return new_id - -validateSettings :: CanReject m => Settings -> m () -validateSettings r = do - forM_ (r .! #compute_allocation) $ \n -> do - unless (n <= 100) $ - reject RC_SYS_FATAL "Compute allocation not <= 100" - forM_ (r .! #memory_allocation) $ \n -> do - unless (n <= 2^(48::Int)) $ - reject RC_SYS_FATAL "Memory allocation not <= 2^48" - forM_ (r .! #freezing_threshold) $ \n -> do - unless (n < 2^(64::Int)) $ - reject RC_SYS_FATAL "Memory allocation not < 2^64" - forM_ (r .! #controllers) $ \n -> do - unless (length n <= 10) $ - reject RC_SYS_FATAL "Controllers cannot be > 10" - -applySettings :: ICM m => EntityId -> Settings -> m () -applySettings cid r = do - forM_ (r .! #controllers) $ setControllers cid . S.fromList . map principalToEntityId . Vec.toList - forM_ (r .! #compute_allocation) $ setComputeAllocation cid - forM_ (r .! #memory_allocation) $ setMemoryAllocation cid - forM_ (r .! #freezing_threshold) $ setFreezingThreshold cid - -onlyController :: - (ICM m, CanReject m, (r .! "canister_id") ~ Principal) => - EntityId -> (R.Rec r -> m a) -> (R.Rec r -> m a) -onlyController caller act r = do - let canister_id = principalToEntityId (r .! #canister_id) - canisterMustExist canister_id - controllers <- getControllers canister_id - if caller `S.member` controllers - then act r - else reject RC_SYS_FATAL $ - prettyID caller <> " is not authorized to manage canister " <> - prettyID canister_id <> ", Controllers are: " <> intercalate ", " (map prettyID (S.toList controllers)) - -icInstallCode :: (ICM m, CanReject m) => EntityId -> ICManagement m .! "install_code" -icInstallCode caller r = do - let canister_id = principalToEntityId (r .! #canister_id) - let arg = r .! #arg - new_can_mod <- return (parseCanister (r .! #wasm_module)) - `onErr` (\err -> reject RC_SYS_FATAL $ "Parsing failed: " ++ err) - was_empty <- isCanisterEmpty canister_id - env <- canisterEnv canister_id - - let - reinstall = do - (wasm_state, ca) <- return (init_method new_can_mod caller env arg) - `onTrap` (\msg -> reject RC_CANISTER_ERROR $ "Initialization trapped: " ++ msg) - setCanisterContent canister_id $ CanisterContent - { can_mod = new_can_mod - , wasm_state = wasm_state - } - performCanisterActions canister_id ca - - install = do - unless was_empty $ - reject RC_DESTINATION_INVALID "canister is not empty during installation" - reinstall - - upgrade = do - when was_empty $ - reject RC_DESTINATION_INVALID "canister is empty during upgrade" - old_wasm_state <- getCanisterState canister_id - old_can_mod <- getCanisterMod canister_id - (ca1, mem) <- return (pre_upgrade_method old_can_mod old_wasm_state caller env) - `onTrap` (\msg -> reject RC_CANISTER_ERROR $ "Pre-upgrade trapped: " ++ msg) - -- TODO: update balance in env based on ca1 here, once canister actions - -- can change balances - let env2 = env - (new_wasm_state, ca2) <- return (post_upgrade_method new_can_mod caller env2 mem arg) - `onTrap` (\msg -> reject RC_CANISTER_ERROR $ "Post-upgrade trapped: " ++ msg) - - setCanisterContent canister_id $ CanisterContent - { can_mod = new_can_mod - , wasm_state = new_wasm_state - } - performCanisterActions canister_id (ca1 <> ca2) - - R.switch (r .! #mode) $ R.empty - .+ #install .== (\() -> install) - .+ #reinstall .== (\() -> reinstall) - .+ #upgrade .== (\() -> upgrade) - -icUninstallCode :: (ICM m, CanReject m) => ICManagement m .! "uninstall_code" -icUninstallCode r = do - let canister_id = principalToEntityId (r .! #canister_id) - -- empty canister, resetting selected state - modCanister canister_id $ \can_state -> can_state - { content = Nothing - , certified_data = "" - } - -- reject all call open contexts of this canister - gets (M.toList . call_contexts) >>= mapM_ (\(ctxt_id, ctxt) -> - when (canister ctxt == canister_id && responded ctxt == Responded False) $ do - rejectCallContext ctxt_id (RC_CANISTER_REJECT, "Canister has been uninstalled") - deleteCallContext ctxt_id - ) - -icUpdateCanisterSettings :: (ICM m, CanReject m) => ICManagement m .! "update_settings" -icUpdateCanisterSettings r = do - let canister_id = principalToEntityId (r .! #canister_id) - validateSettings (r .! #settings) - applySettings canister_id (r .! #settings) - -icStartCanister :: (ICM m, CanReject m) => ICManagement m .! "start_canister" -icStartCanister r = do - let canister_id = principalToEntityId (r .! #canister_id) - getRunStatus canister_id >>= \case - IsRunning -> return () - IsStopping pending -> forM_ pending $ \ctxt_id -> - rejectCallContext ctxt_id (RC_CANISTER_ERROR, "Canister has been restarted") - IsStopped -> setRunStatus canister_id IsRunning - IsDeleted -> error "deleted canister encountered" - -icStopCanister :: - (ICM m, CanReject m) => - (a -> m b) ~ (ICManagement m .! "stop_canister") => - CallId -> a -> m () -icStopCanister ctxt_id r = do - let canister_id = principalToEntityId (r .! #canister_id) - getRunStatus canister_id >>= \case - IsRunning -> setRunStatus canister_id (IsStopping [ctxt_id]) - IsStopping pending -> setRunStatus canister_id (IsStopping (pending ++ [ctxt_id])) - IsStopped -> replyCallContext ctxt_id (encode ()) - IsDeleted -> error "deleted canister encountered" - -actuallyStopCanister :: ICM m => CanisterId -> m () -actuallyStopCanister canister_id = - getRunStatus canister_id >>= \case - IsStopping pending -> do - setRunStatus canister_id IsStopped - forM_ pending $ \ctxt_id -> - replyCallContext ctxt_id (Codec.Candid.encode ()) - IsRunning -> error "unexpected canister status" - IsStopped -> error "unexpected canister status" - IsDeleted -> error "deleted canister encountered" - -icCanisterStatus :: (ICM m, CanReject m) => ICManagement m .! "canister_status" -icCanisterStatus r = do - let canister_id = principalToEntityId (r .! #canister_id) - s <- getRunStatus canister_id >>= \case - IsRunning -> return (V.IsJust #running ()) - IsStopping _pending -> return (V.IsJust #stopping ()) - IsStopped -> return (V.IsJust #stopped ()) - IsDeleted -> error "deleted canister encountered" - can_state <- getCanister canister_id - hash <- module_hash <$> getCanister canister_id - cycles <- getBalance canister_id - return $ R.empty - .+ #status .== s - .+ #settings .== (R.empty - .+ #controllers .== Vec.fromList (map entityIdToPrincipal (S.toList (controllers can_state))) - .+ #memory_allocation .== memory_allocation can_state - .+ #compute_allocation .== compute_allocation can_state - .+ #freezing_threshold .== freezing_threshold can_state - ) - .+ #memory_size .== 0 -- not implemented here - .+ #module_hash .== hash - .+ #cycles .== cycles - - -icDeleteCanister :: (ICM m, CanReject m) => ICManagement m .! "delete_canister" -icDeleteCanister r = do - let canister_id = principalToEntityId (r .! #canister_id) - getRunStatus canister_id >>= \case - IsRunning -> reject RC_SYS_FATAL "Cannot delete running canister" - IsStopping _pending -> reject RC_SYS_FATAL "Cannot delete stopping canister" - IsStopped -> return () - IsDeleted -> error "deleted canister encountered" - - setRunStatus canister_id IsDeleted - -icDepositCycles :: (ICM m, CanReject m) => CallId -> ICManagement m .! "deposit_cycles" -icDepositCycles ctxt_id r = do - let canister_id = principalToEntityId (r .! #canister_id) - canisterMustExist canister_id - - cycles <- getCallContextCycles ctxt_id - available <- getCallContextCycles ctxt_id - setCallContextCycles ctxt_id (available - cycles) - prev_balance <- getBalance canister_id - setBalance canister_id $ prev_balance + cycles - -icTopUpCanister :: (ICM m, CanReject m) => ICManagement m .! "provisional_top_up_canister" -icTopUpCanister r = do - let canister_id = principalToEntityId (r .! #canister_id) - canisterMustExist canister_id - - prev_balance <- getBalance canister_id - setBalance canister_id $ prev_balance + (r .! #amount) - -icRawRand :: ICM m => ICManagement m .! "raw_rand" -icRawRand _r = runRandIC $ BS.pack <$> replicateM 32 getRandom - -runRandIC :: ICM m => Rand StdGen a -> m a -runRandIC a = state $ \ic -> - let (x, g) = runRand a (rng ic) - in (x, ic { rng = g }) - -invokeEntry :: ICM m => - CallId -> WasmState -> CanisterModule -> Env -> EntryPoint -> - m (TrapOr (WasmState, UpdateResult)) -invokeEntry ctxt_id wasm_state can_mod env entry = do - responded <- respondedCallID ctxt_id - available <- getCallContextCycles ctxt_id - case entry of - Public method dat -> do - caller <- callerOfCallID ctxt_id - case lookupUpdate method can_mod of - Just f -> return $ f caller env responded available dat wasm_state - Nothing -> do - let reject = Reject (RC_DESTINATION_INVALID, "method does not exist: " ++ method) - return $ Return (wasm_state, (noCallActions { ca_response = Just reject}, noCanisterActions)) - Closure cb r refund -> return $ do - case callbacks can_mod cb env responded available r refund wasm_state of - Trap err -> case cleanup_callback cb of - Just closure -> case cleanup can_mod closure env wasm_state of - Trap err' -> Trap err' - Return (wasm_state', ()) -> - Return (wasm_state', (noCallActions, noCanisterActions)) - Nothing -> Trap err - Return (wasm_state, actions) -> Return (wasm_state, actions) - where - lookupUpdate method can_mod - | Just f <- M.lookup method (update_methods can_mod) = Just f - | Just f <- M.lookup method (query_methods can_mod) = Just (asUpdate f) - | otherwise = Nothing - -newCall :: ICM m => CallId -> MethodCall -> m () -newCall from_ctxt_id call = do - new_ctxt_id <- newCallContext $ CallContext - { canister = call_callee call - , origin = FromCanister from_ctxt_id (call_callback call) - , responded = Responded False - , deleted = False - , last_trap = Nothing - , available_cycles = call_transferred_cycles call - } - enqueueMessage $ CallMessage - { call_context = new_ctxt_id - , entry = Public (call_method_name call) (call_arg call) - } - --- Scheduling - --- | Pick next request in state `received` -nextReceived :: ICM m => m (Maybe (RequestID, CallRequest)) -nextReceived = gets $ \ic -> listToMaybe - [ (rid,r) | (rid, (r, Received)) <- M.toList (requests ic) ] - --- A call context is still waiting for a response if… -willReceiveResponse :: IC -> CallId -> Bool -willReceiveResponse ic c = c `elem` - -- there is another call context promising to respond to this - [ c' - | CallContext { responded = Responded False, deleted = False, origin = FromCanister c' _} - <- M.elems (call_contexts ic) - ] ++ - -- there is an in-flight call or response message: - [ call_context m | m <- toList (messages ic) ] ++ - -- there this canister is waiting for some canister to stop - [ c' - | CanState { run_status = IsStopping pending } <- M.elems (canisters ic) - , c' <- pending - ] - -- NB: this could be implemented more efficient if kepts a counter of - -- outstanding calls in each call context - --- | Find a starved call context -nextStarved :: ICM m => m (Maybe CallId) -nextStarved = gets $ \ic -> listToMaybe - [ c - | (c, CallContext { responded = Responded False } ) <- M.toList (call_contexts ic) - , not $ willReceiveResponse ic c - ] - --- | Find a canister in stopping state that is, well, stopped -nextStoppedCanister :: ICM m => m (Maybe CanisterId) -nextStoppedCanister = gets $ \ic -> listToMaybe - [ cid - | (cid, CanState { run_status = IsStopping _ }) <- M.toList (canisters ic) - -- no call context still waiting for a response - , null [ () - | (c, ctxt) <- M.toList (call_contexts ic) - , canister ctxt == cid - , not (deleted ctxt) - , willReceiveResponse ic c - ] - ] - - --- | Pick (and remove) next message from queue -popMessage :: ICM m => m (Maybe Message) -popMessage = state $ \ic -> - case messages ic of - Empty -> (Nothing, ic) - m :<| ms -> (Just m, ic { messages = ms }) - - --- | Fake time increase -bumpTime :: ICM m => m () -bumpTime = modify $ - \ic -> ic { canisters = M.map (\cs -> cs { time = time cs +1 }) (canisters ic) } - -setAllTimesTo :: ICM m => Timestamp -> m () -setAllTimesTo ts = modify $ - \ic -> ic { canisters = M.map (\cs -> cs { time = ts }) (canisters ic) } - - --- | Returns true if a step was taken -runStep :: ICM m => m Bool -runStep = do - bumpTime - try - [ with nextReceived processRequest - , with popMessage processMessage - , with nextStarved starveCallContext - , with nextStoppedCanister actuallyStopCanister - ] - where - try = foldr (\g r -> g >>= \case True -> return True; False -> r) (return False) - with sel act = sel >>= maybe (return False) (\x -> act x >> return True) - -runToCompletion :: ICM m => m () -runToCompletion = repeatWhileTrue runStep - - - --- Error handling plumbing - -type CanReject = MonadError (RejectCode, String) -reject :: CanReject m => RejectCode -> String -> m a2 -reject code msg = throwError (code, msg) - --- To maintain the abstraction that the management canister is a canister, --- all its errors are turned into canister errors -rejectAsCanister :: CanReject m => m a -> m a -rejectAsCanister act = catchError act (\(_c,msg) -> reject RC_CANISTER_ERROR msg) - -onReject :: ICM m => - ((RejectCode, String) -> m b) -> - (forall m'. (CanReject m', ICM m') => m' b) -> m b -onReject h act = runExceptT act >>= \case - Left cs -> h cs - Right x -> return x - - -onErr :: Monad m => m (Either a b) -> (a -> m b) -> m b -onErr a b = a >>= either b return - -orElse :: Monad m => m (Maybe a) -> m a -> m a -orElse a b = a >>= maybe b return - -onTrap :: Monad m => m (TrapOr a) -> (String -> m a) -> m a -onTrap a b = a >>= \case { Trap msg -> b msg; Return x -> return x } - - diff --git a/impl/src/IC/Serialise.hs b/impl/src/IC/Serialise.hs deleted file mode 100644 index 7403e0cfa..000000000 --- a/impl/src/IC/Serialise.hs +++ /dev/null @@ -1,131 +0,0 @@ -{-# LANGUAGE StandaloneDeriving #-} -{-# LANGUAGE ConstraintKinds #-} -{-# LANGUAGE DeriveGeneric #-} -{-# LANGUAGE TypeSynonymInstances #-} -{-# LANGUAGE GeneralizedNewtypeDeriving #-} -{-# OPTIONS_GHC -Wno-orphans #-} -{- | -This module defines Serialise instances of the IC state. - -We put them into their own module, despite the usual advise against orphan -instances, to emphasize that these are not part of the `IC.Ref` module with its -“reference implementation” status. - -Also, orphan instances are kinda ok in applications. - --} - -module IC.Serialise () where - -import Codec.Serialise -import GHC.Generics - -import qualified IC.Wasm.Winter as W -import Control.Monad.Random.Lazy -import System.Random.Internal (StdGen(..)) -import System.Random.SplitMix - -import IC.Types -import IC.Wasm.Winter.Persist -import IC.Purify -import IC.Canister.Snapshot -import IC.Canister -import IC.Ref -import IC.Crypto -import qualified IC.Crypto.BLS as BLS - -instance Serialise W.Value - -deriving instance Generic Timestamp -instance Serialise Timestamp where - -deriving instance Generic Responded -instance Serialise Responded where - -deriving instance Generic RejectCode -instance Serialise RejectCode where - -deriving instance Generic Response -instance Serialise Response where - -deriving instance Generic WasmClosure -instance Serialise WasmClosure where - -deriving instance Generic Callback -instance Serialise Callback where - -deriving instance Generic MethodCall -instance Serialise MethodCall where - -deriving instance Generic PInstance -instance Serialise PInstance where - -deriving instance Generic PModuleInst -instance Serialise PModuleInst where - -deriving instance Generic (Snapshot a) -instance Serialise a => Serialise (Snapshot a) where - -deriving instance Generic IC -instance Serialise IC where - -deriving instance Generic CallContext -instance Serialise CallContext where - -deriving instance Generic Message -instance Serialise Message where - -deriving instance Generic RequestStatus -instance Serialise RequestStatus where - -deriving instance Generic CallResponse -instance Serialise CallResponse where - -deriving instance Generic CallRequest -instance Serialise CallRequest where - -deriving instance Generic RunStatus -instance Serialise RunStatus where - -deriving instance Generic CanState -instance Serialise CanState where - -deriving instance Generic CallOrigin -instance Serialise CallOrigin where - -deriving instance Generic EntryPoint -instance Serialise EntryPoint where - -instance Serialise CanisterContent where - encode cc = encode - ( raw_wasm (can_mod cc) - , wsInstances (wasm_state cc) - , wsStableMem (wasm_state cc) - ) - decode = do - (wasm, insts, sm) <- decode - can_mod <- either fail pure $ parseCanister wasm - -- There is some duplication here - wasm_mod <- either fail pure $ W.parseModule wasm - return $ CanisterContent - { can_mod = can_mod - , wasm_state = CanisterSnapshot - { wsModule = wasm_mod - , wsInstances = insts - , wsStableMem = sm - } - } - -deriving instance Serialise EntityId - -deriving instance Serialise StdGen - -instance Serialise SMGen where - encode = encode . unseedSMGen - decode = seedSMGen' <$> decode - -instance Serialise BLS.SecretKey -instance Serialise SecretKey where - encode (BLS sk) = encode sk - encode _ = error "IC.Serialise SecretKey: Only BLS supported" - decode = BLS <$> decode diff --git a/impl/src/IC/StateFile.hs b/impl/src/IC/StateFile.hs deleted file mode 100644 index 6cfc5efa6..000000000 --- a/impl/src/IC/StateFile.hs +++ /dev/null @@ -1,158 +0,0 @@ -{- | -This module provides an (optionally) persistent store for the state of the IC. - -It provides: - * atomic, blocking, exclusive write access - * read access to the latest state, at any time - -If not backed by a file, it is simply kept in memory. - -If backed by a file, state changes are serialized by a separate thread. It -waits for a state change to happen, then waits for 10s, and then serializes the -then youngest state. If the process is shut down (via `conclude`), it -serializes immediately. - -Serialization is atomic (write to a temporary file, then move), so it _should_ -be safe to kill the process; _a_ recent state will be persisted. - --} -{-# LANGUAGE NamedFieldPuns #-} -{-# LANGUAGE BangPatterns #-} -{-# LANGUAGE BlockArguments #-} -module IC.StateFile (Store, withStore, modifyStore, peekStore) where - -import Codec.Serialise -import qualified Data.ByteString.Lazy as BS -import Control.Concurrent -import Control.Exception -import Control.Monad.State -import Data.IORef -import System.AtomicWrite.Writer.LazyByteString.Binary -import System.Directory -import System.Mem.StableName -import System.IO -import System.Exit -import Codec.Compression.GZip -import Codec.Compression.Zlib.Internal (DecompressError) - -data Store a = Store - { stateVar :: MVar a - , lastState :: IORef a - , pingStorage :: a -> IO () - , conclude :: IO () - } - -withStore :: Serialise a => IO a -> Maybe FilePath -> (Store a -> IO b) -> IO b -withStore initial backingFile = bracket (newStore initial backingFile) conclude - -newStore :: Serialise a => IO a -> Maybe FilePath -> IO (Store a) -newStore initial backingFile = do - s <- case backingFile of - Just fn -> do - ex <- doesFileExist fn - if ex - then do - let err e = do - hPutStr stderr $ "Error: Couldn't open " ++ fn ++ ":\n" - hPutStr stderr $ e ++ "\n" - hPutStr stderr "Maybe the file is corrupt or incompatiblen\n" - hPutStr stderr $ "Delete " ++ fn ++ " and try again.\n" - exitFailure - withFile fn ReadMode $ \hnd -> do - input <- BS.hGetContents hnd - case deserialiseOrFail (decompress input) of - Left err -> throwIO err - Right x -> return x - `catches` - [ Handler $ \ex -> err (show (ex :: IOException)) - , Handler $ \ex -> err (show (ex :: DecompressError)) - , Handler $ \ex -> err (show (ex :: DeserialiseFailure)) - ] - else initial - Nothing -> initial - stateVar <- newMVar s - lastState <- newIORef s - (pingStorage, conclude) <- case backingFile of - Nothing -> pure (\_ -> return (), return ()) - Just fn -> writerThread $ \s -> atomicWriteFile fn (compress (serialise s)) - return $ Store { stateVar, lastState, pingStorage, conclude } - -modifyStore :: Serialise a => Store a -> StateT a IO b -> IO b -modifyStore store action = - modifyMVar (stateVar store) $ \(!s) -> do - n1 <- makeStableName s - (x, s') <- runStateT action s - n2 <- makeStableName s' - -- If the stable names are the same, this means - -- that the state is unchanged. No need to write new state - unless (n1 == n2) $ do - -- Remember last state (for peekStore) - writeIORef (lastState store) s' - -- Tell serializer about this - pingStorage store s' - return (s', x) - -peekStore :: Store a -> IO a -peekStore Store{lastState} = readIORef lastState - - --- The abstract logic of the separte writer thread. --- --- I do not claim to be satisfied with this implementation, surely there must --- be an easier way to get something that --- --- * waits 10s after a change --- * always writes the latest version --- * when shutting down, interrupts that 10s wait, and waits for the writing --- to happen. --- --- So within this function signature and specification, rewrites are welcome. --- --- Also see https://www.reddit.com/r/haskell/comments/ky1llf/concurrent_programming_puzzle_debouncing_events/ --- which didn’t turn up anything decisively better --- -writerThread :: - (a -> IO ()) -> -- ^ how to write the thing - IO (a -> IO (), IO ()) -- notify about new value (async); shutdown (sync) -writerThread doSomething = do - latest_unwritten <- newEmptyMVar - write_soon <- newEmptyMVar - write_now <- newEmptyMVar - please_shut_down <- newIORef False - - has_shut_down <- newEmptyMVar - - -- A lock for exclusive access to latest_unwritten - lock <- newMVar () - let atomically act = withMVar lock (const act) - - -- the debouncer of the todo notification - debouncer <- forkIO $ forever $ do - takeMVar write_soon - threadDelay (10 * 1000 * 1000) -- 10s - void $ tryTakeMVar write_soon - tryPutMVar write_now () - - -- the writer thread - void $ forkIO $ fix $ \loop -> do - takeMVar write_now - shutdown <- readIORef please_shut_down - value <- atomically $ tryTakeMVar latest_unwritten - mapM_ doSomething value - if shutdown - then putMVar has_shut_down () - else loop - - let notify x = do - atomically $ do - void $ tryTakeMVar latest_unwritten - putMVar latest_unwritten x - void $ tryPutMVar write_soon () - - let shutdown = do - killThread debouncer - writeIORef please_shut_down True - void $ tryPutMVar write_now () - takeMVar has_shut_down - - return (notify, shutdown) diff --git a/impl/src/IC/Test/BLS.hs b/impl/src/IC/Test/BLS.hs deleted file mode 100644 index 5e4579ac6..000000000 --- a/impl/src/IC/Test/BLS.hs +++ /dev/null @@ -1,51 +0,0 @@ --- Unit test for IC.Test.Crypto.BLS -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE BinaryLiterals #-} -{-# LANGUAGE ViewPatterns #-} -module IC.Test.BLS (blsTests) where - -import Test.Tasty -import Test.Tasty.QuickCheck -import qualified Data.ByteString.Lazy as BS -import Data.Bits - -import qualified IC.Crypto.BLS as BLS - -blsTests :: TestTree -blsTests = testGroup "BLS crypto tests" - [ testProperty "public key is 96 bytes" $ - \(BS.pack -> seed) -> - let sk = BLS.createKey seed in - let pk = BLS.toPublicKey sk in - BS.length pk === 96 - , testProperty "public key high bits are either 0b100 or 0b100" $ - \(BS.pack -> seed) -> - let sk = BLS.createKey seed in - let pk = BLS.toPublicKey sk in - BS.head pk `shiftR` 5 === 0b100 .||. BS.head pk `shiftR` 5 === 0b101 - , testProperty "signature is 48 bytes" $ - \(BS.pack -> seed) (BS.pack -> msg) -> - let sk = BLS.createKey seed in - let sig = BLS.sign sk msg in - BS.length sig === 48 - , testProperty "signature high bits are either 0b100 or 0b100" $ - \(BS.pack -> seed) (BS.pack -> msg) -> - let sk = BLS.createKey seed in - let sig = BLS.sign sk msg in - BS.head sig `shiftR` 5 === 0b100 .||. BS.head sig `shiftR` 5 === 0b101 - , testProperty "create-sign-verify" $ - \(BS.pack -> seed) (BS.pack -> msg) -> - let sk = BLS.createKey seed in - let sig = BLS.sign sk msg in - BLS.verify (BLS.toPublicKey sk) msg sig - , testProperty "invalid sig" $ - \(BS.pack -> seed) (BS.pack -> msg) (BS.pack -> sig) -> - let sk = BLS.createKey seed in - not (BLS.verify (BLS.toPublicKey sk) msg sig) - , testProperty "wrong message" $ - \(BS.pack -> seed) (BS.pack -> msg1) (BS.pack -> msg2) -> - let sk = BLS.createKey seed in - let sig = BLS.sign sk msg2 in - msg1 /= msg2 ==> not (BLS.verify (BLS.toPublicKey sk) msg1 sig) - ] - diff --git a/impl/src/IC/Test/ECDSA.hs b/impl/src/IC/Test/ECDSA.hs deleted file mode 100644 index 708a816d2..000000000 --- a/impl/src/IC/Test/ECDSA.hs +++ /dev/null @@ -1,34 +0,0 @@ --- Unit test for IC.Test.Crypto.ECDSA -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE BinaryLiterals #-} -{-# LANGUAGE ViewPatterns #-} -module IC.Test.ECDSA (ecdsaTests) where - -import qualified Data.ByteString.Lazy as BS - -import Test.Tasty -import Test.Tasty.QuickCheck -import Test.Tasty.HUnit -import Test.QuickCheck.IO () - -import qualified IC.Crypto.ECDSA as ECDSA - -ecdsaTests :: TestTree -ecdsaTests = testGroup "ECDSA crypto tests" - [ testProperty "create-sign-verify" $ - \(BS.pack -> seed) (BS.pack -> msg) -> do - let sk = ECDSA.createKey seed - sig <- ECDSA.sign sk msg - assertBool "verifies" $ ECDSA.verify (ECDSA.toPublicKey sk) msg sig - , testProperty "invalid sig" $ - \(BS.pack -> seed) (BS.pack -> msg) (BS.pack -> sig) -> - let sk = ECDSA.createKey seed in - assertBool "does not verify" $ not $ ECDSA.verify (ECDSA.toPublicKey sk) msg sig - , testProperty "wrong message" $ - \(BS.pack -> seed) (BS.pack -> msg1) (BS.pack -> msg2) -> - msg1 /= msg2 ==> do - let sk = ECDSA.createKey seed - sig <- ECDSA.sign sk msg2 - assertBool "does not verify" $ not $ ECDSA.verify (ECDSA.toPublicKey sk) msg1 sig - ] - diff --git a/impl/src/IC/Test/HashTree.hs b/impl/src/IC/Test/HashTree.hs deleted file mode 100644 index fb7ecd12c..000000000 --- a/impl/src/IC/Test/HashTree.hs +++ /dev/null @@ -1,152 +0,0 @@ --- Unit/Prop tests for IC.HashTree -{-# LANGUAGE OverloadedStrings #-} -{-# OPTIONS_GHC -Wno-orphans #-} -module IC.Test.HashTree (hashTreeTests) where - -import Data.List -import Test.Tasty -import Test.Tasty.HUnit -import Test.Tasty.QuickCheck -import qualified Data.Map.Lazy as M -import qualified Data.ByteString.Lazy as BS -import qualified Text.Hex as T -import Codec.CBOR.Term -import Codec.CBOR.Write - -import IC.HashTree -import IC.HashTree.CBOR -import Data.Bifunctor - -hashTreeTests :: TestTree -hashTreeTests = testGroup "Hash tree tests" - [ testGroup "Examples in spec document" - [ testCase "CBOR of full tree" $ - asHex (toLazyByteString $ encodeTerm $ encodeHashTree exampleTree) - @?= exampleTreeCBOR - , testCase "root hash of full tree" $ - asHex (reconstruct exampleTree) - @?= "eb5c5b2195e62d996b84c9bcc8259d19a83786a2f59e0878cec84c811f669aa0" - , testCase "CBOR of pruned tree" $ - asHex (toLazyByteString $ encodeTerm $ encodeHashTree prunedTree) - @?= prunedTreeCBOR - , testCase "root hash of pruned tree" $ - asHex (reconstruct exampleTree) - @?= asHex (reconstruct prunedTree) - , testCase "tree lokups" $ do - lookupPath prunedTree ["a", "a"] @?= Unknown - lookupPath prunedTree ["a", "y"] @?= Found "world" - lookupPath prunedTree ["aa"] @?= Absent - lookupPath prunedTree ["ax"] @?= Absent - lookupPath prunedTree ["b"] @?= Unknown - lookupPath prunedTree ["bb"] @?= Unknown - lookupPath prunedTree ["d"] @?= Found "morning" - lookupPath prunedTree ["e"] @?= Absent - ] - , testProperty "lookup succeeds" $ \lt (AsPath p)-> - lookupPath (construct lt) p === lookupL lt p - , testProperty "prune preserves hash" $ \lt (AsPaths ps) -> - let ht = construct lt - in reconstruct ht === reconstruct (prune ht ps) - , testProperty "prune preserves lookups" $ \lt (AsPaths ps) (AsPath p) -> - let ht = construct lt - in notError (lookupPath ht p) ==> - if any (`isPrefixOf` p) ps - then lookupPath (prune ht ps) p === lookupPath ht p - else lookupPath (prune ht ps) p `elemP` [Unknown, Absent] - ] - -asHex :: BS.ByteString -> T.Text -asHex = T.encodeHex . BS.toStrict - -exampleTree :: HashTree -exampleTree = - Fork - (Fork - (Labeled "a" - (Fork - (Fork - (Labeled "x" (Leaf "hello")) - EmptyTree - ) - (Labeled "y" (Leaf "world")) - ) - ) - (Labeled "b" (Leaf "good")) - ) - (Fork - (Labeled "c" EmptyTree) - (Labeled "d" (Leaf "morning") - ) - ) - -prunedTree :: HashTree -prunedTree = prune exampleTree [["a","y"], ["ax"], ["d"]] - -exampleTreeCBOR :: T.Text -exampleTreeCBOR = "8301830183024161830183018302417882034568656c6c6f810083024179820345776f726c6483024162820344676f6f648301830241638100830241648203476d6f726e696e67" - -prunedTreeCBOR :: T.Text -prunedTreeCBOR = "83018301830241618301820458201b4feff9bef8131788b0c9dc6dbad6e81e524249c879e9f10f71ce3749f5a63883024179820345776f726c6483024162820458207b32ac0c6ba8ce35ac82c255fc7906f7fc130dab2a090f80fe12f9c2cae83ba6830182045820ec8324b8a1f1ac16bd2e806edba78006479c9877fed4eb464a25485465af601d830241648203476d6f726e696e67" - - --- This is, in a way, the spec for lookupPath -lookupL :: LabeledTree -> Path -> Res -lookupL (Value _) (_:_) = Error "Found leaf when expecting subtree" -lookupL (SubTrees sts) (l:ls) = case M.lookup l sts of - Just st -> lookupL st ls - Nothing -> Absent -lookupL (Value v) [] = Found v -lookupL (SubTrees _) [] = Error "Found forest when expecting leaf" - -notError :: Res -> Bool -notError (Error _) = False -notError _ = True - --- Property based testing infrastructure --- (slightly more verbose because IC.HashTree is not very typed - -elemP :: (Eq a, Show a) => a -> [a] -> Property -x `elemP` xs = disjoin $ map (x ===) xs - -genValue :: Gen Value -genValue = BS.pack <$> arbitrary - -genLabel :: Gen Label -genLabel = oneof [ pure "", pure "hello", pure "world", BS.pack <$> arbitrary ] - - -newtype AsLabel = AsLabel { asLabel :: Label } -instance Arbitrary AsLabel where arbitrary = AsLabel <$> genLabel -instance Show AsLabel where show (AsLabel l) = show l - -newtype AsPath = AsPath { asPath :: Path } -instance Arbitrary AsPath where - arbitrary = AsPath . map asLabel <$> arbitrary - shrink (AsPath ps) = map AsPath (init (inits ps)) -instance Show AsPath where show (AsPath l) = show l - -newtype AsPaths = AsPaths { _asPaths :: [Path] } -instance Arbitrary AsPaths where - arbitrary = AsPaths . map asPath <$> arbitrary - shrink (AsPaths ps) = AsPaths <$> - [ as ++ bs | (as,_,bs) <- splits ] ++ - [ as ++ [v'] ++ bs | (as,v,bs) <- splits, AsPath v' <- shrink (AsPath v) ] - where - splits = [(as,v,bs) | i <- [0..length ps-1], (as,v:bs) <- pure $ splitAt i ps ] -instance Show AsPaths where show (AsPaths l) = show l - -instance Arbitrary LabeledTree where - arbitrary = sized go - where - go 0 = Value <$> genValue - go n = oneof - [ Value <$> genValue - , resize (n `div` 2) $ - SubTrees . M.fromList . map (first asLabel) <$> arbitrary - ] - shrink (Value _) = [Value ""] - shrink (SubTrees m) = SubTrees <$> - [ M.delete k m | k <- M.keys m ] ++ - [ M.insert k v' m | (k,v) <- M.toList m, v' <- shrink v ] - - diff --git a/impl/src/IC/Test/Options.hs b/impl/src/IC/Test/Options.hs deleted file mode 100644 index 895563136..000000000 --- a/impl/src/IC/Test/Options.hs +++ /dev/null @@ -1,23 +0,0 @@ -module IC.Test.Options where - -import Data.Proxy -import Data.List -import Test.Tasty.Options -import Options.Applicative hiding (str) - --- Configuration: The URL of of the endpoint to test - -newtype Endpoint = Endpoint String - -instance IsOption Endpoint where - defaultValue = Endpoint "http://0.0.0.0:8001" - parseValue s = Just $ Endpoint base - where base | "/" `isSuffixOf` s = init s - | otherwise = s - optionName = return "endpoint" - optionHelp = return "Internet Computer endpoint to connect to (default: http://0.0.0.0:8001)" - optionCLParser = mkOptionCLParser (metavar "URL") - - -endpointOption :: OptionDescription -endpointOption = Option (Proxy :: Proxy Endpoint) diff --git a/impl/src/IC/Test/Secp256k1.hs b/impl/src/IC/Test/Secp256k1.hs deleted file mode 100644 index 2eb9f4d63..000000000 --- a/impl/src/IC/Test/Secp256k1.hs +++ /dev/null @@ -1,43 +0,0 @@ --- Unit test for IC.Test.Crypto.Secp256k1 -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE BinaryLiterals #-} -{-# LANGUAGE ViewPatterns #-} -module IC.Test.Secp256k1 (secp256k1Tests) where - -import qualified Data.ByteString.Lazy as BS -import qualified Data.Text as T - -import Test.Tasty -import Test.Tasty.QuickCheck -import Test.Tasty.HUnit -import Test.QuickCheck.IO () - -import qualified IC.Crypto.Secp256k1 as Secp256k1 - -assertRight :: Either T.Text () -> Assertion -assertRight (Right ()) = return () -assertRight (Left err) = assertFailure (T.unpack err) - -assertLeft :: Either T.Text () -> Assertion -assertLeft (Left _) = return () -assertLeft (Right _) = assertFailure "Unexpected success" - -secp256k1Tests :: TestTree -secp256k1Tests = testGroup "Secp256k1 crypto tests" - [ testProperty "create-sign-verify" $ - \(BS.pack -> seed) (BS.pack -> msg) -> do - let sk = Secp256k1.createKey seed - sig <- Secp256k1.sign sk msg - assertRight $ Secp256k1.verify (Secp256k1.toPublicKey sk) msg sig - , testProperty "invalid sig" $ - \(BS.pack -> seed) (BS.pack -> msg) (BS.pack -> sig) -> - let sk = Secp256k1.createKey seed in - assertLeft $ Secp256k1.verify (Secp256k1.toPublicKey sk) msg sig - , testProperty "wrong message" $ - \(BS.pack -> seed) (BS.pack -> msg1) (BS.pack -> msg2) -> - msg1 /= msg2 ==> do - let sk = Secp256k1.createKey seed - sig <- Secp256k1.sign sk msg2 - assertLeft $ Secp256k1.verify (Secp256k1.toPublicKey sk) msg1 sig - ] - diff --git a/impl/src/IC/Test/Spec.hs b/impl/src/IC/Test/Spec.hs deleted file mode 100644 index ad00a30c2..000000000 --- a/impl/src/IC/Test/Spec.hs +++ /dev/null @@ -1,3156 +0,0 @@ -{- | - -This module contains a test suite for the Internet Computer - --} -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE ImplicitParams #-} -{-# LANGUAGE TypeApplications #-} -{-# LANGUAGE ConstraintKinds #-} -{-# LANGUAGE LambdaCase #-} -{-# LANGUAGE ScopedTypeVariables #-} -{-# LANGUAGE RankNTypes #-} -{-# LANGUAGE OverloadedLabels #-} -{-# LANGUAGE TypeOperators #-} -{-# LANGUAGE FlexibleContexts #-} -{-# LANGUAGE TypeFamilies #-} -{-# LANGUAGE NumericUnderscores #-} -{-# LANGUAGE DataKinds #-} -{-# LANGUAGE TupleSections #-} -{-# LANGUAGE NamedFieldPuns #-} -{-# LANGUAGE QuantifiedConstraints #-} -{-# LANGUAGE AllowAmbiguousTypes #-} -{-# OPTIONS_GHC -Wno-orphans #-} -module IC.Test.Spec (preFlight, TestConfig, connect, ReplWrapper(..), icTests) where - -import qualified Data.Text as T -import qualified Data.Text.Encoding as T -import qualified Data.Text.Encoding.Error as T -import qualified Text.Hex as H -import qualified Data.ByteString.Lazy as BS -import qualified Data.ByteString.Builder as BS -import qualified Data.HashMap.Lazy as HM -import qualified Data.Map.Lazy as M -import qualified Data.Set as S -import qualified Data.Vector as Vec -import Network.HTTP.Client -import Network.HTTP.Client.TLS -import Network.HTTP.Types -import Numeric.Natural -import Data.List -import Data.Char -import Test.Tasty -import Test.Tasty.HUnit -import Test.Tasty.Options -import Control.Monad.Trans -import Control.Concurrent -import Control.Monad -import Control.Exception (catch) -import Data.Traversable -import Data.Word -import Data.Functor -import GHC.TypeLits -import System.FilePath -import System.Directory -import System.Environment -import System.Random -import System.Exit -import Data.Time.Clock.POSIX -import Codec.Candid (Principal(..), prettyPrincipal) -import qualified Data.Binary.Get as Get -import qualified Codec.Candid as Candid -import Data.Row -import qualified Data.Row.Records as R -import qualified Data.Row.Variants as V -import qualified Data.Row.Internal as R - -import IC.Types (EntityId(..)) -import IC.Version -import IC.HTTP.GenR -import IC.HTTP.GenR.Parse -import IC.HTTP.CBOR (decode, encode) -import IC.HTTP.RequestId -import IC.Management -import IC.Crypto -import qualified IC.Crypto.CanisterSig as CanisterSig -import qualified IC.Crypto.DER as DER -import qualified IC.Crypto.DER_BLS as DER_BLS -import IC.Id.Forms hiding (Blob) -import IC.Test.Options -import IC.Test.Universal -import IC.HashTree hiding (Blob, Label) -import IC.Certificate -import IC.Certificate.Value -import IC.Certificate.CBOR -import IC.Hash - --- * Test data, standard requests - -doesn'tExist :: Blob -doesn'tExist = "\xDE\xAD\xBE\xEF" -- hopefully no such canister/user exists - -defaultSK :: SecretKey -defaultSK = createSecretKeyEd25519 "fixed32byteseedfortesting" - -otherSK :: SecretKey -otherSK = createSecretKeyEd25519 "anotherfixed32byteseedfortesting" - -webAuthnSK :: SecretKey -webAuthnSK = createSecretKeyWebAuthn "webauthnseed" - -ecdsaSK :: SecretKey -ecdsaSK = createSecretKeyECDSA "ecdsaseed" - -secp256k1SK :: SecretKey -secp256k1SK = createSecretKeySecp256k1 "secp256k1seed" - -defaultUser :: Blob -defaultUser = mkSelfAuthenticatingId $ toPublicKey defaultSK -otherUser :: Blob -otherUser = mkSelfAuthenticatingId $ toPublicKey otherSK -webAuthnUser :: Blob -webAuthnUser = mkSelfAuthenticatingId $ toPublicKey webAuthnSK -ecdsaUser :: Blob -ecdsaUser = mkSelfAuthenticatingId $ toPublicKey ecdsaSK -secp256k1User :: Blob -secp256k1User = mkSelfAuthenticatingId $ toPublicKey secp256k1SK -anonymousUser :: Blob -anonymousUser = "\x04" - -queryToNonExistant :: GenR -queryToNonExistant = rec - [ "request_type" =: GText "query" - , "sender" =: GBlob anonymousUser - , "canister_id" =: GBlob doesn'tExist - , "method_name" =: GText "foo" - , "arg" =: GBlob "nothing to see here" - ] - -readStateEmpty :: GenR -readStateEmpty = rec - [ "request_type" =: GText "read_state" - , "sender" =: GBlob defaultUser - , "paths" =: GList [] - ] - -trivialWasmModule :: Blob -trivialWasmModule = "\0asm\1\0\0\0" - -addIfNotThere :: Monad m => T.Text -> m GenR -> GenR -> m GenR -addIfNotThere f _ (GRec hm)| f `HM.member` hm = return (GRec hm) -addIfNotThere f a (GRec hm) = do - x <- a - return $ GRec $ HM.insert f x hm -addIfNotThere _ _ _ = error "addIfNotThere: not a record" - -deleteField :: T.Text -> GenR -> GenR -deleteField f (GRec hm) = GRec $ HM.delete f hm -deleteField _ _ = error "deleteField: not a record" - -modNatField :: T.Text -> (Natural -> Natural) -> GenR -> GenR -modNatField f g (GRec hm) = GRec $ HM.adjust underNat f hm - where underNat :: GenR -> GenR - underNat (GNat n) = GNat (g n) - underNat _ = error "modNatField: not a nat field" -modNatField _ _ _ = error "modNatField: not a record" - -addNonce :: GenR -> IO GenR -addNonce = addIfNotThere "nonce" $ - GBlob <$> getRand8Bytes - --- Adds expiry 1 minute -addExpiry :: GenR -> IO GenR -addExpiry = addIfNotThere "ingress_expiry" $ do - t <- getPOSIXTime - return $ GNat $ round ((t + 60) * 1000_000_000) - -envelopeFor :: Blob -> GenR -> IO GenR -envelopeFor u content | u == anonymousUser = return $ rec [ "content" =: content ] -envelopeFor u content = envelope key content - where - key :: SecretKey - key | u == defaultUser = defaultSK - | u == otherUser = otherSK - | u == webAuthnUser = webAuthnSK - | u == ecdsaUser = ecdsaSK - | u == secp256k1User = secp256k1SK - | u == anonymousUser = error "No key for the anonymous user" - | otherwise = error $ "Don't know key for user " ++ show u - -envelope :: SecretKey -> GenR -> IO GenR -envelope sk = delegationEnv sk [] - -delegationEnv :: SecretKey -> [(SecretKey, Maybe [Blob])] -> GenR -> IO GenR -delegationEnv sk1 dels content = do - let sks = sk1 : map fst dels - - t <- getPOSIXTime - let expiry = round ((t + 60) * 1000_000_000) - delegations <- for (zip sks dels) $ \(sk1, (sk2,targets)) -> do - let delegation = rec $ - [ "pubkey" =: GBlob (toPublicKey sk2) - , "expiration" =: GNat expiry - ] ++ - [ "targets" =: GList (map GBlob ts) | Just ts <- pure targets ] - sig <- sign "ic-request-auth-delegation" sk1 (requestId delegation) - return $ rec [ "delegation" =: delegation, "signature" =: GBlob sig ] - sig <- sign "ic-request" (last sks) (requestId content) - return $ rec $ - [ "sender_pubkey" =: GBlob (toPublicKey sk1) - , "sender_sig" =: GBlob sig - , "content" =: content - ] ++ - [ "sender_delegation" =: GList delegations | not (null delegations) ] - --- a little bit of smartness in our combinators - -senderOf :: GenR -> Blob -senderOf (GRec hm) | Just (GBlob id) <- HM.lookup "sender" hm = id -senderOf _ = anonymousUser - -addNonceExpiryEnv :: GenR -> IO GenR -addNonceExpiryEnv req = do - addNonce req >>= addExpiry >>= envelopeFor (senderOf req) - -badEnvelope :: GenR -> GenR -badEnvelope content = rec - [ "sender_pubkey" =: GBlob (toPublicKey defaultSK) - , "sender_sig" =: GBlob (BS.replicate 64 0x42) - , "content" =: content - ] - -noDomainSepEnv :: SecretKey -> GenR -> IO GenR -noDomainSepEnv sk content = do - sig <- sign "" sk (requestId content) - return $ rec - [ "sender_pubkey" =: GBlob (toPublicKey sk) - , "sender_sig" =: GBlob sig - , "content" =: content - ] - -noExpiryEnv, pastExpiryEnv, futureExpiryEnv :: GenR -> GenR -noExpiryEnv = deleteField "ingress_expiry" -pastExpiryEnv = modNatField "ingress_expiry" (subtract 3600_000_000_000) -futureExpiryEnv = modNatField "ingress_expiry" (+ 3600_000_000_000) - --- * Preflight checks: Get the root key, and tell user about versions - -data TestConfig = TestConfig - { tc_root_key :: Blob - , tc_manager :: Manager - , tc_endPoint :: String - } - -makeTestConfig :: String -> IO TestConfig -makeTestConfig ep' = do - manager <- newTlsManagerWith $ tlsManagerSettings - { managerResponseTimeout = responseTimeoutMicro 60_000_000 -- 60s - } - request <- parseRequest $ ep ++ "/api/v2/status" - putStrLn $ "Fetching endpoint status from " ++ show ep ++ "..." - s <- (httpLbs request manager >>= okCBOR >>= statusResonse) - `catch` (\(HUnitFailure _ r) -> putStrLn r >> exitFailure) - - putStrLn $ "Spec version tested: " ++ T.unpack specVersion - putStrLn $ "Spec version claimed: " ++ T.unpack (status_api_version s) - - return TestConfig - { tc_root_key = status_root_key s - , tc_manager = manager - , tc_endPoint = ep - } - where - -- strip trailing slash - ep | null ep' = error "empty endpoint" - | last ep' == '/' = init ep' - | otherwise = ep' - -preFlight :: OptionSet -> IO TestConfig -preFlight os = do - let Endpoint ep = lookupOption os - makeTestConfig ep - - -newtype ReplWrapper = R (forall a. (HasTestConfig => a) -> a) --- | This is for use from the Haskell REPL, see README.md -connect :: String -> IO ReplWrapper -connect ep = do - testConfig <- makeTestConfig ep - let ?testConfig = testConfig - return (R id) - - --- * The actual test suite (see below for helper functions) - -icTests :: TestConfig -> TestTree -icTests = withTestConfig $ testGroup "Interface Spec acceptance tests" - [ simpleTestCase "create and install" $ \_ -> - return () - - , testCase "create_canister necessary" $ - ic_install'' defaultUser (enum #install) doesn'tExist trivialWasmModule "" - >>= isErrOrReject [3,5] - - , testCaseSteps "management requests" $ \step -> do - step "Create (provisional)" - can_id <- create - - step "Install" - ic_install ic00 (enum #install) can_id trivialWasmModule "" - - step "Install again fails" - ic_install'' defaultUser (enum #install) can_id trivialWasmModule "" - >>= isErrOrReject [3,5] - - step "Reinstall" - ic_install ic00 (enum #reinstall) can_id trivialWasmModule "" - - step "Reinstall as wrong user" - ic_install'' otherUser (enum #reinstall) can_id trivialWasmModule "" - >>= isErrOrReject [3,5] - - step "Upgrade" - ic_install ic00 (enum #upgrade) can_id trivialWasmModule "" - - step "Upgrade as wrong user" - ic_install'' otherUser (enum #upgrade) can_id trivialWasmModule "" - >>= isErrOrReject [3,5] - - step "Change controller" - ic_set_controllers ic00 can_id [otherUser] - - step "Change controller (with wrong controller)" - ic_set_controllers'' defaultUser can_id [otherUser] - >>= isErrOrReject [3,5] - - step "Reinstall as new controller" - ic_install (ic00as otherUser) (enum #reinstall) can_id trivialWasmModule "" - - , testCaseSteps "reinstall on empty" $ \step -> do - step "Create" - can_id <- create - - step "Reinstall over empty canister" - ic_install ic00 (enum #reinstall) can_id trivialWasmModule "" - - , testCaseSteps "canister_status" $ \step -> do - step "Create empty" - cid <- create - cs <- ic_canister_status ic00 cid - cs .! #status @?= enum #running - cs .! #settings .! #controllers @?= Vec.fromList [Principal defaultUser] - cs .! #module_hash @?= Nothing - - step "Install" - ic_install ic00 (enum #install) cid trivialWasmModule "" - cs <- ic_canister_status ic00 cid - cs .! #module_hash @?= Just (sha256 trivialWasmModule) - - , testCaseSteps "canister lifecycle" $ \step -> do - cid <- install $ - onPreUpgrade $ callback $ - ignore (stableGrow (int 1)) >>> - stableWrite (int 0) (i2b getStatus) - - step "Is running (via management)?" - cs <- ic_canister_status ic00 cid - cs .! #status @?= enum #running - - step "Is running (local)?" - query cid (replyData (i2b getStatus)) >>= asWord32 >>= is 1 - - step "Stop" - ic_stop_canister ic00 cid - - step "Is stopped (via management)?" - cs <- ic_canister_status ic00 cid - cs .! #status @?= enum #stopped - - step "Stop is noop" - ic_stop_canister ic00 cid - - step "Cannot call (update)?" - call'' cid reply >>= isErrOrReject [5] - - step "Cannot call (query)?" - query' cid reply >>= isReject [5] - - step "Upgrade" - upgrade cid $ setGlobal (i2b getStatus) - - step "Start canister" - ic_start_canister ic00 cid - - step "Is running (via managemnet)?" - cs <- ic_canister_status ic00 cid - cs .! #status @?= enum #running - - step "Is running (local)?" - query cid (replyData (i2b getStatus)) >>= asWord32 >>= is 1 - - step "Was stopped during pre-upgrade?" - query cid (replyData (stableRead (int 0) (int 4))) >>= asWord32 >>= is 3 - - step "Was stopped during post-upgrade?" - query cid (replyData getGlobal) >>= asWord32 >>= is 3 - - step "Can call (update)?" - call_ cid reply - - step "Can call (query)?" - query_ cid reply - - step "Start is noop" - ic_start_canister ic00 cid - - , testCaseSteps "canister deletion" $ \step -> do - cid <- install noop - - step "Deletion fails" - ic_delete_canister' ic00 cid >>= isReject [5] - - step "Stop" - ic_stop_canister ic00 cid - - step "Is stopped?" - cs <- ic_canister_status ic00 cid - cs .! #status @?= enum #stopped - - step "Deletion succeeds" - ic_delete_canister ic00 cid - - -- Disabled; such a call gets accepted (200) but - -- then the status never shows up, which causes a timeout - -- - -- step "Cannot call (update)?" - -- call' cid reply >>= isReject [3] - - step "Cannot call (inter-canister)?" - cid2 <- install noop - do call cid2 $ inter_update cid defArgs - >>= isRelay >>= isReject [3] - - step "Cannot call (query)?" - query' cid reply >>= isReject [3] - - step "Cannot query canister_status" - ic_canister_status'' defaultUser cid >>= isErrOrReject [3,5] - - step "Deletion fails" - ic_delete_canister'' defaultUser cid >>= isErrOrReject [3,5] - - - , testCaseSteps "canister lifecycle (wrong controller)" $ \step -> do - cid <- install noop - - step "Start as wrong user" - ic_start_canister'' otherUser cid >>= isErrOrReject [3,5] - step "Stop as wrong user" - ic_stop_canister'' otherUser cid >>= isErrOrReject [3,5] - step "Canister Status as wrong user" - ic_canister_status'' otherUser cid >>= isErrOrReject [3,5] - step "Delete as wrong user" - ic_delete_canister'' otherUser cid >>= isErrOrReject [3,5] - - - , testCaseSteps "aaaaa-aa (inter-canister)" $ \step -> do - -- install universal canisters to proxy the requests - cid <- install noop - cid2 <- install noop - - step "Create" - can_id <- ic_provisional_create (ic00via cid) Nothing empty - - step "Install" - ic_install (ic00via cid) (enum #install) can_id trivialWasmModule "" - - step "Install again fails" - ic_install' (ic00via cid) (enum #install) can_id trivialWasmModule "" - >>= isReject [3,5] - - step "Reinstall" - ic_install (ic00via cid) (enum #reinstall) can_id trivialWasmModule "" - - step "Reinstall as wrong user" - ic_install' (ic00via cid2) (enum #reinstall) can_id trivialWasmModule "" - >>= isReject [3,5] - - step "Upgrade" - ic_install (ic00via cid) (enum #upgrade) can_id trivialWasmModule "" - - step "Change controller" - ic_set_controllers (ic00via cid) can_id [cid2] - - step "Change controller (with wrong controller)" - ic_set_controllers' (ic00via cid) can_id [cid2] - >>= isReject [3,5] - - step "Reinstall as new controller" - ic_install (ic00via cid2) (enum #reinstall) can_id trivialWasmModule "" - - step "Create" - can_id2 <- ic_provisional_create (ic00via cid) Nothing empty - - step "Reinstall on empty" - ic_install (ic00via cid) (enum #reinstall) can_id2 trivialWasmModule "" - - , simpleTestCase "aaaaa-aa (inter-canister, large)" $ \cid -> do - universal_wasm <- getTestWasm "universal_canister" - can_id <- ic_provisional_create (ic00via cid) Nothing empty - ic_install (ic00via cid) (enum #install) can_id universal_wasm "" - do call can_id $ replyData "Hi" - >>= is "Hi" - - , simpleTestCase "randomness" $ \cid -> do - r1 <- ic_raw_rand (ic00via cid) - r2 <- ic_raw_rand (ic00via cid) - BS.length r1 @?= 32 - BS.length r2 @?= 32 - assertBool "random blobs are different" $ r1 /= r2 - - , testGroup "simple calls" - [ simpleTestCase "Call" $ \cid -> - call cid (replyData "ABCD") >>= is "ABCD" - - , simpleTestCase "Call (query)" $ \cid -> do - query cid (replyData "ABCD") >>= is "ABCD" - - , simpleTestCase "Call no non-existant update method" $ \cid -> - do awaitCall' cid $ rec - [ "request_type" =: GText "call" - , "sender" =: GBlob defaultUser - , "canister_id" =: GBlob cid - , "method_name" =: GText "no_such_update" - , "arg" =: GBlob "" - ] - >>= isErrOrReject [3] - - , simpleTestCase "Call no non-existant query method" $ \cid -> - do queryCBOR cid $ rec - [ "request_type" =: GText "query" - , "sender" =: GBlob defaultUser - , "canister_id" =: GBlob cid - , "method_name" =: GText "no_such_update" - , "arg" =: GBlob "" - ] - >>= queryResponse >>= isReject [3] - - , simpleTestCase "reject" $ \cid -> - call' cid (reject "ABCD") >>= isReject [4] - - , simpleTestCase "reject (query)" $ \cid -> - query' cid (reject "ABCD") >>= isReject [4] - - , simpleTestCase "No response" $ \cid -> - call' cid noop >>= isReject [5] - - , simpleTestCase "No response does not rollback" $ \cid -> do - call'' cid (setGlobal "FOO") >>= isErrOrReject [5] - query cid (replyData getGlobal) >>= is "FOO" - - , simpleTestCase "No response (query)" $ \cid -> - query' cid noop >>= isReject [5] - - , simpleTestCase "Double reply" $ \cid -> - call' cid (reply >>> reply) >>= isReject [5] - - , simpleTestCase "Double reply (query)" $ \cid -> - query' cid (reply >>> reply) >>= isReject [5] - - , simpleTestCase "Reply data append after reply" $ \cid -> - call' cid (reply >>> replyDataAppend "foo") >>= isReject [5] - - , simpleTestCase "Reply data append after reject" $ \cid -> - call' cid (reject "bar" >>> replyDataAppend "foo") >>= isReject [5] - - , simpleTestCase "Caller" $ \cid -> - call cid (replyData caller) >>= is defaultUser - - , simpleTestCase "Caller (query)" $ \cid -> - query cid (replyData caller) >>= is defaultUser - ] - - , testGroup "Settings" - [ testGroup "Controllers" - [testCase "Multiple controllers (provisional)" $ do - let controllers = [Principal defaultUser, Principal otherUser] - cid <- ic_provisional_create ic00 Nothing (#controllers .== Vec.fromList controllers) - - -- Controllers should be able to fetch the canister status. - cs <- ic_canister_status (ic00as defaultUser) cid - Vec.toList (cs .! #settings .! #controllers) `isSet` controllers - cs <- ic_canister_status (ic00as otherUser) cid - Vec.toList (cs .! #settings .! #controllers) `isSet` controllers - - -- Non-controllers cannot fetch the canister status - ic_canister_status'' ecdsaUser cid >>= isErrOrReject [3, 5] - ic_canister_status'' anonymousUser cid >>= isErrOrReject [3, 5] - ic_canister_status'' secp256k1User cid >>= isErrOrReject [3, 5] - - -- Set new controller - ic_set_controllers (ic00as defaultUser) cid [ecdsaUser] - - -- Only that controller can get canister status - ic_canister_status'' defaultUser cid >>= isErrOrReject [3, 5] - ic_canister_status'' otherUser cid >>= isErrOrReject [3, 5] - ic_canister_status'' anonymousUser cid >>= isErrOrReject [3, 5] - ic_canister_status'' secp256k1User cid >>= isErrOrReject [3, 5] - cs <- ic_canister_status (ic00as ecdsaUser) cid - cs .! #settings .! #controllers @?= Vec.fromList [Principal ecdsaUser] - - , simpleTestCase "Multiple controllers (aaaaa-aa)" $ \cid -> do - let controllers = [Principal cid, Principal otherUser] - cid2 <- ic_create (ic00viaWithCycles cid 20_000_000_000_000) (#controllers .== Vec.fromList controllers) - - -- Controllers should be able to fetch the canister status. - cs <- ic_canister_status (ic00via cid) cid2 - Vec.toList (cs .! #settings .! #controllers) `isSet` controllers - cs <- ic_canister_status (ic00as otherUser) cid2 - Vec.toList (cs .! #settings .! #controllers) `isSet` controllers - - -- Set new controller - ic_set_controllers (ic00via cid) cid2 [ecdsaUser] - - -- Only that controller can get canister status - ic_canister_status'' defaultUser cid2 >>= isErrOrReject [3, 5] - ic_canister_status'' otherUser cid2 >>= isErrOrReject [3, 5] - cs <- ic_canister_status (ic00as ecdsaUser) cid2 - cs .! #settings .! #controllers @?= Vec.fromList [Principal ecdsaUser] - - , simpleTestCase "> 10 controllers" $ \cid -> do - ic_create' (ic00via cid) (#controllers .== Vec.fromList (replicate 11 (Principal cid))) - >>= isReject [3,5] - - , simpleTestCase "No controller" $ \cid -> do - cid2 <- ic_create (ic00viaWithCycles cid 20_000_000_000_000) (#controllers .== Vec.fromList []) - ic_canister_status'' defaultUser cid2 >>= isErrOrReject [3, 5] - ic_canister_status'' otherUser cid2 >>= isErrOrReject [3, 5] - - , testCase "Controller is self" $ do - cid <- install noop - ic_set_controllers ic00 cid [cid] -- Set controller of cid to be itself - - -- cid can now request its own status - cs <- ic_canister_status (ic00via cid) cid - cs .! #settings .! #controllers @?= Vec.fromList [Principal cid] - - , testCase "Duplicate controllers" $ do - let controllers = [Principal defaultUser, Principal defaultUser, Principal otherUser] - cid <- ic_provisional_create ic00 Nothing (#controllers .== Vec.fromList controllers) - cs <- ic_canister_status (ic00as defaultUser) cid - Vec.toList (cs .! #settings .! #controllers) `isSet` controllers - ] - - , simpleTestCase "Valid allocations" $ \cid -> do - cid2 <- ic_create (ic00viaWithCycles cid 20_000_000_000_000) $ empty - .+ #compute_allocation .== (1::Natural) - .+ #memory_allocation .== (1024*1024::Natural) - .+ #freezing_threshold .== 1000_000 - cs <- ic_canister_status (ic00via cid) cid2 - cs .! #settings .! #compute_allocation @?= 1 - cs .! #settings .! #memory_allocation @?= 1024*1024 - cs .! #settings .! #freezing_threshold @?= 1000_000 - - , testGroup "via provisional_create_canister_with_cycles:" - [ testCase "Invalid compute allocation" $ do - ic_provisional_create' ic00 Nothing (#compute_allocation .== 101) - >>= isReject [3,5] - , testCase "Invalid memory allocation (2^49)" $ do - ic_provisional_create' ic00 Nothing (#compute_allocation .== 2^(49::Int)) - >>= isReject [3,5] - , testCase "Invalid memory allocation (2^70)" $ do - ic_provisional_create' ic00 Nothing (#compute_allocation .== 2^(70::Int)) - >>= isReject [3,5] - , testCase "Invalid freezing threshold (2^70)" $ do - ic_provisional_create' ic00 Nothing (#freezing_threshold .== 2^(70::Int)) - >>= isReject [3,5] - ] - , testGroup "via create_canister:" - [ simpleTestCase "Invalid compute allocation" $ \cid -> do - ic_create' (ic00via cid) (#compute_allocation .== 101) - >>= isReject [3,5] - , simpleTestCase "Invalid memory allocation (2^49)" $ \cid -> do - ic_create' (ic00via cid) (#compute_allocation .== 2^(49::Int)) - >>= isReject [3,5] - , simpleTestCase "Invalid memory allocation (2^70)" $ \cid -> do - ic_create' (ic00via cid) (#compute_allocation .== 2^(70::Int)) - >>= isReject [3,5] - , simpleTestCase "Invalid freezing threshold (2^70)" $ \cid -> do - ic_create' (ic00via cid) (#freezing_threshold .== 2^(70::Int)) - >>= isReject [3,5] - ] - , testGroup "via update_settings" - [ simpleTestCase "Invalid compute allocation" $ \cid -> do - ic_update_settings' ic00 cid (#compute_allocation .== 101) - >>= isReject [3,5] - , simpleTestCase "Invalid memory allocation (2^49)" $ \cid -> do - ic_update_settings' ic00 cid (#compute_allocation .== 2^(49::Int)) - >>= isReject [3,5] - , simpleTestCase "Invalid memory allocation (2^70)" $ \cid -> do - ic_update_settings' ic00 cid (#compute_allocation .== 2^(70::Int)) - >>= isReject [3,5] - , simpleTestCase "Invalid freezing threshold (2^70)" $ \cid -> do - ic_update_settings' ic00 cid (#freezing_threshold .== 2^(70::Int)) - >>= isReject [3,5] - ] - ] - - , testGroup "anonymous user" - [ simpleTestCase "update, sender absent fails" $ \cid -> - do envelopeFor anonymousUser $ rec - [ "request_type" =: GText "call" - , "canister_id" =: GBlob cid - , "method_name" =: GText "update" - , "arg" =: GBlob (run (replyData caller)) - ] - >>= postCallCBOR cid >>= code4xx - , simpleTestCase "query, sender absent fails" $ \cid -> - do envelopeFor anonymousUser $ rec - [ "request_type" =: GText "query" - , "canister_id" =: GBlob cid - , "method_name" =: GText "query" - , "arg" =: GBlob (run (replyData caller)) - ] - >>= postQueryCBOR cid >>= code4xx - , simpleTestCase "update, sender explicit" $ \cid -> - do awaitCall cid $ rec - [ "request_type" =: GText "call" - , "canister_id" =: GBlob cid - , "sender" =: GBlob anonymousUser - , "method_name" =: GText "update" - , "arg" =: GBlob (run (replyData caller)) - ] - >>= isReply >>= is anonymousUser - , simpleTestCase "query, sender explicit" $ \cid -> - do queryCBOR cid $ rec - [ "request_type" =: GText "query" - , "canister_id" =: GBlob cid - , "sender" =: GBlob anonymousUser - , "method_name" =: GText "query" - , "arg" =: GBlob (run (replyData caller)) - ] - >>= queryResponse >>= isReply >>= is anonymousUser - ] - - , testGroup "state" - [ simpleTestCase "set/get" $ \cid -> do - call_ cid $ setGlobal "FOO" >>> reply - query cid (replyData getGlobal) >>= is "FOO" - - , simpleTestCase "set/set/get" $ \cid -> do - call_ cid $ setGlobal "FOO" >>> reply - call_ cid $ setGlobal "BAR" >>> reply - query cid (replyData getGlobal) >>= is "BAR" - - , simpleTestCase "resubmission" $ \cid -> do - -- Submits the same request (same nonce) twice, checks that - -- the IC does not act twice. - -- (Using growing stable memory as non-idempotent action) - callTwice' cid (ignore (stableGrow (int 1)) >>> reply) >>= isReply >>= is "" - query cid (replyData (i2b stableSize)) >>= is "\1\0\0\0" - ] - - , testGroup "API availability" $ - {- - This section checks various API calls in various contexts, to see - if they trap when they should - This mirros the table in https://docs.dfinity.systems/public/#system-api-imports - - -} - let - {- - Contexts - - A context is a function of type - (String, Prog -> TestCase, Prog -> TestCase) - building a test for does-not-trap or does-trap - -} - contexts = mconcat - [ "I" =: twoContexts - (reqResponse (\prog -> do - cid <- create - install' cid prog - )) - (reqResponse (\prog -> do - cid <- install noop - upgrade' cid prog - )) - , "G" =: reqResponse (\prog -> do - cid <- install (onPreUpgrade (callback prog)) - upgrade' cid noop - ) - , "U" =: twoContexts - (reqResponse (\prog -> do - cid <- install noop - call' cid (prog >>> reply) - )) - (reqResponse (\prog -> do - cid <- install noop - call cid >=> isRelay $ inter_update cid defArgs{ - other_side = prog >>> reply - } - )) - , "Q" =: twoContexts - (reqResponse (\prog -> do - cid <- install noop - query' cid (prog >>> reply) - )) - (reqResponse (\prog -> do - cid <- install noop - call cid >=> isRelay $ inter_query cid defArgs{ - other_side = prog >>> reply - } - )) - , "Ry" =: reqResponse (\prog -> do - cid <- install noop - call' cid $ inter_query cid defArgs{ - on_reply = prog >>> reply - } - ) - , "Rt" =: reqResponse (\prog -> do - cid <- install noop - call' cid $ inter_query cid defArgs{ - on_reject = prog >>> reply, - other_side = trap "trap!" - } - ) - , "C" =: boolTest (\prog -> do - cid <- install noop - call' cid >=> isReject [5] $ inter_query cid defArgs - { other_side = reply - , on_reply = trap "Trapping in on_reply" - , on_cleanup = Just $ prog >>> setGlobal "Did not trap" - } - g <- query cid $ replyData getGlobal - return (g == "Did not trap") - ) - , "F" =: httpResponse (\prog -> do - cid <- install (onInspectMessage (callback (prog >>> acceptMessage))) - call'' cid reply - ) - ] - - -- context builder helpers - httpResponse act = (act >=> is2xx >=> void . isReply, act >=> isErrOrReject [5]) - reqResponse act = (act >=> void . isReply, act >=> isReject [5]) - boolTest act = (act >=> is True, act >=> is False) - twoContexts (aNT1, aT1) (aNT2, aT2) = (\p -> aNT1 p >> aNT2 p,\p -> aT1 p >> aT2 p) - - -- assembling it all - t name trapping prog - | Just n <- find (not . (`HM.member` contexts)) s - = error $ "Undefined context " ++ T.unpack n - | otherwise = - [ if cname `S.member` s - then testCase (name ++ " works in " ++ T.unpack cname) $ actNT prog - else testCase (name ++ " traps in " ++ T.unpack cname) $ actTrap prog - | (cname, (actNT, actTrap)) <- HM.toList contexts - ] - where s = S.fromList (T.words trapping) - - star = "I G U Q Ry Rt C F" - never = "" - - in concat - [ t "msg_arg_data" "I U Q Ry F" $ ignore argData - , t "msg_caller" "I G U Q F" $ ignore caller - , t "msg_reject_code" "Ry Rt" $ ignore reject_code - , t "msg_reject_msg" "Rt" $ ignore reject_msg - , t "msg_reply_data_append" "U Q Ry Rt" $ replyDataAppend "Hey!" - , t "msg_reply_data_append (\"\")" "U Q Ry Rt" $ replyDataAppend "" - , t "msg_reply" never reply -- due to double reply - , t "msg_reject" never $ reject "rejecting" -- due to double reply - , t "msg_cycles_available" "U Rt Ry" $ ignore getAvailableCycles - , t "msg_cycles_refunded" "Rt Ry" $ ignore getRefund - , t "msg_cycles_accept" "U Rt Ry" $ ignore (acceptCycles (int64 0)) - , t "canister_self" star $ ignore self - , t "canister_cycle_balance" star $ ignore getBalance - , t "call_new…call_perform" "U Rt Ry" $ - callNew "foo" "bar" "baz" "quux" >>> - callDataAppend "foo" >>> - callCyclesAdd (int64 0) >>> - callPerform - , t "call_set_cleanup" never $ callOnCleanup (callback noop) - , t "call_data_append" never $ callDataAppend "foo" - , t "call_cycles_add" never $ callCyclesAdd (int64 0) - , t "call_perform" never callPerform - , t "stable_size" star $ ignore stableSize - , t "stable_grow" star $ ignore $ stableGrow (int 1) - , t "stable_write" star $ stableWrite (int 0) "" - , t "stable_read" star $ ignore $ stableRead (int 0) (int 0) - , t "certified_data_set" "I G U Ry Rt" $ setCertifiedData "foo" - , t "data_certificate_present" star $ ignore getCertificatePresent - , t "msg_method_name" "F" $ ignore methodName - , t "accept_message" never acceptMessage -- due to double accept - , t "time" star $ ignore getTime - , t "debug_print" star $ debugPrint "hello" - , t "trap" never $ trap "this better traps" - ] - - , simpleTestCase "self" $ \cid -> - query cid (replyData self) >>= is cid - - , testGroup "wrong url path" - [ simpleTestCase "call request to query" $ \cid -> do - let req = rec - [ "request_type" =: GText "call" - , "sender" =: GBlob defaultUser - , "canister_id" =: GBlob cid - , "method_name" =: GText "update" - , "arg" =: GBlob (run reply) - ] - addNonceExpiryEnv req >>= postQueryCBOR cid >>= code4xx - - , simpleTestCase "query request to call" $ \cid -> do - let req = rec - [ "request_type" =: GText "query" - , "sender" =: GBlob defaultUser - , "canister_id" =: GBlob cid - , "method_name" =: GText "query" - , "arg" =: GBlob (run reply) - ] - addNonceExpiryEnv req >>= postCallCBOR cid >>= code4xx - - , simpleTestCase "query request to read_state" $ \cid -> do - let req = rec - [ "request_type" =: GText "query" - , "sender" =: GBlob defaultUser - , "canister_id" =: GBlob cid - , "method_name" =: GText "query" - , "arg" =: GBlob (run reply) - ] - addNonceExpiryEnv req >>= postReadStateCBOR cid >>= code4xx - - , simpleTestCase "read_state request to query" $ \cid -> do - addNonceExpiryEnv readStateEmpty >>= postQueryCBOR cid >>= code4xx - ] - - , testGroup "wrong effective canister id" - [ simpleTestCase "in call" $ \cid1 -> do - cid2 <- create - let req = rec - [ "request_type" =: GText "call" - , "sender" =: GBlob defaultUser - , "canister_id" =: GBlob cid1 - , "method_name" =: GText "update" - , "arg" =: GBlob (run reply) - ] - addNonceExpiryEnv req >>= postCallCBOR cid2 >>= code4xx - - , simpleTestCase "in query" $ \cid1 -> do - cid2 <- create - let req = rec - [ "request_type" =: GText "query" - , "sender" =: GBlob defaultUser - , "canister_id" =: GBlob cid1 - , "method_name" =: GText "query" - , "arg" =: GBlob (run reply) - ] - addNonceExpiryEnv req >>= postQueryCBOR cid2 >>= code4xx - - -- read_state tested in read_state group - -- - , simpleTestCase "in mangement call" $ \cid1 -> do - cid2 <- create - let req = rec - [ "request_type" =: GText "call" - , "sender" =: GBlob defaultUser - , "canister_id" =: GBlob "" - , "method_name" =: GText "canister_status" - , "arg" =: GBlob (Candid.encode (#canister_id .== Principal cid1)) - ] - addNonceExpiryEnv req >>= postCallCBOR cid2 >>= code4xx - - , simpleTestCase "non-existing (and likely invalid)" $ \cid1 -> do - let req = rec - [ "request_type" =: GText "call" - , "sender" =: GBlob defaultUser - , "canister_id" =: GBlob cid1 - , "method_name" =: GText "update" - , "arg" =: GBlob (run reply) - ] - addNonceExpiryEnv req >>= postCallCBOR "foobar" >>= code4xx - - , simpleTestCase "invalid textual represenation" $ \cid1 -> do - let req = rec - [ "request_type" =: GText "call" - , "sender" =: GBlob defaultUser - , "canister_id" =: GBlob cid1 - , "method_name" =: GText "update" - , "arg" =: GBlob (run reply) - ] - let path = "/api/v2/canister/" ++ filter (/= '-') (textual cid1) ++ "/call" - addNonceExpiryEnv req >>= postCBOR path >>= code4xx - ] - - , testGroup "inter-canister calls" - [ testGroup "builder interface" - [ testGroup "traps without call_new" - [ simpleTestCase "call_data_append" $ \cid -> - call' cid (callDataAppend "Foo" >>> reply) >>= isReject [5] - , simpleTestCase "call_on_cleanup" $ \cid -> - call' cid (callOnCleanup (callback noop) >>> reply) >>= isReject [5] - , simpleTestCase "call_cycles_add" $ \cid -> - call' cid (callCyclesAdd (int64 0) >>> reply) >>= isReject [5] - , simpleTestCase "call_perform" $ \cid -> - call' cid (callPerform >>> reply) >>= isReject [5] - ] - , simpleTestCase "call_new clears pending call" $ \cid -> do - do call cid $ - callNew "foo" "bar" "baz" "quux" >>> - callDataAppend "hey" >>> - inter_query cid defArgs - >>= isRelay >>= isReply >>= is ("Hello " <> cid <> " this is " <> cid) - , simpleTestCase "call_data_append really appends" $ \cid -> do - do call cid $ - callNew (bytes cid) (bytes "query") - (callback relayReply) (callback relayReject) >>> - callDataAppend (bytes (BS.take 3 (run defaultOtherSide))) >>> - callDataAppend (bytes (BS.drop 3 (run defaultOtherSide))) >>> - callPerform - >>= isRelay >>= isReply >>= is ("Hello " <> cid <> " this is " <> cid) - , simpleTestCase "call_on_cleanup traps if called twice" $ \cid -> - do call' cid $ - callNew (bytes cid) (bytes "query") - (callback relayReply) (callback relayReject) >>> - callOnCleanup (callback noop) >>> - callOnCleanup (callback noop) >>> - reply - >>= isReject [5] - ] - - , simpleTestCase "to nonexistant canister" $ \cid -> - call cid (inter_call "foo" "bar" defArgs) >>= isRelay >>= isReject [3] - - , simpleTestCase "to nonexistant method" $ \cid -> - call cid (inter_call cid "bar" defArgs) >>= isRelay >>= isReject [3] - - , simpleTestCase "Call from query method traps (in update call)" $ \cid -> - callToQuery'' cid (inter_query cid defArgs) >>= is2xx >>= isReject [5] - - , simpleTestCase "Call from query method traps (in query call)" $ \cid -> - query' cid (inter_query cid defArgs) >>= isReject [5] - - , simpleTestCase "Call from query method traps (in inter-canister-call)" $ \cid -> - do call cid $ - inter_call cid "query" defArgs { - other_side = inter_query cid defArgs - } - >>= isRelay >>= isReject [5] - - , simpleTestCase "Self-call (to update)" $ \cid -> - call cid (inter_update cid defArgs) - >>= isRelay >>= isReply >>= is ("Hello " <> cid <> " this is " <> cid) - - , simpleTestCase "Self-call (to query)" $ \cid -> do - call cid (inter_query cid defArgs) - >>= isRelay >>= isReply >>= is ("Hello " <> cid <> " this is " <> cid) - - , simpleTestCase "update commits" $ \cid -> do - do call cid $ - setGlobal "FOO" >>> - inter_update cid defArgs{ other_side = setGlobal "BAR" >>> reply } - >>= isRelay >>= isReply >>= is "" - - query cid (replyData getGlobal) >>= is "BAR" - - , simpleTestCase "query does not commit" $ \cid -> do - do call cid $ - setGlobal "FOO" >>> - inter_query cid defArgs{ other_side = setGlobal "BAR" >>> reply } - >>= isRelay >>= isReply >>= is "" - - do query cid $ replyData getGlobal - >>= is "FOO" - - , simpleTestCase "query no response" $ \cid -> - do call cid $ inter_query cid defArgs{ other_side = noop } - >>= isRelay >>= isReject [5] - - , simpleTestCase "query double reply" $ \cid -> - do call cid $ inter_query cid defArgs{ other_side = reply >>> reply } - >>= isRelay >>= isReject [5] - - , simpleTestCase "Reject code is 0 in reply" $ \cid -> - do call cid $ inter_query cid defArgs{ on_reply = replyData (i2b reject_code) } - >>= asWord32 >>= is 0 - - , simpleTestCase "Second reply in callback" $ \cid -> do - do call cid $ - setGlobal "FOO" >>> - replyData "First reply" >>> - inter_query cid defArgs{ - on_reply = setGlobal "BAR" >>> replyData "Second reply", - on_reject = setGlobal "BAZ" >>> relayReject - } - >>= is "First reply" - - -- now check that the callback trapped and did not actual change the global - -- to make this test reliabe, stop and start the canister, this will - -- ensure all outstanding callbacks are handled - barrier [cid] - - query cid (replyData getGlobal) >>= is "FOO" - - , simpleTestCase "partial reply" $ \cid -> - do call cid $ - replyDataAppend "FOO" >>> - inter_query cid defArgs{ on_reply = replyDataAppend "BAR" >>> reply } - >>= is "BAR" -- check that the FOO is silently dropped - - , simpleTestCase "cleanup not executed when reply callback does not trap" $ \cid -> do - call_ cid $ inter_query cid defArgs - { on_reply = reply - , on_cleanup = Just (setGlobal "BAD") - } - query cid (replyData getGlobal) >>= is "" - - , simpleTestCase "cleanup not executed when reject callback does not trap" $ \cid -> do - call_ cid $ inter_query cid defArgs - { other_side = reject "meh" - , on_reject = reply - , on_cleanup = Just (setGlobal "BAD") - } - query cid (replyData getGlobal) >>= is "" - - , testGroup "two callbacks" - [ simpleTestCase "reply after trap" $ \cid -> - do call cid $ - inter_query cid defArgs{ on_reply = trap "first callback traps" } >>> - inter_query cid defArgs{ on_reply = replyData "good" } - >>= is "good" - - , simpleTestCase "trap after reply" $ \cid -> - do call cid $ - inter_query cid defArgs{ on_reply = replyData "good" } >>> - inter_query cid defArgs{ on_reply = trap "second callback traps" } - >>= is "good" - - , simpleTestCase "both trap" $ \cid -> - do call' cid $ - inter_query cid defArgs{ on_reply = trap "first callback traps" } >>> - inter_query cid defArgs{ on_reply = trap "second callback traps" } - >>= isReject [5] - ] - - , simpleTestCase "Call to other canister (to update)" $ \cid -> do - cid2 <- install noop - do call cid $ inter_update cid2 defArgs - >>= isRelay >>= isReply >>= is ("Hello " <> cid <> " this is " <> cid2) - - , simpleTestCase "Call to other canister (to query)" $ \cid -> do - cid2 <- install noop - do call cid $ inter_query cid2 defArgs - >>= isRelay >>= isReply >>= is ("Hello " <> cid <> " this is " <> cid2) - ] - - , testCaseSteps "stable memory" $ \step -> do - cid <- install noop - - step "Stable mem size is zero" - query cid (replyData (i2b stableSize)) >>= is "\x0\x0\x0\x0" - - step "Writing stable memory (failing)" - call' cid (stableWrite (int 0) "FOO") >>= isReject [5] - step "Set stable mem (failing, query)" - query' cid (stableWrite (int 0) "FOO") >>= isReject [5] - - step "Growing stable memory" - call cid (replyData (i2b (stableGrow (int 1)))) >>= is "\x0\x0\x0\x0" - - step "Growing stable memory again" - call cid (replyData (i2b (stableGrow (int 1)))) >>= is "\x1\x0\x0\x0" - - step "Growing stable memory in query" - query cid (replyData (i2b (stableGrow (int 1)))) >>= is "\x2\x0\x0\x0" - - step "Stable mem size is two" - query cid (replyData (i2b stableSize)) >>= is "\x2\x0\x0\x0" - - step "Writing stable memory" - call_ cid $ stableWrite (int 0) "FOO" >>> reply - - step "Writing stable memory (query)" - query_ cid $ stableWrite (int 0) "BAR" >>> reply - - step "Reading stable memory" - call cid (replyData (stableRead (int 0) (int 3))) >>= is "FOO" - - , testGroup "time" $ - let getTimeTwice = cat (i64tob getTime) (i64tob getTime) in - [ simpleTestCase "in query" $ \cid -> - query cid (replyData getTimeTwice) >>= as2Word64 >>= bothSame - , simpleTestCase "in update" $ \cid -> - query cid (replyData getTimeTwice) >>= as2Word64 >>= bothSame - , testCase "in install" $ do - cid <- install $ setGlobal (getTimeTwice) - query cid (replyData getGlobal) >>= as2Word64 >>= bothSame - , testCase "in pre_upgrade" $ do - cid <- install $ - ignore (stableGrow (int 1)) >>> - onPreUpgrade (callback $ stableWrite (int 0) (getTimeTwice)) - upgrade cid noop - query cid (replyData (stableRead (int 0) (int (2*8)))) >>= as2Word64 >>= bothSame - , simpleTestCase "in post_upgrade" $ \cid -> do - upgrade cid $ setGlobal (getTimeTwice) - query cid (replyData getGlobal) >>= as2Word64 >>= bothSame - ] - - , testGroup "upgrades" $ - let installForUpgrade on_pre_upgrade = install $ - setGlobal "FOO" >>> - ignore (stableGrow (int 1)) >>> - stableWrite (int 0) "BAR______" >>> - onPreUpgrade (callback on_pre_upgrade) - - checkNoUpgrade cid = do - query cid (replyData getGlobal) >>= is "FOO" - query cid (replyData (stableRead (int 0) (int 9))) >>= is "BAR______" - in - [ testCase "succeeding" $ do - -- check that the pre-upgrade hook has access to the old state - cid <- installForUpgrade $ stableWrite (int 3) getGlobal - checkNoUpgrade cid - - upgrade cid $ stableWrite (int 6) (stableRead (int 0) (int 3)) - - query cid (replyData getGlobal) >>= is "" - query cid (replyData (stableRead (int 0) (int 9))) >>= is "BARFOOBAR" - - , testCase "trapping in pre-upgrade" $ do - cid <- installForUpgrade $ trap "trap in pre-upgrade" - checkNoUpgrade cid - - upgrade' cid noop >>= isReject [5] - checkNoUpgrade cid - - , testCase "trapping in pre-upgrade (by calling)" $ do - cid <- installForUpgrade $ trap "trap in pre-upgrade" - call_ cid $ - reply >>> - onPreUpgrade (callback ( - inter_query cid defArgs { other_side = noop } - )) - checkNoUpgrade cid - - upgrade' cid noop >>= isReject [5] - checkNoUpgrade cid - - , testCase "trapping in pre-upgrade (by accessing arg)" $ do - cid <- installForUpgrade $ ignore argData - checkNoUpgrade cid - - upgrade' cid noop >>= isReject [5] - checkNoUpgrade cid - - , testCase "trapping in post-upgrade" $ do - cid <- installForUpgrade $ stableWrite (int 3) getGlobal - checkNoUpgrade cid - - upgrade' cid (trap "trap in post-upgrade") >>= isReject [5] - checkNoUpgrade cid - - , testCase "trapping in post-upgrade (by calling)" $ do - cid <- installForUpgrade $ stableWrite (int 3) getGlobal - checkNoUpgrade cid - - do upgrade' cid $ inter_query cid defArgs{ other_side = noop } - >>= isReject [5] - checkNoUpgrade cid - ] - - , testGroup "reinstall" - [ testCase "succeeding" $ do - cid <- install $ - setGlobal "FOO" >>> - ignore (stableGrow (int 1)) >>> - stableWrite (int 0) "FOO______" - query cid (replyData getGlobal) >>= is "FOO" - query cid (replyData (stableRead (int 0) (int 9))) >>= is "FOO______" - query cid (replyData (i2b stableSize)) >>= asWord32 >>= is 1 - - reinstall cid $ - setGlobal "BAR" >>> - ignore (stableGrow (int 2)) >>> - stableWrite (int 0) "BAR______" - - query cid (replyData getGlobal) >>= is "BAR" - query cid (replyData (stableRead (int 0) (int 9))) >>= is "BAR______" - query cid (replyData (i2b stableSize)) >>= asWord32 >>= is 2 - - reinstall cid noop - - query cid (replyData getGlobal) >>= is "" - query cid (replyData (i2b stableSize)) >>= asWord32 >>= is 0 - - , testCase "trapping" $ do - cid <- install $ - setGlobal "FOO" >>> - ignore (stableGrow (int 1)) >>> - stableWrite (int 0) "FOO______" - query cid (replyData getGlobal) >>= is "FOO" - query cid (replyData (stableRead (int 0) (int 9))) >>= is "FOO______" - query cid (replyData (i2b stableSize)) >>= is "\1\0\0\0" - - reinstall' cid (trap "Trapping the reinstallation") >>= isReject [5] - - query cid (replyData getGlobal) >>= is "FOO" - query cid (replyData (stableRead (int 0) (int 9))) >>= is "FOO______" - query cid (replyData (i2b stableSize)) >>= is "\1\0\0\0" - ] - - , testGroup "uninstall" $ - let inter_management method_name cid on_reply = - callNew "" method_name (callback on_reply) (callback relayReject) >>> - callDataAppend (bytes (Candid.encode (#canister_id .== Principal cid))) >>> - callPerform - - inter_install_code cid wasm_module on_reply = - callNew "" "install_code" (callback on_reply) (callback relayReject) >>> - callDataAppend (bytes (Candid.encode (empty - .+ #canister_id .== Principal cid - .+ #mode .== (enum #install :: InstallMode) - .+ #wasm_module .== wasm_module - .+ #arg .== run (setGlobal "NONE") - ))) >>> - callPerform - in - [ testCase "uninstall empty canister" $ do - cid <- create - cs <- ic_canister_status ic00 cid - cs .! #status @?= enum #running - cs .! #settings .! #controllers @?= Vec.fromList [Principal defaultUser] - cs .! #module_hash @?= Nothing - ic_uninstall ic00 cid - cs <- ic_canister_status ic00 cid - cs .! #status @?= enum #running - cs .! #settings .! #controllers @?= Vec.fromList [Principal defaultUser] - cs .! #module_hash @?= Nothing - - , testCase "uninstall as wrong user" $ do - cid <- create - ic_uninstall'' otherUser cid >>= isErrOrReject [3,5] - - , testCase "uninstall and reinstall wipes state" $ do - cid <- install (setGlobal "FOO") - ic_uninstall ic00 cid - universal_wasm <- getTestWasm "universal_canister" - ic_install ic00 (enum #install) cid universal_wasm (run (setGlobal "BAR")) - query cid (replyData getGlobal) >>= is "BAR" - - , testCase "uninstall and reinstall wipes stable memory" $ do - cid <- install (ignore (stableGrow (int 1)) >>> stableWrite (int 0) "FOO") - ic_uninstall ic00 cid - universal_wasm <- getTestWasm "universal_canister" - ic_install ic00 (enum #install) cid universal_wasm (run (setGlobal "BAR")) - query cid (replyData (i2b stableSize)) >>= asWord32 >>= is 0 - do query cid $ - ignore (stableGrow (int 1)) >>> - replyData (stableRead (int 0) (int 3)) - >>= is "\0\0\0" - do call cid $ - ignore (stableGrow (int 1)) >>> - replyData (stableRead (int 0) (int 3)) - >>= is "\0\0\0" - - , testCase "uninstall and reinstall wipes certified data" $ do - cid <- install $ setCertifiedData "FOO" - query cid (replyData getCertificate) >>= extractCertData cid >>= is "FOO" - ic_uninstall ic00 cid - universal_wasm <- getTestWasm "universal_canister" - ic_install ic00 (enum #install) cid universal_wasm (run noop) - query cid (replyData getCertificate) >>= extractCertData cid >>= is "" - - , simpleTestCase "uninstalled rejects calls" $ \cid -> do - call cid (replyData "Hi") >>= is "Hi" - query cid (replyData "Hi") >>= is "Hi" - ic_uninstall ic00 cid - -- should be http error, due to inspection - call'' cid (replyData "Hi") >>= isErrOrReject [] - query' cid (replyData "Hi") >>= isReject [3] - - , testCase "open call contexts are rejected" $ do - cid1 <- install noop - cid2 <- install noop - ic_set_controllers ic00 cid1 [cid2] - -- Step A: 2 calls 1 - -- Step B: 1 calls 2. Now 2 is waiting on a call from 1 - -- Step C: 2 uninstalls 1 - -- Step D: 2 replies "FOO" - -- What should happen: The system rejects the call from step A - -- What should not happen: The reply "FOO" makes it to canister 2 - -- (This is not a great test, because even if the call context is - -- not rejected by the system during uninstallation it will somehow be rejected - -- later when the callback is delivered to the empty canister. Maybe the reject - -- code will catch it.) - do call cid2 $ inter_update cid1 defArgs - { other_side = - inter_update cid2 defArgs - { other_side = - inter_management "uninstall_code" cid1 $ - replyData "FOO" - } - } - >>= isRelay >>= isReject [4] - - , testCase "deleted call contexts do not prevent stopping" $ do - -- Similar to above, but 2, after uninstalling, imediatelly - -- stops and deletes 1. This can only work if the call - -- contexts at 1 are indeed deleted - cid1 <- install noop - cid2 <- install noop - ic_set_controllers ic00 cid1 [cid2] - do call cid2 $ inter_update cid1 defArgs - { other_side = - inter_update cid2 defArgs - { other_side = - inter_management "uninstall_code" cid1 $ - inter_management "stop_canister" cid1 $ - inter_management "delete_canister" cid1 $ - replyData "FOO" - } - } - >>= isRelay >>= isReject [4] - - -- wait for cid2 to finish its stuff - barrier [cid2] - - -- check that cid1 is deleted - query' cid1 reply >>= isReject [3] - - , testCase "deleted call contexts are not delivered" $ do - -- This is a very tricky one: - -- Like above, but before replying, 2 installs code, - -- calls into 1, so that 1 can call back to 2. This - -- creates a new callback at 1, presumably with the same - -- internal `env` as the one from step B. - -- 2 rejects the new call, and replies to the original one. - -- Only the on_reject callback at 1 should be called, not the on_reply - -- We observe that using the “global” - cid1 <- install noop - cid2 <- install noop - universal_wasm <- getTestWasm "universal_canister" - ic_set_controllers ic00 cid1 [cid2] - do call cid2 $ inter_update cid1 defArgs - { other_side = - inter_update cid2 defArgs - { other_side = - inter_management "uninstall_code" cid1 $ - inter_install_code cid1 universal_wasm $ - inter_update cid1 defArgs { - other_side = - inter_update cid2 defArgs - { other_side = reject "FOO" - , on_reply = setGlobal "REPLY" >>> reply - , on_reject = setGlobal "REJECT" >>> reply - } - } - } - } - >>= isRelay >>= isReject [4] - - -- wait for cid2 to finish its stuff - barrier [cid2] - - -- check cid1’s global - query cid1 (replyData getGlobal) >>= is "REJECT" - ] - - , testGroup "debug facilities" - [ simpleTestCase "Using debug_print" $ \cid -> - call_ cid (debugPrint "ic-ref-test print" >>> reply) - , simpleTestCase "Using debug_print (query)" $ \cid -> - query_ cid $ debugPrint "ic-ref-test print" >>> reply - , simpleTestCase "Using debug_print with invalid bounds" $ \cid -> - query_ cid $ badPrint >>> reply - , simpleTestCase "Explicit trap" $ \cid -> - call' cid (trap "trapping") >>= isReject [5] - , simpleTestCase "Explicit trap (query)" $ \cid -> do - query' cid (trap "trapping") >>= isReject [5] - ] - - , testCase "caller (in init)" $ do - cid <- install $ setGlobal caller - query cid (replyData getGlobal) >>= is defaultUser - - , testCase "self (in init)" $ do - cid <- install $ setGlobal self - query cid (replyData getGlobal) >>= is cid - - , testGroup "trapping in init" $ - let - failInInit pgm = do - cid <- create - install' cid pgm >>= isReject [5] - -- canister does not exist - query' cid noop >>= isReject [3] - in - [ testCase "explicit trap" $ failInInit $ trap "trapping in install" - , testCase "call" $ failInInit $ inter_query "dummy" defArgs - , testCase "reply" $ failInInit reply - , testCase "reject" $ failInInit $ reject "rejecting in init" - ] - - , testGroup "query" - [ testGroup "required fields" $ do - -- TODO: Begin with a succeeding request to a real canister, to rule - -- out other causes of failure than missing fields - omitFields queryToNonExistant $ \req -> do - cid <- create - addExpiry req >>= envelope defaultSK >>= postQueryCBOR cid >>= code4xx - - , simpleTestCase "non-existing (deleted) canister" $ \cid -> do - ic_stop_canister ic00 cid - ic_delete_canister ic00 cid - query' cid reply >>= isReject [3] - - , simpleTestCase "does not commit" $ \cid -> do - call_ cid (setGlobal "FOO" >>> reply) - query cid (setGlobal "BAR" >>> replyData getGlobal) >>= is "BAR" - query cid (replyData getGlobal) >>= is "FOO" - ] - - , testGroup "read state" $ - let ensure_request_exists cid user = do - req <- addNonce >=> addExpiry $ rec - [ "request_type" =: GText "call" - , "sender" =: GBlob user - , "canister_id" =: GBlob cid - , "method_name" =: GText "query" - , "arg" =: GBlob (run (replyData "\xff\xff")) - ] - awaitCall cid req >>= isReply >>= is "\xff\xff" - - -- check that the request is there - getRequestStatus user cid (requestId req) >>= is (Responded (Reply "\xff\xff")) - - return (requestId req) - in - [ testGroup "required fields" $ - omitFields readStateEmpty $ \req -> do - cid <- create - addExpiry req >>= envelope defaultSK >>= postReadStateCBOR cid >>= code4xx - - , simpleTestCase "certificate validates" $ \cid -> do - cert <- getStateCert defaultUser cid [] - validateStateCert cert - - , testCaseSteps "time is present" $ \step -> do - cid <- create - cert <- getStateCert defaultUser cid [] - time <- certValue @Natural cert ["time"] - step $ "Reported time: " ++ show time - - , testCase "time can be asked for" $ do - cid <- create - cert <- getStateCert defaultUser cid [["time"]] - void $ certValue @Natural cert ["time"] - - , testCase "controller of empty canister" $ do - cid <- create - cert <- getStateCert defaultUser cid [["canister", cid, "controllers"]] - certValue @Blob cert ["canister", cid, "controllers"] >>= asCBORBlobList >>= isSet [defaultUser] - - , testCase "module_hash of empty canister" $ do - cid <- create - cert <- getStateCert defaultUser cid [["canister", cid, "module_hash"]] - lookupPath (cert_tree cert) ["canister", cid, "module_hash"] @?= Absent - - , testCase "single controller of installed canister" $ do - cid <- install noop - -- also vary user, just for good measure - cert <- getStateCert anonymousUser cid [["canister", cid, "controllers"]] - certValue @Blob cert ["canister", cid, "controllers"] >>= asCBORBlobList >>= isSet [defaultUser] - - , testCase "multiple controllers of installed canister" $ do - cid <- ic_provisional_create ic00 Nothing (#controllers .== Vec.fromList [Principal defaultUser, Principal otherUser]) - cert <- getStateCert defaultUser cid [["canister", cid, "controllers"]] - certValue @Blob cert ["canister", cid, "controllers"] >>= asCBORBlobList >>= isSet [defaultUser, otherUser] - - , testCase "zero controllers of installed canister" $ do - cid <- ic_provisional_create ic00 Nothing (#controllers .== Vec.fromList []) - cert <- getStateCert defaultUser cid [["canister", cid, "controllers"]] - certValue @Blob cert ["canister", cid, "controllers"] >>= asCBORBlobList >>= isSet [] - - , testCase "module_hash of empty canister" $ do - cid <- install noop - universal_wasm <- getTestWasm "universal_canister" - cert <- getStateCert anonymousUser cid [["canister", cid, "module_hash"]] - certValue @Blob cert ["canister", cid, "module_hash"] >>= is (sha256 universal_wasm) - - , testGroup "non-existence proofs for non-existing request id" - [ simpleTestCase ("rid \"" ++ shorten 8 (asHex rid) ++ "\"") $ \cid -> do - cert <- getStateCert defaultUser cid [["request_status", rid]] - certValueAbsent cert ["request_status", rid, "status"] - | rid <- [ "", BS.replicate 32 0, BS.replicate 32 8, BS.replicate 32 255 ] - ] - - , simpleTestCase "can ask for portion of request status " $ \cid -> do - rid <- ensure_request_exists cid defaultUser - cert <- getStateCert defaultUser cid - [["request_status", rid, "status"], ["request_status", rid, "reply"]] - void $ certValue @T.Text cert ["request_status", rid, "status"] - void $ certValue @Blob cert ["request_status", rid, "reply"] - - , simpleTestCase "access denied for other users request" $ \cid -> do - rid <- ensure_request_exists cid defaultUser - getStateCert' otherUser cid [["request_status", rid]] >>= code4xx - - , simpleTestCase "reading two statuses to same canister in one go" $ \cid -> do - rid1 <- ensure_request_exists cid defaultUser - rid2 <- ensure_request_exists cid defaultUser - cert <- getStateCert defaultUser cid [["request_status", rid1], ["request_status", rid2]] - void $ certValue @T.Text cert ["request_status", rid1, "status"] - void $ certValue @T.Text cert ["request_status", rid2, "status"] - - , simpleTestCase "access denied for other users request (mixed request)" $ \cid -> do - rid1 <- ensure_request_exists cid defaultUser - rid2 <- ensure_request_exists cid otherUser - getStateCert' defaultUser cid [["request_status", rid1], ["request_status", rid2]] >>= code4xx - - , simpleTestCase "access denied two status to different canisters" $ \cid -> do - cid2 <- install noop - rid1 <- ensure_request_exists cid defaultUser - rid2 <- ensure_request_exists cid2 defaultUser - getStateCert' defaultUser cid [["request_status", rid1], ["request_status", rid2]] >>= code4xx - - , simpleTestCase "access denied for bogus path" $ \cid -> do - getStateCert' otherUser cid [["hello", "world"]] >>= code4xx - - , simpleTestCase "access denied for fetching full state tree" $ \cid -> do - getStateCert' otherUser cid [[]] >>= code4xx - ] - - , testGroup "certified variables" - [ simpleTestCase "initially empty" $ \cid -> do - query cid (replyData getCertificate) >>= extractCertData cid >>= is "" - , simpleTestCase "validates" $ \cid -> do - query cid (replyData getCertificate) - >>= decodeCert' >>= validateStateCert - , simpleTestCase "present in query method (query call)" $ \cid -> do - query cid (replyData (i2b getCertificatePresent)) - >>= is "\1\0\0\0" - , simpleTestCase "not present in query method (update call)" $ \cid -> do - callToQuery'' cid (replyData (i2b getCertificatePresent)) - >>= is2xx >>= isReply >>= is "\0\0\0\0" - , simpleTestCase "not present in query method (inter-canister call)" $ \cid -> do - do call cid $ - inter_call cid "query" defArgs { - other_side = replyData (i2b getCertificatePresent) - } - >>= isRelay >>= isReply >>= is "\0\0\0\0" - , simpleTestCase "not present in update method" $ \cid -> do - call cid (replyData (i2b getCertificatePresent)) - >>= is "\0\0\0\0" - - , simpleTestCase "set and get" $ \cid -> do - call_ cid $ setCertifiedData "FOO" >>> reply - query cid (replyData getCertificate) >>= extractCertData cid >>= is "FOO" - , simpleTestCase "set twice" $ \cid -> do - call_ cid $ setCertifiedData "FOO" >>> setCertifiedData "BAR" >>> reply - query cid (replyData getCertificate) >>= extractCertData cid >>= is "BAR" - , simpleTestCase "set then trap" $ \cid -> do - call_ cid $ setCertifiedData "FOO" >>> reply - call' cid (setCertifiedData "BAR" >>> trap "Trapped") >>= isReject [5] - query cid (replyData getCertificate) >>= extractCertData cid >>= is "FOO" - , simpleTestCase "too large traps, old value retained" $ \cid -> do - call_ cid $ setCertifiedData "FOO" >>> reply - call' cid (setCertifiedData (bytes (BS.replicate 33 0x42)) >>> reply) - >>= isReject [5] - query cid (replyData getCertificate) >>= extractCertData cid >>= is "FOO" - , testCase "set in init" $ do - cid <- install $ setCertifiedData "FOO" - query cid (replyData getCertificate) >>= extractCertData cid >>= is "FOO" - , testCase "set in pre-upgrade" $ do - cid <- install $ onPreUpgrade (callback $ setCertifiedData "FOO") - upgrade cid noop - query cid (replyData getCertificate) >>= extractCertData cid >>= is "FOO" - , simpleTestCase "set in post-upgrade" $ \cid -> do - upgrade cid $ setCertifiedData "FOO" - query cid (replyData getCertificate) >>= extractCertData cid >>= is "FOO" - ] - - , testGroup "cycles" $ - let replyBalance = replyData (i64tob getBalance) - rememberBalance = - ignore (stableGrow (int 1)) >>> - stableWrite (int 0) (i64tob getBalance) - recallBalance = replyData (stableRead (int 0) (int 8)) - acceptAll = ignore (acceptCycles getAvailableCycles) - queryBalance cid = query cid replyBalance >>= asWord64 - - -- At the time of writing, creating a canister needs at least 1T - -- and the freezing limit is 5T - -- (At some point, the max was 100T, but that is no longer the case) - -- So lets try to stay away from these limits. - -- The lowest denomination we deal with below is def_cycles`div`4 - def_cycles = 80_000_000_000_000 :: Word64 - - -- The system burns cycles at unspecified rates. To cater for such behaviour, - -- we make the assumption that no test burns more than the following epsilon. - -- - -- The biggest fee we currenlty deal with is the system deducing 1T - -- upon canister creation. So our epsilon needs to allow that and then - -- some more. - eps = 3_000_000_000_000 :: Integer - - isRoughly :: (HasCallStack, Show a, Num a, Integral a) => a -> a -> Assertion - isRoughly exp act = assertBool - (show act ++ " not near " ++ show exp) - (abs (fromIntegral exp - fromIntegral act) < eps) - - create prog = do - cid <- ic_provisional_create ic00 (Just (fromIntegral def_cycles)) empty - installAt cid prog - return cid - create_via cid initial_cycles = do - cid2 <- ic_create (ic00viaWithCycles cid initial_cycles) empty - universal_wasm <- getTestWasm "universal_canister" - ic_install (ic00via cid) (enum #install) cid2 universal_wasm (run noop) - return cid2 - in - [ testGroup "can use balance API" $ - let getBalanceTwice = join cat (i64tob getBalance) - test = replyData getBalanceTwice - in - [ simpleTestCase "in query" $ \cid -> - query cid test >>= as2Word64 >>= bothSame - , simpleTestCase "in update" $ \cid -> - call cid test >>= as2Word64 >>= bothSame - , testCase "in init" $ do - cid <- install (setGlobal getBalanceTwice) - query cid (replyData getGlobal) >>= as2Word64 >>= bothSame - , simpleTestCase "in callback" $ \cid -> - call cid (inter_query cid defArgs{ on_reply = test }) >>= as2Word64 >>= bothSame - ] - , testGroup "can use available cycles API" $ - let getAvailableCyclesTwice = join cat (i64tob getAvailableCycles) - test = replyData getAvailableCyclesTwice - in - [ simpleTestCase "in update" $ \cid -> - call cid test >>= as2Word64 >>= bothSame - , simpleTestCase "in callback" $ \cid -> - call cid (inter_query cid defArgs{ on_reply = test }) >>= as2Word64 >>= bothSame - ] - , simpleTestCase "can accept zero cycles" $ \cid -> - call cid (replyData (i64tob (acceptCycles (int64 0)))) >>= asWord64 >>= is 0 - , simpleTestCase "can accept more than available cycles" $ \cid -> - call cid (replyData (i64tob (acceptCycles (int64 1)))) >>= asWord64 >>= is 0 - , simpleTestCase "cant accept absurd amount of cycles" $ \cid -> - call cid (replyData (i64tob (acceptCycles (int64 maxBound)))) >>= asWord64 >>= is 0 - - , testGroup "provisional_create_canister_with_cycles" - [ testCase "balance as expected" $ do - cid <- create noop - queryBalance cid >>= isRoughly def_cycles - - , testCaseSteps "default (i.e. max) balance" $ \step -> do - cid <- ic_provisional_create ic00 Nothing empty - installAt cid noop - cycles <- queryBalance cid - step $ "Cycle balance now at " ++ show cycles - - , testCaseSteps "> 2^128 succeeds" $ \step -> do - cid <- ic_provisional_create ic00 (Just (10 * 2^(128::Int))) empty - installAt cid noop - cycles <- queryBalance cid - step $ "Cycle balance now at " ++ show cycles - ] - - , testCase "cycles in canister_status" $ do - cid <- create noop - cs <- ic_canister_status ic00 cid - isRoughly (fromIntegral def_cycles) (cs .! #cycles) - - , testGroup "cycle balance" - [ testCase "install" $ do - cid <- create rememberBalance - query cid recallBalance >>= asWord64 >>= isRoughly def_cycles - , testCase "update" $ do - cid <- create noop - call cid replyBalance >>= asWord64 >>= isRoughly def_cycles - , testCase "query" $ do - cid <- create noop - query cid replyBalance >>= asWord64 >>= isRoughly def_cycles - , testCase "in pre_upgrade" $ do - cid <- create $ onPreUpgrade (callback rememberBalance) - upgrade cid noop - query cid recallBalance >>= asWord64 >>= isRoughly def_cycles - , testCase "in post_upgrade" $ do - cid <- create noop - upgrade cid rememberBalance - query cid recallBalance >>= asWord64 >>= isRoughly def_cycles - queryBalance cid >>= isRoughly def_cycles - ] - , testCase "can send cycles" $ do - cid1 <- create noop - cid2 <- create noop - do call cid1 $ inter_call cid2 "update" defArgs - { other_side = - replyDataAppend (i64tob getAvailableCycles) >>> - acceptAll >>> - reply - , cycles = def_cycles `div` 4 - } - >>= isRelay >>= isReply >>= asWord64 >>= isRoughly (def_cycles `div` 4) - queryBalance cid1 >>= isRoughly (def_cycles - def_cycles `div` 4) - queryBalance cid2 >>= isRoughly (def_cycles + def_cycles `div` 4) - - , testCase "sending more cycles than in balance traps" $ do - cid <- create noop - cycles <- queryBalance cid - call' cid (inter_call cid cid defArgs { cycles = cycles + 1000_000 }) - >>= isReject [5] - - , testCase "relay cycles before accept traps" $ do - cid1 <- create noop - cid2 <- create noop - cid3 <- create noop - do call cid1 $ inter_call cid2 "update" defArgs - { cycles = def_cycles `div` 2 - , other_side = - inter_call cid3 "update" defArgs - { other_side = acceptAll >>> reply - , cycles = def_cycles + def_cycles `div` 4 - , on_reply = noop -- must not double reply - } >>> - acceptAll >>> reply - , on_reply = trap "unexpected reply" - , on_reject = replyData (i64tob getRefund) - } - >>= asWord64 >>= isRoughly (def_cycles `div` 2) - queryBalance cid1 >>= isRoughly def_cycles - queryBalance cid2 >>= isRoughly def_cycles - queryBalance cid3 >>= isRoughly def_cycles - , testCase "relay cycles after accept works" $ do - cid1 <- create noop - cid2 <- create noop - cid3 <- create noop - do call cid1 $ inter_call cid2 "update" defArgs - { cycles = def_cycles `div` 2 - , other_side = - acceptAll >>> - inter_call cid3 "update" defArgs - { other_side = acceptAll >>> reply - , cycles = def_cycles + def_cycles `div` 4 - } - , on_reply = replyData (i64tob getRefund) - , on_reject = trap "unexpected reject" - } - >>= asWord64 >>= isRoughly 0 - queryBalance cid1 >>= isRoughly (def_cycles `div` 2) - queryBalance cid2 >>= isRoughly (def_cycles `div` 4) - queryBalance cid3 >>= isRoughly (2*def_cycles + def_cycles `div` 4) - , testCase "aborting call resets balance" $ do - cid <- create noop - (a,b) <- do - call cid $ - callNew "Foo" "Bar" "baz" "quux" >>> - callCyclesAdd (int64 (def_cycles `div` 2)) >>> - replyDataAppend (i64tob getBalance) >>> - callNew "Foo" "Bar" "baz" "quux" >>> - replyDataAppend (i64tob getBalance) >>> - reply - >>= as2Word64 - isRoughly (def_cycles `div` 2) a - isRoughly def_cycles b - - , testCase "partial refund" $ do - cid1 <- create noop - cid2 <- create noop - do call cid1 $ inter_call cid2 "update" defArgs - { cycles = def_cycles `div` 2 - , other_side = ignore (acceptCycles (int64 (def_cycles `div` 4))) >>> reply - , on_reply = replyData (i64tob getRefund) - , on_reject = trap "unexpected reject" - } - >>= asWord64 >>= isRoughly (def_cycles `div` 4) - queryBalance cid1 >>= isRoughly (def_cycles - def_cycles `div` 4) - queryBalance cid2 >>= isRoughly (def_cycles + def_cycles `div` 4) - , testCase "cycles not in balance while in transit" $ do - cid1 <- create noop - do call cid1 $ inter_call cid1 "update" defArgs - { cycles = def_cycles `div` 4 - , other_side = replyBalance - , on_reject = trap "unexpected reject" - } - >>= isRelay >>= isReply >>= asWord64 >>= isRoughly (def_cycles - def_cycles `div` 4) - queryBalance cid1 >>= isRoughly def_cycles - , testCase "create and delete canister with cycles" $ do - cid1 <- create noop - cid2 <- create_via cid1 (def_cycles`div`2) - queryBalance cid1 >>= isRoughly (def_cycles `div` 2) - queryBalance cid2 >>= isRoughly (def_cycles `div` 2) - ic_stop_canister (ic00via cid1) cid2 - -- We load some cycles on the deletion call, just to check that they are refunded - ic_delete_canister (ic00viaWithCycles cid1 (def_cycles`div`4)) cid2 - queryBalance cid1 >>= isRoughly (def_cycles`div`2) - - , testGroup "deposit_cycles" - [ testCase "as controller" $ do - cid1 <- create noop - cid2 <- create_via cid1 (def_cycles`div`2) - queryBalance cid1 >>= isRoughly (def_cycles `div` 2) - queryBalance cid2 >>= isRoughly (def_cycles `div` 2) - ic_deposit_cycles (ic00viaWithCycles cid1 (def_cycles`div`4)) cid2 - queryBalance cid1 >>= isRoughly (def_cycles `div` 4) - queryBalance cid2 >>= isRoughly (def_cycles - def_cycles `div` 4) - , testCase "as other non-controlling canister" $ do - cid1 <- create noop - cid2 <- create_via cid1 (def_cycles`div`2) - queryBalance cid1 >>= isRoughly (def_cycles `div` 2) - queryBalance cid2 >>= isRoughly (def_cycles `div` 2) - ic_deposit_cycles (ic00viaWithCycles cid2 (def_cycles`div`4)) cid1 - queryBalance cid1 >>= isRoughly (def_cycles - def_cycles `div` 4) - queryBalance cid2 >>= isRoughly (def_cycles `div` 4) - , testCase "to non-existing canister" $ do - cid1 <- create noop - queryBalance cid1 >>= isRoughly def_cycles - ic_deposit_cycles' (ic00viaWithCycles cid1 (def_cycles`div`4)) doesn'tExist - >>= isReject [3,4,5] - queryBalance cid1 >>= isRoughly def_cycles - ] - - , testCase "two-step-refund" $ do - cid1 <- create noop - do call cid1 $ inter_call cid1 "update" defArgs - { cycles = 10 - , other_side = inter_call cid1 "update" defArgs - { cycles = 5 - , other_side = reply -- no accept - , on_reply = - -- remember refund - replyDataAppend (i64tob getRefund) >>> - reply - , on_reject = trap "unexpected reject" - } - , on_reply = - -- remember the refund above and this refund - replyDataAppend argData >>> - replyDataAppend (i64tob getRefund) >>> - reply - , on_reject = trap "unexpected reject" - } - >>= as2Word64 >>= is (5,10) - queryBalance cid1 >>= isRoughly def_cycles -- nothing lost? - - , testGroup "provisional top up" - [ testCase "as user" $ do - cid <- create noop - queryBalance cid >>= isRoughly def_cycles - ic_top_up ic00 cid (fromIntegral def_cycles) - queryBalance cid >>= isRoughly (2 * def_cycles) - , testCase "as self" $ do - cid <- create noop - queryBalance cid >>= isRoughly def_cycles - ic_top_up (ic00via cid) cid (fromIntegral def_cycles) - queryBalance cid >>= isRoughly (2 * def_cycles) - , testCase "as other canister" $ do - cid <- create noop - cid2 <- create noop - queryBalance cid >>= isRoughly def_cycles - ic_top_up (ic00via cid2) cid (fromIntegral def_cycles) - queryBalance cid >>= isRoughly (2 * def_cycles) - , testCaseSteps "more than 2^128" $ \step -> do - cid <- create noop - ic_top_up ic00 cid (10 * 2^(128::Int)) - cycles <- queryBalance cid - step $ "Cycle balance now at " ++ show cycles - , testCase "nonexisting canister" $ do - ic_top_up' ic00 doesn'tExist (fromIntegral def_cycles) - >>= isReject [3,5] - ] - ] - - , testGroup "canister_inspect_message" - [ testCase "empty canister" $ do - cid <- create - call'' cid reply >>= isErrOrReject [] - callToQuery'' cid reply >>= isErrOrReject [] - - , testCase "accept all" $ do - cid <- install $ onInspectMessage $ callback acceptMessage - call_ cid reply - callToQuery'' cid reply >>= is2xx >>= isReply >>= is "" - - , testCase "no accept_message" $ do - cid <- install $ onInspectMessage $ callback noop - call'' cid reply >>= isErrOrReject [] - callToQuery'' cid reply >>= isErrOrReject [] - -- check that inter-canister calls still work - cid2 <- install noop - call cid2 (inter_update cid defArgs) - >>= isRelay >>= isReply >>= is ("Hello " <> cid2 <> " this is " <> cid) - - , testCase "two calls to accept_message" $ do - cid <- install $ onInspectMessage $ callback $ acceptMessage >>> acceptMessage - call'' cid reply >>= isErrOrReject [] - callToQuery'' cid reply >>= isErrOrReject [] - - , testCase "trap" $ do - cid <- install $ onInspectMessage $ callback $ trap "no no no" - call'' cid reply >>= isErrOrReject [] - callToQuery'' cid reply >>= isErrOrReject [] - - , testCase "method_name correct" $ do - cid <- install $ onInspectMessage $ callback $ - trapIfEq methodName "update" "no no no" >>> acceptMessage - - call'' cid reply >>= isErrOrReject [] - callToQuery'' cid reply >>= is2xx >>= isReply >>= is "" - - , testCase "caller correct" $ do - cid <- install $ onInspectMessage $ callback $ - trapIfEq caller (bytes defaultUser) "no no no" >>> acceptMessage - - call'' cid reply >>= isErrOrReject [] - callToQuery'' cid reply >>= isErrOrReject [] - - awaitCall' cid (callRequestAs otherUser cid reply) - >>= is2xx >>= isReply >>= is "" - awaitCall' cid (callToQueryRequestAs otherUser cid reply) - >>= is2xx >>= isReply >>= is "" - - , testCase "arg correct" $ do - cid <- install $ onInspectMessage $ callback $ - trapIfEq argData (callback reply) "no no no" >>> acceptMessage - - call'' cid reply >>= isErrOrReject [] - callToQuery'' cid reply >>= isErrOrReject [] - - call cid (replyData "foo") >>= is "foo" - callToQuery'' cid (replyData "foo") >>= is2xx >>= isReply >>= is "foo" - - , testCase "management canister: raw_rand not accepted" $ do - ic_raw_rand'' defaultUser >>= isErrOrReject [] - - , simpleTestCase "management canister: deposit_cycles not accepted" $ \cid -> do - ic_deposit_cycles'' defaultUser cid >>= isErrOrReject [] - - , simpleTestCase "management canister: wrong sender not accepted" $ \cid -> do - ic_canister_status'' otherUser cid >>= isErrOrReject [] - ] - - , testGroup "Delegation targets" $ let - callReq cid = rec - [ "request_type" =: GText "call" - , "sender" =: GBlob defaultUser - , "canister_id" =: GBlob cid - , "method_name" =: GText "update" - , "arg" =: GBlob (run reply) - ] - - mgmtReq cid = rec - [ "request_type" =: GText "call" - , "sender" =: GBlob defaultUser - , "canister_id" =: GBlob "" - , "method_name" =: GText "canister_status" - , "arg" =: GBlob (Candid.encode (#canister_id .== Principal cid)) - ] - - good cid req dels = do - req <- addExpiry req - let rid = requestId req - -- sign request with delegations - delegationEnv defaultSK dels req >>= postCallCBOR cid >>= code2xx - -- wait for it - void $ awaitStatus defaultUser cid rid >>= isReply - -- also read status with delegation - sreq <- addExpiry $ rec - [ "request_type" =: GText "read_state" - , "sender" =: GBlob defaultUser - , "paths" =: GList [GList [GBlob "request_status", GBlob rid]] - ] - delegationEnv defaultSK dels sreq >>= postReadStateCBOR cid >>= void . code2xx - - badSubmit cid req dels = do - req <- addExpiry req - -- sign request with delegations (should fail) - delegationEnv defaultSK dels req >>= postCallCBOR cid >>= code4xx - - badRead cid req dels = do - req <- addExpiry req - let rid = requestId req - -- submit with plain signature - envelope defaultSK req >>= postCallCBOR cid >>= code202 - -- wait for it - void $ awaitStatus defaultUser cid rid >>= isReply - -- also read status with delegation - sreq <- addExpiry $ rec - [ "request_type" =: GText "read_state" - , "sender" =: GBlob defaultUser - , "paths" =: GList [GList [GBlob "request_status", GBlob rid]] - ] - delegationEnv defaultSK dels sreq >>= postReadStateCBOR cid >>= void . code4xx - - goodTestCase name mkReq mkDels = - simpleTestCase name $ \cid -> good cid (mkReq cid) (mkDels cid) - - badTestCase name mkReq mkDels = testGroup name - [ simpleTestCase "in submit" $ \cid -> badSubmit cid (mkReq cid) (mkDels cid) - , simpleTestCase "in read_state" $ \cid -> badRead cid (mkReq cid) (mkDels cid) - ] - - withEd25519 = zip [createSecretKeyEd25519 (BS.singleton n) | n <- [0..]] - withWebAuthn = zip [createSecretKeyWebAuthn (BS.singleton n) | n <- [0..]] - - in - [ goodTestCase "one delegation, singleton target" callReq $ \cid -> - withEd25519 [Just [cid]] - , badTestCase "one delegation, wrong singleton target" callReq $ \_cid -> - withEd25519 [Just [doesn'tExist]] - , goodTestCase "one delegation, two targets" callReq $ \cid -> - withEd25519 [Just [cid, doesn'tExist]] - , goodTestCase "two delegations, two targets, webauthn" callReq $ \cid -> - withWebAuthn [Just [cid, doesn'tExist], Just [cid, doesn'tExist]] - , goodTestCase "one delegation, redundant targets" callReq $ \cid -> - withEd25519 [Just [cid, cid, doesn'tExist]] - , goodTestCase "two delegations, singletons" callReq $ \cid -> - withEd25519 [Just [cid], Just [cid] ] - , goodTestCase "two delegations, first restricted" callReq $ \cid -> - withEd25519 [Just [cid], Nothing ] - , goodTestCase "two delegations, second restricted" callReq $ \cid -> - withEd25519 [Nothing, Just [cid]] - , badTestCase "two delegations, empty intersection" callReq $ \cid -> - withEd25519 [Just [cid], Just [doesn'tExist]] - , badTestCase "two delegations, first empty target set" callReq $ \cid -> - withEd25519 [Just [], Just [cid]] - , badTestCase "two delegations, second empty target set" callReq $ \cid -> - withEd25519 [Just [cid], Just []] - , goodTestCase "management canister: correct target" mgmtReq $ \_cid -> - withEd25519 [Just [""]] - , badTestCase "management canister: empty target set" mgmtReq $ \_cid -> - withEd25519 [Just []] - , badTestCase "management canister: bogus target" mgmtReq $ \_cid -> - withEd25519 [Just [doesn'tExist]] - , badTestCase "management canister: bogus target (using target canister)" mgmtReq $ \cid -> - withEd25519 [Just [cid]] - ] - - , testGroup "Authentication schemes" $ - let ed25519SK2 = createSecretKeyEd25519 "more keys" - ed25519SK3 = createSecretKeyEd25519 "yet more keys" - ed25519SK4 = createSecretKeyEd25519 "even more keys" - delEnv sks = delegationEnv otherSK (map (, Nothing) sks) -- no targets in these tests - in flip foldMap - [ ("Ed25519", otherUser, envelope otherSK) - , ("ECDSA", ecdsaUser, envelope ecdsaSK) - , ("secp256k1", secp256k1User, envelope secp256k1SK) - , ("WebAuthn", webAuthnUser, envelope webAuthnSK) - , ("empty delegations", otherUser, delEnv []) - , ("same delegations", otherUser, delEnv [otherSK]) - , ("three delegations", otherUser, delEnv [ed25519SK2, ed25519SK3]) - , ("four delegations", otherUser, delEnv [ed25519SK2, ed25519SK3, ed25519SK4]) - , ("mixed delegations", otherUser, delEnv [defaultSK, webAuthnSK, ecdsaSK, secp256k1SK]) - ] $ \ (name, user, env) -> - [ simpleTestCase (name ++ " in query") $ \cid -> do - req <- addExpiry $ rec - [ "request_type" =: GText "query" - , "sender" =: GBlob user - , "canister_id" =: GBlob cid - , "method_name" =: GText "query" - , "arg" =: GBlob (run reply) - ] - signed_req <- env req - postQueryCBOR cid signed_req >>= okCBOR >>= queryResponse >>= isReply >>= is "" - - , simpleTestCase (name ++ " in update") $ \cid -> do - req <- addExpiry $ rec - [ "request_type" =: GText "call" - , "sender" =: GBlob user - , "canister_id" =: GBlob cid - , "method_name" =: GText "update" - , "arg" =: GBlob (run reply) - ] - signed_req <- env req - postCallCBOR cid signed_req >>= code2xx - - awaitStatus user cid (requestId req) >>= isReply >>= is "" - ] - - , testGroup "signature checking" $ - [ ("with bad signature", return . badEnvelope, id) - , ("with wrong key", envelope otherSK, id) - , ("with empty domain separator", noDomainSepEnv defaultSK, id) - , ("with no expiry", envelope defaultSK, noExpiryEnv) - , ("with expiry in the past", envelope defaultSK, pastExpiryEnv) - , ("with expiry in the future", envelope defaultSK, futureExpiryEnv) - ] <&> \(name, env, mod_req) -> testGroup name - [ simpleTestCase "in query" $ \cid -> do - good_req <- addNonce >=> addExpiry $ rec - [ "request_type" =: GText "query" - , "sender" =: GBlob defaultUser - , "canister_id" =: GBlob cid - , "method_name" =: GText "query" - , "arg" =: GBlob (run reply) - ] - queryCBOR cid good_req >>= queryResponse >>= isReply >>= is "" - env (mod_req good_req) >>= postQueryCBOR cid >>= code4xx - - , simpleTestCase "in empty read state request" $ \cid -> do - good_req <- addNonce >=> addExpiry $ readStateEmpty - envelope defaultSK good_req >>= postReadStateCBOR cid >>= code2xx - env (mod_req good_req) >>= postReadStateCBOR cid >>= code4xx - - , simpleTestCase "in call" $ \cid -> do - good_req <- addNonce >=> addExpiry $ rec - [ "request_type" =: GText "call" - , "sender" =: GBlob defaultUser - , "canister_id" =: GBlob cid - , "method_name" =: GText "query" - , "arg" =: GBlob (run reply) - ] - let req = mod_req good_req - env req >>= postCallCBOR cid >>= code202_or_4xx - - -- Also check that the request was not created - ingressDelay - getRequestStatus defaultUser cid (requestId req) >>= is UnknownStatus - - -- check that with a valid signature, this would have worked - awaitCall cid good_req >>= isReply >>= is "" - ] - - , testGroup "Canister signatures" $ - let genId cid seed = - DER.encode DER.CanisterSig $ CanisterSig.genPublicKey (EntityId cid) seed - - genSig cid seed msg = do - -- Create the tree - let tree = construct $ - SubTrees $ M.singleton "sig" $ - SubTrees $ M.singleton (sha256 seed) $ - SubTrees $ M.singleton (sha256 msg) $ - Value "" - -- Store it as certified data - call_ cid (setCertifiedData (bytes (reconstruct tree)) >>> reply) - -- Get certificate - cert <- query cid (replyData getCertificate) >>= decodeCert' - -- double check it certifies - validateStateCert cert - certValue cert ["canister", cid, "certified_data"] >>= is (reconstruct tree) - - return $ CanisterSig.genSig cert tree - - exampleQuery cid userKey = addExpiry $ rec - [ "request_type" =: GText "query" - , "sender" =: GBlob (mkSelfAuthenticatingId userKey) - , "canister_id" =: GBlob cid - , "method_name" =: GText "query" - , "arg" =: GBlob (run (replyData "It works!")) - ] - simpleEnv userKey sig req delegations = rec $ - [ "sender_pubkey" =: GBlob userKey - , "sender_sig" =: GBlob sig - , "content" =: req - ] ++ - [ "sender_delegation" =: GList delegations | not (null delegations) ] - in - [ simpleTestCase "direct signature" $ \cid -> do - let userKey = genId cid "Hello!" - req <- exampleQuery cid userKey - sig <- genSig cid "Hello!" $ "\x0Aic-request" <> requestId req - let env = simpleEnv userKey sig req [] - postQueryCBOR cid env >>= okCBOR >>= queryResponse >>= isReply >>= is "It works!" - - , simpleTestCase "direct signature (empty seed)" $ \cid -> do - let userKey = genId cid "" - req <- exampleQuery cid userKey - sig <- genSig cid "" $ "\x0Aic-request" <> requestId req - let env = simpleEnv userKey sig req [] - postQueryCBOR cid env >>= okCBOR >>= queryResponse >>= isReply >>= is "It works!" - - , simpleTestCase "direct signature (wrong seed)" $ \cid -> do - let userKey = genId cid "Hello" - req <- exampleQuery cid userKey - sig <- genSig cid "Hullo" $ "\x0Aic-request" <> requestId req - let env = simpleEnv userKey sig req [] - postQueryCBOR cid env >>= code4xx - - , simpleTestCase "direct signature (wrong cid)" $ \cid -> do - let userKey = genId doesn'tExist "Hello" - req <- exampleQuery cid userKey - sig <- genSig cid "Hello" $ "\x0Aic-request" <> requestId req - let env = simpleEnv userKey sig req [] - postQueryCBOR cid env >>= code4xx - - , simpleTestCase "direct signature (wrong root key)" $ \cid -> do - let seed = "Hello" - let userKey = genId cid seed - req <- exampleQuery cid userKey - let msg = "\x0Aic-request" <> requestId req - -- Create the tree - let tree = construct $ - SubTrees $ M.singleton "sig" $ - SubTrees $ M.singleton (sha256 seed) $ - SubTrees $ M.singleton (sha256 msg) $ - Value "" - -- Create a fake certificate - let cert_tree = construct $ - SubTrees $ M.singleton "canister" $ - SubTrees $ M.singleton cid $ - SubTrees $ M.singleton "certified_data" $ - Value (reconstruct tree) - let fake_root_key = createSecretKeyBLS "not the root key" - cert_sig <- sign "ic-state-root" fake_root_key (reconstruct cert_tree) - let cert = Certificate { cert_tree, cert_sig, cert_delegation = Nothing } - let sig = CanisterSig.genSig cert tree - let env = simpleEnv userKey sig req [] - postQueryCBOR cid env >>= code4xx - - , simpleTestCase "delegation to Ed25519" $ \cid -> do - let userKey = genId cid "Hello!" - - t <- getPOSIXTime - let expiry = round ((t + 60) * 1000_000_000) - let delegation = rec - [ "pubkey" =: GBlob (toPublicKey otherSK) - , "expiration" =: GNat expiry - ] - sig <- genSig cid "Hello!" $ "\x1Aic-request-auth-delegation" <> requestId delegation - let signed_delegation = rec [ "delegation" =: delegation, "signature" =: GBlob sig ] - - req <- exampleQuery cid userKey - sig <- sign "ic-request" otherSK (requestId req) - let env = simpleEnv userKey sig req [signed_delegation] - postQueryCBOR cid env >>= okCBOR >>= queryResponse >>= isReply >>= is "It works!" - - , simpleTestCase "delegation from Ed25519" $ \cid -> do - let userKey = genId cid "Hello!" - - t <- getPOSIXTime - let expiry = round ((t + 60) * 1000_000_000) - let delegation = rec - [ "pubkey" =: GBlob userKey - , "expiration" =: GNat expiry - ] - sig <- sign "ic-request-auth-delegation" otherSK (requestId delegation) - let signed_delegation = rec [ "delegation" =: delegation, "signature" =: GBlob sig ] - - req <- addExpiry $ rec - [ "request_type" =: GText "query" - , "sender" =: GBlob otherUser - , "canister_id" =: GBlob cid - , "method_name" =: GText "query" - , "arg" =: GBlob (run (replyData "It works!")) - ] - sig <- genSig cid "Hello!" $ "\x0Aic-request" <> requestId req - let env = simpleEnv (toPublicKey otherSK) sig req [signed_delegation] - postQueryCBOR cid env >>= okCBOR >>= queryResponse >>= isReply >>= is "It works!" - - ] - ] - -type Blob = BS.ByteString - --- * HUnit error reporting integration - --- To use IC.HTTP.CBOR with HUnit -instance Parse IO where parseError = assertFailure . T.unpack - -asRight :: HasCallStack => Either T.Text a -> IO a -asRight (Left err) = assertFailure (T.unpack err) -asRight (Right gr) = return gr - --- * Requests - --- | Posting a CBOR request, returning a raw bytestring -postCBOR :: (HasCallStack, HasTestConfig) => String -> GenR -> IO (Response BS.ByteString) -postCBOR path gr = do - request <- parseRequest $ endPoint ++ path - request <- return $ request - { method = "POST" - , requestBody = RequestBodyLBS $ BS.toLazyByteString $ encode gr - , requestHeaders = [(hContentType, "application/cbor")] - } - httpLbs request testManager - --- | postCBOR with url based on effective canister id -postCallCBOR, postQueryCBOR, postReadStateCBOR :: (HasCallStack, HasTestConfig) => Blob -> GenR -> IO (Response BS.ByteString) -postCallCBOR cid = postCBOR $ "/api/v2/canister/" ++ textual cid ++ "/call" -postQueryCBOR cid = postCBOR $ "/api/v2/canister/" ++ textual cid ++ "/query" -postReadStateCBOR cid = postCBOR $ "/api/v2/canister/" ++ textual cid ++ "/read_state" - --- | Add envelope to CBOR request, add a nonce and expiry if it is not there, --- post to "read", return decoded CBOR -queryCBOR :: (HasCallStack, HasTestConfig) => Blob -> GenR -> IO GenR -queryCBOR cid req = do - addNonceExpiryEnv req >>= postQueryCBOR cid >>= okCBOR - --- | Add envelope to CBOR, and a nonce and expiry if not there, post to --- "submit". Returns either a HTTP Error code, or if the status is 2xx, poll --- for the request response, and return decoded CBOR -type HTTPErrOrReqResponse = Either (Int,String) ReqResponse -awaitCall' :: (HasCallStack, HasTestConfig) => Blob -> GenR -> IO HTTPErrOrReqResponse -awaitCall' cid req = do - req <- addNonce req - req <- addExpiry req - res <- envelopeFor (senderOf req) req >>= postCallCBOR cid - let code = statusCode (responseStatus res) - if 200 <= code && code < 300 - then do - assertBool "Response body not empty" (BS.null (responseBody res)) - Right <$> awaitStatus (senderOf req) cid (requestId req) - else do - let msg = T.unpack (T.decodeUtf8With T.lenientDecode (BS.toStrict (BS.take 200 (responseBody res)))) - pure $ Left (code, msg) - --- | Add envelope to CBOR, and a nonce and expiry if not there, post to --- "submit", poll for the request response, and return decoded CBOR -awaitCall :: (HasCallStack, HasTestConfig) => Blob -> GenR -> IO ReqResponse -awaitCall cid req = awaitCall' cid req >>= is2xx - -is2xx :: HasCallStack => HTTPErrOrReqResponse -> IO ReqResponse -is2xx = \case - Left (c,msg) -> assertFailure $ "Status " ++ show c ++ " is not 2xx:\n" ++ msg - Right res -> pure res - --- | Submits twice -awaitCallTwice :: HasTestConfig => Blob -> GenR -> IO ReqResponse -awaitCallTwice cid req = do - req <- addNonce req - req <- addExpiry req - res <- envelopeFor (senderOf req) req >>= postCallCBOR cid - code202 res - res <- envelopeFor (senderOf req) req >>= postCallCBOR cid - code202 res - assertBool "Response body not empty" (BS.null (responseBody res)) - awaitStatus (senderOf req) cid (requestId req) - -getStateCert' :: (HasCallStack, HasTestConfig) => Blob -> Blob -> [[Blob]] -> IO (Response Blob) -getStateCert' sender ecid paths = do - req <- addExpiry $ rec - [ "request_type" =: GText "read_state" - , "sender" =: GBlob sender - , "paths" =: GList (map (GList . map GBlob) paths) - ] - envelopeFor (senderOf req) req >>= postReadStateCBOR ecid - -decodeCert' :: HasCallStack => Blob -> IO Certificate -decodeCert' b = either (assertFailure . T.unpack) return $ decodeCert b - -getStateCert :: (HasCallStack, HasTestConfig) => Blob -> Blob -> [[Blob]] -> IO Certificate -getStateCert sender ecid paths = do - gr <- getStateCert' sender ecid paths >>= okCBOR - b <- record (field blob "certificate") gr - cert <- decodeCert' b - - case wellFormed (cert_tree cert) of - Left err -> assertFailure $ "Hash tree not well formed: " ++ err - Right () -> return () - - return cert - -extractCertData :: Blob -> Blob -> IO Blob -extractCertData cid b = do - cert <- decodeCert' b - case wellFormed (cert_tree cert) of - Left err -> assertFailure $ "Hash tree not well formed: " ++ err - Right () -> return () - certValue cert ["canister", cid, "certified_data"] - -verboseVerify :: String -> Blob -> Blob -> Blob -> Blob -> IO () -verboseVerify what domain_sep pk msg sig = - case DER_BLS.verify domain_sep pk msg sig of - Left err -> assertFailure $ unlines - [ "Signature verification failed on " ++ what - , T.unpack err - , "Domain separator: " ++ prettyBlob domain_sep - , "Public key (DER): " ++ asHex pk - , "Public key decoded: " ++ - case DER.decode pk of - Left err -> T.unpack err - Right (suite, key) -> asHex key ++ " (" ++ show suite ++ ")" - , "Signature: " ++ asHex sig - , "Checked message: " ++ prettyBlob msg - ] - Right () -> return () - -validateDelegation :: (HasCallStack, HasTestConfig) => Maybe Delegation -> IO Blob -validateDelegation Nothing = return (tc_root_key testConfig) -validateDelegation (Just del) = do - cert <- decodeCert' (del_certificate del) - case wellFormed (cert_tree cert) of - Left err -> assertFailure $ "Hash tree not well formed: " ++ err - Right () -> return () - validateStateCert' "certificate delegation" cert - - certValue cert ["subnet", del_subnet_id del, "public_key"] - -validateStateCert' :: (HasCallStack, HasTestConfig) => String -> Certificate -> IO () -validateStateCert' what cert = do - pk <- validateDelegation (cert_delegation cert) - verboseVerify what "ic-state-root" pk (reconstruct (cert_tree cert)) (cert_sig cert) - -validateStateCert :: (HasCallStack, HasTestConfig) => Certificate -> IO () -validateStateCert = validateStateCert' "certificate" - -data ReqResponse = Reply Blob | Reject Natural T.Text - deriving (Eq, Show) -data ReqStatus = Processing | Pending | Responded ReqResponse | UnknownStatus - deriving (Eq, Show) - -prettyPath :: [Blob] -> String -prettyPath = concatMap (("/" ++) . shorten 15 . prettyBlob) - -prettyBlob :: Blob -> String -prettyBlob x = - let s = map (chr . fromIntegral) (BS.unpack x) in - if all isPrint s then s else asHex x - -certValue :: HasCallStack => CertVal a => Certificate -> [Blob] -> IO a -certValue cert path = case lookupPath (cert_tree cert) path of - Found b -> case fromCertVal b of - Just x -> return x - Nothing -> assertFailure $ "Cannot parse " ++ prettyPath path ++ " from " ++ show b - x -> assertFailure $ "Expected to find " ++ prettyPath path ++ ", but got " ++ show x - -certValueAbsent :: HasCallStack => Certificate -> [Blob] -> IO () -certValueAbsent cert path = case lookupPath (cert_tree cert) path of - Absent -> return () - x -> assertFailure $ "Path " ++ prettyPath path ++ " should be absent, but got " ++ show x - -getRequestStatus :: (HasCallStack, HasTestConfig) => Blob -> Blob -> Blob -> IO ReqStatus -getRequestStatus sender cid rid = do - cert <- getStateCert sender cid [["request_status", rid]] - - case lookupPath (cert_tree cert) ["request_status", rid, "status"] of - Absent -> return UnknownStatus - Found "processing" -> return Processing - Found "received" -> return Pending - Found "replied" -> do - b <- certValue cert ["request_status", rid, "reply"] - certValueAbsent cert ["request_status", rid, "reject_code"] - certValueAbsent cert ["request_status", rid, "reject_message"] - return $ Responded (Reply b) - Found "rejected" -> do - certValueAbsent cert ["request_status", rid, "reply"] - code <- certValue cert ["request_status", rid, "reject_code"] - msg <- certValue cert ["request_status", rid, "reject_message"] - return $ Responded (Reject code msg) - Found s -> assertFailure $ "Unexpected status " ++ show s - -- This case should not happen with a compliant IC, but let - -- us be liberal here, and strict in a dedicated test - Unknown -> return UnknownStatus - x -> assertFailure $ "Unexpected request status, got " ++ show x - -awaitStatus :: HasTestConfig => Blob -> Blob -> Blob -> IO ReqResponse -awaitStatus sender cid rid = loop $ pollDelay >> getRequestStatus sender cid rid >>= \case - Responded x -> return $ Just x - _ -> return Nothing - where - loop :: HasCallStack => IO (Maybe a) -> IO a - loop act = go (0::Int) - where - go 10000 = assertFailure "Polling timed out" - go n = act >>= \case - Just r -> return r - Nothing -> go (n+1) - -pollDelay :: IO () -pollDelay = threadDelay $ 10 * 1000 -- 10 milliseonds - --- How long to wait before checking if a request that should _not_ show up on --- the system indeed did not show up -ingressDelay :: IO () -ingressDelay = threadDelay $ 2 * 1000 * 1000 -- 2 seconds - - --- * HTTP Response predicates - -codePred :: HasCallStack => String -> (Int -> Bool) -> Response Blob -> IO () -codePred expt pred response = assertBool - ("Status " ++ show c ++ " is not " ++ expt ++ "\n" ++ msg) - (pred c) - where - c = statusCode (responseStatus response) - msg = T.unpack (T.decodeUtf8With T.lenientDecode (BS.toStrict (BS.take 200 (responseBody response)))) - -code2xx, code202, code4xx, code202_or_4xx :: HasCallStack => Response BS.ByteString -> IO () -code2xx = codePred "2xx" $ \c -> 200 <= c && c < 300 -code202 = codePred "202" $ \c -> c == 202 -code4xx = codePred "4xx" $ \c -> 400 <= c && c < 500 -code202_or_4xx = codePred "202 or 4xx" $ \c -> c == 202 || 400 <= c && c < 500 - --- * CBOR decoding - -okCBOR :: HasCallStack => Response BS.ByteString -> IO GenR -okCBOR response = do - code2xx response - asRight $ decode $ responseBody response - -asCBORBlobList :: Blob -> IO [Blob] -asCBORBlobList blob = do - decoded <- asRight $ decode $ blob - case decoded of - GList list -> mapM cborToBlob list - _ -> assertFailure $ "Failed to decode as CBOR encoded list of blobs: " <> show decoded - -cborToBlob :: GenR -> IO Blob -cborToBlob (GBlob blob) = return blob -cborToBlob r = assertFailure $ "Expected blob, got " <> show r - --- * Response predicates and parsers - -queryResponse :: GenR -> IO ReqResponse -queryResponse = record $ do - s <- field text "status" - case s of - "replied" -> - Reply <$> field (record (field blob "arg")) "reply" - "rejected" -> do - code <- field nat "reject_code" - msg <- field text "reject_message" - return $ Reject code msg - _ -> lift $ assertFailure $ "Unexpected status " ++ show s - -isReject :: HasCallStack => [Natural] -> ReqResponse -> IO () -isReject _ (Reply r) = - assertFailure $ "Expected reject, got reply:" ++ prettyBlob r -isReject codes (Reject n msg) = do - assertBool - ("Reject code " ++ show n ++ " not in " ++ show codes ++ "\n" ++ T.unpack msg) - (n `elem` codes) - -isErrOrReject :: HasCallStack => [Natural] -> HTTPErrOrReqResponse -> IO () -isErrOrReject _codes (Left (c, msg)) - | 400 <= c && c < 600 = return () - | otherwise = assertFailure $ - "Status " ++ show c ++ " is not 4xx or 5xx:\n" ++ msg -isErrOrReject [] (Right _) = assertFailure "Got HTTP response, expected HTTP error" -isErrOrReject codes (Right res) = isReject codes res - - -isReply :: HasCallStack => ReqResponse -> IO Blob -isReply (Reply b) = return b -isReply (Reject n msg) = - assertFailure $ "Unexpected reject (code " ++ show n ++ "): " ++ T.unpack msg - --- Predicates to handle the responses from relayReply and relayReject -isRelay :: HasCallStack => Blob -> IO ReqResponse -isRelay = runGet $ Get.getWord32le >>= \case - 0 -> Reply <$> Get.getRemainingLazyByteString - 0x4c444944 -> fail "Encountered Candid when expectin relayed data. Did you forget to use isRelay?" - c -> do - msg <- Get.getRemainingLazyByteString - return $ Reject (fromIntegral c) (T.decodeUtf8With T.lenientDecode (BS.toStrict msg)) - --- Convenience decoders -asWord32 :: HasCallStack => Blob -> IO Word32 -asWord32 = runGet Get.getWord32le - -asWord64 :: HasCallStack => Blob -> IO Word64 -asWord64 = runGet Get.getWord64le - -as2Word64 :: HasCallStack => Blob -> IO (Word64, Word64) -as2Word64 = runGet $ (,) <$> Get.getWord64le <*> Get.getWord64le - -bothSame :: (Eq a, Show a) => (a, a) -> Assertion -bothSame (x,y) = x @?= y - -runGet :: HasCallStack => Get.Get a -> Blob -> IO a -runGet a b = case Get.runGetOrFail (a <* done) b of - Left (_,_, err) -> - fail $ "Could not parse " ++ show b ++ ": " ++ err - Right (_,_, x) -> return x - where - done = do - nothing_left <- Get.isEmpty - unless nothing_left (fail "left-over bytes") - -is :: (HasCallStack, Eq a, Show a) => a -> a -> Assertion -is exp act = act @?= exp - -isSet :: (HasCallStack, Ord a, Show a) => [a] -> [a] -> Assertion -isSet exp act = S.fromList exp @?= S.fromList act - -data StatusResponse = StatusResponse - { status_api_version :: T.Text - , status_root_key :: Blob - } - -statusResonse :: HasCallStack => GenR -> IO StatusResponse -statusResonse = record $ do - v <- field text "ic_api_version" - _ <- optionalField text "impl_source" - _ <- optionalField text "impl_version" - _ <- optionalField text "impl_revision" - pk <- field blob "root_key" - swallowAllFields -- More fields are explicitly allowed - return StatusResponse {status_api_version = v, status_root_key = pk } - --- * Interacting with aaaaa-aa (via HTTP) - -{- -The code below has some repetition. That’s because we have - - A) multiple ways of _calling_ the Management Canister - (as default user, as specific user, via canister, with or without cycles), - B) different things we want to know - (just the Candid-decoded reply, or the response, or even the HTTP error) - C) and then of course different methods (which affect response decoding) - -So far, there is some duplication here because of that. Eventually, this should -be refactored so that the test can declarative pick A, B and C separately. --} - --- how to reach the management canister -type IC00 = Blob -> T.Text -> Blob -> IO ReqResponse - -ic00as :: (HasTestConfig, HasCallStack) => Blob -> IC00 -ic00as user ecid method_name arg = awaitCall ecid $ rec - [ "request_type" =: GText "call" - , "sender" =: GBlob user - , "canister_id" =: GBlob "" - , "method_name" =: GText method_name - , "arg" =: GBlob arg - ] - -ic00 :: HasTestConfig => IC00 -ic00 = ic00as defaultUser - -ic00via :: HasTestConfig => Blob -> IC00 -ic00via cid = ic00viaWithCycles cid 0 - -ic00viaWithCycles :: HasTestConfig => Blob -> Word64 -> IC00 -ic00viaWithCycles cid cycles _ecid method_name arg = - do call' cid $ - callNew - (bytes "") (bytes (BS.fromStrict (T.encodeUtf8 method_name))) -- aaaaa-aa - (callback relayReply) (callback relayReject) >>> - callDataAppend (bytes arg) >>> - callCyclesAdd (int64 cycles) >>> - callPerform - >>= isReply >>= isRelay - --- A variant that allows non-200 responses to submit -ic00as' :: HasTestConfig => Blob -> Blob -> T.Text -> Blob -> IO HTTPErrOrReqResponse -ic00as' user cid method_name arg = awaitCall' cid $ rec - [ "request_type" =: GText "call" - , "sender" =: GBlob user - , "canister_id" =: GBlob "" - , "method_name" =: GText method_name - , "arg" =: GBlob arg - ] - --- Now wrapping the concrete calls --- (using Candid.toCandidService is tricky because of all stuff like passing through the effective canister id) --- -callIC :: forall s a b. - (HasCallStack, HasTestConfig) => - KnownSymbol s => - (a -> IO b) ~ (ICManagement IO .! s) => - (Candid.CandidArg a, Candid.CandidArg b) => - IC00 -> Blob -> Label s -> a -> IO b -callIC ic00 ecid l x = do - r <- ic00 ecid (T.pack (symbolVal l)) (Candid.encode x) >>= isReply - case Candid.decode r of - Left err -> assertFailure $ "Candid decoding error: " ++ err - Right y -> pure y - --- The following line noise is me getting out of my way --- to be able to use `ic_create` etc. by passing a record that contains --- a subset of settings, without Maybe -type family UnRec r where UnRec (R.Rec r) = r -type PartialSettings r = (R.Forall r R.Unconstrained1, R.Map Maybe r .// UnRec Settings ≈ UnRec Settings) -fromPartialSettings :: PartialSettings r => R.Rec r -> Settings -fromPartialSettings r = - R.map' Just r .// - R.default' @(R.IsA R.Unconstrained1 Maybe) @(UnRec Settings) d - where - d :: forall a. R.IsA R.Unconstrained1 Maybe a => a - d = case R.as @R.Unconstrained1 @Maybe @a of R.As -> Nothing - -ic_create :: (HasCallStack, HasTestConfig, PartialSettings r) => IC00 -> Rec r -> IO Blob -ic_create ic00 ps = do - r <- callIC ic00 "" #create_canister $ empty - .+ #settings .== Just (fromPartialSettings ps) - return (rawPrincipal (r .! #canister_id)) - -ic_provisional_create :: - (HasCallStack, HasTestConfig, PartialSettings r) => - IC00 -> Maybe Natural -> Rec r -> IO Blob -ic_provisional_create ic00 cycles ps = do - r <- callIC ic00 "" #provisional_create_canister_with_cycles $ empty - .+ #amount .== cycles - .+ #settings .== Just (fromPartialSettings ps) - return (rawPrincipal (r .! #canister_id)) - -ic_install :: (HasCallStack, HasTestConfig) => IC00 -> InstallMode -> Blob -> Blob -> Blob -> IO () -ic_install ic00 mode canister_id wasm_module arg = do - callIC ic00 canister_id #install_code $ empty - .+ #mode .== mode - .+ #canister_id .== Principal canister_id - .+ #wasm_module .== wasm_module - .+ #arg .== arg - -ic_uninstall :: (HasCallStack, HasTestConfig) => IC00 -> Blob -> IO () -ic_uninstall ic00 canister_id = do - callIC ic00 canister_id #uninstall_code $ empty - .+ #canister_id .== Principal canister_id - -ic_set_controllers :: HasTestConfig => IC00 -> Blob -> [Blob] -> IO () -ic_set_controllers ic00 canister_id new_controllers = do - callIC ic00 canister_id #update_settings $ empty - .+ #canister_id .== Principal canister_id - .+ #settings .== fromPartialSettings (#controllers .== Vec.fromList (map Principal new_controllers)) - -ic_start_canister :: HasTestConfig => IC00 -> Blob -> IO () -ic_start_canister ic00 canister_id = do - callIC ic00 canister_id #start_canister $ empty - .+ #canister_id .== Principal canister_id - -ic_stop_canister :: HasTestConfig => IC00 -> Blob -> IO () -ic_stop_canister ic00 canister_id = do - callIC ic00 canister_id #stop_canister $ empty - .+ #canister_id .== Principal canister_id - -ic_canister_status :: - forall a b. (a -> IO b) ~ (ICManagement IO .! "canister_status") => - HasTestConfig => IC00 -> Blob -> IO b -ic_canister_status ic00 canister_id = do - callIC ic00 canister_id #canister_status $ empty - .+ #canister_id .== Principal canister_id - -ic_deposit_cycles :: HasTestConfig => IC00 -> Blob -> IO () -ic_deposit_cycles ic00 canister_id = do - callIC ic00 canister_id #deposit_cycles $ empty - .+ #canister_id .== Principal canister_id - -ic_top_up :: HasTestConfig => IC00 -> Blob -> Natural -> IO () -ic_top_up ic00 canister_id amount = do - callIC ic00 canister_id #provisional_top_up_canister $ empty - .+ #canister_id .== Principal canister_id - .+ #amount .== amount - -ic_delete_canister :: HasTestConfig => IC00 -> Blob -> IO () -ic_delete_canister ic00 canister_id = do - callIC ic00 canister_id #delete_canister $ empty - .+ #canister_id .== Principal canister_id - -ic_raw_rand :: HasTestConfig => IC00 -> IO Blob -ic_raw_rand ic00 = - callIC ic00 "" #raw_rand () - --- Primed variants return the response (reply or reject) -callIC' :: forall s a b. - HasTestConfig => - KnownSymbol s => - (a -> IO b) ~ (ICManagement IO .! s) => - Candid.CandidArg a => - IC00 -> Blob -> Label s -> a -> IO ReqResponse -callIC' ic00 ecid l x = ic00 ecid (T.pack (symbolVal l)) (Candid.encode x) - -ic_create' :: - (HasCallStack, HasTestConfig, PartialSettings r) => - IC00 -> Rec r -> IO ReqResponse -ic_create' ic00 ps = do - callIC' ic00 "" #create_canister $ empty - .+ #settings .== Just (fromPartialSettings ps) - -ic_provisional_create' :: - (HasCallStack, HasTestConfig, PartialSettings r) => - IC00 -> Maybe Natural -> Rec r -> IO ReqResponse -ic_provisional_create' ic00 cycles ps = do - callIC' ic00 "" #provisional_create_canister_with_cycles $ empty - .+ #amount .== cycles - .+ #settings .== Just (fromPartialSettings ps) - -ic_install' :: HasTestConfig => IC00 -> InstallMode -> Blob -> Blob -> Blob -> IO ReqResponse -ic_install' ic00 mode canister_id wasm_module arg = - callIC' ic00 canister_id #install_code $ empty - .+ #mode .== mode - .+ #canister_id .== Principal canister_id - .+ #wasm_module .== wasm_module - .+ #arg .== arg - -ic_update_settings' :: (HasTestConfig, PartialSettings r) => IC00 -> Blob -> Rec r -> IO ReqResponse -ic_update_settings' ic00 canister_id r = do - callIC' ic00 canister_id #update_settings $ empty - .+ #canister_id .== Principal canister_id - .+ #settings .== fromPartialSettings r - -ic_set_controllers' :: HasTestConfig => IC00 -> Blob -> [Blob] -> IO ReqResponse -ic_set_controllers' ic00 canister_id new_controllers = do - ic_update_settings' ic00 canister_id (#controllers .== Vec.fromList (map Principal new_controllers)) - -ic_delete_canister' :: HasTestConfig => IC00 -> Blob -> IO ReqResponse -ic_delete_canister' ic00 canister_id = do - callIC' ic00 canister_id #delete_canister $ empty - .+ #canister_id .== Principal canister_id - -ic_deposit_cycles' :: HasTestConfig => IC00 -> Blob -> IO ReqResponse -ic_deposit_cycles' ic00 canister_id = do - callIC' ic00 canister_id #deposit_cycles $ empty - .+ #canister_id .== Principal canister_id - -ic_top_up' :: HasTestConfig => IC00 -> Blob -> Natural -> IO ReqResponse -ic_top_up' ic00 canister_id amount = do - callIC' ic00 canister_id #provisional_top_up_canister $ empty - .+ #canister_id .== Principal canister_id - .+ #amount .== amount - --- Double primed variants are only for requests from users (so they take the user, --- not a generic ic00 thing), and return the HTTP error code or the response --- (reply or reject) - -callIC'' :: forall s a b. - HasTestConfig => - KnownSymbol s => - (a -> IO b) ~ (ICManagement IO .! s) => - Candid.CandidArg a => - Blob -> Blob -> Label s -> a -> IO HTTPErrOrReqResponse -callIC'' user ecid l x = ic00as' user ecid (T.pack (symbolVal l)) (Candid.encode x) - -ic_install'' :: (HasCallStack, HasTestConfig) => Blob -> InstallMode -> Blob -> Blob -> Blob -> IO HTTPErrOrReqResponse -ic_install'' user mode canister_id wasm_module arg = - callIC'' user canister_id #install_code $ empty - .+ #mode .== mode - .+ #canister_id .== Principal canister_id - .+ #wasm_module .== wasm_module - .+ #arg .== arg - -ic_uninstall'' :: HasTestConfig => Blob -> Blob -> IO HTTPErrOrReqResponse -ic_uninstall'' user canister_id = - callIC'' user canister_id #uninstall_code $ empty - .+ #canister_id .== Principal canister_id - -ic_set_controllers'' :: HasTestConfig => Blob -> Blob -> [Blob] -> IO HTTPErrOrReqResponse -ic_set_controllers'' user canister_id new_controllers = do - callIC'' user canister_id #update_settings $ empty - .+ #canister_id .== Principal canister_id - .+ #settings .== fromPartialSettings (#controllers .== Vec.fromList (map Principal new_controllers)) - -ic_start_canister'' :: HasTestConfig => Blob -> Blob -> IO HTTPErrOrReqResponse -ic_start_canister'' user canister_id = do - callIC'' user canister_id #start_canister $ empty - .+ #canister_id .== Principal canister_id - -ic_stop_canister'' :: HasTestConfig => Blob -> Blob -> IO HTTPErrOrReqResponse -ic_stop_canister'' user canister_id = do - callIC'' user canister_id #stop_canister $ empty - .+ #canister_id .== Principal canister_id - -ic_canister_status'' :: HasTestConfig => Blob -> Blob -> IO HTTPErrOrReqResponse -ic_canister_status'' user canister_id = do - callIC'' user canister_id #canister_status $ empty - .+ #canister_id .== Principal canister_id - -ic_delete_canister'' :: HasTestConfig => Blob -> Blob -> IO HTTPErrOrReqResponse -ic_delete_canister'' user canister_id = do - callIC'' user canister_id #delete_canister $ empty - .+ #canister_id .== Principal canister_id - -ic_deposit_cycles'' :: HasTestConfig => Blob -> Blob -> IO HTTPErrOrReqResponse -ic_deposit_cycles'' user canister_id = do - callIC'' user canister_id #deposit_cycles $ empty - .+ #canister_id .== Principal canister_id - -ic_raw_rand'' :: HasTestConfig => Blob -> IO HTTPErrOrReqResponse -ic_raw_rand'' user = do - callIC'' user "" #raw_rand () - - --- A barrier - --- This will stop and start all mentioned canisters. This guarantees --- that all outstanding callbacks are handled -barrier :: HasTestConfig => [Blob] -> IO () -barrier cids = do - mapM_ (ic_stop_canister ic00) cids - mapM_ (ic_start_canister ic00) cids - --- * Interacting with the universal canister - --- Some common operations on the universal canister --- The primed variant (call') return the response record, --- e.g. to check for error conditions. --- The unprimed variant expect a reply. - -install' :: (HasCallStack, HasTestConfig) => Blob -> Prog -> IO ReqResponse -install' cid prog = do - universal_wasm <- getTestWasm "universal_canister" - ic_install' ic00 (enum #install) cid universal_wasm (run prog) - -installAt :: (HasCallStack, HasTestConfig) => Blob -> Prog -> IO () -installAt cid prog = do - universal_wasm <- getTestWasm "universal_canister" - ic_install ic00 (enum #install) cid universal_wasm (run prog) - --- Also calls create, used default 'ic00' -install :: (HasCallStack, HasTestConfig) => Prog -> IO Blob -install prog = do - cid <- create - installAt cid prog - return cid - -create :: (HasCallStack, HasTestConfig) => IO Blob -create = ic_provisional_create ic00 Nothing empty - -upgrade' :: (HasCallStack, HasTestConfig) => Blob -> Prog -> IO ReqResponse -upgrade' cid prog = do - universal_wasm <- getTestWasm "universal_canister" - ic_install' ic00 (enum #upgrade) cid universal_wasm (run prog) - -upgrade :: (HasCallStack, HasTestConfig) => Blob -> Prog -> IO () -upgrade cid prog = do - universal_wasm <- getTestWasm "universal_canister" - ic_install ic00 (enum #upgrade) cid universal_wasm (run prog) - -reinstall' :: (HasCallStack, HasTestConfig) => Blob -> Prog -> IO ReqResponse -reinstall' cid prog = do - universal_wasm <- getTestWasm "universal_canister" - ic_install' ic00 (enum #reinstall) cid universal_wasm (run prog) - -reinstall :: (HasCallStack, HasTestConfig) => Blob -> Prog -> IO () -reinstall cid prog = do - universal_wasm <- getTestWasm "universal_canister" - ic_install ic00 (enum #reinstall) cid universal_wasm (run prog) - -callRequestAs :: (HasCallStack, HasTestConfig) => Blob -> Blob -> Prog -> GenR -callRequestAs user cid prog = rec - [ "request_type" =: GText "call" - , "sender" =: GBlob user - , "canister_id" =: GBlob cid - , "method_name" =: GText "update" - , "arg" =: GBlob (run prog) - ] - -callToQueryRequestAs :: (HasCallStack, HasTestConfig) => Blob -> Blob -> Prog -> GenR -callToQueryRequestAs user cid prog = rec - [ "request_type" =: GText "call" - , "sender" =: GBlob user - , "canister_id" =: GBlob cid - , "method_name" =: GText "query" - , "arg" =: GBlob (run prog) - ] - -callRequest :: (HasCallStack, HasTestConfig) => Blob -> Prog -> GenR -callRequest cid prog = callRequestAs defaultUser cid prog - -callToQuery'' :: (HasCallStack, HasTestConfig) => Blob -> Prog -> IO HTTPErrOrReqResponse -callToQuery'' cid prog = awaitCall' cid $ callToQueryRequestAs defaultUser cid prog - --- The following variants of the call combinator differ in how much failure they allow: --- --- call'' allows HTTP errors at `submit` time already --- call' requires submission to succeed, and allows reject responses --- call requires a reply response --- call_ requires a reply response with an empty blob (a common case) - -call'' :: (HasCallStack, HasTestConfig) => Blob -> Prog -> IO HTTPErrOrReqResponse -call'' cid prog = awaitCall' cid (callRequest cid prog) - -call' :: (HasCallStack, HasTestConfig) => Blob -> Prog -> IO ReqResponse -call' cid prog = call'' cid prog >>= is2xx - -call :: (HasCallStack, HasTestConfig) => Blob -> Prog -> IO Blob -call cid prog = call' cid prog >>= isReply - -call_ :: (HasCallStack, HasTestConfig) => Blob -> Prog -> IO () -call_ cid prog = call cid prog >>= is "" - -callTwice' :: (HasCallStack, HasTestConfig) => Blob -> Prog -> IO ReqResponse -callTwice' cid prog = awaitCallTwice cid (callRequest cid prog) - - -query' :: (HasCallStack, HasTestConfig) => Blob -> Prog -> IO ReqResponse -query' cid prog = - queryCBOR cid >=> queryResponse $ rec - [ "request_type" =: GText "query" - , "sender" =: GBlob defaultUser - , "canister_id" =: GBlob cid - , "method_name" =: GText "query" - , "arg" =: GBlob (run prog) - ] - -query :: (HasCallStack, HasTestConfig) => Blob -> Prog -> IO Blob -query cid prog = query' cid prog >>= isReply - -query_ :: (HasCallStack, HasTestConfig) => Blob -> Prog -> IO () -query_ cid prog = query cid prog >>= is "" - --- Shortcut for test cases that just need one canister -simpleTestCase :: (HasCallStack, HasTestConfig) => String -> (Blob -> IO ()) -> TestTree -simpleTestCase name act = testCase name $ install noop >>= act - --- * Programmatic test generation - --- | Runs test once for each field with that field removed, including nested --- fields -omitFields :: GenR -> (GenR -> Assertion) -> [TestTree] -omitFields (GRec hm) act = - [ let hm' = HM.delete f hm - in testCase ("omitting " ++ T.unpack f) $ act (GRec hm') - | f <- fields - ] ++ concat - [ omitFields val $ \val' -> act (GRec (HM.insert f val' hm)) - | f <- fields - , val@(GRec _) <- return $ hm HM.! f - ] - where fields = sort $ HM.keys hm -omitFields _ _ = error "omitFields needs a GRec" - - -getRand8Bytes :: IO BS.ByteString -getRand8Bytes = BS.pack <$> replicateM 8 randomIO - --- * Endpoint handling - --- Yes, implicit arguments are frowned upon. But they are also very useful. - -type HasTestConfig = (?testConfig :: TestConfig) - -withTestConfig :: (forall. HasTestConfig => a) -> TestConfig -> a -withTestConfig act tc = let ?testConfig = tc in act - -testConfig :: HasTestConfig => TestConfig -testConfig = ?testConfig - -endPoint :: HasTestConfig => String -endPoint = tc_endPoint testConfig - -testManager :: HasTestConfig => Manager -testManager = tc_manager testConfig - --- * Test data access - -getTestFile :: FilePath -> IO FilePath -getTestFile file = - lookupEnv "IC_TEST_DATA" >>= \case - Just fp -> return $ fp file - Nothing -> do - -- nix use - exePath <- getExecutablePath - let exeRelPath = takeDirectory exePath "../test-data" - -- convenient for cabal new-run use - try [ exeRelPath, "test-data", "../test-data", "impl/test-data" ] - where - try (d:ds) = doesFileExist (d file) >>= \case - True -> return (d file) - False -> try ds - try [] = error $ "getTestDir: Could not read " ++ file ++ " from test-data/. Please consult impl/README.md" - -getTestWasm :: FilePath -> IO BS.ByteString -getTestWasm base = do - fp <- getTestFile $ base <.> "wasm" - BS.readFile fp - - --- Convenience around Data.Row.Variants used as enums - -enum :: (AllUniqueLabels r, KnownSymbol l, (r .! l) ~ ()) => Label l -> Var r -enum l = V.IsJust l () - --- Other utilities - -asHex :: Blob -> String -asHex = T.unpack . H.encodeHex . BS.toStrict - -textual :: Blob -> String -textual = T.unpack . prettyPrincipal . Principal - - -shorten :: Int -> String -> String -shorten n s = a ++ (if null b then "" else "…") - where (a,b) = splitAt n s diff --git a/impl/src/IC/Test/Universal.hs b/impl/src/IC/Test/Universal.hs deleted file mode 100644 index 5f3bfe0d4..000000000 --- a/impl/src/IC/Test/Universal.hs +++ /dev/null @@ -1,282 +0,0 @@ -{- | - -Helpers to program the “Universal module”. This is essentially a small, -type-safe DSL to produce the small stack-based programming language interpreted -by the universal canister. - -This DSL is expression-based, not stack based; seems to suite all our needs and is -simpler to work with. - -This language is not stable; therefore there is no separarte documentation of -specification than this file and `impl/universal-canister/src/` --} - -{-# LANGUAGE KindSignatures #-} -{-# LANGUAGE DataKinds #-} -{-# LANGUAGE GADTs #-} -{-# LANGUAGE FlexibleInstances #-} -{-# LANGUAGE OverloadedStrings #-} - -module IC.Test.Universal where - -import qualified Data.ByteString.Lazy as BS -import Data.ByteString.Builder -import Data.Word -import Data.String - --- The types of our little language are i32, i64 and blobs - -data T = I | I64 | B - - --- We deal with expressions (return a value, thus have a type) and programs (do --- something, but do not return a type). They are represented simply --- by the encoded stack program; no need for an AST or something that complicated - -newtype Exp (result :: T) where - Exp :: Builder -> Exp a - -newtype Prog where - Prog :: Builder -> Prog - --- We extracting the actual stack program bytecode from a program. - -run :: Prog -> BS.ByteString -run (Prog x) = toLazyByteString x - --- Programs can be sequenced using (>>>); this naturally forms a Monoid - -(>>>) :: Prog -> Prog -> Prog -Prog a >>> Prog b = Prog (a <> b) - -instance Semigroup Prog where - (<>) = (>>>) - -instance Monoid Prog where - mempty = Prog mempty - --- A utility class to easily defined functions and programs of any arity --- simply by specifying their type. - -class Op a where - mkOp :: Word8 -> Builder -> a - -instance Op Prog - where mkOp x args = Prog $ args <> word8 x -instance Op (Exp t) - where mkOp x args = Exp $ args <> word8 x -instance Op a => Op (Exp t -> a) - where mkOp x args (Exp a) = mkOp x (args <> a) - -op :: Op a => Word8 -> a -op x = mkOp x mempty - --- Now, all the op codes defined by the universal canister. --- Most can be simply be defined by specifiying their type and using the 'op' --- combinator - -noop :: Prog -noop = op 0 - -ignore :: Exp t -> Prog -ignore = op 1 - -int :: Word32 -> Exp 'I -int x = Exp $ word8 2 <> word32LE x - -int64 :: Word64 -> Exp 'I64 -int64 x = Exp $ word8 31 <> word64LE x - -bytes :: BS.ByteString -> Exp 'B -bytes bytes = Exp $ - word8 3 <> - word32LE (fromIntegral (BS.length bytes)) <> - lazyByteString bytes - -replyDataAppend :: Exp 'B -> Prog -replyDataAppend = op 4 - -reply :: Prog -reply = op 5 - -self :: Exp 'B -self = op 6 - -reject :: Exp 'B -> Prog -reject = op 7 - -caller :: Exp 'B -caller = op 8 - -reject_msg :: Exp 'B -reject_msg = op 10 - -reject_code :: Exp 'I -reject_code = op 11 - -i2b :: Exp 'I -> Exp 'B -i2b = op 12 - -i64tob :: Exp 'I64 -> Exp 'B -i64tob = op 25 - -argData :: Exp 'B -argData = op 13 - -cat :: Exp 'B -> Exp 'B -> Exp 'B -cat = op 14 - -stableSize :: Exp 'I -stableSize = op 15 - -stableGrow :: Exp 'I -> Exp 'I -stableGrow = op 16 - -stableRead :: Exp 'I -> Exp 'I -> Exp 'B -stableRead = op 17 - -stableWrite :: Exp 'I -> Exp 'B -> Prog -stableWrite = op 18 - -getTime :: Exp 'I64 -getTime = op 26 - -getAvailableCycles :: Exp 'I64 -getAvailableCycles = op 27 - -getBalance :: Exp 'I64 -getBalance = op 28 - -getRefund :: Exp 'I64 -getRefund = op 29 - -acceptCycles :: Exp 'I64 -> Exp 'I64 -acceptCycles = op 30 - -debugPrint :: Exp 'B -> Prog -debugPrint = op 19 - -trap :: Exp 'B -> Prog -trap = op 20 - -setGlobal :: Exp 'B -> Prog -setGlobal = op 21 - -getGlobal :: Exp 'B -getGlobal = op 22 - -badPrint :: Prog -badPrint = op 23 - -onPreUpgrade :: Exp 'B -> Prog -onPreUpgrade = op 24 - -callNew :: Exp 'B -> Exp 'B -> Exp 'B -> Exp 'B -> Prog -callNew = op 32 - -callDataAppend :: Exp 'B -> Prog -callDataAppend = op 33 - -callCyclesAdd :: Exp 'I64 -> Prog -callCyclesAdd = op 34 - -callPerform :: Prog -callPerform = op 35 - -setCertifiedData :: Exp 'B -> Prog -setCertifiedData = op 36 - -getCertificatePresent :: Exp 'I -getCertificatePresent = op 37 - -getCertificate :: Exp 'B -getCertificate = op 38 - -getStatus :: Exp 'I -getStatus = op 39 - -acceptMessage :: Prog -acceptMessage = op 40 - -onInspectMessage :: Exp 'B -> Prog -onInspectMessage = op 41 - -methodName :: Exp 'B -methodName = op 42 - -trapIfEq :: Exp 'B -> Exp 'B -> Exp 'B -> Prog -trapIfEq = op 43 - -callOnCleanup :: Exp 'B -> Prog -callOnCleanup = op 44 - --- Some convenience combinators - --- This allows us to write byte expressions as plain string literals -instance IsString (Exp 'B) where - fromString s = bytes (fromString s) - -callback :: Prog -> Exp 'B -callback = bytes . run - -replyData :: Exp 'B -> Prog -replyData a = replyDataAppend a >>> reply - --- Convenient inter-canister calling - -data CallArgs = CallArgs - { on_reply :: Prog - , on_reject :: Prog - , on_cleanup :: Maybe Prog - , other_side :: Prog - , cycles :: Word64 - , icpts :: Word64 - } - -inter_call :: BS.ByteString -> BS.ByteString -> CallArgs -> Prog -inter_call callee method_name ca = - callNew (bytes callee) (bytes method_name) - (callback (on_reply ca)) (callback (on_reject ca)) >>> - maybe noop (callOnCleanup . callback) (on_cleanup ca) >>> - callDataAppend (callback (other_side ca)) >>> - (if cycles ca > 0 then callCyclesAdd (int64 (cycles ca)) else noop) >>> - callPerform - -inter_update :: BS.ByteString -> CallArgs -> Prog -inter_update callee = inter_call callee "update" - -inter_query :: BS.ByteString -> CallArgs -> Prog -inter_query callee = inter_call callee "query" - --- | By default, the other side responds with some text --- indicating caller and callee, and the callbacks reply with the response. -defArgs :: CallArgs -defArgs = CallArgs - { on_reply = relayReply - , on_reject = relayReject - , on_cleanup = Nothing - , other_side = defaultOtherSide - , cycles = 0 - , icpts = 0 - } - -defaultOtherSide :: Prog -defaultOtherSide = - replyDataAppend "Hello " >>> - replyDataAppend caller >>> - replyDataAppend " this is " >>> - replyDataAppend self >>> - reply - -relayReply :: Prog -relayReply = - replyDataAppend (i2b (int 0)) >>> - replyDataAppend argData >>> - reply - -relayReject :: Prog -relayReject = - replyDataAppend (i2b reject_code) >>> - replyDataAppend reject_msg >>> - reply - diff --git a/impl/src/IC/Test/WebAuthn.hs b/impl/src/IC/Test/WebAuthn.hs deleted file mode 100644 index c8d8c45ba..000000000 --- a/impl/src/IC/Test/WebAuthn.hs +++ /dev/null @@ -1,43 +0,0 @@ --- Unit test for IC.Test.Crypto.WebAuthn -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE BinaryLiterals #-} -{-# LANGUAGE ViewPatterns #-} -module IC.Test.WebAuthn (webAuthnTests) where - -import qualified Data.ByteString.Lazy as BS -import qualified Data.Text as T - -import Test.Tasty -import Test.Tasty.QuickCheck -import Test.Tasty.HUnit -import Test.QuickCheck.IO () - -import qualified IC.Crypto.WebAuthn as WebAuthn - -assertRight :: Either T.Text () -> Assertion -assertRight (Right ()) = return () -assertRight (Left err) = assertFailure (T.unpack err) - -assertLeft :: Either T.Text () -> Assertion -assertLeft (Left _) = return () -assertLeft (Right _) = assertFailure "Unexpected success" - -webAuthnTests :: TestTree -webAuthnTests = testGroup "WebAuthn crypto tests" - [ testProperty "create-sign-verify" $ - \(BS.pack -> seed) (BS.pack -> msg) -> do - let sk = WebAuthn.createKey seed - sig <- WebAuthn.sign sk msg - assertRight $ WebAuthn.verify (WebAuthn.toPublicKey sk) msg sig - , testProperty "invalid sig" $ - \(BS.pack -> seed) (BS.pack -> msg) (BS.pack -> sig) -> - let sk = WebAuthn.createKey seed in - assertLeft $ WebAuthn.verify (WebAuthn.toPublicKey sk) msg sig - , testProperty "wrong message" $ - \(BS.pack -> seed) (BS.pack -> msg1) (BS.pack -> msg2) -> - msg1 /= msg2 ==> do - let sk = WebAuthn.createKey seed - sig <- WebAuthn.sign sk msg2 - assertLeft $ WebAuthn.verify (WebAuthn.toPublicKey sk) msg1 sig - ] - diff --git a/impl/src/IC/Types.hs b/impl/src/IC/Types.hs deleted file mode 100644 index 6d7a64525..000000000 --- a/impl/src/IC/Types.hs +++ /dev/null @@ -1,180 +0,0 @@ -{-# LANGUAGE TypeOperators #-} -{-# LANGUAGE GeneralizedNewtypeDeriving #-} -{-# LANGUAGE DeriveFunctor #-} -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE NamedFieldPuns #-} -{-# LANGUAGE RankNTypes #-} -{-# LANGUAGE FlexibleContexts #-} -module IC.Types where - -import qualified Data.ByteString.Lazy.Char8 as BS -import qualified Data.ByteString.Builder as BS -import qualified Data.Map as M -import qualified Data.Text as T -import qualified Text.Hex as T hiding (Text) -import Data.Digest.CRC -import Data.Digest.CRC32 -import Data.ByteString.Base32 -import Data.Int -import Data.List -import Data.List.Split (chunksOf) -import Numeric.Natural -import Control.Monad.Except - -type (↦) = M.Map - --- Basic types - -type Blob = BS.ByteString -type PublicKey = Blob -newtype EntityId = EntityId { rawEntityId :: Blob } - deriving (Show, Eq, Ord) - -type CanisterId = EntityId -type SubnetId = EntityId -type UserId = EntityId -type MethodName = String -type RequestID = Blob -type Cycles = Natural - -prettyBlob :: Blob -> String -prettyBlob b = "0x" ++ T.unpack (T.encodeHex (BS.toStrict b)) - -prettyID :: EntityId -> String -prettyID (EntityId blob) = - intercalate "-" (chunksOf 5 (base32 (checkbytes <> blob))) - where - CRC32 checksum = digest (BS.toStrict blob) - checkbytes = BS.toLazyByteString (BS.word32BE checksum) - - base32 = filter (/='=') . T.unpack . T.toLower . encodeBase32 . BS.toStrict - - -newtype Responded = Responded Bool - deriving (Show, Eq) - -newtype Timestamp = Timestamp Natural - deriving (Show, Num, Ord, Eq) - -data RejectCode - = RC_SYS_FATAL - | RC_SYS_TRANSIENT - | RC_DESTINATION_INVALID - | RC_CANISTER_REJECT - | RC_CANISTER_ERROR - deriving Show - -rejectCode :: RejectCode -> Natural -rejectCode RC_SYS_FATAL = 1 -rejectCode RC_SYS_TRANSIENT = 2 -rejectCode RC_DESTINATION_INVALID = 3 -rejectCode RC_CANISTER_REJECT = 4 -rejectCode RC_CANISTER_ERROR = 5 - - -data Response = Reply Blob | Reject (RejectCode, String) - deriving Show - --- Abstract canisters - --- | This data type contains all read-only data that should be available to the --- canister almost always -data Status = Running | Stopping | Stopped -data Env = Env - { env_self :: CanisterId - , env_time :: Timestamp - , env_balance :: Cycles - , env_status :: Status - , env_certificate :: Maybe Blob - } - -data TrapOr a = Trap String | Return a deriving Functor - -data WasmClosure = WasmClosure - { closure_idx :: Int32 - , closure_env :: Int32 - } - deriving Show - -data Callback = Callback - { reply_callback :: WasmClosure - , reject_callback :: WasmClosure - , cleanup_callback :: Maybe WasmClosure - } - deriving Show - -data MethodCall = MethodCall - { call_callee :: CanisterId - , call_method_name :: MethodName - , call_arg :: Blob - , call_callback :: Callback - , call_transferred_cycles :: Cycles - } - deriving Show - -type ExistingCanisters = [CanisterId] - --- Canister actions (independent of calls) -newtype CanisterActions = CanisterActions - { set_certified_data :: Maybe Blob - } - -instance Semigroup CanisterActions where - ca1 <> ca2 = CanisterActions (set_certified_data ca1 `setter` set_certified_data ca2) - where - setter _ (Just x) = Just x - setter x Nothing = x - -noCanisterActions :: CanisterActions -noCanisterActions = CanisterActions Nothing - --- Actions relative to a call context -data CallActions = CallActions - { ca_new_calls :: [MethodCall] - , ca_accept :: Cycles - , ca_response :: Maybe Response - } - -noCallActions :: CallActions -noCallActions = CallActions [] 0 Nothing - -type UpdateResult = (CallActions, CanisterActions) - -type StableMemory = Blob - --- Semantically relevant information from an envelope --- --- * When is it valid --- * Which users can it sign for --- * Which canisters can it be used at --- --- All represented as validation functions - -type ValidityPred a = forall m. MonadError T.Text m => a -> m () -data EnvValidity = EnvValidity - { valid_when :: ValidityPred Timestamp - , valid_for :: ValidityPred EntityId - , valid_where :: ValidityPred EntityId - } - -instance Semigroup EnvValidity where - ed1 <> ed2 = EnvValidity - { valid_when = valid_when ed1 >>> valid_when ed2 - , valid_for = valid_for ed1 >>> valid_for ed2 - , valid_where = valid_where ed1 >>> valid_where ed2 - } where a >>> b = \x -> a x >> b x -instance Monoid EnvValidity where - mempty = EnvValidity x x x - where - x :: ValidityPred a - x = const (return ()) - -validWhen :: ValidityPred Timestamp -> EnvValidity -validWhen valid_when = mempty { valid_when } - -validFor :: ValidityPred EntityId -> EnvValidity -validFor valid_for = mempty { valid_for } - -validWhere :: ValidityPred EntityId -> EnvValidity -validWhere valid_where = mempty { valid_where } - diff --git a/impl/src/IC/Utils.hs b/impl/src/IC/Utils.hs deleted file mode 100644 index 31e0a436d..000000000 --- a/impl/src/IC/Utils.hs +++ /dev/null @@ -1,18 +0,0 @@ -{-# LANGUAGE LambdaCase #-} -{- | -Generic utilities related to standard or imported data structures that we do -don’t want to see in non-plumbing code. --} -module IC.Utils where - -import qualified Data.Map as M - -freshKey :: M.Map Int a -> Int -freshKey m | M.null m = 0 - | otherwise = fst (M.findMax m) + 1 - -repeatWhileTrue :: Monad m => m Bool -> m () -repeatWhileTrue act = act >>= \case - True -> repeatWhileTrue act - False -> return () - diff --git a/impl/src/IC/Version.hs b/impl/src/IC/Version.hs deleted file mode 100644 index 32b73d484..000000000 --- a/impl/src/IC/Version.hs +++ /dev/null @@ -1,9 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -module IC.Version where - -import Data.Text -import SourceId - -specVersion, implVersion :: Text -specVersion = "0.18.0" -implVersion = pack SourceId.id diff --git a/impl/src/IC/Wasm/Imports.hs b/impl/src/IC/Wasm/Imports.hs deleted file mode 100644 index 9858b9f50..000000000 --- a/impl/src/IC/Wasm/Imports.hs +++ /dev/null @@ -1,416 +0,0 @@ -{-# LANGUAGE ScopedTypeVariables #-} -{-# LANGUAGE TypeApplications #-} -{-# LANGUAGE AllowAmbiguousTypes #-} -{-# LANGUAGE DefaultSignatures #-} - -module IC.Wasm.Imports where - -import Data.Int -import Data.Word -import IC.Wasm.Winter -import Text.Printf -import Control.Monad.Except - - -class WasmArg a where - valueType :: ValueType - fromValue :: Value -> Either String a - toValue :: a -> Value - -class WasmArgs a where - stackType :: StackType - fromValues :: [Value] -> Either String a - toValues :: a -> [Value] - - default stackType :: WasmArg a => StackType - stackType = [valueType @a] - default fromValues :: WasmArg a => [Value] -> Either String a - fromValues [x] = fromValue x - fromValues xs = argError 1 xs - default toValues :: WasmArg a => a -> [Value] - toValues x = [toValue x] - -argError :: Int -> [a] -> Either String b -argError n xs = Left $ - printf "expected %d arguments, got %d arguments" n (length xs) - -instance WasmArg Int32 where - valueType = I32Type - fromValue (I32 i) = Right i - fromValue v = Left $ "expected i32, got " ++ show v - toValue = I32 -instance WasmArgs Int32 where - -instance WasmArg Word64 where - valueType = I64Type - fromValue (I64 i) = Right (fromIntegral i) - fromValue v = Left $ "expected i64, got " ++ show v - toValue = I64 . fromIntegral -instance WasmArgs Word64 where - -instance WasmArgs () where - stackType = [] - fromValues [] = Right () - fromValues xs = argError 0 xs - toValues () = [] - --- The formatting is a bit odd, but it allows creating new instances easily by copy and paste and adding lines -instance - ( WasmArg a1 - , WasmArg a2 - ) => WasmArgs - ( a1 - , a2 - ) where - stackType = - [ valueType @a1 - , valueType @a2 - ] - fromValues - [ x1 - , x2 - ] = (,) - <$> fromValue x1 - <*> fromValue x2 - fromValues xs = argError 2 xs - toValues - ( x1 - , x2 - ) = - [ toValue x1 - , toValue x2 - ] - -instance - ( WasmArg a1 - , WasmArg a2 - , WasmArg a3 - , WasmArg a4 - ) => WasmArgs - ( a1 - , a2 - , a3 - , a4 - ) where - stackType = - [ valueType @a1 - , valueType @a2 - , valueType @a3 - , valueType @a4 - ] - fromValues - [ x1 - , x2 - , x3 - , x4 - ] = (,,,) - <$> fromValue x1 - <*> fromValue x2 - <*> fromValue x3 - <*> fromValue x4 - fromValues xs = argError 3 xs - toValues - ( x1 - , x2 - , x3 - , x4 - ) = - [ toValue x1 - , toValue x2 - , toValue x3 - , toValue x4 - ] - -instance - ( WasmArg a1 - , WasmArg a2 - , WasmArg a3 - ) => WasmArgs - ( a1 - , a2 - , a3 - ) where - stackType = - [ valueType @a1 - , valueType @a2 - , valueType @a3 - ] - fromValues - [ x1 - , x2 - , x3 - ] = (,,) - <$> fromValue x1 - <*> fromValue x2 - <*> fromValue x3 - fromValues xs = argError 3 xs - toValues - ( x1 - , x2 - , x3 - ) = - [ toValue x1 - , toValue x2 - , toValue x3 - ] - -instance - ( WasmArg a1 - , WasmArg a2 - , WasmArg a3 - , WasmArg a4 - , WasmArg a5 - , WasmArg a6 - , WasmArg a7 - , WasmArg a8 - ) => WasmArgs - ( a1 - , a2 - , a3 - , a4 - , a5 - , a6 - , a7 - , a8 - ) where - stackType = - [ valueType @a1 - , valueType @a2 - , valueType @a3 - , valueType @a4 - , valueType @a5 - , valueType @a6 - , valueType @a7 - , valueType @a8 - ] - fromValues - [ x1 - , x2 - , x3 - , x4 - , x5 - , x6 - , x7 - , x8 - ] = (,,,,,,,) - <$> fromValue x1 - <*> fromValue x2 - <*> fromValue x3 - <*> fromValue x4 - <*> fromValue x5 - <*> fromValue x6 - <*> fromValue x7 - <*> fromValue x8 - fromValues xs = argError 88888888 xs - toValues - ( x1 - , x2 - , x3 - , x4 - , x5 - , x6 - , x7 - , x8 - ) = - [ toValue x1 - , toValue x2 - , toValue x3 - , toValue x4 - , toValue x5 - , toValue x6 - , toValue x7 - , toValue x8 - ] - -instance - ( WasmArg a1 - , WasmArg a2 - , WasmArg a3 - , WasmArg a4 - , WasmArg a5 - , WasmArg a6 - , WasmArg a7 - , WasmArg a8 - , WasmArg a9 - , WasmArg a10 - ) => WasmArgs - ( a1 - , a2 - , a3 - , a4 - , a5 - , a6 - , a7 - , a8 - , a9 - , a10 - ) where - stackType = - [ valueType @a1 - , valueType @a2 - , valueType @a3 - , valueType @a4 - , valueType @a5 - , valueType @a6 - , valueType @a7 - , valueType @a8 - , valueType @a9 - , valueType @a10 - ] - fromValues - [ x1 - , x2 - , x3 - , x4 - , x5 - , x6 - , x7 - , x8 - , x9 - , x10 - ] = (,,,,,,,,,) - <$> fromValue x1 - <*> fromValue x2 - <*> fromValue x3 - <*> fromValue x4 - <*> fromValue x5 - <*> fromValue x6 - <*> fromValue x7 - <*> fromValue x8 - <*> fromValue x9 - <*> fromValue x10 - fromValues xs = argError 10 xs - toValues - ( x1 - , x2 - , x3 - , x4 - , x5 - , x6 - , x7 - , x8 - , x9 - , x10 - ) = - [ toValue x1 - , toValue x2 - , toValue x3 - , toValue x4 - , toValue x5 - , toValue x6 - , toValue x7 - , toValue x8 - , toValue x9 - , toValue x10 - ] - -instance - ( WasmArg a1 - , WasmArg a2 - , WasmArg a3 - , WasmArg a4 - , WasmArg a5 - , WasmArg a6 - , WasmArg a7 - , WasmArg a8 - , WasmArg a9 - , WasmArg a10 - , WasmArg a11 - , WasmArg a12 - ) => WasmArgs - ( a1 - , a2 - , a3 - , a4 - , a5 - , a6 - , a7 - , a8 - , a9 - , a10 - , a11 - , a12 - ) where - stackType = - [ valueType @a1 - , valueType @a2 - , valueType @a3 - , valueType @a4 - , valueType @a5 - , valueType @a6 - , valueType @a7 - , valueType @a8 - , valueType @a9 - , valueType @a10 - , valueType @a11 - , valueType @a12 - ] - fromValues - [ x1 - , x2 - , x3 - , x4 - , x5 - , x6 - , x7 - , x8 - , x9 - , x10 - , x11 - , x12 - ] = (,,,,,,,,,,,) - <$> fromValue x1 - <*> fromValue x2 - <*> fromValue x3 - <*> fromValue x4 - <*> fromValue x5 - <*> fromValue x6 - <*> fromValue x7 - <*> fromValue x8 - <*> fromValue x9 - <*> fromValue x10 - <*> fromValue x11 - <*> fromValue x12 - fromValues xs = argError 12 xs - toValues - ( x1 - , x2 - , x3 - , x4 - , x5 - , x6 - , x7 - , x8 - , x9 - , x10 - , x11 - , x12 - ) = - [ toValue x1 - , toValue x2 - , toValue x3 - , toValue x4 - , toValue x5 - , toValue x6 - , toValue x7 - , toValue x8 - , toValue x9 - , toValue x10 - , toValue x11 - , toValue x12 - ] - - -toImport :: - forall a b s. - (WasmArgs a, WasmArgs b) => - String -> String -> (a -> HostM s b) -> Import s -toImport mod_name fun_name f = (mod_name, fun_name, stackType @a, stackType @b, f') - where - f' :: [Value] -> HostFunc s - f' xs = do - a <- withExceptT ((mod_name ++ "." ++ fun_name ++ ": ") ++) $ - ExceptT $ return (fromValues xs) - b <- f a - return $ toValues b - diff --git a/impl/src/IC/Wasm/Winter.hs b/impl/src/IC/Wasm/Winter.hs deleted file mode 100644 index 3fe8934cc..000000000 --- a/impl/src/IC/Wasm/Winter.hs +++ /dev/null @@ -1,136 +0,0 @@ -{-# LANGUAGE ScopedTypeVariables #-} -{-| - -This module provides a thin wrapper around the winter Wasm engine, exposing just -the bits needed by the IC ref. - -This is the interface at which one might plug in a different Wasm engine. --} -module IC.Wasm.Winter - ( Module - , parseModule - , exportedFunctions - , Import - , Imports - , HostM - , HostFunc - , W.Value(..) - , W.StackType - , W.ValueType(..) - , W.Address - , W.Size - , getBytes - , setBytes - , initialize - , Instance - , invokeExport - , invokeTable - ) -where - -import qualified Data.ByteString.Lazy as BS -import Control.Monad.Except -import qualified Data.Map as M -import qualified Data.Vector as V -import qualified Data.IntMap as IM -import qualified Data.Text.Lazy as T -import Control.Monad.ST -import Data.Binary.Get (runGetOrFail) -import Data.Default.Class (Default (..)) -import Data.Int -import Data.Foldable - -import qualified Wasm.Binary.Decode as W -import qualified Wasm.Exec.Eval as W -import qualified Wasm.Runtime.Func as W -import qualified Wasm.Runtime.Instance as W -import qualified Wasm.Runtime.Memory as W -import qualified Wasm.Syntax.AST as W -import qualified Wasm.Syntax.Types as W -import qualified Wasm.Syntax.Values as W -import qualified Wasm.Syntax.Memory as W -import qualified Wasm.Util.Source as W - -type Instance s = (IM.IntMap (W.ModuleInst W.Phrase (ST s)), Int) - -type HostM s = ExceptT String (ST s) - -type HostFunc s = HostM s [W.Value] - -type ModName = String -type FuncName = String -type Import s = (ModName, FuncName, W.StackType, W.StackType, [W.Value] -> HostFunc s) -type Imports s = [Import s] - -type Module = W.Module W.Phrase - -parseModule :: BS.ByteString -> Either String Module -parseModule bytes = case runGetOrFail W.getModule bytes of - Left (_,_,err) -> Left err - Right (_,_,wasm_mod) -> Right wasm_mod - - -initialize :: forall s. Module -> Imports s -> HostM s (Instance s) -initialize mod imps = withExceptT show $ do - let by_mod :: [(T.Text, [(T.Text, W.StackType, W.StackType, [W.Value] -> HostFunc s)])] - by_mod = M.toList $ M.fromListWith (<>) - [ (T.pack m, [(T.pack n,t1,t2,f)]) | (m,n,t1,t2,f) <- imps ] - - names :: M.Map T.Text Int - names = M.fromList (zip (map fst by_mod) [1..]) - - mods :: IM.IntMap (W.ModuleInst W.Phrase (ST s)) - mods = IM.fromList $ zip [1..] - [ (W.emptyModuleInst def) - { W._miGlobals = mempty - , W._miTables = mempty - , W._miMemories = mempty - , W._miFuncs = mempty - , W._miExports = M.fromList - [ (,) fname $ W.ExternFunc $ - W.allocHostEff (W.FuncType arg_ty ret_ty) - (\ args -> runExceptT $ f args) - | (fname, arg_ty, ret_ty, f) <- funcs - ] - } - | (_name, funcs) <- by_mod - ] - (ref, inst, start_err) <- W.initialize mod names mods - for_ start_err throwError - let mods' = IM.insert ref inst mods - return (mods', ref) - - -exportedFunctions :: Module -> [FuncName] -exportedFunctions wasm_mod = - [ T.unpack (W._exportName e) - | W.Phrase _ e <- V.toList $ W._moduleExports wasm_mod - , W.FuncExport {} <- return $ W._exportDesc e - ] - - -invokeExport :: Instance s -> FuncName -> [W.Value] -> HostM s [W.Value] -invokeExport (mods', ref) method args = do - let inst = mods' IM.! ref - withExceptT show $ - W.invokeByName mods' inst (T.pack method) args - -invokeTable :: Instance s -> Int32 -> [W.Value] -> HostM s [W.Value] -invokeTable (mods', ref) idx args = do - let inst = mods' IM.! ref - withExceptT show $ do - func <- W.elem inst (0 W.@@ def) idx def - W.invoke mods' inst func args - -getBytes :: Instance s -> W.Address -> W.Size -> HostM s BS.ByteString -getBytes (mods', ref) ptr len = do - let inst = mods' IM.! ref - let mem = V.head (W._miMemories inst) - withExceptT show $ W.loadBytes mem ptr len - -setBytes :: Instance s -> W.Address -> BS.ByteString -> HostM s () -setBytes (mods', ref) ptr blob = do - let inst = mods' IM.! ref - let mem = V.head (W._miMemories inst) - withExceptT show $ W.storeBytes mem (fromIntegral ptr) blob - diff --git a/impl/src/IC/Wasm/Winter/Persist.hs b/impl/src/IC/Wasm/Winter/Persist.hs deleted file mode 100644 index 7849899a5..000000000 --- a/impl/src/IC/Wasm/Winter/Persist.hs +++ /dev/null @@ -1,128 +0,0 @@ -{-# LANGUAGE ExplicitForAll #-} -{-# LANGUAGE TypeFamilies #-} -{-# LANGUAGE MultiParamTypeClasses #-} -{-# LANGUAGE FlexibleInstances #-} -{-# LANGUAGE FlexibleContexts #-} -{- | -This module provides a way to persist the state of a Winter Wasm instance, and -to recover it. - -It is tailored to the use by ic-ref. For example it assumes that the -table of a wasm instance is immutable. --} -module IC.Wasm.Winter.Persist - ( PInstance(..) - , PModuleInst(..) - , persistInstance - , resumeInstance - , persistMemory - , resumeMemory - ) - where - -import Control.Monad -import Control.Monad.ST -import Data.Primitive.MutVar -import qualified Data.IntMap as IM -import qualified Data.Map.Lazy as M -import qualified Data.Vector as V -import Data.ByteString.Lazy (ByteString) - -import qualified Wasm.Runtime.Global as W -import qualified Wasm.Runtime.Instance as W -import qualified Wasm.Runtime.Memory as W -import qualified Wasm.Syntax.Values as W -import qualified Wasm.Util.Source as W - -import IC.Wasm.Winter (Instance) - --- | --- This stores data read from an instance. -newtype PInstance = PInstance (Persisted (Instance ())) - deriving Show - -persistInstance :: Instance s -> ST s PInstance -persistInstance i = PInstance <$> persist i - -resumeInstance :: Instance s -> PInstance -> ST s () -resumeInstance i (PInstance p) = resume i p - -persistMemory :: W.MemoryInst (ST s) -> ST s ByteString -persistMemory i = persist i - -resumeMemory :: W.MemoryInst (ST s) -> ByteString -> ST s () -resumeMemory i p = resume i p - -class Monad (M a) => Persistable a where - type Persisted a :: * - type M a :: * -> * - persist :: a -> M a (Persisted a) - resume :: a -> Persisted a -> M a () - -instance Persistable (W.MemoryInst (ST s)) where - type Persisted (W.MemoryInst (ST s)) = ByteString - type M (W.MemoryInst (ST s)) = ST s - persist = W.exportMemory - resume = W.importMemory - -instance Persistable (W.GlobalInst (ST s)) where - type Persisted (W.GlobalInst (ST s)) = W.Value - type M (W.GlobalInst (ST s)) = ST s - persist m = readMutVar (W._giContent m) - resume m = writeMutVar (W._giContent m) - -data PModuleInst = PModuleInst - { memories :: V.Vector (Persisted (W.MemoryInst (ST ()))) - , globals :: V.Vector (Persisted (W.GlobalInst (ST ()))) - } - deriving Show - -instance Persistable (W.ModuleInst W.Phrase (ST s)) where - type Persisted (W.ModuleInst W.Phrase (ST s)) = PModuleInst - type M (W.ModuleInst W.Phrase (ST s)) = ST s - persist inst = PModuleInst - <$> persist (W._miMemories inst) - <*> persist (W._miGlobals inst) - resume inst pinst = do - resume (W._miMemories inst) (memories pinst) - resume (W._miGlobals inst) (globals pinst) - - -instance Persistable a => Persistable [a] where - type Persisted [a] = [Persisted a] - type M [a] = M a - persist = mapM persist - resume xs ys = do - unless (length xs == length ys) $ error "Lengths don’t match" - zipWithM_ resume xs ys - -instance Persistable a => Persistable (V.Vector a) where - type Persisted (V.Vector a) = V.Vector (Persisted a) - type M (V.Vector a) = M a - persist = mapM persist - resume xs ys = do - unless (V.length xs == V.length ys) $ error "Lengths don’t match" - V.zipWithM_ resume xs ys - -instance (Eq k, Persistable a) => Persistable (M.Map k a) where - type Persisted (M.Map k a) = M.Map k (Persisted a) - type M (M.Map k a) = M a - persist = mapM persist - resume xs ys = do - unless (M.keys xs == M.keys ys) $ error "Map keys don’t match" - zipWithM_ resume (M.elems xs) (M.elems ys) - -instance Persistable a => Persistable (IM.IntMap a) where - type Persisted (IM.IntMap a) = M.Map Int (Persisted a) - type M (IM.IntMap a) = M a - persist = mapM persist . M.fromList . IM.toList - resume xs ys = do - let ys' = IM.fromList (M.toList ys) - unless (IM.keys xs == IM.keys ys') $ error "Map keys don’t match" - zipWithM_ resume (IM.elems xs) (IM.elems ys') - -instance Persistable a => Persistable (a, Int) where - type Persisted (a, Int) = Persisted a - type M (a, Int) = M a - persist (a, _i) = persist a - resume (a, _i) p = resume a p diff --git a/impl/src/IC/Wasm/WinterMemory.hs b/impl/src/IC/Wasm/WinterMemory.hs deleted file mode 100644 index 95053242a..000000000 --- a/impl/src/IC/Wasm/WinterMemory.hs +++ /dev/null @@ -1,58 +0,0 @@ -{-# LANGUAGE ScopedTypeVariables #-} -{-| - -This module provides a thin wrapper around the winter Wasm engine, exposing just -the bits needed for accessing the stable memory. This will hopefully go away once -multiple memories are supported. --} -module IC.Wasm.WinterMemory - ( Memory - , new - , size - , grow - , read - , write - , export - , imp - ) -where - -import Prelude hiding (read) - -import Data.ByteString.Lazy (ByteString) -import Control.Monad.Except -import Control.Monad.ST - -import qualified Wasm.Runtime.Memory as W -import qualified Wasm.Syntax.Types as W -import qualified Wasm.Syntax.Memory as W - -type HostM s = ExceptT String (ST s) -type Memory s = W.MemoryInst (ST s) - -new :: HostM s (Memory s) -new = withExceptT show $ W.alloc (W.Limits 0 Nothing) - -size :: Memory s -> HostM s W.Size -size mem = lift $ W.size mem - -grow :: Memory s -> W.Size -> HostM s W.Size -grow mem delta = lift $ do - -- See memory.grow impl in src/Wasm/Exec/Eval.hs - oldSize <- W.size mem - eres <- runExceptT $ W.grow mem delta - return $ case eres of - Left _ -> -1 - Right () -> oldSize - -read :: Memory s -> W.Address -> W.Size -> HostM s ByteString -read mem ptr len = withExceptT show $ W.loadBytes mem ptr len - -write :: Memory s -> W.Address -> ByteString -> HostM s () -write mem ptr blob = withExceptT show $ W.storeBytes mem (fromIntegral ptr) blob - -export :: Memory s -> ST s ByteString -export = W.exportMemory - -imp :: Memory s -> ByteString -> ST s () -imp = W.importMemory diff --git a/impl/src/SourceId.hs b/impl/src/SourceId.hs deleted file mode 100644 index 0f51c450b..000000000 --- a/impl/src/SourceId.hs +++ /dev/null @@ -1,47 +0,0 @@ -{- | -This module exports a source id in a way that works well within a nix build (no -.git available) and outside nix, according to this logic: - - * If `git` works, use `git describe` - * Else, if $out is set (so this is a nix build), extract an identifer from the out hash - * Else, it says something like unidentified build. - -This is an early experiment. If successful, this logic ought to move into a -library, and maybe also implemented for rust artifacts. (see RPL-101) - -Note that cabal build will not recompile this module if it does not have to, so in local development, this is less reliable than it should. See -https://www.joachim-breitner.de/blog/772-Template_Haskell_recompilation -for more details. - --} -{-# LANGUAGE TemplateHaskell #-} -{-# LANGUAGE LambdaCase #-} -{-# LANGUAGE ScopedTypeVariables #-} -module SourceId where - -import Language.Haskell.TH -import Control.Monad -import Data.List -import Data.List.Split -import System.Process -import System.Environment -import Control.Exception - -id :: String -id = $(stringE <=< runIO $ do - -- Leniently calls git, and removes final newline from git’s output - let readGit args = intercalate "\n" . lines <$> catch - (readCreateProcess ((proc "git" args) {std_err = CreatePipe}) "") - (\(_ :: IOException) -> return "") - - inGit <- readGit ["rev-parse", "--is-inside-work-tree"] - if inGit == "true" - then readGit ["describe", "--tags", "--match=v*", "--dirty"] - else lookupEnv "out" >>= \case - Just path - | ["","nix","store",base] <- splitOn "/" path - , let hash = takeWhile (/= '-') base - -> return $ intercalate "-" (chunksOf 8 hash) - Just path -> fail $ "SouceId: unparsable $out=" ++ path - Nothing -> return "unidentified" - ) diff --git a/impl/src/ic-ref-run.hs b/impl/src/ic-ref-run.hs deleted file mode 100644 index 42bf4e9f6..000000000 --- a/impl/src/ic-ref-run.hs +++ /dev/null @@ -1,185 +0,0 @@ -{-# LANGUAGE ConstraintKinds #-} -{-# LANGUAGE NumericUnderscores #-} -{-# LANGUAGE FlexibleContexts #-} -{-# LANGUAGE LambdaCase #-} -{-# LANGUAGE RankNTypes #-} -{-# LANGUAGE OverloadedLabels #-} -{-# LANGUAGE DataKinds #-} -{-# LANGUAGE ScopedTypeVariables #-} -{-# LANGUAGE TypeFamilies #-} -{-# LANGUAGE TypeOperators #-} - -module Main where - -import Options.Applicative hiding (empty) -import Control.Monad -import qualified Data.Map as M -import qualified Data.ByteString.Lazy as B -import qualified Data.ByteString.Lazy.Char8 as BC -import qualified Data.ByteString.Builder as B -import qualified Data.Text as T -import Control.Monad.Trans -import Control.Monad.Trans.State -import Text.Printf -import Data.List -import Data.Text.Prettyprint.Doc (pretty) -import Data.Time.Clock.POSIX -import Control.Monad.Random.Lazy - -import GHC.TypeLits (KnownSymbol, symbolVal) -import Data.Row (empty, (.==), (.+), type (.!), Label) -import qualified Codec.Candid as Candid -import qualified Data.Row.Variants as V - - -import IC.Version -import IC.Types -import IC.Ref -import IC.DRun.Parse (Ingress(..), parseFile) -import IC.Management - - -type DRun = StateT IC IO - -dummyUserId :: CanisterId -dummyUserId = EntityId $ B.pack [0xCA, 0xFF, 0xEE] - --- Pretty printing - -printCallRequest :: CallRequest -> IO () -printCallRequest (CallRequest _ _ method arg) = - printf "→ update %s%s\n" method (shorten 60 (candidOrPretty arg)) - -printReadStateRequest :: ReadStateRequest -> IO () -printReadStateRequest (ReadStateRequest _ paths) = - printf "→ state? %s\n" (intercalate ", " $ map (intercalate "/" . map show) paths) - -printQueryRequest :: QueryRequest -> IO () -printQueryRequest (QueryRequest _ _ method arg) = - printf "→ query %s%s\n" method (shorten 60 (candidOrPretty arg)) - -printCallResponse :: CallResponse -> IO () -printCallResponse (Rejected (c, s)) = - printf "← rejected (%s): %s\n" (show c) s -printCallResponse (Replied blob) = - printf "← replied: %s\n" (shorten 100 (candidOrPretty blob)) - -printReqStatus :: RequestStatus -> IO () -printReqStatus Received = - printf "← received\n" -printReqStatus Processing = - printf "← processing\n" -printReqStatus (CallResponse c) = printCallResponse c - -printReqResponse :: ReqResponse -> IO () -printReqResponse (QueryResponse c) = printCallResponse c -printReqResponse (ReadStateResponse _ ) = error "dead code in ic-ref" - -candidOrPretty :: Blob -> String -candidOrPretty b - | BC.pack "DIDL" `B.isPrefixOf` b - , Right vs <- Candid.decodeVals b - = show (pretty vs) - | otherwise - = "(" ++ prettyBlob b ++ ")" - - -shorten :: Int -> String -> String -shorten n s = a ++ (if null b then "" else "…") - where (a,b) = splitAt n s - - -submitAndRun :: CallRequest -> DRun () -submitAndRun r = do - lift $ printCallRequest r - rid <- lift mkRequestId - submitRequest rid r - runToCompletion - r <- gets (snd . (M.! rid) . requests) - lift $ printReqStatus r - -submitQuery :: QueryRequest -> DRun () -submitQuery r = do - lift $ printQueryRequest r - t <- lift getTimestamp - r <- handleQuery t r - lift $ printReqResponse r - where - getTimestamp :: IO Timestamp - getTimestamp = do - t <- getPOSIXTime - return $ Timestamp $ round (t * 1000_000_000) - -mkRequestId :: IO RequestID -mkRequestId = B.toLazyByteString . B.word64BE <$> randomIO - -callManagement :: forall s a b. - KnownSymbol s => - (a -> IO b) ~ (ICManagement IO .! s) => - Candid.CandidArg a => - EntityId -> Label s -> a -> StateT IC IO () -callManagement user_id l x = - submitAndRun $ - CallRequest (EntityId mempty) user_id (symbolVal l) (Candid.encode x) - -work :: FilePath -> IO () -work msg_file = do - msgs <- parseFile msg_file - - let user_id = dummyUserId - ic <- initialIC - flip evalStateT ic $ - forM_ msgs $ \case - Create -> - callManagement user_id #create_canister $ empty - .+ #settings .== Nothing - Install cid filename arg -> do - wasm <- liftIO $ B.readFile filename - callManagement user_id #install_code $ empty - .+ #mode .== V.IsJust #install () - .+ #canister_id .== Candid.Principal cid - .+ #wasm_module .== wasm - .+ #arg .== arg - Reinstall cid filename arg -> do - wasm <- liftIO $ B.readFile filename - callManagement user_id #install_code $ empty - .+ #mode .== V.IsJust #reinstall () - .+ #canister_id .== Candid.Principal cid - .+ #wasm_module .== wasm - .+ #arg .== arg - Upgrade cid filename arg -> do - wasm <- liftIO $ B.readFile filename - callManagement user_id #install_code $ empty - .+ #mode .== V.IsJust #upgrade () - .+ #canister_id .== Candid.Principal cid - .+ #wasm_module .== wasm - .+ #arg .== arg - Query cid method arg -> - submitQuery (QueryRequest (EntityId cid) user_id method arg) - Update cid method arg -> - submitAndRun (CallRequest (EntityId cid) user_id method arg) - -main :: IO () -main = join . customExecParser (prefs showHelpOnError) $ - info (helper <*> versions <*> parser) - ( fullDesc - <> header ("Internet Computer canister runner " <> T.unpack implVersion) - <> progDesc "This runs an IC canister against a list of messages." - ) - where - versions :: Parser (a -> a) - versions = - infoOption (T.unpack implVersion) (long "version" <> help "show version number") - <*> infoOption (T.unpack specVersion) (long "spec-version" <> help "show spec version number") - parser :: Parser (IO ()) - parser = work - <$ strOption - ( long "config" - <> short 'c' - <> metavar "CONFIG" - <> value "" - ) - <*> strArgument - ( metavar "script" - <> help "messages to execute" - ) diff --git a/impl/src/ic-ref-test.hs b/impl/src/ic-ref-test.hs deleted file mode 100644 index 0b01594bd..000000000 --- a/impl/src/ic-ref-test.hs +++ /dev/null @@ -1,31 +0,0 @@ -{-# LANGUAGE LambdaCase #-} -{-# LANGUAGE ScopedTypeVariables #-} -{-# OPTIONS_GHC -Wno-orphans #-} -module Main (main) where - -import Test.Tasty -import Test.Tasty.Ingredients -import Test.Tasty.Ingredients.Basic -import Test.Tasty.Ingredients.Rerun -import Test.Tasty.Runners.AntXML -import Test.Tasty.Runners.Html -import Test.Tasty.Runners - -import IC.Test.Options -import IC.Test.Spec -import qualified IC.Crypto.BLS as BLS - -main :: IO () -main = do - BLS.init - os <- parseOptions ingredients (testGroup "dummy" []) - tc <- preFlight os - defaultMainWithIngredients ingredients (icTests tc) - where - ingredients = - [ rerunningTests - [ listingTests - , includingOptions [endpointOption] - , antXMLRunner `composeReporters` htmlRunner `composeReporters` consoleTestReporter - ] - ] diff --git a/impl/src/ic-ref.hs b/impl/src/ic-ref.hs deleted file mode 100644 index f497396a4..000000000 --- a/impl/src/ic-ref.hs +++ /dev/null @@ -1,81 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -import Options.Applicative -import Data.Foldable -import Control.Concurrent -import Control.Monad (join, forever) -import Network.Wai.Middleware.RequestLogger -import Network.Wai.Handler.Warp -import qualified Data.Text as T -import IC.HTTP -import IC.Version -import qualified IC.Crypto.BLS as BLS - -defaultPort :: Port -defaultPort = 8001 - - -work :: Maybe Int -> Maybe FilePath -> Maybe FilePath -> Bool -> IO () -work portToUse writePortTo backingFile log = do - putStrLn "Starting ic-ref..." - BLS.init - withApp backingFile $ \app -> do - let app' = if log then logStdoutDev app else app - case portToUse of - Nothing -> - withApplicationSettings settings (pure app') $ \port -> do - greet port - forever (threadDelay maxBound) - Just port -> do - greet port - runSettings (setPort port settings) app' - where - greet port = do - putStrLn $ "Running at http://127.0.0.1:" ++ show port ++ "/" - for_ writePortTo $ \fn -> writeFile fn (show port) - - settings = setHost "127.0.0.1" defaultSettings - -main :: IO () -main = join . customExecParser (prefs showHelpOnError) $ - info (helper <*> versions <*> parser) - ( fullDesc - <> header ("Internet Computer reference implementation " <> T.unpack implVersion) - <> progDesc ( - "A stand-alone local reference implementation of the Internet Computer. \ - \By default, it listens on http://127.0.0.1:" ++ show defaultPort ++ "/. You \ - \can change the port with --pick-port or --listen-port.") - ) - where - versions :: Parser (a -> a) - versions = - infoOption (T.unpack implVersion) (long "version" <> help "show version number") - <*> infoOption (T.unpack specVersion) (long "spec-version" <> help "show spec version number") - parser :: Parser (IO ()) - parser = work - <$> - ( flag' Nothing - ( long "pick-port" - <> help ("pick a free port (instead of binding to 127.0.0.1:" ++ show defaultPort ++ ")") - ) - <|> - (Just <$> - option auto - ( long "listen-port" - <> help "specify the listen port" - ) - ) - <|> pure (Just defaultPort) - ) - <*> optional (strOption - ( long "write-port-to" - <> help "write port to the given file" - )) - <*> optional (strOption - ( long "state-file" - <> metavar "FILE" - <> help "file to persist IC state in" - )) - <*> switch - ( long "http-log" - <> help "print a HTTP log to stdout" - ) diff --git a/impl/src/ic-request-id.hs b/impl/src/ic-request-id.hs deleted file mode 100644 index a3c93b3a0..000000000 --- a/impl/src/ic-request-id.hs +++ /dev/null @@ -1,45 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -module Main where - -import Options.Applicative -import Control.Monad (join) -import qualified Data.ByteString.Lazy as BS -import qualified Data.Text as T -import qualified Data.Text.IO as T -import qualified Text.Hex as H -import qualified Data.HashMap.Lazy as HM -import System.IO - - -import IC.HTTP.CBOR -import IC.HTTP.GenR -import IC.HTTP.RequestId - -work :: Maybe FilePath -> IO () -work input = do - request <- maybe BS.getContents BS.readFile input - case decode request of - Left err -> do - T.hPutStrLn stderr "Failed to decode CBOR:" - T.hPutStrLn stderr err - Right (GRec m) | Just content <- HM.lookup "content" m -> - T.putStrLn $ H.encodeHex $ BS.toStrict $ requestId content - Right gr -> do - T.hPutStrLn stderr "Request does not look like an envelop (could not find field \"content\"):" - T.hPutStrLn stderr (T.pack (show gr)) - -main :: IO () -main = join . customExecParser (prefs showHelpOnError) $ - info (helper <*> parser) - ( fullDesc - <> header "Internet Computer request id" - <> progDesc "Given a CBOR-encoded request with envelope, calculate the request id" - ) - where - parser :: Parser (IO ()) - parser = - work - <$> optional (strArgument - ( metavar "FILE" - <> help "file to read (default: stdin)" - )) diff --git a/impl/src/unit-tests.hs b/impl/src/unit-tests.hs deleted file mode 100644 index de2dd2708..000000000 --- a/impl/src/unit-tests.hs +++ /dev/null @@ -1,74 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE NamedFieldPuns #-} -module Main (main) where - -import Test.Tasty -import Test.Tasty.HUnit - -import System.IO -import System.IO.Temp -import System.Directory -import qualified Data.Map as M -import qualified Data.Set as S - -import qualified IC.Crypto.BLS as BLS -import IC.Ref -import IC.Types -import IC.Serialise () -import IC.StateFile -import IC.Test.HashTree -import IC.Test.BLS -import IC.Test.WebAuthn -import IC.Test.ECDSA -import IC.Test.Secp256k1 -import IC.HTTP.GenR -import IC.HTTP.RequestId - -main :: IO () -main = do - BLS.init - defaultMain tests - -tests :: TestTree -tests = testGroup "ic-ref unit tests" - [ testCase "Request id calculation from interface spec" $ - let gr = GRec $ mconcat - [ "request_type" =: GText "call" - , "canister_id" =: GBlob "\x00\x00\x00\x00\x00\x00\x04\xD2" - , "method_name" =: GText "hello" - , "arg" =: GBlob "DIDL\x00\xFD*" - ] - in requestId gr @?= "\x87\x81\x29\x1c\x34\x7d\xb3\x2a\x9d\x8c\x10\xeb\x62\xb7\x10\xfc\xe5\xa9\x3b\xe6\x76\x47\x4c\x42\xba\xbc\x74\xc5\x18\x58\xf9\x4b" - , hashTreeTests - , blsTests - , webAuthnTests - , ecdsaTests - , secp256k1Tests - , testGroup "State serialization" - [ testCase "with file" $ - withSystemTempFile "ic-ref-unit-test.state" $ \fn h -> do - -- start with an empty file - hClose h - removeFile fn - - -- Create the state - withStore initialIC (Just fn) $ \store -> do - modifyStore store $ submitRequest "dummyrequestid" $ - CallRequest (EntityId mempty) (EntityId "yay") "create_canister" "DIDL\x01\x6c\0\1\0" - - -- now the file should exist - doesFileExist fn >>= assertBool "File exists" - - withStore initialIC (Just fn) $ \store -> do - ic <- peekStore store - assertBool "No canisters yet expected" (null (canisters ic)) - modifyStore store runToCompletion - - withStore initialIC (Just fn) $ \store -> do - ic <- peekStore store - case M.elems (canisters ic) of - [] -> assertFailure "No canisters created" - [CanState {controllers}] -> controllers @?= S.singleton (EntityId "yay") - _ -> assertFailure "Too many canisters?" - ] - ] diff --git a/impl/test-data/trivial.wat b/impl/test-data/trivial.wat deleted file mode 100644 index 3af8f2545..000000000 --- a/impl/test-data/trivial.wat +++ /dev/null @@ -1 +0,0 @@ -(module) diff --git a/impl/test-data/universal_canister.wasm b/impl/test-data/universal_canister.wasm deleted file mode 120000 index d4d2cdd96..000000000 --- a/impl/test-data/universal_canister.wasm +++ /dev/null @@ -1 +0,0 @@ -../../universal-canister/target/wasm32-unknown-unknown/release/universal_canister.wasm \ No newline at end of file diff --git a/nix/default.nix b/nix/default.nix index 7387d74c4..b4980a4f4 100644 --- a/nix/default.nix +++ b/nix/default.nix @@ -6,43 +6,12 @@ let }; nixpkgs_src = (import sourcesnix { sourcesFile = ./sources.json; inherit pkgs; }).nixpkgs; - bootstrap-pkgs = import nixpkgs_src { - system = builtins.currentSystem; - }; - - nixpkgs-patched = bootstrap-pkgs.applyPatches { - name = "nixpkgs-patched"; - src = nixpkgs_src; - patches = [ - ./patches/0001-ghc865-binary-Use-binary-distribution-which-links-ag.patch - ./patches/0002-openblas-0.3.10-0.3.13.patch - ./patches/fb063991b26b2b93dece6d09f37041451a5ef4cb.patch - ]; - }; - pkgs = - import nixpkgs-patched { + import nixpkgs_src { inherit system; overlays = [ (self: super: { sources = import sourcesnix { sourcesFile = ./sources.json; pkgs = super; }; - - - # nixpkgs's rustc does not inclue the wasm32-unknown-unknown target, so - # lets add it here. With this we can build the universal canister with stock - # nixpkgs + naersk, in particular no dependency on internal repositories. - rustc = super.rustc.overrideAttrs (old: { - configureFlags = self.lib.lists.forEach old.configureFlags (flag: - if self.lib.strings.hasPrefix "--target=" flag - then flag + ",wasm32-unknown-unknown" - else flag - ); - }); - - all-cabal-hashes = self.fetchurl { - url = "https://github.com/commercialhaskell/all-cabal-hashes/archive/f18d8ab7adfbd15acfc5e994dfb973577a5aba5c.tar.gz"; - sha256 = "0kn2wqilpw0nyx54jyz4vp7xrx1893zdv7d54yi9pjl677pnwcs9"; - }; }) ]; }; diff --git a/nix/generate.nix b/nix/generate.nix deleted file mode 100644 index dc36821aa..000000000 --- a/nix/generate.nix +++ /dev/null @@ -1,113 +0,0 @@ -# This file generates the contents of nix/generated/. Use -# -# nix-shell generate.nix -# -# to update - -{ pkgs ? import ../nix {} }: - -let - - # `haskellSrc2nixWithDoc` is used to generate `default.nix` files for - # Haskell packages which are intended to be stored in the repository. - # - # The function generates a directory containing a `default.nix` which - # is the result of running `cabal2nix` with the `extraCabal2nixOptions` - # on the provided `src`. - # - # A header is added to `default.nix` which contains instructions on - # how to regenerate that file. - # - # Finally the `src` attribute in the `default.nix` will be defined as - # `src_subst` such that it can be pointed to local or niv-managed - # sources. - haskellSrc2nixWithDoc = {name, src, src_subst, extraCabal2nixOptions ? ""}: - let - drv = pkgs.haskellPackages.haskellSrc2nix { - inherit name extraCabal2nixOptions src; - }; - in drv.overrideAttrs (oldAttrs: { - message = '' - # THIS IS AN AUTOMATICALLY GENERATED FILE. DO NOT EDIT MANUALLY!\ - # See ./nix/generate.nix for instructions.\ - - ''; - inherit src_subst; - installPhase = oldAttrs.installPhase + '' - sed -i "1i$message;s|src = .*|src = $src_subst;|" $out/default.nix - # Accept `pkgs` as an argument in case the `src_subst` depends on it. - sed -i "s|{ mkDerivation|{ mkDerivation, pkgs|" $out/default.nix - ''; - }); - - # A variant of `haskellSrc2nixWithDoc` for local Haskell packages. - localHaskellSrc2nixWithDoc = name: path: extraCabal2nixOptions: - haskellSrc2nixWithDoc { - inherit name extraCabal2nixOptions; - src = import ./gitSource.nix path; - src_subst = "import ../gitSource.nix \"${path}\""; - }; - - packages = { - winter = haskellSrc2nixWithDoc { - name = "winter"; - src = pkgs.sources.winter; - src_subst = "pkgs.sources.winter"; - extraCabal2nixOptions = "--no-check"; - }; - leb128-cereal = haskellSrc2nixWithDoc { - name = "leb128-cereal"; - src = pkgs.sources.leb128-cereal; - src_subst = "pkgs.sources.leb128-cereal"; - }; - candid = haskellSrc2nixWithDoc { - name = "candid"; - src = pkgs.sources.haskell-candid; - src_subst = "pkgs.sources.haskell-candid"; - }; - - ic-ref = localHaskellSrc2nixWithDoc "ic-ref" "impl" "--no-check -frelease"; - base32 = pkgs.haskellPackages.hackage2nix "base32" "0.1.1.2"; - megaparsec = pkgs.haskellPackages.hackage2nix "megaparsec" "8.0.0"; - base64-bytestring = pkgs.haskellPackages.hackage2nix "base64-bytestring" "1.1.0.0"; - random = pkgs.haskellPackages.hackage2nix "random" "1.2.0"; - splitmix = pkgs.haskellPackages.hackage2nix "splitmix" "0.1.0.3"; - QuickCheck = pkgs.haskellPackages.hackage2nix "QuickCheck" "2.14.2"; - }; - - allGenerated = pkgs.runCommandNoCC "generated" { - buildInputs = [ pkgs.nixpkgs-fmt ]; - } ( - '' - mkdir -p $out - '' + builtins.concatStringsSep "" ( - pkgs.lib.flip pkgs.lib.mapAttrsToList packages ( - n: pkg: '' - cp ${pkg}/default.nix $out/${n}.nix - '' - ) - ) + '' - chmod u+w $out/*.nix - nixpkgs-fmt $out/*.nix - echo <<__END__ > $out/README.md - The contents of this directory are automatically generated. - To update, please run nix-shell generate.nix - __END__ - '' - ); -in -allGenerated.overrideAttrs ( - old: { - shellHook = if pkgs.lib.inNixShell then - '' - dest=${toString ./generated} - - rm -f $dest/*.nix $dest/README.md - cp -v -t $dest/ ${allGenerated}/* - chmod u-w -R $dest/* - - exit 0 - '' else null; - } -) - diff --git a/nix/generated/QuickCheck.nix b/nix/generated/QuickCheck.nix deleted file mode 100644 index 3ba1dffe1..000000000 --- a/nix/generated/QuickCheck.nix +++ /dev/null @@ -1,29 +0,0 @@ -{ mkDerivation -, base -, containers -, deepseq -, process -, random -, splitmix -, stdenv -, template-haskell -, transformers -}: -mkDerivation { - pname = "QuickCheck"; - version = "2.14.2"; - sha256 = "d87b6c85696b601175274361fa62217894401e401e150c3c5d4013ac53cd36f3"; - libraryHaskellDepends = [ - base - containers - deepseq - random - splitmix - template-haskell - transformers - ]; - testHaskellDepends = [ base deepseq process ]; - homepage = "https://github.com/nick8325/quickcheck"; - description = "Automatic testing of Haskell programs"; - license = stdenv.lib.licenses.bsd3; -} diff --git a/nix/generated/base32.nix b/nix/generated/base32.nix deleted file mode 100644 index ddf73a328..000000000 --- a/nix/generated/base32.nix +++ /dev/null @@ -1,39 +0,0 @@ -{ mkDerivation -, base -, bytestring -, criterion -, deepseq -, memory -, random-bytestring -, stdenv -, tasty -, tasty-hunit -, text -}: -mkDerivation { - pname = "base32"; - version = "0.1.1.2"; - sha256 = "0e6211a58cccbaae9c583c800d99db421cdb259170693a82ad9aa2afd795dfd6"; - libraryHaskellDepends = [ base bytestring text ]; - testHaskellDepends = [ - base - bytestring - memory - random-bytestring - tasty - tasty-hunit - text - ]; - benchmarkHaskellDepends = [ - base - bytestring - criterion - deepseq - memory - random-bytestring - text - ]; - homepage = "https://github.com/emilypi/base32"; - description = "RFC 4648-compliant Base32 encodings/decodings"; - license = stdenv.lib.licenses.bsd3; -} diff --git a/nix/generated/base64-bytestring.nix b/nix/generated/base64-bytestring.nix deleted file mode 100644 index f506b856e..000000000 --- a/nix/generated/base64-bytestring.nix +++ /dev/null @@ -1,41 +0,0 @@ -{ mkDerivation -, base -, bytestring -, containers -, criterion -, deepseq -, HUnit -, QuickCheck -, split -, stdenv -, test-framework -, test-framework-hunit -, test-framework-quickcheck2 -}: -mkDerivation { - pname = "base64-bytestring"; - version = "1.1.0.0"; - sha256 = "210d6c9042241ca52ee5d89cf221dbeb4d0e64b37391345369035ad2d9b4aca9"; - libraryHaskellDepends = [ base bytestring ]; - testHaskellDepends = [ - base - bytestring - containers - HUnit - QuickCheck - split - test-framework - test-framework-hunit - test-framework-quickcheck2 - ]; - benchmarkHaskellDepends = [ - base - bytestring - containers - criterion - deepseq - ]; - homepage = "https://github.com/haskell/base64-bytestring"; - description = "Fast base64 encoding and decoding for ByteStrings"; - license = stdenv.lib.licenses.bsd3; -} diff --git a/nix/generated/candid.nix b/nix/generated/candid.nix deleted file mode 100644 index 02cc51973..000000000 --- a/nix/generated/candid.nix +++ /dev/null @@ -1,99 +0,0 @@ -# THIS IS AN AUTOMATICALLY GENERATED FILE. DO NOT EDIT MANUALLY! -# See ./nix/generate.nix for instructions. - -{ mkDerivation -, pkgs -, base -, base32 -, bytestring -, cereal -, constraints -, containers -, crc -, directory -, dlist -, doctest -, filepath -, hex-text -, leb128-cereal -, megaparsec -, mtl -, optparse-applicative -, prettyprinter -, row-types -, scientific -, smallcheck -, split -, stdenv -, tasty -, tasty-hunit -, tasty-quickcheck -, tasty-rerun -, tasty-smallcheck -, template-haskell -, text -, transformers -, unordered-containers -, vector -}: -mkDerivation { - pname = "candid"; - version = "0.1"; - src = pkgs.sources.haskell-candid; - isLibrary = true; - isExecutable = true; - libraryHaskellDepends = [ - base - base32 - bytestring - cereal - constraints - containers - crc - dlist - hex-text - leb128-cereal - megaparsec - mtl - prettyprinter - row-types - scientific - split - template-haskell - text - transformers - unordered-containers - vector - ]; - executableHaskellDepends = [ - base - bytestring - hex-text - optparse-applicative - prettyprinter - text - ]; - testHaskellDepends = [ - base - bytestring - directory - doctest - filepath - leb128-cereal - prettyprinter - row-types - smallcheck - tasty - tasty-hunit - tasty-quickcheck - tasty-rerun - tasty-smallcheck - template-haskell - text - unordered-containers - vector - ]; - homepage = "https://github.com/dfinity/candid"; - description = "Candid integration"; - license = stdenv.lib.licenses.asl20; -} diff --git a/nix/generated/ic-ref.nix b/nix/generated/ic-ref.nix deleted file mode 100644 index ea447eb1e..000000000 --- a/nix/generated/ic-ref.nix +++ /dev/null @@ -1,185 +0,0 @@ -# THIS IS AN AUTOMATICALLY GENERATED FILE. DO NOT EDIT MANUALLY! -# See ./nix/generate.nix for instructions. - -{ mkDerivation -, pkgs -, aeson -, asn1-encoding -, asn1-types -, atomic-write -, base -, base32 -, base64-bytestring -, binary -, bindings-DSL -, bytestring -, candid -, cborg -, cereal -, containers -, crc -, cryptonite -, data-default-class -, directory -, ed25519 -, filepath -, hashable -, hex-text -, http-client -, http-client-tls -, http-types -, leb128-cereal -, memory -, MonadRandom -, mtl -, optparse-applicative -, parallel -, prettyprinter -, primitive -, process -, QuickCheck -, quickcheck-io -, random -, row-types -, serialise -, split -, splitmix -, stdenv -, tasty -, tasty-ant-xml -, tasty-html -, tasty-hunit -, tasty-quickcheck -, tasty-rerun -, template-haskell -, temporary -, text -, time -, transformers -, unordered-containers -, utf8-string -, vector -, wai -, wai-extra -, warp -, winter -, zlib -}: -mkDerivation { - pname = "ic-ref"; - version = "0.0.1"; - src = import ../gitSource.nix "impl"; - configureFlags = [ "-frelease" ]; - isLibrary = false; - isExecutable = true; - executableHaskellDepends = [ - aeson - asn1-encoding - asn1-types - atomic-write - base - base32 - base64-bytestring - binary - bindings-DSL - bytestring - candid - cborg - cereal - containers - crc - cryptonite - data-default-class - directory - ed25519 - filepath - hashable - hex-text - http-client - http-client-tls - http-types - leb128-cereal - memory - MonadRandom - mtl - optparse-applicative - parallel - prettyprinter - primitive - process - random - row-types - serialise - split - splitmix - tasty - tasty-ant-xml - tasty-html - tasty-hunit - tasty-rerun - template-haskell - text - time - transformers - unordered-containers - utf8-string - vector - wai - wai-extra - warp - winter - zlib - ]; - testHaskellDepends = [ - aeson - asn1-encoding - asn1-types - atomic-write - base - base32 - base64-bytestring - binary - bindings-DSL - bytestring - candid - cborg - cereal - containers - crc - cryptonite - data-default-class - directory - ed25519 - filepath - hashable - hex-text - leb128-cereal - memory - MonadRandom - mtl - parallel - primitive - QuickCheck - quickcheck-io - random - row-types - serialise - split - splitmix - tasty - tasty-hunit - tasty-quickcheck - temporary - text - time - transformers - unordered-containers - utf8-string - vector - winter - zlib - ]; - doCheck = false; - license = "unknown"; - hydraPlatforms = stdenv.lib.platforms.none; -} diff --git a/nix/generated/leb128-cereal.nix b/nix/generated/leb128-cereal.nix deleted file mode 100644 index 312740cb5..000000000 --- a/nix/generated/leb128-cereal.nix +++ /dev/null @@ -1,28 +0,0 @@ -# THIS IS AN AUTOMATICALLY GENERATED FILE. DO NOT EDIT MANUALLY! -# See ./nix/generate.nix for instructions. - -{ mkDerivation -, pkgs -, base -, bytestring -, cereal -, stdenv -, tasty -, tasty-hunit -, tasty-quickcheck -}: -mkDerivation { - pname = "leb128-cereal"; - version = "1.2"; - src = pkgs.sources.leb128-cereal; - libraryHaskellDepends = [ base bytestring cereal ]; - testHaskellDepends = [ - base - bytestring - tasty - tasty-hunit - tasty-quickcheck - ]; - description = "LEB128 and SLEB128 encoding"; - license = stdenv.lib.licenses.mit; -} diff --git a/nix/generated/megaparsec.nix b/nix/generated/megaparsec.nix deleted file mode 100644 index 458f5e9d9..000000000 --- a/nix/generated/megaparsec.nix +++ /dev/null @@ -1,43 +0,0 @@ -{ mkDerivation -, base -, bytestring -, case-insensitive -, containers -, criterion -, deepseq -, mtl -, parser-combinators -, scientific -, stdenv -, text -, transformers -, weigh -}: -mkDerivation { - pname = "megaparsec"; - version = "8.0.0"; - sha256 = "b5d7c64646016d12f540a6948396a86e0cd39865569d68fe2018fe9e3fce6318"; - libraryHaskellDepends = [ - base - bytestring - case-insensitive - containers - deepseq - mtl - parser-combinators - scientific - text - transformers - ]; - benchmarkHaskellDepends = [ - base - containers - criterion - deepseq - text - weigh - ]; - homepage = "https://github.com/mrkkrp/megaparsec"; - description = "Monadic parser combinators"; - license = stdenv.lib.licenses.bsd2; -} diff --git a/nix/generated/random.nix b/nix/generated/random.nix deleted file mode 100644 index bb2e42f7e..000000000 --- a/nix/generated/random.nix +++ /dev/null @@ -1,57 +0,0 @@ -{ mkDerivation -, base -, bytestring -, containers -, deepseq -, doctest -, gauge -, mtl -, mwc-random -, primitive -, rdtsc -, smallcheck -, split -, splitmix -, stdenv -, tasty -, tasty-expected-failure -, tasty-hunit -, tasty-smallcheck -, time -, unliftio -, vector -}: -mkDerivation { - pname = "random"; - version = "1.2.0"; - sha256 = "e4519cf7c058bfd5bdbe4acc782284acc9e25e74487208619ca83cbcd63fb9de"; - revision = "4"; - editedCabalFile = "08mq836ganl3sq6mfn3hrj6xm0h30klp21y7gbd9md2882agndrk"; - libraryHaskellDepends = [ base bytestring deepseq mtl splitmix ]; - testHaskellDepends = [ - base - bytestring - containers - doctest - mwc-random - primitive - smallcheck - tasty - tasty-expected-failure - tasty-hunit - tasty-smallcheck - unliftio - vector - ]; - benchmarkHaskellDepends = [ - base - gauge - mtl - rdtsc - split - splitmix - time - ]; - description = "Pseudo-random number generation"; - license = stdenv.lib.licenses.bsd3; -} diff --git a/nix/generated/splitmix.nix b/nix/generated/splitmix.nix deleted file mode 100644 index d6bdd7761..000000000 --- a/nix/generated/splitmix.nix +++ /dev/null @@ -1,53 +0,0 @@ -{ mkDerivation -, async -, base -, base-compat -, base-compat-batteries -, bytestring -, clock -, containers -, criterion -, deepseq -, HUnit -, math-functions -, process -, random -, stdenv -, test-framework -, test-framework-hunit -, tf-random -, vector -}: -mkDerivation { - pname = "splitmix"; - version = "0.1.0.3"; - sha256 = "46009f4b000c9e6613377767b8718bf38476469f2a8e2162d98cc246882d5a35"; - libraryHaskellDepends = [ base deepseq ]; - testHaskellDepends = [ - async - base - base-compat - base-compat-batteries - bytestring - containers - deepseq - HUnit - math-functions - process - random - test-framework - test-framework-hunit - tf-random - vector - ]; - benchmarkHaskellDepends = [ - base - clock - containers - criterion - random - tf-random - ]; - description = "Fast Splittable PRNG"; - license = stdenv.lib.licenses.bsd3; -} diff --git a/nix/generated/winter.nix b/nix/generated/winter.nix deleted file mode 100644 index c4dcaa65d..000000000 --- a/nix/generated/winter.nix +++ /dev/null @@ -1,109 +0,0 @@ -# THIS IS AN AUTOMATICALLY GENERATED FILE. DO NOT EDIT MANUALLY! -# See ./nix/generate.nix for instructions. - -{ mkDerivation -, pkgs -, array -, base -, binary -, byte-order -, bytestring -, cmdargs -, containers -, data-default-class -, data-fix -, deepseq -, directory -, filepath -, FloatingHex -, lifted-base -, microlens-platform -, monad-control -, mtl -, nats -, parsec -, primitive -, primitive-unaligned -, process -, stdenv -, tasty -, tasty-hunit -, tasty-quickcheck -, temporary -, text -, transformers -, vector -}: -mkDerivation { - pname = "winter"; - version = "1.0.0"; - src = pkgs.sources.winter; - isLibrary = true; - isExecutable = true; - libraryHaskellDepends = [ - array - base - binary - byte-order - bytestring - containers - data-default-class - data-fix - deepseq - FloatingHex - lifted-base - microlens-platform - monad-control - mtl - nats - parsec - primitive - primitive-unaligned - text - transformers - vector - ]; - executableHaskellDepends = [ - base - binary - bytestring - cmdargs - containers - data-default-class - mtl - parsec - text - vector - ]; - testHaskellDepends = [ - array - base - binary - bytestring - containers - data-default-class - data-fix - deepseq - directory - filepath - FloatingHex - lifted-base - microlens-platform - monad-control - mtl - parsec - primitive - process - tasty - tasty-hunit - tasty-quickcheck - temporary - text - transformers - vector - ]; - doCheck = false; - homepage = "https://github.com/dfinity/winter"; - description = "Haskell port of the WebAssembly OCaml reference interpreter"; - license = stdenv.lib.licenses.mit; -} diff --git a/nix/haskell-packages.nix b/nix/haskell-packages.nix deleted file mode 100644 index 52f00b620..000000000 --- a/nix/haskell-packages.nix +++ /dev/null @@ -1,60 +0,0 @@ -nix: subpath: - self: super: { - winter = super.callPackage generated/winter.nix {}; - ic-ref = super.callPackage generated/ic-ref.nix {}; - leb128-cereal = super.callPackage generated/leb128-cereal.nix {}; - candid = super.callPackage generated/candid.nix {}; - - # no base32 in nixos-20.03 - base32 = super.callPackage generated/base32.nix {}; - - # need newer version - base64-bytestring = nix.haskell.lib.dontCheck (super.callPackage generated/base64-bytestring.nix {}); - random = nix.haskell.lib.dontCheck (super.callPackage generated/random.nix {}); - splitmix = nix.haskell.lib.dontCheck (super.callPackage generated/splitmix.nix {}); - QuickCheck = super.callPackage generated/QuickCheck.nix {}; - megaparsec = super.callPackage generated/megaparsec.nix {}; - - # Only the test suite of crc is broken - # https://github.com/MichaelXavier/crc/issues/2 - crc = nix.haskell.lib.markUnbroken (nix.haskell.lib.dontCheck super.crc); - - # We want random-1.2, but a lot of packages do not like that yet, - # luckily mostly in the test suite. Hence this set of fix-ups: - # (Remove when going to a nipxkgs that has random-1.2) - - # Wants testing-framework, not compatible with random-1.2 - test-framework-quickcheck2 = nix.haskell.lib.markBroken super.test-framework-quickcheck2; - Glob = nix.haskell.lib.dontCheck super.Glob; - SHA = nix.haskell.lib.dontCheck super.SHA; - blaze-builder = nix.haskell.lib.dontCheck super.blaze-builder; - blaze-html = nix.haskell.lib.dontCheck super.blaze-html; - cereal = nix.haskell.lib.dontCheck super.cereal; - either = nix.haskell.lib.dontCheck super.either; - exceptions = nix.haskell.lib.dontCheck super.exceptions; - hashable = nix.haskell.lib.dontCheck super.hashable; - unordered-containers = nix.haskell.lib.dontCheck super.unordered-containers; - monad-par = nix.haskell.lib.dontCheck super.monad-par; - cassava = nix.haskell.lib.dontCheck super.cassava; - network-uri = nix.haskell.lib.dontCheck super.network-uri; - pem = nix.haskell.lib.dontCheck super.pem; - pureMD5 = nix.haskell.lib.dontCheck super.pureMD5; - - # hedgehog wants older random - hedgehog = nix.haskell.lib.markBroken super.hedgehog; - bsb-http-chunked = nix.haskell.lib.dontCheck super.bsb-http-chunked; - retry = nix.haskell.lib.dontCheck super.retry; - - # wants older quickcheck - quickcheck-instances = nix.haskell.lib.markBroken super.quickcheck-instances; - aeson = nix.haskell.lib.dontCheck super.aeson; - prettyprinter = nix.haskell.lib.dontCheck super.prettyprinter; - http-types = nix.haskell.lib.dontCheck super.http-types; - vector-builder = nix.haskell.lib.dontCheck super.vector-builder; - serialise = nix.haskell.lib.dontCheck super.serialise; - - # not compatible with latest quickcheck - psqueues = nix.haskell.lib.dontCheck super.psqueues; - vector = nix.haskell.lib.dontCheck super.vector; - attoparsec = nix.haskell.lib.dontCheck super.attoparsec; -} diff --git a/nix/patches/0001-ghc865-binary-Use-binary-distribution-which-links-ag.patch b/nix/patches/0001-ghc865-binary-Use-binary-distribution-which-links-ag.patch deleted file mode 100644 index 4e8d88dec..000000000 --- a/nix/patches/0001-ghc865-binary-Use-binary-distribution-which-links-ag.patch +++ /dev/null @@ -1,80 +0,0 @@ -From 35192bf8c4151c57aa6d37064ffeeee5510ce397 Mon Sep 17 00:00:00 2001 -From: Joe Hermaszewski -Date: Sun, 8 Nov 2020 23:51:44 +0800 -Subject: [PATCH 1/2] ghc865-binary: Use binary distribution which links - against ncurses6 for x86_64-linux - -Ben Gamari's patch from #85924. - -Fixes #85924, allowing one to bootstrap GHC in `pkgsMusl` - -`nix-build -A pkgsMusl.haskellPackages.hello` succeeds with this patch. - -(cherry picked from commit 15b3bc33064eeb0cba743cad585c829b6694669c) ---- - .../compilers/ghc/8.6.5-binary.nix | 23 ++++++++++++++----- - 1 file changed, 17 insertions(+), 6 deletions(-) - -diff --git a/pkgs/development/compilers/ghc/8.6.5-binary.nix b/pkgs/development/compilers/ghc/8.6.5-binary.nix -index 41af279e83f..9234e3b1457 100644 ---- a/pkgs/development/compilers/ghc/8.6.5-binary.nix -+++ b/pkgs/development/compilers/ghc/8.6.5-binary.nix -@@ -1,6 +1,6 @@ - { stdenv - , fetchurl, perl, gcc --, ncurses5, gmp, glibc, libiconv -+, ncurses5, ncurses6, gmp, glibc, libiconv - , llvmPackages - }: - -@@ -10,8 +10,12 @@ assert stdenv.targetPlatform == stdenv.hostPlatform; - let - useLLVM = !stdenv.targetPlatform.isx86; - -+ useNcurses6 = stdenv.hostPlatform.system == "x86_64-linux"; -+ -+ ourNcurses = if useNcurses6 then ncurses6 else ncurses5; -+ - libPath = stdenv.lib.makeLibraryPath ([ -- ncurses5 gmp -+ ourNcurses gmp - ] ++ stdenv.lib.optional (stdenv.hostPlatform.isDarwin) libiconv); - - libEnvVar = stdenv.lib.optionalString stdenv.hostPlatform.isDarwin "DY" -@@ -34,12 +38,16 @@ stdenv.mkDerivation rec { - # https://downloads.haskell.org/~ghc/8.6.5/ - src = fetchurl ({ - i686-linux = { -+ # Don't use the Fedora27 build (as below) because there isn't one! - url = "http://haskell.org/ghc/dist/${version}/ghc-${version}-i386-deb9-linux.tar.xz"; - sha256 = "1p2h29qghql19ajk755xa0yxkn85slbds8m9n5196ris743vkp8w"; - }; - x86_64-linux = { -- url = "http://haskell.org/ghc/dist/${version}/ghc-${version}-x86_64-deb9-linux.tar.xz"; -- sha256 = "1pqlx6rdjs2110g0y1i9f8x18lmdizibjqd15f5xahcz39hgaxdw"; -+ # This is the Fedora build because it links against ncurses6 where the -+ # deb9 one links against ncurses5, see here -+ # https://github.com/NixOS/nixpkgs/issues/85924 for a discussion -+ url = "http://haskell.org/ghc/dist/${version}/ghc-${version}-x86_64-fedora27-linux.tar.xz"; -+ sha256 = "18dlqm5d028fqh6ghzn7pgjspr5smw030jjzl3kq6q1kmwzbay6g"; - }; - aarch64-linux = { - url = "http://haskell.org/ghc/dist/${version}/ghc-${version}-aarch64-ubuntu18.04-linux.tar.xz"; -@@ -88,9 +96,12 @@ stdenv.mkDerivation rec { - '' + - # Rename needed libraries and binaries, fix interpreter - stdenv.lib.optionalString stdenv.isLinux '' -- find . -type f -perm -0100 -exec patchelf \ -+ find . -type f -perm -0100 \ -+ -exec patchelf \ - --replace-needed libncurses${stdenv.lib.optionalString stdenv.is64bit "w"}.so.5 libncurses.so \ -- --replace-needed libtinfo.so libtinfo.so.5 \ -+ ${ # This isn't required for x86_64-linux where we use ncurses6 -+ stdenv.lib.optionalString (!useNcurses6) "--replace-needed libtinfo.so libtinfo.so.5" -+ } \ - --interpreter ${glibcDynLinker} {} \; - - sed -i "s|/usr/bin/perl|perl\x00 |" ghc-${version}/ghc/stage2/build/tmp/ghc-stage2 --- -2.29.2 - diff --git a/nix/patches/0002-openblas-0.3.10-0.3.13.patch b/nix/patches/0002-openblas-0.3.10-0.3.13.patch deleted file mode 100644 index 8eb4a448b..000000000 --- a/nix/patches/0002-openblas-0.3.10-0.3.13.patch +++ /dev/null @@ -1,74 +0,0 @@ -From f7fc01b0d74ea06fff934bbb9397615978903726 Mon Sep 17 00:00:00 2001 -From: Joachim Breitner -Date: Thu, 7 Jan 2021 18:32:53 +0000 -Subject: [PATCH 2/2] openblas: 0.3.10 -> 0.3.13 - -fixing https://github.com/NixOS/nixpkgs/issues/92458 -fetching the file from nixpkgs master at cc8db6e19b876e0ee484d8e186fc689ce1e18f6b ---- - .../science/math/openblas/default.nix | 24 +++++++++++-------- - 1 file changed, 14 insertions(+), 10 deletions(-) - -diff --git a/pkgs/development/libraries/science/math/openblas/default.nix b/pkgs/development/libraries/science/math/openblas/default.nix -index 89d88bdf564..8df04e80ceb 100644 ---- a/pkgs/development/libraries/science/math/openblas/default.nix -+++ b/pkgs/development/libraries/science/math/openblas/default.nix -@@ -15,8 +15,8 @@ - # Select a specific optimization target (other than the default) - # See https://github.com/xianyi/OpenBLAS/blob/develop/TargetList.txt - , target ? null --, enableStatic ? false --, enableShared ? true -+, enableStatic ? stdenv.hostPlatform.isStatic -+, enableShared ? !stdenv.hostPlatform.isStatic - }: - - with stdenv.lib; -@@ -71,6 +71,13 @@ let - NO_AVX512 = true; - USE_OPENMP = !stdenv.hostPlatform.isMusl; - }; -+ -+ powerpc64le-linux = { -+ BINARY = 64; -+ TARGET = setTarget "POWER5"; -+ DYNAMIC_ARCH = true; -+ USE_OPENMP = !stdenv.hostPlatform.isMusl; -+ }; - }; - in - -@@ -99,12 +106,15 @@ let - in - stdenv.mkDerivation rec { - pname = "openblas"; -- version = "0.3.10"; -+ version = "0.3.12"; -+ -+ outputs = [ "out" "dev" ]; -+ - src = fetchFromGitHub { - owner = "xianyi"; - repo = "OpenBLAS"; - rev = "v${version}"; -- sha256 = "174id98ga82bhz2v7sy9yj6pqy0h0088p3mkdikip69p9rh3d17b"; -+ sha256 = "0mk1kjkr96bvvcq2zigzjrs0cnhwsf6gfi0855mp9yifn8lvp20y"; - }; - - inherit blas64; -@@ -134,12 +144,6 @@ stdenv.mkDerivation rec { - buildPackages.stdenv.cc - ]; - -- # Disable an optimisation which seems to cause issues, pending an -- # upstream fix: https://github.com/xianyi/OpenBLAS/issues/2496 -- patches = stdenv.lib.optionals stdenv.hostPlatform.isAarch64 [ -- ./0001-Disable-optimised-aarch64-dgemm_beta-pending-fix.patch -- ]; -- - makeFlags = mkMakeFlagsFromConfig (config // { - FC = "${stdenv.cc.targetPrefix}gfortran"; - CC = "${stdenv.cc.targetPrefix}${if stdenv.cc.isClang then "clang" else "cc"}"; --- -2.29.2 - diff --git a/nix/patches/fb063991b26b2b93dece6d09f37041451a5ef4cb.patch b/nix/patches/fb063991b26b2b93dece6d09f37041451a5ef4cb.patch deleted file mode 100644 index 4dd0d4130..000000000 --- a/nix/patches/fb063991b26b2b93dece6d09f37041451a5ef4cb.patch +++ /dev/null @@ -1,51 +0,0 @@ -From fb063991b26b2b93dece6d09f37041451a5ef4cb Mon Sep 17 00:00:00 2001 -From: Joachim Breitner -Date: Tue, 1 Dec 2020 19:11:45 +0100 -Subject: [PATCH] macdylibbundler: Should propagate dependency on otool - (#103163) - -Co-authored-by: Sandro ---- - pkgs/development/tools/misc/macdylibbundler/default.nix | 9 ++++++++- - pkgs/top-level/all-packages.nix | 2 +- - 2 files changed, 9 insertions(+), 2 deletions(-) - -diff --git a/pkgs/development/tools/misc/macdylibbundler/default.nix b/pkgs/development/tools/misc/macdylibbundler/default.nix -index 103c534dfa3d7..5008175363cbf 100644 ---- a/pkgs/development/tools/misc/macdylibbundler/default.nix -+++ b/pkgs/development/tools/misc/macdylibbundler/default.nix -@@ -1,4 +1,4 @@ --{ stdenv, fetchFromGitHub }: -+{ stdenv, makeWrapper, fetchFromGitHub, cctools }: - - stdenv.mkDerivation { - pname = "macdylibbundler"; -@@ -11,8 +11,15 @@ stdenv.mkDerivation { - sha256 = "149p3dcnap4hs3nhq5rfvr3m70rrb5hbr5xkj1h0gsfp0d7gvxnj"; - }; - -+ buildInputs = [ makeWrapper ]; -+ - makeFlags = [ "PREFIX=$(out)" ]; - -+ postInstall = '' -+ wrapProgram $out/bin/dylibbundler \ -+ --prefix PATH ":" "${cctools}/bin" -+ ''; -+ - meta = with stdenv.lib; { - description = "Utility to ease bundling libraries into executables for OSX"; - longDescription = '' -diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix -index 0cfcb9b4abfb2..c0c827bc13b63 100644 ---- a/pkgs/top-level/all-packages.nix -+++ b/pkgs/top-level/all-packages.nix -@@ -22390,7 +22390,7 @@ in - - mac = callPackage ../development/libraries/mac { }; - -- macdylibbundler = callPackage ../development/tools/misc/macdylibbundler { }; -+ macdylibbundler = callPackage ../development/tools/misc/macdylibbundler { inherit (darwin) cctools; }; - - magic-wormhole = with python3Packages; toPythonApplication magic-wormhole; - diff --git a/nix/python-cbor2.nix b/nix/python-cbor2.nix deleted file mode 100644 index e679153e9..000000000 --- a/nix/python-cbor2.nix +++ /dev/null @@ -1,23 +0,0 @@ -{ lib, buildPythonPackage, fetchPypi, pytest, pytestcov, setuptools_scm }: - -buildPythonPackage rec { - pname = "cbor2"; - version = "5.2.0"; - - src = fetchPypi { - inherit pname version; - sha256 = "1gwlgjl70vlv35cgkcw3cg7b5qsmws36hs4mmh0l9msgagjs4fm3"; - }; - - nativeBuildInputs = [ setuptools_scm ]; - checkInputs = [ pytest pytestcov ]; - - checkPhase = "pytest"; - - meta = with lib; { - description = "Pure Python CBOR (de)serializer with extensive tag support"; - homepage = "https://github.com/agronholm/cbor2"; - license = licenses.mit; - maintainers = with maintainers; [ taneb ]; - }; -} diff --git a/shell.nix b/shell.nix deleted file mode 100644 index d6688671e..000000000 --- a/shell.nix +++ /dev/null @@ -1 +0,0 @@ -builtins.throw "Please invoke nix-shell in the appropriate subdirectory (impl/, universal-canister/…)" diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 3e7417f9b..013012b81 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,19 @@ [#changelog] == Changelog +[#0_18_2] +=== 0.18.2 (2021-09-29) + +* Spec: Canister heartbeat +* Spec: Terminology changes +* Spec: Support for 64-bit stable memory + +[#0_18_1] +=== 0.18.1 (2021-08-04) + +* Spec: Support RSA PKCS#1 v1.5 signatures in web authentication +* Spec clarification: Fix various typos and improve textual clarity + [#0_18_0] === 0.18.0 (2021-05-18) diff --git a/spec/index.adoc b/spec/index.adoc index 4fcb061ef..427f1bc88 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,41 +1,41 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.0 +0.18.2 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ == Introduction -Welcome to the Internet Computer! We speak of “the” Internet Computer, because although under the hood a large number of physical computers are working together in non-trivial ways, in the end we have the appearance of a single, shared, secure and world-wide accessible computer. Much, if not all, of the advanced and complex machinery is hidden from those that use the Internet Computer to run their applications and those who use these applications. +Welcome to _the Internet Computer_! We speak of “the” Internet Computer, because although under the hood a large number of physical computers are working together in a blockchain protocol, in the end we have the appearance of a single, shared, secure and world-wide accessible computer. Developers who want to build decentralized applications (or _dapps_ for short) that run on the Internet Computer blockchain and end-users who want to use those dapps need to know very little, if anything, about the underlying protocol. However, knowing some details about the interfaces that the Internet Computer exposes can allow interested developers and architects to take fuller advantages of the unique features that the Internet Computer provides. === Target audience -This documents describes this _external_ view of the Internet Computer, i.e. the low-level interfaces it provides to application developers and users, and what will happen when they use these interfaces. +This document describes this _external_ view of the Internet Computer, i.e. the low-level interfaces it provides to dapp developers and users, and what will happen when they use these interfaces. -NOTE: While this document describes the external interface and behavior of the Internet Computer, it is not end-user or end-developer documentation. Most developers will interact with the Internet Computer through additional tooling like the SDK, Canister Development Kits and Motoko. Please see https://sdk.dfinity.org/ for suitable documentation. +NOTE: While this document describes the external interface and behavior of the Internet Computer, it is not intended as end-user or end-developer documentation. Most developers will interact with the Internet Computer through additional tooling like the SDK, Canister Development Kits and Motoko. Please see https://sdk.dfinity.org/ for suitable documentation. The target audience of this document are -* those who use the these low-level interfaces (e.g. implement agents, canister developments kits, emulators, other tooling). +* those who use these low-level interfaces (e.g. implement agents, canister developments kits, emulators, other tooling). * those who implement these low-level interfaces (e.g. developers of the Internet Computer implementation) -* those who want to understand the intricacies of the Internet Computer’s behaviour in great detail (e.g. to do a security analysis) +* those who want to understand the intricacies of the Internet Computer’s behavior in great detail (e.g. to do a security analysis) WARNING: This document is a rigorous, technically dense reference. It is not an introduction to the Internet Computer, and as such most useful to those who understand the high-level concepts. Please see more high-level documentation first. === Scope of this document -If you think of the Internet Computer as a distributed execution engine that _provides_ a WebAssembly-based service hosting service, then this document describes exclusively the service hosting aspect of it. To the extent possible, this document will _not_ talk about blockchains, consensus protocols, nodes, subnets, orthogonal persistence or governance. +If you think of the Internet Computer as a distributed engine that executes WebAssembly-based dapps, then this document describes exclusively the aspect of executing those dapps. To the extent possible, this document will _not_ talk about consensus protocols, nodes, subnets, orthogonal persistence or governance. This document tries to be implementation agnostic: It would apply just as well to a (hypothetical) compatible reimplementation of the Internet Computer. This implies that this document does not cover interfaces towards those running the Internet Computer (e.g. data center operators, protocol developers, governance users), as topics like node update, monitoring, logging are inherently tied to the actual _implementation_ and its architecture. === Overview of the Internet Computer -If you want to use the Internet Computer as an application developer, you first create a _canister module_ that contains the WebAssembly code and configuration for your application, and deploy it using the <>. You can create canisters using the Motoko language and the SDK, which is more convenient. If you want to use your own tooling, however, then this document describes <> and how the <>. +Dapps on the Internet Computer, or _IC_ for short, are implemented as _canister smart contracts_, or _canisters_ for short. If you want to build on the Internet Computer as a dapp developer, you first create a _canister module_ that contains the WebAssembly code and configuration for your dapp, and deploy it using the <>. You can create canister modules using the Motoko language and the SDK, which is more convenient. If you want to use your own tooling, however, then this document describes <> and how the <>. -Once your application is running on the Internet Computer, it is a _canister_, and users can interact with it. They can use the <> to interact with the canister according to the <>. +Once your dapp is running on the Internet Computer, it is a canister smart contract, and users can interact with it. They can use the <> to interact with the canister according to the <>. -The user can also use the HTTP interface to issue read-only queries, which are faster, but cannot change the state of a canister. +The user can also use the HTTPS interface to issue read-only queries, which are faster, but cannot change the state of a canister. .A typical use of the Internet Computer. (This is a simplified view; some of the arrows represent multiple interaction steps or polling.) [plantuml] @@ -63,29 +63,30 @@ Sections “<>” and “<>” describe these interf To get some consistency in this document, we try to use the following terms with precision: -We avoid the term “client”, as it could be the client of the Internet Computer or the client inside the distributed network that makes up the Internet Computer. Instead, we use the term _user_ to denote the external entity interacting with the internet computer, even if in most cases it will be some code (sometimes called “agent”) acting on behalf of a (human) user. +We avoid the term “client”, as it could be the client of the Internet Computer or the client inside the distributed network that makes up the Internet Computer. Instead, we use the term _user_ to denote the external entity interacting with the Internet Computer, even if in most cases it will be some code (sometimes called “agent”) acting on behalf of a (human) user. The public entry points of canisters are called _methods_. Methods can be declared to be either _update methods_ (state mutation is preserved) or _query methods_ (state mutation is discarded, no further calls can be made). Methods can be _called_, from _caller_ to _callee_, and will eventually incur a _response_ which is either a _reply_ or a _reject_. A method may have _parameters_, which are provided with concrete _arguments_ in a method call. -Inter-canister calls do not distinguish between update and query methods; inter-canister calls can preserve state mutation and are therefore akin to update method calls. Note that calls from a canister to itself also count as "inter-canister". External calls can be update calls, which can call both kinds of methods, and query calls, which can _only_ call query methods. +External calls can be update calls, which can call both kinds of methods, and query calls, which can _only_ call query methods. +Inter-canister calls can also call both kinds of methods. Note that calls from a canister to itself also count as "inter-canister". Internally, a call or a response is transmitted as a _message_ from a _sender_ to a _receiver_. Messages do not have a response. [[define-wasm-fn]]WebAssembly _functions_ are exported by the WebAssembly module or provided by the System API. These are _invoked_ and can either _trap_ or _return_, possibly with a return value. Functions, too, have parameters and take arguments. -External _users_ interact with the system by issuing _requests_ on the HTTPS interface. Requests have responses which can either be replies or rejects. Some requests cause internal messages to be created. +External _users_ interact with the Internet Computer by issuing _requests_ on the HTTPS interface. Requests have responses which can either be replies or rejects. Some requests cause internal messages to be created. Canisters and users are identified by a _principal_, sometimes also called an _id_. == Pervasive concepts -Before going into the details of the four public interfaces describes in this document (namely the agent-facing <>, the canister-facing <>, the <> and the <>), this section introduces some concepts that transcend multiple interfaces. +Before going into the details of the four public interfaces described in this document (namely the agent-facing <>, the canister-facing <>, the <> and the <>), this section introduces some concepts that transcend multiple interfaces. === Unspecified constants and limits -This specification may refer to certain constants and limits without specifying their concrete value (yet), i.e. they are implementation defined. Many are resource limits which are relevant only to specify the error-handling behaviour of the system (which, as mentioned above, is also not yet precisely described in this document). This list is not complete. +This specification may refer to certain constants and limits without specifying their concrete value (yet), i.e. they are implementation defined. Many are resource limits which are relevant only to specify the error-handling behavior of the IC (which, as mentioned above, is also not yet precisely described in this document). This list is not complete. * `MAX_CYCLES_PER_MESSAGE` + @@ -93,16 +94,16 @@ Amount of cycles that a canister has to have before a message is attempted to be * `MAX_CYCLES_PER_RESPONSE` + -Amount of cycles that the system sets aside when a canister performs a call. This is used to pay for processing the response message, and unused cycles after the execution of the response are refunded. See <>. +Amount of cycles that the IC sets aside when a canister performs a call. This is used to pay for processing the response message, and unused cycles after the execution of the response are refunded. See <>. * `MAX_CANISTER_BALANCE` + -Maximum canister cycle balance. Any excess is discarded. Less than 2^64^. +Maximum canister cycle balance. Any excess is discarded. Less than 2^128^. [#principal] === Principals -Principal are generic identifiers for canisters, users and possibly other concepts in the future. As far as most uses of the system are concerned they are _opaque_ binary blobs with a length between 0 and 29 bytes, and there is intentionally no mechanism to tell canister id and users ids apart. +Principals are generic identifiers for canisters, users and possibly other concepts in the future. As far as most uses of the IC are concerned they are _opaque_ binary blobs with a length between 0 and 29 bytes, and there is intentionally no mechanism to tell canister ids and user ids apart. There is, however, some structure to them to encode specific authentication and authorization behavior. @@ -115,7 +116,7 @@ There are several classes of ids: 1. _Opaque ids_. + -These are always generated by the system and have no structure of interest outside the system. +These are always generated by the IC and have no structure of interest outside of it. + NOTE: Typically, these end with the byte `0x01`, but users of the IC should not need to care about that. @@ -138,7 +139,7 @@ NOTE: Derived IDs are currently not explicitly used in this document, but they m + This has the form `0x04`, and is used for the anonymous caller. It can be used in call and query requests without a signature. -When the system creates a _fresh_ id, it never creates a self-authenticating id, an anonymous id or an id derived from what could be a canister or user. +When the IC creates a _fresh_ id, it never creates a self-authenticating id, an anonymous id or an id derived from what could be a canister or user. [#textual-ids] @@ -182,7 +183,7 @@ function textual_decode() { [#canister-lifecycle] === Canister lifecycle -Services on the Internet Computer are called _canisters_. Conceptually, they consist of the following pieces of state: +Dapps on the Internet Computer are called _canisters_. Conceptually, they consist of the following pieces of state: * A canister id (a <>) * Their _controllers_ (a possibly empty list of <>) @@ -194,7 +195,7 @@ A canister can be _empty_ (e.g. directly after creation) or _non-empty_. A non-e * code, in the form of a canister module * state (memories, globals etc.) - * possibly further system-internal data (e.g. queues) + * possibly further data that is specific to the implementation of the IC (e.g. queues) Canisters are empty after creation and uninstallation, and become non-empty through <>. @@ -220,8 +221,8 @@ NOTE: Once the IC frees the resources of a canister, its id, _cycles_ balance, a The canister status can be used to control whether the canister is processing calls: * In status `running`, calls to the canister are processed as normal. -* In status `stopping`, calls to the canister are rejected by the system, but responses to the canister are processed as normal. -* In status `stopped`, calls to the canister are rejected by the system, and there are no outstanding responses. +* In status `stopping`, calls to the canister are rejected by the IC, but responses to the canister are processed as normal. +* In status `stopped`, calls to the canister are rejected by the IC, and there are no outstanding responses. In all cases, calls to the <> are processed, regardless of the state of the managed canister. @@ -255,16 +256,17 @@ Plain signatures are supported for the schemes - See https://tools.ietf.org/html/rfc8410[RFC 8410] for DER encoding of Ed25519 public keys. - See https://tools.ietf.org/rfc/rfc5480[RFC 5480] for DER encoding of ECDSA public keys; the DER encoding must not specify a hash function. For curve `secp256k1`, the OID 1.3.132.0.10 is used. - The points must be specified in uncompressed form (i.e. `0x04` followed by the big-endian 32byte encodings of `x` and `y`). + The points must be specified in uncompressed form (i.e. `0x04` followed by the big-endian 32-byte encodings of `x` and `y`). -* The signatures are encoded as the concatenation of the 32 byte big endian encodings of the two values _r_ and _s_. +* The signatures are encoded as the concatenation of the 32-byte big endian encodings of the two values _r_ and _s_. [#webauthn] ==== Web Authentication -The only allowed signature scheme for web authentication is +The allowed signature schemes for web authentication are * https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf[*ECDSA*] on curve P-256 (also known as `secp256r1`), using SHA-256 as hash function. +* https://datatracker.ietf.org/doc/html/rfc8017#section-8.2[*RSA PKCS#1v1.5 (RSASSA-PKCS1-v1_5)*], using SHA-256 as hash function. The signature is calculated by using the payload as the challenge in the web authentication assertion. @@ -307,7 +309,7 @@ The IC also supports a scheme where a canister can sign a payload by declaring a This section makes forward references to other concepts in this document, in particular the section <>. -* The public key is a DER-wrapped structure that indicates the _signing canister_, and includes a freely choosable seed. Each choice of seed yields a distinct public key for the canister, and the canister can chose to encode information, such as a user id, in the seed. +* The public key is a DER-wrapped structure that indicates the _signing canister_, and includes a freely choosable seed. Each choice of seed yields a distinct public key for the canister, and the canister can choose to encode information, such as a user id, in the seed. + More concretely, it uses the `SubjectPublicKeyInfo` type used for other types of public keys (see, e.g., https://tools.ietf.org/html/rfc8410#section-4[RFC 8410, Section 4]), with OID 1.3.6.1.4.1.56387.1.2 (iso.org.dod.internet.private.enterprise.dfinity.mechanisms.canister-signature). + @@ -340,11 +342,11 @@ where `s` is the SHA-256 hash of the `seed` used in the public key and `m` is th [#state-tree] == The system state tree -Parts of the system state are publicly exposed (e.g. via <> or <>) in a verified way (see <> for the machinery for certifying). This section describes the content of the system state abstractly. +Parts of the IC state are publicly exposed (e.g. via <> or <>) in a verified way (see <> for the machinery for certifying). This section describes the content of this system state abstractly. Conceptually, the system state is a tree with labeled children, and values in the leaves. Equivalently, the system state is a mapping from paths (sequences of labels) to values, where the domain is prefix-free. -Labels are always blobs (but often with an human readable representation). In this document, paths are written suggestively with slashes as separators; the actual encoding is not actually using slashes as delimitors, and labels may contain the 0x2F byte (ASCII `/`) just fine. Values are either natural numbers, text values or blob values. +Labels are always blobs (but often with a human readable representation). In this document, paths are written suggestively with slashes as separators; the actual encoding is not actually using slashes as delimiters, and labels may contain the 0x2F byte (ASCII `/`) just fine. Values are either natural numbers, text values or blob values. This section specifies the publicly relevant paths in the tree. @@ -367,7 +369,7 @@ The public key of the subnet (a DER-encoded BLS key, see <>) [#state-tree-request-status] === Request status -For each asynchronous request known to the system, its status is in a subtree at `/request_status/`. Please see <> for more details on how asynchronous requests work. +For each asynchronous request known to the Internet Computer, its status is in a subtree at `/request_status/`. Please see <> for more details on how asynchronous requests work. * `/request_status//status` (text) + @@ -385,9 +387,9 @@ If the status is `rejected`, then this path contains the reject code (see </controllers` (blob): + -The current controllers of the canister. The value consists of a CBOR data item with major type 6 (“Semantic tag”) and tag value `55799` (see https://tools.ietf.org/html/rfc7049#section-2.4.5[Self-Describe CBOR]), followed by an array of principals in their binary form (CDDL `#6.55799([* bytes .size (0..29)])`) +The current controllers of the canister. The value consists of a CBOR data item with major type 6 (“Semantic tag”) and tag value `55799` (see https://tools.ietf.org/html/rfc7049#section-2.4.5[Self-Describe CBOR]), followed by an array of principals in their binary form (CDDL `#6.55799([* bytes .size (0..29)])`). [#http-interface] == HTTPS Interface -The concrete mechanism that users use to send requests to the Internet Computer is via an HTTPS API, which exposes three endpoints to handle interactions with the internet computer, plus one for diagnostics: +The concrete mechanism that users use to send requests to the Internet Computer is via an HTTPS API, which exposes three endpoints to handle interactions, plus one for diagnostics: * At `/api/v2/canister//call` the user can submit (asynchronous, state-changing) calls. -* At `/api/v2/canister//read_state` the user can read various sytem information. In particular, they can poll for the status of a call here. -* At `/api/v2/canister//query` the user can perform (synchronous, non-state-changing) query calls -* At `/api/v2/status` the user can get additional information about the network +* At `/api/v2/canister//read_state` the user can read various information about the state of the Internet Computer. In particular, they can poll for the status of a call here. +* At `/api/v2/canister//query` the user can perform (synchronous, non-state-changing) query calls. +* At `/api/v2/status` the user can retrieve status information about the Internet Computer. In these paths, the `` is the <> of the <>. @@ -431,19 +433,19 @@ NOTE: This document does not yet explain how to find the location and port of th === Overview of canister calling Users interact with the Internet Computer by calling canisters. -By the very nature of a distributed implementation, they cannot be acted upon immediately, but only with a delay. +By the very nature of a blockchain protocol, they cannot be acted upon immediately, but only with a delay. Moreover, the actual node that the user talks to may not be honest or, for other reasons, may fail to get the request on the way. This implies the following high-level workflow: -1. A user submits a call via the <>. No useful information is returned from the node (as such information cannot be trustworthy anyways). -2. For a certain amount of time, the system behaves as if it does not know about the call. -3. The system asks the targetted canister if it is willing to to accept this message and be charged for the expense of processing it. This uses the <> API for normal calls. For calls to the management canister, the rules in <> apply. -4. At some point, the system may accept the call for processing and set its status to `received`. This indicates that the system as a whole has received the call and plans on processing it (although it may still not get processed if the system is under high load). Furthermore, the user should also be able to ask any endpoint about the status of the pending call. +1. A user submits a call via the <>. No useful information is returned in the immediate response (as such information cannot be trustworthy anyways). +2. For a certain amount of time, the IC behaves as if it does not know about the call. +3. The IC asks the targeted canister if it is willing to accept this message and be charged for the expense of processing it. This uses the <> API for normal calls. For calls to the management canister, the rules in <> apply. +4. At some point, the IC may accept the call for processing and set its status to `received`. This indicates that the IC as a whole has received the call and plans on processing it (although it may still not get processed if the IC is under high load). Furthermore, the user should also be able to ask any endpoint about the status of the pending call. 5. Once it is clear that the call will be acted upon (sufficient resources, call not yet expired), the status changes to `processing`. Now the user has the guarantee that the request will have an effect, e.g. it will reach the target canister. -6. Now the system is processing the call. For some calls this may be atomic, for others this involves multiple internal steps. +6. The IC is processing the call. For some calls this may be atomic, for others this involves multiple internal steps. 7. Eventually, a response will be produced, and can be retrieved for a certain amount of time. The response is either a `reply`, indicating success, or a `reject`, indicating some form of error. -8. In the case that the call has been retained for long enough, but the request has not expired yet, the system can forget the response data and only remember the call as `done`, to prevent a replay attack. -9. Once the expiry time is past, the system can prune the call and its response, and completely forget about it. +8. In the case that the call has been retained for long enough, but the request has not expired yet, the IC can forget the response data and only remember the call as `done`, to prevent a replay attack. +9. Once the expiry time is past, the IC can prune the call and its response, and completely forget about it. This yields the following interaction diagram: @@ -471,21 +473,21 @@ if "" as X then endif .... -State transitions may be instantaneous and not always externally visible. For example, the system may move from `received` via `processing` to `replied` in one go. Similarly, the system may not implement the `done` state at all, and keep calls in state `replied`/`rejected` until they are pruned. +State transitions may be instantaneous and not always externally visible. For example, the state of a request may move from `received` via `processing` to `replied` in one go. Similarly, the IC may not implement the `done` state at all, and keep calls in state `replied`/`rejected` until they are pruned. -All gray states are _not_ explicitly represented in the system state, and are indistinguishable from “call does not exist”. +All gray states are _not_ explicitly represented in the state of the IC, and are indistinguishable from “call does not exist”. -The characteristic property of the `received` state is that the call has made it past the (potentionally malicious) endpoint _into to the system_. It is now pointless (but harmless) to submit the (identical) call again. Before reaching that state, submitting the identical call to further nodes might be a useful safeguard against a malicious or misbehaving node. +The characteristic property of the `received` state is that the call has made it past the (potentially malicious) endpoint _into the state of the IC_. It is now pointless (but harmless) to submit the (identical) call again. Before reaching that state, submitting the identical call to further nodes might be a useful safeguard against a malicious or misbehaving node. -The characteristic property of the `processing` state is _the initial effect of the call has or will happen_. This is best explained by an example: Consider a counter canister. It exports a method `inc` that increases the counter. Assume that the canister is bug free, and is not going to be forcibly removed. A user submits a call to call `inc`. If the user sees request status `processing`, the state change is guaranteed to happen, and the user can stop monitoring the status and does not have to retry submitting. +The characteristic property of the `processing` state is that _the initial effect of the call has happened or will happen_. This is best explained by an example: Consider a counter canister. It exports a method `inc` that increases the counter. Assume that the canister is bug free, and is not going to be forcibly removed. A user submits a call to call `inc`. If the user sees request status `processing`, the state change is guaranteed to happen. The user can stop monitoring the status and does not have to retry submitting. -A call may be rejected by the system or the canister. In either case, there is no guarantee about how much processing of the call has happened. +A call may be rejected by the IC or the canister. In either case, there is no guarantee about how much processing of the call has happened. To avoid replay attacks, the transition from `done` or `received` to `pruned` must happen no earlier than the call’s `ingress_expiry` field. -Calls must stay in `replied` or `rejected` long enough for polling clients to catch the response. +Calls must stay in `replied` or `rejected` long enough for polling users to catch the response. -When asking the system about the state or call of a request, the user uses the request id (see <>) to read the request status (see <>) from the state tree (see <>). +When asking the IC about the state or call of a request, the user uses the request id (see <>) to read the request status (see <>) from the state tree (see <>). [#http-call] === Request: Call @@ -502,7 +504,7 @@ The HTTP response to this request has an empty body and HTTP status 202, or a HT This request type can _also_ be used to call a query method. A user may choose to go this way, instead of via the faster and cheaper <> below, if they want to get a _certified_ response. -NOTE: The system functionality exposed via the <> can be used this way. +NOTE: The functionality exposed via the <> can be used this way. [#http-read-state] @@ -526,7 +528,7 @@ All requested paths must have one of the following paths as prefix: * `/subnet`. Can be requested by anyone. * `/request_status/`. Can only be read if `/request_id/` is not present in the state tree, or if this `read_state` request was signed by same sender as the original request referenced by ``, and the effective canister id of the original request matches the `` (see <>) in this HTTP request’s path. * `/canisters//module_hash`. Can be requested by anyone, if `` matches ``. - * `/canisters//controllers`. Can be requested by anyone, if `` matches ``. The order is system-defined. + * `/canisters//controllers`. Can be requested by anyone, if `` matches ``. The order may vary depending on the implementation. Note that the paths `/canisters//certified_data` are not accessible with this method; these paths are only exposed to the canister themselves via the System API (see <>). @@ -571,31 +573,31 @@ The `` in the URL paths of requests is the _effective_ de + [NOTE] ==== -Production instances do not support `provisional_create_canister_with_cycles` anyways. This means that using an effective canister id that could be an existing canister would lead to the request being routed by the edge component to a node the corresponding subnet and produce an error response there, while using an effective canister id that cannot be an existing canister id would cause an error response from the edge component -- either is fine. +The Internet Computer blockchain mainnet does not support `provisional_create_canister_with_cycles`. This means that using an effective canister id that could be an existing canister would lead to the request being routed to a node on the corresponding subnet and produce an error response there, while using an effective canister id that cannot be an existing canister id would cause an error response from the node receiving the request -- either is fine. -In non-production, multi-subnet instances (e.g. testnets), users with priviledged access to `provisional_create_canister_with_cycles` (or possibly similar, internal methods) and knowledge of the mapping from canisters to subnets can use their choice of the effective canister id in the URL to steer the request to a specific subnet. +In multi-subnet development instances of the Internet Computer Protocol (e.g. testnets), users with privileged access to `provisional_create_canister_with_cycles` (or possibly similar, internal methods) and knowledge of the mapping from canisters to subnets can use their choice of the effective canister id in the URL to steer the request to a specific subnet. -In local, single-subnet instances, the effective canister id is ignored by the single instance, and thus `aaaaa-aa` can be used. +In a local canister execution environment, the effective canister id is ignored, and thus `aaaaa-aa` can be used. ==== * Else, the effective canister id must be the `canister_id` in the request. -NOTE: The expecation is that user-side agent code shields users and developers from this concept, in analogy to how the system interface shields canister developers from worrying about routing. +NOTE: The expectation is that user-side agent code shields users and developers from this concept, in analogy to how the System API interface shields canister developers from worrying about routing. [#authentication] === Authentication -All requests coming in via the HTTP interface need to be either _anonymous_ or _authenticated_ using a cryptographic signature. To that end, the following fields are present in the `content` map in all cases: +All requests coming in via the HTTPS interface need to be either _anonymous_ or _authenticated_ using a cryptographic signature. To that end, the following fields are present in the `content` map in all cases: * `nonce` (`blob`, optional): Arbitrary user-provided data, typically randomly generated. This can be used to create distinct requests with otherwise identical fields. -* `ingress_expiry` (`nat`, required): An upper limit on the validity of the request, expressed in nanoseconds since 1970-01-01 (like <>). This avoids replay attacks: The system will not accept requests, or transition requests from status `received` to status `processing`, if their expiry date is in the past. The system may refuse to accept requests with an ingress expiry date too far in the future. This applies to synchronous and asynchronous requests alike (and could have been called `signature_expiry`). +* `ingress_expiry` (`nat`, required): An upper limit on the validity of the request, expressed in nanoseconds since 1970-01-01 (like <>). This avoids replay attacks: The IC will not accept requests, or transition requests from status `received` to status `processing`, if their expiry date is in the past. The IC may refuse to accept requests with an ingress expiry date too far in the future. This applies to synchronous and asynchronous requests alike (and could have been called `request_expiry`). * `sender` (`Principal`, required): The user who issued the request. The envelope, i.e. the overall request, has the following keys: * `content` (`record`): the actual request content -* `sender_pubkey` (`blob`, optional): Public key used to authenticate this request. Since a user may in the future have more than one key, this field tells the system which key is used. +* `sender_pubkey` (`blob`, optional): Public key used to authenticate this request. Since a user may in the future have more than one key, this field tells the IC which key is used. * `sender_delegation` (`array` of maps, optional): a chain of delegations, starting with the one signed by `sender_pubkey` and ending with the one delegating to the key relating to `sender_sig`. * `sender_sig` (`blob`, optional): Signature to authenticate this request. @@ -609,7 +611,7 @@ The request id (see <>) is calculated from the content record. This The field `sender_pubkey` contains a public key supported by one of the schemes described in <>. -Signing transactions can be delegated from a one key to another one. +Signing transactions can be delegated from one key to another one. If delegation is used, then the `sender_delegation` field contains an array of delegations, each of which is a map with the following fields: * `delegation` (`map`): Map with fields: @@ -649,9 +651,9 @@ The following encodings of field values as blobs are used [#request-id] === Request ids -When signing requests or querying the status of a request (see <>) in the state tree, the user identifies the request using a _request id_, which the <> of the `content` map of the original request. +When signing requests or querying the status of a request (see <>) in the state tree, the user identifies the request using a _request id_, which is the <> of the `content` map of the original request. -NOTE: The request id is independent of the representation of the request (currenly only CBOR), and does not change if the specification adds further optional field to a request type. +NOTE: The request id is independent of the representation of the request (currently only CBOR), and does not change if the specification adds further optional fields to a request type. NOTE: The recommended textual representation of a request id is a hexadecimal string with lower-case letters prefixed with '0x'. E.g., request id consisting of bytes `[00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 0A, 0B, 0C, 0D, 0E, 0F, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 1A, 1B, 1C, 1D, 1E, 1F]` should be displayed as `0x000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f`. @@ -688,11 +690,11 @@ hash_of_map({ request_type: "call", canister_id: 0x00000000000004D2, method_name [#reject-codes] === Reject codes -An API request or inter-canister call that is pending in the system will eventually result in either a _reply_ (indicating success, and carrying data) or a _reject_ (indicating an error of some sorts). A reject contains a _rejection code_ that classifies the error and a hopefully helpful _reject message_ string. +An API request or inter-canister call that is pending in the IC will eventually result in either a _reply_ (indicating success, and carrying data) or a _reject_ (indicating an error of some sorts). A reject contains a _rejection code_ that classifies the error and a hopefully helpful _reject message_ string. Rejection codes are member of the following enumeration: -* `SYS_FATAL` (1): Fatal system error, retry unlikely to be useful. +* `SYS_FATAL` (1): Fatal system error, retry unlikely to be useful. * `SYS_TRANSIENT` (2): Transient system error, retry might be possible. * `DESTINATION_INVALID` (3): Invalid destination (e.g. canister/account does not exist) * `CANISTER_REJECT` (4): Explicit reject by the canister. @@ -702,7 +704,7 @@ The symbolic names of this enumeration are used throughout this specification, b The error message is guaranteed to be a string, i.e. not arbitrary binary data. -When canisters explicitly reject a message (see <>), they can specify the reject message, but _not_ the reject code; it is always `CANISTER_REJECT`. In this sense, the reject code is trustworthy: If the system responds with a `SYS_FATAL` reject, then it really was the system issuing this reject. +When canisters explicitly reject a message (see <>), they can specify the reject message, but _not_ the reject code; it is always `CANISTER_REJECT`. In this sense, the reject code is trustworthy: If the IC responds with a `SYS_FATAL` reject, then it really was the IC issuing this reject. [#api-status] === Status endpoint @@ -712,19 +714,19 @@ Additionally, the Internet Computer provides an API endpoint to obtain various s /api/v2/status .... -For this endpoint, the user performs a GET request, and receives a CBOR value with the following fields. The Internet Computer may include additional implementation-specific fields. +For this endpoint, the user performs a GET request, and receives a CBOR value with the following fields. The IC may include additional implementation-specific fields. * `ic_api_version` (string, mandatory): Identifies the interface version supported, i.e. the version of the present document that the internet computer aims to support, e.g. `0.8.1`. The implementation may also return `unversioned` to indicate that it does _not_ comply to a particular version, e.g. in between releases. -* `impl_source` (string, optional): Identifies the implementation of the Internet Computer, by convention with the canonical location of the source code (e.g. `https://github.com/dfinity/dfinity`). -* `impl_version` (string, optional): If the user is talking to a released version of an Internet Computer implementation, this is the version number. For non-released versions, output of `git describe` like `0.1.13-13-g2414721` would also be very suitable. -* `impl_revision` (string, optional): The precise git revision of the Internet Computer implementation -* `root_key` (blob, only in development instances): The public key (a DER-encoded BLS key) of the root key of this Internet Computer instance. This _must_ be present in short-lived development instances, to allow the client to fetch the public key. In production environments, clients must have an independent trustworthy source for this data, and must not be tempted to fetch it from this insecure location. +* `impl_source` (string, optional): Identifies the implementation of the Internet Computer Protocol, by convention with the canonical location of the source code (e.g. `https://github.com/dfinity/ic`). +* `impl_version` (string, optional): If the user is talking to a released version of an Internet Computer Protocol implementation, this is the version number. For non-released versions, output of `git describe` like `0.1.13-13-g2414721` would also be very suitable. +* `impl_revision` (string, optional): The precise git revision of the Internet Computer Protocol implementation +* `root_key` (blob, only in development instances): The public key (a DER-encoded BLS key) of the root key of this development instance of the Internet Computer Protocol. This _must_ be present in short-lived development instances, to allow the agent to fetch the public key. For the Internet Computer, agents must have an independent trustworthy source for this data, and must not be tempted to fetch it from this insecure location. See <> for details on the precise CBOR encoding of this object. NOTE: Future additions may include local time, geographic location, and other useful implementation-specific information such as blockheight. This data may -possibly signed by the node. +possibly be signed by the node. [#api-cbor] === CBOR encoding of requests and responses @@ -778,9 +780,8 @@ include::{example}requests.cddl[] === Ordering guarantees -In order to allow for a distributed implementation of the Internet Computer, the order in which the various messages between canisters are delivered and executed is not fully specified. - -The guarantee we do give is that function calls between two canisters are executed in order, so that a canister that requires in-order execution need not wait for the response from an earlier message to a canister before sending a later message to that same canister. +The order in which the various messages between canisters are delivered and executed is not fully specified. +The guarantee provided by the IC is that function calls between two canisters are executed in order, so that a canister that requires in-order execution need not wait for the response from an earlier message to a canister before sending a later message to that same canister. More precisely: @@ -790,7 +791,7 @@ More precisely: to the same canister, they are queued in the order of invocations to `ic0.call_perform`. * Responses (including replies with `ic0.msg_reply`, explicit rejects with `ic0.msg_reject` and system-generated error responses) do _not_ have any ordering guarantee relative to each other or to method calls. * There is no particular order guarantee for ingress messages submitted via - the HTTP interface. + the HTTPS interface. === Synchronicity across nodes @@ -813,9 +814,9 @@ A canister module is simply a https://webassembly.github.io/spec/core/index.html [#system-api] == Canister interface (System API) -The System API is the interface between the running canister and the Internet Computer. It allows the WebAssembly module of a canister to expose functionality to the users (method entry points) and the system (e.g. initialization), and exposes system functionality to the canister (e.g. calling other canisters). Because WebAssembly is rather low-level, it also explains how to express higher level concepts (e.g. binary blobs). +The System API is the interface between the running canister and the Internet Computer. It allows the WebAssembly module of a canister to expose functionality to the users (method entry points) and the IC (e.g. initialization), and exposes functionality of the IC to the canister (e.g. calling other canisters). Because WebAssembly is rather low-level, it also explains how to express higher level concepts (e.g. binary blobs). -We want to leverage advanced WebAssembly features, such as WebAssembly host references. But as they are not yet supported by all tools involved, this section describes an initial System API that does not rely on host references. To emphasize that this is just a preliminary interface, we group the system methods under the module name `ic0`, planning to use `ic` for the real deal. +We want to leverage advanced WebAssembly features, such as WebAssembly host references. But as they are not yet supported by all tools involved, this section describes an initial System API that does not rely on host references. In section <>, we outline some of the proposed uses of WebAssembly host references. [#system-api-module] @@ -829,9 +830,11 @@ In order for a WebAssembly module to be usable as the code for the canister, it * It may have a `(start)` function. * If it exports a function called `canister_init`, the function must have type `+() -> ()+`. * If it exports a function called `canister_inspect_message`, the function must have type `+() -> ()+`. +* If it exports a function called `canister_heartbeat`, the function must have type `+() -> ()+`. * If it exports any functions called `canister_update ` or `canister_query ` for some `name`, the functions must have type `+() -> ()+`. * It may not export both `canister_update ` and `canister_query ` with the same `name`. -* The system may reject WebAssembly modules that declare more than 6000 functions or more than 200 globals. +* It may not export other methods the names of which start with the prefix `canister_` besides the methods allowed above. +* The IC may reject WebAssembly modules that declare more than 6000 functions or more than 200 globals. === Interpretation of numbers @@ -839,30 +842,31 @@ WebAssembly number types (`i32`, `i64`) do not indicate if the numbers are to be === Entry points -The canister provides entry points which are invoked by the system under various circumstances: +The canister provides entry points which are invoked by the IC under various circumstances: * The canister may export a function named `canister_init` and type `+() -> ()+`. * The canister may export a function named `canister_pre_upgrade` and type `+() -> ()+`. * The canister may export a function named `canister_post_upgrade` and type `+() -> ()+`. * The canister may export functions named `canister_inspect_message` with type `+() -> ()+`. +* The canister may export a function named `canister_heartbeat` with type `+() -> ()+`. * The canister may export functions named `canister_update ` and type `+() -> ()+`. * The canister may export functions named `canister_query ` and type `+() -> ()+`. * The canister table may contain functions of type `+(env : i32) -> ()+` which may be used as callbacks for inter-canister calls. -If the execution of any of these entry points traps for any reason, then all changes to the WebAssembly state, as well as the effect of any externally visible system call (like `ic0.msg_reply`, `ic0.msg_reject`, `ic0.call_perform`), are discarded. For upgrades, this transactional behaviour applies to the `canister_pre_upgrade`/`canister_post_upgrade` sequence as a whole. +If the execution of any of these entry points traps for any reason, then all changes to the WebAssembly state, as well as the effect of any externally visible system call (like `ic0.msg_reply`, `ic0.msg_reject`, `ic0.call_perform`), are discarded. For upgrades, this transactional behavior applies to the `canister_pre_upgrade`/`canister_post_upgrade` sequence as a whole. [#system-api-init] ==== Canister initialization -If `canister_init` is present, then this is the first exported WebAssembly function invoked by the system. The argument that was passed along with the canister initialization call (see <>) is available to the canister via `ic0.msg_arg_data_size/copy`. +If `canister_init` is present, then this is the first exported WebAssembly function invoked by the IC. The argument that was passed along with the canister initialization call (see <>) is available to the canister via `ic0.msg_arg_data_size/copy`. -The system assumes the canister to be fully instantiated if the `canister_init` method entry point returns. If the `canister_init` method entry point traps, then canister installation has failed, and the canister is reverted to its previous state (i.e. empty with `install`, or whatever it was for a `reinstall`). +The IC assumes the canister to be fully instantiated if the `canister_init` method entry point returns. If the `canister_init` method entry point traps, then canister installation has failed, and the canister is reverted to its previous state (i.e. empty with `install`, or whatever it was for a `reinstall`). [#system-api-upgrades] ==== Canister upgrades -When a canister is upgraded to a new WebAssembly module, the system: +When a canister is upgraded to a new WebAssembly module, the IC: 1. Invokes `canister_pre_upgrade` (if present) on the old instance, to give the canister a chance to clean up (e.g. move data to <>). 2. Instantiates the new module, including the execution of `(start)`, with a fresh WebAssembly state. @@ -881,10 +885,18 @@ To define a public method of name `name`, a WebAssembly module exports a functio NOTE: The space in `canister_update ` resp. `canister_query ` is intentional. There is exactly one space between `canister_update/canister_query` and the ``. -The argument of the call (e.g. the content of the `arg` field in the <>) is copied into the canister on demand using the System functions shown below. +The argument of the call (e.g. the content of the `arg` field in the <>) is copied into the canister on demand using the System API functions shown below. Eventually, a method will want to send a response, using `ic0.reply` or `ic0.reject` +==== Heartbeat + +For periodic or time-based execution, the WebAssembly module can export a function with name `canister_heartbeat`. If present, the IC will invoke this function at regular intervals. The exact interval is implementation-defined. For each heartbeat invocation, the IC guarantees that the time, as returned by <>, is monotonically increasing. + +`canister_heartbeat` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. + +NOTE: While an implementation will likely try to keep the interval between `canister_heartbeat` invocations to within a few seconds, this is not formally part of this specification. + ==== Callbacks Callbacks are addressed by their table index (as a proxy for a Wasm `funcref`). @@ -895,7 +907,7 @@ In the reply callback of a <>, the a [#system-api-imports] === Overview of imports -The following sections describe various system imports, which we summarize here. +The following sections describe various System API functions, also referred to as system calls, which we summarize here. .... ic0.msg_arg_data_size : () -> i32; // I U Q Ry F @@ -923,7 +935,7 @@ ic0.msg_method_name_size : () -> i32 // F ic0.msg_method_name_copy : (dst : i32, offset : i32, size : i32) -> (); // F ic0.accept_message : () -> (); // F -ic0.call_new : // U Ry Rt +ic0.call_new : // U Ry Rt H ( callee_src : i32, callee_size : i32, name_src : i32, @@ -933,17 +945,21 @@ ic0.call_new : // U reject_fun : i32, reject_env : i32 ) -> (); -ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt -ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt -ic0.call_cycles_add : ( amount : i64 ) -> (); // U Ry Rt -ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt +ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt H +ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt H +ic0.call_cycles_add : ( amount : i64 ) -> (); // U Ry Rt H +ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt H ic0.stable_size : () -> (page_count : i32); // * ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32); // * ic0.stable_write : (offset : i32, src : i32, size : i32) -> (); // * ic0.stable_read : (dst : i32, offset : i32, size : i32) -> (); // * +ic0.stable64_size : () -> (page_count : i64); // * +ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64); // * +ic0.stable64_write : (offset : i64, src : i64, size : i64) -> (); // * +ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> (); // * -ic0.certified_data_set : (src: i32, size: i32) -> () // I G U Ry Rt +ic0.certified_data_set : (src: i32, size: i32) -> () // I G U Ry Rt H ic0.data_certificate_present : () -> i32 // * ic0.data_certificate_size : () -> i32 // * ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> () // * @@ -965,9 +981,10 @@ The comment after each function lists from where these functions may be invoked: * `C`: from a cleanup callback * `s`: the `(start)` module initialization function * `F`: from `canister_inspect_message` +* `H`: from `canister_heartbeat` * `*` = `I G U Q Ry Rt C F` (NB: Not `(start)`) -If the canister invokes a system import from somewhere else, it will trap. +If the canister invokes a system call from somewhere else, it will trap. === Blob-typed arguments and results @@ -1003,7 +1020,7 @@ The message argument data. ic0.msg_caller_size : () -> i32 ic0.msg_caller_copy : (dst : i32, offset: i32, size : i32) -> () + -The identity of the caller, which may be a canister id or a user id. During canister installation or upgrade, this is the id of of the user or canister requesting the installation or upgrade. +The identity of the caller, which may be a canister id or a user id. During canister installation or upgrade, this is the id of the user or canister requesting the installation or upgrade. * `+ic0.msg_reject_code : () -> i32+` + @@ -1016,7 +1033,7 @@ It returns the special “no error” code `0` if the callback is _not_ invoked ic0.msg_reject_msg_size : () -> i32 ic0.msg_reject_msg_copy : (dst : i32, offset : i32, size : i32) -> () + -The reject message. Traps if there there is no reject message (i.e. if `reject_code` is `0`). +The reject message. Traps if there is no reject message (i.e. if `reject_code` is `0`). [#responding] === Responding @@ -1025,7 +1042,7 @@ Eventually, the canister will want to respond to the original call, either by re * `+ic0.msg_reply_data_append : (src : i32, size : i32) -> ()+` + -Appends idata it to the (initially empty) data reply. +Appends data it to the (initially empty) data reply. + NOTE: This can be invoked multiple times to build up the argument with data from various places on the Wasm heap. This way, the canister does not have to first copy all the pieces from various places into one location. + @@ -1111,11 +1128,11 @@ When handling an update call (or a callback), a canister can do further calls to Begins assembling a call to the canister specified by `callee_src/_size` at method `name_src/_size`. -The system records two mandatory callback functions, represented by a table entry index `fun` and some additional value `env`. When the response comes back, the table is read at the corresponding index, expected to be a function of type `+(env : i32) -> ()+` and passed the corresponding `env` value. +The IC records two mandatory callback functions, represented by a table entry index `*_fun` and some additional value `*_env`. When the response comes back, the table is read at the corresponding index, expected to be a function of type `+(env : i32) -> ()+`, and passed the corresponding `*_env` value. The reply callback is executed upon successful completion of the method call, which can query the reply using `ic0.msg_arg_data_*`. -The reject callback is executed if the method calls fails asynchronously or the other canister explicitly rejects the call. The reject code and message can be queried using `ic0.msg_reject_code` and `ic0.msg_reject_msg_*`. +The reject callback is executed if the method call fails asynchronously or the other canister explicitly rejects the call. The reject code and message can be queried using `ic0.msg_reject_code` and `ic0.msg_reject_msg_*`. This deducts `MAX_CYCLES_PER_RESPONSE` cycles from the canister balance and sets them aside for response processing. This will trap if not sufficient cycles are available. @@ -1127,7 +1144,7 @@ Subsequent calls to the following functions set further attributes of that call, -- ic0.call_on_cleanup : (fun : i32, env : i32) -> () -If the a cleanup callback is specified for this call, it is executed if and only if the `reply` or the `reject` callback was executed and trapped (for any reason). +If a cleanup callback (of type `+(env : i32) -> ()+`) is specified for this call, it is executed if and only if the `reply` or the `reject` callback was executed and trapped (for any reason). During the execution of the `cleanup` function, only a subset of the System API is available (namely `ic0.debug_print`, `ic0.trap` and the `ic0.stable_*` functions). The cleanup function is expected to run swiftly (within a fixed, yet to be specified cycle limit) and serves to free resources associated with the callback. @@ -1136,14 +1153,14 @@ If this traps (e.g. runs out of cycles), the state changes from the `cleanup` fu There must be at most one call to `ic0.call_on_cleanup` between `ic0.call_new` and `ic0.call_perform`. -- -* `+ic0.call_data_append : (src : i32, size : i32)+` +* `+ic0.call_data_append : (src : i32, size : i32) -> ()+` + Appends the specified bytes to the argument of the call. Initially, the argument is empty. + This may be called multiple times between `ic0.call_new` and `ic0.call_perform`. -* `+ic0.call_cycles_add : ( amount : i64) -> ()+` +* `+ic0.call_cycles_add : (amount : i64) -> ()+` + This adds cycles onto a call. See <>. + @@ -1151,24 +1168,24 @@ This may be called multiple times between `ic0.call_new` and `ic0.call_perform`. * `+ic0.call_perform : () -> ( err_code : i32 )+` + -This call concludes assembling the call. It queues the call message to the given destination, but does not actually act on it until the current WebAssembly function returns without trapping. +This concludes assembling the call. It queues the call message to the given destination, but does not actually act on it until the current WebAssembly function returns without trapping. + -If the system returns `0` as the `err_code`, the system was able to enqueue the call. In this case, the call will either be delivered, returned because the destination canister does not exist or returned because of an out of cycles condition. This also means that exactly one of the reply or reject callback will be executed. +If the function returns `0` as the `err_code`, the IC was able to enqueue the call. In this case, the call will either be delivered, returned because the destination canister does not exist or returned because of an out of cycles condition. This also means that exactly one of the reply or reject callbacks will be executed. + -If the system returns a non-zero value, the call cannot (and will not be) performed. This can happen due to a lack of system internal resources, but also if it would reduce the current cycle balance to a level below where the canister would be frozen. +If the function returns a non-zero value, the call cannot (and will not be) performed. This can happen due to a lack of resources within the IC, but also if it would reduce the current cycle balance to a level below where the canister would be frozen. + -After `ic0.call_perform` and before the next call to `ic0.call_new`, all other `ic0.call_*` function calls trap +After `ic0.call_perform` and before the next call to `ic0.call_new`, all other `ic0.call_*` function calls trap. [#system-api-cycles] === Cycles Each canister maintains a balance of _cycles_, the utility token used to pay for platform usage. -NOTE: This specification currently does not go into details about which actions cost how many cycles and/or when. In general, you must assume that the canister’s cycle balance can change arbitrarily between method executions, and during each system function API call, unless explicitly mentioned otherwise. +NOTE: This specification currently does not go into details about which actions cost how many cycles and/or when. In general, you must assume that the canister’s cycle balance can change arbitrarily between method executions, and during each System API function call, unless explicitly mentioned otherwise. * `ic0.canister_cycle_balance : () -> i64` + -indicates the current cycle balance of the canister. It is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add`. After execution of the message, the system may add unused cycles from the reserve back to the balance. +indicates the current cycle balance of the canister. It is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add`. After execution of the message, the IC may add unused cycles from the reserve back to the balance. + NOTE: Should a future version of this specification allow `MAX_CANISTER_BALANCE` >= 2^64^, then this call will report 2^64^-1 if the balance exceeds 2^64^. A new, wider System API will then be provided for canisters that need to deal precisely with large canister balances. @@ -1176,7 +1193,7 @@ NOTE: Should a future version of this specification allow `MAX_CANISTER_BALANCE` + returns the amount of cycles that were transferred by the caller of the current call, and is still available in this message. + -Initially, in the update method entry point, this is the amount that the caller passed to the canister. When cycles are accepted (`ic0.msg_cycles_accept`), this reports less cycles accordingly. When the call is responsed to (reply or reject), all available cycles are refunded to the caller, and this will return 0. +Initially, in the update method entry point, this is the amount that the caller passed to the canister. When cycles are accepted (`ic0.msg_cycles_accept`), this reports less cycles accordingly. When the call is responded to (reply or reject), all available cycles are refunded to the caller, and this will return 0. * `ic0.msg_cycles_accept : ( max_amount : i64 ) -> ( amount : i64 )` + @@ -1187,13 +1204,13 @@ This moves cycles from the call to the canister balance. It moves as many cycles * It moves no more cycles than available according to `ic0.msg_cycles_available`, and -* The canister balance afterwards does not exceeed `MAX_CANISTER_BALANCE` minus any possible outstanding balances. The precise logic of this limit is not yet specified here. +* The canister balance afterwards does not exceed `MAX_CANISTER_BALANCE` minus any possible outstanding balances. The precise logic of this limit is not yet specified here. It can be called multiple times, each time possibly adding more cycles to the balance. -The return value indicates how much cycles were actually moved. +The return value indicates how many cycles were actually moved. -This does not trap. +This system call is deprecated and does not trap. [TIP] ===== @@ -1205,7 +1222,7 @@ Example: To accept all cycles provided in a call, invoke `ic0.msg_cycles_accept( + This function moves cycles from the canister balance onto the call under construction, to be transferred with that call. + -The cycles are deducted from the balance as shown by `ic0.canister_cycle_balance` immediately, and moved back if the call cannot be performed (e.g. if `ic0.call_perform` signals an error, or if the canister invokes `ic0.call_new` or returns without callling `ic0.call_perform`). +The cycles are deducted from the balance as shown by `ic0.canister_cycle_balance` immediately, and moved back if the call cannot be performed (e.g. if `ic0.call_perform` signals an error, or if the canister invokes `ic0.call_new` or returns without calling `ic0.call_perform`). + This traps if trying to transfer more cycles than are in the current balance of the canister. @@ -1222,34 +1239,74 @@ The stable memory is initially empty. * `ic0.stable_size : () -> (page_count : i32)` + -returns the current size of the stable memory in WebAssembly pages. (One WebAssembly page is 64Ki bytes.) +returns the current size of the stable memory in WebAssembly pages. (One WebAssembly page is 64KiB) ++ +This system call traps if the size of the stable memory exceeds 2^32 bytes. * `ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32)` + tries to grow the memory by `new_pages` many pages containing zeroes. + -If successful, returns the _previous_ size of the memory (in pages). Otherwise, returns `-1`. +This system call traps if the _previous_ size of the memory exceeds 2^32 bytes. ++ +If the _new_ size of the memory exceeds 2^32 bytes or growing is unsuccessful, then it returns `-1`. ++ +Otherwise, it grows the memory and returns the _previous_ size of the memory in pages. * `ic0.stable_write : (offset : i32, src : i32, size : i32) -> ()` + copies the data referred to by `src`/`size` out of the canister and replaces the corresponding segment starting at `offset` in the stable memory. + -This system call traps if `src+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. +This system call traps if the size of the stable memory exceeds 2^32 bytes. ++ +It also traps if `src+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. * `ic0.stable_read : (dst : i32, offset : i32, size : i32) -> ()` + copies the data referred to by `offset`/`size` out of the stable memory and replaces the corresponding bytes starting at `dest` in the canister memory. + +This system call traps if the size of the stable memory exceeds 2^32 bytes. ++ +It also traps if `dst+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory + +* `ic0.stable64_size : () -> (page_count : i64)` ++ +returns the current size of the stable memory in WebAssembly pages. (One WebAssembly page is 64KiB) ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +* `ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64)` ++ +tries to grow the memory by `new_pages` many pages containing zeroes. ++ +If successful, returns the _previous_ size of the memory (in pages). Otherwise, returns `-1`. ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +* `ic0.stable64_write : (offset : i64, src : i64, size : i64) -> ()` ++ +Copies the data from location [src, src+size) of the canister memory to location [offset, offset+size) in the stable memory. ++ +This system call traps if `src+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +* `ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> ()` ++ +Copies the data from location [offset, offset+size) of the stable memory to the location [dst, dst+size) in the canister memory. ++ This system call traps if `dst+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. [#system-api-time] === System time -The canister can query the system for the current time. +The canister can query the IC for the current time. `+ic0.time : () -> i64+` -The time is given as nanoseconds since 1970-01-01. The system guarantees that +The time is given as nanoseconds since 1970-01-01. The IC guarantees that * the time, as observed by the canister, is monotonically increasing, even across canister upgrades. * within an invocation of one entry point, the time is constant. @@ -1261,7 +1318,7 @@ NOTE: While an implementation will likely try to keep the System Time close to t [#system-api-certified-data] === Certified data -For each canister, the system keeps track of “certified data”, a canister-defined blob. For fresh canisters, this blob is the empty blob (`""`). +For each canister, the IC keeps track of “certified data”, a canister-defined blob. For fresh canisters, this blob is the empty blob (`""`). * `+ic0.certified_data_set : (src: i32, size : i32) -> ()+` + @@ -1291,7 +1348,7 @@ This traps if `ic0.data_certificate_present()` returns `0`. === Debugging aids -During local development and execution on a local network, the canister needs a way to emit textual trace messages. On the “real” network, these do not do anything. +In a local canister execution environment, the canister needs a way to emit textual trace messages. On the “real” network, these do not do anything. * `+ic0.debug_print : (src : i32, size : i32) -> ()+` + @@ -1299,7 +1356,7 @@ When executing in an environment that supports debugging, this copies out the da + Semantically, this function is always a no-op, and never traps, even if the `src+size` exceeds the size of the memory, or if this function is executed from `(start)`. If the environment cannot perform the print, it just skips it. -Similarly, the system allows the canister to effectively trap, but give some indication about why it trapped: +Similarly, the System API allows the canister to effectively trap, but give some indication about why it trapped: * `+ic0.trap : (src : i32, size : i32) -> ()+` + @@ -1312,11 +1369,11 @@ The environment may copy out the data specified by `src` and `size`, and log, pr The Internet Computer aims to make the most of the WebAssembly platform, and embraces WebAssembly features. With WebAssembly host references, we can make the platform more secure, the interfaces more abstract and more compositional. The above `ic0` System API does not yet use WebAssembly host references. Once they become available on our platform, a new version of the System API using host references will be available via the `ic` module. The changes will be, at least -1. The introduction of a `api_nonce` reference, which models the capability to use the System API. It is passed as an argument to `canister_init`, `canister_update ` etc., and expected as an argument by almost all system function calls. (The debugging aids remain unconstrainted.) +1. The introduction of a `api_nonce` reference, which models the capability to use the System API. It is passed as an argument to `canister_init`, `canister_update ` etc., and expected as an argument by almost all System API function calls. (The debugging aids remain unconstrained.) 2. The use of references, instead of binary blobs, to address principals (user ids, canister ids), e.g. in `ic0.msg_caller` or in `ic0.call_new`. Additional functions will be provided to convert between the transparent binary representation of principals and references. 3. Making the builder interface to create calls build calls identified by a reference, rather than having an implicit partial call in the background. -A canister may only use the old _or_ the new interface; the system detects which interface the canister intends to use based on the names and types of its function imports and exports. +A canister may only use the old _or_ the new interface; the IC detects which interface the canister intends to use based on the names and types of its function imports and exports. [#ic-management-canister] == The IC management canister @@ -1342,27 +1399,27 @@ The binary encoding of arguments and results are as per Candid specification. [#ic-create_canister] === IC method `create_canister` -Before deploying a canister, the administrator of the canister first has to register it with the system, to get a canister id (with an empty canister behind it), and then separately install the code. +Before deploying a canister, the administrator of the canister first has to register it with the IC, to get a canister id (with an empty canister behind it), and then separately install the code. The optional `settings` parameter can be used to set the following settings: * `controllers` (`vec principal`) + -A list of principles. Must be between 0 and 10 in size. This value is assigned to the _controllers_ attribute of the canister. +A list of principals. Must be between 0 and 10 in size. This value is assigned to the _controllers_ attribute of the canister. + -Default value: A list containing only the caller of the `create_canister` call +Default value: A list containing only the caller of the `create_canister` call. * `compute_allocation` (`nat`) + Must be a number between 0 and 100, inclusively. It indicates how much compute power should be guaranteed to this canister, expressed as a percentage of the maximum compute power that a single canister can allocate. -If the system cannot provide the requested allocation, for example because it is oversubscribed, the call will be rejected. +If the IC cannot provide the requested allocation, for example because it is oversubscribed, the call will be rejected. + Default value: 0 * `memory_allocation` (`nat`) + -Must be a number between 0 and 2^48 (i.e 256TB), inclusively. It indicates how much memory the canister is allowed to use in total. Any attempt to grow memory usage beyond this allocation will fail. If the system cannot provide the requested allocation, for example because it is oversubscribed, the call will be rejected. -If set to 0, then memory growth of the canister will be best-effort and subject to the available memory on the network. +Must be a number between 0 and 2^48^ (i.e 256TB), inclusively. It indicates how much memory the canister is allowed to use in total. Any attempt to grow memory usage beyond this allocation will fail. If the IC cannot provide the requested allocation, for example because it is oversubscribed, the call will be rejected. +If set to 0, then memory growth of the canister will be best-effort and subject to the available memory on the IC. + Default value: 0 @@ -1370,9 +1427,9 @@ Until code is installed, the canister is `Empty` and behaves like a canister tha * `freezing_threshold` (`nat`) + -Must be a number between 0 and 2^64-1, inclusively, and indicates a length of time in seconds. +Must be a number between 0 and 2^64^-1, inclusively, and indicates a length of time in seconds. + -A canister is considered frozen whenever the system estimates that the canister would be depleted of cycles before `freezing_threshold` seconds pass, given the canister’s current size and the system’s current cost for storage. +A canister is considered frozen whenever the IC estimates that the canister would be depleted of cycles before `freezing_threshold` seconds pass, given the canister’s current size and the IC’s current cost for storage. + Calls to a frozen canister will be rejected (like for a stopping canister). Additionally, a canister cannot perform calls if that would, due the cost of the call and transferred cycles, would push the balance into frozen territory; these calls fail with `ic0.call_perform` returning a non-zero error code. + @@ -1393,13 +1450,13 @@ This method installs code into a canister. Only _controllers of the canister can install code. -* If `mode = install`, the canister must be empty before. This will instantiate the canister module and invoke its `canister_init` system method (if present), as explained in Section “<>”, passing the `arg` to the canister. +* If `mode = install`, the canister must be empty before. This will instantiate the canister module and invoke its `canister_init` method (if present), as explained in Section “<>”, passing the `arg` to the canister. * If `mode = reinstall`, if the canister was not empty, its existing code and state is removed before proceeding as for `mode = install`. + Note that this is different from `uninstall_code` followed by `install_code`, as that will forcibly reject all unresponded calls. -* If `mode = upgrade`, this will perform an upgrade of a non-empty canister as described in <>, passing `arg` to the `canister_post_upgrade` system method of the new instance. +* If `mode = upgrade`, this will perform an upgrade of a non-empty canister as described in <>, passing `arg` to the `canister_post_upgrade` method of the new instance. This is atomic: If the response to this request is a `reject`, then this call had no effect. @@ -1410,7 +1467,7 @@ NOTE: Some canisters may not be able to make sense of callbacks after upgrades; This method removes a canister’s code and state, making the canister _empty_ again. -Only _controllers of the canister can uninstall code. +Only controllers of the canister can uninstall code. Uninstalling a canister’s code will reject all calls that the canister has not yet responded to, and drop the canister’s code and state. Outstanding responses to the canister will not be processed, even if they arrive after code has been installed again. @@ -1437,7 +1494,7 @@ Only the controllers of the canister can request its status. The controllers of a canister may stop a canister (e.g., to prepare for a canister upgrade). Stopping a canister is not an atomic action. The immediate effect is that the status of the canister is changed to `stopping` (unless the canister is already stopped). -The system will reject all calls to a stopping canister, indicating that the canister is stopping. +The IC will reject all calls to a stopping canister, indicating that the canister is stopping. Responses to a stopping canister are processed as usual. When all outstanding responses have been processed (so there are no open call contexts), the canister status is changed to `stopped` and the management canister responds to the caller of the `stop_canister` request. @@ -1455,7 +1512,7 @@ If the canister was already `running` then the status stays unchanged. This method deletes a canister from the IC. -Only _controllers of the canister can delete it and the canister must already be stopped. Deleting a canister cannot be undone, any state stored on the canister is permanently deleted and its cycles are discarded. Once a canister is deleted, its ID cannot be reused. +Only controllers of the canister can delete it and the canister must already be stopped. Deleting a canister cannot be undone, any state stored on the canister is permanently deleted and its cycles are discarded. Once a canister is deleted, its ID cannot be reused. [#ic-deposit_cycles] === IC method `deposit_cycles` @@ -1472,29 +1529,29 @@ This method takes no input and returns 32 pseudo-random bytes to the caller. The [#ic-provisional_create_canister_with_cycles] === IC method `provisional_create_canister_with_cycles` -As a provisional method, until developers can convert real ICP tokens to provision a new canister with cycles, the system provides the `provisional_create_canister_with_cycles` method. It behaves as `create_canister`, but initializes the canister’s balance with `amount` fresh cycles (using `MAX_CANISTER_BALANCE` if `amount = null`, else capping the balance at `MAX_CANISTER_BALANCE`). +As a provisional method on development instances, the `provisional_create_canister_with_cycles` method is provided. It behaves as `create_canister`, but initializes the canister’s balance with `amount` fresh cycles (using `MAX_CANISTER_BALANCE` if `amount = null`, else capping the balance at `MAX_CANISTER_BALANCE`). Cycles added to this call via `ic0.call_cycles_add` are returned to the caller. -This method is only available in local development instances, and will be removed in the future. +This method is only available in local development instances. [#ic-provisional_top_up_canister] === IC method `provisional_top_up_canister` -As a provisional method, until developers can convert real ICP tokens to a top up an existing canister, the system provides the `provisional_top_up_canister` method. It adds `amount` cycles to the balance of canister identified by `amount` (implicitly capping it at `MAX_CANISTER_BALANCE`). +As a provisional method on development instances, the `provisional_top_up_canister` method is provided. It adds `amount` cycles to the balance of canister identified by `amount` (implicitly capping it at `MAX_CANISTER_BALANCE`). Cycles added to this call via `ic0.call_cycles_add` are returned to the caller. Any user can top-up any canister this way. -This method is only available in local development instances, and will be removed in the future. +This method is only available in local development instances. [#certification] == Certification -Some parts of the system state are exposed to clients in a tamperproof way via certification: the system can reveal a _partial state tree_ which includes just the data of interest, together with a signature on the root hash of the state tree. This means that a client can be sure that the response is correct, even if the client happens to be communicating with a malicious server, or have received the certificate via some other untrusted way. +Some parts of the IC state are exposed to users in a tamperproof way via certification: the IC can reveal a _partial state tree_ which includes just the data of interest, together with a signature on the root hash of the state tree. This means that a user can be sure that the response is correct, even if the user happens to be communicating with a malicious node, or has received the certificate via some other untrusted way. -To validate a value using a certificate, the client conceptually +To validate a value using a certificate, the user conceptually 1. checks the validity of the partial tree using `verify_cert`, 2. looks up the value in the certificate using `lookup` at a given path, which uses the subroutine `lookup_path` on the certificate's tree @@ -1503,7 +1560,7 @@ This mechanism is used in the `read_state` request type, and eventually also for === Root of trust -The root of trust is the _root public key_, which must be known to the client a priori. For temporary instances of the internet computer, e.g. during local development, the key can be fetched via the <> endpoint. +The root of trust is the _root public key_, which must be known to the user a priori. In a local canister execution environment, the key can be fetched via the <> endpoint. === Certificate @@ -1513,7 +1570,7 @@ A certificate consists of * a signature on the tree root hash valid under some _public key_ * an optional _delegation_ that links that public key to _root public key_. -The system will certify states, by issuing certificates where the tree is a partial state tree. The state tree can be pruned by replacing subtrees with their root hashes (yielding a new and potentially smaller but still valid certificate) to only include paths pertaining to relevant data but still preserving enough information to recover the _tree root hash_. +The IC will certify states by issuing certificates where the tree is a partial state tree. The state tree can be pruned by replacing subtrees with their root hashes (yielding a new and potentially smaller but still valid certificate) to only include paths pertaining to relevant data but still preserving enough information to recover the _tree root hash_. More formally, a certificate is described by the following data structure: .... @@ -1559,11 +1616,11 @@ extract_der : Blob -> Blob .... implements DER decoding of the public key, following https://tools.ietf.org/html/rfc5480[RFC4580] using OID 1.3.6.1.4.1.44668.5.3.1.2.1 for the algorithm and 1.3.6.1.4.1.44668.5.3.2.1 for the curve. -All state trees include the time at path `/time` (see <>). Clients that get a certificate with a state tree can look up the timestamp to guard against working on obsolete data. +All state trees include the time at path `/time` (see <>). Users that get a certificate with a state tree can look up the timestamp to guard against working on obsolete data. === Lookup -Given a (verified) tree, the client can fetch the value at a given path, which is a sequence of labels (blobs). In this document, we write paths suggestively with slashes as separators; the actual encoding is not actually using slashes as delimitors. +Given a (verified) tree, the user can fetch the value at a given path, which is a sequence of labels (blobs). In this document, we write paths suggestively with slashes as separators; the actual encoding is not actually using slashes as delimiters. The following algorithm looks up a `path` in a certificate, and returns either @@ -1600,7 +1657,7 @@ find_label(l, []) = Absent find_label(l, _) = Unknown .... -The system will only produce well-formed state trees, and the above algorithm assumes well-formed trees. These have the property that labeled subtrees appear in strictly increasing order of labels, and are not mixed with leaves. More formally: +The IC will only produce well-formed state trees, and the above algorithm assumes well-formed trees. These have the property that labeled subtrees appear in strictly increasing order of labels, and are not mixed with leaves. More formally: .... well_formed(tree) = (tree = Leaf _) ∨ (well_formed_forest(flatten_forks(tree))) @@ -1614,7 +1671,7 @@ well_formed_forest(trees) = [#certification-delegation] === Delegation -To facilitate a distributed implementation of the Internet Computer, the root key can delegate certification authority to other keys. +The root key can delegate certification authority to other keys. A certificate by the root subnet does not have a delegation field. A certificate by other subnets include a delegation, which is itself a certificate that proves that the subnet is listed in the root subnet’s state tree (see <>), and reveals its public key. @@ -1636,7 +1693,8 @@ check_delegations(Delegation d) : public_bls_key = verify_cert(d.certificate) return lookup(["subnet",d.subnet_id,"public_key"],d.certificate) .... -where `root_public_key` is the a priori known root key of the Internet Computer instance. +where `root_public_key` is the a priori known root key. + [#certification-encoding] === Encoding of certificates @@ -1656,7 +1714,7 @@ The values in the <> are encoded to blobs as follows: === Example -Consider the following tree-shaped data (all single charachter strings denote labels, all other denote values) +Consider the following tree-shaped data (all single character strings denote labels, all other denote values) .... ─┬╴ "a" ─┬─ "x" ─╴"hello" │ └╴ "y" ─╴"world" @@ -1735,7 +1793,7 @@ The design of this abstract specification (e.g. how and where pending messages a === Notation -We specify the behavior of the system using ad hoc pseudocode. +We specify the behavior of the Internet Computer using ad-hoc pseudocode. The manipulated values are primitive values (numbers, text, binary blobs), aggregate values (lists, unordered lists a.k.a. bags, partial maps, records with fixed fields, named constructors) and functions. @@ -1743,19 +1801,19 @@ We use a concatenation operator `·` with various types: to extend sets and maps The shape of values is described using a hand-wavy type system. We use `Foo = Nat` to define type aliases; now `Foo` can be used instead of `Nat`. Often, the right-hand side is a more complex type here, e.g. a record, or multiple possible types separated by a vertical bar (`|`). Partial maps are written as `Key ↦ Value` and the function type as `Argument -> Result`. -NOTE: All values are immutable! State change is specified by describing the new state, not by changing existing state. +NOTE: All values are immutable! State change is specified by describing the new state, not by changing the existing state. Record fields are accessed using dot-notation (e.g. `S.request_id > 0`). To create a new record from an existing record `R` with some fields changed, the syntax `R where field = new_value` is used. This syntax can also be used to create new records with some deeply nested field changed: `R where some_map[key].field = new_value`. -In the state transitions, upper-case variables (`S`, `C`, `Req_id`) are free variables: The state transition may be taken for any possible value of these variables. `S` always refers to the state of the system before. A state transition often comes with a list of _conditions_, which may restrict the values of these free variables. The _state after_ is usually described using the record update syntax by starting with `S where`. +In the state transitions, upper-case variables (`S`, `C`, `Req_id`) are free variables: The state transition may be taken for any possible value of these variables. `S` always refers to the previous state. A state transition often comes with a list of _conditions_, which may restrict the values of these free variables. The _state after_ is usually described using the record update syntax by starting with `S where`. -For example, the condition `S.messages = Older_messages · M · Younger_messages` says that `M` is some message in field `messages` of the record `S`, and that `Younger_messages` and `Older_messages` are the other messages in the system. If the “state after” specifies `S with messages = Older_messages · Younger_messages`, then the message `M` is removed from the state. +For example, the condition `S.messages = Older_messages · M · Younger_messages` says that `M` is some message in field `messages` of the record `S`, and that `Younger_messages` and `Older_messages` are the other messages in the state. If the “state after” specifies `S with messages = Older_messages · Younger_messages`, then the message `M` is removed from the state. === Abstract state -In this specification, we describe the Internet Computer as a state machine. In particular, there is a single piece of data that describes the complete state of the system, called `S`. +In this specification, we describe the Internet Computer as a state machine. In particular, there is a single piece of data that describes the complete state of the IC, called `S`. -Of course, this is a huge simplification: The real Internet Computer is distributed and has a multi-component architecture, and the state is spread over many different components, some physically separated. But this simplification allows us to have a concise description of the system, and to easily make global decisions (such as, “is there any pending message”), without having to specify the bookkeeping that allows such global decision. +Of course, this is a huge simplification: The real Internet Computer is distributed and has a multi-component architecture, and the state is spread over many different components, some physically separated. But this simplification allows us to have a concise description of the behavior, and to easily make global decisions (such as, “is there any pending message”), without having to specify the bookkeeping that allows such global decisions. ==== Identifiers @@ -1852,6 +1910,7 @@ CanisterModule = { post_upgrade : (CanisterId, StableMemory, Arg, Env) -> Trap | Return WasmState update_methods : MethodName ↦ ((Arg, Env, AvailableCycles) -> UpdateFunc) query_methods : MethodName ↦ ((Arg, Env) -> QueryFunc) + heartbeat : (Env) -> Trap | Return NoResponse callbacks : (Callback, Response, RefundedCycles, Env, AvailableCycles) -> UpdateFunc inspect_message : (MethodName, WasmState, Arg, Env) -> Trap | Return (Accept | Reject) } @@ -1875,7 +1934,7 @@ The concrete mapping of this abstract `CanisterModule` to actual WebAssembly con The Internet Computer provides certain messaging guarantees: If a user or a canister calls another canister, it will eventually get a single response (a reply or a rejection), even if some canister code along the way fails. -To ensure that only one response is generated, and also to detect when no response can be generated any more, the system maintains a _call context_. The `responded` field is set to `true` once the call has received a response. Further attempts to respond will now fail. +To ensure that only one response is generated, and also to detect when no response can be generated any more, the IC maintains a _call context_. The `responded` field is set to `true` once the call has received a response. Further attempts to respond will now fail. .... CallCtxt = { @@ -1894,18 +1953,20 @@ CallOrigin calling_context : CallId; callback: Callback } + | FromHeartbeat .... ==== Calls and Messages -Calls into and within the Internet Computer are implemented as messages passed between canisters. During their lifetime, messages change shape: they begin as a call to a public method, which is resolved to a WebAssembly function that is then executed, potentially generating a response which is then delivered. +Calls into and within the IC are implemented as messages passed between canisters. During their lifetime, messages change shape: they begin as a call to a public method, which is resolved to a WebAssembly function that is then executed, potentially generating a response which is then delivered. Therefore, a message can have different shapes: .... -Queue = Unordered | Queue { from : CanisterId; to : CanisterId } +Queue = Unordered | Queue { from : System | CanisterId; to : CanisterId } EntryPoint = PublicMethod MethodName Principal Blob | Callback Callback Response RefundedCycles + | Heartbeat Message = CallMessage { @@ -1936,7 +1997,7 @@ A reference implementation would likely maintain a separate list of `messages` f ==== API requests -We distinguish between the _asynchronous_ API requests passed to `/api/v2/…/call`, which may be present in the system state, and the _synchronous_ API requests passed to `/api/v2/…/read_state` and `/api/v2/…/query`, which are only ephemeral. +We distinguish between the _asynchronous_ API requests passed to `/api/v2/…/call`, which may be present in the IC state, and the _synchronous_ API requests passed to `/api/v2/…/read_state` and `/api/v2/…/query`, which are only ephemeral. .... Envelope = { @@ -1994,7 +2055,7 @@ hash_of_map: Request -> Request .... For the signatures in a `Request`, we assume that the following function implements signature verification as described in <>. -This function picks the corresponding signature scheme according to the on the DER-encoded metadata in the public key. +This function picks the corresponding signature scheme according to the DER-encoded metadata in the public key. .... PublicKey = Blob Signature = Blob @@ -2015,7 +2076,7 @@ SignedDelegation = { ==== The system state -Finally, we can describe the state of the Internet Computer as a record having the following fields: +Finally, we can describe the state of the IC as a record having the following fields: .... S = { @@ -2046,7 +2107,7 @@ CanStatus ==== Initial state -The initial state of the system is +The initial state of the IC is .... { requests = (); @@ -2093,11 +2154,11 @@ The following is an incomplete list of invariants that should hold for the abstr === State transitions -Based on this abstract notion of the state, we can describe the behavior of the system. There are three classes of behaviors: +Based on this abstract notion of the state, we can describe the behavior of the IC. There are three classes of behaviors: * Asynchronous API requests that are submitted via `/api/v2/…/call`. These transitions describe checks that the request must pass to be considered received. - * Spontaneous transitions that model the internal behavior of the system, by describing conditions on the state that allow the transition to happen, and the state after. - * Responses to reads (i.e. `/api/v2/…/read_state`). By definition, these do _not_ change the state of the system, and merely describe the response based on the read request and the current system state. + * Spontaneous transitions that model the internal behavior of the IC, by describing conditions on the state that allow the transition to happen, and the state after. + * Responses to reads (i.e. `/api/v2/…/read_state`). By definition, these do _not_ change the state of the IC, and merely describe the response based on the read request and the current state. The state transitions are not complete with regard to error handling. For example, the behavior of sending a request to a non-existent canister is not specified here. For now, we trust implementors to make sensible decisions there. @@ -2133,7 +2194,7 @@ delegation_targets(DS) ==== Effective canister ids -A `Request` has an effective canister id accoridng to the rules in <<#http-effective-canister-id>>: +A `Request` has an effective canister id according to the rules in <<#http-effective-canister-id>>: .... is_effective_canister_id(CanisterUpdateCall {canister_id = ic_principal, arg = candid({canister_id = p, …}), …}, p) @@ -2141,26 +2202,16 @@ is_effective_canister_id(CanisterUpdateCall {canister_id = ic_principal, method is_effective_canister_id(CanisterUpdateCall {canister_id = p, …}, p), if p ≠ ic_principal .... -=== State transitions - -Based on this abstract notion of the state, we can describe the behavior of the system. There are three classes of behaviors: - - * Asynchronous API requests that are submitted via `/api/v2/…/call`. These transitions describes checks that the request must pass to be considered received. - * Spontaneous transitions that model the internal behavior of the system, by describing conditions on the state that allow the transition to happen, and the state after. - * Responses to synchronous requests (i.e. `/api/v2/…/query`). By definition, these do _not_ change the state of the system, and merely describe the response based on the read request and the current system state. - -The state transitions are not complete with regard to error handling. For example, the behavior of sending a request to a non-existent canister is not specified here. For now, we trust implementors to make sensible decisions there. - ==== API Request submission -After a node accepts a request via `/api/v2/canister//call`, it gets added to the system in the `Received` state. +After a node accepts a request via `/api/v2/canister//call`, the request gets added to the IC state as `Received`. This may only happen if the signature is valid and is created with a correct key. Due to this check, the envelope is discarded after this point. Requests that have expired are dropped here. -Ingress message inspection is applied, and message that are not accepted by the canister are dropped. +Ingress message inspection is applied, and messages that are not accepted by the canister are dropped. Submitted request:: `E : Envelope` Conditions:: @@ -2189,11 +2240,11 @@ S with requests[E.content] = Received .... -NOTE: This is not instantaneous (the system takes some time to agree it accepts the request) nor guaranteed (a node could just drop the request, or maybe it did not pass validation). But once the request has entered the system like this, it will be acted upon. +NOTE: This is not instantaneous (the IC takes some time to agree it accepts the request) nor guaranteed (a node could just drop the request, or maybe it did not pass validation). But once the request has entered the IC state like this, it will be acted upon. ==== Request rejection -The system may reject a received message for internal reasons (high load, low resources) or expiry. The precise conditions are not specified here, but the reject code must indicate this to be a system error. +The IC may reject a received message for internal reasons (high load, low resources) or expiry. The precise conditions are not specified here, but the reject code must indicate this to be a system error. Conditions:: .... @@ -2210,9 +2261,9 @@ S with A first step in processing a canister update call is to create a `CallMessage` in the message queue. -The `request` field of the `FromUser` origin establishes the connection to the api message. One could use the corresponding `hash_of_map` for this purpose, but this formulation is more abstract. +The `request` field of the `FromUser` origin establishes the connection to the API message. One could use the corresponding `hash_of_map` for this purpose, but this formulation is more abstract. -We do not make any guarantees about the order of incoming messages. +The IC does not make any guarantees about the order of incoming messages. Conditions:: .... @@ -2241,7 +2292,7 @@ S with A call to a canister which is stopping, stopped, or frozen is automatically rejected. -The (unspecified) function `freezing_limit(S, cid)` determines the freezing threshold in cycles of the canister with id `cid`, given it’s currrent memory footprint, storage cost, memory and compute allocation, and current `freezing_threshold` setting. +The (unspecified) function `freezing_limit(S, cid)` determines the freezing threshold in cycles of the canister with id `cid`, given its current memory footprint, storage cost, memory and compute allocation, and current `freezing_threshold` setting. Conditions:: .... @@ -2263,18 +2314,20 @@ State after:: ==== Call context creation -Before invoking a message to a public entry point, some bookkeeping is required: - - * A call context is created, and the method is looked up in the list of exports. This happens for both ingress and inter-canister messages. -The canister must be running (so not stopped, or stopping). - -The position of the message in the queue is unchanged. - -This only happens for “real” canisters, not the IC management canister. +Before invoking a heartbeat or a message to a public entry point, a call context is created for bookkeeping purposes. +For these invocations the canister must be running (so not stopped, or stopping). +Additionally, these invocations only happen for "real" canisters, not the IC management canister. This “bookkeeping transition” must be immediately followed by the corresponding <>. +* Call context creation: Public entry points ++ +For a message to a public entry point, the method is looked up in the list of exports. This happens for both ingress and inter-canister messages. ++ +The position of the message in the queue is unchanged. ++ Conditions:: ++ .... S.messages = Older_messages · CallMessage CM · Younger_messages S.canisters[CM.callee] ≠ EmptyCanister @@ -2282,7 +2335,9 @@ Conditions:: balances[CM.callee] ≥ freezing_limit(S, CM.callee) + MAX_CYCLES_PER_MESSAGE Ctxt_id ∉ dom S.call_contexts .... ++ State after:: ++ .... S with messages = @@ -2304,7 +2359,42 @@ S with balances[CM.callee] = balances[CM.callee] - MAX_CYCLES_PER_MESSAGE .... -We can execute any message that is at the head of its queue, i.e. there is no older message with the same abstract `queue` field. The actual message execution, if successful, may enqueue further messages and -- if the function returns a response -- record this response. The new call and response messages are enqueued at the end. +* Call context creation: Heartbeat ++ +If canister `C` exports a method with name `canister_heartbeat`, the IC will add a message to the head of the queue for invoking `canister_heartbeat` and create the corresponding call context. ++ +Conditions:: ++ +.... + S.canisters[C] ≠ EmptyCanister + S.canister_status[C] = Running + balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE + Ctxt_id ∉ dom S.call_contexts +.... ++ +State after:: ++ +.... +S with + messages = + FuncMessage { + call_context = Ctxt_id; + receiver = C; + entry_point = Heartbeat; + queue = Queue { from = System; to = C }; + } + · S.messages + call_contexts[Ctxt_id] = { + canister = C; + origin = FromHeartbeat; + responded = false; + deleted = false; + available_cycles = 0; + } + balances[C] = balances[C] - MAX_CYCLES_PER_MESSAGE +.... + +The IC can execute any message that is at the head of its queue, i.e. there is no older message with the same abstract `queue` field. The actual message execution, if successful, may enqueue further messages and -- if the function returns a response -- record this response. The new call and response messages are enqueued at the end. Note that new messages are executed only if the canister is Running and is not frozen. @@ -2341,6 +2431,10 @@ Conditions:: ( M.entry_point = Callback Callback Response F = Mod.callbacks(Callback, Response, Env, Available) ) + or + ( M.entry_point = Heartbeat + F = Mod.heartbeat(Env) + ) R = F(S.canisters[M.receiver].wasm_state) .... @@ -2409,7 +2503,7 @@ The cycle consumption of executing this message is modeled via the unspecified ` Depending whether this is a call message and a response messages, we have either set aside `MAX_CYCLES_PER_MESSAGE` or `MAX_CYCLES_PER_RESPONSE`, either in the call context creation rule or the Callback invocation rule. -This transition detects certain behaviour that will appear as a trap (and which an implementation may implement by trapping directly in a system call): +This transition detects certain behavior that will appear as a trap (and which an implementation may implement by trapping directly in a system call): * Responding if the present call context has already been responded to * Accepting more cycles than are available on the call context @@ -2420,8 +2514,8 @@ If message execution < If message execution <>, the state is updated and possible outbound calls and responses are enqueued. -Note that returning does _not_ imply that the call associated with this message now _succeeds_ in the sense defined in <>; that would requre a (unique) call to `ic0.reply`. -Note also that the state changes are persisted even when the system is set to synthesize a <> reject immediately afterward (which happens when this returns without calling `ic0.reply` or `ic0.reject`, the corresponding call has not been responded to and there are no outstanding callbacks, see <>). +Note that returning does _not_ imply that the call associated with this message now _succeeds_ in the sense defined in <>; that would require a (unique) call to `ic0.reply`. +Note also that the state changes are persisted even when the IC is set to synthesize a <> reject immediately afterward (which happens when this returns without calling `ic0.reply` or `ic0.reject`, the corresponding call has not been responded to and there are no outstanding callbacks, see <>). The function `as_update` turns a query function into an update function, this is merely a notational trick to simplify the rule .... @@ -2441,11 +2535,12 @@ Note that by construction, a query function will either trap or return with a re [#rule-starvation] ==== Call context starvation -If there is no call, downstream calling context or response that could possibly fulfill a calling context, then a reject is synthesized. The error message below is _not_ indicative. In particular, if the system has an idea about _why_ this starved, it can put that in there (e.g. the initial message handler trapped with an out-of-memory access). +If the call context is not for heartbeat and there is no call, downstream calling context or response that could possibly fulfill a calling context, then a reject is synthesized. The error message below is _not_ indicative. In particular, if the IC has an idea about _why_ this starved, it can put that in there (e.g. the initial message handler trapped with an out-of-memory access). Conditions:: .... S.call_contexts[Ctxt_id].responded = false + S.call_contexts[Ctxt_id].origin ≠ FromHeartbeat ∀ CallMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id ∀ ResponseMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id ∀ ctxt_ids. @@ -2468,11 +2563,17 @@ S with ==== Call context removal -If there is no call, downstream calling context or response that references a call context, and the call context has been replied to, then the call context can be removed. +If there is no call, downstream calling context, or response that references a call context, and the call context has been replied to or the call context corresponds to a heartbeat that had already been executed, then the call context can be removed. Conditions:: .... - S.call_contexts[Ctxt_id].responded = true + ( + S.call_contexts[Ctxt_id].responded = true + ) or + ( + S.call_contexts[Ctxt_id].origin = FromHeartbeat + ∀ FuncMessage M ∈ S.messages. M.call_context ≠ Ctxt_id + ) ∀ CallMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id ∀ ResponseMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id ∀ ctxt_ids. @@ -2488,7 +2589,7 @@ S with ==== IC Management Canister: Canister creation -The system chooses an appropriate canister id and instantiates a new (empty) canister identified by this id. The _controllers_ are set such that the sender of this request is the only controller, unless the `settings` say otherwise. All cycles on this call are now the canister's initial cycles. +The IC chooses an appropriate canister id and instantiates a new (empty) canister identified by this id. The _controllers_ are set such that the sender of this request is the only controller, unless the `settings` say otherwise. All cycles on this call are now the canister's initial cycles. This is also when the System Time of the new canister starts ticking. @@ -2604,7 +2705,7 @@ S with ==== IC Management Canister: Code installation -Only the controllers of the given canister can install code. This transition installs new code over a canister. This involves invoking the `canister_init` system method (see <>), which must succeed. +Only the controllers of the given canister can install code. This transition installs new code over a canister. This involves invoking the `canister_init` method (see <>), which must succeed. Conditions:: .... @@ -2645,7 +2746,7 @@ S with ==== IC Management Canister: Code upgrade -Only the controllers of the given canister can install new code. This changes the code of an _existing_ canister, preserving the state in the stable memory. This involves invoking the `canister_pre_upgrade` system method on the old and `canister_post_upgrade` system method on the new canister, which must succeed and must not invoke other methods. +Only the controllers of the given canister can install new code. This changes the code of an _existing_ canister, preserving the state in the stable memory. This involves invoking the `canister_pre_upgrade` method on the old and `canister_post_upgrade` method on the new canister, which must succeed and must not invoke other methods. Conditions:: .... @@ -2757,7 +2858,7 @@ S with S.status[A.canister_id] = Stopping [(M.origin, M.transferred_cycles)] .... -The next two transition record any additional 'stop_canister' requests that arrive at a stopping (or stopped) canister in its status. +The next two transitions record any additional 'stop_canister' requests that arrive at a stopping (or stopped) canister in its status. Conditions:: .... @@ -3080,7 +3181,7 @@ NB: The refunded cycles, `RM.refunded_cycles` are, by construction, empty. ==== Request clean up -The system will keep the data for a completed or rejected request around for a certain, implementation defined amount of time, to allow clients to poll for the data. After that time, the data of the request will be dropped: +The IC will keep the data for a completed or rejected request around for a certain, implementation defined amount of time, to allow users to poll for the data. After that time, the data of the request will be dropped: Conditions:: .... @@ -3093,7 +3194,7 @@ S with .... -At the same or some later point, the request will be removed from memory of the system. This must happen no earlier than the ingress expiry time set in the request. +At the same or some later point, the request will be removed from the state of the IC. This must happen no earlier than the ingress expiry time set in the request. Conditions:: .... @@ -3106,7 +3207,7 @@ S with requests[M] = (deleted) .... -==== Canister out of cyles +==== Canister out of cycles Once a canister runs out of cycles, its code is uninstalled (cf. <>) and the allocations are set to zero (NB: allocations are currently not modeled in the formal model): @@ -3262,7 +3363,7 @@ The response is a certificate `cert`, as specified in <>, which p .... lookup(path, cert) = lookup_in_tree(path, state_tree(S)) .... -where `state_tree` constructs the a labeled tree from the system state `S` and the (so far underspecified) set of subnets `subnets`, as per <> +where `state_tree` constructs the a labeled tree from the IC state `S` and the (so far underspecified) set of subnets `subnets`, as per <> .... state_tree(S) = { "time": S.system_time; @@ -3295,7 +3396,7 @@ In Section <> we introduced an abstraction over the interfac ==== The concrete `WasmState` -The abstract `WasmState` above models the WebAssembly _store_ `S`, which encompasses the functions, tables, memories and globals of the WebAssembly program, plus additional data maintained by the system, such as the stable memory: +The abstract `WasmState` above models the WebAssembly _store_ `S`, which encompasses the functions, tables, memories and globals of the WebAssembly program, plus additional data maintained by the IC, such as the stable memory: .... WasmState = { store : S; // a store as per WebAssembly spec @@ -3507,6 +3608,22 @@ This formulation checks afterwards that the system call `ic0.call_perform` was n + By construction, the (possibly modified) `es.wasm_state` is discarded. +* The partial map `heartbeat` of the `CanisterModule` is defined if the WebAssembly program exports a function `func` named `canister_heartbeat`, and has value ++ +.... +heartbeat = λ (sysenv) → λ wasm_state → + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = empty_params with { data = NoData; caller = NoCaller; sysenv } + balance = sysenv.balance + } + try func() with Trap then Trap + if es.cycles_accepted ≠ 0 then Trap + if es.ingress_filter ≠ Reject then Trap + if es.response ≠ NoResponse then Trap + Return NoResponse; +.... + * The function `callbacks` of the `CanisterModule` is defined as follows + .... @@ -3610,7 +3727,7 @@ copy_from_canister(src : i32, size : i32) blob = Upon _instantiation_ of the WebAssembly module, we can provide the following functions as imports. -The pseudo-code below does _not_ explicitly enforce the restrictions of which imports are available in which contexts; for that the table in <> is authorative, and is assumed to be part of the implementation. +The pseudo-code below does _not_ explicitly enforce the restrictions of which imports are available in which contexts; for that the table in <> is authoritative, and is assumed to be part of the implementation. .... ic0.msg_arg_data_size() : i32 = @@ -3761,24 +3878,53 @@ discard_pending_call() = es.pending_call := NoPendingCall ic0.stable_size() : (page_count : i32) = - return |es.wasm_state.stable_mem| / 64k + if |es.wasm_state.store.mem| > 2^32 then Trap + page_count := |es.wasm_state.stable_mem| / 64k + return page_count ic0.stable_grow(new_pages : i32) : (old_page_count : i32) = - if arbitrary() - then return -1 + if |es.wasm_state.store.mem| > 2^32 then Trap + if arbitrary() then return -1 else old_size := |es.wasm_state.stable_mem| / 64k + if old_size + new_pages > 2^16 then return -1 es.wasm_state.stable_mem := es.wasm_state.stable_mem · repeat(0x00, new_pages * 64k) return old_size ic0.stable_write(offset : i32, src : i32, size : i32) + if |es.wasm_state.store.mem| > 2^32 then Trap if src+size > |es.wasm_state.store.mem| then Trap if offset+size > |es.wasm_state.stable_mem| then Trap es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] ic0.stable_read(dst : i32, offset : i32, size : i32) + if |es.wasm_state.store.mem| > 2^32 then Trap + if offset+size > |es.wasm_state.stable_mem| then Trap + if dst+size > |es.wasm_state.store.mem| then Trap + + es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] + +ic0.stable64_size() : (page_count : i64) = + return |es.wasm_state.stable_mem| / 64k + +ic0.stable64_grow(new_pages : i64) : (old_page_count : i64) = + if arbitrary() + then return -1 + else + old_size := |es.wasm_state.stable_mem| / 64k + es.wasm_state.stable_mem := + es.wasm_state.stable_mem · repeat(0x00, new_pages * 64k) + return old_size + +ic0.stable64_write(offset : i64, src : i64, size : i64) + if src+size > |es.wasm_state.store.mem| then Trap + if offset+size > |es.wasm_state.stable_mem| then Trap + + es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] + +ic0.stable64_read(dst : i64, offset : i64, size : i64) if offset+size > |es.wasm_state.stable_mem| then Trap if dst+size > |es.wasm_state.store.mem| then Trap diff --git a/universal-canister/.envrc b/universal-canister/.envrc deleted file mode 100644 index be81feddb..000000000 --- a/universal-canister/.envrc +++ /dev/null @@ -1 +0,0 @@ -eval "$(lorri direnv)" \ No newline at end of file diff --git a/universal-canister/.gitignore b/universal-canister/.gitignore deleted file mode 100644 index eb5a316cb..000000000 --- a/universal-canister/.gitignore +++ /dev/null @@ -1 +0,0 @@ -target diff --git a/universal-canister/Cargo.lock b/universal-canister/Cargo.lock deleted file mode 100644 index 9c10486df..000000000 --- a/universal-canister/Cargo.lock +++ /dev/null @@ -1,69 +0,0 @@ -# This file is automatically @generated by Cargo. -# It is not intended for manual editing. -[[package]] -name = "cfg-if" -version = "0.1.10" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "lazy_static" -version = "1.4.0" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "libc" -version = "0.2.67" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "memory_units" -version = "0.4.0" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "universal_canister" -version = "0.1.0" -dependencies = [ - "lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "wee_alloc 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)", -] - -[[package]] -name = "wee_alloc" -version = "0.4.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)", - "libc 0.2.67 (registry+https://github.com/rust-lang/crates.io-index)", - "memory_units 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)", -] - -[[package]] -name = "winapi" -version = "0.3.8" -source = "registry+https://github.com/rust-lang/crates.io-index" -dependencies = [ - "winapi-i686-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", - "winapi-x86_64-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", -] - -[[package]] -name = "winapi-i686-pc-windows-gnu" -version = "0.4.0" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[[package]] -name = "winapi-x86_64-pc-windows-gnu" -version = "0.4.0" -source = "registry+https://github.com/rust-lang/crates.io-index" - -[metadata] -"checksum cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)" = "4785bdd1c96b2a846b2bd7cc02e86b6b3dbf14e7e53446c4f54c92a361040822" -"checksum lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "e2abad23fbc42b3700f2f279844dc832adb2b2eb069b2df918f455c4e18cc646" -"checksum libc 0.2.67 (registry+https://github.com/rust-lang/crates.io-index)" = "eb147597cdf94ed43ab7a9038716637d2d1bf2bc571da995d0028dec06bd3018" -"checksum memory_units 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "8452105ba047068f40ff7093dd1d9da90898e63dd61736462e9cdda6a90ad3c3" -"checksum wee_alloc 0.4.5 (registry+https://github.com/rust-lang/crates.io-index)" = "dbb3b5a6b2bb17cb6ad44a2e68a43e8d2722c997da10e928665c72ec6c0a0b8e" -"checksum winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)" = "8093091eeb260906a183e6ae1abdba2ef5ef2257a21801128899c3fc699229c6" -"checksum winapi-i686-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6" -"checksum winapi-x86_64-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f" diff --git a/universal-canister/Cargo.toml b/universal-canister/Cargo.toml deleted file mode 100644 index 14f0925d5..000000000 --- a/universal-canister/Cargo.toml +++ /dev/null @@ -1,8 +0,0 @@ -[package] -name = "universal_canister" -version = "0.1.0" - -[dependencies] -wee_alloc = "0.4.3" -lazy_static = "1.4.0" - diff --git a/universal-canister/default.nix b/universal-canister/default.nix deleted file mode 100644 index 7895639f5..000000000 --- a/universal-canister/default.nix +++ /dev/null @@ -1,2 +0,0 @@ -{ system ? builtins.currentSystem }: -(import ../default.nix {inherit system;}).universal-canister diff --git a/universal-canister/shell.nix b/universal-canister/shell.nix deleted file mode 100644 index 7895639f5..000000000 --- a/universal-canister/shell.nix +++ /dev/null @@ -1,2 +0,0 @@ -{ system ? builtins.currentSystem }: -(import ../default.nix {inherit system;}).universal-canister diff --git a/universal-canister/src/api.rs b/universal-canister/src/api.rs deleted file mode 100644 index 983abb644..000000000 --- a/universal-canister/src/api.rs +++ /dev/null @@ -1,280 +0,0 @@ -// use `wee_alloc` as the global allocator. -extern crate wee_alloc; - -#[global_allocator] -static ALLOC: wee_alloc::WeeAlloc<'_> = wee_alloc::WeeAlloc::INIT; - -mod ic0 { - #[link(wasm_import_module = "ic0")] - extern "C" { - pub fn accept_message() -> (); - pub fn canister_cycle_balance() -> u64; - pub fn canister_self_copy(dst: u32, offset: u32, size: u32) -> (); - pub fn canister_self_size() -> u32; - pub fn canister_status() -> u32; - pub fn debug_print(offset: u32, size: u32) -> (); - pub fn msg_arg_data_copy(dst: u32, offset: u32, size: u32) -> (); - pub fn msg_arg_data_size() -> u32; - pub fn msg_caller_copy(dst: u32, offset: u32, size: u32) -> (); - pub fn msg_caller_size() -> u32; - pub fn msg_cycles_accept( max_amount : u64 ) -> u64; - pub fn msg_cycles_available() -> u64; - pub fn msg_cycles_refunded() -> u64; - pub fn msg_method_name_copy(dst: u32, offset: u32, size: u32) -> (); - pub fn msg_method_name_size() -> u32; - pub fn msg_reject_code() -> u32; - pub fn msg_reject_msg_copy(dst: u32, offset: u32, size: u32) -> (); - pub fn msg_reject_msg_size() -> u32; - pub fn msg_reject(src: u32, size: u32) -> (); - pub fn msg_reply() -> (); - pub fn msg_reply_data_append(offset: u32, size: u32) -> (); - pub fn trap(offset: u32, size: u32) -> !; - pub fn call_new( - callee_src: u32, - callee_size: u32, - name_src: u32, - name_size: u32, - reply_fun : u32, - reply_env : u32, - reject_fun : u32, - reject_env : u32, - ) -> (); - pub fn call_on_cleanup( fun: u32, env: u32 ) -> (); - pub fn call_data_append( src: u32, size: u32 ) -> (); - pub fn call_cycles_add( amount : u64 ) -> (); - pub fn call_perform() -> u32; - pub fn stable_size() -> u32; - pub fn stable_grow(additional_pages: u32) -> u32; - pub fn stable_read(dst: u32, offset: u32, size: u32) -> (); - pub fn stable_write(offset: u32, src: u32, size: u32) -> (); - pub fn certified_data_set(src: u32, size: u32) -> (); - pub fn data_certificate_present() -> u32; - pub fn data_certificate_size() -> u32; - pub fn data_certificate_copy(dst: u32, offset: u32, size: u32) -> (); - - pub fn time() -> u64; - } -} - -// Convenience wrappers around the DFINTY System API - -pub fn call_new ( - callee: &[u8], - method: &[u8], - reply_fun: fn(u32) -> (), reply_env: u32, - reject_fun: fn(u32) -> (), reject_env: u32, -) { - unsafe { - ic0::call_new( - callee.as_ptr() as u32, - callee.len() as u32, - method.as_ptr() as u32, - method.len() as u32, - reply_fun as u32, - reply_env, - reject_fun as u32, - reject_env, - ) - } -} - -pub fn call_on_cleanup(fun: fn(u32) -> (), env: u32) { - unsafe { - ic0::call_on_cleanup(fun as u32, env as u32) - } -} - -pub fn call_data_append(payload: &[u8]) { - unsafe { - ic0::call_data_append(payload.as_ptr() as u32, payload.len() as u32); - } -} - -pub fn call_cycles_add( amount : u64) { - unsafe { ic0::call_cycles_add(amount); } -} - -pub fn call_perform() -> u32 { - unsafe { - ic0::call_perform() - } -} - -/// Returns the argument extracted from the message payload. -pub fn arg_data() -> Vec { - let len: u32 = unsafe { ic0::msg_arg_data_size() }; - let mut bytes = vec![0; len as usize]; - unsafe { - ic0::msg_arg_data_copy(bytes.as_mut_ptr() as u32, 0, len); - } - bytes -} - -/// Returns the caller of the current call. -pub fn caller() -> Vec { - let len: u32 = unsafe { ic0::msg_caller_size() }; - let mut bytes = vec![0; len as usize]; - unsafe { - ic0::msg_caller_copy(bytes.as_mut_ptr() as u32, 0, len); - } - bytes -} - -/// Returns the canister id as a blob. -pub fn id() -> Vec { - let len: u32 = unsafe { ic0::canister_self_size() }; - let mut bytes = vec![0; len as usize]; - unsafe { - ic0::canister_self_copy(bytes.as_mut_ptr() as u32, 0, len); - } - bytes -} - -pub fn status() -> u32 { - unsafe { ic0::canister_status() } -} - -/// Returns the rejection message. -pub fn reject_message() -> Vec { - let len: u32 = unsafe { ic0::msg_reject_msg_size() }; - let mut bytes = vec![0; len as usize]; - unsafe { - ic0::msg_reject_msg_copy(bytes.as_mut_ptr() as u32, 0, len); - } - bytes -} - -pub fn reply_data_append(payload: &[u8]) { - unsafe { - ic0::msg_reply_data_append(payload.as_ptr() as u32, payload.len() as u32); - } -} - -pub fn reply() { - unsafe { - ic0::msg_reply(); - } -} - -/// Rejects the current call with the given message. -pub fn reject(err_message: &[u8]) { - unsafe { - ic0::msg_reject(err_message.as_ptr() as u32, err_message.len() as u32); - } -} - -pub fn reject_code() -> u32 { - unsafe { ic0::msg_reject_code() } -} - -pub fn cycles_available() -> u64 { - unsafe { ic0::msg_cycles_available() } -} - -pub fn cycles_refunded() -> u64 { - unsafe { ic0::msg_cycles_refunded() } -} - -pub fn accept( amount : u64) -> u64 { - unsafe { ic0::msg_cycles_accept(amount) } -} - -pub fn balance() -> u64 { - unsafe { ic0::canister_cycle_balance() } -} - -pub fn stable_size() -> u32 { - unsafe { ic0::stable_size() } -} - -pub fn stable_grow(additional_pages: u32) -> u32 { - unsafe { ic0::stable_grow(additional_pages) } -} - -pub fn stable_read(offset: u32, size: u32) -> Vec { - let mut bytes = vec![0; size as usize]; - unsafe { - ic0::stable_read(bytes.as_mut_ptr() as u32, offset, size); - } - bytes -} - -pub fn stable_write(offset: u32, data : &[u8]) { - unsafe { - ic0::stable_write(offset, data.as_ptr() as u32, data.len() as u32); - } -} - -pub fn certified_data_set(data : &[u8]) { - unsafe { - ic0::certified_data_set(data.as_ptr() as u32, data.len() as u32); - } -} - -pub fn data_certificate_present() -> u32 { - unsafe { ic0::data_certificate_present() } -} - -pub fn data_certificate() -> Vec { - let len: u32 = unsafe { ic0::data_certificate_size() }; - let mut bytes = vec![0; len as usize]; - unsafe { - ic0::data_certificate_copy(bytes.as_mut_ptr() as u32, 0, len); - } - bytes -} - -pub fn time() -> u64 { - unsafe { - ic0::time() - } -} - - -pub fn accept_message() { - unsafe { ic0::accept_message() } -} - -pub fn method_name() -> Vec { - let len: u32 = unsafe { ic0::msg_method_name_size() }; - let mut bytes = vec![0; len as usize]; - unsafe { - ic0::msg_method_name_copy(bytes.as_mut_ptr() as u32, 0, len); - } - bytes -} - -/// Prints the given message. -pub fn print(data : &[u8]) { - unsafe { - ic0::debug_print(data.as_ptr() as u32, data.len() as u32); - } -} - -pub fn bad_print() { - unsafe { - ic0::debug_print(u32::max_value()-2, 1); - ic0::debug_print(u32::max_value()-2, 3); - } -} - -/// Traps with the given message. -pub fn trap_with_blob(data: &[u8]) -> ! { - unsafe { - ic0::trap(data.as_ptr() as u32, data.len() as u32); - } -} -pub fn trap_with(message: &str) -> ! { - unsafe { - ic0::trap(message.as_ptr() as u32, message.len() as u32); - } -} - -use std::panic; -pub fn set_panic_hook() { - panic::set_hook(Box::new(|i| { - let s = i.to_string(); - trap_with(&s); - })); -} - diff --git a/universal-canister/src/lib.rs b/universal-canister/src/lib.rs deleted file mode 100644 index e5fdf85ee..000000000 --- a/universal-canister/src/lib.rs +++ /dev/null @@ -1 +0,0 @@ -pub mod api; diff --git a/universal-canister/src/main.rs b/universal-canister/src/main.rs deleted file mode 100644 index d9dfd5171..000000000 --- a/universal-canister/src/main.rs +++ /dev/null @@ -1,384 +0,0 @@ -use std::convert::TryInto; - -mod api; - -// A simple dynamically typed stack - -enum Val { I32(u32), I64(u64), Blob(Vec) } - -struct Stack(Vec); - -impl Stack { - fn new() -> Self { - Stack(Vec::new()) - } - - fn drop(self : &mut Self) { - self.0.pop(); - } - - fn push_int(self : &mut Self, x : u32) { - self.0.push(Val::I32(x)); - } - - fn push_int64(self : &mut Self, x : u64) { - self.0.push(Val::I64(x)); - } - - fn push_blob(self : &mut Self, x : Vec) { - self.0.push(Val::Blob(x)); - } - - fn pop_int(self : &mut Self) -> u32 { - if let Some(Val::I32(i)) = self.0.pop() { - i - } else { - api::trap_with("did not find I32 on stack") - } - } - - fn pop_int64(self : &mut Self) -> u64 { - if let Some(Val::I64(i)) = self.0.pop() { - i - } else { - api::trap_with("did not find I64 on stack") - } - } - - fn pop_blob(self : &mut Self) -> Vec { - if let Some(Val::Blob(blob)) = self.0.pop() { - blob - } else { - api::trap_with("did not find blob on stack") - } - } -} - -// Reading data from the operations stream - -type Ops<'a> = &'a[u8]; - -fn read_bytes<'a>(ops : &mut Ops<'a>, len : usize) -> &'a[u8] { - if len < ops.len() { - let (bytes, rest) = ops.split_at(len as usize); - *ops = rest; - bytes - } else { - panic!("cannot read {} bytes of a {} byte string", len, ops.len()); - } -} - -fn read_int(ops : &mut Ops) -> u32 { - let bytes = read_bytes(ops, std::mem::size_of::()); - u32::from_le_bytes(bytes.try_into().unwrap()) -} - -fn read_int64(ops : &mut Ops) -> u64 { - let bytes = read_bytes(ops, std::mem::size_of::()); - u64::from_le_bytes(bytes.try_into().unwrap()) -} - -fn eval(ops : Ops) { - let mut ops : Ops = ops; - let mut stack : Stack = Stack::new(); - - while let Some((op,rest)) = ops.split_first() { - ops = rest; - match op { - // noop - 0 => (), - // drop - 1 => stack.drop(), - // push int - 2 => { - let a = read_int(&mut ops); - stack.push_int(a); - } - // push bytes - 3 => { - let len = read_int(&mut ops); - let blob = read_bytes(&mut ops, len as usize).to_vec(); - stack.push_blob(blob); - } - // reply_data_append - 4 => api::reply_data_append(&stack.pop_blob()), - // reply - 5 => api::reply(), - - // self - 6 => stack.push_blob(api::id()), - - // reject - 7 => api::reject(&stack.pop_blob()), - - // caller - 8 => stack.push_blob(api::caller()), - - // reject_msg - 10 => stack.push_blob(api::reject_message()), - - // reject_code - 11 => stack.push_int(api::reject_code()), - - // int to blob - 12 => { - let i = stack.pop_int(); - stack.push_blob(i.to_le_bytes().to_vec()) - } - - // msg_data - 13 => stack.push_blob(api::arg_data()), - - // concat - 14 => { - let mut b = stack.pop_blob(); - let mut a = stack.pop_blob(); - a.append(&mut b); - stack.push_blob(a); - } - - // stable memory - 15 => stack.push_int(api::stable_size()), - 16 => { - let i = stack.pop_int(); - stack.push_int(api::stable_grow(i)) - } - 17 => { - let size = stack.pop_int(); - let offset = stack.pop_int(); - stack.push_blob(api::stable_read(offset, size)) - } - 18 => { - let data = stack.pop_blob(); - let offset = stack.pop_int(); - api::stable_write(offset, &data) - } - - // debugging - 19 => api::print(&stack.pop_blob()), - 20 => api::trap_with_blob(&stack.pop_blob()), - - // some simple state - 21 => set_global(stack.pop_blob()), - 22 => stack.push_blob(get_global()), - - // bad print - 23 => api::bad_print(), - - // the pre-upgrade script - 24 => set_pre_upgrade(stack.pop_blob()), - - // int64 to blob - 25 => { - let i = stack.pop_int64(); - stack.push_blob(i.to_le_bytes().to_vec()) - } - - // time - 26 => stack.push_int64(api::time()), - - // available cycles - 27 => stack.push_int64(api::cycles_available()), - - // balance - 28 => stack.push_int64(api::balance()), - - // refunded - 29 => stack.push_int64(api::cycles_refunded()), - - // accept - 30 => { - let a = stack.pop_int64(); - stack.push_int64(api::accept(a)) - } - - // push int64 - 31 => { - let a = read_int64(&mut ops); - stack.push_int64(a); - } - - // call_new - 32 => { - // pop in reverse order! - let reject_code = stack.pop_blob(); - let reply_code = stack.pop_blob(); - let method = stack.pop_blob(); - let callee = stack.pop_blob(); - - let reject_env = add_callback(reject_code); - let reply_env = add_callback(reply_code); - - api::call_new(&callee, &method, callback, reply_env, callback, reject_env); - } - - // append arg - 33 => api::call_data_append(&stack.pop_blob()), - - // append cycles - 34 => api::call_cycles_add(stack.pop_int64()), - - // perform - 35 => { - let err_code = api::call_perform(); - if err_code != 0 { - api::trap_with("call_perform failed") - } - } - - // certified variables - 36 => api::certified_data_set(&stack.pop_blob()), - 37 => stack.push_int(api::data_certificate_present()), - 38 => stack.push_blob(api::data_certificate()), - - // canister_status - 39 => stack.push_int(api::status()), - - // accept_message - 40 => api::accept_message(), - - // the pre-upgrade script - 41 => set_inspect_message(stack.pop_blob()), - - 42 => stack.push_blob(api::method_name()), - - // trap if blob equal - 43 => { - let c = stack.pop_blob(); - let b = stack.pop_blob(); - let a = stack.pop_blob(); - if a == b { - api::trap_with_blob(&c) - } - } - - // on_cleanup - 44 => { - // pop in reverse order! - let cleanup_code = stack.pop_blob(); - let cleanup_env = add_callback(cleanup_code); - api::call_on_cleanup(callback, cleanup_env); - } - - - _ => api::trap_with(&format!("unknown op {}", op)), - } - } -} -#[export_name = "canister_update update"] -fn update() { - setup(); - eval(&api::arg_data()); -} - -#[export_name = "canister_query query"] -fn query() { - setup(); - eval(&api::arg_data()); -} - -#[export_name = "canister_init"] -fn init() { - setup(); - eval(&api::arg_data()); -} - -#[export_name = "canister_pre_upgrade"] -fn pre_upgrade() { - setup(); - eval(&get_pre_upgrade()); -} - -#[export_name = "canister_post_upgrade"] -fn post_upgrade() { - setup(); - eval(&api::arg_data()); -} - -#[export_name = "canister_inspect_message"] -fn inspect_message() { - setup(); - eval(&get_inspect_message()); -} - -/* A global variable */ -lazy_static! { - static ref GLOBAL : Mutex> = Mutex::new(Vec::new()); -} -fn set_global(data : Vec) { - *GLOBAL.lock().unwrap() = data; -} -fn get_global() -> Vec { - GLOBAL.lock().unwrap().clone() -} - -/* A variable to store what to execute upon pre_upgrade */ -lazy_static! { - static ref PRE_UPGRADE : Mutex> = Mutex::new(Vec::new()); -} -fn set_pre_upgrade(data : Vec) { - *PRE_UPGRADE.lock().unwrap() = data; -} -fn get_pre_upgrade() -> Vec { - PRE_UPGRADE.lock().unwrap().clone() -} - -/* A variable to store what to execute in canister_inspect_message */ -/* (By default allows all) */ -lazy_static! { - static ref INSPECT_MESSAGE: Mutex> = Mutex::new(vec![40]); -} -fn set_inspect_message(data: Vec) { - *INSPECT_MESSAGE.lock().unwrap() = data; -} -fn get_inspect_message() -> Vec { - INSPECT_MESSAGE.lock().unwrap().clone() -} - - -/* Callback handling */ - -#[macro_use] -extern crate lazy_static; -use std::sync::Mutex; -lazy_static! { - static ref CALLBACKS : Mutex>>> = Mutex::new(Vec::new()); -} - -fn add_callback(code : Vec) -> u32 { - let mut vec = CALLBACKS.lock().unwrap(); - vec.push(Some(code)); - return (vec.len() as u32) - 1; -} - -fn get_callback(idx : u32) -> Vec { - let mut vec = CALLBACKS.lock().unwrap(); - if let Some(entry) = vec.get_mut(idx as usize) { - if let Some(code) = entry.take() { - return code - } else { - panic!("get_callback: {} already taken", idx) - } - } else { - panic!("get_callback: {} out of bounds", idx) - } -} - -fn callback(env: u32) { - eval(&get_callback(env)); -} - - -/* Panic setup */ - -use std::sync::Once; - -static START: Once = Once::new(); - -fn setup() { - START.call_once(|| { - api::set_panic_hook(); - }); -} - -fn main() {} From 9c2f56cd57d352f61b4de8a0ce012b8cc124f537 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bj=C3=B6rn=20Tackmann?= <54846571+Dfinity-Bjoern@users.noreply.github.com> Date: Tue, 11 Jan 2022 11:09:09 +0100 Subject: [PATCH 004/102] Publish release 0.18.3 (#390) * Prepare release 0.18.3 * Remove redundant comment * Update spec/index.adoc Co-authored-by: Jens Groth <46536510+JensGroth@users.noreply.github.com> * Update spec/index.adoc Co-authored-by: Jens Groth <46536510+JensGroth@users.noreply.github.com> * Reformulation Co-authored-by: Jens Groth <46536510+JensGroth@users.noreply.github.com> --- spec/changelog.adoc | 6 ++ spec/index.adoc | 245 ++++++++++++++++++++++++++++++++++---------- 2 files changed, 199 insertions(+), 52 deletions(-) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 013012b81..d17e05ff2 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,12 @@ [#changelog] == Changelog +[#0_18_3] +=== 0.18.3 (2022-01-10) + +* Spec: New System API which uses 128-bit values to represent the amount of cycles +* Spec: Subnet delegations include a canister id scope + [#0_18_2] === 0.18.2 (2021-09-29) diff --git a/spec/index.adoc b/spec/index.adoc index 427f1bc88..dba852204 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,6 +1,6 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.2 +0.18.3 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ @@ -331,6 +331,8 @@ The `BIT STRING` field `subjectPublicKey` is the blob `|signing_canister_id| · where `signing_canister_id` is the id of the signing canister and `reconstruct` is a function that computes a root-hash for the tree. +- If the `certificate` includes subnet delegations (possibly nested), then the `signing_canister_id` must be included in each delegation’s canister id range (see <>). + - The `tree` must be a `well_formed` tree with lookup(/sig//, tree) = Found "" @@ -366,6 +368,20 @@ The state tree contains information about the topology of the Internet Computer. + The public key of the subnet (a DER-encoded BLS key, see <>) +* `/subnet//canister_ranges` (blob) ++ +The set of canister ids assigned to this subnet, represented as a list of closed intervals of canister ids, ordered lexicographically, and encoded as CBOR according to this CDDL: ++ +.... +canister_ranges = tagged<[*canister_range]> +canister_range = [principal principal] +principal = bytes .size (0..29) +tagged = #6.55799(t) ; the CBOR tag +.... ++ +NOTE: Because this uses the lexicographic ordering of princpials, and the byte distinguishing the various classes of ids is at the _end_, this range by construction conceptually includes principals of various classes. This specification needs to take care that the fact that principals that are not canisters may appear in these ranges does not cause confusion. + + [#state-tree-request-status] === Request status @@ -519,6 +535,8 @@ In order to read parts of the <>, the user makes a POST request to ` The HTTP response to this request consists of a CBOR map with the following fields: * `certificate` (`blob`): A certificate (see <>). ++ +If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation’s canister id range (see <>). The returned certificate reveals all values whose path is a suffix of the list of requested paths. It also always reveals `/time`, even if not explicitly requested. @@ -834,23 +852,30 @@ In order for a WebAssembly module to be usable as the code for the canister, it * If it exports any functions called `canister_update ` or `canister_query ` for some `name`, the functions must have type `+() -> ()+`. * It may not export both `canister_update ` and `canister_query ` with the same `name`. * It may not export other methods the names of which start with the prefix `canister_` besides the methods allowed above. -* The IC may reject WebAssembly modules that declare more than 6000 functions or more than 200 globals. +* It may not have both `icp:public ` and `icp:private ` with the same `name` as the custom section name. +* It may not have other custom sections the names of which start with the prefix `icp:` besides the `icp:public ` and `icp:private `. +* The IC may reject WebAssembly modules that + + declare more than 6000 functions, or + + declare more than 200 globals, or + + declare more than 16 exported custom sections (the custom section names with prefix `icp:`), or + + the total size of the exported custom sections exceeds 1MiB === Interpretation of numbers WebAssembly number types (`i32`, `i64`) do not indicate if the numbers are to be interpreted as signed or unsigned. Unless noted otherwise, whenever the System API interprets them as numbers (e.g. memory pointers, buffer offsets, array sizes), they are to be interpreted as unsigned. +[#entry-points] === Entry points The canister provides entry points which are invoked by the IC under various circumstances: -* The canister may export a function named `canister_init` and type `+() -> ()+`. -* The canister may export a function named `canister_pre_upgrade` and type `+() -> ()+`. -* The canister may export a function named `canister_post_upgrade` and type `+() -> ()+`. -* The canister may export functions named `canister_inspect_message` with type `+() -> ()+`. -* The canister may export a function named `canister_heartbeat` with type `+() -> ()+`. -* The canister may export functions named `canister_update ` and type `+() -> ()+`. -* The canister may export functions named `canister_query ` and type `+() -> ()+`. +* The canister may export a function with name `canister_init` and type `+() -> ()+`. +* The canister may export a function with name `canister_pre_upgrade` and type `+() -> ()+`. +* The canister may export a function with name `canister_post_upgrade` and type `+() -> ()+`. +* The canister may export functions with name `canister_inspect_message` with type `+() -> ()+`. +* The canister may export a function with name `canister_heartbeat` with type `+() -> ()+`. +* The canister may export functions with name `canister_update ` and type `+() -> ()+`. +* The canister may export functions with name `canister_query ` and type `+() -> ()+`. * The canister table may contain functions of type `+(env : i32) -> ()+` which may be used as callbacks for inter-canister calls. If the execution of any of these entry points traps for any reason, then all changes to the WebAssembly state, as well as the effect of any externally visible system call (like `ic0.msg_reply`, `ic0.msg_reject`, `ic0.call_perform`), are discarded. For upgrades, this transactional behavior applies to the `canister_pre_upgrade`/`canister_post_upgrade` sequence as a whole. @@ -891,7 +916,7 @@ Eventually, a method will want to send a response, using `ic0.reply` or `ic0.rej ==== Heartbeat -For periodic or time-based execution, the WebAssembly module can export a function with name `canister_heartbeat`. If present, the IC will invoke this function at regular intervals. The exact interval is implementation-defined. For each heartbeat invocation, the IC guarantees that the time, as returned by <>, is monotonically increasing. +For periodic or time-based execution, the WebAssembly module can export a function with name `canister_heartbeat`. The heartbeats scheduling algorithm is implementation-defined. `canister_heartbeat` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. @@ -923,12 +948,17 @@ ic0.msg_reply : () -> (); // U ic0.msg_reject : (src : i32, size : i32) -> (); // U Q Ry Rt ic0.msg_cycles_available : () -> i64; // U Rt Ry +ic0.msg_cycles_available128 : (dst : i32) -> (); // U Rt Ry ic0.msg_cycles_refunded : () -> i64; // Rt Ry -ic0.msg_cycles_accept : ( max_amount : i64 ) -> ( amount : i64 ); // U Rt Ry +ic0.msg_cycles_refunded128 : (dst : i32) -> (); // Rt Ry +ic0.msg_cycles_accept : (max_amount : i64) -> ( amount : i64 ); // U Rt Ry +ic0.msg_cycles_accept128 : (max_amount_high : i64, max_amount_low: i64, dst : i32) + -> (); // U Rt Ry ic0.canister_self_size : () -> i32; // * ic0.canister_self_copy : (dst : i32, offset : i32, size : i32) -> (); // * ic0.canister_cycle_balance : () -> i64; // * +ic0.canister_cycle_balance128 : (dst : i32) -> (); // * ic0.canister_status : () -> i32; // * ic0.msg_method_name_size : () -> i32 // F @@ -947,7 +977,8 @@ ic0.call_new : // U ) -> (); ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt H ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt H -ic0.call_cycles_add : ( amount : i64 ) -> (); // U Ry Rt H +ic0.call_cycles_add : (amount : i64) -> (); // U Ry Rt H +ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt H ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt H ic0.stable_size : () -> (page_count : i32); // * @@ -982,7 +1013,7 @@ The comment after each function lists from where these functions may be invoked: * `s`: the `(start)` module initialization function * `F`: from `canister_inspect_message` * `H`: from `canister_heartbeat` -* `*` = `I G U Q Ry Rt C F` (NB: Not `(start)`) +* `*` = `I G U Q Ry Rt C F H` (NB: Not `(start)`) If the canister invokes a system call from somewhere else, it will trap. @@ -1046,7 +1077,7 @@ Appends data it to the (initially empty) data reply. + NOTE: This can be invoked multiple times to build up the argument with data from various places on the Wasm heap. This way, the canister does not have to first copy all the pieces from various places into one location. + -This traps if the current call already has been responded to. +This traps if the current call already has been or does not need to be responded to. * `+ic0.msg_reply : () -> ()+` + @@ -1060,7 +1091,7 @@ See <> for how this interacts with cycles available on this c + Rejects the call. The data referred to by `src`/`size` is used for the diagnostic message. + -This system call traps if `src+size` exceeds the size of the WebAssembly memory, or if the current call already has been responded to, or if the data referred to by `src`/`size` is not valid UTF8. +This system call traps if `src+size` exceeds the size of the WebAssembly memory, or if the current call already has been or does not need to be responded to, or if the data referred to by `src`/`size` is not valid UTF8. + The other end will receive this reject with reject code `CANISTER_REJECT`, see <>. + @@ -1166,6 +1197,14 @@ This adds cycles onto a call. See <>. + This may be called multiple times between `ic0.call_new` and `ic0.call_perform`. + +* `+ic0.call_cycles_add128 : (amount_high : i64, amount_low : i64) -> ()+` ++ +This adds cycles onto a call. See <>. ++ +This may be called multiple times between `ic0.call_new` and `ic0.call_perform`. + + * `+ic0.call_perform : () -> ( err_code : i32 )+` + This concludes assembling the call. It queues the call message to the given destination, but does not actually act on it until the current WebAssembly function returns without trapping. @@ -1179,23 +1218,43 @@ After `ic0.call_perform` and before the next call to `ic0.call_new`, all other ` [#system-api-cycles] === Cycles -Each canister maintains a balance of _cycles_, the utility token used to pay for platform usage. +Each canister maintains a balance of _cycles_, which are used to pay for platform usage. Cycles are represented by 128-bit values. NOTE: This specification currently does not go into details about which actions cost how many cycles and/or when. In general, you must assume that the canister’s cycle balance can change arbitrarily between method executions, and during each System API function call, unless explicitly mentioned otherwise. * `ic0.canister_cycle_balance : () -> i64` + -indicates the current cycle balance of the canister. It is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add`. After execution of the message, the IC may add unused cycles from the reserve back to the balance. +Indicates the current cycle balance of the canister. It is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add`. After execution of the message, the IC may add unused cycles from the reserve back to the balance. + -NOTE: Should a future version of this specification allow `MAX_CANISTER_BALANCE` >= 2^64^, then this call will report 2^64^-1 if the balance exceeds 2^64^. A new, wider System API will then be provided for canisters that need to deal precisely with large canister balances. +NOTE: This call traps if the current balance does not fit into a 64-bit value. Canisters that need to deal with larger cycles balances should use `ic0.canister_cycles_balance128` instead. +* `ic0.canister_cycle_balance128 : (dst : i32) -> ()` ++ +Indicates the current cycle balance of the canister by copying the value at the location `dst` in the canister memory. +It is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add128`. +After execution of the message, the IC may add unused cycles from the reserve back to the balance. ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. ++ * `ic0.msg_cycles_available : () -> i64` + -returns the amount of cycles that were transferred by the caller of the current call, and is still available in this message. +Returns the amount of cycles that were transferred by the caller of the current call, and is still available in this message. ++ +Initially, in the update method entry point, this is the amount that the caller passed to the canister. When cycles are accepted (`ic0.msg_cycles_accept`), this reports fewer cycles accordingly. When the call is responded to (reply or reject), all available cycles are refunded to the caller, and this will return 0. + -Initially, in the update method entry point, this is the amount that the caller passed to the canister. When cycles are accepted (`ic0.msg_cycles_accept`), this reports less cycles accordingly. When the call is responded to (reply or reject), all available cycles are refunded to the caller, and this will return 0. +NOTE: This call traps if the amount of cycles available does not fit into a 64-bit value. Please use `ic0.msg_cycles_available128` instead. -* `ic0.msg_cycles_accept : ( max_amount : i64 ) -> ( amount : i64 )` +* `ic0.msg_cycles_available128 : (dst : i32) -> ()` ++ +Indicates the number of cycles transferred by the caller of the current call, still available in this message. +The amount of cycles is represented by a 128-bit value. This call copies this value starting at the location `dst` in the canister memory. ++ +Initially, in the update method entry point, this is the amount that the caller passed to the canister. When cycles are accepted (`ic0.msg_cycles_accept128`), this reports fewer cycles accordingly. When the call is responded to (reply or reject), all available cycles are refunded to the caller, and this will report 0 cycles. ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. ++ + +* `ic0.msg_cycles_accept : (max_amount : i64) -> (amount : i64)` + -- This moves cycles from the call to the canister balance. It moves as many cycles as possible, up to these constraints: @@ -1210,7 +1269,7 @@ It can be called multiple times, each time possibly adding more cycles to the ba The return value indicates how many cycles were actually moved. -This system call is deprecated and does not trap. +This system call does not trap. [TIP] ===== @@ -1218,17 +1277,56 @@ Example: To accept all cycles provided in a call, invoke `ic0.msg_cycles_accept( ===== -- -* `ic0.call_cycles_add : ( amount : i64 ) -> ()` +* `ic0.msg_cycles_accept128 : (max_amount_high : i64, max_amount_low : i64, dst : i32) -> ()` ++ +-- +This moves cycles from the call to the canister balance. It moves as many cycles as possible, up to these constraints: + +* It moves no more cycles than the amount obtained by combining `max_amount_high` and `max_amount_low`. Cycles are represented by 128-bit values. + +* It moves no more cycles than available according to `ic0.msg_cycles_available128`, and + +* The canister balance afterwards does not exceed `MAX_CANISTER_BALANCE` minus any possible outstanding balances. The precise logic of this limit is not yet specified here. + +It can be called multiple times, each time possibly adding more cycles to the balance. + +This call also copies the amount of cycles that were actually moved starting at the location `dst` in the canister memory. +-- ++ +This does not trap. ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. +* `ic0.call_cycles_add : (amount : i64) -> ()` + This function moves cycles from the canister balance onto the call under construction, to be transferred with that call. + The cycles are deducted from the balance as shown by `ic0.canister_cycle_balance` immediately, and moved back if the call cannot be performed (e.g. if `ic0.call_perform` signals an error, or if the canister invokes `ic0.call_new` or returns without calling `ic0.call_perform`). + +This system call traps if trying to transfer more cycles than are in the current balance of the canister. + +* `ic0.call_cycles_add128 : (amount_high : i64, amount_low : i64) -> ()` ++ +This function moves cycles from the canister balance onto the call under construction, to be transferred with that call. ++ +The amount of cycles it moves is represented by a 128-bit value which can be obtained by combining the `amount_high` and `amount_low` parameters. ++ +The cycles are deducted from the balance as shown by `ic0.canister_cycles_balance128` immediately, and moved back if the call cannot be performed (e.g. if `ic0.call_perform` signals an error, or if the canister invokes `ic0.call_new` or returns without calling `ic0.call_perform`). ++ This traps if trying to transfer more cycles than are in the current balance of the canister. ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. * `ic0.msg_cycles_refunded : () -> i64` + -this can only be used in a callback handler (reply or reject), and indicates the amount of cycles that came back with the response as a refund. The refund has already been added to the canister balance automatically. +This function can only be used in a callback handler (reply or reject), and indicates the amount of cycles that came back with the response as a refund. The refund has already been added to the canister balance automatically. + +NOTE: This call traps if the amount of cycles refunded does not fit into a 64-bit value. In general, it is recommended to use `ic0.msg_cycles_refunded128` instead. + +* `ic0.msg_cycles_refunded128 : (dst : i32) -> ()` ++ +This function can only be used in a callback handler (reply or reject), and indicates the amount of cycles that came back with the response as a refund. The refund has already been added to the canister balance automatically. ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. [#system-api-stable-memory] === Stable memory @@ -1343,6 +1441,8 @@ Copies the certificate for the current value of the certified data to the canist + The certificate is a blob as described in <> that contains the values at path `/canister//certified_data` and at path `/time` of <>. + +If this `certificate` includes subnet delegations (possibly nested), then the id of the current canister will be included in each delegation’s canister id range. ++ This traps if `ic0.data_certificate_present()` returns `0`. @@ -1454,7 +1554,7 @@ Only _controllers of the canister can install code. * If `mode = reinstall`, if the canister was not empty, its existing code and state is removed before proceeding as for `mode = install`. + -Note that this is different from `uninstall_code` followed by `install_code`, as that will forcibly reject all unresponded calls. +Note that this is different from `uninstall_code` followed by `install_code`, as that will forcibly reject all calls awaiting a response. * If `mode = upgrade`, this will perform an upgrade of a non-empty canister as described in <>, passing `arg` to the `canister_post_upgrade` method of the new instance. @@ -1531,7 +1631,7 @@ This method takes no input and returns 32 pseudo-random bytes to the caller. The As a provisional method on development instances, the `provisional_create_canister_with_cycles` method is provided. It behaves as `create_canister`, but initializes the canister’s balance with `amount` fresh cycles (using `MAX_CANISTER_BALANCE` if `amount = null`, else capping the balance at `MAX_CANISTER_BALANCE`). -Cycles added to this call via `ic0.call_cycles_add` are returned to the caller. +Cycles added to this call via `ic0.call_cycles_add128` are returned to the caller. This method is only available in local development instances. @@ -1540,7 +1640,7 @@ This method is only available in local development instances. As a provisional method on development instances, the `provisional_top_up_canister` method is provided. It adds `amount` cycles to the balance of canister identified by `amount` (implicitly capping it at `MAX_CANISTER_BALANCE`). -Cycles added to this call via `ic0.call_cycles_add` are returned to the caller. +Cycles added to this call via `ic0.call_cycles_add128` are returned to the caller. Any user can top-up any canister this way. @@ -1695,6 +1795,8 @@ check_delegations(Delegation d) : public_bls_key = .... where `root_public_key` is the a priori known root key. +Delegations are _scoped_, i.e., they indicate which set of canister principals the delegatee subnet may certify for. This set can be obtained from a delegation `d` using `lookup(["subnet",d.subnet_id,"canister_ranges"],d.certificate)`, which must be present, and is encoded as described in <>. The various applications of certificates describe if and how the subnet scope comes into play. + [#certification-encoding] === Encoding of certificates @@ -1934,13 +2036,13 @@ The concrete mapping of this abstract `CanisterModule` to actual WebAssembly con The Internet Computer provides certain messaging guarantees: If a user or a canister calls another canister, it will eventually get a single response (a reply or a rejection), even if some canister code along the way fails. -To ensure that only one response is generated, and also to detect when no response can be generated any more, the IC maintains a _call context_. The `responded` field is set to `true` once the call has received a response. Further attempts to respond will now fail. +To ensure that only one response is generated, and also to detect when no response can be generated any more, the IC maintains a _call context_. The `needs_to_respond` field is set to `false` once the call has received a response. Further attempts to respond will now fail. .... CallCtxt = { canister : CanisterId; origin : CallOrigin; - responded : bool; + needs_to_respond : bool; deleted : bool; available_cycles : Nat; } @@ -2128,18 +2230,18 @@ for some time stamp `T`, some DER-encoded BLS public key `PublicKey`, and using The following is an incomplete list of invariants that should hold for the abstract state `S`, and are not already covered by the type annotations in this section. -* Deleted call contexts have been responded to: +* Deleted call contexts were not awaiting a response: + .... ∀ Ctxt_id ↦ Ctxt ∈ S.call_contexts: - if Ctxt.deleted then Ctxt.responded + if Ctxt.deleted then Ctxt.needs_to_respond = false .... * Responded call contexts have no available_cycles left: + .... ∀ Ctxt_id ↦ Ctxt ∈ S.call_contexts: - if Ctxt.responded then Ctxt.available_cycles = 0 + if Ctxt.needs_to_respond = false then Ctxt.available_cycles = 0 .... * Referenced call contexts exists: @@ -2148,7 +2250,7 @@ The following is an incomplete list of invariants that should hold for the abstr ∀ CallMessage M ∈ S.messages. M.origin.calling_context ∈ S.call_contexts ∀ ResponseMessage M ∈ S.messages. M.origin.calling_context ∈ S.call_contexts ∀ _ ↦ Ctxt ∈ S.call_contexts: - if Ctx.responded = false: + if Ctx.needs_to_respond: Ctxt.origin.calling_context ∈ S.call_contexts .... @@ -2352,7 +2454,7 @@ S with call_contexts[Ctxt_id] = { canister = CM.callee; origin = CM.origin; - responded = false; + needs_to_respond = true; deleted = false; available_cycles = CM.transferred_cycles; } @@ -2361,7 +2463,7 @@ S with * Call context creation: Heartbeat + -If canister `C` exports a method with name `canister_heartbeat`, the IC will add a message to the head of the queue for invoking `canister_heartbeat` and create the corresponding call context. +If canister `C` exports a method with name `canister_heartbeat`, the IC will create the corresponding call context. + Conditions:: + @@ -2387,7 +2489,7 @@ S with call_contexts[Ctxt_id] = { canister = C; origin = FromHeartbeat; - responded = false; + needs_to_respond = false; deleted = false; available_cycles = 0; } @@ -2453,7 +2555,7 @@ if - ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ] , MAX_CANISTER_BALANCE) New_balance > if Is_response then 0 else freezing_limit(S, CM.callee); - (res.response = NoResponse) or (S.call_contexts[M.call_context].responded = false) + (res.response = NoResponse) or S.call_contexts[M.call_context].needs_to_respond then S with canisters[M.receiver].wasm_state = res.new_state; @@ -2483,7 +2585,7 @@ then if res.response = NoResponse: call_contexts[M.call_context].available_cycles = Available - res.cycles_accepted else - call_contexts[M.call_context].responded = true + call_contexts[M.call_context].needs_to_respond = false call_contexts[M.call_context].available_cycles = 0 if res.new_certified_data ≠ NoCertifiedData: @@ -2505,7 +2607,7 @@ Depending whether this is a call message and a response messages, we have either This transition detects certain behavior that will appear as a trap (and which an implementation may implement by trapping directly in a system call): - * Responding if the present call context has already been responded to + * Responding if the present call context does not need to be responded to * Accepting more cycles than are available on the call context * Sending out more cycles than available to the canister * Consuming more cycles than allowed (and reserved) @@ -2539,18 +2641,18 @@ If the call context is not for heartbeat and there is no call, downstream callin Conditions:: .... - S.call_contexts[Ctxt_id].responded = false + S.call_contexts[Ctxt_id].needs_to_respond = true S.call_contexts[Ctxt_id].origin ≠ FromHeartbeat ∀ CallMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id ∀ ResponseMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id ∀ ctxt_ids. - S.call_contexts[ctxt_ids].responded = false + S.call_contexts[ctxt_ids].needs_to_respond ==> S.call_contexts[ctxt_ids].origin.calling_context ≠ Ctxt_id .... State after:: .... S with - call_contexts[Ctxt_id].responded = true + call_contexts[Ctxt_id].needs_to_respond = false call_contexts[Ctxt_id].available_cycles = 0 messages = S.messages · @@ -2568,7 +2670,7 @@ If there is no call, downstream calling context, or response that references a c Conditions:: .... ( - S.call_contexts[Ctxt_id].responded = true + S.call_contexts[Ctxt_id].needs_to_respond = false ) or ( S.call_contexts[Ctxt_id].origin = FromHeartbeat @@ -2577,7 +2679,7 @@ Conditions:: ∀ CallMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id ∀ ResponseMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id ∀ ctxt_ids. - S.call_contexts[ctxt_ids].responded = false + S.call_contexts[ctxt_ids].needs_to_respond = true ==> S.call_contexts[ctxt_ids].origin.calling_context ≠ Ctxt_id .... State after:: @@ -2821,13 +2923,13 @@ S with } | Ctxt_id ↦ Ctxt ∈ S.call_contexts , Ctxt.canister = A.canister_id - , Ctxt.responded = false + , Ctxt.needs_to_respond = true ] for Ctxt_id ↦ Ctxt ∈ S.call_contexts: if Ctxt.canister = A.canister_id: call_contexts[Ctxt_id].deleted := true - call_contexts[Ctxt_id].responded := true + call_contexts[Ctxt_id].needs_to_respond := false call_contexts[Ctxt_id].available_cycles := 0 .... @@ -3229,13 +3331,13 @@ S with } | Ctxt_id ↦ Ctxt ∈ S.call_contexts , Ctxt.canister = A.canister_id - , Ctxt.responded = false + , Ctxt.needs_to_respond = true ] for Ctxt_id ↦ Ctxt ∈ S.call_contexts: if Ctxt.canister = A.canister_id: call_contexts[Ctxt_id].deleted := true - call_contexts[Ctxt_id].responded := true + call_contexts[Ctxt_id].needs_to_respond := false call_contexts[Ctxt_id].available_cycles := 0 .... @@ -3721,6 +3823,14 @@ copy_to_canister(dst : i32, offset : i32, size : i32, data : blob) = copy_from_canister(src : i32, size : i32) blob = if src+size > |es.wasm_state.store.mem| then Trap return es.wasm_state.store.mem[src..src+size] + +Cycles are represented by 128-bit values so they require 16 bytes of memory. + +copy_cycles_to_canister(dst : i32, data : blob) = + let size = 16; + if dst+size > |es.wasm_state.store.mem| then Trap + es.wasm_state.store.mem[dst..dst+size] := data[0..size] + .... ==== System imports @@ -3766,10 +3876,20 @@ ic0.msg_reject(src : i32, size : i32) = es.cycles_available := 0 ic0.msg_cycles_available() : i64 = + if es.cycles_available >= 2^64 then Trap return es.cycles_available +ic0.msg_cycles_available128(dst : i32) = + let amount = es.cycles_available + copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) + ic0.msg_cycles_refunded() : i64 = - return es.params.cycles_refundend + if es.params.cycles_refunded >= 2^64 then Trap + return es.params.cycles_refunded + +ic0.msg_cycles_refunded128(dst : i32) = + let amount = es.params.cycles_refunded + copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) ic0.accept_message() = if es.ingress_filter = Accept then Trap @@ -3778,16 +3898,24 @@ ic0.accept_message() = ic0.msg_method_name_size() : i32 = return |es.method_name| -ic0.msg_method_name_copy(dst:i32, offset:i32, size:i32) : i32 = +ic0.msg_method_name_copy(dst : i32, offset : i32, size : i32) : i32 = copy_to_canister(dst, offset, size, es.params.method_name) -ic0.msg_cycles_accept( max_amount : i64 ) : i64 = +ic0.msg_cycles_accept(max_amount : i64) : i64 = let amount = min(max_amount, es.cycles_available, MAX_CANISTER_BALANCE - es.balance) es.cycles_available := es.cycles_available - amount es.cycles_accepted := es.cycles_accepted + amount es.balance := es.balance + amount return amount +ic0.msg_cycles_accept128(max_amount_high : i64, max_amount_low : i64, dst : i32) = + let max_amount = max_amount_high * 2^64 + max_amount_low + let amount = min(max_amount, es.cycles_available, MAX_CANISTER_BALANCE - es.balance) + es.cycles_available := es.cycles_available - amount + es.cycles_accepted := es.cycles_accepted + amount + es.balance := es.balance + amount + copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) + ic0.canister_self_size() : i32 = return |es.wasm_state.self_id| @@ -3795,8 +3923,13 @@ ic0.canister_self_copy(dst:i32, offset:i32, size:i32) = copy_to_canister(dst, offset, size, es.wasm_state.self_id) ic0.canister_cycle_balance() : i64 = + if es.balance >= 2^64 then Trap return es.balance +ic0.canister_cycles_balance128(dst : i32) = + let amount = es.balance + copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) + ic0.canister_status() : i32 = match es.params.sysenv.canister_status with Running -> return 1 @@ -3850,7 +3983,15 @@ ic0.call_on_cleanup (fun : i32, env : i32) = if es.pending_call.callback.on_cleanup ≠ NoClosure then Trap es.pending_call.callback.on_cleanup := Closure { fun = fun; env = env} -ic0.call_cycles_add(amount : i32) = +ic0.call_cycles_add(amount : i64) = + if es.pending_call = NoPendingCall then Trap + if es.balance < amount then Trap + + es.balance := es.balance - amount + es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount + +ic0.call_cycles_add128(amount_high : i64, amount_low : i64) = + let amount = amount_high * 2^64 + amount_low if es.pending_call = NoPendingCall then Trap if es.balance < amount then Trap From 766a5df99e702c3b75c495b8f54dff3a1a9c6019 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bj=C3=B6rn=20Tackmann?= <54846571+Dfinity-Bjoern@users.noreply.github.com> Date: Tue, 11 Jan 2022 12:35:24 +0100 Subject: [PATCH 005/102] Publish version 0.18.3 (#392) --- spec/changelog.adoc | 6 ++ spec/index.adoc | 245 ++++++++++++++++++++++++++++++++++---------- 2 files changed, 199 insertions(+), 52 deletions(-) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 013012b81..d17e05ff2 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,12 @@ [#changelog] == Changelog +[#0_18_3] +=== 0.18.3 (2022-01-10) + +* Spec: New System API which uses 128-bit values to represent the amount of cycles +* Spec: Subnet delegations include a canister id scope + [#0_18_2] === 0.18.2 (2021-09-29) diff --git a/spec/index.adoc b/spec/index.adoc index 427f1bc88..0d1617ef0 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,6 +1,6 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.2 +0.18.3 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ @@ -331,6 +331,8 @@ The `BIT STRING` field `subjectPublicKey` is the blob `|signing_canister_id| · where `signing_canister_id` is the id of the signing canister and `reconstruct` is a function that computes a root-hash for the tree. +- If the `certificate` includes subnet delegations (possibly nested), then the `signing_canister_id` must be included in each delegation’s canister id range (see <>). + - The `tree` must be a `well_formed` tree with lookup(/sig//, tree) = Found "" @@ -366,6 +368,20 @@ The state tree contains information about the topology of the Internet Computer. + The public key of the subnet (a DER-encoded BLS key, see <>) +* `/subnet//canister_ranges` (blob) ++ +The set of canister ids assigned to this subnet, represented as a list of closed intervals of canister ids, ordered lexicographically, and encoded as CBOR according to this CDDL: ++ +.... +canister_ranges = tagged<[*canister_range]> +canister_range = [principal principal] +principal = bytes .size (0..29) +tagged = #6.55799(t) ; the CBOR tag +.... ++ +NOTE: Because this uses the lexicographic ordering of princpials, and the byte distinguishing the various classes of ids is at the _end_, this range by construction conceptually includes principals of various classes. This specification needs to take care that the fact that principals that are not canisters may appear in these ranges does not cause confusion. + + [#state-tree-request-status] === Request status @@ -519,6 +535,8 @@ In order to read parts of the <>, the user makes a POST request to ` The HTTP response to this request consists of a CBOR map with the following fields: * `certificate` (`blob`): A certificate (see <>). ++ +If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation’s canister id range (see <>). The returned certificate reveals all values whose path is a suffix of the list of requested paths. It also always reveals `/time`, even if not explicitly requested. @@ -834,23 +852,30 @@ In order for a WebAssembly module to be usable as the code for the canister, it * If it exports any functions called `canister_update ` or `canister_query ` for some `name`, the functions must have type `+() -> ()+`. * It may not export both `canister_update ` and `canister_query ` with the same `name`. * It may not export other methods the names of which start with the prefix `canister_` besides the methods allowed above. -* The IC may reject WebAssembly modules that declare more than 6000 functions or more than 200 globals. +* It may not have both `icp:public ` and `icp:private ` with the same `name` as the custom section name. +* It may not have other custom sections the names of which start with the prefix `icp:` besides the `icp:public ` and `icp:private `. +* The IC may reject WebAssembly modules that + + declare more than 6000 functions, or + + declare more than 200 globals, or + + declare more than 16 exported custom sections (the custom section names with prefix `icp:`), or + + the total size of the exported custom sections exceeds 1MiB === Interpretation of numbers WebAssembly number types (`i32`, `i64`) do not indicate if the numbers are to be interpreted as signed or unsigned. Unless noted otherwise, whenever the System API interprets them as numbers (e.g. memory pointers, buffer offsets, array sizes), they are to be interpreted as unsigned. +[#entry-points] === Entry points The canister provides entry points which are invoked by the IC under various circumstances: -* The canister may export a function named `canister_init` and type `+() -> ()+`. -* The canister may export a function named `canister_pre_upgrade` and type `+() -> ()+`. -* The canister may export a function named `canister_post_upgrade` and type `+() -> ()+`. -* The canister may export functions named `canister_inspect_message` with type `+() -> ()+`. -* The canister may export a function named `canister_heartbeat` with type `+() -> ()+`. -* The canister may export functions named `canister_update ` and type `+() -> ()+`. -* The canister may export functions named `canister_query ` and type `+() -> ()+`. +* The canister may export a function with name `canister_init` and type `+() -> ()+`. +* The canister may export a function with name `canister_pre_upgrade` and type `+() -> ()+`. +* The canister may export a function with name `canister_post_upgrade` and type `+() -> ()+`. +* The canister may export functions with name `canister_inspect_message` with type `+() -> ()+`. +* The canister may export a function with name `canister_heartbeat` with type `+() -> ()+`. +* The canister may export functions with name `canister_update ` and type `+() -> ()+`. +* The canister may export functions with name `canister_query ` and type `+() -> ()+`. * The canister table may contain functions of type `+(env : i32) -> ()+` which may be used as callbacks for inter-canister calls. If the execution of any of these entry points traps for any reason, then all changes to the WebAssembly state, as well as the effect of any externally visible system call (like `ic0.msg_reply`, `ic0.msg_reject`, `ic0.call_perform`), are discarded. For upgrades, this transactional behavior applies to the `canister_pre_upgrade`/`canister_post_upgrade` sequence as a whole. @@ -891,7 +916,7 @@ Eventually, a method will want to send a response, using `ic0.reply` or `ic0.rej ==== Heartbeat -For periodic or time-based execution, the WebAssembly module can export a function with name `canister_heartbeat`. If present, the IC will invoke this function at regular intervals. The exact interval is implementation-defined. For each heartbeat invocation, the IC guarantees that the time, as returned by <>, is monotonically increasing. +For periodic or time-based execution, the WebAssembly module can export a function with name `canister_heartbeat`. The heartbeats scheduling algorithm is implementation-defined. `canister_heartbeat` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. @@ -923,12 +948,17 @@ ic0.msg_reply : () -> (); // U ic0.msg_reject : (src : i32, size : i32) -> (); // U Q Ry Rt ic0.msg_cycles_available : () -> i64; // U Rt Ry +ic0.msg_cycles_available128 : (dst : i32) -> (); // U Rt Ry ic0.msg_cycles_refunded : () -> i64; // Rt Ry -ic0.msg_cycles_accept : ( max_amount : i64 ) -> ( amount : i64 ); // U Rt Ry +ic0.msg_cycles_refunded128 : (dst : i32) -> (); // Rt Ry +ic0.msg_cycles_accept : (max_amount : i64) -> ( amount : i64 ); // U Rt Ry +ic0.msg_cycles_accept128 : (max_amount_high : i64, max_amount_low: i64, dst : i32) + -> (); // U Rt Ry ic0.canister_self_size : () -> i32; // * ic0.canister_self_copy : (dst : i32, offset : i32, size : i32) -> (); // * ic0.canister_cycle_balance : () -> i64; // * +ic0.canister_cycle_balance128 : (dst : i32) -> (); // * ic0.canister_status : () -> i32; // * ic0.msg_method_name_size : () -> i32 // F @@ -947,7 +977,8 @@ ic0.call_new : // U ) -> (); ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt H ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt H -ic0.call_cycles_add : ( amount : i64 ) -> (); // U Ry Rt H +ic0.call_cycles_add : (amount : i64) -> (); // U Ry Rt H +ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt H ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt H ic0.stable_size : () -> (page_count : i32); // * @@ -982,7 +1013,7 @@ The comment after each function lists from where these functions may be invoked: * `s`: the `(start)` module initialization function * `F`: from `canister_inspect_message` * `H`: from `canister_heartbeat` -* `*` = `I G U Q Ry Rt C F` (NB: Not `(start)`) +* `*` = `I G U Q Ry Rt C F H` (NB: Not `(start)`) If the canister invokes a system call from somewhere else, it will trap. @@ -1046,7 +1077,7 @@ Appends data it to the (initially empty) data reply. + NOTE: This can be invoked multiple times to build up the argument with data from various places on the Wasm heap. This way, the canister does not have to first copy all the pieces from various places into one location. + -This traps if the current call already has been responded to. +This traps if the current call already has been or does not need to be responded to. * `+ic0.msg_reply : () -> ()+` + @@ -1060,7 +1091,7 @@ See <> for how this interacts with cycles available on this c + Rejects the call. The data referred to by `src`/`size` is used for the diagnostic message. + -This system call traps if `src+size` exceeds the size of the WebAssembly memory, or if the current call already has been responded to, or if the data referred to by `src`/`size` is not valid UTF8. +This system call traps if `src+size` exceeds the size of the WebAssembly memory, or if the current call already has been or does not need to be responded to, or if the data referred to by `src`/`size` is not valid UTF8. + The other end will receive this reject with reject code `CANISTER_REJECT`, see <>. + @@ -1166,6 +1197,14 @@ This adds cycles onto a call. See <>. + This may be called multiple times between `ic0.call_new` and `ic0.call_perform`. + +* `+ic0.call_cycles_add128 : (amount_high : i64, amount_low : i64) -> ()+` ++ +This adds cycles onto a call. See <>. ++ +This may be called multiple times between `ic0.call_new` and `ic0.call_perform`. + + * `+ic0.call_perform : () -> ( err_code : i32 )+` + This concludes assembling the call. It queues the call message to the given destination, but does not actually act on it until the current WebAssembly function returns without trapping. @@ -1179,23 +1218,43 @@ After `ic0.call_perform` and before the next call to `ic0.call_new`, all other ` [#system-api-cycles] === Cycles -Each canister maintains a balance of _cycles_, the utility token used to pay for platform usage. +Each canister maintains a balance of _cycles_, which are used to pay for platform usage. Cycles are represented by 128-bit values. NOTE: This specification currently does not go into details about which actions cost how many cycles and/or when. In general, you must assume that the canister’s cycle balance can change arbitrarily between method executions, and during each System API function call, unless explicitly mentioned otherwise. * `ic0.canister_cycle_balance : () -> i64` + -indicates the current cycle balance of the canister. It is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add`. After execution of the message, the IC may add unused cycles from the reserve back to the balance. +Indicates the current cycle balance of the canister. It is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add`. After execution of the message, the IC may add unused cycles from the reserve back to the balance. + -NOTE: Should a future version of this specification allow `MAX_CANISTER_BALANCE` >= 2^64^, then this call will report 2^64^-1 if the balance exceeds 2^64^. A new, wider System API will then be provided for canisters that need to deal precisely with large canister balances. +NOTE: This call traps if the current balance does not fit into a 64-bit value. Canisters that need to deal with larger cycles balances should use `ic0.canister_cycles_balance128` instead. +* `ic0.canister_cycle_balance128 : (dst : i32) -> ()` ++ +Indicates the current cycle balance of the canister by copying the value to the location `dst` in the canister memory. +The value is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add128`. +After execution of the message, the IC may add unused cycles from the reserve back to the balance. ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. ++ * `ic0.msg_cycles_available : () -> i64` + -returns the amount of cycles that were transferred by the caller of the current call, and is still available in this message. +Returns the amount of cycles that were transferred by the caller of the current call, and is still available in this message. ++ +Initially, in the update method entry point, this is the amount that the caller passed to the canister. When cycles are accepted (`ic0.msg_cycles_accept`), this reports fewer cycles accordingly. When the call is responded to (reply or reject), all available cycles are refunded to the caller, and this will return 0. + -Initially, in the update method entry point, this is the amount that the caller passed to the canister. When cycles are accepted (`ic0.msg_cycles_accept`), this reports less cycles accordingly. When the call is responded to (reply or reject), all available cycles are refunded to the caller, and this will return 0. +NOTE: This call traps if the amount of cycles available does not fit into a 64-bit value. Please use `ic0.msg_cycles_available128` instead. -* `ic0.msg_cycles_accept : ( max_amount : i64 ) -> ( amount : i64 )` +* `ic0.msg_cycles_available128 : (dst : i32) -> ()` ++ +Indicates the number of cycles transferred by the caller of the current call, still available in this message. +The amount of cycles is represented by a 128-bit value. This call copies this value starting to the location `dst` in the canister memory. ++ +Initially, in the update method entry point, this is the amount that the caller passed to the canister. When cycles are accepted (`ic0.msg_cycles_accept128`), this reports fewer cycles accordingly. When the call is responded to (reply or reject), all available cycles are refunded to the caller, and this will report 0 cycles. ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. ++ + +* `ic0.msg_cycles_accept : (max_amount : i64) -> (amount : i64)` + -- This moves cycles from the call to the canister balance. It moves as many cycles as possible, up to these constraints: @@ -1210,7 +1269,7 @@ It can be called multiple times, each time possibly adding more cycles to the ba The return value indicates how many cycles were actually moved. -This system call is deprecated and does not trap. +This system call does not trap. [TIP] ===== @@ -1218,17 +1277,56 @@ Example: To accept all cycles provided in a call, invoke `ic0.msg_cycles_accept( ===== -- -* `ic0.call_cycles_add : ( amount : i64 ) -> ()` +* `ic0.msg_cycles_accept128 : (max_amount_high : i64, max_amount_low : i64, dst : i32) -> ()` ++ +-- +This moves cycles from the call to the canister balance. It moves as many cycles as possible, up to these constraints: + +* It moves no more cycles than the amount obtained by combining `max_amount_high` and `max_amount_low`. Cycles are represented by 128-bit values. + +* It moves no more cycles than available according to `ic0.msg_cycles_available128`, and + +* The canister balance afterwards does not exceed `MAX_CANISTER_BALANCE` minus any possible outstanding balances. The precise logic of this limit is not yet specified here. + +It can be called multiple times, each time possibly adding more cycles to the balance. + +This call also copies the amount of cycles that were actually moved starting at the location `dst` in the canister memory. +-- ++ +This does not trap. ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. +* `ic0.call_cycles_add : (amount : i64) -> ()` + This function moves cycles from the canister balance onto the call under construction, to be transferred with that call. + The cycles are deducted from the balance as shown by `ic0.canister_cycle_balance` immediately, and moved back if the call cannot be performed (e.g. if `ic0.call_perform` signals an error, or if the canister invokes `ic0.call_new` or returns without calling `ic0.call_perform`). + +This system call traps if trying to transfer more cycles than are in the current balance of the canister. + +* `ic0.call_cycles_add128 : (amount_high : i64, amount_low : i64) -> ()` ++ +This function moves cycles from the canister balance onto the call under construction, to be transferred with that call. ++ +The amount of cycles it moves is represented by a 128-bit value which can be obtained by combining the `amount_high` and `amount_low` parameters. ++ +The cycles are deducted from the balance as shown by `ic0.canister_cycles_balance128` immediately, and moved back if the call cannot be performed (e.g. if `ic0.call_perform` signals an error, or if the canister invokes `ic0.call_new` or returns without calling `ic0.call_perform`). ++ This traps if trying to transfer more cycles than are in the current balance of the canister. ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. * `ic0.msg_cycles_refunded : () -> i64` + -this can only be used in a callback handler (reply or reject), and indicates the amount of cycles that came back with the response as a refund. The refund has already been added to the canister balance automatically. +This function can only be used in a callback handler (reply or reject), and indicates the amount of cycles that came back with the response as a refund. The refund has already been added to the canister balance automatically. + +NOTE: This call traps if the amount of cycles refunded does not fit into a 64-bit value. In general, it is recommended to use `ic0.msg_cycles_refunded128` instead. + +* `ic0.msg_cycles_refunded128 : (dst : i32) -> ()` ++ +This function can only be used in a callback handler (reply or reject), and indicates the amount of cycles that came back with the response as a refund. The refund has already been added to the canister balance automatically. ++ +This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. [#system-api-stable-memory] === Stable memory @@ -1343,6 +1441,8 @@ Copies the certificate for the current value of the certified data to the canist + The certificate is a blob as described in <> that contains the values at path `/canister//certified_data` and at path `/time` of <>. + +If this `certificate` includes subnet delegations (possibly nested), then the id of the current canister will be included in each delegation’s canister id range. ++ This traps if `ic0.data_certificate_present()` returns `0`. @@ -1454,7 +1554,7 @@ Only _controllers of the canister can install code. * If `mode = reinstall`, if the canister was not empty, its existing code and state is removed before proceeding as for `mode = install`. + -Note that this is different from `uninstall_code` followed by `install_code`, as that will forcibly reject all unresponded calls. +Note that this is different from `uninstall_code` followed by `install_code`, as that will forcibly reject all calls awaiting a response. * If `mode = upgrade`, this will perform an upgrade of a non-empty canister as described in <>, passing `arg` to the `canister_post_upgrade` method of the new instance. @@ -1531,7 +1631,7 @@ This method takes no input and returns 32 pseudo-random bytes to the caller. The As a provisional method on development instances, the `provisional_create_canister_with_cycles` method is provided. It behaves as `create_canister`, but initializes the canister’s balance with `amount` fresh cycles (using `MAX_CANISTER_BALANCE` if `amount = null`, else capping the balance at `MAX_CANISTER_BALANCE`). -Cycles added to this call via `ic0.call_cycles_add` are returned to the caller. +Cycles added to this call via `ic0.call_cycles_add128` are returned to the caller. This method is only available in local development instances. @@ -1540,7 +1640,7 @@ This method is only available in local development instances. As a provisional method on development instances, the `provisional_top_up_canister` method is provided. It adds `amount` cycles to the balance of canister identified by `amount` (implicitly capping it at `MAX_CANISTER_BALANCE`). -Cycles added to this call via `ic0.call_cycles_add` are returned to the caller. +Cycles added to this call via `ic0.call_cycles_add128` are returned to the caller. Any user can top-up any canister this way. @@ -1695,6 +1795,8 @@ check_delegations(Delegation d) : public_bls_key = .... where `root_public_key` is the a priori known root key. +Delegations are _scoped_, i.e., they indicate which set of canister principals the delegatee subnet may certify for. This set can be obtained from a delegation `d` using `lookup(["subnet",d.subnet_id,"canister_ranges"],d.certificate)`, which must be present, and is encoded as described in <>. The various applications of certificates describe if and how the subnet scope comes into play. + [#certification-encoding] === Encoding of certificates @@ -1934,13 +2036,13 @@ The concrete mapping of this abstract `CanisterModule` to actual WebAssembly con The Internet Computer provides certain messaging guarantees: If a user or a canister calls another canister, it will eventually get a single response (a reply or a rejection), even if some canister code along the way fails. -To ensure that only one response is generated, and also to detect when no response can be generated any more, the IC maintains a _call context_. The `responded` field is set to `true` once the call has received a response. Further attempts to respond will now fail. +To ensure that only one response is generated, and also to detect when no response can be generated any more, the IC maintains a _call context_. The `needs_to_respond` field is set to `false` once the call has received a response. Further attempts to respond will now fail. .... CallCtxt = { canister : CanisterId; origin : CallOrigin; - responded : bool; + needs_to_respond : bool; deleted : bool; available_cycles : Nat; } @@ -2128,18 +2230,18 @@ for some time stamp `T`, some DER-encoded BLS public key `PublicKey`, and using The following is an incomplete list of invariants that should hold for the abstract state `S`, and are not already covered by the type annotations in this section. -* Deleted call contexts have been responded to: +* Deleted call contexts were not awaiting a response: + .... ∀ Ctxt_id ↦ Ctxt ∈ S.call_contexts: - if Ctxt.deleted then Ctxt.responded + if Ctxt.deleted then Ctxt.needs_to_respond = false .... * Responded call contexts have no available_cycles left: + .... ∀ Ctxt_id ↦ Ctxt ∈ S.call_contexts: - if Ctxt.responded then Ctxt.available_cycles = 0 + if Ctxt.needs_to_respond = false then Ctxt.available_cycles = 0 .... * Referenced call contexts exists: @@ -2148,7 +2250,7 @@ The following is an incomplete list of invariants that should hold for the abstr ∀ CallMessage M ∈ S.messages. M.origin.calling_context ∈ S.call_contexts ∀ ResponseMessage M ∈ S.messages. M.origin.calling_context ∈ S.call_contexts ∀ _ ↦ Ctxt ∈ S.call_contexts: - if Ctx.responded = false: + if Ctx.needs_to_respond: Ctxt.origin.calling_context ∈ S.call_contexts .... @@ -2352,7 +2454,7 @@ S with call_contexts[Ctxt_id] = { canister = CM.callee; origin = CM.origin; - responded = false; + needs_to_respond = true; deleted = false; available_cycles = CM.transferred_cycles; } @@ -2361,7 +2463,7 @@ S with * Call context creation: Heartbeat + -If canister `C` exports a method with name `canister_heartbeat`, the IC will add a message to the head of the queue for invoking `canister_heartbeat` and create the corresponding call context. +If canister `C` exports a method with name `canister_heartbeat`, the IC will create the corresponding call context. + Conditions:: + @@ -2387,7 +2489,7 @@ S with call_contexts[Ctxt_id] = { canister = C; origin = FromHeartbeat; - responded = false; + needs_to_respond = false; deleted = false; available_cycles = 0; } @@ -2453,7 +2555,7 @@ if - ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ] , MAX_CANISTER_BALANCE) New_balance > if Is_response then 0 else freezing_limit(S, CM.callee); - (res.response = NoResponse) or (S.call_contexts[M.call_context].responded = false) + (res.response = NoResponse) or S.call_contexts[M.call_context].needs_to_respond then S with canisters[M.receiver].wasm_state = res.new_state; @@ -2483,7 +2585,7 @@ then if res.response = NoResponse: call_contexts[M.call_context].available_cycles = Available - res.cycles_accepted else - call_contexts[M.call_context].responded = true + call_contexts[M.call_context].needs_to_respond = false call_contexts[M.call_context].available_cycles = 0 if res.new_certified_data ≠ NoCertifiedData: @@ -2505,7 +2607,7 @@ Depending whether this is a call message and a response messages, we have either This transition detects certain behavior that will appear as a trap (and which an implementation may implement by trapping directly in a system call): - * Responding if the present call context has already been responded to + * Responding if the present call context does not need to be responded to * Accepting more cycles than are available on the call context * Sending out more cycles than available to the canister * Consuming more cycles than allowed (and reserved) @@ -2539,18 +2641,18 @@ If the call context is not for heartbeat and there is no call, downstream callin Conditions:: .... - S.call_contexts[Ctxt_id].responded = false + S.call_contexts[Ctxt_id].needs_to_respond = true S.call_contexts[Ctxt_id].origin ≠ FromHeartbeat ∀ CallMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id ∀ ResponseMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id ∀ ctxt_ids. - S.call_contexts[ctxt_ids].responded = false + S.call_contexts[ctxt_ids].needs_to_respond ==> S.call_contexts[ctxt_ids].origin.calling_context ≠ Ctxt_id .... State after:: .... S with - call_contexts[Ctxt_id].responded = true + call_contexts[Ctxt_id].needs_to_respond = false call_contexts[Ctxt_id].available_cycles = 0 messages = S.messages · @@ -2568,7 +2670,7 @@ If there is no call, downstream calling context, or response that references a c Conditions:: .... ( - S.call_contexts[Ctxt_id].responded = true + S.call_contexts[Ctxt_id].needs_to_respond = false ) or ( S.call_contexts[Ctxt_id].origin = FromHeartbeat @@ -2577,7 +2679,7 @@ Conditions:: ∀ CallMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id ∀ ResponseMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id ∀ ctxt_ids. - S.call_contexts[ctxt_ids].responded = false + S.call_contexts[ctxt_ids].needs_to_respond = true ==> S.call_contexts[ctxt_ids].origin.calling_context ≠ Ctxt_id .... State after:: @@ -2821,13 +2923,13 @@ S with } | Ctxt_id ↦ Ctxt ∈ S.call_contexts , Ctxt.canister = A.canister_id - , Ctxt.responded = false + , Ctxt.needs_to_respond = true ] for Ctxt_id ↦ Ctxt ∈ S.call_contexts: if Ctxt.canister = A.canister_id: call_contexts[Ctxt_id].deleted := true - call_contexts[Ctxt_id].responded := true + call_contexts[Ctxt_id].needs_to_respond := false call_contexts[Ctxt_id].available_cycles := 0 .... @@ -3229,13 +3331,13 @@ S with } | Ctxt_id ↦ Ctxt ∈ S.call_contexts , Ctxt.canister = A.canister_id - , Ctxt.responded = false + , Ctxt.needs_to_respond = true ] for Ctxt_id ↦ Ctxt ∈ S.call_contexts: if Ctxt.canister = A.canister_id: call_contexts[Ctxt_id].deleted := true - call_contexts[Ctxt_id].responded := true + call_contexts[Ctxt_id].needs_to_respond := false call_contexts[Ctxt_id].available_cycles := 0 .... @@ -3721,6 +3823,14 @@ copy_to_canister(dst : i32, offset : i32, size : i32, data : blob) = copy_from_canister(src : i32, size : i32) blob = if src+size > |es.wasm_state.store.mem| then Trap return es.wasm_state.store.mem[src..src+size] + +Cycles are represented by 128-bit values so they require 16 bytes of memory. + +copy_cycles_to_canister(dst : i32, data : blob) = + let size = 16; + if dst+size > |es.wasm_state.store.mem| then Trap + es.wasm_state.store.mem[dst..dst+size] := data[0..size] + .... ==== System imports @@ -3766,10 +3876,20 @@ ic0.msg_reject(src : i32, size : i32) = es.cycles_available := 0 ic0.msg_cycles_available() : i64 = + if es.cycles_available >= 2^64 then Trap return es.cycles_available +ic0.msg_cycles_available128(dst : i32) = + let amount = es.cycles_available + copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) + ic0.msg_cycles_refunded() : i64 = - return es.params.cycles_refundend + if es.params.cycles_refunded >= 2^64 then Trap + return es.params.cycles_refunded + +ic0.msg_cycles_refunded128(dst : i32) = + let amount = es.params.cycles_refunded + copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) ic0.accept_message() = if es.ingress_filter = Accept then Trap @@ -3778,16 +3898,24 @@ ic0.accept_message() = ic0.msg_method_name_size() : i32 = return |es.method_name| -ic0.msg_method_name_copy(dst:i32, offset:i32, size:i32) : i32 = +ic0.msg_method_name_copy(dst : i32, offset : i32, size : i32) : i32 = copy_to_canister(dst, offset, size, es.params.method_name) -ic0.msg_cycles_accept( max_amount : i64 ) : i64 = +ic0.msg_cycles_accept(max_amount : i64) : i64 = let amount = min(max_amount, es.cycles_available, MAX_CANISTER_BALANCE - es.balance) es.cycles_available := es.cycles_available - amount es.cycles_accepted := es.cycles_accepted + amount es.balance := es.balance + amount return amount +ic0.msg_cycles_accept128(max_amount_high : i64, max_amount_low : i64, dst : i32) = + let max_amount = max_amount_high * 2^64 + max_amount_low + let amount = min(max_amount, es.cycles_available, MAX_CANISTER_BALANCE - es.balance) + es.cycles_available := es.cycles_available - amount + es.cycles_accepted := es.cycles_accepted + amount + es.balance := es.balance + amount + copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) + ic0.canister_self_size() : i32 = return |es.wasm_state.self_id| @@ -3795,8 +3923,13 @@ ic0.canister_self_copy(dst:i32, offset:i32, size:i32) = copy_to_canister(dst, offset, size, es.wasm_state.self_id) ic0.canister_cycle_balance() : i64 = + if es.balance >= 2^64 then Trap return es.balance +ic0.canister_cycles_balance128(dst : i32) = + let amount = es.balance + copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) + ic0.canister_status() : i32 = match es.params.sysenv.canister_status with Running -> return 1 @@ -3850,7 +3983,15 @@ ic0.call_on_cleanup (fun : i32, env : i32) = if es.pending_call.callback.on_cleanup ≠ NoClosure then Trap es.pending_call.callback.on_cleanup := Closure { fun = fun; env = env} -ic0.call_cycles_add(amount : i32) = +ic0.call_cycles_add(amount : i64) = + if es.pending_call = NoPendingCall then Trap + if es.balance < amount then Trap + + es.balance := es.balance - amount + es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount + +ic0.call_cycles_add128(amount_high : i64, amount_low : i64) = + let amount = amount_high * 2^64 + amount_low if es.pending_call = NoPendingCall then Trap if es.balance < amount then Trap From 32644e684467e29f06df574f1ca70eec8a7ce4eb Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Mon, 20 Jun 2022 13:36:30 +0200 Subject: [PATCH 006/102] Add new versioning information --- spec/changelog.adoc | 3 +++ spec/index.adoc | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index d17e05ff2..1d84da48e 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,9 @@ [#changelog] == Changelog +[#0_18_4] +=== 0.18.4 (2022-06-20) + [#0_18_3] === 0.18.3 (2022-01-10) diff --git a/spec/index.adoc b/spec/index.adoc index dba852204..23f147522 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,6 +1,6 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.3 +0.18.4-wip // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ From b8393fb5b8da86713b3af7e5ce6dac6600998812 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Mon, 20 Jun 2022 13:57:02 +0200 Subject: [PATCH 007/102] Fixes and refactoring --- CODEOWNERS | 2 +- LICENSE | 14 ++++ README.md | 35 +++++++++ spec/README.md | 4 + spec/changelog.adoc | 181 +++++++++++++++----------------------------- spec/index.adoc | 52 +++++++------ 6 files changed, 142 insertions(+), 146 deletions(-) create mode 100644 LICENSE create mode 100644 README.md create mode 100644 spec/README.md diff --git a/CODEOWNERS b/CODEOWNERS index 7d47bcfe6..68ef32cd0 100644 --- a/CODEOWNERS +++ b/CODEOWNERS @@ -1 +1 @@ -/spec/index.adoc @nomeata @jensgroth @Dfinity-Bjoern +** @dfinity/canister-os diff --git a/LICENSE b/LICENSE new file mode 100644 index 000000000..e2a36b348 --- /dev/null +++ b/LICENSE @@ -0,0 +1,14 @@ +Copyright 2021 DFINITY Foundation + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + diff --git a/README.md b/README.md new file mode 100644 index 000000000..3a1843e02 --- /dev/null +++ b/README.md @@ -0,0 +1,35 @@ +# Internet Computer Reference + +This repository contains the source files of the [Interface Spec], which describes the externally visible behaviour of the Internet Computer. + +It used to contain a reference implementation and acceptance test suite; these can now be found at . + +## About the Interface Spec + +This document describes the external interface of the Internet Computer. It is the authoritative source for interface details (request and function names, parameters, encodings). The goal is to have a document that is authoritative, and provides a place and a language to discuss external features of the Internet Computer in a hopefully concrete way. However, this document intentionally does not address _how_ to implement this behavior, and cannot be used as an implementation spec. + +## Versioning + +The Interface Spec is versioned, using a three-component version like + + 0.2.1 + +Releases from this repository are tagged using a three-component _code +version_ number: + + 0.8.1 + ┬ ┬ ┬ + │ │ └ The third component is bumped upon non-breaking changes to the spec. + │ └ The second component is bumped with a breaking change to the spec + └ Always zero for now. + +Each major spec version has a release branch (e.g. `release-0.8`) that only sees +non-breaking changes and bugfixes. A release branch should typically be “ahead” of all previous release branches. + +The `master` branch contains finished designs, but is not directly scheduled +for implementation. It lists version version number `∞`. The reference +implementation on this branch typically does _not_ fully implement the spec. This branch should always be “ahead” of all the release branches. + +## Contributing + +This repository currently does not accept external contributions. diff --git a/spec/README.md b/spec/README.md new file mode 100644 index 000000000..c626e12b6 --- /dev/null +++ b/spec/README.md @@ -0,0 +1,4 @@ +The Interface Spec +=============== + +This directory contains the sources to the IC Interface Spec. See the top-level README for more information about the Interface Spec. diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 1d84da48e..f186e5fc0 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -7,71 +7,71 @@ [#0_18_3] === 0.18.3 (2022-01-10) -* Spec: New System API which uses 128-bit values to represent the amount of cycles -* Spec: Subnet delegations include a canister id scope +* New System API which uses 128-bit values to represent the amount of cycles +* Subnet delegations include a canister id scope [#0_18_2] === 0.18.2 (2021-09-29) -* Spec: Canister heartbeat -* Spec: Terminology changes -* Spec: Support for 64-bit stable memory +* Canister heartbeat +* Terminology changes +* Support for 64-bit stable memory [#0_18_1] === 0.18.1 (2021-08-04) -* Spec: Support RSA PKCS#1 v1.5 signatures in web authentication +* Support RSA PKCS#1 v1.5 signatures in web authentication * Spec clarification: Fix various typos and improve textual clarity [#0_18_0] === 0.18.0 (2021-05-18) -* Spec: A canister has a set of controllers, instead of always one +* A canister has a set of controllers, instead of always one [#0_17_0] === 0.17.0 (2021-04-22) -* Spec: Canister Signatures are introduced +* Canister Signatures are introduced * Spec clarification: the signature in the WebAuthn scheme is prefixed by the CBOR self-identifying tag -* Spec: Cycle-depleted canisters are forcibly uninstalled -* Spec: Canister settings in `create_canister` and `update_settings`. `install_code` no longer takes allocation settings. -* Spec: A freezing threshold can be configured via the canister settings +* Cycle-depleted canisters are forcibly uninstalled +* Canister settings in `create_canister` and `update_settings`. `install_code` no longer takes allocation settings. +* A freezing threshold can be configured via the canister settings [#0_16_1] === 0.16.1 (2021-04-14) -* Spec: The cleanup callback is introduced +* The cleanup callback is introduced [#0_16_0] === 0.16.0 (2021-03-25) -* Spec: New http v2 API that allows for stateless boundary nodes +* New http v2 API that allows for stateless boundary nodes [#0_15_6] === 0.15.6 (2021-03-25) -* Spec: The system may impose limits on the number of globals and functions -* Spec: No ingress messages towards empty canisters are accepted -* Spec: No ingress messages towards `raw_rand` and `deposit_cycles` are accepted -* Spec: A memory allocation of `0` means “best effort” +* The system may impose limits on the number of globals and functions +* No ingress messages towards empty canisters are accepted +* No ingress messages towards `raw_rand` and `deposit_cycles` are accepted +* A memory allocation of `0` means “best effort” [#0_15_5] === 0.15.5 (2021-03-11) -* Spec: deposit_cycles(): any caller allowed +* deposit_cycles(): any caller allowed [#0_15_4] === 0.15.4 (2021-03-04) -* Spec: Ingress message filtering -* Spec: Add ECDSA signatures on curve secp256k1 -* Spec: Clarify that the `ic0.data_certificate_present` system function may be +* Ingress message filtering +* Add ECDSA signatures on curve secp256k1 +* Clarify that the `ic0.data_certificate_present` system function may be called in all contexts. [#0_15_3] === 0.15.3 (2021-02-26) -* Spec: Expose module hash and controller via `read_state` +* Expose module hash and controller via `read_state` [#0_15_2] === 0.15.2 (2021-02-09) @@ -81,12 +81,12 @@ [#0_15_0] === 0.15.0 (2020-12-17) -* Spec: Support for raw Ed25519 keys is removed +* Support for raw Ed25519 keys is removed [#0_14_1] === 0.14.1 (2020-12-08) -* Spec: The default `memory_allocation` becomes unspecified +* The default `memory_allocation` becomes unspecified [#0_14_0] === 0.14.0 (2020-11-18) @@ -99,56 +99,50 @@ [#0_13_2] === 0.13.2 (2020-11-12) -* Spec: The `ic0.canister_status` system call +* The `ic0.canister_status` system call [#0_13_1] === 0.13.1 (2020-11-06) -* Spec: Delegation between user public keys +* Delegation between user public keys [#0_13_0] === 0.13.0 (2020-10-19) -* Spec: Certification (also removes “request-status” request) +* Certification (also removes “request-status” request) [#0_12_2] === 0.12.2 (2020-10-23) -* Spec: User authentication method based on WebAuthn is introduced -* Spec: User authentication can use ECDSA -* Spec: Public keys are DER-encoded +* User authentication method based on WebAuthn is introduced +* User authentication can use ECDSA +* Public keys are DER-encoded [#0_12_1] === 0.12.1 (2020-10-16) -* Spec: Return more information in the `canister_status` management call +* Return more information in the `canister_status` management call [#0_12_0] === 0.12.0 (2020-10-13) -* Spec: Anonymous requests must have the sender field set +* Anonymous requests must have the sender field set [#0_11_1] === 0.11.1 (2020-10-01) -* Spec: The `deposit_funds` call -* ic-ref(-test): Implement and test the above changes +* The `deposit_funds` call [#0_11_0] === 0.11.0 (2020-09-23) -* Spec: Inter-canister calls are now performed using a builder-like API -* Spec: Support for funds (balances and transfers) +* Inter-canister calls are now performed using a builder-like API +* Support for funds (balances and transfers) [#v0_10_3] === 0.10.3 (2020-09-21) -* Spec: The anonymous user is introduced - -[#v0_10_2] -=== 0.10.2 (2020-09-08) - -* ic-ref(-test): Bugfix: A query response should _not_ include a `time` field +* The anonymous user is introduced [#v0_10_1] === 0.10.1 (2020-09-01) @@ -158,23 +152,19 @@ [#v0_10_0] === 0.10.0 (2020-08-06) -* Spec: Users can set/update a memory allocation when installing/upgrading a canister. -* Spec: The `expiry` field is added to requests -* ic-ref(-test): Implement and test the above changes, as far as possible +* Users can set/update a memory allocation when installing/upgrading a canister. +* The `expiry` field is added to requests [#v0_9_3] === 0.9.3 (2020-09-01) -* Spec: The management canister supports the `raw_rand` method -* ic-ref(-test): Implement and test the above changes +* The management canister supports the `raw_rand` method [#v0_9_2] === 0.9.2 (2020-08-05) -* Spec: Canister controllers can stop/start canisters and can query their status. -* Spec: Canister controllers can delete canisters -* ic-ref(-test): Implement and test the above changes -* ic-ref: refactorings +* Canister controllers can stop/start canisters and can query their status. +* Canister controllers can delete canisters [#v0_9_1] === 0.9.1 (2020-07-20) @@ -184,55 +174,46 @@ [#v0_9_0] === 0.9.0 (2020-07-15) -* Spec: Introduction of a domain separator (again) -* Spec: The calculation of “derived ids” has changed -* Spec: The self-authenticating and derived id forms use a truncated hash -* Spec: The textual representation of principals has changed -* ic-ref(-test): Implement the above changes -* ic-ref-test: Also send read requests with nonces +* Introduction of a domain separator (again) +* The calculation of “derived ids” has changed +* The self-authenticating and derived id forms use a truncated hash +* The textual representation of principals has changed [#v0_8_2] === 0.8.2 (2020-07-17) -* ic-ref-test: Also send read requests with nonces -* Spec: Installing code via `reinstall` works also on the empty canister -* ic-ref(-test): Implement and test the above changes +* Installing code via `reinstall` works also on the empty canister [#v0_8_1] === 0.8.1 (2020-07-10) * Reflect refined process in README and intro. -* Spec: `ic0.time` added -* ic-ref(-test): Implement and test `ic0.time` +* `ic0.time` added [#v0_8_0] === 0.8.0 (2020-06-23) -* Spec: Revert the introduction of a domain separator -* ic-ref(-test): Implement and test the above changes +* Revert the introduction of a domain separator [#v0_6_2] === 0.6.2 (2020-06-23) -* Spec: Fix meaning-changing typos in `ic.did` -* ic-ref-test: Be more liberal about the precise reject code in some cases. +* Fix meaning-changing typos in `ic.did` [#v0_6_0] === 0.6.0 (2020-06-08) -* Spec: Make all canister ids system-chosen -* Spec: HTTP requests for management features are removed -* ic-ref(-test): Implement and test the above changes +* Make all canister ids system-chosen +* HTTP requests for management features are removed [#v0_4_0] === 0.4.0 (2020-05-25) -* Spec (editorial): the term “principal” is now used for the _id_ of a canister or +* (editorial) the term “principal” is now used for the _id_ of a canister or user, not the canister or user itself -* Spec: The signature of a request needs to be calculated using a domain separator -* Spec: Describe the `controller` attribute, add a request to change it -* Spec: The IC management canister is introduced -* ic-ref(-test): Implement and test the above changes +* The signature of a request needs to be calculated using a domain separator +* Describe the `controller` attribute, add a request to change it +* The IC management canister is introduced [#v0_2_16] === 0.2.16 (2020-05-29) @@ -242,63 +223,19 @@ [#v0_2_14] === 0.2.14 (2020-05-14) -* Spec: Bugfix: Mode should be `reinstall`, not `replace` -* ic-ref-test: A few more tests, refactorings - -[#v0_2_12] -=== 0.2.12 (2020-05-06) - -* ic-ref-test: Remove code to work around lack of creater canister. -* ic-ref-test: Stricter tests for bad signatures -* ic-ref: Also accept CBOR maps of indeterminate length - -[#v0_2_10] -=== 0.2.10 (2020-04-29) - -* ic-ref: Bind to 127.0.0.1 instead of 0.0.0.0 -* ic-ref: Set content-type even for error responses -* ic-ref-test: Tests related to query calls -* ic-ref-test: Test “reply after trap in prior callback” +* Bugfix: Mode should be `reinstall`, not `replace` [#v0_2_8] === 0.2.8 (2020-04-23) -* Spec: Include section with CDDL description -* ic-ref-test: Block less tests on `create_canister` support - -[#v0_2_6] -=== 0.2.6 (2020-04-01) - -* ic-ref-run: Accept any canister id in `install` commands -* ic-ref-test: More defensive printing of HTTP bodies +* Include section with CDDL description [#v0_2_4] === 0.2.4 (2020-03-23) * simplify versioning (only three components), skip 0.2.2 to avoid confusion with 0.2.0.2 -* spec: Clarification: `reply` field is always present -* spec: General cleanup based on front-to-back reading -* ic-ref(-test): Enforce signature checking -* ic-ref(-test): Desired canister ids must be derived from sender -* ic-ref(-test): Require the 55799 semantic CBOR tag, as specified -* ic-ref: Ignore duplicate requests -* ic-ref-test: Run more tests independent of each other (try `-j 8`) -* ic-ref-test: Submit requests with nonces -* ic-ref-test: Test various trap conditions in reply and reject callbacks. -* ic-ref-test: Test that `ic0.debug_print` with invalid bounds does _not_ trap -* ic-ref-test: Allow unspecified fields to appear in the status response -* ic-ref-test: Canister upgrade tests - -[#v0_2_0_2] -=== 0.2.0.2 (2020-03-19) - -* ic-ref: Return status 202, empty body, on `submit`, to match spec -* ic-ref: Allow update or inter-canister calls to query methods -* ic-ref: Trap upon calls from queries -* ic-ref-test: If the IC does not claim to be spec compliant, always succeed - (but still report errors) -* ic-ref-test: Support --html reports -* ic-ref-test: Use the “Universal Canister” to drive tests; more tests. +* Clarification: `reply` field is always present +* General cleanup based on front-to-back reading [#v0_2_0_0] === 0.2.0.0 (2020-03-11) diff --git a/spec/index.adoc b/spec/index.adoc index 23f147522..8753b9be2 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -70,7 +70,7 @@ The public entry points of canisters are called _methods_. Methods can be declar Methods can be _called_, from _caller_ to _callee_, and will eventually incur a _response_ which is either a _reply_ or a _reject_. A method may have _parameters_, which are provided with concrete _arguments_ in a method call. External calls can be update calls, which can call both kinds of methods, and query calls, which can _only_ call query methods. -Inter-canister calls can also call both kinds of methods. Note that calls from a canister to itself also count as "inter-canister". +Inter-canister calls can also call both kinds of methods. Note that calls from a canister to itself also count as "inter-canister". Internally, a call or a response is transmitted as a _message_ from a _sender_ to a _receiver_. Messages do not have a response. @@ -1548,7 +1548,7 @@ Not including a setting in the `settings` record means not changing that field. This method installs code into a canister. -Only _controllers of the canister can install code. +Only controllers of the canister can install code. * If `mode = install`, the canister must be empty before. This will instantiate the canister module and invoke its `canister_init` method (if present), as explained in Section “<>”, passing the `arg` to the canister. @@ -1787,9 +1787,9 @@ Delegation = A chain of delegations is verified using the following algorithm, which also returns the delegated key (a DER-encoded BLS key): .... -check_delegations(NoDelegation) : public_bls_key = +check_delegation(NoDelegation) : public_bls_key = return root_public_key -check_delegations(Delegation d) : public_bls_key = +check_delegation(Delegation d) : public_bls_key = verify_cert(d.certificate) return lookup(["subnet",d.subnet_id,"public_key"],d.certificate) .... @@ -2012,7 +2012,7 @@ CanisterModule = { post_upgrade : (CanisterId, StableMemory, Arg, Env) -> Trap | Return WasmState update_methods : MethodName ↦ ((Arg, Env, AvailableCycles) -> UpdateFunc) query_methods : MethodName ↦ ((Arg, Env) -> QueryFunc) - heartbeat : (Env) -> Trap | Return NoResponse + heartbeat : (Env) -> WasmState -> Trap | Return WasmState callbacks : (Callback, Response, RefundedCycles, Env, AvailableCycles) -> UpdateFunc inspect_message : (MethodName, WasmState, Arg, Env) -> Trap | Return (Accept | Reject) } @@ -2483,9 +2483,9 @@ S with call_context = Ctxt_id; receiver = C; entry_point = Heartbeat; - queue = Queue { from = System; to = C }; + queue = Queue { from = System; to = C }; } - · S.messages + · S.messages call_contexts[Ctxt_id] = { canister = C; origin = FromHeartbeat; @@ -2512,7 +2512,7 @@ Conditions:: S.canisters[M.receiver] ≠ EmptyCanister Mod = S.canisters[M.receiver].module - Is_response = M.entry_point == Callback _ _ + Is_response = M.entry_point == Callback _ _ _ Env = { time = S.time[M.receiver]; @@ -2525,17 +2525,17 @@ Conditions:: Available = S.call_contexts[M.call_contexts].available_cycles ( M.entry_point = PublicMethod Name Caller Data Arg = { data = Data; caller = Caller } - (F = M.update_methods[M.method_name](Arg, Env, Available) + (F = Mod.update_methods[Name](Arg, Env, Available) or - (F = as_update(Mod.query_methods[M.method_name], Arg, Env)) + (F = query_as_update(Mod.query_methods[Name], Arg, Env)) ) or - ( M.entry_point = Callback Callback Response - F = Mod.callbacks(Callback, Response, Env, Available) + ( M.entry_point = Callback Callback Response RefundedCycles + F = Mod.callbacks(Callback, Response, RefundedCycles, Env, Available) ) or ( M.entry_point = Heartbeat - F = Mod.heartbeat(Env) + F = heartbeat_as_update(Mod.heartbeat, Env) ) R = F(S.canisters[M.receiver].wasm_state) @@ -2567,7 +2567,7 @@ then call_context = M.call_context; callback = call.callback }; - caller = C.callee; + caller = M.receiver; callee = call.callee; method_name = call.method_name; arg = call.arg; @@ -2704,7 +2704,7 @@ Conditions:: (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) M.callee = ic_principal M.method_name = 'create_canister' - M.arg = candid() + M.arg = candid(A) is_system_assigned CanisterId CanisterId ∉ dom S.canisters .... @@ -2716,7 +2716,7 @@ S with if A.settings.controllers is not null: controllers[CanisterId] = A.settings.controllers else: - controllers[CanisterId] = [M.caller] + controllers[CanisterId] = [M.caller] balances[CanisterId] = M.transferred_cycles certified_data[CanisterId] = "" messages = Older_messages · Younger_messages · @@ -3231,7 +3231,7 @@ S with messages = Older_messages · FuncMessage { - call_context = Ctxt_id2 + call_context = Ctxt_id receiver = C entry_point = Callback Callback FM.response RM.refunded_cycles queue = Unordered @@ -3535,7 +3535,7 @@ Params = { reject_code : 0 | SYS_FATAL | SYS_TRANSIENT | …; reject_message : Text; sysenv : Env; - cycles_refundend : Nat; + cycles_refunded : Nat; method_name : Text; } ExecutionState = { @@ -3572,7 +3572,7 @@ empty_params = { caller = NoCaller; reject_code = 0; reject_message = ""; - cycles_refundend = 0; + cycles_refunded = 0; } empty_execution_state = { @@ -3710,7 +3710,7 @@ This formulation checks afterwards that the system call `ic0.call_perform` was n + By construction, the (possibly modified) `es.wasm_state` is discarded. -* The partial map `heartbeat` of the `CanisterModule` is defined if the WebAssembly program exports a function `func` named `canister_heartbeat`, and has value +* The function `heartbeat` of the `CanisterModule` is defined if the WebAssembly program exports a function `func` named `canister_heartbeat`, and has value + .... heartbeat = λ (sysenv) → λ wasm_state → @@ -3723,16 +3723,20 @@ heartbeat = λ (sysenv) → λ wasm_state → if es.cycles_accepted ≠ 0 then Trap if es.ingress_filter ≠ Reject then Trap if es.response ≠ NoResponse then Trap - Return NoResponse; + Return es.wasm_state; +.... +otherwise it is +.... +heartbeat = λ (sysenv) → λ wasm_state → Trap .... * The function `callbacks` of the `CanisterModule` is defined as follows + .... -callbacks = λ(callbacks, response, sysenv, available) → λ wasm_state → +callbacks = λ(callbacks, response, refunded_cycles, sysenv, available) → λ wasm_state → let params0 = { empty_params with sysenv - cycles_received = refund; + cycles_refunded = refund_cycles; } let (fun, env, params) = match response with Reply data -> @@ -3823,9 +3827,11 @@ copy_to_canister(dst : i32, offset : i32, size : i32, data : blob) = copy_from_canister(src : i32, size : i32) blob = if src+size > |es.wasm_state.store.mem| then Trap return es.wasm_state.store.mem[src..src+size] +.... Cycles are represented by 128-bit values so they require 16 bytes of memory. +.... copy_cycles_to_canister(dst : i32, data : blob) = let size = 16; if dst+size > |es.wasm_state.store.mem| then Trap From 49f60a2aa32ed42d02fc73fb0e4ad9f1c8544511 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Mon, 20 Jun 2022 14:10:01 +0200 Subject: [PATCH 008/102] Changes related to 128-bit cycles --- spec/changelog.adoc | 2 ++ spec/index.adoc | 43 ++++++++++++++++++------------------------- 2 files changed, 20 insertions(+), 25 deletions(-) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index f186e5fc0..11ffa5838 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -4,6 +4,8 @@ [#0_18_4] === 0.18.4 (2022-06-20) +* Canister cycle balances are represented by 128 bits, and no system-defined upper limit exists anymore + [#0_18_3] === 0.18.3 (2022-01-10) diff --git a/spec/index.adoc b/spec/index.adoc index 8753b9be2..815e40c36 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -96,9 +96,9 @@ Amount of cycles that a canister has to have before a message is attempted to be + Amount of cycles that the IC sets aside when a canister performs a call. This is used to pay for processing the response message, and unused cycles after the execution of the response are refunded. See <>. -* `MAX_CANISTER_BALANCE` +* `DEFAULT_PROVISIONAL_CYCLES_BALANCE` + -Maximum canister cycle balance. Any excess is discarded. Less than 2^128^. +Amount of cycles allocated to a new canister by default, if not explicitly specified. See <>. [#principal] === Principals @@ -205,6 +205,7 @@ If an empty canister receives a response, that response is dropped, as if the ca ==== Canister cycles The IC relies on _cycles_, a utility token, to manage its resources. A canister pays for the resources it uses from its _cycle balance_. +The _cycle_balance_ is stored as 128-bit unsigned integers and operations on them are saturating. In particular, if _cycles_ are added to a canister that would bring its total balance beyond 2^128-1, then the balance will be capped at 2^128-1 and any additional cycles will be lost. When the cycle balance of a canister falls to zero, the canister is _deallocated_. This has the same effect as @@ -1263,8 +1264,6 @@ This moves cycles from the call to the canister balance. It moves as many cycles * It moves no more cycles than available according to `ic0.msg_cycles_available`, and -* The canister balance afterwards does not exceed `MAX_CANISTER_BALANCE` minus any possible outstanding balances. The precise logic of this limit is not yet specified here. - It can be called multiple times, each time possibly adding more cycles to the balance. The return value indicates how many cycles were actually moved. @@ -1286,8 +1285,6 @@ This moves cycles from the call to the canister balance. It moves as many cycles * It moves no more cycles than available according to `ic0.msg_cycles_available128`, and -* The canister balance afterwards does not exceed `MAX_CANISTER_BALANCE` minus any possible outstanding balances. The precise logic of this limit is not yet specified here. - It can be called multiple times, each time possibly adding more cycles to the balance. This call also copies the amount of cycles that were actually moved starting at the location `dst` in the canister memory. @@ -1629,7 +1626,7 @@ This method takes no input and returns 32 pseudo-random bytes to the caller. The [#ic-provisional_create_canister_with_cycles] === IC method `provisional_create_canister_with_cycles` -As a provisional method on development instances, the `provisional_create_canister_with_cycles` method is provided. It behaves as `create_canister`, but initializes the canister’s balance with `amount` fresh cycles (using `MAX_CANISTER_BALANCE` if `amount = null`, else capping the balance at `MAX_CANISTER_BALANCE`). +As a provisional method on development instances, the `provisional_create_canister_with_cycles` method is provided. It behaves as `create_canister`, but initializes the canister’s balance with `amount` fresh cycles (using `DEFAULT_PROVISIONAL_CYCLES_BALANCE` if `amount = null`). Cycles added to this call via `ic0.call_cycles_add128` are returned to the caller. @@ -1638,7 +1635,7 @@ This method is only available in local development instances. [#ic-provisional_top_up_canister] === IC method `provisional_top_up_canister` -As a provisional method on development instances, the `provisional_top_up_canister` method is provided. It adds `amount` cycles to the balance of canister identified by `amount` (implicitly capping it at `MAX_CANISTER_BALANCE`). +As a provisional method on development instances, the `provisional_top_up_canister` method is provided. It adds `amount` cycles to the balance of canister identified by `amount`. Cycles added to this call via `ic0.call_cycles_add128` are returned to the caller. @@ -2547,13 +2544,11 @@ if Cycles_used ≤ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) res.cycles_accepted ≤ Available New_balance = - min( - S.balances[M.receiver] - + res.cycles_accepted - + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - - Cycles_used - - ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ] - , MAX_CANISTER_BALANCE) + S.balances[M.receiver] + + res.cycles_accepted + + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + - Cycles_used + - ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ] New_balance > if Is_response then 0 else freezing_limit(S, CM.callee); (res.response = NoResponse) or S.call_contexts[M.call_context].needs_to_respond then @@ -3124,7 +3119,7 @@ State after:: .... S with balances[CanisterId] = - min(S.balances[A.canister_id] + M.transferred_cycles, MAX_CANISTER_BALANCE) + S.balances[A.canister_id] + M.transferred_cycles messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin @@ -3179,7 +3174,7 @@ S with canisters[CanisterId] = EmptyCanister time[CanisterId] = CurrentTime controllers[CanisterId] = [M.caller] - balances[CanisterId] = min(A.amount, MAX_CANISTER_BALANCE) + balances[CanisterId] = A.amount certified_data[CanisterId] = "" messages = Older_messages · Younger_messages · ResponseMessage { @@ -3204,7 +3199,7 @@ Conditions:: State after:: .... S with - balances[CanisterId] = min(balances[CanisterId] + A.amount, MAX_CANISTER_BALANCE) + balances[CanisterId] = balances[CanisterId] + A.amount .... ==== Callback invocation @@ -3226,8 +3221,7 @@ State after:: .... S with balances[S.call_contexts[Ctxt_id].canister] = - min(balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles, - MAX_CANISTER_BALANCE) + balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles messages = Older_messages · FuncMessage { @@ -3255,8 +3249,7 @@ State after:: .... S with balances[S.call_contexts[Ctxt_id].canister] = - min(balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles + MAX_CYCLES_PER_RESPONSE, - MAX_CANISTER_BALANCE) + balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles + MAX_CYCLES_PER_RESPONSE messages = Older_messages · Younger_messages .... @@ -3908,7 +3901,7 @@ ic0.msg_method_name_copy(dst : i32, offset : i32, size : i32) : i32 = copy_to_canister(dst, offset, size, es.params.method_name) ic0.msg_cycles_accept(max_amount : i64) : i64 = - let amount = min(max_amount, es.cycles_available, MAX_CANISTER_BALANCE - es.balance) + let amount = min(max_amount, es.cycles_available) es.cycles_available := es.cycles_available - amount es.cycles_accepted := es.cycles_accepted + amount es.balance := es.balance + amount @@ -3916,7 +3909,7 @@ ic0.msg_cycles_accept(max_amount : i64) : i64 = ic0.msg_cycles_accept128(max_amount_high : i64, max_amount_low : i64, dst : i32) = let max_amount = max_amount_high * 2^64 + max_amount_low - let amount = min(max_amount, es.cycles_available, MAX_CANISTER_BALANCE - es.balance) + let amount = min(max_amount, es.cycles_available) es.cycles_available := es.cycles_available - amount es.cycles_accepted := es.cycles_accepted + amount es.balance := es.balance + amount @@ -4021,7 +4014,7 @@ ic0.call_peform() : ( err_code : i32 ) = // helper function discard_pending_call() = if es.pending_call ≠ NoPendingCall then - es.balance := min(es.balance + MAX_CYCLES_PER_RESPONSE + es.pending_call.transferred_cycles, MAX_CANISTER_BALANCE) + es.balance := es.balance + MAX_CYCLES_PER_RESPONSE + es.pending_call.transferred_cycles es.pending_call := NoPendingCall ic0.stable_size() : (page_count : i32) = From 2fb7da2c87750ac66ed880610b04c547b5f3f63a Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Mon, 20 Jun 2022 14:17:59 +0200 Subject: [PATCH 009/102] Move ic0 interface to external file --- spec/ic0.txt | 65 +++++++++++++++++++++++++++++++++++++++++++++ spec/index.adoc | 70 +++---------------------------------------------- 2 files changed, 69 insertions(+), 66 deletions(-) create mode 100644 spec/ic0.txt diff --git a/spec/ic0.txt b/spec/ic0.txt new file mode 100644 index 000000000..53e9d2716 --- /dev/null +++ b/spec/ic0.txt @@ -0,0 +1,65 @@ +ic0.msg_arg_data_size : () -> i32; // I U Q Ry F +ic0.msg_arg_data_copy : (dst : i32, offset : i32, size : i32) -> (); // I U Q Ry F +ic0.msg_caller_size : () -> i32; // I G U Q F +ic0.msg_caller_copy : (dst : i32, offset: i32, size : i32) -> (); // I G U Q F +ic0.msg_reject_code : () -> i32; // Ry Rt +ic0.msg_reject_msg_size : () -> i32; // Rt +ic0.msg_reject_msg_copy : (dst : i32, offset : i32, size : i32) -> (); // Rt + +ic0.msg_reply_data_append : (src : i32, size : i32) -> (); // U Q Ry Rt +ic0.msg_reply : () -> (); // U Q Ry Rt +ic0.msg_reject : (src : i32, size : i32) -> (); // U Q Ry Rt + +ic0.msg_cycles_available : () -> i64; // U Rt Ry +ic0.msg_cycles_available128 : (dst : i32) -> (); // U Rt Ry +ic0.msg_cycles_refunded : () -> i64; // Rt Ry +ic0.msg_cycles_refunded128 : (dst : i32) -> (); // Rt Ry +ic0.msg_cycles_accept : (max_amount : i64) -> (amount : i64); // U Rt Ry +ic0.msg_cycles_accept128 : (max_amount_high : i64, max_amount_low: i64, dst : i32) + -> (); // U Rt Ry + +ic0.canister_self_size : () -> i32; // * +ic0.canister_self_copy : (dst : i32, offset : i32, size : i32) -> (); // * +ic0.canister_cycle_balance : () -> i64; // * +ic0.canister_cycle_balance128 : (dst : i32) -> (); // * +ic0.canister_status : () -> i32; // * + +ic0.msg_method_name_size : () -> i32; // F +ic0.msg_method_name_copy : (dst : i32, offset : i32, size : i32) -> (); // F +ic0.accept_message : () -> (); // F + +ic0.call_new : // U Ry Rt H + ( callee_src : i32, + callee_size : i32, + name_src : i32, + name_size : i32, + reply_fun : i32, + reply_env : i32, + reject_fun : i32, + reject_env : i32 + ) -> (); +ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt H +ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt H +ic0.call_cycles_add : (amount : i64) -> (); // U Ry Rt H +ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt H +ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt H + +ic0.stable_size : () -> (page_count : i32); // * +ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32); // * +ic0.stable_write : (offset : i32, src : i32, size : i32) -> (); // * +ic0.stable_read : (dst : i32, offset : i32, size : i32) -> (); // * +ic0.stable64_size : () -> (page_count : i64); // * +ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64); // * +ic0.stable64_write : (offset : i64, src : i64, size : i64) -> (); // * +ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> (); // * + +ic0.certified_data_set : (src: i32, size: i32) -> (); // I G U Ry Rt H +ic0.data_certificate_present : () -> i32; // * +ic0.data_certificate_size : () -> i32; // * +ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> (); // * + +ic0.time : () -> (timestamp : i64); // * +ic0.performance_counter : (counter_type : i32) -> (counter : i64); // * s + +ic0.debug_print : (src : i32, size : i32) -> (); // * s +ic0.trap : (src : i32, size : i32) -> (); // * s \ No newline at end of file diff --git a/spec/index.adoc b/spec/index.adoc index 815e40c36..c49f628eb 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -935,72 +935,10 @@ In the reply callback of a <>, the a The following sections describe various System API functions, also referred to as system calls, which we summarize here. -.... -ic0.msg_arg_data_size : () -> i32; // I U Q Ry F -ic0.msg_arg_data_copy : (dst : i32, offset : i32, size : i32) -> (); // I U Q Ry F -ic0.msg_caller_size : () -> i32; // I G U Q F -ic0.msg_caller_copy : (dst : i32, offset: i32, size : i32) -> (); // I G U Q F -ic0.msg_reject_code : () -> i32; // Ry Rt -ic0.msg_reject_msg_size : () -> i32; // Rt -ic0.msg_reject_msg_copy : (dst : i32, offset : i32, size : i32) -> (); // Rt - -ic0.msg_reply_data_append : (src : i32, size : i32) -> (); // U Q Ry Rt -ic0.msg_reply : () -> (); // U Q Ry Rt -ic0.msg_reject : (src : i32, size : i32) -> (); // U Q Ry Rt - -ic0.msg_cycles_available : () -> i64; // U Rt Ry -ic0.msg_cycles_available128 : (dst : i32) -> (); // U Rt Ry -ic0.msg_cycles_refunded : () -> i64; // Rt Ry -ic0.msg_cycles_refunded128 : (dst : i32) -> (); // Rt Ry -ic0.msg_cycles_accept : (max_amount : i64) -> ( amount : i64 ); // U Rt Ry -ic0.msg_cycles_accept128 : (max_amount_high : i64, max_amount_low: i64, dst : i32) - -> (); // U Rt Ry - -ic0.canister_self_size : () -> i32; // * -ic0.canister_self_copy : (dst : i32, offset : i32, size : i32) -> (); // * -ic0.canister_cycle_balance : () -> i64; // * -ic0.canister_cycle_balance128 : (dst : i32) -> (); // * -ic0.canister_status : () -> i32; // * - -ic0.msg_method_name_size : () -> i32 // F -ic0.msg_method_name_copy : (dst : i32, offset : i32, size : i32) -> (); // F -ic0.accept_message : () -> (); // F - -ic0.call_new : // U Ry Rt H - ( callee_src : i32, - callee_size : i32, - name_src : i32, - name_size : i32, - reply_fun : i32, - reply_env : i32, - reject_fun : i32, - reject_env : i32 - ) -> (); -ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt H -ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt H -ic0.call_cycles_add : (amount : i64) -> (); // U Ry Rt H -ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt H -ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt H - -ic0.stable_size : () -> (page_count : i32); // * -ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32); // * -ic0.stable_write : (offset : i32, src : i32, size : i32) -> (); // * -ic0.stable_read : (dst : i32, offset : i32, size : i32) -> (); // * -ic0.stable64_size : () -> (page_count : i64); // * -ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64); // * -ic0.stable64_write : (offset : i64, src : i64, size : i64) -> (); // * -ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> (); // * - -ic0.certified_data_set : (src: i32, size: i32) -> () // I G U Ry Rt H -ic0.data_certificate_present : () -> i32 // * -ic0.data_certificate_size : () -> i32 // * -ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> () // * - -ic0.time : () -> (timestamp : i64); // * - -ic0.debug_print : (src : i32, size : i32) -> (); // * s -ic0.trap : (src : i32, size : i32) -> (); // * s -.... +[source,rust] +---- +include::{example}ic0.txt[] +---- The comment after each function lists from where these functions may be invoked: From 061d0ffe048015845e15abb56be29cb08aadae07 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Mon, 20 Jun 2022 14:20:01 +0200 Subject: [PATCH 010/102] Gzip encoded canister modules --- spec/changelog.adoc | 1 + spec/index.adoc | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 11ffa5838..c0b428649 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -5,6 +5,7 @@ === 0.18.4 (2022-06-20) * Canister cycle balances are represented by 128 bits, and no system-defined upper limit exists anymore +* Canister modules can be gzip-encoded [#0_18_3] === 0.18.3 (2022-01-10) diff --git a/spec/index.adoc b/spec/index.adoc index c49f628eb..2e18d320c 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1497,6 +1497,12 @@ This is atomic: If the response to this request is a `reject`, then this call ha NOTE: Some canisters may not be able to make sense of callbacks after upgrades; these should be stopped first, to wait for all outstanding callbacks, or be uninstalled first, to prevent outstanding callbacks from being invoked. It is expected that the canister admin (or their tooling) does that separately. +The `wasm_module` field specifies the canister module to be installed. +The system supports multiple encodings of the `wasm_module` field: + + * If the `wasm_module` starts with byte sequence `[0x00, 'a', 's', 'm']`, the system parses `wasm_module` as a raw WebAssembly binary. + * If the `wasm_module` starts with byte sequence `[0x1f, 0x8b, 0x08]`, the system decompresses the contents of `wasm_module` as a gzip stream according to https://datatracker.ietf.org/doc/html/rfc1952.html[RFC-1952] and then parses the output as a WebAssembly binary. + [#ic-uninstall_code] === IC method `uninstall_code` From a98fb2bfee43126dc452cd2387434ed2c9e549c2 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Mon, 20 Jun 2022 14:22:18 +0200 Subject: [PATCH 011/102] Expose custom wasm sections in state tree --- spec/changelog.adoc | 1 + spec/index.adoc | 20 ++++++++++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index c0b428649..f0c3f7449 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -6,6 +6,7 @@ * Canister cycle balances are represented by 128 bits, and no system-defined upper limit exists anymore * Canister modules can be gzip-encoded +* Expose Wasm custom sections in the state tree [#0_18_3] === 0.18.3 (2022-01-10) diff --git a/spec/index.adoc b/spec/index.adoc index 2e18d320c..1b102f78b 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -418,7 +418,7 @@ The certified data of the canister with the given id, see </module_hash` (blob): + @@ -430,6 +430,12 @@ If the canister is not empty, it exists and contains the SHA256 hash of the curr + The current controllers of the canister. The value consists of a CBOR data item with major type 6 (“Semantic tag”) and tag value `55799` (see https://tools.ietf.org/html/rfc7049#section-2.4.5[Self-Describe CBOR]), followed by an array of principals in their binary form (CDDL `#6.55799([* bytes .size (0..29)])`). +* `/canister//metadata/` (blob): ++ +If the canister has a https://webassembly.github.io/spec/core/binary/modules.html#custom-section[custom section] called `icp:public ` or `icp:private `, this path contains the content of the custom section. Otherwise, this path does not exist. ++ +It is recommended for the canister to have a custom section called "icp:public candid:service", which contains the UTF-8 encoding of https://github.com/dfinity/candid/blob/master/spec/Candid.md#core-grammar[the Candid interface] for the canister. + [#http-interface] == HTTPS Interface @@ -548,6 +554,7 @@ All requested paths must have one of the following paths as prefix: * `/request_status/`. Can only be read if `/request_id/` is not present in the state tree, or if this `read_state` request was signed by same sender as the original request referenced by ``, and the effective canister id of the original request matches the `` (see <>) in this HTTP request’s path. * `/canisters//module_hash`. Can be requested by anyone, if `` matches ``. * `/canisters//controllers`. Can be requested by anyone, if `` matches ``. The order may vary depending on the implementation. + * `/canisters//metadata/`. Can be read by anyone if `` matches ``, and `` is a public custom section. If `` is a private custom section, it can only be read if this `read_state` request was signed by one of the controllers of the canister. Note that the paths `/canisters//certified_data` are not accessible with this method; these paths are only exposed to the canister themselves via the System API (see <>). @@ -2141,6 +2148,8 @@ CanState wasm_state : WasmState; module : CanisterModule; raw_module : Blob; + public_custom_sections: Text ↦ Blob; + private_custom_sections: Text ↦ Blob; } CanStatus = Running @@ -3395,6 +3404,12 @@ may_read_path(S, _, ["request_status", Rid] · _) = else True may_read_path(S, _, ["canister", cid, "module_hash"]) = cid == ECID may_read_path(S, _, ["canister", cid, "controllers"]) = cid == ECID +may_read_path(S, _, ["canister", cid, "metadata", name]) = + if name ∈ S.canisters[cid].public_custom_sections + then cid == ECID + else if name ∈ S.canisters[cid].private_custom_sections + then cid == ECID ∧ RS.sender ∈ S.controllers[cid] + else False may_read_path(S, _, _) = False .... @@ -3410,7 +3425,8 @@ state_tree(S) = { "canister": { canister_id : { "module_hash" : SHA256(C.raw_module) | if C ≠ EmptyCanister } ∪ - { "controllers" : CBOR(S.controllers[canister_id]) } + { "controllers" : CBOR(S.controllers[canister_id]) } ∪ + { "metadata": { name: blob | (name, blob) ∈ S.canisters[canister_id].public_custom_sections ∪ S.canisters[canister_id].private_custom_sections } } | (canister_id, C) ∈ S.canisters }; "subnet": { subnet_id : { "public_key" : pub } | (subnet_id, subnet_pk) ∈ subnets }; } From 2198afa59ea825c7a2b1f4c10a72789c3dc7febd Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Mon, 20 Jun 2022 14:33:48 +0200 Subject: [PATCH 012/102] Netlify and nix changes --- .netlify.sh | 19 +++++++++++++++++++ default.nix | 4 ++-- netlify.toml | 3 +++ spec/gen-antora.sh | 4 ++-- 4 files changed, 26 insertions(+), 4 deletions(-) create mode 100755 .netlify.sh create mode 100644 netlify.toml diff --git a/.netlify.sh b/.netlify.sh new file mode 100755 index 000000000..d91fcfdb3 --- /dev/null +++ b/.netlify.sh @@ -0,0 +1,19 @@ +#!/usr/bin/env bash + +# +# Deploy a preview of the interface specification to netlify +# + +set -e + +export NP_GIT=$(which git) + +wget -nv https://github.com/DavHau/nix-portable/releases/download/v009/nix-portable +chmod +x nix-portable + +./nix-portable nix-build -A interface-spec default.nix + +# The "result" symlink only valid inside the nix-portable sandbox, so use nix-shell +./nix-portable nix-shell -p bash --run "cp -rL result/spec/ _netlify-deploy" + +chmod -R u+w _netlify-deploy diff --git a/default.nix b/default.nix index e35311795..834388ac3 100644 --- a/default.nix +++ b/default.nix @@ -34,8 +34,8 @@ rec { -R $PWD -D $out/$doc_path/ index.adoc find . -type f -name '*.png' | cpio -pdm $out/$doc_path/ cp *.cddl $out/$doc_path - cp ic.did $out/$doc_path - + cp *.did $out/$doc_path + cp *.txt $out/$doc_path mkdir -p $out/nix-support echo "report spec $out/$doc_path index.html" >> $out/nix-support/hydra-build-products diff --git a/netlify.toml b/netlify.toml new file mode 100644 index 000000000..1d4a4ec08 --- /dev/null +++ b/netlify.toml @@ -0,0 +1,3 @@ +[build] + command = "./.netlify.sh" + publish = "_netlify-deploy" diff --git a/spec/gen-antora.sh b/spec/gen-antora.sh index 289b1a8cf..1bad3c2c1 100755 --- a/spec/gen-antora.sh +++ b/spec/gen-antora.sh @@ -50,7 +50,7 @@ __END__ cp -v "$in/index.adoc" "$out/modules/interface-spec/pages/index.adoc" cp -v "$in/changelog.adoc" "$out/modules/interface-spec/partials/changelog.adoc" -cp -v "$in"/*.did "$in"/*.cddl "$out/modules/interface-spec/attachments" -cp -v "$in"/*.did "$in"/*.cddl "$out/modules/interface-spec/examples" +cp -v "$in"/*.did "$in"/*.cddl "$in"/*.txt "$out/modules/interface-spec/attachments" +cp -v "$in"/*.did "$in"/*.cddl "$in"/*.txt "$out/modules/interface-spec/examples" From 52f946c01a85dadf2a9c67f4a711ffee3ffb5f30 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Mon, 20 Jun 2022 14:40:39 +0200 Subject: [PATCH 013/102] Add Bitcoin and ECDSA API --- spec/changelog.adoc | 2 + spec/ic.did | 71 ++++++++++++++++++++++ spec/index.adoc | 139 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 212 insertions(+) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index f0c3f7449..fd54e02ab 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -7,6 +7,8 @@ * Canister cycle balances are represented by 128 bits, and no system-defined upper limit exists anymore * Canister modules can be gzip-encoded * Expose Wasm custom sections in the state tree +* EXPERIMENTAL: Canister API for accessing Bitcoin transactions +* EXPERIMENTAL: Canister API for threshold ECDSA signatures [#0_18_3] === 0.18.3 (2022-01-10) diff --git a/spec/ic.did b/spec/ic.did index 5b64725d1..8958df886 100644 --- a/spec/ic.did +++ b/spec/ic.did @@ -16,6 +16,59 @@ type definite_canister_settings = record { freezing_threshold : nat; }; +type ecdsa_curve = variant { secp256k1; }; + +type satoshi = nat64; + +type bitcoin_network = variant { + mainnet; + testnet; +}; + +type bitcoin_address = text; + +type block_hash = blob; + +type outpoint = record { + txid : blob; + vout : nat32 +}; + +type utxo = record { + outpoint: outpoint; + value: satoshi; + height: nat32; +}; + +type get_utxos_request = record { + address : bitcoin_address; + network: bitcoin_network; + filter: opt variant { + min_confirmations: nat32; + page: blob; + }; +}; + +type get_utxos_response = record { + utxos: vec utxo; + tip_block_hash: block_hash; + tip_height: nat32; + next_page: opt blob; +}; + +type get_balance_request = record { + address : bitcoin_address; + network: bitcoin_network; + min_confirmations: opt nat32; +}; + +type send_transaction_request = record { + transaction: blob; + network: bitcoin_network; +}; + +type millisatoshi_per_byte = nat64; + service ic : { create_canister : (record { settings : opt canister_settings @@ -44,6 +97,24 @@ service ic : { deposit_cycles : (record {canister_id : canister_id}) -> (); raw_rand : () -> (blob); + // Threshold ECDSA signature + ecdsa_public_key : (record { + canister_id : opt canister_id; + derivation_path : vec blob; + key_id : record { curve: ecdsa_curve; name: text }; + }) -> (record { public_key : blob; chain_code : blob; }); + sign_with_ecdsa : (record { + message_hash : blob; + derivation_path : vec blob; + key_id : record { curve: ecdsa_curve; name: text }; + }) -> (record { signature : blob }); + + // bitcoin interface + bitcoin_get_balance: (get_balance_request) -> (satoshi); + bitcoin_get_utxos: (get_utxos_request) -> (get_utxos_response); + bitcoin_send_transaction: (send_transaction_request) -> (); + bitcoin_get_current_fee_percentiles: () -> (vec millisatoshi_per_byte); + // provisional interfaces for the pre-ledger world provisional_create_canister_with_cycles : (record { amount: opt nat; diff --git a/spec/index.adoc b/spec/index.adoc index 1b102f78b..96d164f8c 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1574,6 +1574,36 @@ There is no restriction on who can invoke this method. This method takes no input and returns 32 pseudo-random bytes to the caller. The return value is unknown to any part of the IC at time of the submission of this call. A new return value is generated for each call to this method. +[#ic-ecdsa_public_key] +=== IC method `ecdsa_public_key` + +NOTE: The ECDSA API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. + +This method returns a https://www.secg.org/sec1-v2.pdf[SEC1] encoded ECDSA public key for the given canister using the given derivation path. If the `canister_id` is unspecified, it will default to the canister id of the caller. +The `derivation_path` is a vector of variable length byte strings. +The `key_id` is a struct specifying both a curve and a name. +The availability of a particular `key_id` depends on implementation. + +For curve `secp256k1`, the public key is derived using a generalization of BIP32 (see https://ia.cr/2021/1330[ia.cr/2021/1330, Appendix D]). To derive (non-hardened) https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki[BIP-0032]-compatible public keys, each byte string (`blob`) in the `derivation_path` must be a 4-byte big-endian encoding of an unsigned integer less than 2^31^. + +The return result is an extended public key consisting of an ECDSA `public_key`, encoded in https://www.secg.org/sec2-v2.pdf[SEC1] compressed form, and a `chain_code`, which can be used to deterministically derive child keys of the `public_key`. + +This call requires that the ECDSA feature is enabled, and the `canister_id` meets the requirement of a canister id. +Otherwise it will be rejected. + +[#ic-sign_with_ecdsa] +=== IC method `sign_with_ecdsa` + +NOTE: The ECDSA API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. + +This method returns a new https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf[ECDSA] signature of the given `message_hash` that can be separately verified against a derived ECDSA public key. +This public key can be obtained by calling `ecdsa_public_key` with the caller's `canister_id`, and the same `derivation_path` and `key_id` used here. + +The signatures are encoded as the concatenation of the https://www.secg.org/sec2-v2.pdf[SEC1] encodings of the two values r and s. For curve `secp256k1`, this corresponds to 32-byte big-endian encoding. + +This call requires that the ECDSA feature is enabled, the caller is a canister, and `message_hash` is 32 bytes long. +Otherwise it will be rejected. + [#ic-provisional_create_canister_with_cycles] === IC method `provisional_create_canister_with_cycles` @@ -1594,6 +1624,115 @@ Any user can top-up any canister this way. This method is only available in local development instances. + +[#ic-bitcoin-api] +== The IC Bitcoin API + +NOTE: The IC Bitcoin API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. + +The Bitcoin functionality is exposed via the management canister. +Information about Bitcoin can be found in the https://developer.bitcoin.org/devguide/[Bitcoin developer guides]. +Invoking the functions of the Bitcoin API will cost cycles. The concrete costs for each function are yet to be determined. + +[#ic-bitcoin_get_utxos] +=== IC method `bitcoin_get_utxos` + +Given a `get_utxos_request`, which must specify a Bitcoin address and a Bitcoin network (`mainnet` or `testnet`), the function returns all unspent transaction outputs (UTXOs) associated with the +provided address in the specified Bitcoin network based on the current view of the Bitcoin blockchain available to the Bitcoin component. +The UTXOs are returned sorted by block height in descending order. + +The following address formats are supported: + +* Pay to public key hash (P2PKH) +* Pay to script hash (P2SH) +* Pay to witness public key hash (P2WPKH) +* Pay to witness script hash (P2WSH) +* Pay to taproot (P2TR) + +If the address is malformed, the call is rejected. + +The optional `filter` parameter can be used to restrict the set of returned UTXOs, either providing a +minimum number of confirmations or a page reference when pagination is used for addresses with many UTXOs. +In the first case, only UTXOs with at least the provided number of confirmations +are returned, i.e., transactions with fewer than this number of +confirmations are not considered. In other words, if the number of confirmations is `c`, an output +is returned if it occurred in a transaction with at least `c` confirmations +and there is no transaction that spends the same output with at least `c` confirmations. + +There is an upper bound of 144 on the minimum number of confirmations. +If a larger minimum number of confirmations is specified, the call is rejected. +Note that this is not a severe restriction as the minimum number of confirmations is typically set +to a value around 6 in practice. + +It is important to note that the validity of transactions is not verified in the Bitcoin component. The Bitcoin component relies on the +proof of work that goes into the blocks and the verification of the blocks in the Bitcoin network. +For a newly discovered block, a regular Bitcoin (full) node therefore provides a higher level of security +than the Bitcoin component, which implies that it is advisable to set the number of confirmations to a reasonably large value, such as 6, +to gain confidence in the correctness of the returned UTXOs. + +There is an upper bound of 100,000 UTXOs that can be returned in a single request. For addresses that contain sufficiently large UTXOs, +the partial set of the address' UTXOs are returned along with a page reference. + +In the second case, a page reference (a series of bytes) must be provided, which instructs the Bitcoin component to +collect UTXOs starting from the corresponding "page". + +A `get_utxos_request` without the optional `filter` results in a request that +considers the full blockchain, which is equivalent to setting +`min_confirmations` to 0. + +The recommended workflow is to issue a request with the desired number of confirmations. +If the `next_page` field in the response is not empty, there are more UTXOs than in the returned vector. +In that case, the `page` field should be set to the `next_page` bytes in the subsequent request to obtain the next batch of UTXOs. + +[#ic-bitcoin_get_balance] +=== IC method `bitcoin_get_balance` + +Given a `get_balance_request`, which must specify a Bitcoin address and a Bitcoin network (`mainnet` or `testnet`), the function returns the current balance of this address in `Satoshi` (10^8 Satoshi = 1 Bitcoin) in the specified Bitcoin network. The same address formats as for link:#ic-bitcoin_get_utxos[`bitcoin_get_utxos`] are supported. + +If the address is malformed, the call is rejected. + +The optional `min_confirmations` parameter can be used to limit the set of considered UTXOs +for the calculation of the balance to those with at least the provided number of confirmations +in the same manner as for the link:#ic-bitcoin_get_utxos[`bitcoin_get_utxos`] call. + +Given an address and the optional `min_confirmations` parameter, `bitcoin_get_balance` iterates over +all UTXOs, i.e., the same balance is returned as when calling +link:#ic-bitcoin_get_utxos[`bitcoin_get_utxos`] +for the same address and the same number of confirmations and, if necessary, +using pagination to get all UTXOs for the same tip hash. + +[#ic-bitcoin_send_transaction] +=== IC method `bitcoin_send_transaction` + +Given a `send_transaction_request`, which must specify a `blob` of a Bitcoin transaction and a Bitcoin network (`mainnet` or `testnet`), several checks are performed: + +- The transaction is well formed. +- The transaction only consumes unspent outputs with respect to the current (longest) +blockchain, i.e., there is no block on the (longest) chain that consumes any of these outputs. +- There is a positive transaction fee. + +If at least one of these checks fails, the call is rejected. + +If the transaction passes these tests, the transaction is forwarded to the +specified Bitcoin network. + +The Bitcoin component periodically forwards the transaction +until the transaction appears in a block appended to the blockchain or the transaction +times out after 24 hours. As soon as it appears in a block or expires, +the Bitcoin component drops the transaction. +It follows that the function does not provide any guarantees that a submitted +transaction will ever appear in a block. + +[#ic-bitcoin_get_current_fee_percentiles] +=== IC method `bitcoin_get_current_fee_percentiles` + +The transaction fees in the Bitcoin network change dynamically based on the number of +pending transactions. +It must be possible for a canister to determine an adequate fee when creating a Bitcoin transaction. + +This function returns the 100 fee percentiles, measured in millisatoshi/byte (10^3 millisatoshi = 1 satoshi), over the last 10,000 transactions, i.e., +over the transactions in the last approximately 4-10 blocks. + [#certification] == Certification From bee9a66c4b63ae7fdd8a55f35a085a8b2561040d Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Mon, 20 Jun 2022 14:42:33 +0200 Subject: [PATCH 014/102] Release to branch --- spec/index.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/index.adoc b/spec/index.adoc index 96d164f8c..f1cc49483 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,6 +1,6 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.4-wip +0.18.4 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ From 395ae75f09adbadcbaf14f8c49b6aee1e3120f2b Mon Sep 17 00:00:00 2001 From: Jens Groth Date: Mon, 20 Jun 2022 15:47:46 +0200 Subject: [PATCH 015/102] Update index.adoc Fix exponentiations 2^n -> 2^n^ --- spec/index.adoc | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/spec/index.adoc b/spec/index.adoc index f1cc49483..73a81ad5b 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -205,7 +205,7 @@ If an empty canister receives a response, that response is dropped, as if the ca ==== Canister cycles The IC relies on _cycles_, a utility token, to manage its resources. A canister pays for the resources it uses from its _cycle balance_. -The _cycle_balance_ is stored as 128-bit unsigned integers and operations on them are saturating. In particular, if _cycles_ are added to a canister that would bring its total balance beyond 2^128-1, then the balance will be capped at 2^128-1 and any additional cycles will be lost. +The _cycle_balance_ is stored as 128-bit unsigned integers and operations on them are saturating. In particular, if _cycles_ are added to a canister that would bring its total balance beyond 2^128^-1, then the balance will be capped at 2^128^-1 and any additional cycles will be lost. When the cycle balance of a canister falls to zero, the canister is _deallocated_. This has the same effect as @@ -1281,15 +1281,15 @@ The stable memory is initially empty. + returns the current size of the stable memory in WebAssembly pages. (One WebAssembly page is 64KiB) + -This system call traps if the size of the stable memory exceeds 2^32 bytes. +This system call traps if the size of the stable memory exceeds 2^32^ bytes. * `ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32)` + tries to grow the memory by `new_pages` many pages containing zeroes. + -This system call traps if the _previous_ size of the memory exceeds 2^32 bytes. +This system call traps if the _previous_ size of the memory exceeds 2^32^ bytes. + -If the _new_ size of the memory exceeds 2^32 bytes or growing is unsuccessful, then it returns `-1`. +If the _new_ size of the memory exceeds 2^32^ bytes or growing is unsuccessful, then it returns `-1`. + Otherwise, it grows the memory and returns the _previous_ size of the memory in pages. @@ -1297,7 +1297,7 @@ Otherwise, it grows the memory and returns the _previous_ size of the memory in + copies the data referred to by `src`/`size` out of the canister and replaces the corresponding segment starting at `offset` in the stable memory. + -This system call traps if the size of the stable memory exceeds 2^32 bytes. +This system call traps if the size of the stable memory exceeds 2^32^ bytes. + It also traps if `src+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. @@ -1305,7 +1305,7 @@ It also traps if `src+size` exceeds the size of the WebAssembly memory or `offse + copies the data referred to by `offset`/`size` out of the stable memory and replaces the corresponding bytes starting at `dest` in the canister memory. + -This system call traps if the size of the stable memory exceeds 2^32 bytes. +This system call traps if the size of the stable memory exceeds 2^32^ bytes. + It also traps if `dst+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory @@ -3974,7 +3974,7 @@ ic0.msg_reject(src : i32, size : i32) = es.cycles_available := 0 ic0.msg_cycles_available() : i64 = - if es.cycles_available >= 2^64 then Trap + if es.cycles_available >= 2^64^ then Trap return es.cycles_available ic0.msg_cycles_available128(dst : i32) = @@ -3982,7 +3982,7 @@ ic0.msg_cycles_available128(dst : i32) = copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) ic0.msg_cycles_refunded() : i64 = - if es.params.cycles_refunded >= 2^64 then Trap + if es.params.cycles_refunded >= 2^64^ then Trap return es.params.cycles_refunded ic0.msg_cycles_refunded128(dst : i32) = @@ -4007,7 +4007,7 @@ ic0.msg_cycles_accept(max_amount : i64) : i64 = return amount ic0.msg_cycles_accept128(max_amount_high : i64, max_amount_low : i64, dst : i32) = - let max_amount = max_amount_high * 2^64 + max_amount_low + let max_amount = max_amount_high * 2^64^ + max_amount_low let amount = min(max_amount, es.cycles_available) es.cycles_available := es.cycles_available - amount es.cycles_accepted := es.cycles_accepted + amount @@ -4021,7 +4021,7 @@ ic0.canister_self_copy(dst:i32, offset:i32, size:i32) = copy_to_canister(dst, offset, size, es.wasm_state.self_id) ic0.canister_cycle_balance() : i64 = - if es.balance >= 2^64 then Trap + if es.balance >= 2^64^ then Trap return es.balance ic0.canister_cycles_balance128(dst : i32) = @@ -4089,7 +4089,7 @@ ic0.call_cycles_add(amount : i64) = es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount ic0.call_cycles_add128(amount_high : i64, amount_low : i64) = - let amount = amount_high * 2^64 + amount_low + let amount = amount_high * 2^64^ + amount_low if es.pending_call = NoPendingCall then Trap if es.balance < amount then Trap @@ -4117,12 +4117,12 @@ discard_pending_call() = es.pending_call := NoPendingCall ic0.stable_size() : (page_count : i32) = - if |es.wasm_state.store.mem| > 2^32 then Trap + if |es.wasm_state.store.mem| > 2^32^ then Trap page_count := |es.wasm_state.stable_mem| / 64k return page_count ic0.stable_grow(new_pages : i32) : (old_page_count : i32) = - if |es.wasm_state.store.mem| > 2^32 then Trap + if |es.wasm_state.store.mem| > 2^32^ then Trap if arbitrary() then return -1 else old_size := |es.wasm_state.stable_mem| / 64k @@ -4132,14 +4132,14 @@ ic0.stable_grow(new_pages : i32) : (old_page_count : i32) = return old_size ic0.stable_write(offset : i32, src : i32, size : i32) - if |es.wasm_state.store.mem| > 2^32 then Trap + if |es.wasm_state.store.mem| > 2^32^ then Trap if src+size > |es.wasm_state.store.mem| then Trap if offset+size > |es.wasm_state.stable_mem| then Trap es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] ic0.stable_read(dst : i32, offset : i32, size : i32) - if |es.wasm_state.store.mem| > 2^32 then Trap + if |es.wasm_state.store.mem| > 2^32^ then Trap if offset+size > |es.wasm_state.stable_mem| then Trap if dst+size > |es.wasm_state.store.mem| then Trap From aa9c314ca69a01a8bd73e7bcaf836f7ab6ebaf8b Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Tue, 21 Jun 2022 09:59:17 +0200 Subject: [PATCH 016/102] Hotfix for Bitcoin --- spec/ic.did | 6 +++++- spec/index.adoc | 2 +- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/spec/ic.did b/spec/ic.did index 8958df886..e9856c442 100644 --- a/spec/ic.did +++ b/spec/ic.did @@ -49,6 +49,10 @@ type get_utxos_request = record { }; }; +type get_current_fee_percentiles_request = record { + network: bitcoin_network; +}; + type get_utxos_response = record { utxos: vec utxo; tip_block_hash: block_hash; @@ -113,7 +117,7 @@ service ic : { bitcoin_get_balance: (get_balance_request) -> (satoshi); bitcoin_get_utxos: (get_utxos_request) -> (get_utxos_response); bitcoin_send_transaction: (send_transaction_request) -> (); - bitcoin_get_current_fee_percentiles: () -> (vec millisatoshi_per_byte); + bitcoin_get_current_fee_percentiles: (get_current_fee_percentiles_request) -> (vec millisatoshi_per_byte); // provisional interfaces for the pre-ledger world provisional_create_canister_with_cycles : (record { diff --git a/spec/index.adoc b/spec/index.adoc index 73a81ad5b..f1eaa447e 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1730,7 +1730,7 @@ The transaction fees in the Bitcoin network change dynamically based on the numb pending transactions. It must be possible for a canister to determine an adequate fee when creating a Bitcoin transaction. -This function returns the 100 fee percentiles, measured in millisatoshi/byte (10^3 millisatoshi = 1 satoshi), over the last 10,000 transactions, i.e., +This function returns the 100 fee percentiles, measured in millisatoshi/byte (10^3 millisatoshi = 1 satoshi), over the last 10,000 transactions in the specified network, i.e., over the transactions in the last approximately 4-10 blocks. [#certification] From dccdbb6862886df9db9e82d84c40142d34e8009d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bj=C3=B6rn=20Tackmann?= <54846571+Dfinity-Bjoern@users.noreply.github.com> Date: Tue, 21 Jun 2022 12:03:08 +0200 Subject: [PATCH 017/102] Update spec/changelog.adoc Co-authored-by: Jens Groth <46536510+JensGroth@users.noreply.github.com> --- spec/changelog.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index fd54e02ab..26f712f51 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -215,7 +215,7 @@ [#v0_4_0] === 0.4.0 (2020-05-25) -* (editorial) the term “principal” is now used for the _id_ of a canister or +* (Editorial) the term “principal” is now used for the _id_ of a canister or user, not the canister or user itself * The signature of a request needs to be calculated using a domain separator * Describe the `controller` attribute, add a request to change it From 62e85dedf624aa3f3aede2fe4677f9bd7cd1a8c8 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Tue, 21 Jun 2022 15:37:20 +0200 Subject: [PATCH 018/102] Merge from release-0.18 --- .netlify.sh | 19 +++ CODEOWNERS | 2 +- LICENSE | 14 ++ README.md | 35 +++++ default.nix | 4 +- netlify.toml | 3 + spec/README.md | 4 + spec/changelog.adoc | 190 ++++++++--------------- spec/gen-antora.sh | 4 +- spec/ic.did | 75 +++++++++ spec/ic0.txt | 65 ++++++++ spec/index.adoc | 366 ++++++++++++++++++++++++++++---------------- 12 files changed, 520 insertions(+), 261 deletions(-) create mode 100755 .netlify.sh create mode 100644 LICENSE create mode 100644 README.md create mode 100644 netlify.toml create mode 100644 spec/README.md create mode 100644 spec/ic0.txt diff --git a/.netlify.sh b/.netlify.sh new file mode 100755 index 000000000..d91fcfdb3 --- /dev/null +++ b/.netlify.sh @@ -0,0 +1,19 @@ +#!/usr/bin/env bash + +# +# Deploy a preview of the interface specification to netlify +# + +set -e + +export NP_GIT=$(which git) + +wget -nv https://github.com/DavHau/nix-portable/releases/download/v009/nix-portable +chmod +x nix-portable + +./nix-portable nix-build -A interface-spec default.nix + +# The "result" symlink only valid inside the nix-portable sandbox, so use nix-shell +./nix-portable nix-shell -p bash --run "cp -rL result/spec/ _netlify-deploy" + +chmod -R u+w _netlify-deploy diff --git a/CODEOWNERS b/CODEOWNERS index 7d47bcfe6..68ef32cd0 100644 --- a/CODEOWNERS +++ b/CODEOWNERS @@ -1 +1 @@ -/spec/index.adoc @nomeata @jensgroth @Dfinity-Bjoern +** @dfinity/canister-os diff --git a/LICENSE b/LICENSE new file mode 100644 index 000000000..e2a36b348 --- /dev/null +++ b/LICENSE @@ -0,0 +1,14 @@ +Copyright 2021 DFINITY Foundation + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + diff --git a/README.md b/README.md new file mode 100644 index 000000000..3a1843e02 --- /dev/null +++ b/README.md @@ -0,0 +1,35 @@ +# Internet Computer Reference + +This repository contains the source files of the [Interface Spec], which describes the externally visible behaviour of the Internet Computer. + +It used to contain a reference implementation and acceptance test suite; these can now be found at . + +## About the Interface Spec + +This document describes the external interface of the Internet Computer. It is the authoritative source for interface details (request and function names, parameters, encodings). The goal is to have a document that is authoritative, and provides a place and a language to discuss external features of the Internet Computer in a hopefully concrete way. However, this document intentionally does not address _how_ to implement this behavior, and cannot be used as an implementation spec. + +## Versioning + +The Interface Spec is versioned, using a three-component version like + + 0.2.1 + +Releases from this repository are tagged using a three-component _code +version_ number: + + 0.8.1 + ┬ ┬ ┬ + │ │ └ The third component is bumped upon non-breaking changes to the spec. + │ └ The second component is bumped with a breaking change to the spec + └ Always zero for now. + +Each major spec version has a release branch (e.g. `release-0.8`) that only sees +non-breaking changes and bugfixes. A release branch should typically be “ahead” of all previous release branches. + +The `master` branch contains finished designs, but is not directly scheduled +for implementation. It lists version version number `∞`. The reference +implementation on this branch typically does _not_ fully implement the spec. This branch should always be “ahead” of all the release branches. + +## Contributing + +This repository currently does not accept external contributions. diff --git a/default.nix b/default.nix index e35311795..834388ac3 100644 --- a/default.nix +++ b/default.nix @@ -34,8 +34,8 @@ rec { -R $PWD -D $out/$doc_path/ index.adoc find . -type f -name '*.png' | cpio -pdm $out/$doc_path/ cp *.cddl $out/$doc_path - cp ic.did $out/$doc_path - + cp *.did $out/$doc_path + cp *.txt $out/$doc_path mkdir -p $out/nix-support echo "report spec $out/$doc_path index.html" >> $out/nix-support/hydra-build-products diff --git a/netlify.toml b/netlify.toml new file mode 100644 index 000000000..1d4a4ec08 --- /dev/null +++ b/netlify.toml @@ -0,0 +1,3 @@ +[build] + command = "./.netlify.sh" + publish = "_netlify-deploy" diff --git a/spec/README.md b/spec/README.md new file mode 100644 index 000000000..c626e12b6 --- /dev/null +++ b/spec/README.md @@ -0,0 +1,4 @@ +The Interface Spec +=============== + +This directory contains the sources to the IC Interface Spec. See the top-level README for more information about the Interface Spec. diff --git a/spec/changelog.adoc b/spec/changelog.adoc index d17e05ff2..26f712f51 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,74 +1,83 @@ [#changelog] == Changelog +[#0_18_4] +=== 0.18.4 (2022-06-20) + +* Canister cycle balances are represented by 128 bits, and no system-defined upper limit exists anymore +* Canister modules can be gzip-encoded +* Expose Wasm custom sections in the state tree +* EXPERIMENTAL: Canister API for accessing Bitcoin transactions +* EXPERIMENTAL: Canister API for threshold ECDSA signatures + [#0_18_3] === 0.18.3 (2022-01-10) -* Spec: New System API which uses 128-bit values to represent the amount of cycles -* Spec: Subnet delegations include a canister id scope +* New System API which uses 128-bit values to represent the amount of cycles +* Subnet delegations include a canister id scope [#0_18_2] === 0.18.2 (2021-09-29) -* Spec: Canister heartbeat -* Spec: Terminology changes -* Spec: Support for 64-bit stable memory +* Canister heartbeat +* Terminology changes +* Support for 64-bit stable memory [#0_18_1] === 0.18.1 (2021-08-04) -* Spec: Support RSA PKCS#1 v1.5 signatures in web authentication +* Support RSA PKCS#1 v1.5 signatures in web authentication * Spec clarification: Fix various typos and improve textual clarity [#0_18_0] === 0.18.0 (2021-05-18) -* Spec: A canister has a set of controllers, instead of always one +* A canister has a set of controllers, instead of always one [#0_17_0] === 0.17.0 (2021-04-22) -* Spec: Canister Signatures are introduced +* Canister Signatures are introduced * Spec clarification: the signature in the WebAuthn scheme is prefixed by the CBOR self-identifying tag -* Spec: Cycle-depleted canisters are forcibly uninstalled -* Spec: Canister settings in `create_canister` and `update_settings`. `install_code` no longer takes allocation settings. -* Spec: A freezing threshold can be configured via the canister settings +* Cycle-depleted canisters are forcibly uninstalled +* Canister settings in `create_canister` and `update_settings`. `install_code` no longer takes allocation settings. +* A freezing threshold can be configured via the canister settings [#0_16_1] === 0.16.1 (2021-04-14) -* Spec: The cleanup callback is introduced +* The cleanup callback is introduced [#0_16_0] === 0.16.0 (2021-03-25) -* Spec: New http v2 API that allows for stateless boundary nodes +* New http v2 API that allows for stateless boundary nodes [#0_15_6] === 0.15.6 (2021-03-25) -* Spec: The system may impose limits on the number of globals and functions -* Spec: No ingress messages towards empty canisters are accepted -* Spec: No ingress messages towards `raw_rand` and `deposit_cycles` are accepted -* Spec: A memory allocation of `0` means “best effort” +* The system may impose limits on the number of globals and functions +* No ingress messages towards empty canisters are accepted +* No ingress messages towards `raw_rand` and `deposit_cycles` are accepted +* A memory allocation of `0` means “best effort” [#0_15_5] === 0.15.5 (2021-03-11) -* Spec: deposit_cycles(): any caller allowed +* deposit_cycles(): any caller allowed [#0_15_4] === 0.15.4 (2021-03-04) -* Spec: Ingress message filtering -* Spec: Add ECDSA signatures on curve secp256k1 -* Spec: Clarify that the `ic0.data_certificate_present` system function may be +* Ingress message filtering +* Add ECDSA signatures on curve secp256k1 +* Clarify that the `ic0.data_certificate_present` system function may be called in all contexts. [#0_15_3] === 0.15.3 (2021-02-26) -* Spec: Expose module hash and controller via `read_state` +* Expose module hash and controller via `read_state` [#0_15_2] === 0.15.2 (2021-02-09) @@ -78,12 +87,12 @@ [#0_15_0] === 0.15.0 (2020-12-17) -* Spec: Support for raw Ed25519 keys is removed +* Support for raw Ed25519 keys is removed [#0_14_1] === 0.14.1 (2020-12-08) -* Spec: The default `memory_allocation` becomes unspecified +* The default `memory_allocation` becomes unspecified [#0_14_0] === 0.14.0 (2020-11-18) @@ -96,56 +105,50 @@ [#0_13_2] === 0.13.2 (2020-11-12) -* Spec: The `ic0.canister_status` system call +* The `ic0.canister_status` system call [#0_13_1] === 0.13.1 (2020-11-06) -* Spec: Delegation between user public keys +* Delegation between user public keys [#0_13_0] === 0.13.0 (2020-10-19) -* Spec: Certification (also removes “request-status” request) +* Certification (also removes “request-status” request) [#0_12_2] === 0.12.2 (2020-10-23) -* Spec: User authentication method based on WebAuthn is introduced -* Spec: User authentication can use ECDSA -* Spec: Public keys are DER-encoded +* User authentication method based on WebAuthn is introduced +* User authentication can use ECDSA +* Public keys are DER-encoded [#0_12_1] === 0.12.1 (2020-10-16) -* Spec: Return more information in the `canister_status` management call +* Return more information in the `canister_status` management call [#0_12_0] === 0.12.0 (2020-10-13) -* Spec: Anonymous requests must have the sender field set +* Anonymous requests must have the sender field set [#0_11_1] === 0.11.1 (2020-10-01) -* Spec: The `deposit_funds` call -* ic-ref(-test): Implement and test the above changes +* The `deposit_funds` call [#0_11_0] === 0.11.0 (2020-09-23) -* Spec: Inter-canister calls are now performed using a builder-like API -* Spec: Support for funds (balances and transfers) +* Inter-canister calls are now performed using a builder-like API +* Support for funds (balances and transfers) [#v0_10_3] === 0.10.3 (2020-09-21) -* Spec: The anonymous user is introduced - -[#v0_10_2] -=== 0.10.2 (2020-09-08) - -* ic-ref(-test): Bugfix: A query response should _not_ include a `time` field +* The anonymous user is introduced [#v0_10_1] === 0.10.1 (2020-09-01) @@ -155,23 +158,19 @@ [#v0_10_0] === 0.10.0 (2020-08-06) -* Spec: Users can set/update a memory allocation when installing/upgrading a canister. -* Spec: The `expiry` field is added to requests -* ic-ref(-test): Implement and test the above changes, as far as possible +* Users can set/update a memory allocation when installing/upgrading a canister. +* The `expiry` field is added to requests [#v0_9_3] === 0.9.3 (2020-09-01) -* Spec: The management canister supports the `raw_rand` method -* ic-ref(-test): Implement and test the above changes +* The management canister supports the `raw_rand` method [#v0_9_2] === 0.9.2 (2020-08-05) -* Spec: Canister controllers can stop/start canisters and can query their status. -* Spec: Canister controllers can delete canisters -* ic-ref(-test): Implement and test the above changes -* ic-ref: refactorings +* Canister controllers can stop/start canisters and can query their status. +* Canister controllers can delete canisters [#v0_9_1] === 0.9.1 (2020-07-20) @@ -181,55 +180,46 @@ [#v0_9_0] === 0.9.0 (2020-07-15) -* Spec: Introduction of a domain separator (again) -* Spec: The calculation of “derived ids” has changed -* Spec: The self-authenticating and derived id forms use a truncated hash -* Spec: The textual representation of principals has changed -* ic-ref(-test): Implement the above changes -* ic-ref-test: Also send read requests with nonces +* Introduction of a domain separator (again) +* The calculation of “derived ids” has changed +* The self-authenticating and derived id forms use a truncated hash +* The textual representation of principals has changed [#v0_8_2] === 0.8.2 (2020-07-17) -* ic-ref-test: Also send read requests with nonces -* Spec: Installing code via `reinstall` works also on the empty canister -* ic-ref(-test): Implement and test the above changes +* Installing code via `reinstall` works also on the empty canister [#v0_8_1] === 0.8.1 (2020-07-10) * Reflect refined process in README and intro. -* Spec: `ic0.time` added -* ic-ref(-test): Implement and test `ic0.time` +* `ic0.time` added [#v0_8_0] === 0.8.0 (2020-06-23) -* Spec: Revert the introduction of a domain separator -* ic-ref(-test): Implement and test the above changes +* Revert the introduction of a domain separator [#v0_6_2] === 0.6.2 (2020-06-23) -* Spec: Fix meaning-changing typos in `ic.did` -* ic-ref-test: Be more liberal about the precise reject code in some cases. +* Fix meaning-changing typos in `ic.did` [#v0_6_0] === 0.6.0 (2020-06-08) -* Spec: Make all canister ids system-chosen -* Spec: HTTP requests for management features are removed -* ic-ref(-test): Implement and test the above changes +* Make all canister ids system-chosen +* HTTP requests for management features are removed [#v0_4_0] === 0.4.0 (2020-05-25) -* Spec (editorial): the term “principal” is now used for the _id_ of a canister or +* (Editorial) the term “principal” is now used for the _id_ of a canister or user, not the canister or user itself -* Spec: The signature of a request needs to be calculated using a domain separator -* Spec: Describe the `controller` attribute, add a request to change it -* Spec: The IC management canister is introduced -* ic-ref(-test): Implement and test the above changes +* The signature of a request needs to be calculated using a domain separator +* Describe the `controller` attribute, add a request to change it +* The IC management canister is introduced [#v0_2_16] === 0.2.16 (2020-05-29) @@ -239,63 +229,19 @@ [#v0_2_14] === 0.2.14 (2020-05-14) -* Spec: Bugfix: Mode should be `reinstall`, not `replace` -* ic-ref-test: A few more tests, refactorings - -[#v0_2_12] -=== 0.2.12 (2020-05-06) - -* ic-ref-test: Remove code to work around lack of creater canister. -* ic-ref-test: Stricter tests for bad signatures -* ic-ref: Also accept CBOR maps of indeterminate length - -[#v0_2_10] -=== 0.2.10 (2020-04-29) - -* ic-ref: Bind to 127.0.0.1 instead of 0.0.0.0 -* ic-ref: Set content-type even for error responses -* ic-ref-test: Tests related to query calls -* ic-ref-test: Test “reply after trap in prior callback” +* Bugfix: Mode should be `reinstall`, not `replace` [#v0_2_8] === 0.2.8 (2020-04-23) -* Spec: Include section with CDDL description -* ic-ref-test: Block less tests on `create_canister` support - -[#v0_2_6] -=== 0.2.6 (2020-04-01) - -* ic-ref-run: Accept any canister id in `install` commands -* ic-ref-test: More defensive printing of HTTP bodies +* Include section with CDDL description [#v0_2_4] === 0.2.4 (2020-03-23) * simplify versioning (only three components), skip 0.2.2 to avoid confusion with 0.2.0.2 -* spec: Clarification: `reply` field is always present -* spec: General cleanup based on front-to-back reading -* ic-ref(-test): Enforce signature checking -* ic-ref(-test): Desired canister ids must be derived from sender -* ic-ref(-test): Require the 55799 semantic CBOR tag, as specified -* ic-ref: Ignore duplicate requests -* ic-ref-test: Run more tests independent of each other (try `-j 8`) -* ic-ref-test: Submit requests with nonces -* ic-ref-test: Test various trap conditions in reply and reject callbacks. -* ic-ref-test: Test that `ic0.debug_print` with invalid bounds does _not_ trap -* ic-ref-test: Allow unspecified fields to appear in the status response -* ic-ref-test: Canister upgrade tests - -[#v0_2_0_2] -=== 0.2.0.2 (2020-03-19) - -* ic-ref: Return status 202, empty body, on `submit`, to match spec -* ic-ref: Allow update or inter-canister calls to query methods -* ic-ref: Trap upon calls from queries -* ic-ref-test: If the IC does not claim to be spec compliant, always succeed - (but still report errors) -* ic-ref-test: Support --html reports -* ic-ref-test: Use the “Universal Canister” to drive tests; more tests. +* Clarification: `reply` field is always present +* General cleanup based on front-to-back reading [#v0_2_0_0] === 0.2.0.0 (2020-03-11) diff --git a/spec/gen-antora.sh b/spec/gen-antora.sh index 289b1a8cf..1bad3c2c1 100755 --- a/spec/gen-antora.sh +++ b/spec/gen-antora.sh @@ -50,7 +50,7 @@ __END__ cp -v "$in/index.adoc" "$out/modules/interface-spec/pages/index.adoc" cp -v "$in/changelog.adoc" "$out/modules/interface-spec/partials/changelog.adoc" -cp -v "$in"/*.did "$in"/*.cddl "$out/modules/interface-spec/attachments" -cp -v "$in"/*.did "$in"/*.cddl "$out/modules/interface-spec/examples" +cp -v "$in"/*.did "$in"/*.cddl "$in"/*.txt "$out/modules/interface-spec/attachments" +cp -v "$in"/*.did "$in"/*.cddl "$in"/*.txt "$out/modules/interface-spec/examples" diff --git a/spec/ic.did b/spec/ic.did index 5b64725d1..e9856c442 100644 --- a/spec/ic.did +++ b/spec/ic.did @@ -16,6 +16,63 @@ type definite_canister_settings = record { freezing_threshold : nat; }; +type ecdsa_curve = variant { secp256k1; }; + +type satoshi = nat64; + +type bitcoin_network = variant { + mainnet; + testnet; +}; + +type bitcoin_address = text; + +type block_hash = blob; + +type outpoint = record { + txid : blob; + vout : nat32 +}; + +type utxo = record { + outpoint: outpoint; + value: satoshi; + height: nat32; +}; + +type get_utxos_request = record { + address : bitcoin_address; + network: bitcoin_network; + filter: opt variant { + min_confirmations: nat32; + page: blob; + }; +}; + +type get_current_fee_percentiles_request = record { + network: bitcoin_network; +}; + +type get_utxos_response = record { + utxos: vec utxo; + tip_block_hash: block_hash; + tip_height: nat32; + next_page: opt blob; +}; + +type get_balance_request = record { + address : bitcoin_address; + network: bitcoin_network; + min_confirmations: opt nat32; +}; + +type send_transaction_request = record { + transaction: blob; + network: bitcoin_network; +}; + +type millisatoshi_per_byte = nat64; + service ic : { create_canister : (record { settings : opt canister_settings @@ -44,6 +101,24 @@ service ic : { deposit_cycles : (record {canister_id : canister_id}) -> (); raw_rand : () -> (blob); + // Threshold ECDSA signature + ecdsa_public_key : (record { + canister_id : opt canister_id; + derivation_path : vec blob; + key_id : record { curve: ecdsa_curve; name: text }; + }) -> (record { public_key : blob; chain_code : blob; }); + sign_with_ecdsa : (record { + message_hash : blob; + derivation_path : vec blob; + key_id : record { curve: ecdsa_curve; name: text }; + }) -> (record { signature : blob }); + + // bitcoin interface + bitcoin_get_balance: (get_balance_request) -> (satoshi); + bitcoin_get_utxos: (get_utxos_request) -> (get_utxos_response); + bitcoin_send_transaction: (send_transaction_request) -> (); + bitcoin_get_current_fee_percentiles: (get_current_fee_percentiles_request) -> (vec millisatoshi_per_byte); + // provisional interfaces for the pre-ledger world provisional_create_canister_with_cycles : (record { amount: opt nat; diff --git a/spec/ic0.txt b/spec/ic0.txt new file mode 100644 index 000000000..53e9d2716 --- /dev/null +++ b/spec/ic0.txt @@ -0,0 +1,65 @@ +ic0.msg_arg_data_size : () -> i32; // I U Q Ry F +ic0.msg_arg_data_copy : (dst : i32, offset : i32, size : i32) -> (); // I U Q Ry F +ic0.msg_caller_size : () -> i32; // I G U Q F +ic0.msg_caller_copy : (dst : i32, offset: i32, size : i32) -> (); // I G U Q F +ic0.msg_reject_code : () -> i32; // Ry Rt +ic0.msg_reject_msg_size : () -> i32; // Rt +ic0.msg_reject_msg_copy : (dst : i32, offset : i32, size : i32) -> (); // Rt + +ic0.msg_reply_data_append : (src : i32, size : i32) -> (); // U Q Ry Rt +ic0.msg_reply : () -> (); // U Q Ry Rt +ic0.msg_reject : (src : i32, size : i32) -> (); // U Q Ry Rt + +ic0.msg_cycles_available : () -> i64; // U Rt Ry +ic0.msg_cycles_available128 : (dst : i32) -> (); // U Rt Ry +ic0.msg_cycles_refunded : () -> i64; // Rt Ry +ic0.msg_cycles_refunded128 : (dst : i32) -> (); // Rt Ry +ic0.msg_cycles_accept : (max_amount : i64) -> (amount : i64); // U Rt Ry +ic0.msg_cycles_accept128 : (max_amount_high : i64, max_amount_low: i64, dst : i32) + -> (); // U Rt Ry + +ic0.canister_self_size : () -> i32; // * +ic0.canister_self_copy : (dst : i32, offset : i32, size : i32) -> (); // * +ic0.canister_cycle_balance : () -> i64; // * +ic0.canister_cycle_balance128 : (dst : i32) -> (); // * +ic0.canister_status : () -> i32; // * + +ic0.msg_method_name_size : () -> i32; // F +ic0.msg_method_name_copy : (dst : i32, offset : i32, size : i32) -> (); // F +ic0.accept_message : () -> (); // F + +ic0.call_new : // U Ry Rt H + ( callee_src : i32, + callee_size : i32, + name_src : i32, + name_size : i32, + reply_fun : i32, + reply_env : i32, + reject_fun : i32, + reject_env : i32 + ) -> (); +ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt H +ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt H +ic0.call_cycles_add : (amount : i64) -> (); // U Ry Rt H +ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt H +ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt H + +ic0.stable_size : () -> (page_count : i32); // * +ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32); // * +ic0.stable_write : (offset : i32, src : i32, size : i32) -> (); // * +ic0.stable_read : (dst : i32, offset : i32, size : i32) -> (); // * +ic0.stable64_size : () -> (page_count : i64); // * +ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64); // * +ic0.stable64_write : (offset : i64, src : i64, size : i64) -> (); // * +ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> (); // * + +ic0.certified_data_set : (src: i32, size: i32) -> (); // I G U Ry Rt H +ic0.data_certificate_present : () -> i32; // * +ic0.data_certificate_size : () -> i32; // * +ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> (); // * + +ic0.time : () -> (timestamp : i64); // * +ic0.performance_counter : (counter_type : i32) -> (counter : i64); // * s + +ic0.debug_print : (src : i32, size : i32) -> (); // * s +ic0.trap : (src : i32, size : i32) -> (); // * s \ No newline at end of file diff --git a/spec/index.adoc b/spec/index.adoc index 0d1617ef0..f1eaa447e 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,6 +1,6 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.3 +0.18.4 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ @@ -70,7 +70,7 @@ The public entry points of canisters are called _methods_. Methods can be declar Methods can be _called_, from _caller_ to _callee_, and will eventually incur a _response_ which is either a _reply_ or a _reject_. A method may have _parameters_, which are provided with concrete _arguments_ in a method call. External calls can be update calls, which can call both kinds of methods, and query calls, which can _only_ call query methods. -Inter-canister calls can also call both kinds of methods. Note that calls from a canister to itself also count as "inter-canister". +Inter-canister calls can also call both kinds of methods. Note that calls from a canister to itself also count as "inter-canister". Internally, a call or a response is transmitted as a _message_ from a _sender_ to a _receiver_. Messages do not have a response. @@ -96,9 +96,9 @@ Amount of cycles that a canister has to have before a message is attempted to be + Amount of cycles that the IC sets aside when a canister performs a call. This is used to pay for processing the response message, and unused cycles after the execution of the response are refunded. See <>. -* `MAX_CANISTER_BALANCE` +* `DEFAULT_PROVISIONAL_CYCLES_BALANCE` + -Maximum canister cycle balance. Any excess is discarded. Less than 2^128^. +Amount of cycles allocated to a new canister by default, if not explicitly specified. See <>. [#principal] === Principals @@ -205,6 +205,7 @@ If an empty canister receives a response, that response is dropped, as if the ca ==== Canister cycles The IC relies on _cycles_, a utility token, to manage its resources. A canister pays for the resources it uses from its _cycle balance_. +The _cycle_balance_ is stored as 128-bit unsigned integers and operations on them are saturating. In particular, if _cycles_ are added to a canister that would bring its total balance beyond 2^128^-1, then the balance will be capped at 2^128^-1 and any additional cycles will be lost. When the cycle balance of a canister falls to zero, the canister is _deallocated_. This has the same effect as @@ -417,7 +418,7 @@ The certified data of the canister with the given id, see </module_hash` (blob): + @@ -429,6 +430,12 @@ If the canister is not empty, it exists and contains the SHA256 hash of the curr + The current controllers of the canister. The value consists of a CBOR data item with major type 6 (“Semantic tag”) and tag value `55799` (see https://tools.ietf.org/html/rfc7049#section-2.4.5[Self-Describe CBOR]), followed by an array of principals in their binary form (CDDL `#6.55799([* bytes .size (0..29)])`). +* `/canister//metadata/` (blob): ++ +If the canister has a https://webassembly.github.io/spec/core/binary/modules.html#custom-section[custom section] called `icp:public ` or `icp:private `, this path contains the content of the custom section. Otherwise, this path does not exist. ++ +It is recommended for the canister to have a custom section called "icp:public candid:service", which contains the UTF-8 encoding of https://github.com/dfinity/candid/blob/master/spec/Candid.md#core-grammar[the Candid interface] for the canister. + [#http-interface] == HTTPS Interface @@ -547,6 +554,7 @@ All requested paths must have one of the following paths as prefix: * `/request_status/`. Can only be read if `/request_id/` is not present in the state tree, or if this `read_state` request was signed by same sender as the original request referenced by ``, and the effective canister id of the original request matches the `` (see <>) in this HTTP request’s path. * `/canisters//module_hash`. Can be requested by anyone, if `` matches ``. * `/canisters//controllers`. Can be requested by anyone, if `` matches ``. The order may vary depending on the implementation. + * `/canisters//metadata/`. Can be read by anyone if `` matches ``, and `` is a public custom section. If `` is a private custom section, it can only be read if this `read_state` request was signed by one of the controllers of the canister. Note that the paths `/canisters//certified_data` are not accessible with this method; these paths are only exposed to the canister themselves via the System API (see <>). @@ -934,72 +942,10 @@ In the reply callback of a <>, the a The following sections describe various System API functions, also referred to as system calls, which we summarize here. -.... -ic0.msg_arg_data_size : () -> i32; // I U Q Ry F -ic0.msg_arg_data_copy : (dst : i32, offset : i32, size : i32) -> (); // I U Q Ry F -ic0.msg_caller_size : () -> i32; // I G U Q F -ic0.msg_caller_copy : (dst : i32, offset: i32, size : i32) -> (); // I G U Q F -ic0.msg_reject_code : () -> i32; // Ry Rt -ic0.msg_reject_msg_size : () -> i32; // Rt -ic0.msg_reject_msg_copy : (dst : i32, offset : i32, size : i32) -> (); // Rt - -ic0.msg_reply_data_append : (src : i32, size : i32) -> (); // U Q Ry Rt -ic0.msg_reply : () -> (); // U Q Ry Rt -ic0.msg_reject : (src : i32, size : i32) -> (); // U Q Ry Rt - -ic0.msg_cycles_available : () -> i64; // U Rt Ry -ic0.msg_cycles_available128 : (dst : i32) -> (); // U Rt Ry -ic0.msg_cycles_refunded : () -> i64; // Rt Ry -ic0.msg_cycles_refunded128 : (dst : i32) -> (); // Rt Ry -ic0.msg_cycles_accept : (max_amount : i64) -> ( amount : i64 ); // U Rt Ry -ic0.msg_cycles_accept128 : (max_amount_high : i64, max_amount_low: i64, dst : i32) - -> (); // U Rt Ry - -ic0.canister_self_size : () -> i32; // * -ic0.canister_self_copy : (dst : i32, offset : i32, size : i32) -> (); // * -ic0.canister_cycle_balance : () -> i64; // * -ic0.canister_cycle_balance128 : (dst : i32) -> (); // * -ic0.canister_status : () -> i32; // * - -ic0.msg_method_name_size : () -> i32 // F -ic0.msg_method_name_copy : (dst : i32, offset : i32, size : i32) -> (); // F -ic0.accept_message : () -> (); // F - -ic0.call_new : // U Ry Rt H - ( callee_src : i32, - callee_size : i32, - name_src : i32, - name_size : i32, - reply_fun : i32, - reply_env : i32, - reject_fun : i32, - reject_env : i32 - ) -> (); -ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt H -ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt H -ic0.call_cycles_add : (amount : i64) -> (); // U Ry Rt H -ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt H -ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt H - -ic0.stable_size : () -> (page_count : i32); // * -ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32); // * -ic0.stable_write : (offset : i32, src : i32, size : i32) -> (); // * -ic0.stable_read : (dst : i32, offset : i32, size : i32) -> (); // * -ic0.stable64_size : () -> (page_count : i64); // * -ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64); // * -ic0.stable64_write : (offset : i64, src : i64, size : i64) -> (); // * -ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> (); // * - -ic0.certified_data_set : (src: i32, size: i32) -> () // I G U Ry Rt H -ic0.data_certificate_present : () -> i32 // * -ic0.data_certificate_size : () -> i32 // * -ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> () // * - -ic0.time : () -> (timestamp : i64); // * - -ic0.debug_print : (src : i32, size : i32) -> (); // * s -ic0.trap : (src : i32, size : i32) -> (); // * s -.... +[source,rust] +---- +include::{example}ic0.txt[] +---- The comment after each function lists from where these functions may be invoked: @@ -1230,8 +1176,8 @@ NOTE: This call traps if the current balance does not fit into a 64-bit value. C * `ic0.canister_cycle_balance128 : (dst : i32) -> ()` + -Indicates the current cycle balance of the canister by copying the value to the location `dst` in the canister memory. -The value is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add128`. +Indicates the current cycle balance of the canister by copying the value at the location `dst` in the canister memory. +It is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add128`. After execution of the message, the IC may add unused cycles from the reserve back to the balance. + This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. @@ -1247,7 +1193,7 @@ NOTE: This call traps if the amount of cycles available does not fit into a 64-b * `ic0.msg_cycles_available128 : (dst : i32) -> ()` + Indicates the number of cycles transferred by the caller of the current call, still available in this message. -The amount of cycles is represented by a 128-bit value. This call copies this value starting to the location `dst` in the canister memory. +The amount of cycles is represented by a 128-bit value. This call copies this value starting at the location `dst` in the canister memory. + Initially, in the update method entry point, this is the amount that the caller passed to the canister. When cycles are accepted (`ic0.msg_cycles_accept128`), this reports fewer cycles accordingly. When the call is responded to (reply or reject), all available cycles are refunded to the caller, and this will report 0 cycles. + @@ -1263,8 +1209,6 @@ This moves cycles from the call to the canister balance. It moves as many cycles * It moves no more cycles than available according to `ic0.msg_cycles_available`, and -* The canister balance afterwards does not exceed `MAX_CANISTER_BALANCE` minus any possible outstanding balances. The precise logic of this limit is not yet specified here. - It can be called multiple times, each time possibly adding more cycles to the balance. The return value indicates how many cycles were actually moved. @@ -1286,8 +1230,6 @@ This moves cycles from the call to the canister balance. It moves as many cycles * It moves no more cycles than available according to `ic0.msg_cycles_available128`, and -* The canister balance afterwards does not exceed `MAX_CANISTER_BALANCE` minus any possible outstanding balances. The precise logic of this limit is not yet specified here. - It can be called multiple times, each time possibly adding more cycles to the balance. This call also copies the amount of cycles that were actually moved starting at the location `dst` in the canister memory. @@ -1339,15 +1281,15 @@ The stable memory is initially empty. + returns the current size of the stable memory in WebAssembly pages. (One WebAssembly page is 64KiB) + -This system call traps if the size of the stable memory exceeds 2^32 bytes. +This system call traps if the size of the stable memory exceeds 2^32^ bytes. * `ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32)` + tries to grow the memory by `new_pages` many pages containing zeroes. + -This system call traps if the _previous_ size of the memory exceeds 2^32 bytes. +This system call traps if the _previous_ size of the memory exceeds 2^32^ bytes. + -If the _new_ size of the memory exceeds 2^32 bytes or growing is unsuccessful, then it returns `-1`. +If the _new_ size of the memory exceeds 2^32^ bytes or growing is unsuccessful, then it returns `-1`. + Otherwise, it grows the memory and returns the _previous_ size of the memory in pages. @@ -1355,7 +1297,7 @@ Otherwise, it grows the memory and returns the _previous_ size of the memory in + copies the data referred to by `src`/`size` out of the canister and replaces the corresponding segment starting at `offset` in the stable memory. + -This system call traps if the size of the stable memory exceeds 2^32 bytes. +This system call traps if the size of the stable memory exceeds 2^32^ bytes. + It also traps if `src+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. @@ -1363,7 +1305,7 @@ It also traps if `src+size` exceeds the size of the WebAssembly memory or `offse + copies the data referred to by `offset`/`size` out of the stable memory and replaces the corresponding bytes starting at `dest` in the canister memory. + -This system call traps if the size of the stable memory exceeds 2^32 bytes. +This system call traps if the size of the stable memory exceeds 2^32^ bytes. + It also traps if `dst+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory @@ -1548,7 +1490,7 @@ Not including a setting in the `settings` record means not changing that field. This method installs code into a canister. -Only _controllers of the canister can install code. +Only controllers of the canister can install code. * If `mode = install`, the canister must be empty before. This will instantiate the canister module and invoke its `canister_init` method (if present), as explained in Section “<>”, passing the `arg` to the canister. @@ -1562,6 +1504,12 @@ This is atomic: If the response to this request is a `reject`, then this call ha NOTE: Some canisters may not be able to make sense of callbacks after upgrades; these should be stopped first, to wait for all outstanding callbacks, or be uninstalled first, to prevent outstanding callbacks from being invoked. It is expected that the canister admin (or their tooling) does that separately. +The `wasm_module` field specifies the canister module to be installed. +The system supports multiple encodings of the `wasm_module` field: + + * If the `wasm_module` starts with byte sequence `[0x00, 'a', 's', 'm']`, the system parses `wasm_module` as a raw WebAssembly binary. + * If the `wasm_module` starts with byte sequence `[0x1f, 0x8b, 0x08]`, the system decompresses the contents of `wasm_module` as a gzip stream according to https://datatracker.ietf.org/doc/html/rfc1952.html[RFC-1952] and then parses the output as a WebAssembly binary. + [#ic-uninstall_code] === IC method `uninstall_code` @@ -1626,10 +1574,40 @@ There is no restriction on who can invoke this method. This method takes no input and returns 32 pseudo-random bytes to the caller. The return value is unknown to any part of the IC at time of the submission of this call. A new return value is generated for each call to this method. +[#ic-ecdsa_public_key] +=== IC method `ecdsa_public_key` + +NOTE: The ECDSA API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. + +This method returns a https://www.secg.org/sec1-v2.pdf[SEC1] encoded ECDSA public key for the given canister using the given derivation path. If the `canister_id` is unspecified, it will default to the canister id of the caller. +The `derivation_path` is a vector of variable length byte strings. +The `key_id` is a struct specifying both a curve and a name. +The availability of a particular `key_id` depends on implementation. + +For curve `secp256k1`, the public key is derived using a generalization of BIP32 (see https://ia.cr/2021/1330[ia.cr/2021/1330, Appendix D]). To derive (non-hardened) https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki[BIP-0032]-compatible public keys, each byte string (`blob`) in the `derivation_path` must be a 4-byte big-endian encoding of an unsigned integer less than 2^31^. + +The return result is an extended public key consisting of an ECDSA `public_key`, encoded in https://www.secg.org/sec2-v2.pdf[SEC1] compressed form, and a `chain_code`, which can be used to deterministically derive child keys of the `public_key`. + +This call requires that the ECDSA feature is enabled, and the `canister_id` meets the requirement of a canister id. +Otherwise it will be rejected. + +[#ic-sign_with_ecdsa] +=== IC method `sign_with_ecdsa` + +NOTE: The ECDSA API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. + +This method returns a new https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf[ECDSA] signature of the given `message_hash` that can be separately verified against a derived ECDSA public key. +This public key can be obtained by calling `ecdsa_public_key` with the caller's `canister_id`, and the same `derivation_path` and `key_id` used here. + +The signatures are encoded as the concatenation of the https://www.secg.org/sec2-v2.pdf[SEC1] encodings of the two values r and s. For curve `secp256k1`, this corresponds to 32-byte big-endian encoding. + +This call requires that the ECDSA feature is enabled, the caller is a canister, and `message_hash` is 32 bytes long. +Otherwise it will be rejected. + [#ic-provisional_create_canister_with_cycles] === IC method `provisional_create_canister_with_cycles` -As a provisional method on development instances, the `provisional_create_canister_with_cycles` method is provided. It behaves as `create_canister`, but initializes the canister’s balance with `amount` fresh cycles (using `MAX_CANISTER_BALANCE` if `amount = null`, else capping the balance at `MAX_CANISTER_BALANCE`). +As a provisional method on development instances, the `provisional_create_canister_with_cycles` method is provided. It behaves as `create_canister`, but initializes the canister’s balance with `amount` fresh cycles (using `DEFAULT_PROVISIONAL_CYCLES_BALANCE` if `amount = null`). Cycles added to this call via `ic0.call_cycles_add128` are returned to the caller. @@ -1638,7 +1616,7 @@ This method is only available in local development instances. [#ic-provisional_top_up_canister] === IC method `provisional_top_up_canister` -As a provisional method on development instances, the `provisional_top_up_canister` method is provided. It adds `amount` cycles to the balance of canister identified by `amount` (implicitly capping it at `MAX_CANISTER_BALANCE`). +As a provisional method on development instances, the `provisional_top_up_canister` method is provided. It adds `amount` cycles to the balance of canister identified by `amount`. Cycles added to this call via `ic0.call_cycles_add128` are returned to the caller. @@ -1646,6 +1624,115 @@ Any user can top-up any canister this way. This method is only available in local development instances. + +[#ic-bitcoin-api] +== The IC Bitcoin API + +NOTE: The IC Bitcoin API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. + +The Bitcoin functionality is exposed via the management canister. +Information about Bitcoin can be found in the https://developer.bitcoin.org/devguide/[Bitcoin developer guides]. +Invoking the functions of the Bitcoin API will cost cycles. The concrete costs for each function are yet to be determined. + +[#ic-bitcoin_get_utxos] +=== IC method `bitcoin_get_utxos` + +Given a `get_utxos_request`, which must specify a Bitcoin address and a Bitcoin network (`mainnet` or `testnet`), the function returns all unspent transaction outputs (UTXOs) associated with the +provided address in the specified Bitcoin network based on the current view of the Bitcoin blockchain available to the Bitcoin component. +The UTXOs are returned sorted by block height in descending order. + +The following address formats are supported: + +* Pay to public key hash (P2PKH) +* Pay to script hash (P2SH) +* Pay to witness public key hash (P2WPKH) +* Pay to witness script hash (P2WSH) +* Pay to taproot (P2TR) + +If the address is malformed, the call is rejected. + +The optional `filter` parameter can be used to restrict the set of returned UTXOs, either providing a +minimum number of confirmations or a page reference when pagination is used for addresses with many UTXOs. +In the first case, only UTXOs with at least the provided number of confirmations +are returned, i.e., transactions with fewer than this number of +confirmations are not considered. In other words, if the number of confirmations is `c`, an output +is returned if it occurred in a transaction with at least `c` confirmations +and there is no transaction that spends the same output with at least `c` confirmations. + +There is an upper bound of 144 on the minimum number of confirmations. +If a larger minimum number of confirmations is specified, the call is rejected. +Note that this is not a severe restriction as the minimum number of confirmations is typically set +to a value around 6 in practice. + +It is important to note that the validity of transactions is not verified in the Bitcoin component. The Bitcoin component relies on the +proof of work that goes into the blocks and the verification of the blocks in the Bitcoin network. +For a newly discovered block, a regular Bitcoin (full) node therefore provides a higher level of security +than the Bitcoin component, which implies that it is advisable to set the number of confirmations to a reasonably large value, such as 6, +to gain confidence in the correctness of the returned UTXOs. + +There is an upper bound of 100,000 UTXOs that can be returned in a single request. For addresses that contain sufficiently large UTXOs, +the partial set of the address' UTXOs are returned along with a page reference. + +In the second case, a page reference (a series of bytes) must be provided, which instructs the Bitcoin component to +collect UTXOs starting from the corresponding "page". + +A `get_utxos_request` without the optional `filter` results in a request that +considers the full blockchain, which is equivalent to setting +`min_confirmations` to 0. + +The recommended workflow is to issue a request with the desired number of confirmations. +If the `next_page` field in the response is not empty, there are more UTXOs than in the returned vector. +In that case, the `page` field should be set to the `next_page` bytes in the subsequent request to obtain the next batch of UTXOs. + +[#ic-bitcoin_get_balance] +=== IC method `bitcoin_get_balance` + +Given a `get_balance_request`, which must specify a Bitcoin address and a Bitcoin network (`mainnet` or `testnet`), the function returns the current balance of this address in `Satoshi` (10^8 Satoshi = 1 Bitcoin) in the specified Bitcoin network. The same address formats as for link:#ic-bitcoin_get_utxos[`bitcoin_get_utxos`] are supported. + +If the address is malformed, the call is rejected. + +The optional `min_confirmations` parameter can be used to limit the set of considered UTXOs +for the calculation of the balance to those with at least the provided number of confirmations +in the same manner as for the link:#ic-bitcoin_get_utxos[`bitcoin_get_utxos`] call. + +Given an address and the optional `min_confirmations` parameter, `bitcoin_get_balance` iterates over +all UTXOs, i.e., the same balance is returned as when calling +link:#ic-bitcoin_get_utxos[`bitcoin_get_utxos`] +for the same address and the same number of confirmations and, if necessary, +using pagination to get all UTXOs for the same tip hash. + +[#ic-bitcoin_send_transaction] +=== IC method `bitcoin_send_transaction` + +Given a `send_transaction_request`, which must specify a `blob` of a Bitcoin transaction and a Bitcoin network (`mainnet` or `testnet`), several checks are performed: + +- The transaction is well formed. +- The transaction only consumes unspent outputs with respect to the current (longest) +blockchain, i.e., there is no block on the (longest) chain that consumes any of these outputs. +- There is a positive transaction fee. + +If at least one of these checks fails, the call is rejected. + +If the transaction passes these tests, the transaction is forwarded to the +specified Bitcoin network. + +The Bitcoin component periodically forwards the transaction +until the transaction appears in a block appended to the blockchain or the transaction +times out after 24 hours. As soon as it appears in a block or expires, +the Bitcoin component drops the transaction. +It follows that the function does not provide any guarantees that a submitted +transaction will ever appear in a block. + +[#ic-bitcoin_get_current_fee_percentiles] +=== IC method `bitcoin_get_current_fee_percentiles` + +The transaction fees in the Bitcoin network change dynamically based on the number of +pending transactions. +It must be possible for a canister to determine an adequate fee when creating a Bitcoin transaction. + +This function returns the 100 fee percentiles, measured in millisatoshi/byte (10^3 millisatoshi = 1 satoshi), over the last 10,000 transactions in the specified network, i.e., +over the transactions in the last approximately 4-10 blocks. + [#certification] == Certification @@ -1787,9 +1874,9 @@ Delegation = A chain of delegations is verified using the following algorithm, which also returns the delegated key (a DER-encoded BLS key): .... -check_delegations(NoDelegation) : public_bls_key = +check_delegation(NoDelegation) : public_bls_key = return root_public_key -check_delegations(Delegation d) : public_bls_key = +check_delegation(Delegation d) : public_bls_key = verify_cert(d.certificate) return lookup(["subnet",d.subnet_id,"public_key"],d.certificate) .... @@ -2012,7 +2099,7 @@ CanisterModule = { post_upgrade : (CanisterId, StableMemory, Arg, Env) -> Trap | Return WasmState update_methods : MethodName ↦ ((Arg, Env, AvailableCycles) -> UpdateFunc) query_methods : MethodName ↦ ((Arg, Env) -> QueryFunc) - heartbeat : (Env) -> Trap | Return NoResponse + heartbeat : (Env) -> WasmState -> Trap | Return WasmState callbacks : (Callback, Response, RefundedCycles, Env, AvailableCycles) -> UpdateFunc inspect_message : (MethodName, WasmState, Arg, Env) -> Trap | Return (Accept | Reject) } @@ -2200,6 +2287,8 @@ CanState wasm_state : WasmState; module : CanisterModule; raw_module : Blob; + public_custom_sections: Text ↦ Blob; + private_custom_sections: Text ↦ Blob; } CanStatus = Running @@ -2483,9 +2572,9 @@ S with call_context = Ctxt_id; receiver = C; entry_point = Heartbeat; - queue = Queue { from = System; to = C }; + queue = Queue { from = System; to = C }; } - · S.messages + · S.messages call_contexts[Ctxt_id] = { canister = C; origin = FromHeartbeat; @@ -2512,7 +2601,7 @@ Conditions:: S.canisters[M.receiver] ≠ EmptyCanister Mod = S.canisters[M.receiver].module - Is_response = M.entry_point == Callback _ _ + Is_response = M.entry_point == Callback _ _ _ Env = { time = S.time[M.receiver]; @@ -2525,17 +2614,17 @@ Conditions:: Available = S.call_contexts[M.call_contexts].available_cycles ( M.entry_point = PublicMethod Name Caller Data Arg = { data = Data; caller = Caller } - (F = M.update_methods[M.method_name](Arg, Env, Available) + (F = Mod.update_methods[Name](Arg, Env, Available) or - (F = as_update(Mod.query_methods[M.method_name], Arg, Env)) + (F = query_as_update(Mod.query_methods[Name], Arg, Env)) ) or - ( M.entry_point = Callback Callback Response - F = Mod.callbacks(Callback, Response, Env, Available) + ( M.entry_point = Callback Callback Response RefundedCycles + F = Mod.callbacks(Callback, Response, RefundedCycles, Env, Available) ) or ( M.entry_point = Heartbeat - F = Mod.heartbeat(Env) + F = heartbeat_as_update(Mod.heartbeat, Env) ) R = F(S.canisters[M.receiver].wasm_state) @@ -2547,13 +2636,11 @@ if Cycles_used ≤ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) res.cycles_accepted ≤ Available New_balance = - min( - S.balances[M.receiver] - + res.cycles_accepted - + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - - Cycles_used - - ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ] - , MAX_CANISTER_BALANCE) + S.balances[M.receiver] + + res.cycles_accepted + + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + - Cycles_used + - ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ] New_balance > if Is_response then 0 else freezing_limit(S, CM.callee); (res.response = NoResponse) or S.call_contexts[M.call_context].needs_to_respond then @@ -2567,7 +2654,7 @@ then call_context = M.call_context; callback = call.callback }; - caller = C.callee; + caller = M.receiver; callee = call.callee; method_name = call.method_name; arg = call.arg; @@ -2704,7 +2791,7 @@ Conditions:: (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) M.callee = ic_principal M.method_name = 'create_canister' - M.arg = candid() + M.arg = candid(A) is_system_assigned CanisterId CanisterId ∉ dom S.canisters .... @@ -2716,7 +2803,7 @@ S with if A.settings.controllers is not null: controllers[CanisterId] = A.settings.controllers else: - controllers[CanisterId] = [M.caller] + controllers[CanisterId] = [M.caller] balances[CanisterId] = M.transferred_cycles certified_data[CanisterId] = "" messages = Older_messages · Younger_messages · @@ -3124,7 +3211,7 @@ State after:: .... S with balances[CanisterId] = - min(S.balances[A.canister_id] + M.transferred_cycles, MAX_CANISTER_BALANCE) + S.balances[A.canister_id] + M.transferred_cycles messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin @@ -3179,7 +3266,7 @@ S with canisters[CanisterId] = EmptyCanister time[CanisterId] = CurrentTime controllers[CanisterId] = [M.caller] - balances[CanisterId] = min(A.amount, MAX_CANISTER_BALANCE) + balances[CanisterId] = A.amount certified_data[CanisterId] = "" messages = Older_messages · Younger_messages · ResponseMessage { @@ -3204,7 +3291,7 @@ Conditions:: State after:: .... S with - balances[CanisterId] = min(balances[CanisterId] + A.amount, MAX_CANISTER_BALANCE) + balances[CanisterId] = balances[CanisterId] + A.amount .... ==== Callback invocation @@ -3226,12 +3313,11 @@ State after:: .... S with balances[S.call_contexts[Ctxt_id].canister] = - min(balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles, - MAX_CANISTER_BALANCE) + balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles messages = Older_messages · FuncMessage { - call_context = Ctxt_id2 + call_context = Ctxt_id receiver = C entry_point = Callback Callback FM.response RM.refunded_cycles queue = Unordered @@ -3255,8 +3341,7 @@ State after:: .... S with balances[S.call_contexts[Ctxt_id].canister] = - min(balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles + MAX_CYCLES_PER_RESPONSE, - MAX_CANISTER_BALANCE) + balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles + MAX_CYCLES_PER_RESPONSE messages = Older_messages · Younger_messages .... @@ -3458,6 +3543,12 @@ may_read_path(S, _, ["request_status", Rid] · _) = else True may_read_path(S, _, ["canister", cid, "module_hash"]) = cid == ECID may_read_path(S, _, ["canister", cid, "controllers"]) = cid == ECID +may_read_path(S, _, ["canister", cid, "metadata", name]) = + if name ∈ S.canisters[cid].public_custom_sections + then cid == ECID + else if name ∈ S.canisters[cid].private_custom_sections + then cid == ECID ∧ RS.sender ∈ S.controllers[cid] + else False may_read_path(S, _, _) = False .... @@ -3473,7 +3564,8 @@ state_tree(S) = { "canister": { canister_id : { "module_hash" : SHA256(C.raw_module) | if C ≠ EmptyCanister } ∪ - { "controllers" : CBOR(S.controllers[canister_id]) } + { "controllers" : CBOR(S.controllers[canister_id]) } ∪ + { "metadata": { name: blob | (name, blob) ∈ S.canisters[canister_id].public_custom_sections ∪ S.canisters[canister_id].private_custom_sections } } | (canister_id, C) ∈ S.canisters }; "subnet": { subnet_id : { "public_key" : pub } | (subnet_id, subnet_pk) ∈ subnets }; } @@ -3535,7 +3627,7 @@ Params = { reject_code : 0 | SYS_FATAL | SYS_TRANSIENT | …; reject_message : Text; sysenv : Env; - cycles_refundend : Nat; + cycles_refunded : Nat; method_name : Text; } ExecutionState = { @@ -3572,7 +3664,7 @@ empty_params = { caller = NoCaller; reject_code = 0; reject_message = ""; - cycles_refundend = 0; + cycles_refunded = 0; } empty_execution_state = { @@ -3710,7 +3802,7 @@ This formulation checks afterwards that the system call `ic0.call_perform` was n + By construction, the (possibly modified) `es.wasm_state` is discarded. -* The partial map `heartbeat` of the `CanisterModule` is defined if the WebAssembly program exports a function `func` named `canister_heartbeat`, and has value +* The function `heartbeat` of the `CanisterModule` is defined if the WebAssembly program exports a function `func` named `canister_heartbeat`, and has value + .... heartbeat = λ (sysenv) → λ wasm_state → @@ -3723,16 +3815,20 @@ heartbeat = λ (sysenv) → λ wasm_state → if es.cycles_accepted ≠ 0 then Trap if es.ingress_filter ≠ Reject then Trap if es.response ≠ NoResponse then Trap - Return NoResponse; + Return es.wasm_state; +.... +otherwise it is +.... +heartbeat = λ (sysenv) → λ wasm_state → Trap .... * The function `callbacks` of the `CanisterModule` is defined as follows + .... -callbacks = λ(callbacks, response, sysenv, available) → λ wasm_state → +callbacks = λ(callbacks, response, refunded_cycles, sysenv, available) → λ wasm_state → let params0 = { empty_params with sysenv - cycles_received = refund; + cycles_refunded = refund_cycles; } let (fun, env, params) = match response with Reply data -> @@ -3823,9 +3919,11 @@ copy_to_canister(dst : i32, offset : i32, size : i32, data : blob) = copy_from_canister(src : i32, size : i32) blob = if src+size > |es.wasm_state.store.mem| then Trap return es.wasm_state.store.mem[src..src+size] +.... Cycles are represented by 128-bit values so they require 16 bytes of memory. +.... copy_cycles_to_canister(dst : i32, data : blob) = let size = 16; if dst+size > |es.wasm_state.store.mem| then Trap @@ -3876,7 +3974,7 @@ ic0.msg_reject(src : i32, size : i32) = es.cycles_available := 0 ic0.msg_cycles_available() : i64 = - if es.cycles_available >= 2^64 then Trap + if es.cycles_available >= 2^64^ then Trap return es.cycles_available ic0.msg_cycles_available128(dst : i32) = @@ -3884,7 +3982,7 @@ ic0.msg_cycles_available128(dst : i32) = copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) ic0.msg_cycles_refunded() : i64 = - if es.params.cycles_refunded >= 2^64 then Trap + if es.params.cycles_refunded >= 2^64^ then Trap return es.params.cycles_refunded ic0.msg_cycles_refunded128(dst : i32) = @@ -3902,15 +4000,15 @@ ic0.msg_method_name_copy(dst : i32, offset : i32, size : i32) : i32 = copy_to_canister(dst, offset, size, es.params.method_name) ic0.msg_cycles_accept(max_amount : i64) : i64 = - let amount = min(max_amount, es.cycles_available, MAX_CANISTER_BALANCE - es.balance) + let amount = min(max_amount, es.cycles_available) es.cycles_available := es.cycles_available - amount es.cycles_accepted := es.cycles_accepted + amount es.balance := es.balance + amount return amount ic0.msg_cycles_accept128(max_amount_high : i64, max_amount_low : i64, dst : i32) = - let max_amount = max_amount_high * 2^64 + max_amount_low - let amount = min(max_amount, es.cycles_available, MAX_CANISTER_BALANCE - es.balance) + let max_amount = max_amount_high * 2^64^ + max_amount_low + let amount = min(max_amount, es.cycles_available) es.cycles_available := es.cycles_available - amount es.cycles_accepted := es.cycles_accepted + amount es.balance := es.balance + amount @@ -3923,7 +4021,7 @@ ic0.canister_self_copy(dst:i32, offset:i32, size:i32) = copy_to_canister(dst, offset, size, es.wasm_state.self_id) ic0.canister_cycle_balance() : i64 = - if es.balance >= 2^64 then Trap + if es.balance >= 2^64^ then Trap return es.balance ic0.canister_cycles_balance128(dst : i32) = @@ -3991,7 +4089,7 @@ ic0.call_cycles_add(amount : i64) = es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount ic0.call_cycles_add128(amount_high : i64, amount_low : i64) = - let amount = amount_high * 2^64 + amount_low + let amount = amount_high * 2^64^ + amount_low if es.pending_call = NoPendingCall then Trap if es.balance < amount then Trap @@ -4015,16 +4113,16 @@ ic0.call_peform() : ( err_code : i32 ) = // helper function discard_pending_call() = if es.pending_call ≠ NoPendingCall then - es.balance := min(es.balance + MAX_CYCLES_PER_RESPONSE + es.pending_call.transferred_cycles, MAX_CANISTER_BALANCE) + es.balance := es.balance + MAX_CYCLES_PER_RESPONSE + es.pending_call.transferred_cycles es.pending_call := NoPendingCall ic0.stable_size() : (page_count : i32) = - if |es.wasm_state.store.mem| > 2^32 then Trap + if |es.wasm_state.store.mem| > 2^32^ then Trap page_count := |es.wasm_state.stable_mem| / 64k return page_count ic0.stable_grow(new_pages : i32) : (old_page_count : i32) = - if |es.wasm_state.store.mem| > 2^32 then Trap + if |es.wasm_state.store.mem| > 2^32^ then Trap if arbitrary() then return -1 else old_size := |es.wasm_state.stable_mem| / 64k @@ -4034,14 +4132,14 @@ ic0.stable_grow(new_pages : i32) : (old_page_count : i32) = return old_size ic0.stable_write(offset : i32, src : i32, size : i32) - if |es.wasm_state.store.mem| > 2^32 then Trap + if |es.wasm_state.store.mem| > 2^32^ then Trap if src+size > |es.wasm_state.store.mem| then Trap if offset+size > |es.wasm_state.stable_mem| then Trap es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] ic0.stable_read(dst : i32, offset : i32, size : i32) - if |es.wasm_state.store.mem| > 2^32 then Trap + if |es.wasm_state.store.mem| > 2^32^ then Trap if offset+size > |es.wasm_state.stable_mem| then Trap if dst+size > |es.wasm_state.store.mem| then Trap From d6cb09fec82b62895f3e5719b6114e684f167912 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Fri, 8 Jul 2022 09:28:39 +0200 Subject: [PATCH 019/102] Cut 0.18.5 --- README.md | 2 +- spec/changelog.adoc | 6 ++ spec/ic.did | 1 + spec/index.adoc | 190 +++++++++++++++++++++++++++++++++++++++++++- 4 files changed, 194 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 3a1843e02..25bac53e3 100644 --- a/README.md +++ b/README.md @@ -32,4 +32,4 @@ implementation on this branch typically does _not_ fully implement the spec. Thi ## Contributing -This repository currently does not accept external contributions. +This repository accepts external contributions, conditioned on acceptance of the [https://github.com/dfinity/cla/](Contributor Lincense Agreement). diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 26f712f51..47eddf347 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,12 @@ [#changelog] == Changelog +[#0_18_5] +=== 0.18.5 (2022-07-08) +* Idle consumption of resources in cycles per day can be obtain via `canister_status` method of the management canister +* Include the HTTP Gateway Protocol in this spec +* Clarifications in definition of cycles consumption + [#0_18_4] === 0.18.4 (2022-06-20) diff --git a/spec/ic.did b/spec/ic.did index e9856c442..8a4f91dc0 100644 --- a/spec/ic.did +++ b/spec/ic.did @@ -96,6 +96,7 @@ service ic : { module_hash: opt blob; memory_size: nat; cycles: nat; + idle_cycles_burned_per_day: nat; }); delete_canister : (record {canister_id : canister_id}) -> (); deposit_cycles : (record {canister_id : canister_id}) -> (); diff --git a/spec/index.adoc b/spec/index.adoc index f1eaa447e..07168890d 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1604,6 +1604,42 @@ The signatures are encoded as the concatenation of the https://www.secg.org/sec2 This call requires that the ECDSA feature is enabled, the caller is a canister, and `message_hash` is 32 bytes long. Otherwise it will be rejected. +[#ic-http_request] +=== IC method `http_request` + +This method makes an HTTP request to a given URL and returns the HTTP response, possibly after a transformation. + +The canister should aim to issue _idempotent_ requests, meaning that it must not change the state at the remote server, or the remote server has the means to identify duplicated requests. Otherwise, the risk of failure increases. + +The responses for all identical requests must match too. However, a web service could return slightly different responses for identical idempotent requests. For example, it may include some unique identification or a timestamp that would vary across responses. + +For this reason, the calling canister can supply a transformation function, which the IC uses to let the canister sanitize the responses from such unique values. The transformation function is executed separately on the corresponding response received for a request. The final response will only be available to the calling canister. + +Currently, only the `GET`, `HEAD`, and `POST` methods are supported for HTTP requests. Note that when using `POST`, the calling canister must make sure that the remote server is able to handle idempotent requests sent from multiple sources. This may require, for example, to set a certain request header to uniquely identify the request. + +For security reasons, only HTTPS connections are allowed (URLs must start with `https://`). The IC uses industry-standard root CA lists to validate certificates of remote web servers. + +Each request can specify the maximal expected size for the response from the remote HTTP server. The upper limit on the size of a response defaults to `2MiB` if no maximal size value is specified. +An error will be returned when the response is larger than the maximal size. +The `2MiB` size limit also applies to the value returned by the `transform` function. + +The following parameters should be supplied for the call: + +- `url` - the requested URL +- `max_response_bytes` - optional, specifies the maximal size of the response in bytes. The call will be charged based on this parameter. If not provided, the maximum of `2MiB` will be used. +- `method` - currently, only GET, HEAD, and POST are supported +- `headers` - list of HTTP request headers and their corresponding values +- `transform` - an optional function that transforms raw responses to sanitized responses. If provided, the calling canister itself must export this function. + +The returned response (and the response provided to the `transform` function, if specified) contains the following fields: + +- `status` - the response status (e.g., 200, 404) +- `headers` - list of HTTP response headers and their corresponding values +- `body` - the response's body + +The `transform` function may, for example, transform the body in any way, add or remove headers, modify headers, etc. +When the transform function was invoked due to a canister HTTP request, the caller's identity is the principal of the management canister. + [#ic-provisional_create_canister_with_cycles] === IC method `provisional_create_canister_with_cycles` @@ -1970,6 +2006,146 @@ lookup_path(["d"], pruned_tree) = Found "morning" lookup_path(["e"], pruned_tree) = Absent .... +[#http-gateway] +== The HTTP Gateway protocol + +This section specifies the _HTTP Gateway protocol_, which allows canisters to handle conventional HTTP requests. + +This feature involves the help of a _HTTP Gateway_ that translates between HTTP requests and the IC protocol. Such a gateway could be a stand-alone proxy, it could be implemented in a web browsers (natively, via plugin or via a service worker) or in other ways. This document describes the interface and semantics of this protocol independent of a concrete Gateway, so that all Gateway implementations can be compatible. + +Conceptually, this protocol builds on top of the interface specified in the remainder of this document, and therefore is an “application-level” interface, not a feature of the core Internet Computer system described in the other sections, and could be a separate document. We nevertheless include this protocol in the Internet Computer Interface Specification because of its important role in the ecosystem and due to the importance of keeping multiple Gateway implementations in sync. + +=== Overview + +A HTTP request by an HTTP client is handled by these steps: + +1. The Gateway resolves the Host of the request to a canister id. +2. The Gateway Candid-encodes the HTTP request data. +3. The Gateway invokes the canister via a query call to `http_request`. +4. The canister handles the request and returns a HTTP response, encoded in Candid, together with additional metadata. +5. If requested by the canister, the Gateway sends the request again via an update call to `http_request_update`. +6. If applicable, the Gateway fetches further body data via streaming query calls. +7. If applicable, the Gateway validates the certificate of the response. +8. The Gateway sends the response to the HTTP client. + +[#http-gateway-interface] +=== Candid interface + +The following interface description, in https://github.com/dfinity/candid/blob/master/spec/Candid.md[Candid syntax], describes the expected Canister interface. You can also link:{attachmentsdir}/http-gateway.did[download the file]. +---- +include::{example}http-gateway.did[] +---- + +Only canisters that use the “Upgrade to update calls” feature need to provide the `http_request_update` method. + +NOTE: Canisters not using these features can completely leave out the `streaming_strategy` and/or `upgrade` fields in the `HttpResponse` they return, due to how Candid subtyping works. This might simplify their code. + +[#http-gateway-name-resolution] +=== Canister resolution + +The Gateway needs to know the canister id of the canister to talk to, and obtains that information from the hostname as follows: + +1. Check that the hostname, taken from the `Host` field of the HTTP request, is of the form `.raw.ic0.app` or `.ic0.app`, or fail. + +2. If the `` is in the following table, use the given canister ids: ++ +.Canister hostname resolution +|============================================ +| Hostname | Canister id +| `identity` | `rdmx6-jaaaa-aaaaa-aaadq-cai` +| `nns` | `qoctq-giaaa-aaaaa-aaaea-cai` +| `dscvr` | `h5aet-waaaa-aaaab-qaamq-cai` +| `personhood` | `g3wsl-eqaaa-aaaan-aaaaa-cai` +|============================================ + +3. Else, if `` is a valid textual encoding of a principal, use that principal as the canister id. + +4. Else fail. + +If the hostname was of the form `.ic0.app`, it is a _safe_ hostname; if it was of the form `.raw.ic0.app` it is a _raw_ hostname. + +=== Request encoding + +The HTTP request is encoded into the `HttpRequest` Candid structure. + +* The `method` field contains the HTTP method (e.g. `HTTP`), in upper case. + +* The `url` field contains the URL from the HTTP request line, i.e. without protocol or hostname, and including query parameters. + +* The `headers` field contains the headers of the HTTP request. + +* The `body` field contains the body of the HTTP request (without any content encodings processed by the Gateway). + +=== Upgrade to update calls + +If the canister sets `update = opt true` in the `HttpResponse` reply from `http_request`, then the Gateway ignores all other fields of the reply. The Gateway performs an _update_ call to `http_request_update`, passing the same `HttpRequest` record as the argument, and uses that response instead. + +The value of the `update` field returned from `http_request_update` is ignored. + +=== Response decoding + +The Gateway assembles the HTTP response from the given `HttpResponse` record: + +* The HTTP response status code is taken from the `status_code` field. + +* The HTTP response headers are taken from the `headers` field. ++ +NOTE: Not all Gateway implementations may be able to pass on all forms of headers. In particular, Service Workers are unable to pass on the `Set-Cookie` header. ++ +[NOTE] +==== +HTTP Gateways may add additional headers. In particular, the following headers may be set: + +.... +access-control-allow-origin: * +access-control-allow-methods: GET, POST, HEAD, OPTIONS +access-control-allow-headers: DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Cookie +access-control-expose-headers: Content-Length,Content-Range +x-cache-status: MISS +.... +==== + +* The HTTP response body is initialized with the value of the `body` field, and further assembled as per the <>. + +[#http-gateway-streaming] +=== Response body streaming + +The HTTP Gateway protocol has provisions to transfer further chunks of the body data from the canister to the HTTP Gateway, to overcome the message limit of the Internet Computer. This streaming protocol is independent of any possible streaming of data between the HTTP Gateway and the HTTP client. The gateway may assemble the response in whole before passing it on, or pass the chunks on directly, on the TCP or HTTP level, as it sees fit. When the Gateway is <>, it must not pass on uncertified chunks. + +If the `streaming_strategy` field of the `HttpResponse` is set, the HTTP Gateway then uses further query calls to obtain further chunks to append to the body: + +1. If the function reference in the `callback` field of the `streaming_strategy` is not a method of the given canister, the Gateway fails the request. + +2. Else, it makes a query call to the given method, passing the `token` value given in the `streaming_strategy` as the argument. + +3. That query method returns a `StreamingCallbackHttpResponse`. The `body` therein is appended to the body of the HTTP response. This is repeated as long as the method returns some token in the `token` field, until that field is `null`. + +WARNING: The type of the `token` value is chosen by the canister; the HTTP Gateway obtains the Candid type of the encoded message from the canister, and uses it when passing the token back to the canister. This generic use of Candid is not covered by the Candid specification, and may not be possible in some cases (e.g. when using “future types”). Canister authors may have to use “simple” types. + + +[#http-gateway-certification] +=== Response certification + +If the hostname was safe, the HTTP Gateway performs _certificate validation_: + +1. It searches for a response header called `Ic-Certificate` (case-insensitive). + +2. The value of the header must be a structured header according to RFC 8941 with fields `certificate` and `tree`, both being byte sequences. + +3. The `certificate` must be a valid certificate as per <>, signed by the root key. If the certificate contains a subnet delegation, the delegation must be valid for the given canister. The timestamp in `/time` must be recent. The subnet state tree in the certificate must reveal the canister’s <>. + +4. The `tree` must be a hash tree as per <>. + +5. The root hash of that `tree` must match the canister’s certified data. + +6. The path `["http_assets",]`, where `url` is the utf8-encoded `url` from the `HttpRequest` must exist and be a leaf. Else, if it does not exist, `["http_assets","/index.html"]` must exist and be a leaf. + +7. That leaf must contain the SHA-256 hash of the _decoded_ body. ++ +The decoded body is the body of the HTTP response (in particular, after assembling streaming chunks), decoded according to the `Content-Encoding` header, if present. Supported encodings for `Content-Encoding` are `gzip` and `deflate.` + +WARNING: The certification protocol only covers the mapping from request URL to response body. It completely ignores the request method and headers, and does not cover the response headers and status code. + [#abstract-behavior] == Abstract behavior @@ -2688,9 +2864,11 @@ else - Cycles_used .... -The cycle consumption of executing this message is modeled via the unspecified `Cycles_used` variable. -Depending whether this is a call message and a response messages, we have either set aside `MAX_CYCLES_PER_MESSAGE` or `MAX_CYCLES_PER_RESPONSE`, either in the call context creation rule or the Callback invocation rule. +Depending on whether this is a call message and a response messages, we have either set aside `MAX_CYCLES_PER_MESSAGE` or `MAX_CYCLES_PER_RESPONSE`, either in the call context creation rule or the Callback invocation rule. + +The cycle consumption of executing this message is modeled via the unspecified `Cycles_used` variable; the variable takes some value between 0 and `MAX_CYCLES_PER_MESSAGE`/`MAX_CYCLES_PER_RESPONSE` (for call execution and response execution, respectively). + This transition detects certain behavior that will appear as a trap (and which an implementation may implement by trapping directly in a system call): @@ -2699,7 +2877,7 @@ This transition detects certain behavior that will appear as a trap (and which a * Sending out more cycles than available to the canister * Consuming more cycles than allowed (and reserved) -If message execution <>, the message gets dropped. No response is generated (as some other message may still fulfill this calling context). Any state mutation is discarded. +If message execution <>, the message gets dropped. No response is generated (as some other message may still fulfill this calling context). Any state mutation is discarded. If the message was a call, the associated cycles are held by its associated call context and will be refunded to the caller, see <>. If message execution <>, the state is updated and possible outbound calls and responses are enqueued. @@ -2863,6 +3041,8 @@ The controllers of a canister can obtain information about the canister. The `Memory_size` is the (in this specification underspecified) total size of storage in bytes. +The `idle_cycles_burned_per_day` is the idle consumption of resources in cycles per day. Therefore, the freezing threshold in cycles can be obtained using the following formula: `freezing_threshold` (in seconds) * `idle_cycles_burned_per_day` / (3600 * 24) (seconds). + Conditions:: .... S.messages = Older_messages · CallMessage M · Younger_messages @@ -2887,6 +3067,8 @@ S with controllers = S.controllers[A.canister_id]; memory_size = Memory_size; cycles = S.balance[A.canister_id]; + freezing_threshold = S.freezing_threshold[A.canister_id]; + idle_cycles_burned_per_day = freezing_limit(S, A.canister_id) / freezing_threshold * 3600 * 24; }) refunded_cycles = M.transferred_cycles } @@ -3028,7 +3210,7 @@ We encode this behavior via three (types of) transitions: 1. First, any `stop_canister` call sets the state of the canister to `Stopping`; we record in the status the origin (and cycles) of all `stop_canister` calls which arrive at the canister while it is stopping (or stopped). 2. Next, when the canister has no open call contexts (so, in particular, all outstanding responses to the canister have been processed), the status of the canister is set to `Stopped`. -3. Finally, each pending `stop_canister` call (which are encoded in the status) is responded to, to indicate that that the canister is stopped. +3. Finally, each pending `stop_canister` call (which are encoded in the status) is responded to, to indicate that the canister is stopped. Conditions:: .... From 7517e90002a83a27385b2722d23795675c5db1b7 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Fri, 8 Jul 2022 14:59:38 +0200 Subject: [PATCH 020/102] Add did file, bump version --- spec/http-gateway.did | 41 +++++++++++++++++++++++++++++++++++++++++ spec/index.adoc | 2 +- 2 files changed, 42 insertions(+), 1 deletion(-) create mode 100644 spec/http-gateway.did diff --git a/spec/http-gateway.did b/spec/http-gateway.did new file mode 100644 index 000000000..4ec844f65 --- /dev/null +++ b/spec/http-gateway.did @@ -0,0 +1,41 @@ +type HeaderField = record { text; text; }; + +type HttpRequest = record { + method: text; + url: text; + headers: vec HeaderField; + body: blob; +}; + +type HttpResponse = record { + status_code: nat16; + headers: vec HeaderField; + body: blob; + upgrade : opt bool; + streaming_strategy: opt StreamingStrategy; +}; + +// Each canister that uses the streaming feature gets to choose their concrete +// type; the HTTP Gateway will treat it as an opaque value that is only fed to +// the callback method + +type StreamingToken = /* application-specific type */ + + +type StreamingCallbackHttpResponse = record { + body: blob; + token: opt StreamingToken; +}; + +type StreamingStrategy = variant { + Callback: record { + callback: func (StreamingToken) -> (opt StreamingCallbackHttpResponse) query; + token: StreamingToken; + }; +}; + +service : { + http_request: (request: HttpRequest) -> (HttpResponse) query; + http_request_update: (request: HttpRequest) -> (HttpResponse); +} + diff --git a/spec/index.adoc b/spec/index.adoc index 07168890d..e2ad3b736 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,6 +1,6 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.4 +0.18.5 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ From fd4c65971509c7f8c24d570647a88bc9a2cf86a9 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Fri, 8 Jul 2022 15:23:35 +0200 Subject: [PATCH 021/102] Release 0.18.5 --- README.md | 2 +- spec/changelog.adoc | 6 ++ spec/http-gateway.did | 41 +++++++++ spec/ic.did | 1 + spec/index.adoc | 192 ++++++++++++++++++++++++++++++++++++++++-- 5 files changed, 236 insertions(+), 6 deletions(-) create mode 100644 spec/http-gateway.did diff --git a/README.md b/README.md index 3a1843e02..25bac53e3 100644 --- a/README.md +++ b/README.md @@ -32,4 +32,4 @@ implementation on this branch typically does _not_ fully implement the spec. Thi ## Contributing -This repository currently does not accept external contributions. +This repository accepts external contributions, conditioned on acceptance of the [https://github.com/dfinity/cla/](Contributor Lincense Agreement). diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 26f712f51..47eddf347 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,12 @@ [#changelog] == Changelog +[#0_18_5] +=== 0.18.5 (2022-07-08) +* Idle consumption of resources in cycles per day can be obtain via `canister_status` method of the management canister +* Include the HTTP Gateway Protocol in this spec +* Clarifications in definition of cycles consumption + [#0_18_4] === 0.18.4 (2022-06-20) diff --git a/spec/http-gateway.did b/spec/http-gateway.did new file mode 100644 index 000000000..4ec844f65 --- /dev/null +++ b/spec/http-gateway.did @@ -0,0 +1,41 @@ +type HeaderField = record { text; text; }; + +type HttpRequest = record { + method: text; + url: text; + headers: vec HeaderField; + body: blob; +}; + +type HttpResponse = record { + status_code: nat16; + headers: vec HeaderField; + body: blob; + upgrade : opt bool; + streaming_strategy: opt StreamingStrategy; +}; + +// Each canister that uses the streaming feature gets to choose their concrete +// type; the HTTP Gateway will treat it as an opaque value that is only fed to +// the callback method + +type StreamingToken = /* application-specific type */ + + +type StreamingCallbackHttpResponse = record { + body: blob; + token: opt StreamingToken; +}; + +type StreamingStrategy = variant { + Callback: record { + callback: func (StreamingToken) -> (opt StreamingCallbackHttpResponse) query; + token: StreamingToken; + }; +}; + +service : { + http_request: (request: HttpRequest) -> (HttpResponse) query; + http_request_update: (request: HttpRequest) -> (HttpResponse); +} + diff --git a/spec/ic.did b/spec/ic.did index e9856c442..8a4f91dc0 100644 --- a/spec/ic.did +++ b/spec/ic.did @@ -96,6 +96,7 @@ service ic : { module_hash: opt blob; memory_size: nat; cycles: nat; + idle_cycles_burned_per_day: nat; }); delete_canister : (record {canister_id : canister_id}) -> (); deposit_cycles : (record {canister_id : canister_id}) -> (); diff --git a/spec/index.adoc b/spec/index.adoc index f1eaa447e..e2ad3b736 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,6 +1,6 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.4 +0.18.5 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ @@ -1604,6 +1604,42 @@ The signatures are encoded as the concatenation of the https://www.secg.org/sec2 This call requires that the ECDSA feature is enabled, the caller is a canister, and `message_hash` is 32 bytes long. Otherwise it will be rejected. +[#ic-http_request] +=== IC method `http_request` + +This method makes an HTTP request to a given URL and returns the HTTP response, possibly after a transformation. + +The canister should aim to issue _idempotent_ requests, meaning that it must not change the state at the remote server, or the remote server has the means to identify duplicated requests. Otherwise, the risk of failure increases. + +The responses for all identical requests must match too. However, a web service could return slightly different responses for identical idempotent requests. For example, it may include some unique identification or a timestamp that would vary across responses. + +For this reason, the calling canister can supply a transformation function, which the IC uses to let the canister sanitize the responses from such unique values. The transformation function is executed separately on the corresponding response received for a request. The final response will only be available to the calling canister. + +Currently, only the `GET`, `HEAD`, and `POST` methods are supported for HTTP requests. Note that when using `POST`, the calling canister must make sure that the remote server is able to handle idempotent requests sent from multiple sources. This may require, for example, to set a certain request header to uniquely identify the request. + +For security reasons, only HTTPS connections are allowed (URLs must start with `https://`). The IC uses industry-standard root CA lists to validate certificates of remote web servers. + +Each request can specify the maximal expected size for the response from the remote HTTP server. The upper limit on the size of a response defaults to `2MiB` if no maximal size value is specified. +An error will be returned when the response is larger than the maximal size. +The `2MiB` size limit also applies to the value returned by the `transform` function. + +The following parameters should be supplied for the call: + +- `url` - the requested URL +- `max_response_bytes` - optional, specifies the maximal size of the response in bytes. The call will be charged based on this parameter. If not provided, the maximum of `2MiB` will be used. +- `method` - currently, only GET, HEAD, and POST are supported +- `headers` - list of HTTP request headers and their corresponding values +- `transform` - an optional function that transforms raw responses to sanitized responses. If provided, the calling canister itself must export this function. + +The returned response (and the response provided to the `transform` function, if specified) contains the following fields: + +- `status` - the response status (e.g., 200, 404) +- `headers` - list of HTTP response headers and their corresponding values +- `body` - the response's body + +The `transform` function may, for example, transform the body in any way, add or remove headers, modify headers, etc. +When the transform function was invoked due to a canister HTTP request, the caller's identity is the principal of the management canister. + [#ic-provisional_create_canister_with_cycles] === IC method `provisional_create_canister_with_cycles` @@ -1970,6 +2006,146 @@ lookup_path(["d"], pruned_tree) = Found "morning" lookup_path(["e"], pruned_tree) = Absent .... +[#http-gateway] +== The HTTP Gateway protocol + +This section specifies the _HTTP Gateway protocol_, which allows canisters to handle conventional HTTP requests. + +This feature involves the help of a _HTTP Gateway_ that translates between HTTP requests and the IC protocol. Such a gateway could be a stand-alone proxy, it could be implemented in a web browsers (natively, via plugin or via a service worker) or in other ways. This document describes the interface and semantics of this protocol independent of a concrete Gateway, so that all Gateway implementations can be compatible. + +Conceptually, this protocol builds on top of the interface specified in the remainder of this document, and therefore is an “application-level” interface, not a feature of the core Internet Computer system described in the other sections, and could be a separate document. We nevertheless include this protocol in the Internet Computer Interface Specification because of its important role in the ecosystem and due to the importance of keeping multiple Gateway implementations in sync. + +=== Overview + +A HTTP request by an HTTP client is handled by these steps: + +1. The Gateway resolves the Host of the request to a canister id. +2. The Gateway Candid-encodes the HTTP request data. +3. The Gateway invokes the canister via a query call to `http_request`. +4. The canister handles the request and returns a HTTP response, encoded in Candid, together with additional metadata. +5. If requested by the canister, the Gateway sends the request again via an update call to `http_request_update`. +6. If applicable, the Gateway fetches further body data via streaming query calls. +7. If applicable, the Gateway validates the certificate of the response. +8. The Gateway sends the response to the HTTP client. + +[#http-gateway-interface] +=== Candid interface + +The following interface description, in https://github.com/dfinity/candid/blob/master/spec/Candid.md[Candid syntax], describes the expected Canister interface. You can also link:{attachmentsdir}/http-gateway.did[download the file]. +---- +include::{example}http-gateway.did[] +---- + +Only canisters that use the “Upgrade to update calls” feature need to provide the `http_request_update` method. + +NOTE: Canisters not using these features can completely leave out the `streaming_strategy` and/or `upgrade` fields in the `HttpResponse` they return, due to how Candid subtyping works. This might simplify their code. + +[#http-gateway-name-resolution] +=== Canister resolution + +The Gateway needs to know the canister id of the canister to talk to, and obtains that information from the hostname as follows: + +1. Check that the hostname, taken from the `Host` field of the HTTP request, is of the form `.raw.ic0.app` or `.ic0.app`, or fail. + +2. If the `` is in the following table, use the given canister ids: ++ +.Canister hostname resolution +|============================================ +| Hostname | Canister id +| `identity` | `rdmx6-jaaaa-aaaaa-aaadq-cai` +| `nns` | `qoctq-giaaa-aaaaa-aaaea-cai` +| `dscvr` | `h5aet-waaaa-aaaab-qaamq-cai` +| `personhood` | `g3wsl-eqaaa-aaaan-aaaaa-cai` +|============================================ + +3. Else, if `` is a valid textual encoding of a principal, use that principal as the canister id. + +4. Else fail. + +If the hostname was of the form `.ic0.app`, it is a _safe_ hostname; if it was of the form `.raw.ic0.app` it is a _raw_ hostname. + +=== Request encoding + +The HTTP request is encoded into the `HttpRequest` Candid structure. + +* The `method` field contains the HTTP method (e.g. `HTTP`), in upper case. + +* The `url` field contains the URL from the HTTP request line, i.e. without protocol or hostname, and including query parameters. + +* The `headers` field contains the headers of the HTTP request. + +* The `body` field contains the body of the HTTP request (without any content encodings processed by the Gateway). + +=== Upgrade to update calls + +If the canister sets `update = opt true` in the `HttpResponse` reply from `http_request`, then the Gateway ignores all other fields of the reply. The Gateway performs an _update_ call to `http_request_update`, passing the same `HttpRequest` record as the argument, and uses that response instead. + +The value of the `update` field returned from `http_request_update` is ignored. + +=== Response decoding + +The Gateway assembles the HTTP response from the given `HttpResponse` record: + +* The HTTP response status code is taken from the `status_code` field. + +* The HTTP response headers are taken from the `headers` field. ++ +NOTE: Not all Gateway implementations may be able to pass on all forms of headers. In particular, Service Workers are unable to pass on the `Set-Cookie` header. ++ +[NOTE] +==== +HTTP Gateways may add additional headers. In particular, the following headers may be set: + +.... +access-control-allow-origin: * +access-control-allow-methods: GET, POST, HEAD, OPTIONS +access-control-allow-headers: DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Cookie +access-control-expose-headers: Content-Length,Content-Range +x-cache-status: MISS +.... +==== + +* The HTTP response body is initialized with the value of the `body` field, and further assembled as per the <>. + +[#http-gateway-streaming] +=== Response body streaming + +The HTTP Gateway protocol has provisions to transfer further chunks of the body data from the canister to the HTTP Gateway, to overcome the message limit of the Internet Computer. This streaming protocol is independent of any possible streaming of data between the HTTP Gateway and the HTTP client. The gateway may assemble the response in whole before passing it on, or pass the chunks on directly, on the TCP or HTTP level, as it sees fit. When the Gateway is <>, it must not pass on uncertified chunks. + +If the `streaming_strategy` field of the `HttpResponse` is set, the HTTP Gateway then uses further query calls to obtain further chunks to append to the body: + +1. If the function reference in the `callback` field of the `streaming_strategy` is not a method of the given canister, the Gateway fails the request. + +2. Else, it makes a query call to the given method, passing the `token` value given in the `streaming_strategy` as the argument. + +3. That query method returns a `StreamingCallbackHttpResponse`. The `body` therein is appended to the body of the HTTP response. This is repeated as long as the method returns some token in the `token` field, until that field is `null`. + +WARNING: The type of the `token` value is chosen by the canister; the HTTP Gateway obtains the Candid type of the encoded message from the canister, and uses it when passing the token back to the canister. This generic use of Candid is not covered by the Candid specification, and may not be possible in some cases (e.g. when using “future types”). Canister authors may have to use “simple” types. + + +[#http-gateway-certification] +=== Response certification + +If the hostname was safe, the HTTP Gateway performs _certificate validation_: + +1. It searches for a response header called `Ic-Certificate` (case-insensitive). + +2. The value of the header must be a structured header according to RFC 8941 with fields `certificate` and `tree`, both being byte sequences. + +3. The `certificate` must be a valid certificate as per <>, signed by the root key. If the certificate contains a subnet delegation, the delegation must be valid for the given canister. The timestamp in `/time` must be recent. The subnet state tree in the certificate must reveal the canister’s <>. + +4. The `tree` must be a hash tree as per <>. + +5. The root hash of that `tree` must match the canister’s certified data. + +6. The path `["http_assets",]`, where `url` is the utf8-encoded `url` from the `HttpRequest` must exist and be a leaf. Else, if it does not exist, `["http_assets","/index.html"]` must exist and be a leaf. + +7. That leaf must contain the SHA-256 hash of the _decoded_ body. ++ +The decoded body is the body of the HTTP response (in particular, after assembling streaming chunks), decoded according to the `Content-Encoding` header, if present. Supported encodings for `Content-Encoding` are `gzip` and `deflate.` + +WARNING: The certification protocol only covers the mapping from request URL to response body. It completely ignores the request method and headers, and does not cover the response headers and status code. + [#abstract-behavior] == Abstract behavior @@ -2688,9 +2864,11 @@ else - Cycles_used .... -The cycle consumption of executing this message is modeled via the unspecified `Cycles_used` variable. -Depending whether this is a call message and a response messages, we have either set aside `MAX_CYCLES_PER_MESSAGE` or `MAX_CYCLES_PER_RESPONSE`, either in the call context creation rule or the Callback invocation rule. +Depending on whether this is a call message and a response messages, we have either set aside `MAX_CYCLES_PER_MESSAGE` or `MAX_CYCLES_PER_RESPONSE`, either in the call context creation rule or the Callback invocation rule. + +The cycle consumption of executing this message is modeled via the unspecified `Cycles_used` variable; the variable takes some value between 0 and `MAX_CYCLES_PER_MESSAGE`/`MAX_CYCLES_PER_RESPONSE` (for call execution and response execution, respectively). + This transition detects certain behavior that will appear as a trap (and which an implementation may implement by trapping directly in a system call): @@ -2699,7 +2877,7 @@ This transition detects certain behavior that will appear as a trap (and which a * Sending out more cycles than available to the canister * Consuming more cycles than allowed (and reserved) -If message execution <>, the message gets dropped. No response is generated (as some other message may still fulfill this calling context). Any state mutation is discarded. +If message execution <>, the message gets dropped. No response is generated (as some other message may still fulfill this calling context). Any state mutation is discarded. If the message was a call, the associated cycles are held by its associated call context and will be refunded to the caller, see <>. If message execution <>, the state is updated and possible outbound calls and responses are enqueued. @@ -2863,6 +3041,8 @@ The controllers of a canister can obtain information about the canister. The `Memory_size` is the (in this specification underspecified) total size of storage in bytes. +The `idle_cycles_burned_per_day` is the idle consumption of resources in cycles per day. Therefore, the freezing threshold in cycles can be obtained using the following formula: `freezing_threshold` (in seconds) * `idle_cycles_burned_per_day` / (3600 * 24) (seconds). + Conditions:: .... S.messages = Older_messages · CallMessage M · Younger_messages @@ -2887,6 +3067,8 @@ S with controllers = S.controllers[A.canister_id]; memory_size = Memory_size; cycles = S.balance[A.canister_id]; + freezing_threshold = S.freezing_threshold[A.canister_id]; + idle_cycles_burned_per_day = freezing_limit(S, A.canister_id) / freezing_threshold * 3600 * 24; }) refunded_cycles = M.transferred_cycles } @@ -3028,7 +3210,7 @@ We encode this behavior via three (types of) transitions: 1. First, any `stop_canister` call sets the state of the canister to `Stopping`; we record in the status the origin (and cycles) of all `stop_canister` calls which arrive at the canister while it is stopping (or stopped). 2. Next, when the canister has no open call contexts (so, in particular, all outstanding responses to the canister have been processed), the status of the canister is set to `Stopped`. -3. Finally, each pending `stop_canister` call (which are encoded in the status) is responded to, to indicate that that the canister is stopped. +3. Finally, each pending `stop_canister` call (which are encoded in the status) is responded to, to indicate that the canister is stopped. Conditions:: .... From 24ca9b8339b911643072f022766b5dc12b3aaed3 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Tue, 9 Aug 2022 14:24:05 +0200 Subject: [PATCH 022/102] Merge notational fixes, performance counter, Isabelle model --- .github/workflows/isabelle.yml | 23 + .gitignore | 1 + README.md | 20 + spec/index.adoc | 236 ++++++---- theories/IC.thy | 777 +++++++++++++++++++++++++++++++++ theories/ROOT | 8 + 6 files changed, 968 insertions(+), 97 deletions(-) create mode 100644 .github/workflows/isabelle.yml create mode 100644 theories/IC.thy create mode 100644 theories/ROOT diff --git a/.github/workflows/isabelle.yml b/.github/workflows/isabelle.yml new file mode 100644 index 000000000..2b2a018d6 --- /dev/null +++ b/.github/workflows/isabelle.yml @@ -0,0 +1,23 @@ +name: Build on Ubuntu +on: + push: + paths: + - 'theories/**' + +jobs: + isabelle: + runs-on: ubuntu-latest + steps: + - name: Check out the repo + uses: actions/checkout@v2 + - name: Run Isabelle with Docker + uses: addnab/docker-run-action@v3 + with: + image: martin2718/isabelle + options: -v ${{ github.workspace }}:/interface-spec + run: Isabelle/bin/isabelle build -e -v -D /interface-spec/theories/ + - name: Haskell Code + uses: actions/upload-artifact@v2 + with: + name: haskell-code + path: ${{ github.workspace }}/theories/code diff --git a/.gitignore b/.gitignore index 6f138f2cb..5db325438 100644 --- a/.gitignore +++ b/.gitignore @@ -8,3 +8,4 @@ spec/index.html spec/*.png spec/.asciidoctor/ impl/.tasty-rerun-log +theories/code diff --git a/README.md b/README.md index 25bac53e3..0117bb19e 100644 --- a/README.md +++ b/README.md @@ -30,6 +30,26 @@ The `master` branch contains finished designs, but is not directly scheduled for implementation. It lists version version number `∞`. The reference implementation on this branch typically does _not_ fully implement the spec. This branch should always be “ahead” of all the release branches. +## Formal Model + +We are developing a formal model of Interface Spec in the interactive theorem prover [Isabelle/HOL](https://isabelle.in.tum.de/). +The formal development is included in the directory `theories/`. + +To setup the environment, follow the standard [instructions](https://isabelle.in.tum.de/installation.html) for Isabelle/HOL. +Additionally, you may want to setup `isabelle` as an alias for the path `bin/isabelle` in your local Isabelle directory. + +To browse the formal model, open Isabelle/jEdit: +``` +isabelle jedit theories/IC.thy +``` +from the root directory of this repository. + +To build the formal model and export Haskell code from the formal model, run +``` +isabelle build -e -v -D theories/ +``` +in the root directory of this repository. The exported Haskell code can then be found under `theories/code/`. + ## Contributing This repository accepts external contributions, conditioned on acceptance of the [https://github.com/dfinity/cla/](Contributor Lincense Agreement). diff --git a/spec/index.adoc b/spec/index.adoc index e2ad3b736..20c892be0 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -229,9 +229,7 @@ In all cases, calls to the <> are pr The controllers of the canister can initiate transitions between these states using <> and <>, and query the state using <>. The canister itself can also query its state using <>. -NOTE: -This status is orthogonal to the question of whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected, but because the canister is empty. - +NOTE: This status is orthogonal to the question of whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected, but because the canister is empty. [#signatures] === Signatures @@ -543,7 +541,7 @@ The HTTP response to this request consists of a CBOR map with the following fiel * `certificate` (`blob`): A certificate (see <>). + -If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation’s canister id range (see <>). +If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation’s canister id range (see <>), unless the `effective_canister_id` is that of the Management Canister (`aaaaa-aa`). In other words, we define `aaaaa-aa` to belong to every canister range for the purposes of certificate checking. Note that `aaaaa-aa` is a valid canister id only for calls to `provisional_create_canister_with_cycles`, so it cannot appear in `read_state` requests on production networks. The returned certificate reveals all values whose path is a suffix of the list of requested paths. It also always reveals `/time`, even if not explicitly requested. @@ -591,11 +589,13 @@ Canister methods that do not change the canister state can be executed more effi The `` in the URL paths of requests is the _effective_ destination of the request. -* If the call is to the Management Canister (`aaaaa-aa`), and the `arg` is Candid-encoded where the first argument is a record with a `canister_id` field of type `principal`, then the effective canister id is that principal. +* If the call is to the Management Canister (`aaaaa-aa`), then: + - If the `arg` is Candid-encoded where the first argument is a record with a `canister_id` field of type `principal`, then the effective canister id is that principal. + - Otherwise, if the call is to the `provisional_create_canister_with_cycles` method, then any principal is a valid effective canister id for this call. + - Otherwise, there is no effective canister id. In particular, the `create_canister` method has no effective canister ID, and this method cannot be called by users, only via canisters. * If the call is to the `raw_rand` method of the Management Canister (`aaaaa-aa`), then there is no effective canister id. This implies that this method cannot be called by users, only via canisters. -* If the call is to the `provisional_create_canister_with_cycles` method of the Management Canister (`aaaaa-aa`), any principal is a valid effective canister id for this call. + [NOTE] ==== @@ -1351,9 +1351,26 @@ The time is given as nanoseconds since 1970-01-01. The IC guarantees that * the time, as observed by the canister, is monotonically increasing, even across canister upgrades. * within an invocation of one entry point, the time is constant. -The system times of different canisters are unrelated, and calls from one canister to another may appear to travel “backwards in time”. +The times observed by different canisters are unrelated, and calls from one canister to another may appear to travel “backwards in time”. + +NOTE: While an implementation will likely try to keep the time returned by `ic0.time` close to the real time, this is not formally part of this specification. + +[#system-api-performance-counter] +=== Performance counter + +The canister can query the "performance counter", which is a deterministic monotonically increasing integer approximating the amount of work the canister has done since the beginning of the current execution. + +`+ic0.performance_counter : (counter_type : i32) -> i64+` + +The argument `type` decides which performance counter to return: + +* 0 : instruction counter. The number of WebAssembly instructions the system has determined that the canister has executed. + +In the future, we might expose more performance counters. -NOTE: While an implementation will likely try to keep the System Time close to the real time, this is not formally part of this specification. +The system resets the counter at the beginning of each <> invocation. + +The main purpose of this counter is to facilitate in-canister performance profiling. [#system-api-certified-data] === Certified data @@ -2271,7 +2288,7 @@ RefundedCycles = Nat CanisterModule = { init : (CanisterId, Arg, Env) -> Trap | Return WasmState - pre_upgrade : (WasmState, caller : Principal, Env) -> Trap | Return StableMemory + pre_upgrade : (WasmState, Principal, Env) -> Trap | Return StableMemory post_upgrade : (CanisterId, StableMemory, Arg, Env) -> Trap | Return WasmState update_methods : MethodName ↦ ((Arg, Env, AvailableCycles) -> UpdateFunc) query_methods : MethodName ↦ ((Arg, Env) -> QueryFunc) @@ -2302,13 +2319,14 @@ The Internet Computer provides certain messaging guarantees: If a user or a cani To ensure that only one response is generated, and also to detect when no response can be generated any more, the IC maintains a _call context_. The `needs_to_respond` field is set to `false` once the call has received a response. Further attempts to respond will now fail. .... -CallCtxt = { - canister : CanisterId; - origin : CallOrigin; - needs_to_respond : bool; - deleted : bool; - available_cycles : Nat; -} +Request = { + nonce : Blob; + ingress_expiry : Nat; + sender : UserId; + canister_id : CanisterId; + method_name : Text; + data : Blob; + } CallId = (abstract) CallOrigin = FromUser { @@ -2319,6 +2337,13 @@ CallOrigin callback: Callback } | FromHeartbeat +CallCtxt = { + canister : CanisterId; + origin : CallOrigin; + needs_to_respond : bool; + deleted : bool; + available_cycles : Nat; +} .... ==== Calls and Messages @@ -2362,36 +2387,7 @@ A reference implementation would likely maintain a separate list of `messages` f ==== API requests -We distinguish between the _asynchronous_ API requests passed to `/api/v2/…/call`, which may be present in the IC state, and the _synchronous_ API requests passed to `/api/v2/…/read_state` and `/api/v2/…/query`, which are only ephemeral. - -.... -Envelope = { - content : Request | APIReadRequest; - sender_pubkey : PublicKey | NoPublicKey; - sender_sig : Signature | NoSignature; - sender_delegation: [SignedDelegation] -} - -Request - = CanisterUpdateCall = { - nonce : Blob; - ingress_expiry : Nat; - sender : UserId; - canister_id : CanisterId; - method_name : Text; - data : Blob; - } -.... - -The evolution of a `Request` goes through these states, as explained in <>: -.... -RequestStatus - = Received - | Processing - | Rejected (RejectCode, Text) - | Replied Blob - | Done -.... +We distinguish between the _asynchronous_ API requests (type `Request`) passed to `/api/v2/…/call`, which may be present in the IC state, and the _synchronous_ API requests passed to `/api/v2/…/read_state` and `/api/v2/…/query`, which are only ephemeral. These are the synchronous read messages: .... @@ -2413,10 +2409,29 @@ APIReadRequest } .... +.... +Envelope = { + content : Request | APIReadRequest; + sender_pubkey : PublicKey | NoPublicKey; + sender_sig : Signature | NoSignature; + sender_delegation: [SignedDelegation] +} +.... + +The evolution of a `Request` goes through these states, as explained in <>: +.... +RequestStatus + = Received + | Processing + | Rejected (RejectCode, Text) + | Replied Blob + | Done +.... + A `Path` may refer to a request by way of a _request id_, as specified in <>: .... -Request = Blob -hash_of_map: Request -> Request +RequestId = Blob +hash_of_map: Request -> RequestId .... For the signatures in a `Request`, we assume that the following function implements signature verification as described in <>. @@ -2444,6 +2459,18 @@ SignedDelegation = { Finally, we can describe the state of the IC as a record having the following fields: .... +CanState + = EmptyCanister | { + wasm_state : WasmState; + module : CanisterModule; + raw_module : Blob; + public_custom_sections: Text ↦ Blob; + private_custom_sections: Text ↦ Blob; +} +CanStatus + = Running + | Stopping (List (CallOrigin, Nat)) + | Stopped S = { requests : Request ↦ RequestStatus; canisters : CanisterId ↦ CanState; @@ -2458,18 +2485,6 @@ S = { messages : List Message; // ordered! root_key : PublicKey } -CanState - = EmptyCanister | { - wasm_state : WasmState; - module : CanisterModule; - raw_module : Blob; - public_custom_sections: Text ↦ Blob; - private_custom_sections: Text ↦ Blob; -} -CanStatus - = Running - | Stopping (List (CallOrigin, Nat)) - | Stopped .... ==== Initial state @@ -2481,8 +2496,10 @@ The initial state of the IC is canisters = (); controllers = (); freezing_threshold = (); + canister_status = (); time = (); balances = (); + certified_data = (); system_time = T; call_contexts = (); messages = (); @@ -2495,6 +2512,13 @@ for some time stamp `T`, some DER-encoded BLS public key `PublicKey`, and using The following is an incomplete list of invariants that should hold for the abstract state `S`, and are not already covered by the type annotations in this section. +* No method name is the name of an update and query method in a CanisterModule at the same time: ++ +.... +∀ _ ↦ CanState ∈ S.canisters: + dom(CanState.module.update_methods) ∩ dom(CanState.module.query_methods) = ∅ +.... + * Deleted call contexts were not awaiting a response: + .... @@ -2512,8 +2536,8 @@ The following is an incomplete list of invariants that should hold for the abstr * Referenced call contexts exists: + .... -∀ CallMessage M ∈ S.messages. M.origin.calling_context ∈ S.call_contexts -∀ ResponseMessage M ∈ S.messages. M.origin.calling_context ∈ S.call_contexts +∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ∈ S.call_contexts +∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ∈ S.call_contexts ∀ _ ↦ Ctxt ∈ S.call_contexts: if Ctx.needs_to_respond: Ctxt.origin.calling_context ∈ S.call_contexts @@ -2525,7 +2549,7 @@ Based on this abstract notion of the state, we can describe the behavior of the * Asynchronous API requests that are submitted via `/api/v2/…/call`. These transitions describe checks that the request must pass to be considered received. * Spontaneous transitions that model the internal behavior of the IC, by describing conditions on the state that allow the transition to happen, and the state after. - * Responses to reads (i.e. `/api/v2/…/read_state`). By definition, these do _not_ change the state of the IC, and merely describe the response based on the read request and the current state. + * Responses to reads (i.e. `/api/v2/…/read_state` and `/api/v2/…/query`). By definition, these do _not_ change the state of the IC, and merely describe the response based on the read request (or query, respectively) and the current state. The state transitions are not complete with regard to error handling. For example, the behavior of sending a request to a non-existent canister is not specified here. For now, we trust implementors to make sensible decisions there. @@ -2564,9 +2588,9 @@ delegation_targets(DS) A `Request` has an effective canister id according to the rules in <<#http-effective-canister-id>>: .... -is_effective_canister_id(CanisterUpdateCall {canister_id = ic_principal, arg = candid({canister_id = p, …}), …}, p) -is_effective_canister_id(CanisterUpdateCall {canister_id = ic_principal, method = provisional_create_canister_with_cycles, p) -is_effective_canister_id(CanisterUpdateCall {canister_id = p, …}, p), if p ≠ ic_principal +is_effective_canister_id(Request {canister_id = ic_principal, arg = candid({canister_id = p, …}), …}, p) +is_effective_canister_id(Request {canister_id = ic_principal, method = provisional_create_canister_with_cycles, p) +is_effective_canister_id(Request {canister_id = p, …}, p), if p ≠ ic_principal .... ==== API Request submission @@ -2591,7 +2615,7 @@ Conditions:: E.content.arg = candid({canister_id = CanisterId, …}) E.content.sender ∈ S.controllers[CanisterId] E.content.method_name ∈ - { "install_code", "set_controller", "start_canister", "stop_canister", + { "install_code", "update_settings", "start_canister", "stop_canister", "canister_status", "delete_canister" } ) ∨ ( E.content.canister_id ≠ ic_principal @@ -2634,17 +2658,17 @@ The IC does not make any guarantees about the order of incoming messages. Conditions:: .... - S.requests[CanisterUpdateCall R] = Received + S.requests[R] = Received S.system_time <= R.ingress_expiry C = S.canisters[R.canister_id] .... State after:: .... S with - requests[CanisterUpdateCall R] = Processing + requests[R] = Processing messages = CallMessage { - origin = FromUser { request = CanisterUpdateCall R }; + origin = FromUser { request = R }; caller = R.sender; callee = R.canister_id; method_name = R.method_name; @@ -2659,20 +2683,27 @@ S with A call to a canister which is stopping, stopped, or frozen is automatically rejected. -The (unspecified) function `freezing_limit(S, cid)` determines the freezing threshold in cycles of the canister with id `cid`, given its current memory footprint, storage cost, memory and compute allocation, and current `freezing_threshold` setting. +The (unspecified) function `idle_cycles_burned_rate(S, cid)` determines the idle resource consumption rate in cycles per day of the canister with id `cid`, +given its current memory footprint, compute and storage cost, and memory and compute allocation. +The function `freezing_limit(S, cid)` determines the freezing threshold in cycles of the canister with id `cid`, +given its current memory footprint, compute and storage cost, memory and compute allocation, and current `freezing_threshold` setting. +The value `freezing_limit(S, cid)` is derived from `idle_cycles_burned_rate(S, cid)` and `freezing_threshold` as follows: +.... + freezing_limit(S, cid) = idle_cycles_burned_rate(S, cid) * S.freezing_threshold[cid] / (24 * 60 * 60) +.... Conditions:: .... S.messages = Older_messages · CallMessage CM · Younger_messages (CM.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ CM.queue) - S.canister_status[CM.callee] = Stopped or S.canister_status[CM.callee] = Stopping or balances[CM.callee] < freezing_limit(S, CM.callee) + S.canister_status[CM.callee] = Stopped or S.canister_status[CM.callee] = Stopping _ or balances[CM.callee] < freezing_limit(S, CM.callee) .... State after:: .... S.messages = Older_messages · Younger_messages · ResponseMessage { - origin = S.call_contexts[CM.call_context].origin + origin = CM.origin; response = Reject (CANISTER_ERROR, "canister not running"); refunded_cycles = CM.transferred_cycles; } @@ -2699,7 +2730,7 @@ Conditions:: S.messages = Older_messages · CallMessage CM · Younger_messages S.canisters[CM.callee] ≠ EmptyCanister S.canister_status[CM.callee] = Running - balances[CM.callee] ≥ freezing_limit(S, CM.callee) + MAX_CYCLES_PER_MESSAGE + S.balances[CM.callee] ≥ freezing_limit(S, CM.callee) + MAX_CYCLES_PER_MESSAGE Ctxt_id ∉ dom S.call_contexts .... + @@ -2712,7 +2743,7 @@ S with FuncMessage { call_context = Ctxt_id; receiver = CM.callee; - entry_point = PublicMethod CM.method_name CM.caller CM.data + entry_point = PublicMethod CM.method_name CM.caller CM.data; queue = CM.queue; } · Younger_messages @@ -2790,9 +2821,7 @@ Conditions:: Available = S.call_contexts[M.call_contexts].available_cycles ( M.entry_point = PublicMethod Name Caller Data Arg = { data = Data; caller = Caller } - (F = Mod.update_methods[Name](Arg, Env, Available) - or - (F = query_as_update(Mod.query_methods[Name], Arg, Env)) + (F = Mod.update_methods[Name](Arg, Env, Available)) or (F = query_as_update(Mod.query_methods[Name], Arg, Env)) ) or ( M.entry_point = Callback Callback Response RefundedCycles @@ -2811,12 +2840,11 @@ if R = Return res Cycles_used ≤ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) res.cycles_accepted ≤ Available + (Cycles_used + ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ]) ≤ + (S.balances[M.receiver] + res.cycles_accepted + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) New_balance = - S.balances[M.receiver] - + res.cycles_accepted - + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - - Cycles_used - - ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ] + (S.balances[M.receiver] + res.cycles_accepted + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) + - (Cycles_used + ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ]) New_balance > if Is_response then 0 else freezing_limit(S, CM.callee); (res.response = NoResponse) or S.call_contexts[M.call_context].needs_to_respond then @@ -2859,9 +2887,8 @@ else S with messages = Older_messages · Younger_messages balances[M.receiver] = - S.balances[M.receiver] - + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - - Cycles_used + S.balances[M.receiver] + + ((if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - Cycles_used) .... @@ -2884,17 +2911,28 @@ If message execution <>; that would require a (unique) call to `ic0.reply`. Note also that the state changes are persisted even when the IC is set to synthesize a <> reject immediately afterward (which happens when this returns without calling `ic0.reply` or `ic0.reject`, the corresponding call has not been responded to and there are no outstanding callbacks, see <>). -The function `as_update` turns a query function into an update function, this is merely a notational trick to simplify the rule +The functions `query_as_update` and `heartbeat_as_update` turns a query function resp the heartbeat into an update function; this is merely a notational trick to simplify the rule: .... -as_update(f, arg, env) = λ wasm_state → +query_as_update(f, arg, env) = λ wasm_state → match f(arg, env)(wasm_state) with Trap → Trap Return res → Return { new_state = wasm_state; new_calls = []; + new_certified_data = NoCertifiedData; response = res; cycles_accepted = 0; + } + +heartbeat_as_update(f, env) = λ wasm_state → + match f(env)(wasm_state) with + Trap → Trap + Return wasm_state → Return { + new_state = wasm_state; + new_calls = []; new_certified_data = NoCertifiedData; + response = NoResponse; + cycles_accepted = 0; } .... Note that by construction, a query function will either trap or return with a response; it will never send calls, and it will never change the state of the canister. @@ -2908,9 +2946,9 @@ Conditions:: .... S.call_contexts[Ctxt_id].needs_to_respond = true S.call_contexts[Ctxt_id].origin ≠ FromHeartbeat - ∀ CallMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id - ∀ ResponseMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id - ∀ ctxt_ids. + ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id + ∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id + ∀ ctxt_ids ∈ dom(S.call_contexts). S.call_contexts[ctxt_ids].needs_to_respond ==> S.call_contexts[ctxt_ids].origin.calling_context ≠ Ctxt_id .... @@ -2941,8 +2979,8 @@ Conditions:: S.call_contexts[Ctxt_id].origin = FromHeartbeat ∀ FuncMessage M ∈ S.messages. M.call_context ≠ Ctxt_id ) - ∀ CallMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id - ∀ ResponseMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id + ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id + ∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id ∀ ctxt_ids. S.call_contexts[ctxt_ids].needs_to_respond = true ==> S.call_contexts[ctxt_ids].origin.calling_context ≠ Ctxt_id @@ -3041,7 +3079,7 @@ The controllers of a canister can obtain information about the canister. The `Memory_size` is the (in this specification underspecified) total size of storage in bytes. -The `idle_cycles_burned_per_day` is the idle consumption of resources in cycles per day. Therefore, the freezing threshold in cycles can be obtained using the following formula: `freezing_threshold` (in seconds) * `idle_cycles_burned_per_day` / (3600 * 24) (seconds). +The `idle_cycles_burned_per_day` is the idle consumption of resources in cycles per day. Conditions:: .... @@ -3068,7 +3106,7 @@ S with memory_size = Memory_size; cycles = S.balance[A.canister_id]; freezing_threshold = S.freezing_threshold[A.canister_id]; - idle_cycles_burned_per_day = freezing_limit(S, A.canister_id) / freezing_threshold * 3600 * 24; + idle_cycles_burned_per_day = idle_cycles_burned_rate(S, A.canister_id); }) refunded_cycles = M.transferred_cycles } @@ -3101,6 +3139,7 @@ Conditions:: status = S.status[M.receiver]; } Mod.init(A.canister_id, Arg, Env) = Return New_state + dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ .... State after:: .... @@ -3145,6 +3184,7 @@ Conditions:: caller = M.caller; } Mod.post_upgrade(A.canister_id, Stable_memory, Arg, Env) = Return New_state + dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ .... State after:: .... @@ -3652,7 +3692,7 @@ S with ==== Query call -Canister query calls to `/api/v2/canister//query` can be executed directly. They can only be executed against canisters which are `Running`. +Canister query calls to `/api/v2/canister//query` can be executed directly. They can only be executed against canisters which have a status of `Running` and are also not frozen. During the execution of a query call, a certificate is provided to the canister that is valid, contains a current state tree (or “recent enough”; the specification is currently vague about how old the certificate may be) and reveals the canister’s <>. @@ -3664,7 +3704,7 @@ Conditions:: is_effective_canister_id(E.content, ECID) S.system_time <= Q.ingress_expiry S.canisters[Q.canister_id] ≠ EmptyCanister - S.canister_status[Q.canister_id] = Running + S.canister_status[Q.canister_id] = Running ∧ S.balances[Q.canister_id] >= freezing_limit(S, Q.canister_id) C = S.canisters[Q.canister_id] F = C.module.query_methods[Q.method_name] Arg = { @@ -4370,6 +4410,8 @@ ic0.data_certificate_copy(dst: i32, offset: i32, size: i32) = if es.params.sysenv.certificate = NoCertificate then Trap copy_to_canister(dst, offset, size, es.params.sysenv.certificate) +ic0.performance_counter(counter_type : i32) : i64 = + arbitrary() ic0.debug_print(src : i32, size : i32) = return diff --git a/theories/IC.thy b/theories/IC.thy new file mode 100644 index 000000000..193784430 --- /dev/null +++ b/theories/IC.thy @@ -0,0 +1,777 @@ +theory IC + imports "HOL-Library.AList" +begin + +(* Partial maps *) + +typedef ('a, 'b) list_map = "{f :: ('a \ 'b) list. distinct (map fst f)}" + by (auto intro: exI[of _ "[]"]) + +setup_lifting type_definition_list_map + +lift_definition list_map_dom :: "('a, 'b) list_map \ 'a set" is + "set \ map fst" . + +lift_definition list_map_vals :: "('a, 'b) list_map \ 'b set" is + "set \ map snd" . + +lift_definition list_map_sum_vals :: "('b \ nat) \ ('a, 'b) list_map \ nat" is + "\g. sum_list \ (map (g \ snd))" . + +lift_definition list_map_get :: "('a, 'b) list_map \ 'a \ 'b option" is + "map_of" . + +lift_definition list_map_set :: "('a, 'b) list_map \ 'a \ 'b \ ('a, 'b) list_map" is + "\f x y. AList.update x y f" + by (rule distinct_update) + +lift_definition list_map_del :: "('a, 'b) list_map \ 'a \ ('a, 'b) list_map" is + "\f x. AList.delete x f" + by (rule distinct_delete) + +lift_definition list_map_empty :: "('a, 'b) list_map" is "[]" + by auto + +lemma list_map_empty_dom[simp]: "list_map_dom list_map_empty = {}" + by transfer auto + +lemma list_map_sum_in_ge_aux: + fixes g :: "'a \ nat" + shows "distinct (map fst f) \ map_of f x = Some y \ g y \ sum_list (map g (map snd f))" + by (induction f) (auto split: if_splits) + +lemma list_map_sum_in_ge: "list_map_get f x = Some y \ list_map_sum_vals g f \ g y" + apply transfer + using list_map_sum_in_ge_aux[OF _ map_of_is_SomeI] + by fastforce + +lemma list_map_sum_in_aux: fixes g :: "'a \ nat" + shows "distinct (map fst f) \ map_of f x = Some y \ + sum_list (map (g \ snd) (AList.update x y' f)) = sum_list (map (g \ snd) f) + g y' - g y" + apply (induction f) + apply auto[1] + subgoal for a f + using list_map_sum_in_ge_aux[OF _ map_of_is_SomeI, of f x y g] + by auto + done + +lemma list_map_sum_in: "list_map_get f x = Some y \ list_map_sum_vals g (list_map_set f x y') = list_map_sum_vals g f + g y' - g y" + apply transfer + using list_map_sum_in_aux + by fastforce + +lemma list_map_sum_out_aux: + "x \ set (map fst f) \ sum_list (map (g \ snd) (AList.update x y f)) = sum_list (map (g \ snd) f) + g y" + by (induction f) (auto simp: add.assoc) + +lemma list_map_sum_out: "x \ list_map_dom f \ list_map_sum_vals g (list_map_set f x y) = list_map_sum_vals g f + g y" + apply transfer + using list_map_sum_out_aux + by fastforce + +lemma list_map_del_sum_aux: + fixes g :: "'a \ nat" + shows "distinct (map fst f) \ map_of f x = Some y \ sum_list (map (g \ snd) f) = sum_list (map (g \ snd) (AList.delete x f)) + g y" + by (induction f) auto + +lemma list_map_del_sum: "list_map_get f x = Some y \ list_map_sum_vals g f = list_map_sum_vals g (list_map_del f x) + g y" + apply transfer + using list_map_del_sum_aux + by fastforce + +(* Abstract behaviour *) + +(* Abstract canisters *) + +record ('b, 'p) arg = + data :: 'b + caller :: 'p + +type_synonym timestamp = nat +datatype status = Running | Stopping | Stopped +record ('b) env = + env_time :: timestamp + balance :: nat + freezing_limit :: nat + certificate :: "'b option" + status :: status + +type_synonym reject_code = nat +datatype ('b, 's) response = + Reply "'b" +| Reject reject_code 's +record ('p, 'canid, 's, 'b, 'c) method_call = + callee :: 'canid + method_name :: 's + arg :: 'b + transferred_cycles :: nat + callback :: 'c + +record ('w, 'p, 'canid, 's, 'b, 'c) update_return = + new_state :: 'w + new_calls :: "('p, 'canid, 's, 'b, 'c) method_call list" + new_certified_data :: "'b option" + response :: "('b, 's) response option" + cycles_accepted :: nat +type_synonym ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func = "'w \ 'tr + ('w, 'p, 'canid, 's, 'b, 'c) update_return" +type_synonym ('w, 'b, 's, 'tr) query_func = "'w \ 'tr + ('b, 's) response" + +type_synonym available_cycles = nat +type_synonym refunded_cycles = nat + +datatype inspect_method_result = Accept | Reject +record ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module_rec = + init :: "'canid \ ('b, 'p) arg \ 'b env \ 'tr + 'w" + pre_upgrade :: "'w \ 'p \ 'b env \ 'tr + 'sm" + post_upgrade :: "'canid \ 'sm \ ('b, 'p) arg \ 'b env \ 'tr + 'w" + update_methods :: "('s, (('b, 'p) arg \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func) list_map" + query_methods :: "('s, (('b, 'p) arg \ 'b env) \ ('w, 'b, 's, 'tr) query_func) list_map" + heartbeat :: "'b env \ 'w \ 'tr + 'w" + callbacks :: "('c \ ('b, 's) response \ refunded_cycles \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func" + inspect_message :: "('s \ 'w \ ('b, 'p) arg \ 'b env) \ 'tr + inspect_method_result" +typedef ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module = + "{m :: ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module_rec. list_map_dom (update_methods m) \ list_map_dom (query_methods m) = {}}" + by (auto intro: exI[of _ "\init = undefined, pre_upgrade = undefined, post_upgrade = undefined, + update_methods = list_map_empty, query_methods = list_map_empty, heartbeat = undefined, callbacks = undefined, + inspect_message = undefined\"]) + +setup_lifting type_definition_canister_module + +lift_definition dispatch_method :: "'s \ ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module \ + (((('b, 'p) arg \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func) + + ((('b, 'p) arg \ 'b env) \ ('w, 'b, 's, 'tr) query_func)) option" is + "\f m. case list_map_get (update_methods m) f of Some f' \ None | None \ (case list_map_get (query_methods m) f of Some f' \ None | None \ None)" . + +lift_definition canister_module_callbacks :: "('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module \ + ('c \ ('b, 's) response \ refunded_cycles \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func" is + callbacks . + +lift_definition canister_module_heartbeat :: "('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module \ 'b env \ 'w \ 'tr + 'w" is + heartbeat . + +(* Call contexts *) + +record ('b, 'p, 'uid, 'canid, 's) request = + nonce :: 'b + ingress_expiry :: nat + sender :: 'uid + canister_id :: 'canid + method_name :: 's + data :: 'b +datatype ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin = + From_user "('b, 'p, 'uid, 'canid, 's) request" +| From_canister "'cid" "'c" +| From_heartbeat +record ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt_rep = + canister :: 'canid + origin :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin" + needs_to_respond :: bool + deleted :: bool + available_cycles :: nat + +typedef ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt = "{ctxt :: ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt_rep. + (deleted ctxt \ \needs_to_respond ctxt) \ (\needs_to_respond ctxt \ available_cycles ctxt = 0)}" + by (auto intro: exI[of _ "\canister = undefined, origin = undefined, needs_to_respond = True, deleted = False, available_cycles = 0\"]) + +setup_lifting type_definition_call_ctxt + +lift_definition call_ctxt_origin :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin" is + "\ctxt. origin ctxt" . + +lift_definition call_ctxt_needs_to_respond :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ bool" is + "\ctxt. needs_to_respond ctxt" . + +lift_definition call_ctxt_available_cycles :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ nat" is + "\ctxt. available_cycles ctxt" . + +lemma call_ctxt_inv: "\call_ctxt_needs_to_respond x2 \ call_ctxt_available_cycles x2 = 0" + by transfer auto + +lift_definition call_ctxt_respond :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt" is + "\ctxt. ctxt\available_cycles := 0, needs_to_respond := False\" + by auto + +lemma call_ctxt_respond_available_cycles[simp]: "call_ctxt_available_cycles (call_ctxt_respond ctxt) = 0" + by transfer auto + +lemma call_ctxt_respond_needs_to_respond[dest]: "call_ctxt_needs_to_respond (call_ctxt_respond ctxt) \ False" + by transfer auto + +lift_definition call_ctxt_deduct_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt" is + "\n ctxt. ctxt\available_cycles := available_cycles ctxt - n\" + by auto + +lemma call_ctxt_deduct_cycles_origin[simp]: "call_ctxt_origin (call_ctxt_deduct_cycles n ctxt) = call_ctxt_origin ctxt" + by transfer auto + +lemma call_ctxt_deduct_cycles_needs_to_respond[simp]: "call_ctxt_needs_to_respond (call_ctxt_deduct_cycles n ctxt) = call_ctxt_needs_to_respond ctxt" + by transfer auto + +lemma call_ctxt_deduct_cycles_available_cycles[simp]: "call_ctxt_available_cycles (call_ctxt_deduct_cycles n ctxt) = call_ctxt_available_cycles ctxt - n" + by transfer auto + +(* Calls and Messages *) + +datatype 'canid queue_origin = System | Canister 'canid +datatype 'canid queue = Unordered | Queue "'canid queue_origin" 'canid +datatype ('s, 'p, 'b, 'c) entry_point = + Public_method "'s" "'p" "'b" +| Callback "'c" "('b, 's) response" "refunded_cycles" +| Heartbeat + +datatype ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message = + Call_message "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin" 'p 'canid 's 'b nat "'canid queue" +| Func_message 'cid 'canid "('s, 'p, 'b, 'c) entry_point" "'canid queue" +| Response_message "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin" "('b, 's) response" nat + +(* API requests *) + +type_synonym 'b path = "'b list" +record ('b, 'uid) StateRead = + nonce :: 'b + ingress_expiry :: nat + sender :: 'uid + paths :: "'b path list" +record ('b, 'uid, 'canid, 's) CanisterQuery = + nonce :: 'b + ingress_expiry :: nat + sender :: 'uid + canister_id :: 'canid + method_name :: 's + data :: 'b +type_synonym ('b, 'uid, 'canid, 's) APIReadRequest = "('b, 'uid) StateRead + ('b, 'uid, 'canid, 's) CanisterQuery" +record ('b, 'p, 'uid, 'canid, 's, 'pk, 'sig, 'sd) envelope = + content :: "('b, 'p, 'uid, 'canid, 's) request + ('b, 'uid, 'canid, 's) APIReadRequest" + sender_pubkey :: "'pk option" + sender_sig :: "'sig option" + sender_delegation :: 'sd + +datatype ('b, 's) request_status = Received | Processing | Rejected reject_code 's | Replied 'b | Done + +(* The system state *) + +record ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) can_state_rec = + wasm_state :: 'w + module :: "('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module" + raw_module :: 'b + public_custom_sections :: "('s, 'b) list_map" + private_custom_sections :: "('s, 'b) list_map" +type_synonym ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) can_state = "('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) can_state_rec option" +datatype ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status = Running | Stopping "(('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin \ nat) list" | Stopped +record ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic = + requests :: "(('b, 'p, 'uid, 'canid, 's) request, ('b, 's) request_status) list_map" + canisters :: "('canid, ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) can_state) list_map" + controllers :: "('canid, 'p set) list_map" + freezing_threshold :: "('canid, nat) list_map" + canister_status :: "('canid, ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status) list_map" + time :: "('canid, timestamp) list_map" + balances :: "('canid, nat) list_map" + certified_data :: "('canid, 'b) list_map" + system_time :: timestamp + call_contexts :: "('cid, ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt) list_map" + messages :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message list" + root_key :: 'pk + +(* State transitions *) + +context fixes + CANISTER_ERROR :: reject_code + and SYS_FATAL :: reject_code + and SYS_TRANSIENT :: reject_code + and MAX_CYCLES_PER_MESSAGE :: nat + and MAX_CYCLES_PER_RESPONSE :: nat + and MAX_CANISTER_BALANCE :: nat + and ic_freezing_limit :: "('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ 'canid \ nat" + and encode_string :: "string \ 's" + and principal_of_uid :: "'uid \ 'p" + and principal_of_canid :: "'canid \ 'p" +begin + +(* Cycles *) + +fun carried_cycles :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin \ nat" where + "carried_cycles (From_canister _ _) = MAX_CYCLES_PER_RESPONSE" +| "carried_cycles _ = 0" + +fun cycles_reserved :: "('s, 'p, 'b, 'c) entry_point \ nat" where + "cycles_reserved (entry_point.Public_method _ _ _) = MAX_CYCLES_PER_MESSAGE" +| "cycles_reserved (entry_point.Callback _ _ _) = MAX_CYCLES_PER_RESPONSE" +| "cycles_reserved (entry_point.Heartbeat) = MAX_CYCLES_PER_MESSAGE" + +fun message_cycles :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message \ nat" where + "message_cycles (Call_message orig _ _ _ _ trans_cycles q) = carried_cycles orig + trans_cycles" +| "message_cycles (Func_message _ _ ep _) = cycles_reserved ep" +| "message_cycles (Response_message orig _ ref_cycles) = carried_cycles orig + ref_cycles" + +lift_definition call_ctxt_carried_cycles :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ nat" is + "\ctxt. (if needs_to_respond ctxt + then available_cycles ctxt + carried_cycles (origin ctxt) + else 0)" . + +lemma call_ctxt_respond_carried_cycles[simp]: "call_ctxt_carried_cycles (call_ctxt_respond ctxt) = 0" + by transfer auto + +lemma call_ctxt_carried_cycles: "call_ctxt_carried_cycles ctxt = (if call_ctxt_needs_to_respond ctxt + then call_ctxt_available_cycles ctxt + carried_cycles (call_ctxt_origin ctxt) else 0)" + by transfer auto + +definition total_cycles :: "('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "total_cycles ic = ( + let cycles_in_balances = list_map_sum_vals id (balances ic) in + let cycles_in_messages = sum_list (map message_cycles (messages ic)) in + let cycles_in_contexts = list_map_sum_vals call_ctxt_carried_cycles (call_contexts ic) in + cycles_in_balances + cycles_in_messages + cycles_in_contexts)" + +(* Accessor functions *) + +fun calling_context :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin \ 'cid option" where + "calling_context (From_canister c _) = Some c" +| "calling_context _ = None" + +fun message_queue :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message \ 'canid queue option" where + "message_queue (Call_message _ _ _ _ _ _ q) = Some q" +| "message_queue (Func_message _ _ _ q) = Some q" +| "message_queue _ = None" + +(* Type conversion functions *) + +fun to_status :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status \ status" where + "to_status can_status.Running = status.Running" +| "to_status (can_status.Stopping _) = status.Stopping" +| "to_status can_status.Stopped = status.Stopped" + + + +(* System transition: API Request submission [DONE] *) + +definition request_submission_pre :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig, 'sd) envelope \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "request_submission_pre E S = (case content E of Inl req \ req \ list_map_dom (requests S) | _ \ False)" + +definition request_submission_post :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig, 'sd) envelope \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "request_submission_post E S = S\requests := list_map_set (requests S) (projl (content E)) Received\" + +lemma request_submission_cycles_inv: + "request_submission_pre E S \ total_cycles S = total_cycles (request_submission_post E S)" + by (auto simp: request_submission_pre_def request_submission_post_def total_cycles_def) + + + +(* System transition: Request rejection [DONE] *) + +definition request_rejection_pre :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig, 'sd) envelope \ ('b, 'p, 'uid, 'canid, 's) request \ reject_code \ 's \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "request_rejection_pre E req code msg S = (list_map_get (requests S) req = Some Received \ (code = SYS_FATAL \ code = SYS_TRANSIENT))" + +definition request_rejection_post :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig, 'sd) envelope \ ('b, 'p, 'uid, 'canid, 's) request \ reject_code \ 's \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "request_rejection_post E req code msg S = S\requests := list_map_set (requests S) req (Rejected code msg)\" + +lemma request_rejection_cycles_inv: + "request_rejection_pre E req code msg S \ total_cycles S = total_cycles (request_rejection_post E req code msg S)" + by (auto simp: request_rejection_pre_def request_rejection_post_def total_cycles_def) + + + +(* System transition: Initiating canister calls [DONE] *) + +definition initiate_canister_call_pre :: "('b, 'p, 'uid, 'canid, 's) request \ + ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "initiate_canister_call_pre req S = (list_map_get (requests S) req = Some Received \ + system_time S \ request.ingress_expiry req \ + request.canister_id req \ list_map_dom (canisters S))" + +definition initiate_canister_call_post :: "('b, 'p, 'uid, 'canid, 's) request \ + ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ + ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "initiate_canister_call_post req S = + S\requests := list_map_set (requests S) req Processing, messages := + Call_message (From_user req) (principal_of_uid (request.sender req)) (request.canister_id req) (request.method_name req) + (request.data req) 0 Unordered # messages S\" + +lemma initiate_canister_call_cycles_inv: + "initiate_canister_call_pre R S \ + total_cycles S = total_cycles (initiate_canister_call_post R S)" + by (auto simp: initiate_canister_call_pre_def initiate_canister_call_post_def total_cycles_def) + + + +(* System transition: Calls to stopped/stopping/frozen canisters are rejected [DONE] *) + +definition call_reject_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "call_reject_pre n S = (n < length (messages S) \ (case messages S ! n of + Call_message orig cer cee mn d trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + (case list_map_get (canister_status S) cee of Some Stopped \ True + | Some (Stopping l) \ True + | _ \ (case list_map_get (balances S) cee of Some bal \ bal < ic_freezing_limit S cee | _ \ False)) + | _ \ False))" + +definition call_reject_post :: "nat \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "call_reject_post n S = (case messages S ! n of Call_message orig cer cee mn d trans_cycles q \ + S\messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (response.Reject CANISTER_ERROR (encode_string ''canister not running'')) trans_cycles]\)" + +lemma call_reject_cycles_inv: + assumes "call_reject_pre n S" + shows "total_cycles S = total_cycles (call_reject_post n S)" +proof - + obtain orig cer cee mn d trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn d trans_cycles q" + using assms + by (auto simp: call_reject_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn d trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: call_reject_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: call_reject_pre_def call_reject_post_def total_cycles_def Let_def msgs split: message.splits option.splits) +qed + + + +(* System transition: Call context creation: Public entry points [DONE] *) + +definition call_context_create_pre :: "nat \ 'cid + \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "call_context_create_pre n ctxt_id S = (n < length (messages S) \ (case messages S ! n of + Call_message orig cer cee mn d trans_cycles q \ + (case list_map_get (canisters S) cee of Some (Some can) \ True | _ \ False) \ + list_map_get (canister_status S) cee = Some Running \ + (case list_map_get (balances S) cee of Some bal \ bal \ ic_freezing_limit S cee + MAX_CYCLES_PER_MESSAGE | None \ False) \ + ctxt_id \ list_map_dom (call_contexts S) + | _ \ False))" + +lift_definition create_call_ctxt :: "'canid \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin \ nat \ + ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt" is + "\cee orig trans_cycles. \canister = cee, origin = orig, needs_to_respond = True, deleted = False, available_cycles = trans_cycles\" + by auto + +lemma create_call_ctxt_carried_cycles[simp]: "call_ctxt_carried_cycles (create_call_ctxt cee orig trans_cycles) = carried_cycles orig + trans_cycles" + by transfer auto + +definition call_context_create_post :: "nat \ 'cid \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "call_context_create_post n ctxt_id S = (case messages S ! n of Call_message orig cer cee mn d trans_cycles q \ + case list_map_get (balances S) cee of Some bal \ + S\messages := list_update (messages S) n (Func_message ctxt_id cee (Public_method mn cer d) q), + call_contexts := list_map_set (call_contexts S) ctxt_id (create_call_ctxt cee orig trans_cycles), + balances := list_map_set (balances S) cee (bal - MAX_CYCLES_PER_MESSAGE)\)" + +lemma call_context_create_cycles_inv: + assumes "call_context_create_pre n ctxt_id S" + shows "total_cycles S = total_cycles (call_context_create_post n ctxt_id S)" +proof - + obtain orig cer cee mn d trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn d trans_cycles q" + using assms + by (auto simp: call_context_create_pre_def split: message.splits) + define xs where "xs = take n (messages S)" + define older where "older = drop (Suc n) (messages S)" + have msgs: "messages S = xs @ Call_message orig cer cee mn d trans_cycles q # older" "(xs @ m # older) ! n = m" + and msgs_upd: "(xs @ Call_message orig cer cee mn d trans_cycles q # older)[n := m] = xs @ m # older" for m + using id_take_nth_drop[of n "messages S"] upd_conv_take_nth_drop[of n "messages S"] assms + by (auto simp: call_context_create_pre_def msg xs_def older_def nth_append) + show ?thesis + using assms list_map_sum_in_ge[of "balances S" cee, where ?g=id] + by (auto simp: call_context_create_pre_def call_context_create_post_def total_cycles_def + list_map_sum_in[where ?g=id, simplified] list_map_sum_out msgs msgs_upd split: option.splits) +qed + + + +(* System transition: Call context creation: Heartbeat [DONE] *) + +definition call_context_heartbeat_pre :: "'canid \ 'cid \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "call_context_heartbeat_pre cee ctxt_id S = ( + (case list_map_get (canisters S) cee of Some (Some can) \ True | _ \ False) \ + list_map_get (canister_status S) cee = Some Running \ + (case list_map_get (balances S) cee of Some bal \ bal \ ic_freezing_limit S cee + MAX_CYCLES_PER_MESSAGE | _ \ False) \ + ctxt_id \ list_map_dom (call_contexts S))" + +lift_definition create_call_ctxt_heartbeat :: "'canid \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt" is + "\cee. \canister = cee, origin = From_heartbeat, needs_to_respond = False, deleted = False, available_cycles = 0\" + by auto + +lemma create_call_ctxt_heartbeat_carried_cycles[simp]: "call_ctxt_carried_cycles (create_call_ctxt_heartbeat cee) = 0" + by transfer auto + +definition call_context_heartbeat_post :: "'canid \ 'cid \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "call_context_heartbeat_post cee ctxt_id S = + (case list_map_get (balances S) cee of Some bal \ + S\messages := Func_message ctxt_id cee Heartbeat (Queue System cee) # messages S, + call_contexts := list_map_set (call_contexts S) ctxt_id (create_call_ctxt_heartbeat cee), + balances := list_map_set (balances S) cee (bal - MAX_CYCLES_PER_MESSAGE)\)" + +lemma call_context_heartbeat_cycles_inv: + "call_context_heartbeat_pre cee ctxt_id S \ + total_cycles S = total_cycles (call_context_heartbeat_post cee ctxt_id S)" + using list_map_sum_in_ge[of "balances S" cee, where ?g=id, simplified] + by (auto simp: call_context_heartbeat_pre_def call_context_heartbeat_post_def total_cycles_def + list_map_sum_in[where ?g=id, simplified] list_map_sum_out split: option.splits) + + + +(* System transition: Message execution [DONE] *) + +fun query_as_update :: "((('b, 'p) arg \ 'b env) \ ('w, 'b, 's, 'tr) query_func) \ ('b, 'p) arg \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func" where + "query_as_update (f, a, e) = (\w. case f (a, e) w of Inl t \ Inl t | + Inr res \ Inr \new_state = w, new_calls = [], new_certified_data = None, response = Some res, cycles_accepted = 0\)" + +fun heartbeat_as_update :: "('b env \ 'w \ 'tr + 'w) \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func" where + "heartbeat_as_update (f, e) w = (case f e w of Inl t \ Inl t | + Inr w' \ Inr \update_return.new_state = w', update_return.new_calls = [], update_return.new_certified_data = None, + update_return.response = None, update_return.cycles_accepted = 0\)" + +fun exec_function :: "('s, 'p, 'b, 'c) entry_point \ 'b env \ nat \ ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func" where + "exec_function (entry_point.Public_method mn c d) e bal m = ( + let arg = \arg.data = d, arg.caller = c\ in + case dispatch_method mn m of Some (Inl upd) \ upd (arg, e, bal) + | Some (Inr query) \ query_as_update (query, arg, e) | None \ + undefined)" +| "exec_function (entry_point.Callback cb resp ref_cycles) e bal m = + canister_module_callbacks m (cb, resp, ref_cycles, e, bal)" +| "exec_function (entry_point.Heartbeat) e bal m = heartbeat_as_update ((canister_module_heartbeat m), e)" + +definition message_execution_pre :: "nat \ nat \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "message_execution_pre n cycles_used S = + (n < length (messages S) \ (case messages S ! n of Func_message ctxt_id recv ep q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + (case (list_map_get (canisters S) recv, list_map_get (balances S) recv, list_map_get (canister_status S) recv, + list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id) of + (Some (Some can), Some bal, Some can_status, Some t, Some ctxt) \ True | _ \ False) + | _ \ False))" + +definition message_execution_post :: "nat \ nat \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "message_execution_post n cycles_used S = (case messages S ! n of Func_message ctxt_id recv ep q \ + (case (list_map_get (canisters S) recv, list_map_get (balances S) recv, list_map_get (canister_status S) recv, + list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id) of + (Some (Some can), Some bal, Some can_status, Some t, Some ctxt) \ ( + let Mod = module can; + Is_response = (case ep of Callback _ _ _ \ True | _ \ False); + Env = \env_time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = to_status can_status\; + Available = call_ctxt_available_cycles ctxt; + F = exec_function ep Env Available Mod; + R = F (wasm_state can); + (cycles_accepted_res, new_calls_res) = (case R of Inr res \ (cycles_accepted res, new_calls res)); + New_balance = bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + - (cycles_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res)); + no_response = (case R of Inr result \ update_return.response result = None) in + if \isl R \ cycles_used \ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + cycles_accepted_res \ Available \ + cycles_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res) \ + bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + New_balance > (if Is_response then 0 else ic_freezing_limit S recv) \ + (no_response \ call_ctxt_needs_to_respond ctxt) then + (let result = projr R; + new_call_to_message = (\call. Call_message (From_canister ctxt_id (callback call)) (principal_of_canid recv) + (callee call) (method_call.method_name call) (arg call) (transferred_cycles call) (Queue (Canister recv) (callee call))); + response_messages = (case response result of None \ [] + | Some resp \ [Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res)]); + messages = take n (messages S) @ drop (Suc n) (messages S) @ map new_call_to_message new_calls_res @ response_messages; + new_ctxt = (case response result of + None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt + | Some _ \ call_ctxt_respond ctxt); + certified_data = (case new_certified_data result of None \ certified_data S + | Some cd \ list_map_set (certified_data S) recv cd) + in S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := new_state result\)), + messages := messages, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, + balances := list_map_set (balances S) recv New_balance, certified_data := certified_data\) + else S\messages := take n (messages S) @ drop (Suc n) (messages S), + balances := list_map_set (balances S) recv (bal + ((if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - cycles_used))\)) + | _ \ undefined)" + +definition message_execution_lost_cycles :: "nat \ nat \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "message_execution_lost_cycles n cycles_used S = (case messages S ! n of Func_message ctxt_id recv ep q \ + let Is_response = (case ep of Callback _ _ _ \ True | _ \ False) in + min cycles_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))" + +lemma message_execution_cycles_monotonic: + assumes pre: "message_execution_pre n cycles_used S" + shows "total_cycles S = total_cycles (message_execution_post n cycles_used S) + message_execution_lost_cycles n cycles_used S" +proof - + obtain ctxt_id recv ep q can bal can_status t ctxt where msg: "messages S ! n = Func_message ctxt_id recv ep q" + and prod: "list_map_get (canisters S) recv = Some (Some can)" + "list_map_get (balances S) recv = Some bal" + "list_map_get (canister_status S) recv = Some can_status" + "list_map_get (time S) recv = Some t" + "list_map_get (call_contexts S) ctxt_id = Some ctxt" + using pre + by (auto simp: message_execution_pre_def split: message.splits option.splits) + define Mod where "Mod = can_state_rec.module can" + define Is_response where "Is_response = (case ep of Callback _ _ _ \ True | _ \ False)" + define Env :: "'b env" where "Env = \env_time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = to_status can_status\" + define Available where "Available = call_ctxt_available_cycles ctxt" + define F where "F = exec_function ep Env Available Mod" + define R where "R = F (wasm_state can)" + obtain cycles_accepted_res new_calls_res where res: "(cycles_accepted_res, new_calls_res) = (case R of Inr res \ (cycles_accepted res, new_calls res))" + by (cases "(case R of Inr res \ (cycles_accepted res, new_calls res))") auto + define New_balance where "New_balance = bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + - (cycles_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res))" + define no_response where "no_response = (case R of Inr result \ update_return.response result = None)" + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Func_message ctxt_id recv ep q # younger" + "take n older = older" "drop (Suc n) older = []" + "take (n - length older) ws = []" "drop (Suc n - length older) (w # ws) = ws" + for w and ws :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message list" + using id_take_nth_drop[of n "messages S"] pre + by (auto simp: message_execution_pre_def msg older_def younger_def) + note lm = list_map_sum_in[OF prod(2), where ?g=id, simplified] list_map_sum_in_ge[OF prod(2), where ?g=id, simplified] + list_map_sum_in[OF prod(5), where ?g=call_ctxt_carried_cycles] list_map_sum_in_ge[OF prod(5), where ?g=call_ctxt_carried_cycles] + define S'' where "S'' = S\messages := take n (messages S) @ drop (Suc n) (messages S), + balances := list_map_set (balances S) recv (bal + ((if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - cycles_used))\" + define cond where "cond = (\isl R \ cycles_used \ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + cycles_accepted_res \ Available \ + cycles_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res) \ + bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + New_balance > (if Is_response then 0 else ic_freezing_limit S recv) \ + (no_response \ call_ctxt_needs_to_respond ctxt))" + have reserved: "(if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) = cycles_reserved ep" + by (auto simp: Is_response_def split: entry_point.splits) + show ?thesis + proof (cases cond) + case False + have "message_execution_post n cycles_used S = S''" + "message_execution_lost_cycles n cycles_used S = min cycles_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)" + using False + by (simp_all add: message_execution_post_def message_execution_lost_cycles_def Let_def msg prod + Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] res[symmetric] + New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric] del: min_less_iff_conj split del: if_split) + then show ?thesis + using lm(2) + by (auto simp: total_cycles_def S''_def msgs lm(1) reserved) + next + case True + define result where "result = projr R" + have R_Inr: "R = Inr result" + using True + by (auto simp: cond_def result_def) + define response_messages where "response_messages = (case response result of None \ [] + | Some resp \ [Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res)])" + define new_call_to_message :: "(?'p, 'canid, 's, 'b, 'c) method_call \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message" where + "new_call_to_message = (\call. Call_message (From_canister ctxt_id (callback call)) (principal_of_canid recv) + (callee call) (method_call.method_name call) (arg call) (transferred_cycles call) (Queue (Canister recv) (callee call)))" + define messages where "messages = take n (ic.messages S) @ drop (Suc n) (ic.messages S) @ map new_call_to_message new_calls_res @ response_messages" + define new_ctxt where "new_ctxt = (case response result of + None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt + | Some _ \ call_ctxt_respond ctxt)" + define certified_data where "certified_data = (case new_certified_data result of None \ ic.certified_data S + | Some cd \ list_map_set (ic.certified_data S) recv cd)" + define S' where "S' = S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := new_state result\)), + messages := messages, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, + balances := list_map_set (balances S) recv New_balance, certified_data := certified_data\" + have cycles_accepted_res_def: "cycles_accepted_res = cycles_accepted result" + and new_calls_res_def: "new_calls_res = new_calls result" + using res + by (auto simp: R_Inr) + have no_response: "no_response = (response result = None)" + by (auto simp: no_response_def R_Inr) + have msg_exec: "message_execution_post n cycles_used S = S'" + and lost: "message_execution_lost_cycles n cycles_used S = min cycles_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)" + using True + by (simp_all add: message_execution_post_def message_execution_lost_cycles_def Let_def msg prod + Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] res[symmetric] + New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric] + messages_def[symmetric] new_ctxt_def[symmetric] certified_data_def[symmetric] S'_def[symmetric] + result_def[symmetric] response_messages_def[symmetric] new_call_to_message_def[symmetric] + del: min_less_iff_conj split del: if_split) + have "message_cycles \ new_call_to_message = (\c. MAX_CYCLES_PER_RESPONSE + transferred_cycles c)" for c :: "(?'p, 'canid, 's, 'b, 'c) method_call" + by (auto simp: new_call_to_message_def) + then have A1: "sum_list (map (message_cycles \ new_call_to_message) new_calls_res) = (\x\new_calls_res. MAX_CYCLES_PER_RESPONSE + transferred_cycles x)" + by auto + have A2: "sum_list (map local.message_cycles response_messages) = (case response result of None \ 0 + | _ \ carried_cycles (call_ctxt_origin ctxt) + (call_ctxt_available_cycles ctxt - cycles_accepted result))" + by (auto simp: response_messages_def Available_def cycles_accepted_res_def split: option.splits) + have A3: "call_ctxt_carried_cycles new_ctxt = (case response result of Some _ \ 0 + | _ \ if call_ctxt_needs_to_respond ctxt then carried_cycles (call_ctxt_origin ctxt) + (call_ctxt_available_cycles ctxt - cycles_accepted result) else 0)" + by (auto simp: new_ctxt_def Available_def cycles_accepted_res_def call_ctxt_carried_cycles split: option.splits) + have A4: "call_ctxt_carried_cycles ctxt = (if call_ctxt_needs_to_respond ctxt then carried_cycles (call_ctxt_origin ctxt) + call_ctxt_available_cycles ctxt else 0)" + using call_ctxt_carried_cycles + by auto + have reserve: "cycles_reserved ep = (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)" + by (auto simp: Is_response_def split: entry_point.splits) + have messages_msgs: "messages = older @ younger @ map new_call_to_message new_calls_res @ response_messages" + by (auto simp: messages_def older_def younger_def) + show ?thesis + using lm(2,4) True call_ctxt_inv[of ctxt] + by (auto simp: cond_def msg_exec S'_def total_cycles_def lm(1,3) msgs messages_msgs A1 A2 A3 A4 New_balance_def + reserve cycles_accepted_res_def no_response_def R_Inr lost Available_def split: option.splits) + qed +qed + + + +(* System transition: Call context starvation [DONE] *) + +definition call_context_starvation_pre :: "'cid \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "call_context_starvation_pre ctxt_id S = + (case list_map_get (call_contexts S) ctxt_id of Some call_context \ call_ctxt_needs_to_respond call_context \ + call_ctxt_origin call_context \ From_heartbeat \ + (\msg \ set (messages S). case msg of + Call_message orig _ _ _ _ _ _ \ calling_context orig \ Some ctxt_id + | Response_message orig _ _ \ calling_context orig \ Some ctxt_id + | _ \ True) \ + (\other_call_context \ list_map_vals (call_contexts S). + call_ctxt_needs_to_respond other_call_context \ + calling_context (call_ctxt_origin other_call_context) \ Some ctxt_id) + | None \ False)" + +definition call_context_starvation_post :: "'cid \ + ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "call_context_starvation_post ctxt_id S = ( + case list_map_get (call_contexts S) ctxt_id of Some call_context \ + let msg = Response_message (call_ctxt_origin call_context) (response.Reject CANISTER_ERROR (encode_string ''starvation'')) (call_ctxt_available_cycles call_context) + in S\call_contexts := list_map_set (call_contexts S) ctxt_id (call_ctxt_respond call_context), + messages := messages S @ [msg]\)" + +lemma call_context_starvation_cycles_inv: + "call_context_starvation_pre ctxt_id S \ + total_cycles S = total_cycles (call_context_starvation_post ctxt_id S)" + using list_map_sum_in_ge[where ?f="call_contexts S" and ?x=ctxt_id and ?g=call_ctxt_carried_cycles] + by (auto simp: call_context_starvation_pre_def call_context_starvation_post_def total_cycles_def + call_ctxt_carried_cycles list_map_sum_in[where ?g=call_ctxt_carried_cycles] split: option.splits) + + + +(* System transition: Call context removal [DONE] *) + +definition call_context_removal_pre :: "'cid \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "call_context_removal_pre ctxt_id S = ( + (case list_map_get (call_contexts S) ctxt_id of Some call_context \ + (\call_ctxt_needs_to_respond call_context \ + (call_ctxt_origin call_context = From_heartbeat \ + (\msg \ set (messages S). case msg of + Func_message other_ctxt_id _ _ _ \ other_ctxt_id \ ctxt_id + | _ \ True))) \ + (\msg \ set (messages S). case msg of + Call_message ctxt _ _ _ _ _ _ \ calling_context ctxt \ Some ctxt_id + | Response_message ctxt _ _ \ calling_context ctxt \ Some ctxt_id + | _ \ True) \ + (\other_call_context \ list_map_vals (call_contexts S). + call_ctxt_needs_to_respond other_call_context \ + calling_context (call_ctxt_origin other_call_context) \ Some ctxt_id) + | None \ False))" + +definition call_context_removal_post :: "'cid \ + ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "call_context_removal_post ctxt_id S = S\call_contexts := list_map_del (call_contexts S) ctxt_id\" + +lemma call_context_removal_cycles_inv: + "call_context_removal_pre ctxt_id S \ + total_cycles S = total_cycles (call_context_removal_post ctxt_id S) + + (case list_map_get (call_contexts S) ctxt_id of Some call_context \ call_ctxt_available_cycles call_context)" + using call_ctxt_inv + by (auto simp: call_context_removal_pre_def call_context_removal_post_def total_cycles_def call_ctxt_carried_cycles list_map_del_sum split: option.splits) + +end + +export_code request_submission_pre request_submission_post + request_rejection_pre request_rejection_post + initiate_canister_call_pre initiate_canister_call_post + call_reject_pre call_reject_post + call_context_create_pre call_context_create_post + call_context_heartbeat_pre call_context_heartbeat_post + message_execution_pre message_execution_post + call_context_starvation_pre call_context_starvation_post + call_context_removal_pre call_context_removal_post +in Haskell module_name IC file_prefix code + +end diff --git a/theories/ROOT b/theories/ROOT new file mode 100644 index 000000000..7db8ae08d --- /dev/null +++ b/theories/ROOT @@ -0,0 +1,8 @@ +chapter IC + +session Internet_Computer (IC) = "HOL-Library" + + options [timeout=600] + theories + IC +export_files (in ".") [2] + "Internet_Computer.IC:code/**" From 368c71c7e524038cc1618f22f657372b7749c322 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Tue, 9 Aug 2022 14:29:35 +0200 Subject: [PATCH 023/102] Allow error codes set by the replica --- spec/index.adoc | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/spec/index.adoc b/spec/index.adoc index 20c892be0..a68d3fc1d 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -402,6 +402,10 @@ If the status is `rejected`, then this path contains the reject code (see </error_code` (text) ++ +If the status is `rejected`, then this path might be present and contain an implementation-specific error code (see <>), else it is not present. + NOTE: Immediately after submitting a request, the request may not show up yet as the Internet Computer is still working on accepting the request as pending. NOTE: Request statuses will not actually be kept around indefinitely, and eventually the Internet Computer forgets about the request. This will happen no sooner than the request’s expiry time, so that replay attacks are prevented. @@ -581,6 +585,7 @@ If the call resulted in a reject, the response is a CBOR map with the following * `status` (`text`): `rejected` * `reject_code` (`nat`): The reject code (see <>). * `reject_message` (`text`): a textual diagnostic message. +* `error_code` (text): an optional implementation-specific textual error code (see <>). Canister methods that do not change the canister state can be executed more efficiently. This method provides that ability, and returns the canister’s response directly within the HTTP response. @@ -732,6 +737,13 @@ The error message is guaranteed to be a string, i.e. not arbitrary binary data. When canisters explicitly reject a message (see <>), they can specify the reject message, but _not_ the reject code; it is always `CANISTER_REJECT`. In this sense, the reject code is trustworthy: If the IC responds with a `SYS_FATAL` reject, then it really was the IC issuing this reject. +[#error-codes] +=== Error codes + +Implementations of the API can provide additional details for rejected messages in the form of a textual label identifying the error condition. +API clients can use these labels to handle errors programmatically or suggest recovery paths to the user. +The specification reserves error codes matching the regular expression `IC[0-9]+` (e.g., `IC502`) for the DFINITY implementation of the API. + [#api-status] === Status endpoint @@ -3726,12 +3738,12 @@ Read response:: * If `F(Arg, Env) = Trap` then + .... -{status: failed; error: "Query execution trapped"} +{status: rejected; reject_code: CANISTER_ERROR, reject_message: , error_code: } .... * Else if `F(Arg, Env) = Return (Reject (code, msg))` then + .... -{status: rejected; reject_code: : reject_message: } +{status: rejected; reject_code: : reject_message: , error_code: } .... * Else if `F(Arg, Env) = Return (Reply R)` then + @@ -3797,7 +3809,7 @@ request_status_tree(Received) = request_status_tree(Processing) = { "status": "processing" } request_status_tree(Rejected (code,msg)) = - { "status": "rejected"; "reject_code": code; "reject_message": msg } + { "status": "rejected"; "reject_code": code; "reject_message": msg, error_code: } request_status_tree(Replied arg) = { "status": "replied"; "reply": arg } request_status_tree(Done) = From 6c3146f0f75d800f36cd768fd36b29088f6fca8f Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Tue, 9 Aug 2022 14:33:27 +0200 Subject: [PATCH 024/102] Add changelog --- spec/changelog.adoc | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 47eddf347..a15cd166b 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,12 @@ [#changelog] == Changelog +[#0_18_6] +=== 0.18.6 (2022-08-09) +* Canister access to performance metrics +* Query calls are rejected when the canister is frozen +* Support for implementation-specific error codes for requests + [#0_18_5] === 0.18.5 (2022-07-08) * Idle consumption of resources in cycles per day can be obtain via `canister_status` method of the management canister From d50507454563b24f7ba19216fe65f8af1b52e8ba Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Tue, 9 Aug 2022 14:36:06 +0200 Subject: [PATCH 025/102] Add changelog item --- spec/changelog.adoc | 1 + 1 file changed, 1 insertion(+) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index a15cd166b..26e70bba7 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -6,6 +6,7 @@ * Canister access to performance metrics * Query calls are rejected when the canister is frozen * Support for implementation-specific error codes for requests +* Formal model in Isabelle [#0_18_5] === 0.18.5 (2022-07-08) From ebcfce1bd354dd6c51217d8ef342b0cae7a45055 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Mon, 15 Aug 2022 11:22:51 +0200 Subject: [PATCH 026/102] Merge more fixes, bump version --- spec/index.adoc | 142 ++++++++++++++++++++++----------------------- spec/requests.cddl | 1 + theories/IC.thy | 47 +++++++-------- 3 files changed, 93 insertions(+), 97 deletions(-) diff --git a/spec/index.adoc b/spec/index.adoc index a68d3fc1d..354cbaf89 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -145,8 +145,6 @@ When the IC creates a _fresh_ id, it never creates a self-authenticating id, an [#textual-ids] ==== Textual representation of principals -NOTE: This textual representation does not actually show up in the interface (which always deals with blobs), so it is merely a recommended convention. - We specify a _canonical textual format_ that is recommended whenever principals need to be printed or read in textual format, e.g. in log messages, transactions browser, command line tools, source code. The textual representation of a blob `b` is `Grouped(Base32(CRC32(b) · b))` where @@ -2262,10 +2260,8 @@ WasmState = (abstract) StableMemory = (abstract) Callback = (abstract) -Arg = { - data : Blob - caller: Principal -} +Arg = Blob; +CallerId = Principal; Timestamp = Nat; Env = { @@ -2299,14 +2295,14 @@ AvailableCycles = Nat RefundedCycles = Nat CanisterModule = { - init : (CanisterId, Arg, Env) -> Trap | Return WasmState + init : (CanisterId, Arg, CallerId, Env) -> Trap | Return WasmState pre_upgrade : (WasmState, Principal, Env) -> Trap | Return StableMemory - post_upgrade : (CanisterId, StableMemory, Arg, Env) -> Trap | Return WasmState - update_methods : MethodName ↦ ((Arg, Env, AvailableCycles) -> UpdateFunc) - query_methods : MethodName ↦ ((Arg, Env) -> QueryFunc) + post_upgrade : (CanisterId, StableMemory, Arg, CallerId, Env) -> Trap | Return WasmState + update_methods : MethodName ↦ ((Arg, CallerId, Env, AvailableCycles) -> UpdateFunc) + query_methods : MethodName ↦ ((Arg, CallerId, Env) -> QueryFunc) heartbeat : (Env) -> WasmState -> Trap | Return WasmState callbacks : (Callback, Response, RefundedCycles, Env, AvailableCycles) -> UpdateFunc - inspect_message : (MethodName, WasmState, Arg, Env) -> Trap | Return (Accept | Reject) + inspect_message : (MethodName, WasmState, Arg, CallerId, Env) -> Trap | Return (Accept | Reject) } .... @@ -2337,7 +2333,7 @@ Request = { sender : UserId; canister_id : CanisterId; method_name : Text; - data : Blob; + arg : Blob; } CallId = (abstract) CallOrigin @@ -2376,7 +2372,7 @@ Message caller : Principal; callee : CanisterId; method_name : Text; - data : Blob; + arg : Blob; transferred_cycles : Nat; queue : Queue; } @@ -2417,7 +2413,7 @@ APIReadRequest sender : UserId; canister_id : CanisterId; method_name : Text; - data : Blob; + arg : Blob; } .... @@ -2499,6 +2495,13 @@ S = { } .... +To convert `CanStatus` into `status : Running | Stopping | Stopped` from `Env`, we define the following conversion function: +.... +simple_status(Running) = Running +simple_status(Stopping _) = Stopping +simple_status(Stopped) = Stopped +.... + ==== Initial state The initial state of the IC is @@ -2632,9 +2635,15 @@ Conditions:: ) ∨ ( E.content.canister_id ≠ ic_principal S.canisters[E.content.canister_id] ≠ EmptyCanister - Arg = { data = E.content.arg; caller = E.content.sender; method = E.content.method_name; time = S.time[E.content.canister_id] } + Env = { + time = S.time[E.content.canister_id]; + balance = S.balances[E.content.canister_id] + freezing_limit = freezing_limit(S, E.content.canister_id); + certificate = NoCertificate; + status = simple_status(S.canister_status[E.content.canister_id]); + } S.canisters[E.content.canister_id].module.inspect_message - (E.content.method_name, C.wasm_state, Arg, S.balance[E.content.canister_id]) = Return Accept + (E.content.method_name, C.wasm_state, E.content.arg, E.content.sender, Env) = Return Accept ) .... State after:: @@ -2755,7 +2764,7 @@ S with FuncMessage { call_context = Ctxt_id; receiver = CM.callee; - entry_point = PublicMethod CM.method_name CM.caller CM.data; + entry_point = PublicMethod CM.method_name CM.caller CM.arg; queue = CM.queue; } · Younger_messages @@ -2827,13 +2836,12 @@ Conditions:: balance = S.balances[M.receiver] freezing_limit = freezing_limit(S, M.receiver); certificate = NoCertificate; - status = S.status[M.receiver]; + status = simple_status(S.canister_status[M.receiver]); } Available = S.call_contexts[M.call_contexts].available_cycles - ( M.entry_point = PublicMethod Name Caller Data - Arg = { data = Data; caller = Caller } - (F = Mod.update_methods[Name](Arg, Env, Available)) or (F = query_as_update(Mod.query_methods[Name], Arg, Env)) + ( M.entry_point = PublicMethod Name Caller Arg + (F = Mod.update_methods[Name](Arg, Caller, Env, Available)) or (F = query_as_update(Mod.query_methods[Name], Arg, Caller, Env)) ) or ( M.entry_point = Callback Callback Response RefundedCycles @@ -3109,7 +3117,7 @@ S with ResponseMessage { origin = M.origin response = candid({ - status = S.canister_status[A.canister_id]; + status = simple_status(S.canister_status[A.canister_id]); module_hash = if S.canisters[A.canister_id] = EmptyCanister then null @@ -3139,18 +3147,14 @@ Conditions:: (A.mode = install && S.canisters[A.canister_id] = EmptyCanister) or A.mode = reinstall M.caller ∈ S.controllers[A.canister_id] - Arg = { - data = A.arg; - caller = M.caller; - } Env = { time = S.time[M.receiver]; balance = S.balances[M.receiver]; freezing_limit = freezing_limit(S, M.receiver); certificate = NoCertificate; - status = S.status[M.receiver]; + status = simple_status(S.canister_status[M.receiver]); } - Mod.init(A.canister_id, Arg, Env) = Return New_state + Mod.init(A.canister_id, A.arg, M.caller, Env) = Return New_state dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ .... State after:: @@ -3188,14 +3192,10 @@ Conditions:: balance = S.balances[M.receiver]; freezing_limit = freezing_limit(S, M.receiver); certificate = NoCertificate; - status = S.status[M.receiver]; + status = simple_status(S.canister_status[M.receiver]); } Old_module.pre_upgrade(Old_State, M.caller, Env) = Return Stable_memory - Arg = { - data = A.arg; - caller = M.caller; - } - Mod.post_upgrade(A.canister_id, Stable_memory, Arg, Env) = Return New_state + Mod.post_upgrade(A.canister_id, Stable_memory, A.arg, M.caller, Env) = Return New_state dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ .... State after:: @@ -3278,7 +3278,7 @@ State after:: .... S with messages = Older_messages · Younger_messages - S.status[A.canister_id] = Stopping [(M.origin, M.transferred_cycles)] + canister_status[A.canister_id] = Stopping [(M.origin, M.transferred_cycles)] .... The next two transitions record any additional 'stop_canister' requests that arrive at a stopping (or stopped) canister in its status. @@ -3297,7 +3297,7 @@ State after:: .... S with messages = Older_messages · Younger_messages - S.status[A.canister_id] = Stopping (Origins · (M.origin, M.transferred_cycles)) + canister_status[A.canister_id] = Stopping (Origins · [(M.origin, M.transferred_cycles)]) .... @@ -3310,8 +3310,9 @@ Conditions:: .... State after:: .... - S.canister_status[CanisterId] = Stopped - S.messages = Messages · +S with + canister_status[CanisterId] = Stopped + messages = Messages · [ ResponseMessage { origin = O response = Accepted (candid()) @@ -3355,13 +3356,13 @@ Conditions:: M.callee = ic_principal M.method_name = 'start_canister' M.arg = candid(A) - S.status[A.canister_id] = Running or S.status[A.canister_id] = Stopped + S.canister_status[A.canister_id] = Running or S.canister_status[A.canister_id] = Stopped M.caller ∈ S.controllers[A.canister_id] .... State after:: .... S with - S.status[A.canister_id] = Running + canister_status[A.canister_id] = Running messages = Older_messages · Younger_messages · ResponseMessage{ origin = M.origin @@ -3380,13 +3381,13 @@ Conditions:: M.callee = ic_principal M.method_name = 'start_canister' M.arg = candid(A) - S.status[A.canister_id] = Stopping Origins + S.canister_status[A.canister_id] = Stopping Origins M.caller ∈ S.controllers[A.canister_id] .... State after:: .... S with - S.status[A.canister_id] = Running + canister_status[A.canister_id] = Running messages = Older_messages · Younger_messages · ResponseMessage{ origin = M.origin @@ -3559,7 +3560,7 @@ S with Younger_messages .... -If the responded call context does not exist anymore, because the canister has been uninstalled since, the refundend cycles are still added to the canister balance, but +If the responded call context does not exist anymore, because the canister has been uninstalled since, the refunded cycles are still added to the canister balance, but no function invocation is enqueued: Conditions:: @@ -3719,10 +3720,6 @@ Conditions:: S.canister_status[Q.canister_id] = Running ∧ S.balances[Q.canister_id] >= freezing_limit(S, Q.canister_id) C = S.canisters[Q.canister_id] F = C.module.query_methods[Q.method_name] - Arg = { - data = Q.arg; - caller = Q.sender; - } verify_cert(Cert) lookup(["canister",Q.canister_id,"certified_data"], Cert) = Found S.certified_data[Q.canister_id] lookup(["time"], Cert) = Found S.system_time // or “recent enough” @@ -3731,21 +3728,21 @@ Conditions:: balance = S.balances[Q.canister_id]; freezing_limit = freezing_limit(S, Q.canister_id); certificate = Cert; - status = S.status[Q.receiver]; + status = simple_status(S.canister_status[Q.receiver]); } .... Read response:: -* If `F(Arg, Env) = Trap` then +* If `F(Q.Arg, Q.sender, Env) = Trap` then + .... {status: rejected; reject_code: CANISTER_ERROR, reject_message: , error_code: } .... -* Else if `F(Arg, Env) = Return (Reject (code, msg))` then +* Else if `F(Q.Arg, Q.sender, Env) = Return (Reject (code, msg))` then + .... {status: rejected; reject_code: : reject_message: , error_code: } .... -* Else if `F(Arg, Env) = Return (Reply R)` then +* Else if `F(Q.Arg, Q.sender, Env) = Return (Reply R)` then + .... {status: success; reply: { arg : } } @@ -3856,13 +3853,13 @@ Callback = { We can model the execution of WebAssembly functions as stateful functions that have access to the WebAssembly store. In order to also model the behavior of the system imports, which have access to additional data structures, we extend the state as follows: .... Params = { - data : NoData | Blob; + arg : NoArg | Blob; caller : NoCaller | Principal; reject_code : 0 | SYS_FATAL | SYS_TRANSIENT | …; reject_message : Text; sysenv : Env; cycles_refunded : Nat; - method_name : Text; + method_name : NoText | Text; } ExecutionState = { wasm_state : WasmState; @@ -3894,11 +3891,12 @@ Finally we can specify the abstract `CanisterModule` that models a concrete WebA + .... empty_params = { - data = NoData; + arg = NoArg; caller = NoCaller; reject_code = 0; reject_message = ""; cycles_refunded = 0; + method_name = NoText; } empty_execution_state = { @@ -3922,17 +3920,17 @@ empty_execution_state = { If the WebAssembly module does not export a function called under the name `canister_init`, then the argument blob is ignored and the `initial_wasm_store` is returned: + .... -init = λ (self_id, arg, sysenv) → +init = λ (self_id, arg, caller, sysenv) → Return { store = initial_wasm_store; self_id = self_id; stable_mem = "" } .... + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_init`, it is + .... -init = λ (self_id, arg, sysenv) → +init = λ (self_id, arg, caller, sysenv) → let es = ref {empty_execution_state with wasm_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" } - params = empty_params with { data = arg.data; caller = arg.caller; sysenv } + params = empty_params with { arg = arg; caller = caller; sysenv } balance = sysenv.balance } try func() with Trap then Trap @@ -3974,17 +3972,17 @@ pre_upgrade = λ (old_state, caller, sysenv) → If the WebAssembly module does not export a function called under the name `canister_post_upgrade`, then the argument blob is ignored and the `initial_wasm_store` is returned: + .... -post_upgrade = λ (self_id, stable_mem, arg, sysenv) → +post_upgrade = λ (self_id, stable_mem, arg, caller, sysenv) → Return { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem } .... + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_post_upgrade`, it is + .... -post_upgrade = λ (self_id, stable_mem, arg, sysenv) → +post_upgrade = λ (self_id, stable_mem, arg, caller, sysenv) → let es = ref {empty_execution_state with wasm_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem } - params = { empty_params with data = arg.data; caller = arg.caller; sysenv } + params = { empty_params with arg = arg; caller = caller; sysenv } balance = sysenv.balance } try func() with Trap then Trap @@ -3997,12 +3995,12 @@ post_upgrade = λ (self_id, stable_mem, arg, sysenv) → * The partial map `update_methods` of the `CanisterModule` is defined for all method names `method` for which the WebAssembly program exports a function `func` named `canister_update `, and has value + .... -update_methods[method] = λ (arg, sysenv, available) → λ wasm_state → +update_methods[method] = λ (arg, caller, sysenv, available) → λ wasm_state → let es = ref {empty_execution_state with wasm_state = wasm_state; - params = empty_params with { data = arg.data; caller = arg.caller; sysenv } + params = empty_params with { arg = arg; caller = caller; sysenv } balance = sysenv.balance - cycles_available = arg.cycles; + cycles_available = available; } try func() with Trap then Trap if es.ingress_filter ≠ Reject then Trap @@ -4018,10 +4016,10 @@ update_methods[method] = λ (arg, sysenv, available) → λ wasm_state → * The partial map `query_methods` of the `CanisterModule` is defined for all method names `method` for which the WebAssembly program exports a function `func` named `canister_query `, and has value + .... -query_methods[method] = λ (arg, sysenv) → λ wasm_state → +query_methods[method] = λ (arg, caller, sysenv) → λ wasm_state → let es = ref {empty_execution_state with wasm_state = wasm_state; - params = empty_params with { data = arg.data; caller = arg.caller; sysenv } + params = empty_params with { arg = arg; caller = caller; sysenv } balance = sysenv.balance } try func() with Trap then Trap @@ -4042,7 +4040,7 @@ By construction, the (possibly modified) `es.wasm_state` is discarded. heartbeat = λ (sysenv) → λ wasm_state → let es = ref {empty_execution_state with wasm_state = wasm_state; - params = empty_params with { data = NoData; caller = NoCaller; sysenv } + params = empty_params with { arg = NoArg; caller = NoCaller; sysenv } balance = sysenv.balance } try func() with Trap then Trap @@ -4116,19 +4114,19 @@ present) is executed, and the canister has the chance to update its state. If the WebAssembly module does not export a function called under the name `canister_inspect_message`, then access is always granted: + .... -inspect_message = λ (method_name, wasm_state, arg, sysenv) → +inspect_message = λ (method_name, wasm_state, arg, caller, sysenv) → Return Accept .... Otherwise, if the WebAssembly module exports a function `func` under the name `canister_inspect_message`, it is + .... -inspect_message = λ (method_name, wasm_state, arg, sysenv) → +inspect_message = λ (method_name, wasm_state, arg, caller, sysenv) → let es = ref {empty_execution_state with wasm_state = wasm_state; params = empty_params with { - data = arg.data; - caller = arg.caller; - method_name = arg.method_name; + arg = arg; + caller = caller; + method_name = method_name; sysenv } balance = sysenv.balance; @@ -4176,7 +4174,7 @@ ic0.msg_arg_data_size() : i32 = return |es.params.arg| ic0.msg_arg_data_copy(dst:i32, offset:i32, size:i32) = - copy_to_canister(dst, offset, size, es.param.arg) + copy_to_canister(dst, offset, size, es.params.arg) ic0.msg_caller_size() : i32 = return |es.params.caller| diff --git a/spec/requests.cddl b/spec/requests.cddl index 21d1f9f28..f964bafad 100644 --- a/spec/requests.cddl +++ b/spec/requests.cddl @@ -70,6 +70,7 @@ query-response = tagged<{ status: "rejected" reject_code: unsigned reject_message: text + ? error_code: text }> call-reply = { diff --git a/theories/IC.thy b/theories/IC.thy index 193784430..8aaac612b 100644 --- a/theories/IC.thy +++ b/theories/IC.thy @@ -83,9 +83,7 @@ lemma list_map_del_sum: "list_map_get f x = Some y \ list_map_su (* Abstract canisters *) -record ('b, 'p) arg = - data :: 'b - caller :: 'p +type_synonym 'b arg = 'b type_synonym timestamp = nat datatype status = Running | Stopping | Stopped @@ -121,14 +119,14 @@ type_synonym refunded_cycles = nat datatype inspect_method_result = Accept | Reject record ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module_rec = - init :: "'canid \ ('b, 'p) arg \ 'b env \ 'tr + 'w" + init :: "'canid \ 'b arg \ 'p \ 'b env \ 'tr + 'w" pre_upgrade :: "'w \ 'p \ 'b env \ 'tr + 'sm" - post_upgrade :: "'canid \ 'sm \ ('b, 'p) arg \ 'b env \ 'tr + 'w" - update_methods :: "('s, (('b, 'p) arg \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func) list_map" - query_methods :: "('s, (('b, 'p) arg \ 'b env) \ ('w, 'b, 's, 'tr) query_func) list_map" + post_upgrade :: "'canid \ 'sm \ 'b arg \ 'p \ 'b env \ 'tr + 'w" + update_methods :: "('s, ('b arg \ 'p \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func) list_map" + query_methods :: "('s, ('b arg \ 'p \ 'b env) \ ('w, 'b, 's, 'tr) query_func) list_map" heartbeat :: "'b env \ 'w \ 'tr + 'w" callbacks :: "('c \ ('b, 's) response \ refunded_cycles \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func" - inspect_message :: "('s \ 'w \ ('b, 'p) arg \ 'b env) \ 'tr + inspect_method_result" + inspect_message :: "('s \ 'w \ 'b arg \ 'p \ 'b env) \ 'tr + inspect_method_result" typedef ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module = "{m :: ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module_rec. list_map_dom (update_methods m) \ list_map_dom (query_methods m) = {}}" by (auto intro: exI[of _ "\init = undefined, pre_upgrade = undefined, post_upgrade = undefined, @@ -138,8 +136,8 @@ typedef ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module = setup_lifting type_definition_canister_module lift_definition dispatch_method :: "'s \ ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module \ - (((('b, 'p) arg \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func) + - ((('b, 'p) arg \ 'b env) \ ('w, 'b, 's, 'tr) query_func)) option" is + ((('b arg \ 'p \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func) + + (('b arg \ 'p \ 'b env) \ ('w, 'b, 's, 'tr) query_func)) option" is "\f m. case list_map_get (update_methods m) f of Some f' \ None | None \ (case list_map_get (query_methods m) f of Some f' \ None | None \ None)" . lift_definition canister_module_callbacks :: "('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module \ @@ -157,7 +155,7 @@ record ('b, 'p, 'uid, 'canid, 's) request = sender :: 'uid canister_id :: 'canid method_name :: 's - data :: 'b + arg :: 'b datatype ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin = From_user "('b, 'p, 'uid, 'canid, 's) request" | From_canister "'cid" "'c" @@ -238,7 +236,7 @@ record ('b, 'uid, 'canid, 's) CanisterQuery = sender :: 'uid canister_id :: 'canid method_name :: 's - data :: 'b + arg :: 'b type_synonym ('b, 'uid, 'canid, 's) APIReadRequest = "('b, 'uid) StateRead + ('b, 'uid, 'canid, 's) CanisterQuery" record ('b, 'p, 'uid, 'canid, 's, 'pk, 'sig, 'sd) envelope = content :: "('b, 'p, 'uid, 'canid, 's) request + ('b, 'uid, 'canid, 's) APIReadRequest" @@ -335,10 +333,10 @@ fun message_queue :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message \ (* Type conversion functions *) -fun to_status :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status \ status" where - "to_status can_status.Running = status.Running" -| "to_status (can_status.Stopping _) = status.Stopping" -| "to_status can_status.Stopped = status.Stopped" +fun simple_status :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status \ status" where + "simple_status can_status.Running = status.Running" +| "simple_status (can_status.Stopping _) = status.Stopping" +| "simple_status can_status.Stopped = status.Stopped" @@ -384,7 +382,7 @@ definition initiate_canister_call_post :: "('b, 'p, 'uid, 'canid, 's) request \< "initiate_canister_call_post req S = S\requests := list_map_set (requests S) req Processing, messages := Call_message (From_user req) (principal_of_uid (request.sender req)) (request.canister_id req) (request.method_name req) - (request.data req) 0 Unordered # messages S\" + (request.arg req) 0 Unordered # messages S\" lemma initiate_canister_call_cycles_inv: "initiate_canister_call_pre R S \ @@ -511,7 +509,7 @@ lemma call_context_heartbeat_cycles_inv: (* System transition: Message execution [DONE] *) -fun query_as_update :: "((('b, 'p) arg \ 'b env) \ ('w, 'b, 's, 'tr) query_func) \ ('b, 'p) arg \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func" where +fun query_as_update :: "(('b arg \ 'p \ 'b env) \ ('w, 'b, 's, 'tr) query_func) \ 'b arg \ 'p \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func" where "query_as_update (f, a, e) = (\w. case f (a, e) w of Inl t \ Inl t | Inr res \ Inr \new_state = w, new_calls = [], new_certified_data = None, response = Some res, cycles_accepted = 0\)" @@ -522,9 +520,8 @@ fun heartbeat_as_update :: "('b env \ 'w \ 'tr + 'w) \ 'b env \ nat \ ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func" where "exec_function (entry_point.Public_method mn c d) e bal m = ( - let arg = \arg.data = d, arg.caller = c\ in - case dispatch_method mn m of Some (Inl upd) \ upd (arg, e, bal) - | Some (Inr query) \ query_as_update (query, arg, e) | None \ + case dispatch_method mn m of Some (Inl upd) \ upd (d, c, e, bal) + | Some (Inr query) \ query_as_update (query, d, c, e) | None \ undefined)" | "exec_function (entry_point.Callback cb resp ref_cycles) e bal m = canister_module_callbacks m (cb, resp, ref_cycles, e, bal)" @@ -546,7 +543,7 @@ definition message_execution_post :: "nat \ nat \ ('p, ' (Some (Some can), Some bal, Some can_status, Some t, Some ctxt) \ ( let Mod = module can; Is_response = (case ep of Callback _ _ _ \ True | _ \ False); - Env = \env_time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = to_status can_status\; + Env = \env_time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\; Available = call_ctxt_available_cycles ctxt; F = exec_function ep Env Available Mod; R = F (wasm_state can); @@ -562,7 +559,7 @@ definition message_execution_post :: "nat \ nat \ ('p, ' (no_response \ call_ctxt_needs_to_respond ctxt) then (let result = projr R; new_call_to_message = (\call. Call_message (From_canister ctxt_id (callback call)) (principal_of_canid recv) - (callee call) (method_call.method_name call) (arg call) (transferred_cycles call) (Queue (Canister recv) (callee call))); + (callee call) (method_call.method_name call) (method_call.arg call) (transferred_cycles call) (Queue (Canister recv) (callee call))); response_messages = (case response result of None \ [] | Some resp \ [Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res)]); messages = take n (messages S) @ drop (Suc n) (messages S) @ map new_call_to_message new_calls_res @ response_messages; @@ -597,7 +594,7 @@ proof - by (auto simp: message_execution_pre_def split: message.splits option.splits) define Mod where "Mod = can_state_rec.module can" define Is_response where "Is_response = (case ep of Callback _ _ _ \ True | _ \ False)" - define Env :: "'b env" where "Env = \env_time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = to_status can_status\" + define Env :: "'b env" where "Env = \env_time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\" define Available where "Available = call_ctxt_available_cycles ctxt" define F where "F = exec_function ep Env Available Mod" define R where "R = F (wasm_state can)" @@ -648,7 +645,7 @@ proof - | Some resp \ [Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res)])" define new_call_to_message :: "(?'p, 'canid, 's, 'b, 'c) method_call \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message" where "new_call_to_message = (\call. Call_message (From_canister ctxt_id (callback call)) (principal_of_canid recv) - (callee call) (method_call.method_name call) (arg call) (transferred_cycles call) (Queue (Canister recv) (callee call)))" + (callee call) (method_call.method_name call) (method_call.arg call) (transferred_cycles call) (Queue (Canister recv) (callee call)))" define messages where "messages = take n (ic.messages S) @ drop (Suc n) (ic.messages S) @ map new_call_to_message new_calls_res @ response_messages" define new_ctxt where "new_ctxt = (case response result of None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt From a156891eaa72dfa8cea57a7f29641af423467b2e Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Mon, 15 Aug 2022 11:29:31 +0200 Subject: [PATCH 027/102] Bump version, now for real --- spec/index.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/index.adoc b/spec/index.adoc index 354cbaf89..968949643 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,6 +1,6 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.5 +0.18.6 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ From 36aea6c52fd6241a0bfadbdefa9b8bd6e72f61ff Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Mon, 15 Aug 2022 11:30:16 +0200 Subject: [PATCH 028/102] Add remark on big-endian CRC --- spec/index.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/index.adoc b/spec/index.adoc index 968949643..f51bb9c74 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -149,7 +149,7 @@ We specify a _canonical textual format_ that is recommended whenever principals The textual representation of a blob `b` is `Grouped(Base32(CRC32(b) · b))` where - * `CRC32` is a four byte check sequence, calculated as defined by ISO 3309, ITU-T V.42 and https://www.w3.org/TR/2003/REC-PNG-20031110/#5CRC-algorithm[elsewhere] + * `CRC32` is a four byte check sequence, calculated as defined by ISO 3309, ITU-T V.42, and https://www.w3.org/TR/2003/REC-PNG-20031110/#5CRC-algorithm[elsewhere], and stored as big-endian, i.e., the most significant byte comes first and then the less significant bytes come in descending order of significance (MSB B2 B1 LSB). * `Base32` is the Base32 encoding as defined in https://tools.ietf.org/html/rfc4648#section-6[RFC 4648], with no padding character added. * The middle dot denotes concatenation. * `Grouped` takes an ASCII string and inserts the separator `-` (dash) every 5 characters. The last group may contain less than 5 characters. A separator never appears at the beginning or end. From f9c473e7127a8103b61bd8193097efc8bd982c17 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Mon, 22 Aug 2022 13:49:21 +0200 Subject: [PATCH 029/102] Full isabelle model --- spec/index.adoc | 540 ++++++----- theories/IC.thy | 2421 +++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 2549 insertions(+), 412 deletions(-) diff --git a/spec/index.adoc b/spec/index.adoc index f51bb9c74..9bee7368a 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -595,9 +595,7 @@ The `` in the URL paths of requests is the _effective_ de * If the call is to the Management Canister (`aaaaa-aa`), then: - If the `arg` is Candid-encoded where the first argument is a record with a `canister_id` field of type `principal`, then the effective canister id is that principal. - Otherwise, if the call is to the `provisional_create_canister_with_cycles` method, then any principal is a valid effective canister id for this call. - - Otherwise, there is no effective canister id. In particular, the `create_canister` method has no effective canister ID, and this method cannot be called by users, only via canisters. - -* If the call is to the `raw_rand` method of the Management Canister (`aaaaa-aa`), then there is no effective canister id. This implies that this method cannot be called by users, only via canisters. + - Otherwise, there is no effective canister id. In particular, the `create_canister` and `raw_rand` methods have no effective canister ID, and these methods cannot be called by users, only via canisters. + [NOTE] @@ -2282,27 +2280,47 @@ MethodCall = { callback: Callback; } -UpdateFunc = WasmState -> Trap | Return { +UpdateFunc = WasmState -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; new_calls : List MethodCall; new_certified_data : NoCertifiedData | Blob response : NoResponse | Response; cycles_accepted : Nat; + cycles_used : Nat; +} +QueryFunc = WasmState -> Trap { cycles_used : Nat; } | Return { + response : Response; + cycles_used : Nat; +} +HeartbeatFunc = WasmState -> Trap { cycles_used : Nat; } | Return { + new_state : WasmState; + cycles_used : Nat; } -QueryFunc = WasmState -> Trap | Return Response AvailableCycles = Nat RefundedCycles = Nat CanisterModule = { - init : (CanisterId, Arg, CallerId, Env) -> Trap | Return WasmState - pre_upgrade : (WasmState, Principal, Env) -> Trap | Return StableMemory - post_upgrade : (CanisterId, StableMemory, Arg, CallerId, Env) -> Trap | Return WasmState + init : (CanisterId, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { + new_state : WasmState; + cycles_used : Nat; + } + pre_upgrade : (WasmState, Principal, Env) -> Trap { cycles_used : Nat; } | Return { + stable_memory : StableMemory; + cycles_used : Nat; + } + post_upgrade : (CanisterId, StableMemory, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { + new_state : WasmState; + cycles_used : Nat; + } update_methods : MethodName ↦ ((Arg, CallerId, Env, AvailableCycles) -> UpdateFunc) query_methods : MethodName ↦ ((Arg, CallerId, Env) -> QueryFunc) - heartbeat : (Env) -> WasmState -> Trap | Return WasmState + heartbeat : (Env) -> HeartbeatFunc callbacks : (Callback, Response, RefundedCycles, Env, AvailableCycles) -> UpdateFunc - inspect_message : (MethodName, WasmState, Arg, CallerId, Env) -> Trap | Return (Accept | Reject) + inspect_message : (MethodName, WasmState, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { + status : Accept | Reject; + cycles_used : Nat; + } } .... @@ -2312,9 +2330,11 @@ The `CanisterId` parameter of `init` and `post_upgrade` is merely passed through The `Env` parameter provides synchronous read-only access to portions of the system state and canister metadata that are always available. -The parsing of a blob to a canister module is modelled via the (possibly implicitly failing) function +The parsing of a blob to a canister module and its public and private custom sections is modelled via the (possibly implicitly failing) functions .... parse_wasm_mod : Blob -> CanisterModule +parse_public_custom_sections : Blob -> Text ↦ Blob +parse_private_custom_sections : Blob -> Text ↦ Blob .... The concrete mapping of this abstract `CanisterModule` to actual WebAssembly concepts and the System API is described separately in section <>. @@ -2417,6 +2437,27 @@ APIReadRequest } .... +Signed delegations contain the (unsigned) delegation data in a nested record, next to the signature of that data. +.... +PublicKey = Blob +Signature = Blob +SignedDelegation = { + delegation : { + pubkey : PublicKey; + targets : [CanisterId] | Unrestricted; + senders : [Principal] | Unrestricted; + expiration : Timestamp + }; + signature : Signature +} +.... + +For the signatures in a `Request`, we assume that the following function implements signature verification as described in <>. +This function picks the corresponding signature scheme according to the DER-encoded metadata in the public key. +.... +verify_signature : PublicKey -> Signature -> Blob -> Bool +.... + .... Envelope = { content : Request | APIReadRequest; @@ -2442,26 +2483,6 @@ RequestId = Blob hash_of_map: Request -> RequestId .... -For the signatures in a `Request`, we assume that the following function implements signature verification as described in <>. -This function picks the corresponding signature scheme according to the DER-encoded metadata in the public key. -.... -PublicKey = Blob -Signature = Blob -verify_signature : PublicKey -> Signature -> Blob -> Bool -.... - -Signed delegations contain the (unsigned) delegation data in a nested record, next to the signature of that data. -.... -SignedDelegation = { - delegation : { - pubkey : PublicKey; - targets : [CanisterId] | Unrestricted; - expiration : Timestamp - }; - signature : Signature -} -.... - ==== The system state Finally, we can describe the state of the IC as a record having the following fields: @@ -2517,7 +2538,7 @@ The initial state of the IC is certified_data = (); system_time = T; call_contexts = (); - messages = (); + messages = []; root_key = PublicKey; } .... @@ -2548,14 +2569,13 @@ The following is an incomplete list of invariants that should hold for the abstr if Ctxt.needs_to_respond = false then Ctxt.available_cycles = 0 .... -* Referenced call contexts exists: +* Referenced call contexts exist: + .... -∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ∈ S.call_contexts -∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ∈ S.call_contexts -∀ _ ↦ Ctxt ∈ S.call_contexts: - if Ctx.needs_to_respond: - Ctxt.origin.calling_context ∈ S.call_contexts +∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ∈ dom(S.call_contexts) +∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ∈ dom(S.call_contexts) +∀ _ ↦ {needs_to_respond = true, origin = FromCanister O, …} ∈ S.call_contexts: O.calling_context ∈ dom(S.call_contexts) +∀ _ ↦ Stopping Origins ∈ S.canister_status: ∀(FromCanister O, _) ∈ Origins. O.calling_context ∈ dom(S.call_contexts) .... === State transitions @@ -2619,19 +2639,29 @@ Requests that have expired are dropped here. Ingress message inspection is applied, and messages that are not accepted by the canister are dropped. +The (unspecified) function `idle_cycles_burned_rate(S, cid)` determines the idle resource consumption rate in cycles per day of the canister with id `cid`, +given its current memory footprint, compute and storage cost, and memory and compute allocation. +The function `freezing_limit(S, cid)` determines the freezing threshold in cycles of the canister with id `cid`, +given its current memory footprint, compute and storage cost, memory and compute allocation, and current `freezing_threshold` setting. +The value `freezing_limit(S, cid)` is derived from `idle_cycles_burned_rate(S, cid)` and `freezing_threshold` as follows: +.... + freezing_limit(S, cid) = idle_cycles_burned_rate(S, cid) * S.freezing_threshold[cid] / (24 * 60 * 60) +.... + Submitted request:: `E : Envelope` Conditions:: .... E.content.canister_id ∈ verify_envelope(E, E.content.sender, S.system_time) - E.content ∉ requests + E.content ∉ dom(S.requests) S.system_time <= E.content.ingress_expiry is_effective_canister_id(E.content, ECID) ( E.content.canister_id = ic_principal E.content.arg = candid({canister_id = CanisterId, …}) E.content.sender ∈ S.controllers[CanisterId] E.content.method_name ∈ - { "install_code", "update_settings", "start_canister", "stop_canister", - "canister_status", "delete_canister" } + { "install_code", "uninstall_code", "update_settings", "start_canister", "stop_canister", + "canister_status", "delete_canister", + "provisional_create_canister_with_cycles", "provisional_top_up_canister" } ) ∨ ( E.content.canister_id ≠ ic_principal S.canisters[E.content.canister_id] ≠ EmptyCanister @@ -2643,13 +2673,16 @@ Conditions:: status = simple_status(S.canister_status[E.content.canister_id]); } S.canisters[E.content.canister_id].module.inspect_message - (E.content.method_name, C.wasm_state, E.content.arg, E.content.sender, Env) = Return Accept + (E.content.method_name, S.canisters[E.content.canister_id].wasm_state, E.content.arg, E.content.sender, Env) = Return {status = Accept; cycles_used = Cycles_used;} + Cycles_used ≤ S.balances[E.content.canister_id] ) .... State after:: .... S with requests[E.content] = Received + if E.content.canister_id ≠ ic_principal then + balances[E.content.canister_id] = S.balances[E.content.canister_id] - Cycles_used .... NOTE: This is not instantaneous (the IC takes some time to agree it accepts the request) nor guaranteed (a node could just drop the request, or maybe it did not pass validation). But once the request has entered the IC state like this, it will be acted upon. @@ -2704,15 +2737,6 @@ S with A call to a canister which is stopping, stopped, or frozen is automatically rejected. -The (unspecified) function `idle_cycles_burned_rate(S, cid)` determines the idle resource consumption rate in cycles per day of the canister with id `cid`, -given its current memory footprint, compute and storage cost, and memory and compute allocation. -The function `freezing_limit(S, cid)` determines the freezing threshold in cycles of the canister with id `cid`, -given its current memory footprint, compute and storage cost, memory and compute allocation, and current `freezing_threshold` setting. -The value `freezing_limit(S, cid)` is derived from `idle_cycles_burned_rate(S, cid)` and `freezing_threshold` as follows: -.... - freezing_limit(S, cid) = idle_cycles_burned_rate(S, cid) * S.freezing_threshold[cid] / (24 * 60 * 60) -.... - Conditions:: .... S.messages = Older_messages · CallMessage CM · Younger_messages @@ -2752,7 +2776,7 @@ Conditions:: S.canisters[CM.callee] ≠ EmptyCanister S.canister_status[CM.callee] = Running S.balances[CM.callee] ≥ freezing_limit(S, CM.callee) + MAX_CYCLES_PER_MESSAGE - Ctxt_id ∉ dom S.call_contexts + Ctxt_id ∉ dom(S.call_contexts) .... + State after:: @@ -2775,7 +2799,7 @@ S with deleted = false; available_cycles = CM.transferred_cycles; } - balances[CM.callee] = balances[CM.callee] - MAX_CYCLES_PER_MESSAGE + balances[CM.callee] = S.balances[CM.callee] - MAX_CYCLES_PER_MESSAGE .... * Call context creation: Heartbeat @@ -2788,7 +2812,7 @@ Conditions:: S.canisters[C] ≠ EmptyCanister S.canister_status[C] = Running balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE - Ctxt_id ∉ dom S.call_contexts + Ctxt_id ∉ dom(S.call_contexts) .... + State after:: @@ -2810,7 +2834,7 @@ S with deleted = false; available_cycles = 0; } - balances[C] = balances[C] - MAX_CYCLES_PER_MESSAGE + balances[C] = S.balances[C] - MAX_CYCLES_PER_MESSAGE .... The IC can execute any message that is at the head of its queue, i.e. there is no older message with the same abstract `queue` field. The actual message execution, if successful, may enqueue further messages and -- if the function returns a response -- record this response. The new call and response messages are enqueued at the end. @@ -2858,14 +2882,14 @@ State after:: .... if R = Return res - Cycles_used ≤ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + res.cycles_used ≤ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) res.cycles_accepted ≤ Available - (Cycles_used + ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ]) ≤ + (res.cycles_used + ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ]) ≤ (S.balances[M.receiver] + res.cycles_accepted + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) New_balance = (S.balances[M.receiver] + res.cycles_accepted + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) - - (Cycles_used + ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ]) - New_balance > if Is_response then 0 else freezing_limit(S, CM.callee); + - (res.cycles_used + ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ]) + New_balance ≥ if Is_response then 0 else freezing_limit(S, M.receiver); (res.response = NoResponse) or S.call_contexts[M.call_context].needs_to_respond then S with @@ -2907,8 +2931,8 @@ else S with messages = Older_messages · Younger_messages balances[M.receiver] = - S.balances[M.receiver] + - ((if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - Cycles_used) + (S.balances[M.receiver] + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) + - min (R.cycles_used, (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) .... @@ -2935,24 +2959,26 @@ The functions `query_as_update` and `heartbeat_as_update` turns a query function .... query_as_update(f, arg, env) = λ wasm_state → match f(arg, env)(wasm_state) with - Trap → Trap + Trap trap → Trap trap Return res → Return { new_state = wasm_state; new_calls = []; new_certified_data = NoCertifiedData; - response = res; + response = res.response; cycles_accepted = 0; + cycles_used = res.cycles_used; } heartbeat_as_update(f, env) = λ wasm_state → match f(env)(wasm_state) with - Trap → Trap - Return wasm_state → Return { - new_state = wasm_state; + Trap trap → Trap trap + Return res → Return { + new_state = res.new_state; new_calls = []; new_certified_data = NoCertifiedData; response = NoResponse; cycles_accepted = 0; + cycles_used = res.cycles_used; } .... Note that by construction, a query function will either trap or return with a response; it will never send calls, and it will never change the state of the canister. @@ -2968,9 +2994,7 @@ Conditions:: S.call_contexts[Ctxt_id].origin ≠ FromHeartbeat ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id ∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id - ∀ ctxt_ids ∈ dom(S.call_contexts). - S.call_contexts[ctxt_ids].needs_to_respond - ==> S.call_contexts[ctxt_ids].origin.calling_context ≠ Ctxt_id + ∀ _ ↦ {needs_to_respond = true, origin = FromCanister O, …} ∈ S.call_contexts: O.calling_context ≠ Ctxt_id .... State after:: .... @@ -3001,9 +3025,8 @@ Conditions:: ) ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id ∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id - ∀ ctxt_ids. - S.call_contexts[ctxt_ids].needs_to_respond = true - ==> S.call_contexts[ctxt_ids].origin.calling_context ≠ Ctxt_id + ∀ _ ↦ {needs_to_respond = true, origin = FromCanister O, …} ∈ S.call_contexts: O.calling_context ≠ Ctxt_id + ∀ _ ↦ Stopping Origins ∈ S.canister_status: ∀(FromCanister O, _) ∈ Origins. O.calling_context ≠ Ctxt_id .... State after:: .... @@ -3029,7 +3052,7 @@ Conditions:: M.method_name = 'create_canister' M.arg = candid(A) is_system_assigned CanisterId - CanisterId ∉ dom S.canisters + CanisterId ∉ dom(S.canisters) .... State after:: .... @@ -3040,12 +3063,13 @@ S with controllers[CanisterId] = A.settings.controllers else: controllers[CanisterId] = [M.caller] + freezing_threshold[CanisterId] = 2592000 balances[CanisterId] = M.transferred_cycles certified_data[CanisterId] = "" messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin - response = Accepted (candid({canister_id = CanisterId})) + response = Reply (candid({canister_id = CanisterId})) refunded_cycles = 0 } canister_status[CanisterId] = Running @@ -3083,12 +3107,12 @@ State after:: S with if A.settings.controllers is not null: controllers[A.canister_id] = A.settings.controllers - if A.settings.freezing_threshold exists: + if A.settings.freezing_threshold is not null: freezing_threshold[A.canister_id] = A.settings.freezing_threshold messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin - response = Accepted (candid()) + response = Reply (candid()) refunded_cycles = M.transferred_cycles } .... @@ -3121,10 +3145,10 @@ S with module_hash = if S.canisters[A.canister_id] = EmptyCanister then null - else opt (SHA-265(S.canisters[A.canister_id].raw_module)); + else opt (SHA-256(S.canisters[A.canister_id].raw_module)); controllers = S.controllers[A.canister_id]; memory_size = Memory_size; - cycles = S.balance[A.canister_id]; + cycles = S.balances[A.canister_id]; freezing_threshold = S.freezing_threshold[A.canister_id]; idle_cycles_burned_per_day = idle_cycles_burned_rate(S, A.canister_id); }) @@ -3144,29 +3168,38 @@ Conditions:: M.method_name = 'install_code' M.arg = candid(A) Mod = parse_wasm_mod(A.wasm_module) - (A.mode = install && S.canisters[A.canister_id] = EmptyCanister) - or A.mode = reinstall + Public_custom_sections = parse_public_custom_sections(A.wasm_module); + Private_custom_sections = parse_private_custom_sections(A.wasm_module); + (A.mode = install and S.canisters[A.canister_id] = EmptyCanister) or A.mode = reinstall M.caller ∈ S.controllers[A.canister_id] Env = { - time = S.time[M.receiver]; - balance = S.balances[M.receiver]; - freezing_limit = freezing_limit(S, M.receiver); + time = S.time[A.canister_id]; + balance = S.balances[A.canister_id]; + freezing_limit = freezing_limit(S, A.canister_id); certificate = NoCertificate; - status = simple_status(S.canister_status[M.receiver]); + status = simple_status(S.canister_status[A.canister_id]); } - Mod.init(A.canister_id, A.arg, M.caller, Env) = Return New_state + Mod.init(A.canister_id, A.arg, M.caller, Env) = Return {new_state = New_state; cycles_used = Cycles_used;} + Cycles_used ≤ S.balances[A.canister_id] dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ + .... State after:: .... S with - canisters[A.canister_id] = - { wasm_state = New_state; module = Mod; raw_module = A.wasm_module } + canisters[A.canister_id] = { + wasm_state = New_state; + module = Mod; + raw_module = A.wasm_module; + public_custom_sections = Public_custom_sections; + private_custom_sections = Private_custom_sections; + } + balances[A.canister_id] = S.balances[A.canister_id] - Cycles_used messages = Older_messages · Younger_messages · ResponseMessage { - origin = M.origin - response = Accepted (candid()) - refunded_cycles = M.transferred_cycles + origin = M.origin; + response = Reply (candid()); + refunded_cycles = M.transferred_cycles; } .... @@ -3182,32 +3215,39 @@ Conditions:: M.method_name = 'install_code' M.arg = candid(A) Mod = parse_wasm_mod(A.wasm_module) - + Public_custom_sections = parse_public_custom_sections(A.wasm_module) + Private_custom_sections = parse_private_custom_sections(A.wasm_module) A.mode = upgrade - S.canisters[A.canister_id] ≠ EmptyCanister M.caller ∈ S.controllers[A.canister_id] - S.canisters[A.canister_id] = { wasm_state = Old_state; module = Old_module } + S.canisters[A.canister_id] = { wasm_state = Old_state; module = Old_module, …} Env = { - time = S.time[M.receiver]; - balance = S.balances[M.receiver]; - freezing_limit = freezing_limit(S, M.receiver); + time = S.time[A.canister_id]; + balance = S.balances[A.canister_id]; + freezing_limit = freezing_limit(S, A.canister_id); certificate = NoCertificate; - status = simple_status(S.canister_status[M.receiver]); + status = simple_status(S.canister_status[A.canister_id]); } - Old_module.pre_upgrade(Old_State, M.caller, Env) = Return Stable_memory - Mod.post_upgrade(A.canister_id, Stable_memory, A.arg, M.caller, Env) = Return New_state + Old_module.pre_upgrade(Old_State, M.caller, Env) = Return {stable_memory = Stable_memory; cycles_used = Cycles_used;} + Mod.post_upgrade(A.canister_id, Stable_memory, A.arg, M.caller, Env) = Return {new_state = New_state; cycles_used = Cycles_used';} + Cycles_used + Cycles_used' ≤ S.balances[A.canister_id] dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ .... State after:: .... S with - canisters[A.canister_id] = - { wasm_state = New_state; module = Mod; raw_module = A.wasm_module } + canisters[A.canister_id] = { + wasm_state = New_state; + module = Mod; + raw_module = A.wasm_module; + public_custom_sections = Public_custom_sections; + private_custom_sections = Private_custom_sections; + } + balances[A.canister_id] = S.balances[A.canister_id] - (Cycles_used + Cycles_used'); messages = Older_messages · Younger_messages · ResponseMessage { - origin = M.origin - response = Accepted (candid()) - refunded_cycles = M.transferred_cycles + origin = M.origin; + response = Reply (candid()); + refunded_cycles = M.transferred_cycles; } .... @@ -3234,7 +3274,7 @@ S with messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin - response = Accepted (candid()) + response = Reply (candid()) refunded_cycles = M.transferred_cycles } · [ ResponseMessage { @@ -3305,17 +3345,17 @@ The status of a stopping canister which has no open call contexts is set to `Sto Conditions:: .... - S.canister_status[A.canister_id] = Stopping Origins - ∀ Ctxt_id. S.call_contexts[Ctxt_id].canister ≠ A.canister_id + S.canister_status[Canister_id] = Stopping Origins + ∀ Ctxt_id. S.call_contexts[Ctxt_id].canister ≠ Canister_id .... State after:: .... S with canister_status[CanisterId] = Stopped - messages = Messages · + messages = S.Messages · [ ResponseMessage { origin = O - response = Accepted (candid()) + response = Reply (candid()) refunded_cycles = C } | (O, C) ∈ Origins @@ -3337,12 +3377,12 @@ Conditions:: State after:: .... S with - messages = Older_messages · Younger_messages - S.messages = Messages · - ResponseMessage { - origin = M.origin - response = Accepted (candid()) - } + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin; + response = Reply (candid()); + refunded_cycles = M.transferred_cycles; + } .... ==== IC Management Canister: Starting a canister @@ -3366,7 +3406,7 @@ S with messages = Older_messages · Younger_messages · ResponseMessage{ origin = M.origin - response = Accepted (candid()) + response = Reply (candid()) refunded_cycles = M.transferred_cycles } .... @@ -3391,7 +3431,7 @@ S with messages = Older_messages · Younger_messages · ResponseMessage{ origin = M.origin - response = Accepted (candid()) + response = Reply (candid()) refunded_cycles = M.transferred_cycles } · [ ResponseMessage { @@ -3412,21 +3452,23 @@ Conditions:: M.callee = ic_principal M.method_name = 'delete_canister' M.arg = candid(A) - S.canister_status[A.canister_id] = stopped + S.canister_status[A.canister_id] = Stopped M.caller ∈ S.controllers[A.canister_id] .... State after:: .... S with - canisters[CanisterId] = (deleted) - controllers[CanisterId] = (deleted) - canister_status[CanisterId] = (deleted) - time[CanisterId] = (deleted) - balances[CanisterId] = (deleted) + canisters[A.canister_id] = (deleted) + controllers[A.canister_id] = (deleted) + freezing_threshold[A.canister_id] = (deleted) + canister_status[A.canister_id] = (deleted) + time[A.canister_id] = (deleted) + balances[A.canister_id] = (deleted) + certified_data[A.canister_id] = (deleted) messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin - response = Accepted (candid()) + response = Reply (candid()) refunded_cycles = M.transferred_cycles } .... @@ -3440,17 +3482,17 @@ Conditions:: M.callee = ic_principal M.method_name = 'deposit_cycles' M.arg = candid(A) - Cycle_cost ≤ S.balances[A.canister_id] + M.transferred_cycles + A.canister_id ∈ dom(S.balances) .... State after:: .... S with - balances[CanisterId] = + balances[A.canister_id] = S.balances[A.canister_id] + M.transferred_cycles messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin - response = Accepted (candid()) + response = Reply (candid()) refunded_cycles = 0 } .... @@ -3476,7 +3518,7 @@ S with messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin - response = Accepted (candid(B)) + response = Reply (candid(B)) refunded_cycles = M.transferred_cycles } .... @@ -3493,7 +3535,7 @@ Conditions:: M.method_name = 'provisional_create_canister_with_cycles' M.arg = candid(A) is_system_assigned CanisterId - CanisterId ∉ dom S.canisters + CanisterId ∉ dom(S.canisters) .... State after:: .... @@ -3501,12 +3543,13 @@ S with canisters[CanisterId] = EmptyCanister time[CanisterId] = CurrentTime controllers[CanisterId] = [M.caller] + freezing_threshold[CanisterId] = 2592000 balances[CanisterId] = A.amount certified_data[CanisterId] = "" messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin - response = Accepted (candid({canister_id = CanisterId})) + response = Reply (candid({canister_id = CanisterId})) transferred_cycles = M.transferred_cycles } canister_status[CanisterId] = Running @@ -3521,12 +3564,12 @@ Conditions:: M.callee = ic_principal M.method_name = 'provisional_top_up_canister' M.arg = candid(A) - A.canister_id ∈ dom S.canisters + A.canister_id ∈ dom(S.canisters) .... State after:: .... S with - balances[CanisterId] = balances[CanisterId] + A.amount + balances[A.canister_id] = S.balances[A.canister_id] + A.amount .... ==== Callback invocation @@ -3543,18 +3586,19 @@ Conditions:: callback = Callback } not S.call_contexts[Ctxt_id].deleted + S.call_contexts[Ctxt_id].canister ∈ dom(S.balances) .... State after:: .... S with balances[S.call_contexts[Ctxt_id].canister] = - balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles + S.balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles messages = Older_messages · FuncMessage { call_context = Ctxt_id - receiver = C - entry_point = Callback Callback FM.response RM.refunded_cycles + receiver = S.call_contexts[Ctxt_id].canister + entry_point = Callback Callback RM.response RM.refunded_cycles queue = Unordered } · Younger_messages @@ -3571,12 +3615,13 @@ Conditions:: callback = Callback } S.call_contexts[Ctxt_id].deleted + S.call_contexts[Ctxt_id].canister ∈ dom(S.balances) .... State after:: .... S with balances[S.call_contexts[Ctxt_id].canister] = - balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles + MAX_CYCLES_PER_RESPONSE + S.balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles + MAX_CYCLES_PER_RESPONSE messages = Older_messages · Younger_messages .... @@ -3586,17 +3631,17 @@ When an ingress method call has been responded to, we can record the response in Conditions:: .... - S.requests[M] = Processing S.messages = Older_messages · ResponseMessage RM · Younger_messages RM.origin = FromUser { request = M } + S.requests[M] = Processing .... State after:: .... S with messages = Older_messages · Younger_messages requests[M] = - | Replied R if response = Reply R - | Rejected R if response = Reject R + | Replied R if M.response = Reply R + | Rejected (c, R) if M.response = Reject (c, R) .... NB: The refunded cycles, `RM.refunded_cycles` are, by construction, empty. @@ -3641,21 +3686,21 @@ State after:: .... S with canisters[CanisterId] = EmptyCanister - certified_data[A.canister_id] = "" + certified_data[Canister_id] = "" - messages = Older_messages · Younger_messages · + messages = S.messages · [ ResponseMessage { origin = Ctxt.origin response = Reject (CANISTER_REJECT, 'Canister has been uninstalled') refunded_cycles = Ctxt.available_cycles } | Ctxt_id ↦ Ctxt ∈ S.call_contexts - , Ctxt.canister = A.canister_id + , Ctxt.canister = CanisterId , Ctxt.needs_to_respond = true ] for Ctxt_id ↦ Ctxt ∈ S.call_contexts: - if Ctxt.canister = A.canister_id: + if Ctxt.canister = CanisterId: call_contexts[Ctxt_id].deleted := true call_contexts[Ctxt_id].needs_to_respond := false call_contexts[Ctxt_id].available_cycles := 0 @@ -3732,17 +3777,17 @@ Conditions:: } .... Read response:: -* If `F(Q.Arg, Q.sender, Env) = Trap` then +* If `F(Q.Arg, Q.sender, Env) = Trap trap` then + .... {status: rejected; reject_code: CANISTER_ERROR, reject_message: , error_code: } .... -* Else if `F(Q.Arg, Q.sender, Env) = Return (Reject (code, msg))` then +* Else if `F(Q.Arg, Q.sender, Env) = Return {response = Reject (code, msg); …}` then + .... {status: rejected; reject_code: : reject_message: , error_code: } .... -* Else if `F(Q.Arg, Q.sender, Env) = Return (Reply R)` then +* Else if `F(Q.Arg, Q.sender, Env) = Return {response = Reply R; …}` then + .... {status: success; reply: { arg : } } @@ -3775,9 +3820,9 @@ may_read_path(S, _, ["request_status", Rid] · _) = may_read_path(S, _, ["canister", cid, "module_hash"]) = cid == ECID may_read_path(S, _, ["canister", cid, "controllers"]) = cid == ECID may_read_path(S, _, ["canister", cid, "metadata", name]) = - if name ∈ S.canisters[cid].public_custom_sections + if name ∈ dom(S.canisters[cid].public_custom_sections) then cid == ECID - else if name ∈ S.canisters[cid].private_custom_sections + else if name ∈ dom(S.canisters[cid].private_custom_sections) then cid == ECID ∧ RS.sender ∈ S.controllers[cid] else False may_read_path(S, _, _) = False @@ -3867,6 +3912,7 @@ ExecutionState = { response : NoResponse | Response; cycles_accepted : Nat; cycles_available : Nat; + cycles_used : Nat; balance : Funds; reply_params : { arg : Blob }; pending_call : MethodCall | NoPendingCall; @@ -3905,6 +3951,7 @@ empty_execution_state = { response = NoResponse; cycles_accepted = 0; cycles_available = 0; + cycles_used = 0; balance = 0; reply_params = { arg = "" }; pending_call = NoPendingCall; @@ -3921,7 +3968,7 @@ If the WebAssembly module does not export a function called under the name `cani + .... init = λ (self_id, arg, caller, sysenv) → - Return { store = initial_wasm_store; self_id = self_id; stable_mem = "" } + Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" }; cycles_used = 0;} .... + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_init`, it is @@ -3933,11 +3980,11 @@ init = λ (self_id, arg, caller, sysenv) → params = empty_params with { arg = arg; caller = caller; sysenv } balance = sysenv.balance } - try func() with Trap then Trap - if es.performed_calls ≠ [] then Trap - if es.response ≠ NoResponse then Trap - if es.ingress_filter ≠ Reject then Trap - Return es.wasm_state + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} + Return {new_state = es.wasm_state; cycles_used = es.cycles_used;} .... + This formulation checks afterwards that the system calls `call.perform` or `msg.reply` were not invoked; an implementation can of course trap as soon as these system calls are invoked. @@ -3947,7 +3994,7 @@ This formulation checks afterwards that the system calls `call.perform` or `msg. If the WebAssembly module does not export a function called under the name `canister_pre_upgrade`, then it simply returns the stable memory: + .... -pre_upgrade = λ (old_state, caller, sysenv) → Return old_state.stable_mem +pre_upgrade = λ (old_state, caller, sysenv) → Return {stable_memory = old_state.stable_mem; cycles_used = 0;} .... + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_pre_upgrade`, it is @@ -3959,11 +4006,11 @@ pre_upgrade = λ (old_state, caller, sysenv) → params = { empty_params with caller = caller; sysenv } balance = sysenv.balance } - try func() with Trap then Trap - if es.performed_calls ≠ [] then Trap - if es.response ≠ NoResponse then Trap - if es.ingress_filter ≠ Reject then Trap - Return es.wasm_state.stable_mem + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} + Return {stable_memory = es.wasm_state.stable_mem; cycles_used = es.cycles_used;} .... @@ -3973,7 +4020,7 @@ If the WebAssembly module does not export a function called under the name `cani + .... post_upgrade = λ (self_id, stable_mem, arg, caller, sysenv) → - Return { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem } + Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem }; cycles_used = 0;} .... + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_post_upgrade`, it is @@ -3985,11 +4032,11 @@ post_upgrade = λ (self_id, stable_mem, arg, caller, sysenv) → params = { empty_params with arg = arg; caller = caller; sysenv } balance = sysenv.balance } - try func() with Trap then Trap - if es.performed_calls ≠ [] then Trap - if es.response ≠ NoResponse then Trap - if es.ingress_filter ≠ Reject then Trap - Return es.wasm_state + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} + Return {new_state = es.wasm_state; cycles_used = es.cycles_used;} .... * The partial map `update_methods` of the `CanisterModule` is defined for all method names `method` for which the WebAssembly program exports a function `func` named `canister_update `, and has value @@ -4002,14 +4049,15 @@ update_methods[method] = λ (arg, caller, sysenv, available) → λ wasm_state balance = sysenv.balance cycles_available = available; } - try func() with Trap then Trap - if es.ingress_filter ≠ Reject then Trap + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} Return { new_state = es.wasm_state; new_calls = es.calls; response = es.response; cycles_accepted = es.cycles_accepted; - new_certified_data = es.new_certified_data + cycles_used = es.cycles_used; + new_certified_data = es.new_certified_data; } .... @@ -4022,12 +4070,15 @@ query_methods[method] = λ (arg, caller, sysenv) → λ wasm_state → params = empty_params with { arg = arg; caller = caller; sysenv } balance = sysenv.balance } - try func() with Trap then Trap - if es.cycles_accepted ≠ 0 then Trap - if es.calls ≠ () then Trap - if es.ingress_filter ≠ Reject then Trap - if es.response = NoResponse then Trap - Return es.response; + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used;} + if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} + if es.response = NoResponse then Trap {cycles_used = es.cycles_used;} + Return { + response = es.response; + cycles_used = es.cycles_used; + } .... + This formulation checks afterwards that the system call `ic0.call_perform` was not invoked; an implementation can of course trap already when these system calls have been invoked. @@ -4043,15 +4094,18 @@ heartbeat = λ (sysenv) → λ wasm_state → params = empty_params with { arg = NoArg; caller = NoCaller; sysenv } balance = sysenv.balance } - try func() with Trap then Trap - if es.cycles_accepted ≠ 0 then Trap - if es.ingress_filter ≠ Reject then Trap - if es.response ≠ NoResponse then Trap - Return es.wasm_state; + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + cycles_used = es.cycles_used; + } .... otherwise it is .... -heartbeat = λ (sysenv) → λ wasm_state → Trap +heartbeat = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} .... * The function `callbacks` of the `CanisterModule` is defined as follows @@ -4069,40 +4123,46 @@ callbacks = λ(callbacks, response, refunded_cycles, sysenv, available) → λ w Reject (reject_code, reject_message)-> (callbacks.on_reject.fun, callbacks.on_reject.env, { params0 with reject_code; reject_message}) + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = params; + balance = sysenv.balance; + cycles_available = available; + } try if fun > |es.wasm_state.store.table| then Trap let func = es.wasm_state.store.table[fun] if typeof(func) ≠ func (i32) -> () then Trap - let es = ref {empty_execution_state with - wasm_state = wasm_state; - params = params; - balance = sysenv.balance; - cycles_available = available; - } func(env) Return { new_state = es.wasm_state; new_calls = es.calls; response = es.response; cycles_accepted = es.cycles_accepted; + cycles_used = es.cycles_used; new_certified_data = es.certified_data; } with Trap - if callbacks.on_cleanup = NoClosure then Trap - if callbacks.on_cleanup.fun > |es.wasm_state.store.table| then Trap + if callbacks.on_cleanup = NoClosure then Trap {cycles_used = es.cycles_used;} + if callbacks.on_cleanup.fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} let func = es.wasm_state.store.table[callbacks.on_cleanup.fun] - if typeof(func) ≠ func (i32) -> () then Trap + if typeof(func) ≠ func (i32) -> () then Trap {cycles_used = es.cycles_used;} - let es = ref { empty_execution_state with - wasm_state; + let es' = ref { empty_execution_state with + wasm_state = wasm_state; } - func(callbacks.on_cleanup.env) + try func(callbacks.on_cleanup.env) with Trap then Trap {cycles_used = es.cycles_used + es'.cycles_used;} + if es'.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used + es'.cycles_used;} + if es'.calls ≠ [] then Trap {cycles_used = es.cycles_used + es'.cycles_used;} + if es'.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used + es'.cycles_used;} + if es'.response ≠ NoResponse then Trap {cycles_used = es.cycles_used + es'.cycles_used;} Return { - new_state = es.wasm_state; + new_state = es'.wasm_state; new_calls = []; response = NoResponse; cycles_accepted = 0; + cycles_used = es.cycles_used + es'.cycles_used; } .... + @@ -4115,7 +4175,7 @@ If the WebAssembly module does not export a function called under the name `cani + .... inspect_message = λ (method_name, wasm_state, arg, caller, sysenv) → - Return Accept + Return {status = Accept; cycles_used = 0;} .... Otherwise, if the WebAssembly module exports a function `func` under the name `canister_inspect_message`, it is + @@ -4132,10 +4192,10 @@ inspect_message = λ (method_name, wasm_state, arg, caller, sysenv) → balance = sysenv.balance; cycles_available = 0; // ingress requests have no funds } - try func() with Trap then Trap - if es.calls ≠ () then Trap - if es.response ≠ NoResponse then Trap - Return es.ingress_filter; + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + Return {status = es.ingress_filter; cycles_used = es.cycles_used;}; .... ==== Helper functions @@ -4144,12 +4204,12 @@ In the following section, we use the these helper functions .... copy_to_canister(dst : i32, offset : i32, size : i32, data : blob) = - if offset+size > |data| then Trap - if dst+size > |es.wasm_state.store.mem| then Trap + if offset+size > |data| then Trap {cycles_used = es.cycles_used;} + if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} es.wasm_state.store.mem[dst..dst+size] := data[offset..offset+size] copy_from_canister(src : i32, size : i32) blob = - if src+size > |es.wasm_state.store.mem| then Trap + if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} return es.wasm_state.store.mem[src..src+size] .... @@ -4158,7 +4218,7 @@ Cycles are represented by 128-bit values so they require 16 bytes of memory. .... copy_cycles_to_canister(dst : i32, data : blob) = let size = 16; - if dst+size > |es.wasm_state.store.mem| then Trap + if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} es.wasm_state.store.mem[dst..dst+size] := data[0..size] .... @@ -4192,21 +4252,21 @@ ic0.msg_reject_msg_copy(dst:i32, offset:i32, size:i32) : i32 = copy_to_canister(dst, offset, size, es.params.reject_msg) ic0.msg_reply_data_append(src : i32, size : i32) = - if es.response ≠ NoResponse then Trap + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.reply_params.arg := es.reply_params.arg · copy_from_canister(src, size) ic0.msg_reply() = - if es.response ≠ NoResponse then Trap + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.response := Reply (es.reply_params.arg) es.cycles_available := 0 ic0.msg_reject(src : i32, size : i32) = - if es.response ≠ NoResponse then Trap + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.response := Reject (CANISTER_REJECT, copy_from_canister(src, size)) es.cycles_available := 0 ic0.msg_cycles_available() : i64 = - if es.cycles_available >= 2^64^ then Trap + if es.cycles_available >= 2^64^ then Trap {cycles_used = es.cycles_used;} return es.cycles_available ic0.msg_cycles_available128(dst : i32) = @@ -4214,7 +4274,7 @@ ic0.msg_cycles_available128(dst : i32) = copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) ic0.msg_cycles_refunded() : i64 = - if es.params.cycles_refunded >= 2^64^ then Trap + if es.params.cycles_refunded >= 2^64^ then Trap {cycles_used = es.cycles_used;} return es.params.cycles_refunded ic0.msg_cycles_refunded128(dst : i32) = @@ -4222,7 +4282,7 @@ ic0.msg_cycles_refunded128(dst : i32) = copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) ic0.accept_message() = - if es.ingress_filter = Accept then Trap + if es.ingress_filter = Accept then Trap {cycles_used = es.cycles_used;} es.ingress_filter = Accept ic0.msg_method_name_size() : i32 = @@ -4253,7 +4313,7 @@ ic0.canister_self_copy(dst:i32, offset:i32, size:i32) = copy_to_canister(dst, offset, size, es.wasm_state.self_id) ic0.canister_cycle_balance() : i64 = - if es.balance >= 2^64^ then Trap + if es.balance >= 2^64^ then Trap {cycles_used = es.cycles_used;} return es.balance ic0.canister_cycles_balance128(dst : i32) = @@ -4278,17 +4338,17 @@ ic0.call_new( ) = discard_pending_call() - if es.balance < MAX_CYCLES_PER_RESPONSE then Trap + if es.balance < MAX_CYCLES_PER_RESPONSE then Trap {cycles_used = es.cycles_used;} es.balance := es.balance - MAX_CYCLES_PER_RESPONSE callee := copy_from_canister(callee_src, callee_size); method_name := copy_from_canister(name_src, name_size); - if reply_fun > |es.wasm_state.store.table| then Trap - if typeof(es.wasm_state.store.table[reply_fun]) ≠ func (anyref, i32) -> () then Trap + if reply_fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} + if typeof(es.wasm_state.store.table[reply_fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} - if reject_fun > |es.wasm_state.store.table| then Trap - if typeof(es.wasm_state.store.table[reject_fun]) ≠ func (anyref, i32) -> () then Trap + if reject_fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} + if typeof(es.wasm_state.store.table[reject_fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} es.pending_call = MethodCall { callee = callee; @@ -4303,33 +4363,33 @@ ic0.call_new( } ic0.call_data_append (src : i32, size : i32) = - if es.pending_call = NoPendingCall then Trap + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} es.pending_call.arg := es.pending_call.arg · copy_from_canister(src, size) ic0.call_on_cleanup (fun : i32, env : i32) = - if fun > |es.wasm_state.store.table| then Trap - if typeof(es.wasm_state.store.table[fun]) ≠ func (anyref, i32) -> () then Trap - if es.pending_call = NoPendingCall then Trap - if es.pending_call.callback.on_cleanup ≠ NoClosure then Trap + if fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} + if typeof(es.wasm_state.store.table[fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + if es.pending_call.callback.on_cleanup ≠ NoClosure then Trap {cycles_used = es.cycles_used;} es.pending_call.callback.on_cleanup := Closure { fun = fun; env = env} ic0.call_cycles_add(amount : i64) = - if es.pending_call = NoPendingCall then Trap - if es.balance < amount then Trap + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + if es.balance < amount then Trap {cycles_used = es.cycles_used;} es.balance := es.balance - amount es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount ic0.call_cycles_add128(amount_high : i64, amount_low : i64) = let amount = amount_high * 2^64^ + amount_low - if es.pending_call = NoPendingCall then Trap - if es.balance < amount then Trap + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + if es.balance < amount then Trap {cycles_used = es.cycles_used;} es.balance := es.balance - amount es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount ic0.call_peform() : ( err_code : i32 ) = - if es.pending_call = NoPendingCall then Trap + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} // are we below the threezing threshold? // Or maybe the system has other reasons to not perform this @@ -4349,12 +4409,12 @@ discard_pending_call() = es.pending_call := NoPendingCall ic0.stable_size() : (page_count : i32) = - if |es.wasm_state.store.mem| > 2^32^ then Trap + if |es.wasm_state.store.mem| > 2^32^ then Trap {cycles_used = es.cycles_used;} page_count := |es.wasm_state.stable_mem| / 64k return page_count ic0.stable_grow(new_pages : i32) : (old_page_count : i32) = - if |es.wasm_state.store.mem| > 2^32^ then Trap + if |es.wasm_state.store.mem| > 2^32^ then Trap {cycles_used = es.cycles_used;} if arbitrary() then return -1 else old_size := |es.wasm_state.stable_mem| / 64k @@ -4364,16 +4424,16 @@ ic0.stable_grow(new_pages : i32) : (old_page_count : i32) = return old_size ic0.stable_write(offset : i32, src : i32, size : i32) - if |es.wasm_state.store.mem| > 2^32^ then Trap - if src+size > |es.wasm_state.store.mem| then Trap - if offset+size > |es.wasm_state.stable_mem| then Trap + if |es.wasm_state.store.mem| > 2^32^ then Trap {cycles_used = es.cycles_used;} + if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] ic0.stable_read(dst : i32, offset : i32, size : i32) - if |es.wasm_state.store.mem| > 2^32^ then Trap - if offset+size > |es.wasm_state.stable_mem| then Trap - if dst+size > |es.wasm_state.store.mem| then Trap + if |es.wasm_state.store.mem| > 2^32^ then Trap {cycles_used = es.cycles_used;} + if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} + if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] @@ -4390,14 +4450,14 @@ ic0.stable64_grow(new_pages : i64) : (old_page_count : i64) = return old_size ic0.stable64_write(offset : i64, src : i64, size : i64) - if src+size > |es.wasm_state.store.mem| then Trap - if offset+size > |es.wasm_state.stable_mem| then Trap + if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] ic0.stable64_read(dst : i64, offset : i64, size : i64) - if offset+size > |es.wasm_state.stable_mem| then Trap - if dst+size > |es.wasm_state.store.mem| then Trap + if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} + if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] @@ -4413,11 +4473,11 @@ ic0.data_certificate_present() : i32 = else return 1 ic0.data_certificate_size() : i32 = - if es.params.sysenv.certificate = NoCertificate then Trap + if es.params.sysenv.certificate = NoCertificate then Trap {cycles_used = es.cycles_used;} return |es.params.sysenv.certificate| ic0.data_certificate_copy(dst: i32, offset: i32, size: i32) = - if es.params.sysenv.certificate = NoCertificate then Trap + if es.params.sysenv.certificate = NoCertificate then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.sysenv.certificate) ic0.performance_counter(counter_type : i32) : i64 = @@ -4427,7 +4487,7 @@ ic0.debug_print(src : i32, size : i32) = return ic0.trap(src : i32, size : i32) = - Trap + Trap {cycles_used = es.cycles_used;} .... diff --git a/theories/IC.thy b/theories/IC.thy index 8aaac612b..7eccffa1d 100644 --- a/theories/IC.thy +++ b/theories/IC.thy @@ -2,6 +2,11 @@ theory IC imports "HOL-Library.AList" begin +(* General helper lemmas *) + +lemma in_set_updD: "x \ set (xs[n := z]) \ x \ set xs \ x = z" + by (meson insert_iff set_update_subset_insert subsetD) + (* Partial maps *) typedef ('a, 'b) list_map = "{f :: ('a \ 'b) list. distinct (map fst f)}" @@ -9,32 +14,113 @@ typedef ('a, 'b) list_map = "{f :: ('a \ 'b) list. distinct (map fst f)}" setup_lifting type_definition_list_map +lift_bnf (dead 'k, set: 'v) list_map [wits: "[] :: ('k \ 'v) list"] for map: map rel: rel + by auto + +hide_const (open) map set rel + lift_definition list_map_dom :: "('a, 'b) list_map \ 'a set" is "set \ map fst" . -lift_definition list_map_vals :: "('a, 'b) list_map \ 'b set" is +lift_definition list_map_vals :: "('a, 'b) list_map \ 'b list" is + "map snd" . + +lift_definition list_map_range :: "('a, 'b) list_map \ 'b set" is "set \ map snd" . +lemma in_set_map_filter_vals: "z \ set (List.map_filter g (list_map_vals f)) \ \w \ list_map_range f. g w = Some z" + by transfer (force simp: List.map_filter_def) + lift_definition list_map_sum_vals :: "('b \ nat) \ ('a, 'b) list_map \ nat" is "\g. sum_list \ (map (g \ snd))" . lift_definition list_map_get :: "('a, 'b) list_map \ 'a \ 'b option" is "map_of" . +lemma list_map_get_dom[dest]: "x \ list_map_dom f \ list_map_get f x = None \ False" + by transfer auto + +lemma list_map_get_range: "list_map_get f x = Some y \ y \ list_map_range f" + by transfer force + lift_definition list_map_set :: "('a, 'b) list_map \ 'a \ 'b \ ('a, 'b) list_map" is "\f x y. AList.update x y f" by (rule distinct_update) +lemma list_map_get_set: "list_map_get (list_map_set f x y) z = (if x = z then Some y else list_map_get f z)" + by transfer (auto simp add: update_Some_unfold update_conv) + +lemma list_map_dom_set[simp]: "list_map_dom (list_map_set f x y) = list_map_dom f \ {x}" + by transfer (auto simp add: dom_update) + +lemma list_map_range_setD: "z \ list_map_range (list_map_set f x y) \ z \ list_map_range f \ z = y" + apply transfer + apply auto + apply (metis distinct_update image_iff map_of_eq_Some_iff snd_eqD update_Some_unfold) + done + lift_definition list_map_del :: "('a, 'b) list_map \ 'a \ ('a, 'b) list_map" is "\f x. AList.delete x f" by (rule distinct_delete) +lemma list_map_get_del: "list_map_get (list_map_del f x) z = (if x = z then None else list_map_get f z)" + by transfer (auto simp add: delete_conv') + +lemma list_map_dom_del[simp]: "list_map_dom (list_map_del f x) = list_map_dom f - {x}" + by transfer (simp add: delete_keys) + +lemma list_map_range_del: "z \ list_map_range (list_map_del f x) \ z \ list_map_range f" + apply transfer + apply auto + apply (metis Some_eq_map_of_iff delete_notin_dom distinct_delete fst_eqD imageI map_of_delete snd_eqD) + done + lift_definition list_map_empty :: "('a, 'b) list_map" is "[]" by auto +lemma list_map_get_empty[simp]: "list_map_get list_map_empty x = None" + by transfer auto + lemma list_map_empty_dom[simp]: "list_map_dom list_map_empty = {}" by transfer auto +lemma list_map_empty_range[simp]: "list_map_range list_map_empty = {}" + by transfer auto + +lift_definition list_map_init :: "('a \ 'b) list \ ('a, 'b) list_map" is + "\xys. AList.updates (map fst xys) (map snd xys) []" + using distinct_updates + by force + +lift_definition list_map_map :: "('b \ 'c) \ ('a, 'b) list_map \ ('a, 'c) list_map" is + "\f xs. map (\(k, v). (k, f v)) xs" + by (auto simp: comp_def case_prod_beta) + +lemma list_map_dom_map_map[simp]: "list_map_dom (list_map_map g f) = list_map_dom f" + by transfer force + +lemma list_map_range_map_map[simp]: "list_map_range (list_map_map g f) = g ` list_map_range f" + by transfer force + +lemma list_map_sum_vals_split: "(\ctxt. ctxt \ list_map_range xs \ f (g ctxt) \ f ctxt) \ list_map_sum_vals f xs = + list_map_sum_vals id + (list_map_map (\ctxt. if P ctxt then f ctxt - f (g ctxt) else 0) xs) + + list_map_sum_vals f + (list_map_map (\ctxt. if P ctxt then g ctxt else ctxt) xs)" + apply (transfer fixing: f g P) + subgoal for xs + by (induction xs) auto + done + +lemma list_map_sum_vals_filter: + assumes "\b. b \ list_map_range xs \ P b = None \ f b = 0" "\b y. b \ list_map_range xs \ P b = Some y \ f b = g y" + shows "list_map_sum_vals id (list_map_map f xs) = sum_list (map g (List.map_filter P (list_map_vals xs)))" + using assms + apply (transfer fixing: f g P) + subgoal for xs + by (induction xs) (auto simp: List.map_filter_def) + done + lemma list_map_sum_in_ge_aux: fixes g :: "'a \ nat" shows "distinct (map fst f) \ map_of f x = Some y \ g y \ sum_list (map g (map snd f))" @@ -83,12 +169,15 @@ lemma list_map_del_sum: "list_map_get f x = Some y \ list_map_su (* Abstract canisters *) +type_synonym 's method_name = 's + type_synonym 'b arg = 'b +type_synonym 'p caller_id = 'p type_synonym timestamp = nat datatype status = Running | Stopping | Stopped record ('b) env = - env_time :: timestamp + time :: timestamp balance :: nat freezing_limit :: nat certificate :: "'b option" @@ -100,52 +189,65 @@ datatype ('b, 's) response = | Reject reject_code 's record ('p, 'canid, 's, 'b, 'c) method_call = callee :: 'canid - method_name :: 's + method_name :: "'s method_name" arg :: 'b transferred_cycles :: nat callback :: 'c +record 'x cycles_return = + return :: 'x + cycles_used :: nat +type_synonym trap_return = "unit cycles_return" record ('w, 'p, 'canid, 's, 'b, 'c) update_return = new_state :: 'w new_calls :: "('p, 'canid, 's, 'b, 'c) method_call list" new_certified_data :: "'b option" response :: "('b, 's) response option" cycles_accepted :: nat -type_synonym ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func = "'w \ 'tr + ('w, 'p, 'canid, 's, 'b, 'c) update_return" -type_synonym ('w, 'b, 's, 'tr) query_func = "'w \ 'tr + ('b, 's) response" + cycles_used :: nat +record ('b, 's) query_return = + response :: "('b, 's) response" + cycles_used :: nat +record 'w heartbeat_return = + new_state :: 'w + cycles_used :: nat +type_synonym ('w, 'p, 'canid, 's, 'b, 'c) update_func = "'w \ trap_return + ('w, 'p, 'canid, 's, 'b, 'c) update_return" +type_synonym ('w, 'b, 's) query_func = "'w \ trap_return + ('b, 's) query_return" +type_synonym 'w heartbeat_func = "'w \ trap_return + 'w heartbeat_return" type_synonym available_cycles = nat type_synonym refunded_cycles = nat datatype inspect_method_result = Accept | Reject -record ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module_rec = - init :: "'canid \ 'b arg \ 'p \ 'b env \ 'tr + 'w" - pre_upgrade :: "'w \ 'p \ 'b env \ 'tr + 'sm" - post_upgrade :: "'canid \ 'sm \ 'b arg \ 'p \ 'b env \ 'tr + 'w" - update_methods :: "('s, ('b arg \ 'p \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func) list_map" - query_methods :: "('s, ('b arg \ 'p \ 'b env) \ ('w, 'b, 's, 'tr) query_func) list_map" - heartbeat :: "'b env \ 'w \ 'tr + 'w" - callbacks :: "('c \ ('b, 's) response \ refunded_cycles \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func" - inspect_message :: "('s \ 'w \ 'b arg \ 'p \ 'b env) \ 'tr + inspect_method_result" -typedef ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module = - "{m :: ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module_rec. list_map_dom (update_methods m) \ list_map_dom (query_methods m) = {}}" +record ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module_rec = + init :: "'canid \ 'b arg \ 'p caller_id \ 'b env \ trap_return + 'w cycles_return" + pre_upgrade :: "'w \ 'p \ 'b env \ trap_return + 'sm cycles_return" + post_upgrade :: "'canid \ 'sm \ 'b arg \ 'p caller_id \ 'b env \ trap_return + 'w cycles_return" + update_methods :: "('s method_name, ('b arg \ 'p caller_id \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func) list_map" + query_methods :: "('s method_name, ('b arg \ 'p caller_id \ 'b env) \ ('w, 'b, 's) query_func) list_map" + heartbeat :: "'b env \ 'w heartbeat_func" + callbacks :: "('c \ ('b, 's) response \ refunded_cycles \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" + inspect_message :: "('s method_name \ 'w \ 'b arg \ 'p caller_id \ 'b env) \ trap_return + inspect_method_result cycles_return" +typedef ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module = + "{m :: ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module_rec. list_map_dom (update_methods m) \ list_map_dom (query_methods m) = {}}" by (auto intro: exI[of _ "\init = undefined, pre_upgrade = undefined, post_upgrade = undefined, update_methods = list_map_empty, query_methods = list_map_empty, heartbeat = undefined, callbacks = undefined, inspect_message = undefined\"]) setup_lifting type_definition_canister_module -lift_definition dispatch_method :: "'s \ ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module \ - ((('b arg \ 'p \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func) + - (('b arg \ 'p \ 'b env) \ ('w, 'b, 's, 'tr) query_func)) option" is - "\f m. case list_map_get (update_methods m) f of Some f' \ None | None \ (case list_map_get (query_methods m) f of Some f' \ None | None \ None)" . - -lift_definition canister_module_callbacks :: "('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module \ - ('c \ ('b, 's) response \ refunded_cycles \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func" is - callbacks . +lift_definition canister_module_init :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'canid \ 'b arg \ 'p \ 'b env \ trap_return + 'w cycles_return" is "init" . +lift_definition canister_module_pre_upgrade :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'w \ 'p \ 'b env \ trap_return + 'sm cycles_return" is pre_upgrade . +lift_definition canister_module_post_upgrade :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'canid \ 'sm \ 'b arg \ 'p \ 'b env \ trap_return + 'w cycles_return" is post_upgrade . +lift_definition canister_module_update_methods :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('s, ('b arg \ 'p \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func) list_map" is update_methods . +lift_definition canister_module_query_methods :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('s, ('b arg \ 'p \ 'b env) \ ('w, 'b, 's) query_func) list_map" is query_methods . +lift_definition canister_module_heartbeat :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'b env \ 'w heartbeat_func" is heartbeat . +lift_definition canister_module_callbacks :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('c \ ('b, 's) response \ refunded_cycles \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" is callbacks . +lift_definition canister_module_inspect_message :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('s \ 'w \ 'b arg \ 'p \ 'b env) \ trap_return + inspect_method_result cycles_return" is inspect_message . -lift_definition canister_module_heartbeat :: "('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module \ 'b env \ 'w \ 'tr + 'w" is - heartbeat . +lift_definition dispatch_method :: "'s \ ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ + ((('b arg \ 'p \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func) + (('b arg \ 'p \ 'b env) \ ('w, 'b, 's) query_func)) option" is + "\f m. case list_map_get (update_methods m) f of Some f' \ Some (Inl f') | None \ (case list_map_get (query_methods m) f of Some f' \ Some (Inr f') | None \ None)" . (* Call contexts *) @@ -173,28 +275,28 @@ typedef ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt = "{ctxt :: ('p, 'uid, 'c setup_lifting type_definition_call_ctxt -lift_definition call_ctxt_origin :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin" is - "\ctxt. origin ctxt" . +lift_definition call_ctxt_canister :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ 'canid" is "canister" . +lift_definition call_ctxt_origin :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin" is "origin" . +lift_definition call_ctxt_needs_to_respond :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ bool" is needs_to_respond . +lift_definition call_ctxt_deleted :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ bool" is deleted . +lift_definition call_ctxt_available_cycles :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ nat" is available_cycles . -lift_definition call_ctxt_needs_to_respond :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ bool" is - "\ctxt. needs_to_respond ctxt" . - -lift_definition call_ctxt_available_cycles :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ nat" is - "\ctxt. available_cycles ctxt" . - -lemma call_ctxt_inv: "\call_ctxt_needs_to_respond x2 \ call_ctxt_available_cycles x2 = 0" +lemma call_ctxt_not_needs_to_respond_available_cycles: "\call_ctxt_needs_to_respond x2 \ call_ctxt_available_cycles x2 = 0" by transfer auto lift_definition call_ctxt_respond :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt" is - "\ctxt. ctxt\available_cycles := 0, needs_to_respond := False\" + "\ctxt. ctxt\needs_to_respond := False, available_cycles := 0\" by auto -lemma call_ctxt_respond_available_cycles[simp]: "call_ctxt_available_cycles (call_ctxt_respond ctxt) = 0" +lemma call_ctxt_respond_origin[simp]: "call_ctxt_origin (call_ctxt_respond ctxt) = call_ctxt_origin ctxt" by transfer auto lemma call_ctxt_respond_needs_to_respond[dest]: "call_ctxt_needs_to_respond (call_ctxt_respond ctxt) \ False" by transfer auto +lemma call_ctxt_respond_available_cycles[simp]: "call_ctxt_available_cycles (call_ctxt_respond ctxt) = 0" + by transfer auto + lift_definition call_ctxt_deduct_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt" is "\n ctxt. ctxt\available_cycles := available_cycles ctxt - n\" by auto @@ -208,12 +310,19 @@ lemma call_ctxt_deduct_cycles_needs_to_respond[simp]: "call_ctxt_needs_to_respon lemma call_ctxt_deduct_cycles_available_cycles[simp]: "call_ctxt_available_cycles (call_ctxt_deduct_cycles n ctxt) = call_ctxt_available_cycles ctxt - n" by transfer auto +lift_definition call_ctxt_delete :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt" is + "\ctxt. ctxt\deleted := True, needs_to_respond := False, available_cycles := 0\" + by auto + +lemma call_ctxt_delete_needs_to_respond[simp]: "call_ctxt_needs_to_respond (call_ctxt_delete ctxt) = False" + by transfer auto + (* Calls and Messages *) datatype 'canid queue_origin = System | Canister 'canid datatype 'canid queue = Unordered | Queue "'canid queue_origin" 'canid datatype ('s, 'p, 'b, 'c) entry_point = - Public_method "'s" "'p" "'b" + Public_method "'s method_name" "'p" "'b" | Callback "'c" "('b, 's) response" "refunded_cycles" | Heartbeat @@ -238,27 +347,37 @@ record ('b, 'uid, 'canid, 's) CanisterQuery = method_name :: 's arg :: 'b type_synonym ('b, 'uid, 'canid, 's) APIReadRequest = "('b, 'uid) StateRead + ('b, 'uid, 'canid, 's) CanisterQuery" -record ('b, 'p, 'uid, 'canid, 's, 'pk, 'sig, 'sd) envelope = + +record ('p, 'canid, 'pk, 'sig) delegation = + pubkey :: 'pk + targets :: "'canid list option" + senders :: "'p list option" + expiration :: timestamp +record ('p, 'canid, 'pk, 'sig) signed_delegation = + delegation :: "('p, 'canid, 'pk, 'sig) delegation" + signature :: "'sig" + +record ('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope = content :: "('b, 'p, 'uid, 'canid, 's) request + ('b, 'uid, 'canid, 's) APIReadRequest" sender_pubkey :: "'pk option" sender_sig :: "'sig option" - sender_delegation :: 'sd + sender_delegation :: "('p, 'canid, 'pk, 'sig) delegation list" datatype ('b, 's) request_status = Received | Processing | Rejected reject_code 's | Replied 'b | Done (* The system state *) -record ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) can_state_rec = +record ('p, 'canid, 'b, 'w, 'sm, 'c, 's) can_state_rec = wasm_state :: 'w - module :: "('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module" + module :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module" raw_module :: 'b public_custom_sections :: "('s, 'b) list_map" private_custom_sections :: "('s, 'b) list_map" -type_synonym ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) can_state = "('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) can_state_rec option" +type_synonym ('p, 'canid, 'b, 'w, 'sm, 'c, 's) can_state = "('p, 'canid, 'b, 'w, 'sm, 'c, 's) can_state_rec option" datatype ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status = Running | Stopping "(('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin \ nat) list" | Stopped -record ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic = +record ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic = requests :: "(('b, 'p, 'uid, 'canid, 's) request, ('b, 's) request_status) list_map" - canisters :: "('canid, ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) can_state) list_map" + canisters :: "('canid, ('p, 'canid, 'b, 'w, 'sm, 'c, 's) can_state) list_map" controllers :: "('canid, 'p set) list_map" freezing_threshold :: "('canid, nat) list_map" canister_status :: "('canid, ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status) list_map" @@ -270,21 +389,164 @@ record ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic = messages :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message list" root_key :: 'pk +fun simple_status :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status \ status" where + "simple_status can_status.Running = status.Running" +| "simple_status (can_status.Stopping _) = status.Stopping" +| "simple_status can_status.Stopped = status.Stopped" + +(* Initial state *) + +definition initial_ic :: "nat \ 'pk \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "initial_ic t pk = \requests = list_map_empty, + canisters = list_map_empty, + controllers = list_map_empty, + freezing_threshold = list_map_empty, + canister_status = list_map_empty, + time = list_map_empty, + balances = list_map_empty, + certified_data = list_map_empty, + system_time = t, + call_contexts = list_map_empty, + messages = [], + root_key = pk\" + +(* Invariants *) + +definition ic_can_status_inv :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status set \ 'cid set \ bool" where + "ic_can_status_inv st c = (\can_status \ st. + case can_status of Stopping os \ \(orig, cycles) \ set os. (case orig of + From_canister ctxt_id _ \ ctxt_id \ c + | _ \ True) + | _ \ True)" + +lemma ic_can_status_inv_mono1: "ic_can_status_inv x y \ + z \ x \ {can_status.Running, can_status.Stopped} \ + ic_can_status_inv z y" + by (fastforce simp: ic_can_status_inv_def split: can_status.splits call_origin.splits) + +lemma ic_can_status_inv_mono2: "ic_can_status_inv x y \ + y \ z \ + ic_can_status_inv x z" + by (force simp: ic_can_status_inv_def split: can_status.splits call_origin.splits) + +lemma ic_can_status_inv_stopping: "ic_can_status_inv x y \ + (\ctxt_id c. os = From_canister ctxt_id c \ ctxt_id \ y) \ + z \ x \ {can_status.Stopping [(os, cyc)]} \ + ic_can_status_inv z y" + by (fastforce simp: ic_can_status_inv_def split: can_status.splits call_origin.splits) + +lemma ic_can_status_inv_stopping_app: "ic_can_status_inv x y \ + can_status.Stopping w \ x \ + (\ctxt_id c. os = From_canister ctxt_id c \ ctxt_id \ y) \ + z \ x \ {can_status.Stopping (w @ [(os, cyc)])} \ + ic_can_status_inv z y" + by (force simp: ic_can_status_inv_def split: can_status.splits call_origin.splits dest!: subsetD[where ?A=z]) + +lemma ic_can_status_inv_del: "ic_can_status_inv x z \ + (\os other_ctxt_id c cyc. Stopping os \ x \ (From_canister other_ctxt_id c, cyc) \ set os \ ctxt_id \ other_ctxt_id) \ + ic_can_status_inv x (z - {ctxt_id})" + by (fastforce simp: ic_can_status_inv_def split: can_status.splits call_origin.splits) + +definition ic_inv :: "('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_inv S = ((\msg \ set (messages S). case msg of + Call_message (From_canister ctxt_id _) _ _ _ _ _ _ \ ctxt_id \ list_map_dom (call_contexts S) + | Response_message (From_canister ctxt_id _) _ _ \ ctxt_id \ list_map_dom (call_contexts S) + | _ \ True) \ + (\ctxt \ list_map_range (call_contexts S). call_ctxt_needs_to_respond ctxt \ + (case call_ctxt_origin ctxt of From_canister ctxt_id _ \ ctxt_id \ list_map_dom (call_contexts S) + | _ \ True)) \ + ic_can_status_inv (list_map_range (canister_status S)) (list_map_dom (call_contexts S)))" + +lemma ic_initial_inv: "ic_inv (initial_ic t pk)" + by (auto simp: ic_inv_def ic_can_status_inv_def initial_ic_def split: call_origin.splits) + +(* Candid *) + +datatype ('s, 'b, 'p) candid = + Candid_nat nat + | Candid_text 's + | Candid_blob (candid_unwrap_blob: 'b) + | Candid_opt "('s, 'b, 'p) candid" + | Candid_vec "('s, 'b, 'p) candid list" + | Candid_record "('s, ('s, 'b, 'p) candid) list_map" + | Candid_null + | Candid_empty + +fun candid_is_blob :: "('s, 'b, 'p) candid \ bool" where + "candid_is_blob (Candid_blob b) = True" +| "candid_is_blob _ = False" + +fun candid_lookup :: "('s, 'b, 'p) candid \ 's \ ('s, 'b, 'p) candid option" where + "candid_lookup (Candid_record m) s = list_map_get m s" +| "candid_lookup _ _ = None" + (* State transitions *) context fixes CANISTER_ERROR :: reject_code + and CANISTER_REJECT :: reject_code and SYS_FATAL :: reject_code and SYS_TRANSIENT :: reject_code and MAX_CYCLES_PER_MESSAGE :: nat and MAX_CYCLES_PER_RESPONSE :: nat and MAX_CANISTER_BALANCE :: nat - and ic_freezing_limit :: "('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ 'canid \ nat" + and ic_idle_cycles_burned_rate :: "('p :: linorder, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ 'canid \ nat" + and blob_length :: "'b \ nat" + and sha_256 :: "'b \ 'b" + and ic_principal :: 'canid + and blob_of_candid :: "('s, 'b, 'p) candid \ 'b" + and parse_candid :: "'b \ ('s, 'b, 'p) candid option" + and parse_principal :: "'b \ 'p option" + and blob_of_principal :: "'p \ 'b" + and empty_blob :: 'b + and is_system_assigned :: "'p \ bool" and encode_string :: "string \ 's" and principal_of_uid :: "'uid \ 'p" and principal_of_canid :: "'canid \ 'p" + and canid_of_principal :: "'p \ 'canid option" + and parse_wasm_mod :: "'b \ ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module option" + and parse_public_custom_sections :: "'b \ ('s, 'b) list_map option" + and parse_private_custom_sections :: "'b \ ('s, 'b) list_map option" + and verify_envelope :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ 'p \ nat \ 'p set" (* TODO *) + and principal_list_of_set :: "'p set \ 'p list" begin +(* Type conversion functions *) + +definition canid_of_blob :: "'b \ 'canid option" where + "canid_of_blob b = (case parse_principal b of Some p \ canid_of_principal p | _ \ None)" + +definition blob_of_canid :: "'canid \ 'b" where + "blob_of_canid = blob_of_principal \ principal_of_canid" + +(* Candid helper functions *) + +definition candid_nested_lookup :: "'b \ 's list \ ('s, 'b, 'p) candid option" where + "candid_nested_lookup b = foldl (\c s. case c of Some c' \ candid_lookup c' s | _ \ None) (parse_candid b)" + +definition candid_parse_nat :: "'b \ 's list \ nat option" where + "candid_parse_nat b s = (case candid_nested_lookup b s of Some (Candid_nat n') \ Some n' | _ \ None)" + +definition candid_parse_text :: "'b \ 's list \ 's option" where + "candid_parse_text b s = (case candid_nested_lookup b s of Some (Candid_text t') \ Some t' | _ \ None)" + +definition candid_parse_blob :: "'b \ 's list \ 'b option" where + "candid_parse_blob b s = (case candid_nested_lookup b s of Some (Candid_blob b') \ Some b' | _ \ None)" + +definition candid_parse_cid :: "'b \ 'canid option" where + "candid_parse_cid b = (case candid_parse_blob b [encode_string ''canister_id''] of Some b' \ canid_of_blob b' | _ \ None)" + +definition candid_parse_controllers :: "'b \ 'p set option" where + "candid_parse_controllers b = (case candid_nested_lookup b [encode_string ''settings'', encode_string ''controllers''] of Some (Candid_vec xs) \ + if (\c'' \ set xs. candid_is_blob c'' \ parse_principal (candid_unwrap_blob c'') \ None) then + Some (the ` parse_principal ` candid_unwrap_blob ` set xs) + else None | _ \ None)" + +fun candid_of_status :: "status \ ('s, 'b, 'p) candid" where + "candid_of_status status.Running = Candid_text (encode_string ''Running'')" +| "candid_of_status status.Stopping = Candid_text (encode_string ''Stopping'')" +| "candid_of_status status.Stopped = Candid_text (encode_string ''Stopped'')" + (* Cycles *) fun carried_cycles :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin \ nat" where @@ -301,6 +563,10 @@ fun message_cycles :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message \ nat" where + "status_cycles (Stopping os) = sum_list (map (carried_cycles \ fst) os) + sum_list (map snd os)" +| "status_cycles _ = 0" + lift_definition call_ctxt_carried_cycles :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ nat" is "\ctxt. (if needs_to_respond ctxt then available_cycles ctxt + carried_cycles (origin ctxt) @@ -313,12 +579,16 @@ lemma call_ctxt_carried_cycles: "call_ctxt_carried_cycles ctxt = (if call_ctxt_n then call_ctxt_available_cycles ctxt + carried_cycles (call_ctxt_origin ctxt) else 0)" by transfer auto -definition total_cycles :: "('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where +lemma call_ctxt_delete_carried_cycles[simp]: "call_ctxt_carried_cycles (call_ctxt_delete ctxt) = 0" + by transfer auto + +definition total_cycles :: "('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where "total_cycles ic = ( let cycles_in_balances = list_map_sum_vals id (balances ic) in let cycles_in_messages = sum_list (map message_cycles (messages ic)) in let cycles_in_contexts = list_map_sum_vals call_ctxt_carried_cycles (call_contexts ic) in - cycles_in_balances + cycles_in_messages + cycles_in_contexts)" + let cycles_in_statuses = list_map_sum_vals status_cycles (canister_status ic) in + cycles_in_balances + cycles_in_messages + cycles_in_contexts + cycles_in_statuses)" (* Accessor functions *) @@ -331,91 +601,199 @@ fun message_queue :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message \ | "message_queue (Func_message _ _ _ q) = Some q" | "message_queue _ = None" -(* Type conversion functions *) +(* Effective canister IDs *) -fun simple_status :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status \ status" where - "simple_status can_status.Running = status.Running" -| "simple_status (can_status.Stopping _) = status.Stopping" -| "simple_status can_status.Stopped = status.Stopped" +definition is_effective_canister_id :: "('b, 'p, 'uid, 'canid, 's) request \ 'p \ bool" where + "is_effective_canister_id r p = (if request.canister_id r = ic_principal then + (case candid_parse_cid (request.arg r) of Some cid \ principal_of_canid cid = p + | _ \ request.method_name r = encode_string ''provisional_create_canister_with_cycles'') + else principal_of_canid (request.canister_id r) = p)" + +lemma is_effective_canister_id_code[code_unfold]: + "(\p. is_effective_canister_id r p) = (if request.canister_id r = ic_principal then + (case candid_parse_cid (request.arg r) of Some cid \ True + | _ \ request.method_name r = encode_string ''provisional_create_canister_with_cycles'') + else True)" + by (auto simp: is_effective_canister_id_def split: option.splits) (* System transition: API Request submission [DONE] *) -definition request_submission_pre :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig, 'sd) envelope \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where - "request_submission_pre E S = (case content E of Inl req \ req \ list_map_dom (requests S) | _ \ False)" +definition ic_freezing_limit :: "('p :: linorder, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ 'canid \ nat" where + "ic_freezing_limit S cid = ic_idle_cycles_burned_rate S cid * (the (list_map_get (freezing_threshold S) cid)) div (24 * 60 * 60)" -definition request_submission_post :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig, 'sd) envelope \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where - "request_submission_post E S = S\requests := list_map_set (requests S) (projl (content E)) Received\" +definition request_submission_pre :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "request_submission_pre E S = (case content E of Inl req \ + principal_of_canid (request.canister_id req) \ verify_envelope E (principal_of_uid (request.sender req)) (system_time S) \ + req \ list_map_dom (requests S) \ + system_time S \ request.ingress_expiry req \ + (\ECID. is_effective_canister_id req ECID) \ + ( + ( + request.canister_id req = ic_principal \ + (case candid_parse_cid (request.arg req) of Some cid \ + (case list_map_get (controllers S) cid of Some ctrls \ + principal_of_uid (request.sender req) \ ctrls \ + request.method_name req \ {encode_string ''install_code'', encode_string ''uninstall_code'', encode_string ''update_settings'', + encode_string ''start_canister'', encode_string ''stop_canister'', + encode_string ''canister_status'', encode_string ''delete_canister'', + encode_string ''provisional_create_canister_with_cycles'', encode_string ''provisional_top_up_canister''} + | _ \ False) + | _ \ False) + ) + \ + ( + request.canister_id req \ ic_principal \ + (case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req)) of + (Some (Some can), Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status\ in + (case canister_module_inspect_message (module can) (request.method_name req, wasm_state can, request.arg req, principal_of_uid (request.sender req), env) of Inr ret \ + cycles_return.return ret = Accept \ cycles_return.cycles_used ret \ bal + | _ \ False) + | _ \ False) + ) + ) + | _ \ False)" + +definition request_submission_post :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "request_submission_post E S = ( + let req = projl (content E); + cid = request.canister_id req; + balances = (if cid \ ic_principal then + (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of + (Some (Some can), Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + (case canister_module_inspect_message (module can) (request.method_name req, wasm_state can, request.arg req, principal_of_uid (request.sender req), env) of Inr ret \ + list_map_set (balances S) cid (bal - cycles_return.cycles_used ret))) + else balances S) in + S\requests := list_map_set (requests S) req Received, balances := balances\)" + +definition request_submission_burned_cycles :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "request_submission_burned_cycles E S = ( + let req = projl (content E); + cid = request.canister_id req in + (if request.canister_id req \ ic_principal then + (case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req)) of + (Some (Some can), Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status\ in + (case canister_module_inspect_message (module can) (request.method_name req, wasm_state can, request.arg req, principal_of_uid (request.sender req), env) of Inr ret \ + cycles_return.cycles_used ret)) + else 0))" lemma request_submission_cycles_inv: - "request_submission_pre E S \ total_cycles S = total_cycles (request_submission_post E S)" - by (auto simp: request_submission_pre_def request_submission_post_def total_cycles_def) + assumes "request_submission_pre E S" + shows "total_cycles S = total_cycles (request_submission_post E S) + request_submission_burned_cycles E S" +proof - + obtain req where req_def: "content E = Inl req" + using assms + by (auto simp: request_submission_pre_def split: sum.splits) + { + assume "request.canister_id req \ ic_principal" + then have "(case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req)) of + (Some (Some can), Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status\ in + (case canister_module_inspect_message (module can) (request.method_name req, wasm_state can, request.arg req, principal_of_uid (request.sender req), env) of Inr ret \ + cycles_return.return ret = Accept \ cycles_return.cycles_used ret \ bal + | _ \ False) + | _ \ False)" + using assms + by (auto simp: request_submission_pre_def req_def split: option.splits sum.splits prod.splits) + then have ?thesis + using list_map_sum_in_ge[where ?f="balances S" and ?g=id and ?x="request.canister_id req"] + by (auto simp: request_submission_pre_def request_submission_post_def request_submission_burned_cycles_def total_cycles_def req_def list_map_sum_in[where ?f="balances S"] + split: option.splits sum.splits) + } + then show ?thesis + by (auto simp: request_submission_pre_def request_submission_post_def request_submission_burned_cycles_def total_cycles_def req_def) +qed + +lemma request_submission_ic_inv: + assumes "request_submission_pre E S" "ic_inv S" + shows "ic_inv (request_submission_post E S)" + using assms + by (auto simp: ic_inv_def request_submission_pre_def request_submission_post_def Let_def + split: sum.splits message.splits call_origin.splits) (* System transition: Request rejection [DONE] *) -definition request_rejection_pre :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig, 'sd) envelope \ ('b, 'p, 'uid, 'canid, 's) request \ reject_code \ 's \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where +definition request_rejection_pre :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ ('b, 'p, 'uid, 'canid, 's) request \ reject_code \ 's \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "request_rejection_pre E req code msg S = (list_map_get (requests S) req = Some Received \ (code = SYS_FATAL \ code = SYS_TRANSIENT))" -definition request_rejection_post :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig, 'sd) envelope \ ('b, 'p, 'uid, 'canid, 's) request \ reject_code \ 's \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where +definition request_rejection_post :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ ('b, 'p, 'uid, 'canid, 's) request \ reject_code \ 's \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where "request_rejection_post E req code msg S = S\requests := list_map_set (requests S) req (Rejected code msg)\" lemma request_rejection_cycles_inv: - "request_rejection_pre E req code msg S \ total_cycles S = total_cycles (request_rejection_post E req code msg S)" + assumes "request_rejection_pre E req code msg S" + shows "total_cycles S = total_cycles (request_rejection_post E req code msg S)" by (auto simp: request_rejection_pre_def request_rejection_post_def total_cycles_def) +lemma request_rejection_ic_inv: + assumes "request_rejection_pre E req code msg S" "ic_inv S" + shows "ic_inv (request_rejection_post E req code msg S)" + using assms + by (auto simp: ic_inv_def request_rejection_pre_def request_rejection_post_def Let_def + split: sum.splits message.splits call_origin.splits) + (* System transition: Initiating canister calls [DONE] *) definition initiate_canister_call_pre :: "('b, 'p, 'uid, 'canid, 's) request \ - ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "initiate_canister_call_pre req S = (list_map_get (requests S) req = Some Received \ system_time S \ request.ingress_expiry req \ request.canister_id req \ list_map_dom (canisters S))" definition initiate_canister_call_post :: "('b, 'p, 'uid, 'canid, 's) request \ - ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ - ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ + ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where "initiate_canister_call_post req S = S\requests := list_map_set (requests S) req Processing, messages := Call_message (From_user req) (principal_of_uid (request.sender req)) (request.canister_id req) (request.method_name req) (request.arg req) 0 Unordered # messages S\" lemma initiate_canister_call_cycles_inv: - "initiate_canister_call_pre R S \ - total_cycles S = total_cycles (initiate_canister_call_post R S)" + assumes "initiate_canister_call_pre R S" + shows "total_cycles S = total_cycles (initiate_canister_call_post R S)" by (auto simp: initiate_canister_call_pre_def initiate_canister_call_post_def total_cycles_def) +lemma initiate_canister_call_ic_inv: + assumes "initiate_canister_call_pre R S" "ic_inv S" + shows "ic_inv (initiate_canister_call_post R S)" + using assms + by (auto simp: ic_inv_def initiate_canister_call_pre_def initiate_canister_call_post_def Let_def + split: sum.splits message.splits call_origin.splits) + (* System transition: Calls to stopped/stopping/frozen canisters are rejected [DONE] *) -definition call_reject_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where +definition call_reject_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "call_reject_pre n S = (n < length (messages S) \ (case messages S ! n of - Call_message orig cer cee mn d trans_cycles q \ + Call_message orig cer cee mn a trans_cycles q \ (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ (case list_map_get (canister_status S) cee of Some Stopped \ True | Some (Stopping l) \ True | _ \ (case list_map_get (balances S) cee of Some bal \ bal < ic_freezing_limit S cee | _ \ False)) | _ \ False))" -definition call_reject_post :: "nat \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where - "call_reject_post n S = (case messages S ! n of Call_message orig cer cee mn d trans_cycles q \ +definition call_reject_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "call_reject_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ S\messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (response.Reject CANISTER_ERROR (encode_string ''canister not running'')) trans_cycles]\)" lemma call_reject_cycles_inv: assumes "call_reject_pre n S" shows "total_cycles S = total_cycles (call_reject_post n S)" proof - - obtain orig cer cee mn d trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn d trans_cycles q" + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" using assms by (auto simp: call_reject_pre_def split: message.splits) define older where "older = take n (messages S)" define younger where "younger = drop (Suc n) (messages S)" - have msgs: "messages S = older @ Call_message orig cer cee mn d trans_cycles q # younger" "(older @ w # younger) ! n = w" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" "drop (Suc n - length older) (w # ws) = ws" for w ws using id_take_nth_drop[of n "messages S"] assms @@ -425,14 +803,22 @@ proof - by (auto simp: call_reject_pre_def call_reject_post_def total_cycles_def Let_def msgs split: message.splits option.splits) qed +lemma call_reject_ic_inv: + assumes "call_reject_pre n S" "ic_inv S" + shows "ic_inv (call_reject_post n S)" + using assms + by (auto simp: ic_inv_def call_reject_pre_def call_reject_post_def Let_def + split: sum.splits message.splits call_origin.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"]) + (* System transition: Call context creation: Public entry points [DONE] *) definition call_context_create_pre :: "nat \ 'cid - \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "call_context_create_pre n ctxt_id S = (n < length (messages S) \ (case messages S ! n of - Call_message orig cer cee mn d trans_cycles q \ + Call_message orig cer cee mn a trans_cycles q \ (case list_map_get (canisters S) cee of Some (Some can) \ True | _ \ False) \ list_map_get (canister_status S) cee = Some Running \ (case list_map_get (balances S) cee of Some bal \ bal \ ic_freezing_limit S cee + MAX_CYCLES_PER_MESSAGE | None \ False) \ @@ -444,13 +830,16 @@ lift_definition create_call_ctxt :: "'canid \ ('b, 'p, 'uid, 'canid, "\cee orig trans_cycles. \canister = cee, origin = orig, needs_to_respond = True, deleted = False, available_cycles = trans_cycles\" by auto +lemma create_call_ctxt_origin[simp]: "call_ctxt_origin (create_call_ctxt cee orig trans_cycles) = orig" + by transfer auto + lemma create_call_ctxt_carried_cycles[simp]: "call_ctxt_carried_cycles (create_call_ctxt cee orig trans_cycles) = carried_cycles orig + trans_cycles" by transfer auto -definition call_context_create_post :: "nat \ 'cid \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where - "call_context_create_post n ctxt_id S = (case messages S ! n of Call_message orig cer cee mn d trans_cycles q \ +definition call_context_create_post :: "nat \ 'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "call_context_create_post n ctxt_id S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ case list_map_get (balances S) cee of Some bal \ - S\messages := list_update (messages S) n (Func_message ctxt_id cee (Public_method mn cer d) q), + S\messages := list_update (messages S) n (Func_message ctxt_id cee (Public_method mn cer a) q), call_contexts := list_map_set (call_contexts S) ctxt_id (create_call_ctxt cee orig trans_cycles), balances := list_map_set (balances S) cee (bal - MAX_CYCLES_PER_MESSAGE)\)" @@ -458,26 +847,35 @@ lemma call_context_create_cycles_inv: assumes "call_context_create_pre n ctxt_id S" shows "total_cycles S = total_cycles (call_context_create_post n ctxt_id S)" proof - - obtain orig cer cee mn d trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn d trans_cycles q" + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" using assms by (auto simp: call_context_create_pre_def split: message.splits) - define xs where "xs = take n (messages S)" - define older where "older = drop (Suc n) (messages S)" - have msgs: "messages S = xs @ Call_message orig cer cee mn d trans_cycles q # older" "(xs @ m # older) ! n = m" - and msgs_upd: "(xs @ Call_message orig cer cee mn d trans_cycles q # older)[n := m] = xs @ m # older" for m + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ m # younger) ! n = m" + and msgs_upd: "(older @ Call_message orig cer cee mn a trans_cycles q # younger)[n := m] = older @ m # younger" for m using id_take_nth_drop[of n "messages S"] upd_conv_take_nth_drop[of n "messages S"] assms - by (auto simp: call_context_create_pre_def msg xs_def older_def nth_append) + by (auto simp: call_context_create_pre_def msg older_def younger_def nth_append) show ?thesis using assms list_map_sum_in_ge[of "balances S" cee, where ?g=id] by (auto simp: call_context_create_pre_def call_context_create_post_def total_cycles_def list_map_sum_in[where ?g=id, simplified] list_map_sum_out msgs msgs_upd split: option.splits) qed +lemma call_context_create_ic_inv: + assumes "call_context_create_pre n ctxt_id S" "ic_inv S" + shows "ic_inv (call_context_create_post n ctxt_id S)" + using assms + by (auto simp: ic_inv_def call_context_create_pre_def call_context_create_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD + intro: ic_can_status_inv_mono2) + (* System transition: Call context creation: Heartbeat [DONE] *) -definition call_context_heartbeat_pre :: "'canid \ 'cid \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where +definition call_context_heartbeat_pre :: "'canid \ 'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "call_context_heartbeat_pre cee ctxt_id S = ( (case list_map_get (canisters S) cee of Some (Some can) \ True | _ \ False) \ list_map_get (canister_status S) cee = Some Running \ @@ -488,10 +886,13 @@ lift_definition create_call_ctxt_heartbeat :: "'canid \ ('p, 'uid, ' "\cee. \canister = cee, origin = From_heartbeat, needs_to_respond = False, deleted = False, available_cycles = 0\" by auto +lemma create_call_ctxt_heartbeat_needs_to_respond[simp]: "call_ctxt_needs_to_respond (create_call_ctxt_heartbeat cee) = False" + by transfer auto + lemma create_call_ctxt_heartbeat_carried_cycles[simp]: "call_ctxt_carried_cycles (create_call_ctxt_heartbeat cee) = 0" by transfer auto -definition call_context_heartbeat_post :: "'canid \ 'cid \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where +definition call_context_heartbeat_post :: "'canid \ 'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where "call_context_heartbeat_post cee ctxt_id S = (case list_map_get (balances S) cee of Some bal \ S\messages := Func_message ctxt_id cee Heartbeat (Queue System cee) # messages S, @@ -499,36 +900,47 @@ definition call_context_heartbeat_post :: "'canid \ 'cid \)" lemma call_context_heartbeat_cycles_inv: - "call_context_heartbeat_pre cee ctxt_id S \ - total_cycles S = total_cycles (call_context_heartbeat_post cee ctxt_id S)" - using list_map_sum_in_ge[of "balances S" cee, where ?g=id, simplified] + assumes "call_context_heartbeat_pre cee ctxt_id S" + shows "total_cycles S = total_cycles (call_context_heartbeat_post cee ctxt_id S)" + using assms list_map_sum_in_ge[of "balances S" cee, where ?g=id, simplified] by (auto simp: call_context_heartbeat_pre_def call_context_heartbeat_post_def total_cycles_def list_map_sum_in[where ?g=id, simplified] list_map_sum_out split: option.splits) +lemma call_context_heartbeat_ic_inv: + assumes "call_context_heartbeat_pre cee ctxt_id S" "ic_inv S" + shows "ic_inv (call_context_heartbeat_post cee ctxt_id S)" + using assms + by (auto simp: ic_inv_def call_context_heartbeat_pre_def call_context_heartbeat_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD + intro: ic_can_status_inv_mono2) + (* System transition: Message execution [DONE] *) -fun query_as_update :: "(('b arg \ 'p \ 'b env) \ ('w, 'b, 's, 'tr) query_func) \ 'b arg \ 'p \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func" where +fun query_as_update :: "(('b arg \ 'p \ 'b env) \ ('w, 'b, 's) query_func) \ 'b arg \ 'p \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" where "query_as_update (f, a, e) = (\w. case f (a, e) w of Inl t \ Inl t | - Inr res \ Inr \new_state = w, new_calls = [], new_certified_data = None, response = Some res, cycles_accepted = 0\)" - -fun heartbeat_as_update :: "('b env \ 'w \ 'tr + 'w) \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func" where - "heartbeat_as_update (f, e) w = (case f e w of Inl t \ Inl t | - Inr w' \ Inr \update_return.new_state = w', update_return.new_calls = [], update_return.new_certified_data = None, - update_return.response = None, update_return.cycles_accepted = 0\)" - -fun exec_function :: "('s, 'p, 'b, 'c) entry_point \ 'b env \ nat \ ('p, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's) canister_module \ ('w, 'p, 'canid, 's, 'b, 'c, 'tr) update_func" where - "exec_function (entry_point.Public_method mn c d) e bal m = ( - case dispatch_method mn m of Some (Inl upd) \ upd (d, c, e, bal) - | Some (Inr query) \ query_as_update (query, d, c, e) | None \ - undefined)" -| "exec_function (entry_point.Callback cb resp ref_cycles) e bal m = - canister_module_callbacks m (cb, resp, ref_cycles, e, bal)" + Inr res \ Inr \update_return.new_state = w, update_return.new_calls = [], update_return.new_certified_data = None, + update_return.response = Some (query_return.response res), update_return.cycles_accepted = 0, update_return.cycles_used = query_return.cycles_used res\)" + +fun heartbeat_as_update :: "('b env \ 'w heartbeat_func) \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" where + "heartbeat_as_update (f, e) = (\w. case f e w of Inl t \ Inl t | + Inr res \ Inr \update_return.new_state = heartbeat_return.new_state res, update_return.new_calls = [], update_return.new_certified_data = None, + update_return.response = None, update_return.cycles_accepted = 0, update_return.cycles_used = heartbeat_return.cycles_used res\)" + +fun exec_function :: "('s, 'p, 'b, 'c) entry_point \ 'b env \ nat \ ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" where + "exec_function (entry_point.Public_method mn c a) e bal m = ( + case dispatch_method mn m of Some (Inl upd) \ upd (a, c, e, bal) + | Some (Inr query) \ query_as_update (query, a, c, e) + | None \ undefined + )" +| "exec_function (entry_point.Callback c resp ref_cycles) e bal m = + canister_module_callbacks m (c, resp, ref_cycles, e, bal)" | "exec_function (entry_point.Heartbeat) e bal m = heartbeat_as_update ((canister_module_heartbeat m), e)" -definition message_execution_pre :: "nat \ nat \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where - "message_execution_pre n cycles_used S = +definition message_execution_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "message_execution_pre n S = (n < length (messages S) \ (case messages S ! n of Func_message ctxt_id recv ep q \ (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ (case (list_map_get (canisters S) recv, list_map_get (balances S) recv, list_map_get (canister_status S) recv, @@ -536,53 +948,65 @@ definition message_execution_pre :: "nat \ nat \ ('p, 'u (Some (Some can), Some bal, Some can_status, Some t, Some ctxt) \ True | _ \ False) | _ \ False))" -definition message_execution_post :: "nat \ nat \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where - "message_execution_post n cycles_used S = (case messages S ! n of Func_message ctxt_id recv ep q \ +definition message_execution_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "message_execution_post n S = (case messages S ! n of Func_message ctxt_id recv ep q \ (case (list_map_get (canisters S) recv, list_map_get (balances S) recv, list_map_get (canister_status S) recv, list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id) of (Some (Some can), Some bal, Some can_status, Some t, Some ctxt) \ ( let Mod = module can; Is_response = (case ep of Callback _ _ _ \ True | _ \ False); - Env = \env_time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\; + Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\; Available = call_ctxt_available_cycles ctxt; F = exec_function ep Env Available Mod; R = F (wasm_state can); - (cycles_accepted_res, new_calls_res) = (case R of Inr res \ (cycles_accepted res, new_calls res)); + cyc_used = (case R of Inr res \ update_return.cycles_used res | Inl trap \ cycles_return.cycles_used trap); + (cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, update_return.new_calls res)); New_balance = bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - - (cycles_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res)); + - (cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res)); no_response = (case R of Inr result \ update_return.response result = None) in - if \isl R \ cycles_used \ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + if \isl R \ cyc_used \ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ cycles_accepted_res \ Available \ - cycles_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res) \ + cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res) \ bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ - New_balance > (if Is_response then 0 else ic_freezing_limit S recv) \ + New_balance \ (if Is_response then 0 else ic_freezing_limit S recv) \ (no_response \ call_ctxt_needs_to_respond ctxt) then (let result = projr R; - new_call_to_message = (\call. Call_message (From_canister ctxt_id (callback call)) (principal_of_canid recv) - (callee call) (method_call.method_name call) (method_call.arg call) (transferred_cycles call) (Queue (Canister recv) (callee call))); - response_messages = (case response result of None \ [] + new_call_to_message = (\call. Call_message (From_canister ctxt_id (method_call.callback call)) (principal_of_canid recv) + (method_call.callee call) (method_call.method_name call) (method_call.arg call) (method_call.transferred_cycles call) (Queue (Canister recv) (method_call.callee call))); + response_messages = (case update_return.response result of None \ [] | Some resp \ [Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res)]); messages = take n (messages S) @ drop (Suc n) (messages S) @ map new_call_to_message new_calls_res @ response_messages; - new_ctxt = (case response result of + new_ctxt = (case update_return.response result of None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt | Some _ \ call_ctxt_respond ctxt); certified_data = (case new_certified_data result of None \ certified_data S | Some cd \ list_map_set (certified_data S) recv cd) - in S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := new_state result\)), + in S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), messages := messages, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, - balances := list_map_set (balances S) recv New_balance, certified_data := certified_data\) + certified_data := certified_data, balances := list_map_set (balances S) recv New_balance\) else S\messages := take n (messages S) @ drop (Suc n) (messages S), - balances := list_map_set (balances S) recv (bal + ((if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - cycles_used))\)) + balances := list_map_set (balances S) recv ((bal + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) + - min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))\)) | _ \ undefined)" -definition message_execution_lost_cycles :: "nat \ nat \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where - "message_execution_lost_cycles n cycles_used S = (case messages S ! n of Func_message ctxt_id recv ep q \ - let Is_response = (case ep of Callback _ _ _ \ True | _ \ False) in - min cycles_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))" +definition message_execution_burned_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "message_execution_burned_cycles n S = (case messages S ! n of Func_message ctxt_id recv ep q \ + (case (list_map_get (canisters S) recv, list_map_get (balances S) recv, list_map_get (canister_status S) recv, + list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id) of + (Some (Some can), Some bal, Some can_status, Some t, Some ctxt) \ ( + let Mod = module can; + Is_response = (case ep of Callback _ _ _ \ True | _ \ False); + Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\; + Available = call_ctxt_available_cycles ctxt; + F = exec_function ep Env Available Mod; + R = F (wasm_state can); + cyc_used = (case R of Inr res \ update_return.cycles_used res | Inl trap \ cycles_return.cycles_used trap) in + min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + )))" lemma message_execution_cycles_monotonic: - assumes pre: "message_execution_pre n cycles_used S" - shows "total_cycles S = total_cycles (message_execution_post n cycles_used S) + message_execution_lost_cycles n cycles_used S" + assumes pre: "message_execution_pre n S" + shows "total_cycles S = total_cycles (message_execution_post n S) + message_execution_burned_cycles n S" proof - obtain ctxt_id recv ep q can bal can_status t ctxt where msg: "messages S ! n = Func_message ctxt_id recv ep q" and prod: "list_map_get (canisters S) recv = Some (Some can)" @@ -594,14 +1018,15 @@ proof - by (auto simp: message_execution_pre_def split: message.splits option.splits) define Mod where "Mod = can_state_rec.module can" define Is_response where "Is_response = (case ep of Callback _ _ _ \ True | _ \ False)" - define Env :: "'b env" where "Env = \env_time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\" + define Env :: "'b env" where "Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\" define Available where "Available = call_ctxt_available_cycles ctxt" define F where "F = exec_function ep Env Available Mod" define R where "R = F (wasm_state can)" - obtain cycles_accepted_res new_calls_res where res: "(cycles_accepted_res, new_calls_res) = (case R of Inr res \ (cycles_accepted res, new_calls res))" - by (cases "(case R of Inr res \ (cycles_accepted res, new_calls res))") auto + define cyc_used where "cyc_used = (case R of Inr res \ update_return.cycles_used res | Inl trap \ cycles_return.cycles_used trap)" + obtain cycles_accepted_res new_calls_res where res: "(cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, new_calls res))" + by (cases "(case R of Inr res \ (update_return.cycles_accepted res, new_calls res))") auto define New_balance where "New_balance = bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - - (cycles_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res))" + - (cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res))" define no_response where "no_response = (case R of Inr result \ update_return.response result = None)" define older where "older = take n (messages S)" define younger where "younger = drop (Suc n) (messages S)" @@ -614,23 +1039,23 @@ proof - note lm = list_map_sum_in[OF prod(2), where ?g=id, simplified] list_map_sum_in_ge[OF prod(2), where ?g=id, simplified] list_map_sum_in[OF prod(5), where ?g=call_ctxt_carried_cycles] list_map_sum_in_ge[OF prod(5), where ?g=call_ctxt_carried_cycles] define S'' where "S'' = S\messages := take n (messages S) @ drop (Suc n) (messages S), - balances := list_map_set (balances S) recv (bal + ((if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - cycles_used))\" - define cond where "cond = (\isl R \ cycles_used \ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + balances := list_map_set (balances S) recv ((bal + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) - min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))\" + define cond where "cond = (\isl R \ cyc_used \ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ cycles_accepted_res \ Available \ - cycles_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res) \ + cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res) \ bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ - New_balance > (if Is_response then 0 else ic_freezing_limit S recv) \ + New_balance \ (if Is_response then 0 else ic_freezing_limit S recv) \ (no_response \ call_ctxt_needs_to_respond ctxt))" have reserved: "(if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) = cycles_reserved ep" by (auto simp: Is_response_def split: entry_point.splits) show ?thesis proof (cases cond) case False - have "message_execution_post n cycles_used S = S''" - "message_execution_lost_cycles n cycles_used S = min cycles_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)" + have "message_execution_post n S = S''" + "message_execution_burned_cycles n S = min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)" using False - by (simp_all add: message_execution_post_def message_execution_lost_cycles_def Let_def msg prod - Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] res[symmetric] + by (simp_all add: message_execution_post_def message_execution_burned_cycles_def Let_def msg prod + Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] cyc_used_def[symmetric] res[symmetric] New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric] del: min_less_iff_conj split del: if_split) then show ?thesis using lm(2) @@ -640,32 +1065,32 @@ proof - define result where "result = projr R" have R_Inr: "R = Inr result" using True - by (auto simp: cond_def result_def) - define response_messages where "response_messages = (case response result of None \ [] + by (auto simp: cond_def result_def split: option.splits) + define response_messages where "response_messages = (case update_return.response result of None \ [] | Some resp \ [Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res)])" define new_call_to_message :: "(?'p, 'canid, 's, 'b, 'c) method_call \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message" where "new_call_to_message = (\call. Call_message (From_canister ctxt_id (callback call)) (principal_of_canid recv) (callee call) (method_call.method_name call) (method_call.arg call) (transferred_cycles call) (Queue (Canister recv) (callee call)))" define messages where "messages = take n (ic.messages S) @ drop (Suc n) (ic.messages S) @ map new_call_to_message new_calls_res @ response_messages" - define new_ctxt where "new_ctxt = (case response result of + define new_ctxt where "new_ctxt = (case update_return.response result of None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt | Some _ \ call_ctxt_respond ctxt)" define certified_data where "certified_data = (case new_certified_data result of None \ ic.certified_data S | Some cd \ list_map_set (ic.certified_data S) recv cd)" - define S' where "S' = S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := new_state result\)), + define S' where "S' = S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), messages := messages, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, - balances := list_map_set (balances S) recv New_balance, certified_data := certified_data\" - have cycles_accepted_res_def: "cycles_accepted_res = cycles_accepted result" + certified_data := certified_data, balances := list_map_set (balances S) recv New_balance\" + have cycles_accepted_res_def: "cycles_accepted_res = update_return.cycles_accepted result" and new_calls_res_def: "new_calls_res = new_calls result" using res by (auto simp: R_Inr) - have no_response: "no_response = (response result = None)" + have no_response: "no_response = (update_return.response result = None)" by (auto simp: no_response_def R_Inr) - have msg_exec: "message_execution_post n cycles_used S = S'" - and lost: "message_execution_lost_cycles n cycles_used S = min cycles_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)" + have msg_exec: "message_execution_post n S = S'" + and lost: "message_execution_burned_cycles n S = min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)" using True - by (simp_all add: message_execution_post_def message_execution_lost_cycles_def Let_def msg prod - Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] res[symmetric] + by (simp_all add: message_execution_post_def message_execution_burned_cycles_def Let_def msg prod + Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] cyc_used_def[symmetric] res[symmetric] New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric] messages_def[symmetric] new_ctxt_def[symmetric] certified_data_def[symmetric] S'_def[symmetric] result_def[symmetric] response_messages_def[symmetric] new_call_to_message_def[symmetric] @@ -674,11 +1099,11 @@ proof - by (auto simp: new_call_to_message_def) then have A1: "sum_list (map (message_cycles \ new_call_to_message) new_calls_res) = (\x\new_calls_res. MAX_CYCLES_PER_RESPONSE + transferred_cycles x)" by auto - have A2: "sum_list (map local.message_cycles response_messages) = (case response result of None \ 0 - | _ \ carried_cycles (call_ctxt_origin ctxt) + (call_ctxt_available_cycles ctxt - cycles_accepted result))" + have A2: "sum_list (map local.message_cycles response_messages) = (case update_return.response result of None \ 0 + | _ \ carried_cycles (call_ctxt_origin ctxt) + (call_ctxt_available_cycles ctxt - update_return.cycles_accepted result))" by (auto simp: response_messages_def Available_def cycles_accepted_res_def split: option.splits) - have A3: "call_ctxt_carried_cycles new_ctxt = (case response result of Some _ \ 0 - | _ \ if call_ctxt_needs_to_respond ctxt then carried_cycles (call_ctxt_origin ctxt) + (call_ctxt_available_cycles ctxt - cycles_accepted result) else 0)" + have A3: "call_ctxt_carried_cycles new_ctxt = (case update_return.response result of Some _ \ 0 + | _ \ if call_ctxt_needs_to_respond ctxt then carried_cycles (call_ctxt_origin ctxt) + (call_ctxt_available_cycles ctxt - update_return.cycles_accepted result) else 0)" by (auto simp: new_ctxt_def Available_def cycles_accepted_res_def call_ctxt_carried_cycles split: option.splits) have A4: "call_ctxt_carried_cycles ctxt = (if call_ctxt_needs_to_respond ctxt then carried_cycles (call_ctxt_origin ctxt) + call_ctxt_available_cycles ctxt else 0)" using call_ctxt_carried_cycles @@ -688,31 +1113,152 @@ proof - have messages_msgs: "messages = older @ younger @ map new_call_to_message new_calls_res @ response_messages" by (auto simp: messages_def older_def younger_def) show ?thesis - using lm(2,4) True call_ctxt_inv[of ctxt] + using lm(2,4) True call_ctxt_not_needs_to_respond_available_cycles[of ctxt] by (auto simp: cond_def msg_exec S'_def total_cycles_def lm(1,3) msgs messages_msgs A1 A2 A3 A4 New_balance_def reserve cycles_accepted_res_def no_response_def R_Inr lost Available_def split: option.splits) qed qed +lemma message_execution_ic_inv: + assumes "message_execution_pre n S" "ic_inv S" + shows "ic_inv (message_execution_post n S)" +proof - + obtain ctxt_id recv ep q can bal can_status t ctxt where msg: "messages S ! n = Func_message ctxt_id recv ep q" + and prod: "list_map_get (canisters S) recv = Some (Some can)" + "list_map_get (balances S) recv = Some bal" + "list_map_get (canister_status S) recv = Some can_status" + "list_map_get (time S) recv = Some t" + "list_map_get (call_contexts S) ctxt_id = Some ctxt" + using assms + by (auto simp: message_execution_pre_def split: message.splits option.splits) + define Mod where "Mod = can_state_rec.module can" + define Is_response where "Is_response = (case ep of Callback _ _ _ \ True | _ \ False)" + define Env :: "'b env" where "Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\" + define Available where "Available = call_ctxt_available_cycles ctxt" + define F where "F = exec_function ep Env Available Mod" + define R where "R = F (wasm_state can)" + define cyc_used where "cyc_used = (case R of Inr res \ update_return.cycles_used res | Inl trap \ cycles_return.cycles_used trap)" + obtain cycles_accepted_res new_calls_res where res: "(cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, new_calls res))" + by (cases "(case R of Inr res \ (update_return.cycles_accepted res, new_calls res))") auto + define New_balance where "New_balance = bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + - (cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res))" + define no_response where "no_response = (case R of Inr result \ update_return.response result = None)" + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Func_message ctxt_id recv ep q # younger" + "take n older = older" "drop (Suc n) older = []" + "take (n - length older) ws = []" "drop (Suc n - length older) (w # ws) = ws" + for w and ws :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message list" + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: message_execution_pre_def msg older_def younger_def) + define S'' where "S'' = S\messages := take n (messages S) @ drop (Suc n) (messages S), + balances := list_map_set (balances S) recv ((bal + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) - min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))\" + define cond where "cond = (\isl R \ cyc_used \ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + cycles_accepted_res \ Available \ + cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res) \ + bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + New_balance \ (if Is_response then 0 else ic_freezing_limit S recv) \ + (no_response \ call_ctxt_needs_to_respond ctxt))" + show ?thesis + proof (cases cond) + case False + have "message_execution_post n S = S''" + using False + by (simp_all add: message_execution_post_def Let_def msg prod + Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] cyc_used_def[symmetric] res[symmetric] + New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric]) + then show ?thesis + using assms(2) + apply (auto simp: ic_inv_def S''_def msgs split: message.splits call_origin.splits) + apply force + apply fast + apply force + apply fast + done + next + case True + define result where "result = projr R" + have R_Inr: "R = Inr result" + using True + by (auto simp: cond_def result_def split: option.splits) + define response_messages where "response_messages = (case update_return.response result of None \ [] + | Some resp \ [Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res)])" + define new_call_to_message :: "(?'p, 'canid, 's, 'b, 'c) method_call \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message" where + "new_call_to_message = (\call. Call_message (From_canister ctxt_id (callback call)) (principal_of_canid recv) + (callee call) (method_call.method_name call) (method_call.arg call) (transferred_cycles call) (Queue (Canister recv) (callee call)))" + define messages where "messages = take n (ic.messages S) @ drop (Suc n) (ic.messages S) @ map new_call_to_message new_calls_res @ response_messages" + define new_ctxt where "new_ctxt = (case update_return.response result of + None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt + | Some _ \ call_ctxt_respond ctxt)" + define certified_data where "certified_data = (case new_certified_data result of None \ ic.certified_data S + | Some cd \ list_map_set (ic.certified_data S) recv cd)" + define S' where "S' = S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), + messages := messages, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, + certified_data := certified_data, balances := list_map_set (balances S) recv New_balance\" + have cycles_accepted_res_def: "cycles_accepted_res = update_return.cycles_accepted result" + and new_calls_res_def: "new_calls_res = new_calls result" + using res + by (auto simp: R_Inr) + have no_response: "no_response = (update_return.response result = None)" + by (auto simp: no_response_def R_Inr) + have msg_exec: "message_execution_post n S = S'" + using True + by (simp_all add: message_execution_post_def Let_def msg prod + Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] cyc_used_def[symmetric] res[symmetric] + New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric] + messages_def[symmetric] new_ctxt_def[symmetric] certified_data_def[symmetric] S'_def[symmetric] + result_def[symmetric] response_messages_def[symmetric] new_call_to_message_def[symmetric]) + have messages_msgs: "messages = older @ younger @ map new_call_to_message new_calls_res @ response_messages" + by (auto simp: messages_def older_def younger_def) + have ctxt_in_range: "ctxt \ list_map_range (call_contexts S)" + using prod(5) + by (simp add: list_map_get_range) + have response_msgsD: "msg \ set response_messages \ update_return.response result \ None \ + (\resp. msg = Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res))" for msg + by (auto simp: response_messages_def) metis + have call_ctxt_origin_new_ctxt: "call_ctxt_origin new_ctxt = call_ctxt_origin ctxt" + by (auto simp: new_ctxt_def split: option.splits) + have call_ctxt_needs_to_respond_new_ctxtD: "call_ctxt_needs_to_respond new_ctxt \ call_ctxt_needs_to_respond ctxt" + by (auto simp: new_ctxt_def split: option.splits) + show ?thesis + using assms(2) ctxt_in_range True call_ctxt_not_needs_to_respond_available_cycles[of ctxt] + apply (auto simp: cond_def msg_exec S'_def ic_inv_def msgs messages_msgs new_call_to_message_def + no_response_def R_Inr call_ctxt_origin_new_ctxt + split: option.splits message.splits call_origin.splits + dest!: list_map_range_setD response_msgsD call_ctxt_needs_to_respond_new_ctxtD + intro: ic_can_status_inv_mono2) + apply blast + apply fast + apply blast + apply fast + apply blast + apply fast + apply blast + apply fast + done + qed +qed + (* System transition: Call context starvation [DONE] *) -definition call_context_starvation_pre :: "'cid \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where +definition call_context_starvation_pre :: "'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "call_context_starvation_pre ctxt_id S = - (case list_map_get (call_contexts S) ctxt_id of Some call_context \ call_ctxt_needs_to_respond call_context \ + (case list_map_get (call_contexts S) ctxt_id of Some call_context \ + call_ctxt_needs_to_respond call_context \ call_ctxt_origin call_context \ From_heartbeat \ (\msg \ set (messages S). case msg of Call_message orig _ _ _ _ _ _ \ calling_context orig \ Some ctxt_id | Response_message orig _ _ \ calling_context orig \ Some ctxt_id | _ \ True) \ - (\other_call_context \ list_map_vals (call_contexts S). + (\other_call_context \ list_map_range (call_contexts S). call_ctxt_needs_to_respond other_call_context \ calling_context (call_ctxt_origin other_call_context) \ Some ctxt_id) | None \ False)" definition call_context_starvation_post :: "'cid \ - ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where "call_context_starvation_post ctxt_id S = ( case list_map_get (call_contexts S) ctxt_id of Some call_context \ let msg = Response_message (call_ctxt_origin call_context) (response.Reject CANISTER_ERROR (encode_string ''starvation'')) (call_ctxt_available_cycles call_context) @@ -720,17 +1266,26 @@ definition call_context_starvation_post :: "'cid \ messages := messages S @ [msg]\)" lemma call_context_starvation_cycles_inv: - "call_context_starvation_pre ctxt_id S \ - total_cycles S = total_cycles (call_context_starvation_post ctxt_id S)" - using list_map_sum_in_ge[where ?f="call_contexts S" and ?x=ctxt_id and ?g=call_ctxt_carried_cycles] + assumes "call_context_starvation_pre ctxt_id S" + shows "total_cycles S = total_cycles (call_context_starvation_post ctxt_id S)" + using assms list_map_sum_in_ge[where ?f="call_contexts S" and ?x=ctxt_id and ?g=call_ctxt_carried_cycles] by (auto simp: call_context_starvation_pre_def call_context_starvation_post_def total_cycles_def call_ctxt_carried_cycles list_map_sum_in[where ?g=call_ctxt_carried_cycles] split: option.splits) +lemma call_context_starvation_ic_inv: + assumes "call_context_starvation_pre ctxt_id S" "ic_inv S" + shows "ic_inv (call_context_starvation_post ctxt_id S)" + using assms + by (auto simp: ic_inv_def call_context_starvation_pre_def call_context_starvation_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range + intro: ic_can_status_inv_mono2) + (* System transition: Call context removal [DONE] *) -definition call_context_removal_pre :: "'cid \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where +definition call_context_removal_pre :: "'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "call_context_removal_pre ctxt_id S = ( (case list_map_get (call_contexts S) ctxt_id of Some call_context \ (\call_ctxt_needs_to_respond call_context \ @@ -742,21 +1297,1517 @@ definition call_context_removal_pre :: "'cid \ ('p, 'uid, 'canid, 'b Call_message ctxt _ _ _ _ _ _ \ calling_context ctxt \ Some ctxt_id | Response_message ctxt _ _ \ calling_context ctxt \ Some ctxt_id | _ \ True) \ - (\other_call_context \ list_map_vals (call_contexts S). + (\other_call_context \ list_map_range (call_contexts S). call_ctxt_needs_to_respond other_call_context \ - calling_context (call_ctxt_origin other_call_context) \ Some ctxt_id) + calling_context (call_ctxt_origin other_call_context) \ Some ctxt_id) \ + (\can_status \ list_map_range (canister_status S). case can_status of Stopping os \ + \(orig, cyc) \ set os. (case orig of From_canister other_ctxt_id _ \ ctxt_id \ other_ctxt_id | _ \ True) + | _ \ True) | None \ False))" definition call_context_removal_post :: "'cid \ - ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'tr, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where "call_context_removal_post ctxt_id S = S\call_contexts := list_map_del (call_contexts S) ctxt_id\" -lemma call_context_removal_cycles_inv: - "call_context_removal_pre ctxt_id S \ - total_cycles S = total_cycles (call_context_removal_post ctxt_id S) + - (case list_map_get (call_contexts S) ctxt_id of Some call_context \ call_ctxt_available_cycles call_context)" - using call_ctxt_inv - by (auto simp: call_context_removal_pre_def call_context_removal_post_def total_cycles_def call_ctxt_carried_cycles list_map_del_sum split: option.splits) +definition call_context_removal_burned_cycles :: "'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "call_context_removal_burned_cycles ctxt_id S = call_ctxt_available_cycles (the (list_map_get (call_contexts S) ctxt_id))" + +lemma call_context_removal_cycles_monotonic: + assumes "call_context_removal_pre ctxt_id S" + shows "total_cycles S = total_cycles (call_context_removal_post ctxt_id S) + call_context_removal_burned_cycles ctxt_id S" + using assms call_ctxt_not_needs_to_respond_available_cycles + by (auto simp: call_context_removal_pre_def call_context_removal_post_def call_context_removal_burned_cycles_def total_cycles_def call_ctxt_carried_cycles list_map_del_sum split: option.splits) + +lemma call_context_removal_ic_inv: + assumes "call_context_removal_pre ctxt_id S" "ic_inv S" + shows "ic_inv (call_context_removal_post ctxt_id S)" + using assms + by (force simp: ic_inv_def call_context_removal_pre_def call_context_removal_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del + intro!: ic_can_status_inv_del[where ?z="list_map_dom (call_contexts S)"]) + + + +(* System transition: IC Management Canister: Canister creation [DONE] *) + +definition ic_canister_creation_pre :: "nat \ 'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_creation_pre n cid t S = (n < length (messages S) \ (case messages S ! n of + Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''create_canister'' \ + parse_candid a \ None \ + is_system_assigned (principal_of_canid cid) \ + cid \ list_map_dom (canisters S) \ + cid \ list_map_dom (time S) \ + cid \ list_map_dom (controllers S) \ + cid \ list_map_dom (balances S) \ + cid \ list_map_dom (certified_data S) \ + cid \ list_map_dom (canister_status S) + | _ \ False))" + +definition ic_canister_creation_post :: "nat \ 'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_creation_post n cid t S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let ctrls = (case candid_parse_controllers a of Some ctrls \ ctrls | _ \ {cer}) in + S\canisters := list_map_set (canisters S) cid None, + time := list_map_set (time S) cid t, + controllers := list_map_set (controllers S) cid ctrls, + freezing_threshold := list_map_set (freezing_threshold S) cid 2592000, + balances := list_map_set (balances S) cid trans_cycles, + certified_data := list_map_set (certified_data S) cid empty_blob, + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid + (Candid_record (list_map_init [(encode_string ''canister_id'', Candid_blob (blob_of_canid cid))])))) 0], + canister_status := list_map_set (canister_status S) cid Running\)" + +lemma ic_canister_creation_cycles_inv: + assumes "ic_canister_creation_pre n cid t S" + shows "total_cycles S = total_cycles (ic_canister_creation_post n cid t S)" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: ic_canister_creation_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_canister_creation_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_canister_creation_pre_def ic_canister_creation_post_def total_cycles_def Let_def msgs + list_map_sum_out[where ?g=id] list_map_sum_out[where ?g=status_cycles] split: message.splits option.splits) +qed + +lemma ic_canister_creation_ic_inv: + assumes "ic_canister_creation_pre n cid t S" "ic_inv S" + shows "ic_inv (ic_canister_creation_post n cid t S)" + using assms + by (auto simp: ic_inv_def ic_canister_creation_pre_def ic_canister_creation_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + intro: ic_can_status_inv_mono1) + + + +(* System transition: IC Management Canister: Changing settings [DONE] *) + +definition ic_update_settings_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_update_settings_pre n S = (n < length (messages S) \ (case messages S ! n of + Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''update_settings'' \ + (case candid_parse_cid a of Some cid \ + (case list_map_get (controllers S) cid of Some ctrls \ cer \ ctrls + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_update_settings_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_update_settings_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a); + ctrls = (case candid_parse_controllers a of Some ctrls \ list_map_set (controllers S) cid ctrls | _ \ controllers S); + freezing_thres = (case candid_nested_lookup a [encode_string '''settings'', encode_string ''freezing_threshold''] of Some (Candid_nat freeze) \ + list_map_set (freezing_threshold S) cid freeze | _ \ freezing_threshold S) in + S\controllers := ctrls, freezing_threshold := freezing_thres, messages := take n (messages S) @ drop (Suc n) (messages S) @ + [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)" + +lemma ic_update_settings_cycles_inv: + assumes "ic_update_settings_pre n S" + shows "total_cycles S = total_cycles (ic_update_settings_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: ic_update_settings_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_update_settings_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_update_settings_pre_def ic_update_settings_post_def total_cycles_def Let_def msgs split: message.splits option.splits) +qed + +lemma ic_update_settings_ic_inv: + assumes "ic_update_settings_pre n S" "ic_inv S" + shows "ic_inv (ic_update_settings_post n S)" + using assms + by (auto simp: ic_inv_def ic_update_settings_pre_def ic_update_settings_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del) + + + +(* System transition: IC Management Canister: Canister status [DONE] *) + +definition ic_canister_status_pre :: "nat \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_status_pre n m S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''canister_status'' \ + (case candid_parse_cid a of Some cid \ + cid \ list_map_dom (canisters S) \ + cid \ list_map_dom (canister_status S) \ + cid \ list_map_dom (balances S) \ + cid \ list_map_dom (freezing_threshold S) \ + (case list_map_get (controllers S) cid of Some ctrls \ cer \ ctrls | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_canister_status_post :: "nat \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_status_post n m S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a); + hash = (case the (list_map_get (canisters S) cid) of None \ Candid_null + | Some can \ Candid_opt (Candid_blob (sha_256 (raw_module can)))) in + S\messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid + (Candid_record (list_map_init [(encode_string ''status'', candid_of_status (simple_status (the (list_map_get (canister_status S) cid)))), + (encode_string ''module_hash'', hash), + (encode_string ''controllers'', Candid_vec (map (Candid_blob \ blob_of_principal) (principal_list_of_set (the (list_map_get (controllers S) cid))))), + (encode_string ''memory_size'', Candid_nat m), + (encode_string ''cycles'', Candid_nat (the (list_map_get (balances S) cid))), + (encode_string ''freezing_threshold'', Candid_nat (the (list_map_get (freezing_threshold S) cid))), + (encode_string ''idle_cycles_burned_per_day'', Candid_nat (ic_idle_cycles_burned_rate S cid))])))) trans_cycles]\)" + +lemma ic_canister_status_cycles_inv: + assumes "ic_canister_status_pre n m S" + shows "total_cycles S = total_cycles (ic_canister_status_post n m S)" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: ic_canister_status_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_canister_status_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_canister_status_pre_def ic_canister_status_post_def total_cycles_def Let_def msgs split: message.splits option.splits) +qed + +lemma ic_canister_status_ic_inv: + assumes "ic_canister_status_pre n m S" "ic_inv S" + shows "ic_inv (ic_canister_status_post n m S)" + using assms + by (auto simp: ic_inv_def ic_canister_status_pre_def ic_canister_status_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del) + + + +(* System transition: IC Management Canister: Code installation [DONE] *) + +definition ic_code_installation_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_code_installation_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''install_code'' \ + (case candid_parse_cid a of Some cid \ + (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of + (Some mode, Some w, Some ar) \ + (case parse_wasm_mod w of Some m \ + parse_public_custom_sections w \ None \ + parse_private_custom_sections w \ None \ + (case (list_map_get (controllers S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of + (Some ctrls, Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + ((mode = encode_string ''install'' \ (case list_map_get (canisters S) cid of Some None \ True | _ \ False)) \ mode = encode_string ''reinstall'') \ + cer \ ctrls \ + (case canister_module_init m (cid, ar, cer, env) of Inl _ \ False + | Inr ret \ cycles_return.cycles_used ret \ bal) \ + list_map_dom (canister_module_update_methods m) \ list_map_dom (canister_module_query_methods m) = {} + | _ \ False) | _ \ False) | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_code_installation_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_code_installation_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (case candid_parse_cid a of Some cid \ + (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of + (Some mode, Some w, Some a) \ + (case parse_wasm_mod w of Some m \ + (case (list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of + (Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\; + (new_state, cyc_used) = (case canister_module_init m (cid, a, cer, env) of Inr ret \ (cycles_return.return ret, cycles_return.cycles_used ret)) in + S\canisters := list_map_set (canisters S) cid (Some \wasm_state = new_state, module = m, raw_module = w, + public_custom_sections = the (parse_public_custom_sections w), private_custom_sections = the (parse_private_custom_sections w)\), + balances := list_map_set (balances S) cid (bal - cyc_used), + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)))))" + +definition ic_code_installation_burned_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "ic_code_installation_burned_cycles n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (case candid_parse_cid a of Some cid \ + (case (candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of + (Some w, Some a) \ + (case parse_wasm_mod w of Some m \ + (case (list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of + (Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + (case canister_module_init m (cid, a, cer, env) of Inr ret \ cycles_return.cycles_used ret))))))" + +lemma ic_code_installation_cycles_inv: + assumes "ic_code_installation_pre n S" + shows "total_cycles S = total_cycles (ic_code_installation_post n S) + ic_code_installation_burned_cycles n S" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: ic_code_installation_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_code_installation_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms list_map_sum_in_ge[where ?f="balances S" and ?g=id and ?x="the (candid_parse_cid a)"] + by (auto simp: ic_code_installation_pre_def ic_code_installation_post_def ic_code_installation_burned_cycles_def total_cycles_def Let_def msgs list_map_sum_in[where ?f="balances S"] split: message.splits option.splits sum.splits prod.splits) +qed + +lemma ic_code_installation_ic_inv: + assumes "ic_code_installation_pre n S" "ic_inv S" + shows "ic_inv (ic_code_installation_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid mode w ar m ctrls t bal can_status where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and parse: "candid_parse_cid a = Some cid" + "candid_parse_text a [encode_string ''mode''] = Some mode" + "candid_parse_blob a [encode_string ''wasm_module''] = Some w" + "candid_parse_blob a [encode_string ''arg''] = Some ar" + "parse_wasm_mod w = Some m" + and list_map_get: + "list_map_get (controllers S) cid = Some ctrls" + "list_map_get (time S) cid = Some t" + "list_map_get (balances S) cid = Some bal" + "list_map_get (canister_status S) cid = Some can_status" + using assms + by (auto simp: ic_code_installation_pre_def split: message.splits option.splits) + show ?thesis + using assms + by (auto simp: ic_inv_def ic_code_installation_pre_def ic_code_installation_post_def msg parse list_map_get Let_def + split: sum.splits call_origin.splits message.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del) +qed + + + +(* System transition: IC Management Canister: Code upgrade [DONE] *) + +definition ic_code_upgrade_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_code_upgrade_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''install_code'' \ + (case candid_parse_cid a of Some cid \ + (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of + (Some mode, Some w, Some ar) \ + (case parse_wasm_mod w of Some m \ + parse_public_custom_sections w \ None \ + parse_private_custom_sections w \ None \ + (case (list_map_get (canisters S) cid, list_map_get (controllers S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of + (Some (Some can), Some ctrls, Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + mode = encode_string ''upgrade'' \ + cer \ ctrls \ + (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env) of Inr pre_ret \ + (case canister_module_post_upgrade m (cid, cycles_return.return pre_ret, ar, cer, env) of Inr post_ret \ + cycles_return.cycles_used pre_ret + cycles_return.cycles_used post_ret \ bal + | _ \ False) | _ \ False) \ + list_map_dom (canister_module_update_methods m) \ list_map_dom (canister_module_query_methods m) = {} + | _ \ False) | _ \ False) | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_code_upgrade_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_code_upgrade_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (case candid_parse_cid a of Some cid \ + (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of + (Some mode, Some w, Some ar) \ + (case parse_wasm_mod w of Some m \ + (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of + (Some (Some can), Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env) of Inr pre_ret \ + (case canister_module_post_upgrade m (cid, cycles_return.return pre_ret, ar, cer, env) of Inr post_ret \ + S\canisters := list_map_set (canisters S) cid (Some \wasm_state = cycles_return.return post_ret, module = m, raw_module = w, + public_custom_sections = the (parse_public_custom_sections w), private_custom_sections = the (parse_private_custom_sections w)\), + balances := list_map_set (balances S) cid (bal - (cycles_return.cycles_used pre_ret + cycles_return.cycles_used post_ret)), + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)))))))" + +definition ic_code_upgrade_burned_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "ic_code_upgrade_burned_cycles n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (case candid_parse_cid a of Some cid \ + (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of + (Some mode, Some w, Some ar) \ + (case parse_wasm_mod w of Some m \ + (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of + (Some (Some can), Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env) of Inr pre_ret \ + (case canister_module_post_upgrade m (cid, cycles_return.return pre_ret, ar, cer, env) of Inr post_ret \ + cycles_return.cycles_used pre_ret + cycles_return.cycles_used post_ret)))))))" + +lemma ic_code_upgrade_cycles_inv: + assumes "ic_code_upgrade_pre n S" + shows "total_cycles S = total_cycles (ic_code_upgrade_post n S) + ic_code_upgrade_burned_cycles n S" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: ic_code_upgrade_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_code_upgrade_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms list_map_sum_in_ge[where ?f="balances S" and ?g=id and ?x="the (candid_parse_cid a)"] + by (auto simp: ic_code_upgrade_pre_def ic_code_upgrade_post_def ic_code_upgrade_burned_cycles_def total_cycles_def Let_def msgs list_map_sum_in[where ?f="balances S"] split: message.splits option.splits sum.splits) +qed + +lemma ic_code_upgrade_ic_inv: + assumes "ic_code_upgrade_pre n S" "ic_inv S" + shows "ic_inv (ic_code_upgrade_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid mode w ar m can ctrls t bal can_status where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and parse: "candid_parse_cid a = Some cid" + "candid_parse_text a [encode_string ''mode''] = Some mode" + "candid_parse_blob a [encode_string ''wasm_module''] = Some w" + "candid_parse_blob a [encode_string ''arg''] = Some ar" + "parse_wasm_mod w = Some m" + and list_map_get: + "list_map_get (canisters S) cid = Some (Some can)" + "list_map_get (controllers S) cid = Some ctrls" + "list_map_get (time S) cid = Some t" + "list_map_get (balances S) cid = Some bal" + "list_map_get (canister_status S) cid = Some can_status" + using assms + by (auto simp: ic_code_upgrade_pre_def split: message.splits option.splits) + show ?thesis + using assms + by (auto simp: ic_inv_def ic_code_upgrade_pre_def ic_code_upgrade_post_def msg parse list_map_get Let_def + split: sum.splits call_origin.splits message.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del) +qed + + + +(* System transition: IC Management Canister: Code uninstallation [DONE] *) + +definition ic_code_uninstallation_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_code_uninstallation_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''uninstall_code'' \ + (case candid_parse_cid a of Some cid \ + (case list_map_get (controllers S) cid of Some ctrls \ cer \ ctrls + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_code_uninstallation_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_code_uninstallation_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (case candid_parse_cid a of Some cid \ + let call_ctxt_to_msg = (\ctxt. + if call_ctxt_canister ctxt = cid \ call_ctxt_needs_to_respond ctxt then + Some (Response_message (call_ctxt_origin ctxt) (response.Reject CANISTER_REJECT (encode_string ''Canister has been uninstalled'')) (call_ctxt_available_cycles ctxt)) + else None); + call_ctxt_to_ctxt = (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt) in + S\canisters := list_map_set (canisters S) cid None, certified_data := list_map_set (certified_data S) cid empty_blob, + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles] @ + List.map_filter call_ctxt_to_msg (list_map_vals (call_contexts S)), + call_contexts := list_map_map call_ctxt_to_ctxt (call_contexts S)\))" + +lemma ic_code_uninstallation_cycles_inv: + assumes "ic_code_uninstallation_pre n S" + shows "total_cycles S = total_cycles (ic_code_uninstallation_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_code_uninstallation_pre_def split: message.splits option.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_code_uninstallation_pre_def msg younger_def older_def nth_append) + have F1: "list_map_sum_vals call_ctxt_carried_cycles (call_contexts S) = + list_map_sum_vals id + (list_map_map (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_carried_cycles ctxt else 0) (call_contexts S)) + + list_map_sum_vals call_ctxt_carried_cycles + (list_map_map (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt) (call_contexts S))" + using list_map_sum_vals_split[where ?f="call_ctxt_carried_cycles" and ?g="call_ctxt_delete", unfolded call_ctxt_delete_carried_cycles diff_zero] + by auto + show ?thesis + using assms + by (auto simp: ic_code_uninstallation_pre_def ic_code_uninstallation_post_def total_cycles_def call_ctxt_carried_cycles cid_def Let_def msgs F1 + split: message.splits option.splits sum.splits if_splits intro!: list_map_sum_vals_filter) +qed + +lemma ic_code_uninstallation_ic_inv: + assumes "ic_code_uninstallation_pre n S" "ic_inv S" + shows "ic_inv (ic_code_uninstallation_post n S)" + using assms + by (auto simp: ic_inv_def ic_code_uninstallation_pre_def ic_code_uninstallation_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: IC Management Canister: Stopping a canister (running) [DONE] *) + +definition ic_canister_stop_running_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_stop_running_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''stop_canister'' \ + (case candid_parse_cid a of Some cid \ + (case (list_map_get (canister_status S) cid, list_map_get (controllers S) cid) of (Some Running, Some ctrls) \ + cer \ ctrls + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_canister_stop_running_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_stop_running_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a) in + S\messages := take n (messages S) @ drop (Suc n) (messages S), + canister_status := list_map_set (canister_status S) cid (Stopping [(orig, trans_cycles)])\)" + +lemma ic_canister_stop_running_cycles_inv: + assumes "ic_canister_stop_running_pre n S" + shows "total_cycles S = total_cycles (ic_canister_stop_running_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_canister_stop_running_pre_def split: message.splits option.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_canister_stop_running_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_canister_stop_running_pre_def ic_canister_stop_running_post_def total_cycles_def call_ctxt_carried_cycles cid_def Let_def msgs + list_map_sum_in[where ?g=status_cycles] split: message.splits option.splits sum.splits can_status.splits) +qed + +lemma ic_canister_stop_running_ic_inv: + assumes "ic_canister_stop_running_pre n S" "ic_inv S" + shows "ic_inv (ic_canister_stop_running_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_canister_stop_running_pre_def split: message.splits option.splits) + show ?thesis + using assms + by (force simp: ic_inv_def ic_canister_stop_running_pre_def ic_canister_stop_running_post_def msg Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals intro!: ic_can_status_inv_stopping[where ?x="list_map_range (canister_status S)" and ?os=orig]) +qed + + + +(* System transition: IC Management Canister: Stopping a canister (stopping) [DONE] *) + +definition ic_canister_stop_stopping_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_stop_stopping_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''stop_canister'' \ + (case candid_parse_cid a of Some cid \ + (case (list_map_get (canister_status S) cid, list_map_get (controllers S) cid) of (Some (Stopping os), Some ctrls) \ + cer \ ctrls + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_canister_stop_stopping_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_stop_stopping_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a) in + (case list_map_get (canister_status S) cid of Some (Stopping os) \ + S\messages := take n (messages S) @ drop (Suc n) (messages S), + canister_status := list_map_set (canister_status S) cid (Stopping (os @ [(orig, trans_cycles)]))\))" + +lemma ic_canister_stop_stopping_cycles_inv: + assumes "ic_canister_stop_stopping_pre n S" + shows "total_cycles S = total_cycles (ic_canister_stop_stopping_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_canister_stop_stopping_pre_def split: message.splits option.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_canister_stop_stopping_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_canister_stop_stopping_pre_def ic_canister_stop_stopping_post_def total_cycles_def call_ctxt_carried_cycles cid_def Let_def msgs + list_map_sum_in[where ?g=status_cycles] split: message.splits option.splits sum.splits can_status.splits) +qed + +lemma ic_canister_stop_stopping_ic_inv: + assumes "ic_canister_stop_stopping_pre n S" "ic_inv S" + shows "ic_inv (ic_canister_stop_stopping_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_canister_stop_stopping_pre_def split: message.splits option.splits) + show ?thesis + using assms + by (force simp: ic_inv_def ic_canister_stop_stopping_pre_def ic_canister_stop_stopping_post_def msg Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals intro!: ic_can_status_inv_stopping_app[where ?x="list_map_range (canister_status S)" and ?os=orig]) +qed + + + +(* System transition: IC Management Canister: Stopping a canister (done stopping) [DONE] *) + +definition ic_canister_stop_done_stopping_pre :: "'canid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_stop_done_stopping_pre cid S = + (case list_map_get (canister_status S) cid of Some (Stopping os) \ + (\ctxt \ list_map_range (call_contexts S). call_ctxt_canister ctxt \ cid) + | _ \ False)" + +definition ic_canister_stop_done_stopping_post :: "'canid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_stop_done_stopping_post cid S = ( + let orig_cycles_to_msg = (\(or, cyc). Response_message or (Reply (blob_of_candid Candid_empty)) cyc) in + (case list_map_get (canister_status S) cid of Some (Stopping os) \ + S\canister_status := list_map_set (canister_status S) cid Stopped, + messages := messages S @ map orig_cycles_to_msg os\))" + +lemma ic_canister_stop_done_stopping_cycles_inv: + assumes "ic_canister_stop_done_stopping_pre cid S" + shows "total_cycles S = total_cycles (ic_canister_stop_done_stopping_post cid S)" +proof - + have F1: "(message_cycles \ (\(or, y). Response_message or (Reply (blob_of_candid Candid_empty)) y)) = (\(or, y). carried_cycles or + y)" + by auto + have F2: "(\(or, y)\xs. carried_cycles or + y) = sum_list (map (carried_cycles \ fst) xs) + sum_list (map snd xs)" for xs + by (induction xs) auto + show ?thesis + using assms list_map_sum_in_ge[where ?g=status_cycles and ?f="canister_status S" and ?x=cid] + by (auto simp: ic_canister_stop_done_stopping_pre_def ic_canister_stop_done_stopping_post_def total_cycles_def call_ctxt_carried_cycles Let_def + F1 F2 list_map_sum_in[where ?g=status_cycles] split: message.splits option.splits sum.splits can_status.splits) +qed + +lemma ic_canister_stop_done_stopping_ic_inv: + assumes "ic_canister_stop_done_stopping_pre cid S" "ic_inv S" + shows "ic_inv (ic_canister_stop_done_stopping_post cid S)" + using assms + by (force simp: ic_inv_def ic_can_status_inv_def ic_canister_stop_done_stopping_pre_def ic_canister_stop_done_stopping_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: IC Management Canister: Stopping a canister (stopped) [DONE] *) + +definition ic_canister_stop_stopped_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_stop_stopped_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''stop_canister'' \ + (case candid_parse_cid a of Some cid \ + (case (list_map_get (canister_status S) cid, list_map_get (controllers S) cid) of (Some Stopped, Some ctrls) \ + cer \ ctrls + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_canister_stop_stopped_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_stop_stopped_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + S\messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)" + +lemma ic_canister_stop_stopped_cycles_inv: + assumes "ic_canister_stop_stopped_pre n S" + shows "total_cycles S = total_cycles (ic_canister_stop_stopped_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_canister_stop_stopped_pre_def split: message.splits option.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_canister_stop_stopped_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_canister_stop_stopped_pre_def ic_canister_stop_stopped_post_def total_cycles_def call_ctxt_carried_cycles cid_def Let_def msgs + split: message.splits option.splits sum.splits can_status.splits) +qed + +lemma ic_canister_stop_stopped_ic_inv: + assumes "ic_canister_stop_stopped_pre n S" "ic_inv S" + shows "ic_inv (ic_canister_stop_stopped_post n S)" + using assms + by (auto simp: ic_inv_def ic_canister_stop_stopped_pre_def ic_canister_stop_stopped_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: IC Management Canister: Starting a canister (not stopping) [DONE] *) + +definition ic_canister_start_not_stopping_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_start_not_stopping_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''start_canister'' \ + (case candid_parse_cid a of Some cid \ + (case (list_map_get (canister_status S) cid, list_map_get (controllers S) cid) of (Some can_status, Some ctrls) \ + (can_status = Running \ can_status = Stopped) \ + cer \ ctrls + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_canister_start_not_stopping_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_start_not_stopping_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a) in + S\canister_status := list_map_set (canister_status S) cid Running, + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)" + +lemma ic_canister_start_not_stopping_cycles_inv: + assumes "ic_canister_start_not_stopping_pre n S" + shows "total_cycles S = total_cycles (ic_canister_start_not_stopping_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_canister_start_not_stopping_pre_def split: message.splits option.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_canister_start_not_stopping_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_canister_start_not_stopping_pre_def ic_canister_start_not_stopping_post_def total_cycles_def call_ctxt_carried_cycles cid_def Let_def msgs + list_map_sum_in[where ?g=status_cycles] split: message.splits option.splits sum.splits) +qed + +lemma ic_canister_start_not_stopping_ic_inv: + assumes "ic_canister_start_not_stopping_pre n S" "ic_inv S" + shows "ic_inv (ic_canister_start_not_stopping_post n S)" + using assms + by (auto simp: ic_inv_def ic_canister_start_not_stopping_pre_def ic_canister_start_not_stopping_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals intro: ic_can_status_inv_mono1) + + + +(* System transition: IC Management Canister: Starting a canister (stopping) [DONE] *) + +definition ic_canister_start_stopping_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_start_stopping_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''start_canister'' \ + (case candid_parse_cid a of Some cid \ + (case (list_map_get (canister_status S) cid, list_map_get (controllers S) cid) of (Some (Stopping _), Some ctrls) \ + cer \ ctrls + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_canister_start_stopping_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_start_stopping_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a); + orig_cycles_to_msg = (\(or, cyc). Response_message or (response.Reject CANISTER_REJECT (encode_string ''Canister has been restarted'')) cyc) in + (case list_map_get (canister_status S) cid of Some (Stopping os) \ + S\canister_status := list_map_set (canister_status S) cid Running, + messages := take n (messages S) @ drop (Suc n) (messages S) @ Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles # + map orig_cycles_to_msg os\))" + +lemma ic_canister_start_stopping_cycles_inv: + assumes "ic_canister_start_stopping_pre n S" + shows "total_cycles S = total_cycles (ic_canister_start_stopping_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_canister_start_stopping_pre_def split: message.splits option.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_canister_start_stopping_pre_def msg younger_def older_def nth_append) + have F1: "(message_cycles \ (\(or, y). Response_message or (response.Reject CANISTER_REJECT (encode_string ''Canister has been restarted'')) y)) = (\(or, y). carried_cycles or + y)" + by auto + have F2: "(\(or, y)\xs. carried_cycles or + y) = sum_list (map (carried_cycles \ fst) xs) + sum_list (map snd xs)" for xs + by (induction xs) auto + show ?thesis + using assms list_map_sum_in_ge[where ?g=status_cycles and ?f="canister_status S" and ?x=cid] + by (auto simp: ic_canister_start_stopping_pre_def ic_canister_start_stopping_post_def total_cycles_def call_ctxt_carried_cycles cid_def Let_def msgs F1 F2 + list_map_sum_in[where ?g=status_cycles and ?f="canister_status S"] + split: message.splits option.splits sum.splits can_status.splits) +qed + +lemma ic_canister_start_stopping_ic_inv: + assumes "ic_canister_start_stopping_pre n S" "ic_inv S" + shows "ic_inv (ic_canister_start_stopping_post n S)" + using assms + apply (auto simp: ic_inv_def ic_canister_start_stopping_pre_def ic_canister_start_stopping_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del in_set_map_filter_vals + intro: ic_can_status_inv_mono1) + apply (fastforce simp: ic_can_status_inv_def)+ + done + + + +(* System transition: IC Management Canister: Canister deletion [DONE] *) + +definition ic_canister_deletion_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_deletion_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''delete_canister'' \ + (case candid_parse_cid a of Some cid \ + (case (list_map_get (canister_status S) cid, list_map_get (controllers S) cid, list_map_get (balances S) cid) of (Some Stopped, Some ctrls, Some bal) \ + cer \ ctrls + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_canister_deletion_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_deletion_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a) in + S\canisters := list_map_del (canisters S) cid, + controllers := list_map_del (controllers S) cid, + freezing_threshold := list_map_del (freezing_threshold S) cid, + canister_status := list_map_del (canister_status S) cid, + time := list_map_del (time S) cid, + balances := list_map_del (balances S) cid, + certified_data := list_map_del (certified_data S) cid, + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)" + +definition ic_canister_deletion_burned_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "ic_canister_deletion_burned_cycles n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a) in the (list_map_get (balances S) cid))" + +lemma ic_canister_deletion_cycles_monotonic: + assumes "ic_canister_deletion_pre n S" + shows "total_cycles S = total_cycles (ic_canister_deletion_post n S) + ic_canister_deletion_burned_cycles n S" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_canister_deletion_pre_def split: message.splits option.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_canister_deletion_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_canister_deletion_pre_def ic_canister_deletion_post_def ic_canister_deletion_burned_cycles_def total_cycles_def call_ctxt_carried_cycles cid_def Let_def msgs + list_map_del_sum[where ?g=id and ?f="balances S"] list_map_del_sum[where ?g=status_cycles and ?f="canister_status S"] + split: message.splits option.splits sum.splits can_status.splits) +qed + +lemma ic_canister_deletion_ic_inv: + assumes "ic_canister_deletion_pre n S" "ic_inv S" + shows "ic_inv (ic_canister_deletion_post n S)" + using assms + by (auto simp: ic_inv_def ic_canister_deletion_pre_def ic_canister_deletion_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del in_set_map_filter_vals + intro: ic_can_status_inv_mono1) + + + +(* System transition: IC Management Canister: Depositing cycles [DONE] *) + +definition ic_depositing_cycles_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_depositing_cycles_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''deposit_cycles'' \ + (case candid_parse_cid a of Some cid \ + (case list_map_get (balances S) cid of Some bal \ + True + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_depositing_cycles_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_depositing_cycles_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a) in + (case list_map_get (balances S) cid of Some bal \ + S\balances := list_map_set (balances S) cid (bal + trans_cycles), + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) 0]\))" + +lemma ic_depositing_cycles_cycles_monotonic: + assumes "ic_depositing_cycles_pre n S" + shows "total_cycles S = total_cycles (ic_depositing_cycles_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_depositing_cycles_pre_def split: message.splits option.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_depositing_cycles_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms list_map_sum_in_ge[where ?g=id and ?f="balances S" and ?x=cid] + by (auto simp: ic_depositing_cycles_pre_def ic_depositing_cycles_post_def total_cycles_def call_ctxt_carried_cycles cid_def Let_def msgs + list_map_sum_in[where ?g=id and ?f="balances S"] min_def split: message.splits option.splits sum.splits) +qed + +lemma ic_depositing_cycles_ic_inv: + assumes "ic_depositing_cycles_pre n S" "ic_inv S" + shows "ic_inv (ic_depositing_cycles_post n S)" + using assms + by (auto simp: ic_inv_def ic_depositing_cycles_pre_def ic_depositing_cycles_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: IC Management Canister: Random numbers [DONE] *) + +definition ic_random_numbers_pre :: "nat \ 'b \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_random_numbers_pre n b S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''raw_rand'' \ + a = blob_of_candid Candid_empty \ + blob_length b = 32 + | _ \ False))" + +definition ic_random_numbers_post :: "nat \ 'b \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_random_numbers_post n b S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + S\messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid (Candid_blob b))) trans_cycles]\)" + +lemma ic_random_numbers_cycles_inv: + assumes "ic_random_numbers_pre n b S" + shows "total_cycles S = total_cycles (ic_random_numbers_post n b S)" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: ic_random_numbers_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_random_numbers_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_random_numbers_pre_def ic_random_numbers_post_def total_cycles_def call_ctxt_carried_cycles Let_def msgs) +qed + +lemma ic_random_numbers_ic_inv: + assumes "ic_random_numbers_pre n b S" "ic_inv S" + shows "ic_inv (ic_random_numbers_post n b S)" + using assms + by (auto simp: ic_inv_def ic_random_numbers_pre_def ic_random_numbers_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: IC Management Canister: Canister creation with cycles [DONE] *) + +definition ic_provisional_canister_creation_pre :: "nat \ 'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_provisional_canister_creation_pre n cid t S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (case candid_parse_nat a [encode_string ''amount''] of Some cyc \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''provisional_create_canister_with_cycles'' \ + is_system_assigned (principal_of_canid cid) \ + cid \ list_map_dom (canisters S) \ + cid \ list_map_dom (time S) \ + cid \ list_map_dom (controllers S) \ + cid \ list_map_dom (balances S) \ + cid \ list_map_dom (certified_data S) \ + cid \ list_map_dom (canister_status S) + | _ \ False) | _ \ False))" + +definition ic_provisional_canister_creation_post :: "nat \ 'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_provisional_canister_creation_post n cid t S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cyc = the (candid_parse_nat a [encode_string ''amount'']) in + S\canisters := list_map_set (canisters S) cid None, + time := list_map_set (time S) cid t, + controllers := list_map_set (controllers S) cid {cer}, + freezing_threshold := list_map_set (freezing_threshold S) cid 2592000, + balances := list_map_set (balances S) cid cyc, + certified_data := list_map_set (certified_data S) cid empty_blob, + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid + (Candid_record (list_map_init [(encode_string ''canister_id'', Candid_blob (blob_of_canid cid))])))) trans_cycles], + canister_status := list_map_set (canister_status S) cid Running\)" + +definition ic_provisional_canister_creation_minted_cycles :: "nat \ 'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "ic_provisional_canister_creation_minted_cycles n cid t S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + the (candid_parse_nat a [encode_string ''amount'']))" + +lemma ic_provisional_canister_creation_cycles_antimonotonic: + assumes "ic_provisional_canister_creation_pre n cid t S" + shows "total_cycles S + ic_provisional_canister_creation_minted_cycles n cid t S = total_cycles (ic_provisional_canister_creation_post n cid t S)" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: ic_provisional_canister_creation_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_provisional_canister_creation_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_provisional_canister_creation_pre_def ic_provisional_canister_creation_post_def ic_provisional_canister_creation_minted_cycles_def total_cycles_def Let_def msgs + list_map_sum_out[where ?g=id] list_map_sum_out[where ?g=status_cycles] split: message.splits option.splits) +qed + +lemma ic_provisional_canister_creation_ic_inv: + assumes "ic_provisional_canister_creation_pre n cid t S" "ic_inv S" + shows "ic_inv (ic_provisional_canister_creation_post n cid t S)" + using assms + by (auto simp: ic_inv_def ic_provisional_canister_creation_pre_def ic_provisional_canister_creation_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del in_set_map_filter_vals + intro: ic_can_status_inv_mono1) + + + +(* System transition: IC Management Canister: Top up canister [DONE] *) + +definition ic_top_up_canister_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_top_up_canister_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (case (candid_parse_cid a, candid_parse_nat a [encode_string ''amount'']) of (Some cid, Some cyc) \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''provisional_top_up_canister'' \ + cid \ list_map_dom (balances S) + | _ \ False) | _ \ False))" + +definition ic_top_up_canister_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_top_up_canister_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a); + cyc = the (candid_parse_nat a [encode_string ''amount'']); + bal = the (list_map_get (balances S) cid) in + S\balances := list_map_set (balances S) cid (bal + cyc)\)" + +definition ic_top_up_canister_minted_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "ic_top_up_canister_minted_cycles n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + the (candid_parse_nat a [encode_string ''amount'']))" + +lemma ic_top_up_canister_cycles_antimonotonic: + assumes "ic_top_up_canister_pre n S" + shows "total_cycles S + ic_top_up_canister_minted_cycles n S = total_cycles (ic_top_up_canister_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: ic_top_up_canister_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_top_up_canister_pre_def msg younger_def older_def nth_append) + define cid where "cid = the (candid_parse_cid a)" + show ?thesis + using assms + by (cases "list_map_get (balances S) cid") + (auto simp: ic_top_up_canister_pre_def ic_top_up_canister_post_def ic_top_up_canister_minted_cycles_def total_cycles_def Let_def msgs cid_def + list_map_sum_in[where ?g=id] split: message.splits option.splits) +qed + +lemma ic_top_up_canister_ic_inv: + assumes "ic_top_up_canister_pre n S" "ic_inv S" + shows "ic_inv (ic_top_up_canister_post n S)" + using assms + by (auto simp: ic_inv_def ic_top_up_canister_pre_def ic_top_up_canister_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: Callback invocation (not deleted) [DONE] *) + +definition callback_invocation_not_deleted_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "callback_invocation_not_deleted_pre n S = (n < length (messages S) \ (case messages S ! n of Response_message (From_canister ctxt_id c) resp ref_cycles \ + (case list_map_get (call_contexts S) ctxt_id of Some ctxt \ + let cid = call_ctxt_canister ctxt in + \call_ctxt_deleted ctxt \ + (case list_map_get (balances S) cid of Some bal \ True + | _ \ False) + | _ \ False) + | _ \ False))" + +definition callback_invocation_not_deleted_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "callback_invocation_not_deleted_post n S = (case messages S ! n of Response_message (From_canister ctxt_id c) resp ref_cycles \ + (case list_map_get (call_contexts S) ctxt_id of Some ctxt \ + let cid = call_ctxt_canister ctxt in + (case list_map_get (balances S) cid of Some bal \ + S\balances := list_map_set (balances S) cid (bal + ref_cycles), + messages := list_update (messages S) n (Func_message ctxt_id cid (Callback c resp ref_cycles) Unordered)\)))" + +lemma callback_invocation_not_deleted_cycles_inv: + assumes "callback_invocation_not_deleted_pre n S" + shows "total_cycles S = total_cycles (callback_invocation_not_deleted_post n S)" +proof - + obtain ctxt_id c resp ref_cycles where msg: "messages S ! n = Response_message (From_canister ctxt_id c) resp ref_cycles" + using assms + by (auto simp: callback_invocation_not_deleted_pre_def split: message.splits option.splits call_origin.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Response_message (From_canister ctxt_id c) resp ref_cycles # younger" "(older @ w # younger) ! n = w" + and msgs_upd: "(older @ Response_message (From_canister ctxt_id c) resp ref_cycles # younger)[n := m] = older @ m # younger" for w m + using assms id_take_nth_drop[of n "messages S"] upd_conv_take_nth_drop[of n "messages S"] assms + by (auto simp: callback_invocation_not_deleted_pre_def msg older_def younger_def nth_append) + show ?thesis + using assms + by (auto simp: callback_invocation_not_deleted_pre_def callback_invocation_not_deleted_post_def total_cycles_def call_ctxt_carried_cycles Let_def msgs msgs_upd + list_map_sum_in[where ?g=id and ?f="balances S"] split: option.splits) +qed + +lemma callback_invocation_not_deleted_ic_inv: + assumes "callback_invocation_not_deleted_pre n S" "ic_inv S" + shows "ic_inv (callback_invocation_not_deleted_post n S)" + using assms + by (auto simp: ic_inv_def callback_invocation_not_deleted_pre_def callback_invocation_not_deleted_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD) + + + +(* System transition: Callback invocation (deleted) [DONE] *) + +definition callback_invocation_deleted_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "callback_invocation_deleted_pre n S = (n < length (messages S) \ (case messages S ! n of Response_message (From_canister ctxt_id c) resp ref_cycles \ + (case list_map_get (call_contexts S) ctxt_id of Some ctxt \ + let cid = call_ctxt_canister ctxt in + call_ctxt_deleted ctxt \ + (case list_map_get (balances S) cid of Some bal \ True + | _ \ False) + | _ \ False) + | _ \ False))" + +definition callback_invocation_deleted_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "callback_invocation_deleted_post n S = (case messages S ! n of Response_message (From_canister ctxt_id c) resp ref_cycles \ + (case list_map_get (call_contexts S) ctxt_id of Some ctxt \ + let cid = call_ctxt_canister ctxt in + (case list_map_get (balances S) cid of Some bal \ + S\balances := list_map_set (balances S) cid (bal + ref_cycles + MAX_CYCLES_PER_RESPONSE), + messages := take n (messages S) @ drop (Suc n) (messages S)\)))" + +lemma callback_invocation_deleted_cycles_inv: + assumes "callback_invocation_deleted_pre n S" + shows "total_cycles S = total_cycles (callback_invocation_deleted_post n S)" +proof - + obtain ctxt_id c resp ref_cycles where msg: "messages S ! n = Response_message (From_canister ctxt_id c) resp ref_cycles" + using assms + by (auto simp: callback_invocation_deleted_pre_def split: message.splits option.splits call_origin.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Response_message (From_canister ctxt_id c) resp ref_cycles # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: callback_invocation_deleted_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: callback_invocation_deleted_pre_def callback_invocation_deleted_post_def total_cycles_def call_ctxt_carried_cycles Let_def msgs + list_map_sum_in[where ?g=id and ?f="balances S"] split: option.splits) +qed + +lemma callback_invocation_deleted_ic_inv: + assumes "callback_invocation_deleted_pre n S" "ic_inv S" + shows "ic_inv (callback_invocation_deleted_post n S)" + using assms + by (auto simp: ic_inv_def callback_invocation_deleted_pre_def callback_invocation_deleted_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD) + + + +(* System transition: Respond to user request [DONE] *) + +definition respond_to_user_request_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "respond_to_user_request_pre n S = (n < length (messages S) \ (case messages S ! n of Response_message (From_user req) resp ref_cycles \ + (case list_map_get (requests S) req of Some Processing \ True + | _ \ False) + | _ \ False))" + +definition respond_to_user_request_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "respond_to_user_request_post n S = (case messages S ! n of Response_message (From_user req) resp ref_cycles \ + let req_resp = (case resp of Reply b \ Replied b | response.Reject c b \ Rejected c b) in + S\messages := take n (messages S) @ drop (Suc n) (messages S), + requests := list_map_set (requests S) req req_resp\)" + +definition respond_to_user_request_burned_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "respond_to_user_request_burned_cycles n S = (case messages S ! n of Response_message (From_user req) resp ref_cycles \ + ref_cycles)" + +lemma respond_to_user_request_cycles_monotonic: + assumes "respond_to_user_request_pre n S" + shows "total_cycles S = total_cycles (respond_to_user_request_post n S) + respond_to_user_request_burned_cycles n S" +proof - + obtain req resp ref_cycles where msg: "messages S ! n = Response_message (From_user req) resp ref_cycles" + using assms + by (auto simp: respond_to_user_request_pre_def split: message.splits option.splits call_origin.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Response_message (From_user req) resp ref_cycles # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: respond_to_user_request_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: respond_to_user_request_pre_def respond_to_user_request_post_def respond_to_user_request_burned_cycles_def total_cycles_def call_ctxt_carried_cycles Let_def msgs + list_map_sum_in[where ?g=id and ?f="balances S"] split: option.splits request_status.splits) +qed + +lemma respond_to_user_request_ic_inv: + assumes "respond_to_user_request_pre n S" "ic_inv S" + shows "ic_inv (respond_to_user_request_post n S)" + using assms + by (auto simp: ic_inv_def respond_to_user_request_pre_def respond_to_user_request_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD) + + + +(* System transition: Request clean up [DONE] *) + +definition request_cleanup_pre :: "('b, 'p, 'uid, 'canid, 's) request \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "request_cleanup_pre req S = (case list_map_get (requests S) req of Some req_status \ + (case req_status of Replied _ \ True | Rejected _ _ \ True | _ \ False) + | _ \ False)" + +definition request_cleanup_post :: "('b, 'p, 'uid, 'canid, 's) request \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "request_cleanup_post req S = (S\requests := list_map_set (requests S) req Done\)" + +lemma request_cleanup_cycles_inv: + assumes "request_cleanup_pre n S" + shows "total_cycles S = total_cycles (request_cleanup_post n S)" + by (auto simp: request_cleanup_post_def total_cycles_def) + +lemma request_cleanup_ic_inv: + assumes "request_cleanup_pre req S" "ic_inv S" + shows "ic_inv (request_cleanup_post req S)" + using assms + by (auto simp: ic_inv_def request_cleanup_pre_def request_cleanup_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD in_set_updD) + + + +(* System transition: Request clean up (expired) [DONE] *) + +definition request_cleanup_expired_pre :: "('b, 'p, 'uid, 'canid, 's) request \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "request_cleanup_expired_pre req S = (case list_map_get (requests S) req of Some req_status \ + (case req_status of Replied _ \ True | Rejected _ _ \ True | Done \ True | _ \ False) \ + request.ingress_expiry req < system_time S + | _ \ False)" + +definition request_cleanup_expired_post :: "('b, 'p, 'uid, 'canid, 's) request \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "request_cleanup_expired_post req S = (S\requests := list_map_del (requests S) req\)" + +lemma request_cleanup_expired_cycles_inv: + assumes "request_cleanup_expired_pre n S" + shows "total_cycles S = total_cycles (request_cleanup_expired_post n S)" + by (auto simp: request_cleanup_expired_post_def total_cycles_def) + +lemma request_cleanup_expired_ic_inv: + assumes "request_cleanup_expired_pre req S" "ic_inv S" + shows "ic_inv (request_cleanup_expired_post req S)" + using assms + by (auto simp: ic_inv_def request_cleanup_expired_pre_def request_cleanup_expired_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD in_set_updD) + + + +(* System transition: Canister out of cycles [DONE] *) + +definition canister_out_of_cycles_pre :: "'canid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "canister_out_of_cycles_pre cid S = (case list_map_get (balances S) cid of Some 0 \ True + | _ \ False)" + +definition canister_out_of_cycles_post :: "'canid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "canister_out_of_cycles_post cid S = ( + let call_ctxt_to_msg = (\ctxt. + if call_ctxt_canister ctxt = cid \ call_ctxt_needs_to_respond ctxt then + Some (Response_message (call_ctxt_origin ctxt) (response.Reject CANISTER_REJECT (encode_string ''Canister has been uninstalled'')) (call_ctxt_available_cycles ctxt)) + else None); + call_ctxt_to_ctxt = (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt) in + S\canisters := list_map_set (canisters S) cid None, certified_data := list_map_set (certified_data S) cid empty_blob, + messages := messages S @ List.map_filter call_ctxt_to_msg (list_map_vals (call_contexts S)), + call_contexts := list_map_map call_ctxt_to_ctxt (call_contexts S)\)" + +lemma canister_out_of_cycles_cycles_inv: + assumes "canister_out_of_cycles_pre cid S" + shows "total_cycles S = total_cycles (canister_out_of_cycles_post cid S)" +proof - + have F1: "list_map_sum_vals call_ctxt_carried_cycles (call_contexts S) = + list_map_sum_vals id + (list_map_map (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_carried_cycles ctxt else 0) (call_contexts S)) + + list_map_sum_vals call_ctxt_carried_cycles + (list_map_map (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt) (call_contexts S))" + using list_map_sum_vals_split[where ?f="call_ctxt_carried_cycles" and ?g="call_ctxt_delete", unfolded call_ctxt_delete_carried_cycles diff_zero] + by auto + show ?thesis + using assms + by (auto simp: canister_out_of_cycles_pre_def canister_out_of_cycles_post_def total_cycles_def call_ctxt_carried_cycles Let_def F1 + split: message.splits option.splits sum.splits if_splits intro!: list_map_sum_vals_filter) +qed + +lemma canister_out_of_cycles_ic_inv: + assumes "canister_out_of_cycles_pre cid S" "ic_inv S" + shows "ic_inv (canister_out_of_cycles_post cid S)" + using assms + by (auto simp: ic_inv_def canister_out_of_cycles_pre_def canister_out_of_cycles_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: Time progressing and cycle consumption (canister time) [DONE] *) + +definition canister_time_progress_pre :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "canister_time_progress_pre cid t1 S = (case list_map_get (time S) cid of Some t0 \ + t0 < t1 + | _ \ False)" + +definition canister_time_progress_post :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "canister_time_progress_post cid t1 S = (S\time := list_map_set (time S) cid t1\)" + +lemma canister_time_progress_cycles_inv: + assumes "canister_time_progress_pre cid t1 S" + shows "total_cycles S = total_cycles (canister_time_progress_post cid t1 S)" + by (auto simp: canister_time_progress_post_def total_cycles_def) + +lemma canister_time_progress_ic_inv: + assumes "canister_time_progress_pre cid t1 S" "ic_inv S" + shows "ic_inv (canister_time_progress_post cid t1 S)" + using assms + by (auto simp: ic_inv_def canister_time_progress_pre_def canister_time_progress_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: Time progressing and cycle consumption (cycle consumption) [DONE] *) + +definition cycle_consumption_pre :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "cycle_consumption_pre cid b1 S = (case list_map_get (balances S) cid of Some b0 \ + 0 \ b1 \ b1 < b0 + | _ \ False)" + +definition cycle_consumption_post :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "cycle_consumption_post cid b1 S = (S\balances := list_map_set (balances S) cid b1\)" + +definition cycle_consumption_burned_cycles :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "cycle_consumption_burned_cycles cid b1 S = the (list_map_get (balances S) cid) - b1" + +lemma cycle_consumption_cycles_monotonic: + assumes "cycle_consumption_pre cid b1 S" + shows "total_cycles S = total_cycles (cycle_consumption_post cid b1 S) + cycle_consumption_burned_cycles cid b1 S" + using assms list_map_sum_in_ge[where ?g=id and ?f="balances S" and ?x=cid] + by (auto simp: cycle_consumption_pre_def cycle_consumption_post_def cycle_consumption_burned_cycles_def total_cycles_def + list_map_sum_in[where ?g=id and ?f="balances S"] split: option.splits) + +lemma cycle_consumption_ic_inv: + assumes "cycle_consumption_pre cid b1 S" "ic_inv S" + shows "ic_inv (cycle_consumption_post cid b1 S)" + using assms + by (auto simp: ic_inv_def cycle_consumption_pre_def cycle_consumption_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: Time progressing and cycle consumption (system time) [DONE] *) + +definition system_time_progress_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "system_time_progress_pre t1 S = (system_time S < t1)" + +definition system_time_progress_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "system_time_progress_post t1 S = (S\system_time := t1\)" + +lemma system_time_progress_cycles_inv: + assumes "system_time_progress_pre t1 S" + shows "total_cycles S = total_cycles (system_time_progress_post t1 S)" + by (auto simp: system_time_progress_post_def total_cycles_def) + +lemma system_time_progress_ic_inv: + assumes "system_time_progress_pre t1 S" "ic_inv S" + shows "ic_inv (system_time_progress_post t1 S)" + using assms + by (auto simp: ic_inv_def system_time_progress_pre_def system_time_progress_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* State machine *) + +inductive ic_steps :: "'sig itself \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ + nat \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + ic_steps_refl: "ic_steps sig S 0 0 S" +| request_submission: "ic_steps sig S0 minted burned S \ request_submission_pre (E :: ('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope) S \ ic_steps sig S0 minted (burned + request_submission_burned_cycles E S) (request_submission_post E S)" +| request_rejection: "ic_steps sig S0 minted burned S \ request_rejection_pre (E :: ('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope) req code msg S \ ic_steps sig S0 minted burned (request_rejection_post E req code msg S)" +| initiate_canister_call: "ic_steps sig S0 minted burned S \ initiate_canister_call_pre req S \ ic_steps sig S0 minted burned (initiate_canister_call_post req S)" +| call_reject: "ic_steps sig S0 minted burned S \ call_reject_pre n S \ ic_steps sig S0 minted burned (call_reject_post n S)" +| call_context_create: "ic_steps sig S0 minted burned S \ call_context_create_pre n ctxt_id S \ ic_steps sig S0 minted burned (call_context_create_post n ctxt_id S)" +| call_context_heartbeat: "ic_steps sig S0 minted burned S \ call_context_heartbeat_pre cee ctxt_id S \ ic_steps sig S0 minted burned (call_context_heartbeat_post cee ctxt_id S)" +| message_execution: "ic_steps sig S0 minted burned S \ message_execution_pre n S \ ic_steps sig S0 minted (burned + message_execution_burned_cycles n S) (message_execution_post n S)" +| call_context_starvation: "ic_steps sig S0 minted burned S \ call_context_starvation_pre ctxt_id S \ ic_steps sig S0 minted burned (call_context_starvation_post ctxt_id S)" +| call_context_removal: "ic_steps sig S0 minted burned S \ call_context_removal_pre ctxt_id S \ ic_steps sig S0 minted (burned + call_context_removal_burned_cycles ctxt_id S) (call_context_removal_post ctxt_id S)" +| ic_canister_creation: "ic_steps sig S0 minted burned S \ ic_canister_creation_pre n cid t S \ ic_steps sig S0 minted burned (ic_canister_creation_post n cid t S)" +| ic_update_settings: "ic_steps sig S0 minted burned S \ ic_update_settings_pre n S \ ic_steps sig S0 minted burned (ic_update_settings_post n S)" +| ic_canister_status: "ic_steps sig S0 minted burned S \ ic_canister_status_pre n m S \ ic_steps sig S0 minted burned (ic_canister_status_post n m S)" +| ic_code_installation: "ic_steps sig S0 minted burned S \ ic_code_installation_pre n S \ ic_steps sig S0 minted (burned + ic_code_installation_burned_cycles n S) (ic_code_installation_post n S)" +| ic_code_upgrade: "ic_steps sig S0 minted burned S \ ic_code_upgrade_pre n S \ ic_steps sig S0 minted (burned + ic_code_upgrade_burned_cycles n S) (ic_code_upgrade_post n S)" +| ic_code_uninstallation: "ic_steps sig S0 minted burned S \ ic_code_uninstallation_pre n S \ ic_steps sig S0 minted burned (ic_code_uninstallation_post n S)" +| ic_canister_stop_running: "ic_steps sig S0 minted burned S \ ic_canister_stop_running_pre n S \ ic_steps sig S0 minted burned (ic_canister_stop_running_post n S)" +| ic_canister_stop_stopping: "ic_steps sig S0 minted burned S \ ic_canister_stop_stopping_pre n S \ ic_steps sig S0 minted burned (ic_canister_stop_stopping_post n S)" +| ic_canister_stop_done_stopping: "ic_steps sig S0 minted burned S \ ic_canister_stop_done_stopping_pre cid S \ ic_steps sig S0 minted burned (ic_canister_stop_done_stopping_post cid S)" +| ic_canister_stop_stopped: "ic_steps sig S0 minted burned S \ ic_canister_stop_stopped_pre n S \ ic_steps sig S0 minted burned (ic_canister_stop_stopped_post n S)" +| ic_canister_start_not_stopping: "ic_steps sig S0 minted burned S \ ic_canister_start_not_stopping_pre n S \ ic_steps sig S0 minted burned (ic_canister_start_not_stopping_post n S)" +| ic_canister_start_stopping: "ic_steps sig S0 minted burned S \ ic_canister_start_stopping_pre n S \ ic_steps sig S0 minted burned (ic_canister_start_stopping_post n S)" +| ic_canister_deletion: "ic_steps sig S0 minted burned S \ ic_canister_deletion_pre n S \ ic_steps sig S0 minted (burned + ic_canister_deletion_burned_cycles n S) (ic_canister_deletion_post n S)" +| ic_depositing_cycles: "ic_steps sig S0 minted burned S \ ic_depositing_cycles_pre n S \ ic_steps sig S0 minted burned (ic_depositing_cycles_post n S)" +| ic_random_numbers: "ic_steps sig S0 minted burned S \ ic_random_numbers_pre n b S \ ic_steps sig S0 minted burned (ic_random_numbers_post n b S)" +| ic_provisional_canister_creation: "ic_steps sig S0 minted burned S \ ic_provisional_canister_creation_pre n cid t S \ ic_steps sig S0 (minted + ic_provisional_canister_creation_minted_cycles n cid t S) burned (ic_provisional_canister_creation_post n cid t S)" +| ic_top_up_canister: "ic_steps sig S0 minted burned S \ ic_top_up_canister_pre n S \ ic_steps sig S0 (minted + ic_top_up_canister_minted_cycles n S) burned (ic_top_up_canister_post n S)" +| callback_invocation_not_deleted: "ic_steps sig S0 minted burned S \ callback_invocation_not_deleted_pre n S \ ic_steps sig S0 minted burned (callback_invocation_not_deleted_post n S)" +| callback_invocation_deleted: "ic_steps sig S0 minted burned S \ callback_invocation_deleted_pre n S \ ic_steps sig S0 minted burned (callback_invocation_deleted_post n S)" +| respond_to_user_request: "ic_steps sig S0 minted burned S \ respond_to_user_request_pre n S \ ic_steps sig S0 minted (burned + respond_to_user_request_burned_cycles n S) (respond_to_user_request_post n S)" +| request_cleanup: "ic_steps sig S0 minted burned S \ request_cleanup_pre req S \ ic_steps sig S0 minted burned (request_cleanup_post req S)" +| request_cleanup_expired: "ic_steps sig S0 minted burned S \ request_cleanup_expired_pre req S \ ic_steps sig S0 minted burned (request_cleanup_expired_post req S)" +| canister_out_of_cycles: "ic_steps sig S0 minted burned S \ canister_out_of_cycles_pre cid S \ ic_steps sig S0 minted burned (canister_out_of_cycles_post cid S)" +| canister_time_progress: "ic_steps sig S0 minted burned S \ canister_time_progress_pre cid t1 S \ ic_steps sig S0 minted burned (canister_time_progress_post cid t1 S)" +| cycle_consumption: "ic_steps sig S0 minted burned S \ cycle_consumption_pre cid b1 S \ ic_steps sig S0 minted (burned + cycle_consumption_burned_cycles cid b1 S) (cycle_consumption_post cid b1 S)" +| system_time_progress: "ic_steps sig S0 minted burned S \ system_time_progress_pre t1 S \ ic_steps sig S0 minted burned (system_time_progress_post t1 S)" + +lemma total_cycles: + assumes "ic_steps TYPE('sig) S0 minted burned S" + shows "total_cycles S0 + minted = total_cycles S + burned" + using assms + apply (induction "TYPE('sig)" S0 minted burned S rule: ic_steps.induct) + apply auto[1] + using request_submission_cycles_inv apply fastforce + using request_rejection_cycles_inv apply fastforce + using initiate_canister_call_cycles_inv apply fastforce + using call_reject_cycles_inv apply fastforce + using call_context_create_cycles_inv apply fastforce + using call_context_heartbeat_cycles_inv apply fastforce + using message_execution_cycles_monotonic apply fastforce + using call_context_starvation_cycles_inv apply fastforce + using call_context_removal_cycles_monotonic apply fastforce + using ic_canister_creation_cycles_inv apply fastforce + using ic_update_settings_cycles_inv apply fastforce + using ic_canister_status_cycles_inv apply fastforce + using ic_code_installation_cycles_inv apply fastforce + using ic_code_upgrade_cycles_inv apply fastforce + using ic_code_uninstallation_cycles_inv apply fastforce + using ic_canister_stop_running_cycles_inv apply fastforce + using ic_canister_stop_stopping_cycles_inv apply fastforce + using ic_canister_stop_done_stopping_cycles_inv apply fastforce + using ic_canister_stop_stopped_cycles_inv apply fastforce + using ic_canister_start_not_stopping_cycles_inv apply fastforce + using ic_canister_start_stopping_cycles_inv apply fastforce + using ic_canister_deletion_cycles_monotonic apply fastforce + using ic_depositing_cycles_cycles_monotonic apply fastforce + using ic_random_numbers_cycles_inv apply fastforce + using ic_provisional_canister_creation_cycles_antimonotonic apply fastforce + using ic_top_up_canister_cycles_antimonotonic apply fastforce + using callback_invocation_not_deleted_cycles_inv apply fastforce + using callback_invocation_deleted_cycles_inv apply fastforce + using respond_to_user_request_cycles_monotonic apply fastforce + using request_cleanup_cycles_inv apply fastforce + using request_cleanup_expired_cycles_inv apply fastforce + using canister_out_of_cycles_cycles_inv apply fastforce + using canister_time_progress_cycles_inv apply fastforce + using cycle_consumption_cycles_monotonic apply fastforce + using system_time_progress_cycles_inv apply fastforce + done + +lemma ic_inv: + assumes "ic_steps TYPE('sig) S0 minted burned S" + shows "ic_inv S0 \ ic_inv S" + using assms + apply (induction "TYPE('sig)" S0 minted burned S rule: ic_steps.induct) + apply auto[1] + using request_submission_ic_inv apply fastforce + using request_rejection_ic_inv apply fastforce + using initiate_canister_call_ic_inv apply fastforce + using call_reject_ic_inv apply fastforce + using call_context_create_ic_inv apply fastforce + using call_context_heartbeat_ic_inv apply fastforce + using message_execution_ic_inv apply fastforce + using call_context_starvation_ic_inv apply fastforce + using call_context_removal_ic_inv apply fastforce + using ic_canister_creation_ic_inv apply fastforce + using ic_update_settings_ic_inv apply fastforce + using ic_canister_status_ic_inv apply fastforce + using ic_code_installation_ic_inv apply fastforce + using ic_code_upgrade_ic_inv apply fastforce + using ic_code_uninstallation_ic_inv apply fastforce + using ic_canister_stop_running_ic_inv apply fastforce + using ic_canister_stop_stopping_ic_inv apply fastforce + using ic_canister_stop_done_stopping_ic_inv apply fastforce + using ic_canister_stop_stopped_ic_inv apply fastforce + using ic_canister_start_not_stopping_ic_inv apply fastforce + using ic_canister_start_stopping_ic_inv apply fastforce + using ic_canister_deletion_ic_inv apply fastforce + using ic_depositing_cycles_ic_inv apply fastforce + using ic_random_numbers_ic_inv apply fastforce + using ic_provisional_canister_creation_ic_inv apply fastforce + using ic_top_up_canister_ic_inv apply fastforce + using callback_invocation_not_deleted_ic_inv apply fastforce + using callback_invocation_deleted_ic_inv apply fastforce + using respond_to_user_request_ic_inv apply fastforce + using request_cleanup_ic_inv apply fastforce + using request_cleanup_expired_ic_inv apply fastforce + using canister_out_of_cycles_ic_inv apply fastforce + using canister_time_progress_ic_inv apply fastforce + using cycle_consumption_ic_inv apply fastforce + using system_time_progress_ic_inv apply fastforce + done end @@ -769,6 +2820,32 @@ export_code request_submission_pre request_submission_post message_execution_pre message_execution_post call_context_starvation_pre call_context_starvation_post call_context_removal_pre call_context_removal_post + ic_canister_creation_pre ic_canister_creation_post + ic_update_settings_pre ic_update_settings_post + ic_canister_status_pre ic_canister_status_post + ic_code_installation_pre ic_code_installation_post + ic_code_upgrade_pre ic_code_upgrade_post + ic_code_uninstallation_pre ic_code_uninstallation_post + ic_canister_stop_running_pre ic_canister_stop_running_post + ic_canister_stop_stopping_pre ic_canister_stop_stopping_post + ic_canister_stop_done_stopping_pre ic_canister_stop_done_stopping_post + ic_canister_stop_stopped_pre ic_canister_stop_stopped_post + ic_canister_start_not_stopping_pre ic_canister_start_not_stopping_post + ic_canister_start_stopping_pre ic_canister_start_stopping_post + ic_canister_deletion_pre ic_canister_deletion_post + ic_depositing_cycles_pre ic_depositing_cycles_post + ic_random_numbers_pre ic_random_numbers_post + ic_provisional_canister_creation_pre ic_provisional_canister_creation_post + ic_top_up_canister_pre ic_top_up_canister_post + callback_invocation_not_deleted_pre callback_invocation_not_deleted_post + callback_invocation_deleted_pre callback_invocation_deleted_post + respond_to_user_request_pre respond_to_user_request_post + request_cleanup_pre request_cleanup_post + request_cleanup_expired_pre request_cleanup_expired_post + canister_out_of_cycles_pre canister_out_of_cycles_post + canister_time_progress_pre canister_time_progress_post + cycle_consumption_pre cycle_consumption_post + system_time_progress_pre system_time_progress_post in Haskell module_name IC file_prefix code end From 016f0dcb2340a18d0751e3ff4081d79b255afc78 Mon Sep 17 00:00:00 2001 From: mraszyk <31483726+mraszyk@users.noreply.github.com> Date: Fri, 26 Aug 2022 12:26:40 +0200 Subject: [PATCH 030/102] Deleted call contexts do not prevent canister from reaching Stopped state (#83) Co-authored-by: Martin Raszyk --- spec/index.adoc | 14 +++++++------- theories/IC.thy | 2 +- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/spec/index.adoc b/spec/index.adoc index 9bee7368a..c7121cbab 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -221,7 +221,7 @@ The canister status can be used to control whether the canister is processing ca * In status `running`, calls to the canister are processed as normal. * In status `stopping`, calls to the canister are rejected by the IC, but responses to the canister are processed as normal. -* In status `stopped`, calls to the canister are rejected by the IC, and there are no outstanding responses. +* In status `stopped`, calls to the canister are rejected by the IC, and there are no outstanding responses to call contexts that are not marked as deleted. In all cases, calls to the <> are processed, regardless of the state of the managed canister. @@ -1527,7 +1527,7 @@ Note that this is different from `uninstall_code` followed by `install_code`, as This is atomic: If the response to this request is a `reject`, then this call had no effect. -NOTE: Some canisters may not be able to make sense of callbacks after upgrades; these should be stopped first, to wait for all outstanding callbacks, or be uninstalled first, to prevent outstanding callbacks from being invoked. It is expected that the canister admin (or their tooling) does that separately. +NOTE: Some canisters may not be able to make sense of callbacks after upgrades; these should be stopped first, to wait for all outstanding callbacks that are not marked as deleted, or be uninstalled first, to prevent outstanding callbacks from being invoked (by marking the corresponding call contexts as deleted). It is expected that the canister admin (or their tooling) does that separately. The `wasm_module` field specifies the canister module to be installed. The system supports multiple encodings of the `wasm_module` field: @@ -1569,7 +1569,7 @@ The controllers of a canister may stop a canister (e.g., to prepare for a canist Stopping a canister is not an atomic action. The immediate effect is that the status of the canister is changed to `stopping` (unless the canister is already stopped). The IC will reject all calls to a stopping canister, indicating that the canister is stopping. Responses to a stopping canister are processed as usual. -When all outstanding responses have been processed (so there are no open call contexts), the canister status is changed to `stopped` and the management canister responds to the caller of the `stop_canister` request. +When all outstanding responses to call contexts that are not marked as deleted have been processed (so there are no open call contexts that are not marked as deleted), the canister status is changed to `stopped` and the management canister responds to the caller of the `stop_canister` request. [#ic-start_canister] === IC method `start_canister` @@ -3296,12 +3296,12 @@ S with ==== IC Management Canister: Stopping a canister -The controllers of a canister can stop a canister. Stopping a canister goes through two steps. First, the status of the canister is set to `Stopping`; as explained above, a stopping canister rejects all incoming requests and continues processing outstanding responses. When a stopping canister has no more open call contexts, its status is changed to `Stopped` and a response is generated. Note that when processing responses, a stopping canister can make calls to other canisters and thus create new call contexts. In addition, a canister which is stopped, or stopping will accept (and respond) to further `stop_canister` requests. +The controllers of a canister can stop a canister. Stopping a canister goes through two steps. First, the status of the canister is set to `Stopping`; as explained above, a stopping canister rejects all incoming requests and continues processing outstanding responses. When a stopping canister has no more open call contexts that are not marked as deleted, its status is changed to `Stopped` and a response is generated. Note that when processing responses, a stopping canister can make calls to other canisters and thus create new call contexts. In addition, a canister which is stopped, or stopping will accept (and respond) to further `stop_canister` requests. We encode this behavior via three (types of) transitions: 1. First, any `stop_canister` call sets the state of the canister to `Stopping`; we record in the status the origin (and cycles) of all `stop_canister` calls which arrive at the canister while it is stopping (or stopped). -2. Next, when the canister has no open call contexts (so, in particular, all outstanding responses to the canister have been processed), the status of the canister is set to `Stopped`. +2. Next, when the canister has no open call contexts that are not marked as deleted (so, in particular, all outstanding responses to the canister have been processed or will be discared because the call context has been marked as deleted), the status of the canister is set to `Stopped`. 3. Finally, each pending `stop_canister` call (which are encoded in the status) is responded to, to indicate that the canister is stopped. Conditions:: @@ -3341,12 +3341,12 @@ S with .... -The status of a stopping canister which has no open call contexts is set to `Stopped`, and all pending `stop_canister` calls are replied to. +The status of a stopping canister which has no open call contexts that are not marked as deleted is set to `Stopped`, and all pending `stop_canister` calls are replied to. Conditions:: .... S.canister_status[Canister_id] = Stopping Origins - ∀ Ctxt_id. S.call_contexts[Ctxt_id].canister ≠ Canister_id + ∀ Ctxt_id. S.call_contexts[Ctxt_id].deleted or S.call_contexts[Ctxt_id].canister ≠ Canister_id .... State after:: .... diff --git a/theories/IC.thy b/theories/IC.thy index 7eccffa1d..f595ec58c 100644 --- a/theories/IC.thy +++ b/theories/IC.thy @@ -1882,7 +1882,7 @@ qed definition ic_canister_stop_done_stopping_pre :: "'canid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "ic_canister_stop_done_stopping_pre cid S = (case list_map_get (canister_status S) cid of Some (Stopping os) \ - (\ctxt \ list_map_range (call_contexts S). call_ctxt_canister ctxt \ cid) + (\ctxt \ list_map_range (call_contexts S). call_ctxt_deleted ctxt \ call_ctxt_canister ctxt \ cid) | _ \ False)" definition ic_canister_stop_done_stopping_post :: "'canid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where From c7a537e5f7d90d084d3289f62b7190eaef823dbb Mon Sep 17 00:00:00 2001 From: mraszyk <31483726+mraszyk@users.noreply.github.com> Date: Wed, 31 Aug 2022 19:09:11 +0200 Subject: [PATCH 031/102] update spec of ECID (#82) Co-authored-by: Martin Raszyk --- spec/index.adoc | 39 +++++++++++++++++++++------------------ theories/IC.thy | 35 ++++++++++++++--------------------- 2 files changed, 35 insertions(+), 39 deletions(-) diff --git a/spec/index.adoc b/spec/index.adoc index c7121cbab..20b343129 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -543,7 +543,10 @@ The HTTP response to this request consists of a CBOR map with the following fiel * `certificate` (`blob`): A certificate (see <>). + -If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation’s canister id range (see <>), unless the `effective_canister_id` is that of the Management Canister (`aaaaa-aa`). In other words, we define `aaaaa-aa` to belong to every canister range for the purposes of certificate checking. Note that `aaaaa-aa` is a valid canister id only for calls to `provisional_create_canister_with_cycles`, so it cannot appear in `read_state` requests on production networks. +If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation’s canister id range (see <>), unless +- the `effective_canister_id` is that of the Management Canister (`aaaaa-aa`), +- all requested paths have `/request_status/` as prefix where the (single) original request referenced by `` is an update call to the Management Canister (`aaaaa-aa`) and the method name is `provisional_create_canister_with_cycles`, and +- whenever the certificate contains the path `/request_status//reply`, then its value is a Candid-encoded record with a `canister_id` field of type `principal` and the `canister_id` must be included in each delegation’s canister id range. The returned certificate reveals all values whose path is a suffix of the list of requested paths. It also always reveals `/time`, even if not explicitly requested. @@ -551,10 +554,10 @@ All requested paths must have one of the following paths as prefix: * `/time`. Can be requested by anyone. * `/subnet`. Can be requested by anyone. - * `/request_status/`. Can only be read if `/request_id/` is not present in the state tree, or if this `read_state` request was signed by same sender as the original request referenced by ``, and the effective canister id of the original request matches the `` (see <>) in this HTTP request’s path. - * `/canisters//module_hash`. Can be requested by anyone, if `` matches ``. - * `/canisters//controllers`. Can be requested by anyone, if `` matches ``. The order may vary depending on the implementation. - * `/canisters//metadata/`. Can be read by anyone if `` matches ``, and `` is a public custom section. If `` is a private custom section, it can only be read if this `read_state` request was signed by one of the controllers of the canister. + * `/request_status/`. Can only be requested by the same sender as the original request referenced by `` and if the effective canister id of the original request matches ``. + * `/canisters//module_hash`. Can be requested by anyone if `` matches ``. + * `/canisters//controllers`. Can be requested by anyone if `` matches ``. The order may vary depending on the implementation. + * `/canisters//metadata/`. Can be requested by anyone if `` matches `` and `` is a public custom section. If `` is a private custom section, it can only be requested by the controllers of the canister. Note that the paths `/canisters//certified_data` are not accessible with this method; these paths are only exposed to the canister themselves via the System API (see <>). @@ -592,25 +595,24 @@ Canister methods that do not change the canister state can be executed more effi The `` in the URL paths of requests is the _effective_ destination of the request. -* If the call is to the Management Canister (`aaaaa-aa`), then: - - If the `arg` is Candid-encoded where the first argument is a record with a `canister_id` field of type `principal`, then the effective canister id is that principal. - - Otherwise, if the call is to the `provisional_create_canister_with_cycles` method, then any principal is a valid effective canister id for this call. - - Otherwise, there is no effective canister id. In particular, the `create_canister` and `raw_rand` methods have no effective canister ID, and these methods cannot be called by users, only via canisters. +* If the request is an update call to the Management Canister (`aaaaa-aa`), then: + - If the call is to the `provisional_create_canister_with_cycles` method, then any principal can be used as the effective canister id for this call. + - Otherwise, if the `arg` is a Candid-encoded record with a `canister_id` field of type `principal`, then the effective canister id must be that principal. + - Otherwise, the call is rejected by the system independently of the effective canister id. + +* If the request is an update call to a canister that is not the Management Canister (`aaaaa-aa`) or if the request is a query call, then the effective canister id must be the `canister_id` in the request. + [NOTE] ==== -The Internet Computer blockchain mainnet does not support `provisional_create_canister_with_cycles`. This means that using an effective canister id that could be an existing canister would lead to the request being routed to a node on the corresponding subnet and produce an error response there, while using an effective canister id that cannot be an existing canister id would cause an error response from the node receiving the request -- either is fine. - -In multi-subnet development instances of the Internet Computer Protocol (e.g. testnets), users with privileged access to `provisional_create_canister_with_cycles` (or possibly similar, internal methods) and knowledge of the mapping from canisters to subnets can use their choice of the effective canister id in the URL to steer the request to a specific subnet. - -In a local canister execution environment, the effective canister id is ignored, and thus `aaaaa-aa` can be used. -==== +The expectation is that user-side agent code shields users and developers from the notion of effective canister ID, in analogy to how the System API interface shields canister developers from worrying about routing. -* Else, the effective canister id must be the `canister_id` in the request. +The Internet Computer blockchain mainnet rejects all requests whose effective canister id is in no subnet's canister ranges, independently of whether the remaining conditions on the effective canister id are satisfied. -NOTE: The expectation is that user-side agent code shields users and developers from this concept, in analogy to how the System API interface shields canister developers from worrying about routing. +The Internet Computer blockchain mainnet does not support `provisional_create_canister_with_cycles` and thus all calls to this method are rejected independently of the effective canister id. +In development instances of the Internet Computer Protocol (e.g. testnets), the effective canister id of a request submitted to a node must be a canister id from the canister ranges of the subnet to which the node belongs, unless the request is an update call to the Management Canister (`aaaaa-aa`), the method name is `provisional_create_canister_with_cycles`, and the effective canister id is that of the Management Canister (`aaaaa-aa`). +==== [#authentication] === Authentication @@ -2623,8 +2625,8 @@ delegation_targets(DS) A `Request` has an effective canister id according to the rules in <<#http-effective-canister-id>>: .... -is_effective_canister_id(Request {canister_id = ic_principal, arg = candid({canister_id = p, …}), …}, p) is_effective_canister_id(Request {canister_id = ic_principal, method = provisional_create_canister_with_cycles, p) +is_effective_canister_id(Request {canister_id = ic_principal, arg = candid({canister_id = p, …}), …}, p) is_effective_canister_id(Request {canister_id = p, …}, p), if p ≠ ic_principal .... @@ -3086,6 +3088,7 @@ To avoid clashes with potential user ids or is derived from users or canisters, * `is_system_assigned (mk_self_authenticating_id pk) = false` for possible public keys `pk` and * `is_system_assigned (mk_derived_id p dn) = false` for any `p` that could be a user id or canister id. * `is_system_assigned p = false` for `|p| > 29`. + * `is_system_assigned ic_principal = false`. ==== IC Management Canister: Changing settings diff --git a/theories/IC.thy b/theories/IC.thy index f595ec58c..4e542f128 100644 --- a/theories/IC.thy +++ b/theories/IC.thy @@ -605,17 +605,10 @@ fun message_queue :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message \ definition is_effective_canister_id :: "('b, 'p, 'uid, 'canid, 's) request \ 'p \ bool" where "is_effective_canister_id r p = (if request.canister_id r = ic_principal then - (case candid_parse_cid (request.arg r) of Some cid \ principal_of_canid cid = p - | _ \ request.method_name r = encode_string ''provisional_create_canister_with_cycles'') + request.method_name r = encode_string ''provisional_create_canister_with_cycles'' \ + (case candid_parse_cid (request.arg r) of Some cid \ principal_of_canid cid = p | _ \ False) else principal_of_canid (request.canister_id r) = p)" -lemma is_effective_canister_id_code[code_unfold]: - "(\p. is_effective_canister_id r p) = (if request.canister_id r = ic_principal then - (case candid_parse_cid (request.arg r) of Some cid \ True - | _ \ request.method_name r = encode_string ''provisional_create_canister_with_cycles'') - else True)" - by (auto simp: is_effective_canister_id_def split: option.splits) - (* System transition: API Request submission [DONE] *) @@ -623,12 +616,12 @@ lemma is_effective_canister_id_code[code_unfold]: definition ic_freezing_limit :: "('p :: linorder, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ 'canid \ nat" where "ic_freezing_limit S cid = ic_idle_cycles_burned_rate S cid * (the (list_map_get (freezing_threshold S) cid)) div (24 * 60 * 60)" -definition request_submission_pre :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where - "request_submission_pre E S = (case content E of Inl req \ +definition request_submission_pre :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ 'p \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "request_submission_pre E ECID S = (case content E of Inl req \ principal_of_canid (request.canister_id req) \ verify_envelope E (principal_of_uid (request.sender req)) (system_time S) \ req \ list_map_dom (requests S) \ system_time S \ request.ingress_expiry req \ - (\ECID. is_effective_canister_id req ECID) \ + is_effective_canister_id req ECID \ ( ( request.canister_id req = ic_principal \ @@ -656,8 +649,8 @@ definition request_submission_pre :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) enve ) | _ \ False)" -definition request_submission_post :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where - "request_submission_post E S = ( +definition request_submission_post :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ 'p \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "request_submission_post E ECID S = ( let req = projl (content E); cid = request.canister_id req; balances = (if cid \ ic_principal then @@ -669,8 +662,8 @@ definition request_submission_post :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) env else balances S) in S\requests := list_map_set (requests S) req Received, balances := balances\)" -definition request_submission_burned_cycles :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where - "request_submission_burned_cycles E S = ( +definition request_submission_burned_cycles :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ 'p \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "request_submission_burned_cycles E ECID S = ( let req = projl (content E); cid = request.canister_id req in (if request.canister_id req \ ic_principal then @@ -682,8 +675,8 @@ definition request_submission_burned_cycles :: "('b, 'p, 'uid, 'canid, 's, 'pk, else 0))" lemma request_submission_cycles_inv: - assumes "request_submission_pre E S" - shows "total_cycles S = total_cycles (request_submission_post E S) + request_submission_burned_cycles E S" + assumes "request_submission_pre E ECID S" + shows "total_cycles S = total_cycles (request_submission_post E ECID S) + request_submission_burned_cycles E ECID S" proof - obtain req where req_def: "content E = Inl req" using assms @@ -709,8 +702,8 @@ proof - qed lemma request_submission_ic_inv: - assumes "request_submission_pre E S" "ic_inv S" - shows "ic_inv (request_submission_post E S)" + assumes "request_submission_pre E ECID S" "ic_inv S" + shows "ic_inv (request_submission_post E ECID S)" using assms by (auto simp: ic_inv_def request_submission_pre_def request_submission_post_def Let_def split: sum.splits message.splits call_origin.splits) @@ -2687,7 +2680,7 @@ lemma system_time_progress_ic_inv: inductive ic_steps :: "'sig itself \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where ic_steps_refl: "ic_steps sig S 0 0 S" -| request_submission: "ic_steps sig S0 minted burned S \ request_submission_pre (E :: ('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope) S \ ic_steps sig S0 minted (burned + request_submission_burned_cycles E S) (request_submission_post E S)" +| request_submission: "ic_steps sig S0 minted burned S \ request_submission_pre (E :: ('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope) ECID S \ ic_steps sig S0 minted (burned + request_submission_burned_cycles E ECID S) (request_submission_post E ECID S)" | request_rejection: "ic_steps sig S0 minted burned S \ request_rejection_pre (E :: ('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope) req code msg S \ ic_steps sig S0 minted burned (request_rejection_post E req code msg S)" | initiate_canister_call: "ic_steps sig S0 minted burned S \ initiate_canister_call_pre req S \ ic_steps sig S0 minted burned (initiate_canister_call_post req S)" | call_reject: "ic_steps sig S0 minted burned S \ call_reject_pre n S \ ic_steps sig S0 minted burned (call_reject_post n S)" From b02b4dfc236590f0427987e671f025e9822af77a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dominic=20W=C3=B6rner?= Date: Wed, 31 Aug 2022 19:14:18 +0200 Subject: [PATCH 032/102] wrong flag name for upgrading http requests to update calls (#84) Signed-off-by: Dominic Woerner Signed-off-by: Dominic Woerner Co-authored-by: mraszyk <31483726+mraszyk@users.noreply.github.com> --- spec/index.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/spec/index.adoc b/spec/index.adoc index 20b343129..af5b76711 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -2105,9 +2105,9 @@ The HTTP request is encoded into the `HttpRequest` Candid structure. === Upgrade to update calls -If the canister sets `update = opt true` in the `HttpResponse` reply from `http_request`, then the Gateway ignores all other fields of the reply. The Gateway performs an _update_ call to `http_request_update`, passing the same `HttpRequest` record as the argument, and uses that response instead. +If the canister sets `upgrade = opt true` in the `HttpResponse` reply from `http_request`, then the Gateway ignores all other fields of the reply. The Gateway performs an _update_ call to `http_request_update`, passing the same `HttpRequest` record as the argument, and uses that response instead. -The value of the `update` field returned from `http_request_update` is ignored. +The value of the `upgrade` field returned from `http_request_update` is ignored. === Response decoding From af1af96507eca3ca472ed7e93b5abe761580ebd5 Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Wed, 31 Aug 2022 21:30:42 +0200 Subject: [PATCH 033/102] update changelog --- spec/changelog.adoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 26e70bba7..787c2af92 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -6,6 +6,8 @@ * Canister access to performance metrics * Query calls are rejected when the canister is frozen * Support for implementation-specific error codes for requests +* Deleted call contexts do not prevent canister from reaching Stopped state +* Update effective canister id checks in certificate delegations * Formal model in Isabelle [#0_18_5] From 62185f4fb695aef3bd43d2cf40dbb51ed5706f2c Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Fri, 2 Sep 2022 10:50:57 +0200 Subject: [PATCH 034/102] Merge notational changes --- spec/index.adoc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/spec/index.adoc b/spec/index.adoc index af5b76711..55ee440f2 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -544,8 +544,9 @@ The HTTP response to this request consists of a CBOR map with the following fiel * `certificate` (`blob`): A certificate (see <>). + If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation’s canister id range (see <>), unless + - the `effective_canister_id` is that of the Management Canister (`aaaaa-aa`), -- all requested paths have `/request_status/` as prefix where the (single) original request referenced by `` is an update call to the Management Canister (`aaaaa-aa`) and the method name is `provisional_create_canister_with_cycles`, and +- all requested paths have `/time` or `/request_status/` as prefix where the (single) original request referenced by `` is an update call to the Management Canister (`aaaaa-aa`) and the method name is `provisional_create_canister_with_cycles`, and - whenever the certificate contains the path `/request_status//reply`, then its value is a Candid-encoded record with a `canister_id` field of type `principal` and the `canister_id` must be included in each delegation’s canister id range. The returned certificate reveals all values whose path is a suffix of the list of requested paths. It also always reveals `/time`, even if not explicitly requested. From 4d1997f6e135d1f1d79f975bb901a0be98ea414a Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Tue, 27 Sep 2022 10:04:53 +0200 Subject: [PATCH 035/102] Merge 0.18.6 to public --- .github/workflows/isabelle.yml | 23 + .gitignore | 1 + README.md | 20 + spec/changelog.adoc | 9 + spec/index.adoc | 892 +++++----- spec/requests.cddl | 1 + theories/IC.thy | 2844 ++++++++++++++++++++++++++++++++ theories/ROOT | 8 + 8 files changed, 3410 insertions(+), 388 deletions(-) create mode 100644 .github/workflows/isabelle.yml create mode 100644 theories/IC.thy create mode 100644 theories/ROOT diff --git a/.github/workflows/isabelle.yml b/.github/workflows/isabelle.yml new file mode 100644 index 000000000..2b2a018d6 --- /dev/null +++ b/.github/workflows/isabelle.yml @@ -0,0 +1,23 @@ +name: Build on Ubuntu +on: + push: + paths: + - 'theories/**' + +jobs: + isabelle: + runs-on: ubuntu-latest + steps: + - name: Check out the repo + uses: actions/checkout@v2 + - name: Run Isabelle with Docker + uses: addnab/docker-run-action@v3 + with: + image: martin2718/isabelle + options: -v ${{ github.workspace }}:/interface-spec + run: Isabelle/bin/isabelle build -e -v -D /interface-spec/theories/ + - name: Haskell Code + uses: actions/upload-artifact@v2 + with: + name: haskell-code + path: ${{ github.workspace }}/theories/code diff --git a/.gitignore b/.gitignore index 6f138f2cb..5db325438 100644 --- a/.gitignore +++ b/.gitignore @@ -8,3 +8,4 @@ spec/index.html spec/*.png spec/.asciidoctor/ impl/.tasty-rerun-log +theories/code diff --git a/README.md b/README.md index 25bac53e3..0117bb19e 100644 --- a/README.md +++ b/README.md @@ -30,6 +30,26 @@ The `master` branch contains finished designs, but is not directly scheduled for implementation. It lists version version number `∞`. The reference implementation on this branch typically does _not_ fully implement the spec. This branch should always be “ahead” of all the release branches. +## Formal Model + +We are developing a formal model of Interface Spec in the interactive theorem prover [Isabelle/HOL](https://isabelle.in.tum.de/). +The formal development is included in the directory `theories/`. + +To setup the environment, follow the standard [instructions](https://isabelle.in.tum.de/installation.html) for Isabelle/HOL. +Additionally, you may want to setup `isabelle` as an alias for the path `bin/isabelle` in your local Isabelle directory. + +To browse the formal model, open Isabelle/jEdit: +``` +isabelle jedit theories/IC.thy +``` +from the root directory of this repository. + +To build the formal model and export Haskell code from the formal model, run +``` +isabelle build -e -v -D theories/ +``` +in the root directory of this repository. The exported Haskell code can then be found under `theories/code/`. + ## Contributing This repository accepts external contributions, conditioned on acceptance of the [https://github.com/dfinity/cla/](Contributor Lincense Agreement). diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 47eddf347..787c2af92 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,15 @@ [#changelog] == Changelog +[#0_18_6] +=== 0.18.6 (2022-08-09) +* Canister access to performance metrics +* Query calls are rejected when the canister is frozen +* Support for implementation-specific error codes for requests +* Deleted call contexts do not prevent canister from reaching Stopped state +* Update effective canister id checks in certificate delegations +* Formal model in Isabelle + [#0_18_5] === 0.18.5 (2022-07-08) * Idle consumption of resources in cycles per day can be obtain via `canister_status` method of the management canister diff --git a/spec/index.adoc b/spec/index.adoc index e2ad3b736..55ee440f2 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,6 +1,6 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.5 +0.18.6 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ @@ -145,13 +145,11 @@ When the IC creates a _fresh_ id, it never creates a self-authenticating id, an [#textual-ids] ==== Textual representation of principals -NOTE: This textual representation does not actually show up in the interface (which always deals with blobs), so it is merely a recommended convention. - We specify a _canonical textual format_ that is recommended whenever principals need to be printed or read in textual format, e.g. in log messages, transactions browser, command line tools, source code. The textual representation of a blob `b` is `Grouped(Base32(CRC32(b) · b))` where - * `CRC32` is a four byte check sequence, calculated as defined by ISO 3309, ITU-T V.42 and https://www.w3.org/TR/2003/REC-PNG-20031110/#5CRC-algorithm[elsewhere] + * `CRC32` is a four byte check sequence, calculated as defined by ISO 3309, ITU-T V.42, and https://www.w3.org/TR/2003/REC-PNG-20031110/#5CRC-algorithm[elsewhere], and stored as big-endian, i.e., the most significant byte comes first and then the less significant bytes come in descending order of significance (MSB B2 B1 LSB). * `Base32` is the Base32 encoding as defined in https://tools.ietf.org/html/rfc4648#section-6[RFC 4648], with no padding character added. * The middle dot denotes concatenation. * `Grouped` takes an ASCII string and inserts the separator `-` (dash) every 5 characters. The last group may contain less than 5 characters. A separator never appears at the beginning or end. @@ -223,15 +221,13 @@ The canister status can be used to control whether the canister is processing ca * In status `running`, calls to the canister are processed as normal. * In status `stopping`, calls to the canister are rejected by the IC, but responses to the canister are processed as normal. -* In status `stopped`, calls to the canister are rejected by the IC, and there are no outstanding responses. +* In status `stopped`, calls to the canister are rejected by the IC, and there are no outstanding responses to call contexts that are not marked as deleted. In all cases, calls to the <> are processed, regardless of the state of the managed canister. The controllers of the canister can initiate transitions between these states using <> and <>, and query the state using <>. The canister itself can also query its state using <>. -NOTE: -This status is orthogonal to the question of whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected, but because the canister is empty. - +NOTE: This status is orthogonal to the question of whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected, but because the canister is empty. [#signatures] === Signatures @@ -404,6 +400,10 @@ If the status is `rejected`, then this path contains the reject code (see </error_code` (text) ++ +If the status is `rejected`, then this path might be present and contain an implementation-specific error code (see <>), else it is not present. + NOTE: Immediately after submitting a request, the request may not show up yet as the Internet Computer is still working on accepting the request as pending. NOTE: Request statuses will not actually be kept around indefinitely, and eventually the Internet Computer forgets about the request. This will happen no sooner than the request’s expiry time, so that replay attacks are prevented. @@ -543,7 +543,11 @@ The HTTP response to this request consists of a CBOR map with the following fiel * `certificate` (`blob`): A certificate (see <>). + -If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation’s canister id range (see <>). +If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation’s canister id range (see <>), unless + +- the `effective_canister_id` is that of the Management Canister (`aaaaa-aa`), +- all requested paths have `/time` or `/request_status/` as prefix where the (single) original request referenced by `` is an update call to the Management Canister (`aaaaa-aa`) and the method name is `provisional_create_canister_with_cycles`, and +- whenever the certificate contains the path `/request_status//reply`, then its value is a Candid-encoded record with a `canister_id` field of type `principal` and the `canister_id` must be included in each delegation’s canister id range. The returned certificate reveals all values whose path is a suffix of the list of requested paths. It also always reveals `/time`, even if not explicitly requested. @@ -551,10 +555,10 @@ All requested paths must have one of the following paths as prefix: * `/time`. Can be requested by anyone. * `/subnet`. Can be requested by anyone. - * `/request_status/`. Can only be read if `/request_id/` is not present in the state tree, or if this `read_state` request was signed by same sender as the original request referenced by ``, and the effective canister id of the original request matches the `` (see <>) in this HTTP request’s path. - * `/canisters//module_hash`. Can be requested by anyone, if `` matches ``. - * `/canisters//controllers`. Can be requested by anyone, if `` matches ``. The order may vary depending on the implementation. - * `/canisters//metadata/`. Can be read by anyone if `` matches ``, and `` is a public custom section. If `` is a private custom section, it can only be read if this `read_state` request was signed by one of the controllers of the canister. + * `/request_status/`. Can only be requested by the same sender as the original request referenced by `` and if the effective canister id of the original request matches ``. + * `/canisters//module_hash`. Can be requested by anyone if `` matches ``. + * `/canisters//controllers`. Can be requested by anyone if `` matches ``. The order may vary depending on the implementation. + * `/canisters//metadata/`. Can be requested by anyone if `` matches `` and `` is a public custom section. If `` is a private custom section, it can only be requested by the controllers of the canister. Note that the paths `/canisters//certified_data` are not accessible with this method; these paths are only exposed to the canister themselves via the System API (see <>). @@ -583,6 +587,7 @@ If the call resulted in a reject, the response is a CBOR map with the following * `status` (`text`): `rejected` * `reject_code` (`nat`): The reject code (see <>). * `reject_message` (`text`): a textual diagnostic message. +* `error_code` (text): an optional implementation-specific textual error code (see <>). Canister methods that do not change the canister state can be executed more efficiently. This method provides that ability, and returns the canister’s response directly within the HTTP response. @@ -591,25 +596,24 @@ Canister methods that do not change the canister state can be executed more effi The `` in the URL paths of requests is the _effective_ destination of the request. -* If the call is to the Management Canister (`aaaaa-aa`), and the `arg` is Candid-encoded where the first argument is a record with a `canister_id` field of type `principal`, then the effective canister id is that principal. +* If the request is an update call to the Management Canister (`aaaaa-aa`), then: + - If the call is to the `provisional_create_canister_with_cycles` method, then any principal can be used as the effective canister id for this call. + - Otherwise, if the `arg` is a Candid-encoded record with a `canister_id` field of type `principal`, then the effective canister id must be that principal. + - Otherwise, the call is rejected by the system independently of the effective canister id. -* If the call is to the `raw_rand` method of the Management Canister (`aaaaa-aa`), then there is no effective canister id. This implies that this method cannot be called by users, only via canisters. +* If the request is an update call to a canister that is not the Management Canister (`aaaaa-aa`) or if the request is a query call, then the effective canister id must be the `canister_id` in the request. -* If the call is to the `provisional_create_canister_with_cycles` method of the Management Canister (`aaaaa-aa`), any principal is a valid effective canister id for this call. + [NOTE] ==== -The Internet Computer blockchain mainnet does not support `provisional_create_canister_with_cycles`. This means that using an effective canister id that could be an existing canister would lead to the request being routed to a node on the corresponding subnet and produce an error response there, while using an effective canister id that cannot be an existing canister id would cause an error response from the node receiving the request -- either is fine. - -In multi-subnet development instances of the Internet Computer Protocol (e.g. testnets), users with privileged access to `provisional_create_canister_with_cycles` (or possibly similar, internal methods) and knowledge of the mapping from canisters to subnets can use their choice of the effective canister id in the URL to steer the request to a specific subnet. +The expectation is that user-side agent code shields users and developers from the notion of effective canister ID, in analogy to how the System API interface shields canister developers from worrying about routing. -In a local canister execution environment, the effective canister id is ignored, and thus `aaaaa-aa` can be used. -==== +The Internet Computer blockchain mainnet rejects all requests whose effective canister id is in no subnet's canister ranges, independently of whether the remaining conditions on the effective canister id are satisfied. -* Else, the effective canister id must be the `canister_id` in the request. - -NOTE: The expectation is that user-side agent code shields users and developers from this concept, in analogy to how the System API interface shields canister developers from worrying about routing. +The Internet Computer blockchain mainnet does not support `provisional_create_canister_with_cycles` and thus all calls to this method are rejected independently of the effective canister id. +In development instances of the Internet Computer Protocol (e.g. testnets), the effective canister id of a request submitted to a node must be a canister id from the canister ranges of the subnet to which the node belongs, unless the request is an update call to the Management Canister (`aaaaa-aa`), the method name is `provisional_create_canister_with_cycles`, and the effective canister id is that of the Management Canister (`aaaaa-aa`). +==== [#authentication] === Authentication @@ -732,6 +736,13 @@ The error message is guaranteed to be a string, i.e. not arbitrary binary data. When canisters explicitly reject a message (see <>), they can specify the reject message, but _not_ the reject code; it is always `CANISTER_REJECT`. In this sense, the reject code is trustworthy: If the IC responds with a `SYS_FATAL` reject, then it really was the IC issuing this reject. +[#error-codes] +=== Error codes + +Implementations of the API can provide additional details for rejected messages in the form of a textual label identifying the error condition. +API clients can use these labels to handle errors programmatically or suggest recovery paths to the user. +The specification reserves error codes matching the regular expression `IC[0-9]+` (e.g., `IC502`) for the DFINITY implementation of the API. + [#api-status] === Status endpoint @@ -1351,9 +1362,26 @@ The time is given as nanoseconds since 1970-01-01. The IC guarantees that * the time, as observed by the canister, is monotonically increasing, even across canister upgrades. * within an invocation of one entry point, the time is constant. -The system times of different canisters are unrelated, and calls from one canister to another may appear to travel “backwards in time”. +The times observed by different canisters are unrelated, and calls from one canister to another may appear to travel “backwards in time”. + +NOTE: While an implementation will likely try to keep the time returned by `ic0.time` close to the real time, this is not formally part of this specification. + +[#system-api-performance-counter] +=== Performance counter -NOTE: While an implementation will likely try to keep the System Time close to the real time, this is not formally part of this specification. +The canister can query the "performance counter", which is a deterministic monotonically increasing integer approximating the amount of work the canister has done since the beginning of the current execution. + +`+ic0.performance_counter : (counter_type : i32) -> i64+` + +The argument `type` decides which performance counter to return: + +* 0 : instruction counter. The number of WebAssembly instructions the system has determined that the canister has executed. + +In the future, we might expose more performance counters. + +The system resets the counter at the beginning of each <> invocation. + +The main purpose of this counter is to facilitate in-canister performance profiling. [#system-api-certified-data] === Certified data @@ -1502,7 +1530,7 @@ Note that this is different from `uninstall_code` followed by `install_code`, as This is atomic: If the response to this request is a `reject`, then this call had no effect. -NOTE: Some canisters may not be able to make sense of callbacks after upgrades; these should be stopped first, to wait for all outstanding callbacks, or be uninstalled first, to prevent outstanding callbacks from being invoked. It is expected that the canister admin (or their tooling) does that separately. +NOTE: Some canisters may not be able to make sense of callbacks after upgrades; these should be stopped first, to wait for all outstanding callbacks that are not marked as deleted, or be uninstalled first, to prevent outstanding callbacks from being invoked (by marking the corresponding call contexts as deleted). It is expected that the canister admin (or their tooling) does that separately. The `wasm_module` field specifies the canister module to be installed. The system supports multiple encodings of the `wasm_module` field: @@ -1544,7 +1572,7 @@ The controllers of a canister may stop a canister (e.g., to prepare for a canist Stopping a canister is not an atomic action. The immediate effect is that the status of the canister is changed to `stopping` (unless the canister is already stopped). The IC will reject all calls to a stopping canister, indicating that the canister is stopping. Responses to a stopping canister are processed as usual. -When all outstanding responses have been processed (so there are no open call contexts), the canister status is changed to `stopped` and the management canister responds to the caller of the `stop_canister` request. +When all outstanding responses to call contexts that are not marked as deleted have been processed (so there are no open call contexts that are not marked as deleted), the canister status is changed to `stopped` and the management canister responds to the caller of the `stop_canister` request. [#ic-start_canister] === IC method `start_canister` @@ -2078,9 +2106,9 @@ The HTTP request is encoded into the `HttpRequest` Candid structure. === Upgrade to update calls -If the canister sets `update = opt true` in the `HttpResponse` reply from `http_request`, then the Gateway ignores all other fields of the reply. The Gateway performs an _update_ call to `http_request_update`, passing the same `HttpRequest` record as the argument, and uses that response instead. +If the canister sets `upgrade = opt true` in the `HttpResponse` reply from `http_request`, then the Gateway ignores all other fields of the reply. The Gateway performs an _update_ call to `http_request_update`, passing the same `HttpRequest` record as the argument, and uses that response instead. -The value of the `update` field returned from `http_request_update` is ignored. +The value of the `upgrade` field returned from `http_request_update` is ignored. === Response decoding @@ -2233,10 +2261,8 @@ WasmState = (abstract) StableMemory = (abstract) Callback = (abstract) -Arg = { - data : Blob - caller: Principal -} +Arg = Blob; +CallerId = Principal; Timestamp = Nat; Env = { @@ -2257,27 +2283,47 @@ MethodCall = { callback: Callback; } -UpdateFunc = WasmState -> Trap | Return { +UpdateFunc = WasmState -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; new_calls : List MethodCall; new_certified_data : NoCertifiedData | Blob response : NoResponse | Response; cycles_accepted : Nat; + cycles_used : Nat; +} +QueryFunc = WasmState -> Trap { cycles_used : Nat; } | Return { + response : Response; + cycles_used : Nat; +} +HeartbeatFunc = WasmState -> Trap { cycles_used : Nat; } | Return { + new_state : WasmState; + cycles_used : Nat; } -QueryFunc = WasmState -> Trap | Return Response AvailableCycles = Nat RefundedCycles = Nat CanisterModule = { - init : (CanisterId, Arg, Env) -> Trap | Return WasmState - pre_upgrade : (WasmState, caller : Principal, Env) -> Trap | Return StableMemory - post_upgrade : (CanisterId, StableMemory, Arg, Env) -> Trap | Return WasmState - update_methods : MethodName ↦ ((Arg, Env, AvailableCycles) -> UpdateFunc) - query_methods : MethodName ↦ ((Arg, Env) -> QueryFunc) - heartbeat : (Env) -> WasmState -> Trap | Return WasmState + init : (CanisterId, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { + new_state : WasmState; + cycles_used : Nat; + } + pre_upgrade : (WasmState, Principal, Env) -> Trap { cycles_used : Nat; } | Return { + stable_memory : StableMemory; + cycles_used : Nat; + } + post_upgrade : (CanisterId, StableMemory, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { + new_state : WasmState; + cycles_used : Nat; + } + update_methods : MethodName ↦ ((Arg, CallerId, Env, AvailableCycles) -> UpdateFunc) + query_methods : MethodName ↦ ((Arg, CallerId, Env) -> QueryFunc) + heartbeat : (Env) -> HeartbeatFunc callbacks : (Callback, Response, RefundedCycles, Env, AvailableCycles) -> UpdateFunc - inspect_message : (MethodName, WasmState, Arg, Env) -> Trap | Return (Accept | Reject) + inspect_message : (MethodName, WasmState, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { + status : Accept | Reject; + cycles_used : Nat; + } } .... @@ -2287,9 +2333,11 @@ The `CanisterId` parameter of `init` and `post_upgrade` is merely passed through The `Env` parameter provides synchronous read-only access to portions of the system state and canister metadata that are always available. -The parsing of a blob to a canister module is modelled via the (possibly implicitly failing) function +The parsing of a blob to a canister module and its public and private custom sections is modelled via the (possibly implicitly failing) functions .... parse_wasm_mod : Blob -> CanisterModule +parse_public_custom_sections : Blob -> Text ↦ Blob +parse_private_custom_sections : Blob -> Text ↦ Blob .... The concrete mapping of this abstract `CanisterModule` to actual WebAssembly concepts and the System API is described separately in section <>. @@ -2302,13 +2350,14 @@ The Internet Computer provides certain messaging guarantees: If a user or a cani To ensure that only one response is generated, and also to detect when no response can be generated any more, the IC maintains a _call context_. The `needs_to_respond` field is set to `false` once the call has received a response. Further attempts to respond will now fail. .... -CallCtxt = { - canister : CanisterId; - origin : CallOrigin; - needs_to_respond : bool; - deleted : bool; - available_cycles : Nat; -} +Request = { + nonce : Blob; + ingress_expiry : Nat; + sender : UserId; + canister_id : CanisterId; + method_name : Text; + arg : Blob; + } CallId = (abstract) CallOrigin = FromUser { @@ -2319,6 +2368,13 @@ CallOrigin callback: Callback } | FromHeartbeat +CallCtxt = { + canister : CanisterId; + origin : CallOrigin; + needs_to_respond : bool; + deleted : bool; + available_cycles : Nat; +} .... ==== Calls and Messages @@ -2339,7 +2395,7 @@ Message caller : Principal; callee : CanisterId; method_name : Text; - data : Blob; + arg : Blob; transferred_cycles : Nat; queue : Queue; } @@ -2362,36 +2418,7 @@ A reference implementation would likely maintain a separate list of `messages` f ==== API requests -We distinguish between the _asynchronous_ API requests passed to `/api/v2/…/call`, which may be present in the IC state, and the _synchronous_ API requests passed to `/api/v2/…/read_state` and `/api/v2/…/query`, which are only ephemeral. - -.... -Envelope = { - content : Request | APIReadRequest; - sender_pubkey : PublicKey | NoPublicKey; - sender_sig : Signature | NoSignature; - sender_delegation: [SignedDelegation] -} - -Request - = CanisterUpdateCall = { - nonce : Blob; - ingress_expiry : Nat; - sender : UserId; - canister_id : CanisterId; - method_name : Text; - data : Blob; - } -.... - -The evolution of a `Request` goes through these states, as explained in <>: -.... -RequestStatus - = Received - | Processing - | Rejected (RejectCode, Text) - | Replied Blob - | Done -.... +We distinguish between the _asynchronous_ API requests (type `Request`) passed to `/api/v2/…/call`, which may be present in the IC state, and the _synchronous_ API requests passed to `/api/v2/…/read_state` and `/api/v2/…/query`, which are only ephemeral. These are the synchronous read messages: .... @@ -2409,41 +2436,73 @@ APIReadRequest sender : UserId; canister_id : CanisterId; method_name : Text; - data : Blob; + arg : Blob; } .... -A `Path` may refer to a request by way of a _request id_, as specified in <>: -.... -Request = Blob -hash_of_map: Request -> Request -.... - -For the signatures in a `Request`, we assume that the following function implements signature verification as described in <>. -This function picks the corresponding signature scheme according to the DER-encoded metadata in the public key. +Signed delegations contain the (unsigned) delegation data in a nested record, next to the signature of that data. .... PublicKey = Blob Signature = Blob -verify_signature : PublicKey -> Signature -> Blob -> Bool -.... - -Signed delegations contain the (unsigned) delegation data in a nested record, next to the signature of that data. -.... SignedDelegation = { delegation : { pubkey : PublicKey; targets : [CanisterId] | Unrestricted; + senders : [Principal] | Unrestricted; expiration : Timestamp }; signature : Signature } .... +For the signatures in a `Request`, we assume that the following function implements signature verification as described in <>. +This function picks the corresponding signature scheme according to the DER-encoded metadata in the public key. +.... +verify_signature : PublicKey -> Signature -> Blob -> Bool +.... + +.... +Envelope = { + content : Request | APIReadRequest; + sender_pubkey : PublicKey | NoPublicKey; + sender_sig : Signature | NoSignature; + sender_delegation: [SignedDelegation] +} +.... + +The evolution of a `Request` goes through these states, as explained in <>: +.... +RequestStatus + = Received + | Processing + | Rejected (RejectCode, Text) + | Replied Blob + | Done +.... + +A `Path` may refer to a request by way of a _request id_, as specified in <>: +.... +RequestId = Blob +hash_of_map: Request -> RequestId +.... + ==== The system state Finally, we can describe the state of the IC as a record having the following fields: .... +CanState + = EmptyCanister | { + wasm_state : WasmState; + module : CanisterModule; + raw_module : Blob; + public_custom_sections: Text ↦ Blob; + private_custom_sections: Text ↦ Blob; +} +CanStatus + = Running + | Stopping (List (CallOrigin, Nat)) + | Stopped S = { requests : Request ↦ RequestStatus; canisters : CanisterId ↦ CanState; @@ -2458,18 +2517,13 @@ S = { messages : List Message; // ordered! root_key : PublicKey } -CanState - = EmptyCanister | { - wasm_state : WasmState; - module : CanisterModule; - raw_module : Blob; - public_custom_sections: Text ↦ Blob; - private_custom_sections: Text ↦ Blob; -} -CanStatus - = Running - | Stopping (List (CallOrigin, Nat)) - | Stopped +.... + +To convert `CanStatus` into `status : Running | Stopping | Stopped` from `Env`, we define the following conversion function: +.... +simple_status(Running) = Running +simple_status(Stopping _) = Stopping +simple_status(Stopped) = Stopped .... ==== Initial state @@ -2481,11 +2535,13 @@ The initial state of the IC is canisters = (); controllers = (); freezing_threshold = (); + canister_status = (); time = (); balances = (); + certified_data = (); system_time = T; call_contexts = (); - messages = (); + messages = []; root_key = PublicKey; } .... @@ -2495,6 +2551,13 @@ for some time stamp `T`, some DER-encoded BLS public key `PublicKey`, and using The following is an incomplete list of invariants that should hold for the abstract state `S`, and are not already covered by the type annotations in this section. +* No method name is the name of an update and query method in a CanisterModule at the same time: ++ +.... +∀ _ ↦ CanState ∈ S.canisters: + dom(CanState.module.update_methods) ∩ dom(CanState.module.query_methods) = ∅ +.... + * Deleted call contexts were not awaiting a response: + .... @@ -2509,14 +2572,13 @@ The following is an incomplete list of invariants that should hold for the abstr if Ctxt.needs_to_respond = false then Ctxt.available_cycles = 0 .... -* Referenced call contexts exists: +* Referenced call contexts exist: + .... -∀ CallMessage M ∈ S.messages. M.origin.calling_context ∈ S.call_contexts -∀ ResponseMessage M ∈ S.messages. M.origin.calling_context ∈ S.call_contexts -∀ _ ↦ Ctxt ∈ S.call_contexts: - if Ctx.needs_to_respond: - Ctxt.origin.calling_context ∈ S.call_contexts +∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ∈ dom(S.call_contexts) +∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ∈ dom(S.call_contexts) +∀ _ ↦ {needs_to_respond = true, origin = FromCanister O, …} ∈ S.call_contexts: O.calling_context ∈ dom(S.call_contexts) +∀ _ ↦ Stopping Origins ∈ S.canister_status: ∀(FromCanister O, _) ∈ Origins. O.calling_context ∈ dom(S.call_contexts) .... === State transitions @@ -2525,7 +2587,7 @@ Based on this abstract notion of the state, we can describe the behavior of the * Asynchronous API requests that are submitted via `/api/v2/…/call`. These transitions describe checks that the request must pass to be considered received. * Spontaneous transitions that model the internal behavior of the IC, by describing conditions on the state that allow the transition to happen, and the state after. - * Responses to reads (i.e. `/api/v2/…/read_state`). By definition, these do _not_ change the state of the IC, and merely describe the response based on the read request and the current state. + * Responses to reads (i.e. `/api/v2/…/read_state` and `/api/v2/…/query`). By definition, these do _not_ change the state of the IC, and merely describe the response based on the read request (or query, respectively) and the current state. The state transitions are not complete with regard to error handling. For example, the behavior of sending a request to a non-existent canister is not specified here. For now, we trust implementors to make sensible decisions there. @@ -2564,9 +2626,9 @@ delegation_targets(DS) A `Request` has an effective canister id according to the rules in <<#http-effective-canister-id>>: .... -is_effective_canister_id(CanisterUpdateCall {canister_id = ic_principal, arg = candid({canister_id = p, …}), …}, p) -is_effective_canister_id(CanisterUpdateCall {canister_id = ic_principal, method = provisional_create_canister_with_cycles, p) -is_effective_canister_id(CanisterUpdateCall {canister_id = p, …}, p), if p ≠ ic_principal +is_effective_canister_id(Request {canister_id = ic_principal, method = provisional_create_canister_with_cycles, p) +is_effective_canister_id(Request {canister_id = ic_principal, arg = candid({canister_id = p, …}), …}, p) +is_effective_canister_id(Request {canister_id = p, …}, p), if p ≠ ic_principal .... ==== API Request submission @@ -2580,31 +2642,50 @@ Requests that have expired are dropped here. Ingress message inspection is applied, and messages that are not accepted by the canister are dropped. +The (unspecified) function `idle_cycles_burned_rate(S, cid)` determines the idle resource consumption rate in cycles per day of the canister with id `cid`, +given its current memory footprint, compute and storage cost, and memory and compute allocation. +The function `freezing_limit(S, cid)` determines the freezing threshold in cycles of the canister with id `cid`, +given its current memory footprint, compute and storage cost, memory and compute allocation, and current `freezing_threshold` setting. +The value `freezing_limit(S, cid)` is derived from `idle_cycles_burned_rate(S, cid)` and `freezing_threshold` as follows: +.... + freezing_limit(S, cid) = idle_cycles_burned_rate(S, cid) * S.freezing_threshold[cid] / (24 * 60 * 60) +.... + Submitted request:: `E : Envelope` Conditions:: .... E.content.canister_id ∈ verify_envelope(E, E.content.sender, S.system_time) - E.content ∉ requests + E.content ∉ dom(S.requests) S.system_time <= E.content.ingress_expiry is_effective_canister_id(E.content, ECID) ( E.content.canister_id = ic_principal E.content.arg = candid({canister_id = CanisterId, …}) E.content.sender ∈ S.controllers[CanisterId] E.content.method_name ∈ - { "install_code", "set_controller", "start_canister", "stop_canister", - "canister_status", "delete_canister" } + { "install_code", "uninstall_code", "update_settings", "start_canister", "stop_canister", + "canister_status", "delete_canister", + "provisional_create_canister_with_cycles", "provisional_top_up_canister" } ) ∨ ( E.content.canister_id ≠ ic_principal S.canisters[E.content.canister_id] ≠ EmptyCanister - Arg = { data = E.content.arg; caller = E.content.sender; method = E.content.method_name; time = S.time[E.content.canister_id] } + Env = { + time = S.time[E.content.canister_id]; + balance = S.balances[E.content.canister_id] + freezing_limit = freezing_limit(S, E.content.canister_id); + certificate = NoCertificate; + status = simple_status(S.canister_status[E.content.canister_id]); + } S.canisters[E.content.canister_id].module.inspect_message - (E.content.method_name, C.wasm_state, Arg, S.balance[E.content.canister_id]) = Return Accept + (E.content.method_name, S.canisters[E.content.canister_id].wasm_state, E.content.arg, E.content.sender, Env) = Return {status = Accept; cycles_used = Cycles_used;} + Cycles_used ≤ S.balances[E.content.canister_id] ) .... State after:: .... S with requests[E.content] = Received + if E.content.canister_id ≠ ic_principal then + balances[E.content.canister_id] = S.balances[E.content.canister_id] - Cycles_used .... NOTE: This is not instantaneous (the IC takes some time to agree it accepts the request) nor guaranteed (a node could just drop the request, or maybe it did not pass validation). But once the request has entered the IC state like this, it will be acted upon. @@ -2634,17 +2715,17 @@ The IC does not make any guarantees about the order of incoming messages. Conditions:: .... - S.requests[CanisterUpdateCall R] = Received + S.requests[R] = Received S.system_time <= R.ingress_expiry C = S.canisters[R.canister_id] .... State after:: .... S with - requests[CanisterUpdateCall R] = Processing + requests[R] = Processing messages = CallMessage { - origin = FromUser { request = CanisterUpdateCall R }; + origin = FromUser { request = R }; caller = R.sender; callee = R.canister_id; method_name = R.method_name; @@ -2659,20 +2740,18 @@ S with A call to a canister which is stopping, stopped, or frozen is automatically rejected. -The (unspecified) function `freezing_limit(S, cid)` determines the freezing threshold in cycles of the canister with id `cid`, given its current memory footprint, storage cost, memory and compute allocation, and current `freezing_threshold` setting. - Conditions:: .... S.messages = Older_messages · CallMessage CM · Younger_messages (CM.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ CM.queue) - S.canister_status[CM.callee] = Stopped or S.canister_status[CM.callee] = Stopping or balances[CM.callee] < freezing_limit(S, CM.callee) + S.canister_status[CM.callee] = Stopped or S.canister_status[CM.callee] = Stopping _ or balances[CM.callee] < freezing_limit(S, CM.callee) .... State after:: .... S.messages = Older_messages · Younger_messages · ResponseMessage { - origin = S.call_contexts[CM.call_context].origin + origin = CM.origin; response = Reject (CANISTER_ERROR, "canister not running"); refunded_cycles = CM.transferred_cycles; } @@ -2699,8 +2778,8 @@ Conditions:: S.messages = Older_messages · CallMessage CM · Younger_messages S.canisters[CM.callee] ≠ EmptyCanister S.canister_status[CM.callee] = Running - balances[CM.callee] ≥ freezing_limit(S, CM.callee) + MAX_CYCLES_PER_MESSAGE - Ctxt_id ∉ dom S.call_contexts + S.balances[CM.callee] ≥ freezing_limit(S, CM.callee) + MAX_CYCLES_PER_MESSAGE + Ctxt_id ∉ dom(S.call_contexts) .... + State after:: @@ -2712,7 +2791,7 @@ S with FuncMessage { call_context = Ctxt_id; receiver = CM.callee; - entry_point = PublicMethod CM.method_name CM.caller CM.data + entry_point = PublicMethod CM.method_name CM.caller CM.arg; queue = CM.queue; } · Younger_messages @@ -2723,7 +2802,7 @@ S with deleted = false; available_cycles = CM.transferred_cycles; } - balances[CM.callee] = balances[CM.callee] - MAX_CYCLES_PER_MESSAGE + balances[CM.callee] = S.balances[CM.callee] - MAX_CYCLES_PER_MESSAGE .... * Call context creation: Heartbeat @@ -2736,7 +2815,7 @@ Conditions:: S.canisters[C] ≠ EmptyCanister S.canister_status[C] = Running balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE - Ctxt_id ∉ dom S.call_contexts + Ctxt_id ∉ dom(S.call_contexts) .... + State after:: @@ -2758,7 +2837,7 @@ S with deleted = false; available_cycles = 0; } - balances[C] = balances[C] - MAX_CYCLES_PER_MESSAGE + balances[C] = S.balances[C] - MAX_CYCLES_PER_MESSAGE .... The IC can execute any message that is at the head of its queue, i.e. there is no older message with the same abstract `queue` field. The actual message execution, if successful, may enqueue further messages and -- if the function returns a response -- record this response. The new call and response messages are enqueued at the end. @@ -2784,15 +2863,12 @@ Conditions:: balance = S.balances[M.receiver] freezing_limit = freezing_limit(S, M.receiver); certificate = NoCertificate; - status = S.status[M.receiver]; + status = simple_status(S.canister_status[M.receiver]); } Available = S.call_contexts[M.call_contexts].available_cycles - ( M.entry_point = PublicMethod Name Caller Data - Arg = { data = Data; caller = Caller } - (F = Mod.update_methods[Name](Arg, Env, Available) - or - (F = query_as_update(Mod.query_methods[Name], Arg, Env)) + ( M.entry_point = PublicMethod Name Caller Arg + (F = Mod.update_methods[Name](Arg, Caller, Env, Available)) or (F = query_as_update(Mod.query_methods[Name], Arg, Caller, Env)) ) or ( M.entry_point = Callback Callback Response RefundedCycles @@ -2809,15 +2885,14 @@ State after:: .... if R = Return res - Cycles_used ≤ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + res.cycles_used ≤ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) res.cycles_accepted ≤ Available + (res.cycles_used + ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ]) ≤ + (S.balances[M.receiver] + res.cycles_accepted + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) New_balance = - S.balances[M.receiver] - + res.cycles_accepted - + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - - Cycles_used - - ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ] - New_balance > if Is_response then 0 else freezing_limit(S, CM.callee); + (S.balances[M.receiver] + res.cycles_accepted + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) + - (res.cycles_used + ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ]) + New_balance ≥ if Is_response then 0 else freezing_limit(S, M.receiver); (res.response = NoResponse) or S.call_contexts[M.call_context].needs_to_respond then S with @@ -2859,9 +2934,8 @@ else S with messages = Older_messages · Younger_messages balances[M.receiver] = - S.balances[M.receiver] - + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - - Cycles_used + (S.balances[M.receiver] + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) + - min (R.cycles_used, (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) .... @@ -2884,17 +2958,30 @@ If message execution <>; that would require a (unique) call to `ic0.reply`. Note also that the state changes are persisted even when the IC is set to synthesize a <> reject immediately afterward (which happens when this returns without calling `ic0.reply` or `ic0.reject`, the corresponding call has not been responded to and there are no outstanding callbacks, see <>). -The function `as_update` turns a query function into an update function, this is merely a notational trick to simplify the rule +The functions `query_as_update` and `heartbeat_as_update` turns a query function resp the heartbeat into an update function; this is merely a notational trick to simplify the rule: .... -as_update(f, arg, env) = λ wasm_state → +query_as_update(f, arg, env) = λ wasm_state → match f(arg, env)(wasm_state) with - Trap → Trap + Trap trap → Trap trap Return res → Return { new_state = wasm_state; new_calls = []; - response = res; + new_certified_data = NoCertifiedData; + response = res.response; cycles_accepted = 0; + cycles_used = res.cycles_used; + } + +heartbeat_as_update(f, env) = λ wasm_state → + match f(env)(wasm_state) with + Trap trap → Trap trap + Return res → Return { + new_state = res.new_state; + new_calls = []; new_certified_data = NoCertifiedData; + response = NoResponse; + cycles_accepted = 0; + cycles_used = res.cycles_used; } .... Note that by construction, a query function will either trap or return with a response; it will never send calls, and it will never change the state of the canister. @@ -2908,11 +2995,9 @@ Conditions:: .... S.call_contexts[Ctxt_id].needs_to_respond = true S.call_contexts[Ctxt_id].origin ≠ FromHeartbeat - ∀ CallMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id - ∀ ResponseMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id - ∀ ctxt_ids. - S.call_contexts[ctxt_ids].needs_to_respond - ==> S.call_contexts[ctxt_ids].origin.calling_context ≠ Ctxt_id + ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id + ∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id + ∀ _ ↦ {needs_to_respond = true, origin = FromCanister O, …} ∈ S.call_contexts: O.calling_context ≠ Ctxt_id .... State after:: .... @@ -2941,11 +3026,10 @@ Conditions:: S.call_contexts[Ctxt_id].origin = FromHeartbeat ∀ FuncMessage M ∈ S.messages. M.call_context ≠ Ctxt_id ) - ∀ CallMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id - ∀ ResponseMessage M ∈ S.messages. M.origin.calling_context ≠ Ctxt_id - ∀ ctxt_ids. - S.call_contexts[ctxt_ids].needs_to_respond = true - ==> S.call_contexts[ctxt_ids].origin.calling_context ≠ Ctxt_id + ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id + ∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id + ∀ _ ↦ {needs_to_respond = true, origin = FromCanister O, …} ∈ S.call_contexts: O.calling_context ≠ Ctxt_id + ∀ _ ↦ Stopping Origins ∈ S.canister_status: ∀(FromCanister O, _) ∈ Origins. O.calling_context ≠ Ctxt_id .... State after:: .... @@ -2971,7 +3055,7 @@ Conditions:: M.method_name = 'create_canister' M.arg = candid(A) is_system_assigned CanisterId - CanisterId ∉ dom S.canisters + CanisterId ∉ dom(S.canisters) .... State after:: .... @@ -2982,12 +3066,13 @@ S with controllers[CanisterId] = A.settings.controllers else: controllers[CanisterId] = [M.caller] + freezing_threshold[CanisterId] = 2592000 balances[CanisterId] = M.transferred_cycles certified_data[CanisterId] = "" messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin - response = Accepted (candid({canister_id = CanisterId})) + response = Reply (candid({canister_id = CanisterId})) refunded_cycles = 0 } canister_status[CanisterId] = Running @@ -3004,6 +3089,7 @@ To avoid clashes with potential user ids or is derived from users or canisters, * `is_system_assigned (mk_self_authenticating_id pk) = false` for possible public keys `pk` and * `is_system_assigned (mk_derived_id p dn) = false` for any `p` that could be a user id or canister id. * `is_system_assigned p = false` for `|p| > 29`. + * `is_system_assigned ic_principal = false`. ==== IC Management Canister: Changing settings @@ -3025,12 +3111,12 @@ State after:: S with if A.settings.controllers is not null: controllers[A.canister_id] = A.settings.controllers - if A.settings.freezing_threshold exists: + if A.settings.freezing_threshold is not null: freezing_threshold[A.canister_id] = A.settings.freezing_threshold messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin - response = Accepted (candid()) + response = Reply (candid()) refunded_cycles = M.transferred_cycles } .... @@ -3041,7 +3127,7 @@ The controllers of a canister can obtain information about the canister. The `Memory_size` is the (in this specification underspecified) total size of storage in bytes. -The `idle_cycles_burned_per_day` is the idle consumption of resources in cycles per day. Therefore, the freezing threshold in cycles can be obtained using the following formula: `freezing_threshold` (in seconds) * `idle_cycles_burned_per_day` / (3600 * 24) (seconds). +The `idle_cycles_burned_per_day` is the idle consumption of resources in cycles per day. Conditions:: .... @@ -3059,16 +3145,16 @@ S with ResponseMessage { origin = M.origin response = candid({ - status = S.canister_status[A.canister_id]; + status = simple_status(S.canister_status[A.canister_id]); module_hash = if S.canisters[A.canister_id] = EmptyCanister then null - else opt (SHA-265(S.canisters[A.canister_id].raw_module)); + else opt (SHA-256(S.canisters[A.canister_id].raw_module)); controllers = S.controllers[A.canister_id]; memory_size = Memory_size; - cycles = S.balance[A.canister_id]; + cycles = S.balances[A.canister_id]; freezing_threshold = S.freezing_threshold[A.canister_id]; - idle_cycles_burned_per_day = freezing_limit(S, A.canister_id) / freezing_threshold * 3600 * 24; + idle_cycles_burned_per_day = idle_cycles_burned_rate(S, A.canister_id); }) refunded_cycles = M.transferred_cycles } @@ -3086,32 +3172,38 @@ Conditions:: M.method_name = 'install_code' M.arg = candid(A) Mod = parse_wasm_mod(A.wasm_module) - (A.mode = install && S.canisters[A.canister_id] = EmptyCanister) - or A.mode = reinstall + Public_custom_sections = parse_public_custom_sections(A.wasm_module); + Private_custom_sections = parse_private_custom_sections(A.wasm_module); + (A.mode = install and S.canisters[A.canister_id] = EmptyCanister) or A.mode = reinstall M.caller ∈ S.controllers[A.canister_id] - Arg = { - data = A.arg; - caller = M.caller; - } Env = { - time = S.time[M.receiver]; - balance = S.balances[M.receiver]; - freezing_limit = freezing_limit(S, M.receiver); + time = S.time[A.canister_id]; + balance = S.balances[A.canister_id]; + freezing_limit = freezing_limit(S, A.canister_id); certificate = NoCertificate; - status = S.status[M.receiver]; + status = simple_status(S.canister_status[A.canister_id]); } - Mod.init(A.canister_id, Arg, Env) = Return New_state + Mod.init(A.canister_id, A.arg, M.caller, Env) = Return {new_state = New_state; cycles_used = Cycles_used;} + Cycles_used ≤ S.balances[A.canister_id] + dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ + .... State after:: .... S with - canisters[A.canister_id] = - { wasm_state = New_state; module = Mod; raw_module = A.wasm_module } + canisters[A.canister_id] = { + wasm_state = New_state; + module = Mod; + raw_module = A.wasm_module; + public_custom_sections = Public_custom_sections; + private_custom_sections = Private_custom_sections; + } + balances[A.canister_id] = S.balances[A.canister_id] - Cycles_used messages = Older_messages · Younger_messages · ResponseMessage { - origin = M.origin - response = Accepted (candid()) - refunded_cycles = M.transferred_cycles + origin = M.origin; + response = Reply (candid()); + refunded_cycles = M.transferred_cycles; } .... @@ -3127,35 +3219,39 @@ Conditions:: M.method_name = 'install_code' M.arg = candid(A) Mod = parse_wasm_mod(A.wasm_module) - + Public_custom_sections = parse_public_custom_sections(A.wasm_module) + Private_custom_sections = parse_private_custom_sections(A.wasm_module) A.mode = upgrade - S.canisters[A.canister_id] ≠ EmptyCanister M.caller ∈ S.controllers[A.canister_id] - S.canisters[A.canister_id] = { wasm_state = Old_state; module = Old_module } + S.canisters[A.canister_id] = { wasm_state = Old_state; module = Old_module, …} Env = { - time = S.time[M.receiver]; - balance = S.balances[M.receiver]; - freezing_limit = freezing_limit(S, M.receiver); + time = S.time[A.canister_id]; + balance = S.balances[A.canister_id]; + freezing_limit = freezing_limit(S, A.canister_id); certificate = NoCertificate; - status = S.status[M.receiver]; + status = simple_status(S.canister_status[A.canister_id]); } - Old_module.pre_upgrade(Old_State, M.caller, Env) = Return Stable_memory - Arg = { - data = A.arg; - caller = M.caller; - } - Mod.post_upgrade(A.canister_id, Stable_memory, Arg, Env) = Return New_state + Old_module.pre_upgrade(Old_State, M.caller, Env) = Return {stable_memory = Stable_memory; cycles_used = Cycles_used;} + Mod.post_upgrade(A.canister_id, Stable_memory, A.arg, M.caller, Env) = Return {new_state = New_state; cycles_used = Cycles_used';} + Cycles_used + Cycles_used' ≤ S.balances[A.canister_id] + dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ .... State after:: .... S with - canisters[A.canister_id] = - { wasm_state = New_state; module = Mod; raw_module = A.wasm_module } + canisters[A.canister_id] = { + wasm_state = New_state; + module = Mod; + raw_module = A.wasm_module; + public_custom_sections = Public_custom_sections; + private_custom_sections = Private_custom_sections; + } + balances[A.canister_id] = S.balances[A.canister_id] - (Cycles_used + Cycles_used'); messages = Older_messages · Younger_messages · ResponseMessage { - origin = M.origin - response = Accepted (candid()) - refunded_cycles = M.transferred_cycles + origin = M.origin; + response = Reply (candid()); + refunded_cycles = M.transferred_cycles; } .... @@ -3182,7 +3278,7 @@ S with messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin - response = Accepted (candid()) + response = Reply (candid()) refunded_cycles = M.transferred_cycles } · [ ResponseMessage { @@ -3204,12 +3300,12 @@ S with ==== IC Management Canister: Stopping a canister -The controllers of a canister can stop a canister. Stopping a canister goes through two steps. First, the status of the canister is set to `Stopping`; as explained above, a stopping canister rejects all incoming requests and continues processing outstanding responses. When a stopping canister has no more open call contexts, its status is changed to `Stopped` and a response is generated. Note that when processing responses, a stopping canister can make calls to other canisters and thus create new call contexts. In addition, a canister which is stopped, or stopping will accept (and respond) to further `stop_canister` requests. +The controllers of a canister can stop a canister. Stopping a canister goes through two steps. First, the status of the canister is set to `Stopping`; as explained above, a stopping canister rejects all incoming requests and continues processing outstanding responses. When a stopping canister has no more open call contexts that are not marked as deleted, its status is changed to `Stopped` and a response is generated. Note that when processing responses, a stopping canister can make calls to other canisters and thus create new call contexts. In addition, a canister which is stopped, or stopping will accept (and respond) to further `stop_canister` requests. We encode this behavior via three (types of) transitions: 1. First, any `stop_canister` call sets the state of the canister to `Stopping`; we record in the status the origin (and cycles) of all `stop_canister` calls which arrive at the canister while it is stopping (or stopped). -2. Next, when the canister has no open call contexts (so, in particular, all outstanding responses to the canister have been processed), the status of the canister is set to `Stopped`. +2. Next, when the canister has no open call contexts that are not marked as deleted (so, in particular, all outstanding responses to the canister have been processed or will be discared because the call context has been marked as deleted), the status of the canister is set to `Stopped`. 3. Finally, each pending `stop_canister` call (which are encoded in the status) is responded to, to indicate that the canister is stopped. Conditions:: @@ -3226,7 +3322,7 @@ State after:: .... S with messages = Older_messages · Younger_messages - S.status[A.canister_id] = Stopping [(M.origin, M.transferred_cycles)] + canister_status[A.canister_id] = Stopping [(M.origin, M.transferred_cycles)] .... The next two transitions record any additional 'stop_canister' requests that arrive at a stopping (or stopped) canister in its status. @@ -3245,24 +3341,25 @@ State after:: .... S with messages = Older_messages · Younger_messages - S.status[A.canister_id] = Stopping (Origins · (M.origin, M.transferred_cycles)) + canister_status[A.canister_id] = Stopping (Origins · [(M.origin, M.transferred_cycles)]) .... -The status of a stopping canister which has no open call contexts is set to `Stopped`, and all pending `stop_canister` calls are replied to. +The status of a stopping canister which has no open call contexts that are not marked as deleted is set to `Stopped`, and all pending `stop_canister` calls are replied to. Conditions:: .... - S.canister_status[A.canister_id] = Stopping Origins - ∀ Ctxt_id. S.call_contexts[Ctxt_id].canister ≠ A.canister_id + S.canister_status[Canister_id] = Stopping Origins + ∀ Ctxt_id. S.call_contexts[Ctxt_id].deleted or S.call_contexts[Ctxt_id].canister ≠ Canister_id .... State after:: .... - S.canister_status[CanisterId] = Stopped - S.messages = Messages · +S with + canister_status[CanisterId] = Stopped + messages = S.Messages · [ ResponseMessage { origin = O - response = Accepted (candid()) + response = Reply (candid()) refunded_cycles = C } | (O, C) ∈ Origins @@ -3284,12 +3381,12 @@ Conditions:: State after:: .... S with - messages = Older_messages · Younger_messages - S.messages = Messages · - ResponseMessage { - origin = M.origin - response = Accepted (candid()) - } + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin; + response = Reply (candid()); + refunded_cycles = M.transferred_cycles; + } .... ==== IC Management Canister: Starting a canister @@ -3303,17 +3400,17 @@ Conditions:: M.callee = ic_principal M.method_name = 'start_canister' M.arg = candid(A) - S.status[A.canister_id] = Running or S.status[A.canister_id] = Stopped + S.canister_status[A.canister_id] = Running or S.canister_status[A.canister_id] = Stopped M.caller ∈ S.controllers[A.canister_id] .... State after:: .... S with - S.status[A.canister_id] = Running + canister_status[A.canister_id] = Running messages = Older_messages · Younger_messages · ResponseMessage{ origin = M.origin - response = Accepted (candid()) + response = Reply (candid()) refunded_cycles = M.transferred_cycles } .... @@ -3328,17 +3425,17 @@ Conditions:: M.callee = ic_principal M.method_name = 'start_canister' M.arg = candid(A) - S.status[A.canister_id] = Stopping Origins + S.canister_status[A.canister_id] = Stopping Origins M.caller ∈ S.controllers[A.canister_id] .... State after:: .... S with - S.status[A.canister_id] = Running + canister_status[A.canister_id] = Running messages = Older_messages · Younger_messages · ResponseMessage{ origin = M.origin - response = Accepted (candid()) + response = Reply (candid()) refunded_cycles = M.transferred_cycles } · [ ResponseMessage { @@ -3359,21 +3456,23 @@ Conditions:: M.callee = ic_principal M.method_name = 'delete_canister' M.arg = candid(A) - S.canister_status[A.canister_id] = stopped + S.canister_status[A.canister_id] = Stopped M.caller ∈ S.controllers[A.canister_id] .... State after:: .... S with - canisters[CanisterId] = (deleted) - controllers[CanisterId] = (deleted) - canister_status[CanisterId] = (deleted) - time[CanisterId] = (deleted) - balances[CanisterId] = (deleted) + canisters[A.canister_id] = (deleted) + controllers[A.canister_id] = (deleted) + freezing_threshold[A.canister_id] = (deleted) + canister_status[A.canister_id] = (deleted) + time[A.canister_id] = (deleted) + balances[A.canister_id] = (deleted) + certified_data[A.canister_id] = (deleted) messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin - response = Accepted (candid()) + response = Reply (candid()) refunded_cycles = M.transferred_cycles } .... @@ -3387,17 +3486,17 @@ Conditions:: M.callee = ic_principal M.method_name = 'deposit_cycles' M.arg = candid(A) - Cycle_cost ≤ S.balances[A.canister_id] + M.transferred_cycles + A.canister_id ∈ dom(S.balances) .... State after:: .... S with - balances[CanisterId] = + balances[A.canister_id] = S.balances[A.canister_id] + M.transferred_cycles messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin - response = Accepted (candid()) + response = Reply (candid()) refunded_cycles = 0 } .... @@ -3423,7 +3522,7 @@ S with messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin - response = Accepted (candid(B)) + response = Reply (candid(B)) refunded_cycles = M.transferred_cycles } .... @@ -3440,7 +3539,7 @@ Conditions:: M.method_name = 'provisional_create_canister_with_cycles' M.arg = candid(A) is_system_assigned CanisterId - CanisterId ∉ dom S.canisters + CanisterId ∉ dom(S.canisters) .... State after:: .... @@ -3448,12 +3547,13 @@ S with canisters[CanisterId] = EmptyCanister time[CanisterId] = CurrentTime controllers[CanisterId] = [M.caller] + freezing_threshold[CanisterId] = 2592000 balances[CanisterId] = A.amount certified_data[CanisterId] = "" messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin - response = Accepted (candid({canister_id = CanisterId})) + response = Reply (candid({canister_id = CanisterId})) transferred_cycles = M.transferred_cycles } canister_status[CanisterId] = Running @@ -3468,12 +3568,12 @@ Conditions:: M.callee = ic_principal M.method_name = 'provisional_top_up_canister' M.arg = candid(A) - A.canister_id ∈ dom S.canisters + A.canister_id ∈ dom(S.canisters) .... State after:: .... S with - balances[CanisterId] = balances[CanisterId] + A.amount + balances[A.canister_id] = S.balances[A.canister_id] + A.amount .... ==== Callback invocation @@ -3490,24 +3590,25 @@ Conditions:: callback = Callback } not S.call_contexts[Ctxt_id].deleted + S.call_contexts[Ctxt_id].canister ∈ dom(S.balances) .... State after:: .... S with balances[S.call_contexts[Ctxt_id].canister] = - balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles + S.balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles messages = Older_messages · FuncMessage { call_context = Ctxt_id - receiver = C - entry_point = Callback Callback FM.response RM.refunded_cycles + receiver = S.call_contexts[Ctxt_id].canister + entry_point = Callback Callback RM.response RM.refunded_cycles queue = Unordered } · Younger_messages .... -If the responded call context does not exist anymore, because the canister has been uninstalled since, the refundend cycles are still added to the canister balance, but +If the responded call context does not exist anymore, because the canister has been uninstalled since, the refunded cycles are still added to the canister balance, but no function invocation is enqueued: Conditions:: @@ -3518,12 +3619,13 @@ Conditions:: callback = Callback } S.call_contexts[Ctxt_id].deleted + S.call_contexts[Ctxt_id].canister ∈ dom(S.balances) .... State after:: .... S with balances[S.call_contexts[Ctxt_id].canister] = - balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles + MAX_CYCLES_PER_RESPONSE + S.balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles + MAX_CYCLES_PER_RESPONSE messages = Older_messages · Younger_messages .... @@ -3533,17 +3635,17 @@ When an ingress method call has been responded to, we can record the response in Conditions:: .... - S.requests[M] = Processing S.messages = Older_messages · ResponseMessage RM · Younger_messages RM.origin = FromUser { request = M } + S.requests[M] = Processing .... State after:: .... S with messages = Older_messages · Younger_messages requests[M] = - | Replied R if response = Reply R - | Rejected R if response = Reject R + | Replied R if M.response = Reply R + | Rejected (c, R) if M.response = Reject (c, R) .... NB: The refunded cycles, `RM.refunded_cycles` are, by construction, empty. @@ -3588,21 +3690,21 @@ State after:: .... S with canisters[CanisterId] = EmptyCanister - certified_data[A.canister_id] = "" + certified_data[Canister_id] = "" - messages = Older_messages · Younger_messages · + messages = S.messages · [ ResponseMessage { origin = Ctxt.origin response = Reject (CANISTER_REJECT, 'Canister has been uninstalled') refunded_cycles = Ctxt.available_cycles } | Ctxt_id ↦ Ctxt ∈ S.call_contexts - , Ctxt.canister = A.canister_id + , Ctxt.canister = CanisterId , Ctxt.needs_to_respond = true ] for Ctxt_id ↦ Ctxt ∈ S.call_contexts: - if Ctxt.canister = A.canister_id: + if Ctxt.canister = CanisterId: call_contexts[Ctxt_id].deleted := true call_contexts[Ctxt_id].needs_to_respond := false call_contexts[Ctxt_id].available_cycles := 0 @@ -3652,7 +3754,7 @@ S with ==== Query call -Canister query calls to `/api/v2/canister//query` can be executed directly. They can only be executed against canisters which are `Running`. +Canister query calls to `/api/v2/canister//query` can be executed directly. They can only be executed against canisters which have a status of `Running` and are also not frozen. During the execution of a query call, a certificate is provided to the canister that is valid, contains a current state tree (or “recent enough”; the specification is currently vague about how old the certificate may be) and reveals the canister’s <>. @@ -3664,13 +3766,9 @@ Conditions:: is_effective_canister_id(E.content, ECID) S.system_time <= Q.ingress_expiry S.canisters[Q.canister_id] ≠ EmptyCanister - S.canister_status[Q.canister_id] = Running + S.canister_status[Q.canister_id] = Running ∧ S.balances[Q.canister_id] >= freezing_limit(S, Q.canister_id) C = S.canisters[Q.canister_id] F = C.module.query_methods[Q.method_name] - Arg = { - data = Q.arg; - caller = Q.sender; - } verify_cert(Cert) lookup(["canister",Q.canister_id,"certified_data"], Cert) = Found S.certified_data[Q.canister_id] lookup(["time"], Cert) = Found S.system_time // or “recent enough” @@ -3679,21 +3777,21 @@ Conditions:: balance = S.balances[Q.canister_id]; freezing_limit = freezing_limit(S, Q.canister_id); certificate = Cert; - status = S.status[Q.receiver]; + status = simple_status(S.canister_status[Q.receiver]); } .... Read response:: -* If `F(Arg, Env) = Trap` then +* If `F(Q.Arg, Q.sender, Env) = Trap trap` then + .... -{status: failed; error: "Query execution trapped"} +{status: rejected; reject_code: CANISTER_ERROR, reject_message: , error_code: } .... -* Else if `F(Arg, Env) = Return (Reject (code, msg))` then +* Else if `F(Q.Arg, Q.sender, Env) = Return {response = Reject (code, msg); …}` then + .... -{status: rejected; reject_code: : reject_message: } +{status: rejected; reject_code: : reject_message: , error_code: } .... -* Else if `F(Arg, Env) = Return (Reply R)` then +* Else if `F(Q.Arg, Q.sender, Env) = Return {response = Reply R; …}` then + .... {status: success; reply: { arg : } } @@ -3726,9 +3824,9 @@ may_read_path(S, _, ["request_status", Rid] · _) = may_read_path(S, _, ["canister", cid, "module_hash"]) = cid == ECID may_read_path(S, _, ["canister", cid, "controllers"]) = cid == ECID may_read_path(S, _, ["canister", cid, "metadata", name]) = - if name ∈ S.canisters[cid].public_custom_sections + if name ∈ dom(S.canisters[cid].public_custom_sections) then cid == ECID - else if name ∈ S.canisters[cid].private_custom_sections + else if name ∈ dom(S.canisters[cid].private_custom_sections) then cid == ECID ∧ RS.sender ∈ S.controllers[cid] else False may_read_path(S, _, _) = False @@ -3757,7 +3855,7 @@ request_status_tree(Received) = request_status_tree(Processing) = { "status": "processing" } request_status_tree(Rejected (code,msg)) = - { "status": "rejected"; "reject_code": code; "reject_message": msg } + { "status": "rejected"; "reject_code": code; "reject_message": msg, error_code: } request_status_tree(Replied arg) = { "status": "replied"; "reply": arg } request_status_tree(Done) = @@ -3804,13 +3902,13 @@ Callback = { We can model the execution of WebAssembly functions as stateful functions that have access to the WebAssembly store. In order to also model the behavior of the system imports, which have access to additional data structures, we extend the state as follows: .... Params = { - data : NoData | Blob; + arg : NoArg | Blob; caller : NoCaller | Principal; reject_code : 0 | SYS_FATAL | SYS_TRANSIENT | …; reject_message : Text; sysenv : Env; cycles_refunded : Nat; - method_name : Text; + method_name : NoText | Text; } ExecutionState = { wasm_state : WasmState; @@ -3818,6 +3916,7 @@ ExecutionState = { response : NoResponse | Response; cycles_accepted : Nat; cycles_available : Nat; + cycles_used : Nat; balance : Funds; reply_params : { arg : Blob }; pending_call : MethodCall | NoPendingCall; @@ -3842,11 +3941,12 @@ Finally we can specify the abstract `CanisterModule` that models a concrete WebA + .... empty_params = { - data = NoData; + arg = NoArg; caller = NoCaller; reject_code = 0; reject_message = ""; cycles_refunded = 0; + method_name = NoText; } empty_execution_state = { @@ -3855,6 +3955,7 @@ empty_execution_state = { response = NoResponse; cycles_accepted = 0; cycles_available = 0; + cycles_used = 0; balance = 0; reply_params = { arg = "" }; pending_call = NoPendingCall; @@ -3870,24 +3971,24 @@ empty_execution_state = { If the WebAssembly module does not export a function called under the name `canister_init`, then the argument blob is ignored and the `initial_wasm_store` is returned: + .... -init = λ (self_id, arg, sysenv) → - Return { store = initial_wasm_store; self_id = self_id; stable_mem = "" } +init = λ (self_id, arg, caller, sysenv) → + Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" }; cycles_used = 0;} .... + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_init`, it is + .... -init = λ (self_id, arg, sysenv) → +init = λ (self_id, arg, caller, sysenv) → let es = ref {empty_execution_state with wasm_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" } - params = empty_params with { data = arg.data; caller = arg.caller; sysenv } + params = empty_params with { arg = arg; caller = caller; sysenv } balance = sysenv.balance } - try func() with Trap then Trap - if es.performed_calls ≠ [] then Trap - if es.response ≠ NoResponse then Trap - if es.ingress_filter ≠ Reject then Trap - Return es.wasm_state + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} + Return {new_state = es.wasm_state; cycles_used = es.cycles_used;} .... + This formulation checks afterwards that the system calls `call.perform` or `msg.reply` were not invoked; an implementation can of course trap as soon as these system calls are invoked. @@ -3897,7 +3998,7 @@ This formulation checks afterwards that the system calls `call.perform` or `msg. If the WebAssembly module does not export a function called under the name `canister_pre_upgrade`, then it simply returns the stable memory: + .... -pre_upgrade = λ (old_state, caller, sysenv) → Return old_state.stable_mem +pre_upgrade = λ (old_state, caller, sysenv) → Return {stable_memory = old_state.stable_mem; cycles_used = 0;} .... + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_pre_upgrade`, it is @@ -3909,11 +4010,11 @@ pre_upgrade = λ (old_state, caller, sysenv) → params = { empty_params with caller = caller; sysenv } balance = sysenv.balance } - try func() with Trap then Trap - if es.performed_calls ≠ [] then Trap - if es.response ≠ NoResponse then Trap - if es.ingress_filter ≠ Reject then Trap - Return es.wasm_state.stable_mem + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} + Return {stable_memory = es.wasm_state.stable_mem; cycles_used = es.cycles_used;} .... @@ -3922,62 +4023,66 @@ pre_upgrade = λ (old_state, caller, sysenv) → If the WebAssembly module does not export a function called under the name `canister_post_upgrade`, then the argument blob is ignored and the `initial_wasm_store` is returned: + .... -post_upgrade = λ (self_id, stable_mem, arg, sysenv) → - Return { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem } +post_upgrade = λ (self_id, stable_mem, arg, caller, sysenv) → + Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem }; cycles_used = 0;} .... + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_post_upgrade`, it is + .... -post_upgrade = λ (self_id, stable_mem, arg, sysenv) → +post_upgrade = λ (self_id, stable_mem, arg, caller, sysenv) → let es = ref {empty_execution_state with wasm_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem } - params = { empty_params with data = arg.data; caller = arg.caller; sysenv } + params = { empty_params with arg = arg; caller = caller; sysenv } balance = sysenv.balance } - try func() with Trap then Trap - if es.performed_calls ≠ [] then Trap - if es.response ≠ NoResponse then Trap - if es.ingress_filter ≠ Reject then Trap - Return es.wasm_state + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} + Return {new_state = es.wasm_state; cycles_used = es.cycles_used;} .... * The partial map `update_methods` of the `CanisterModule` is defined for all method names `method` for which the WebAssembly program exports a function `func` named `canister_update `, and has value + .... -update_methods[method] = λ (arg, sysenv, available) → λ wasm_state → +update_methods[method] = λ (arg, caller, sysenv, available) → λ wasm_state → let es = ref {empty_execution_state with wasm_state = wasm_state; - params = empty_params with { data = arg.data; caller = arg.caller; sysenv } + params = empty_params with { arg = arg; caller = caller; sysenv } balance = sysenv.balance - cycles_available = arg.cycles; + cycles_available = available; } - try func() with Trap then Trap - if es.ingress_filter ≠ Reject then Trap + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} Return { new_state = es.wasm_state; new_calls = es.calls; response = es.response; cycles_accepted = es.cycles_accepted; - new_certified_data = es.new_certified_data + cycles_used = es.cycles_used; + new_certified_data = es.new_certified_data; } .... * The partial map `query_methods` of the `CanisterModule` is defined for all method names `method` for which the WebAssembly program exports a function `func` named `canister_query `, and has value + .... -query_methods[method] = λ (arg, sysenv) → λ wasm_state → +query_methods[method] = λ (arg, caller, sysenv) → λ wasm_state → let es = ref {empty_execution_state with wasm_state = wasm_state; - params = empty_params with { data = arg.data; caller = arg.caller; sysenv } + params = empty_params with { arg = arg; caller = caller; sysenv } balance = sysenv.balance } - try func() with Trap then Trap - if es.cycles_accepted ≠ 0 then Trap - if es.calls ≠ () then Trap - if es.ingress_filter ≠ Reject then Trap - if es.response = NoResponse then Trap - Return es.response; + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used;} + if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} + if es.response = NoResponse then Trap {cycles_used = es.cycles_used;} + Return { + response = es.response; + cycles_used = es.cycles_used; + } .... + This formulation checks afterwards that the system call `ic0.call_perform` was not invoked; an implementation can of course trap already when these system calls have been invoked. @@ -3990,18 +4095,21 @@ By construction, the (possibly modified) `es.wasm_state` is discarded. heartbeat = λ (sysenv) → λ wasm_state → let es = ref {empty_execution_state with wasm_state = wasm_state; - params = empty_params with { data = NoData; caller = NoCaller; sysenv } + params = empty_params with { arg = NoArg; caller = NoCaller; sysenv } balance = sysenv.balance } - try func() with Trap then Trap - if es.cycles_accepted ≠ 0 then Trap - if es.ingress_filter ≠ Reject then Trap - if es.response ≠ NoResponse then Trap - Return es.wasm_state; + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + cycles_used = es.cycles_used; + } .... otherwise it is .... -heartbeat = λ (sysenv) → λ wasm_state → Trap +heartbeat = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} .... * The function `callbacks` of the `CanisterModule` is defined as follows @@ -4019,40 +4127,46 @@ callbacks = λ(callbacks, response, refunded_cycles, sysenv, available) → λ w Reject (reject_code, reject_message)-> (callbacks.on_reject.fun, callbacks.on_reject.env, { params0 with reject_code; reject_message}) + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = params; + balance = sysenv.balance; + cycles_available = available; + } try if fun > |es.wasm_state.store.table| then Trap let func = es.wasm_state.store.table[fun] if typeof(func) ≠ func (i32) -> () then Trap - let es = ref {empty_execution_state with - wasm_state = wasm_state; - params = params; - balance = sysenv.balance; - cycles_available = available; - } func(env) Return { new_state = es.wasm_state; new_calls = es.calls; response = es.response; cycles_accepted = es.cycles_accepted; + cycles_used = es.cycles_used; new_certified_data = es.certified_data; } with Trap - if callbacks.on_cleanup = NoClosure then Trap - if callbacks.on_cleanup.fun > |es.wasm_state.store.table| then Trap + if callbacks.on_cleanup = NoClosure then Trap {cycles_used = es.cycles_used;} + if callbacks.on_cleanup.fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} let func = es.wasm_state.store.table[callbacks.on_cleanup.fun] - if typeof(func) ≠ func (i32) -> () then Trap + if typeof(func) ≠ func (i32) -> () then Trap {cycles_used = es.cycles_used;} - let es = ref { empty_execution_state with - wasm_state; + let es' = ref { empty_execution_state with + wasm_state = wasm_state; } - func(callbacks.on_cleanup.env) + try func(callbacks.on_cleanup.env) with Trap then Trap {cycles_used = es.cycles_used + es'.cycles_used;} + if es'.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used + es'.cycles_used;} + if es'.calls ≠ [] then Trap {cycles_used = es.cycles_used + es'.cycles_used;} + if es'.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used + es'.cycles_used;} + if es'.response ≠ NoResponse then Trap {cycles_used = es.cycles_used + es'.cycles_used;} Return { - new_state = es.wasm_state; + new_state = es'.wasm_state; new_calls = []; response = NoResponse; cycles_accepted = 0; + cycles_used = es.cycles_used + es'.cycles_used; } .... + @@ -4064,28 +4178,28 @@ present) is executed, and the canister has the chance to update its state. If the WebAssembly module does not export a function called under the name `canister_inspect_message`, then access is always granted: + .... -inspect_message = λ (method_name, wasm_state, arg, sysenv) → - Return Accept +inspect_message = λ (method_name, wasm_state, arg, caller, sysenv) → + Return {status = Accept; cycles_used = 0;} .... Otherwise, if the WebAssembly module exports a function `func` under the name `canister_inspect_message`, it is + .... -inspect_message = λ (method_name, wasm_state, arg, sysenv) → +inspect_message = λ (method_name, wasm_state, arg, caller, sysenv) → let es = ref {empty_execution_state with wasm_state = wasm_state; params = empty_params with { - data = arg.data; - caller = arg.caller; - method_name = arg.method_name; + arg = arg; + caller = caller; + method_name = method_name; sysenv } balance = sysenv.balance; cycles_available = 0; // ingress requests have no funds } - try func() with Trap then Trap - if es.calls ≠ () then Trap - if es.response ≠ NoResponse then Trap - Return es.ingress_filter; + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + Return {status = es.ingress_filter; cycles_used = es.cycles_used;}; .... ==== Helper functions @@ -4094,12 +4208,12 @@ In the following section, we use the these helper functions .... copy_to_canister(dst : i32, offset : i32, size : i32, data : blob) = - if offset+size > |data| then Trap - if dst+size > |es.wasm_state.store.mem| then Trap + if offset+size > |data| then Trap {cycles_used = es.cycles_used;} + if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} es.wasm_state.store.mem[dst..dst+size] := data[offset..offset+size] copy_from_canister(src : i32, size : i32) blob = - if src+size > |es.wasm_state.store.mem| then Trap + if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} return es.wasm_state.store.mem[src..src+size] .... @@ -4108,7 +4222,7 @@ Cycles are represented by 128-bit values so they require 16 bytes of memory. .... copy_cycles_to_canister(dst : i32, data : blob) = let size = 16; - if dst+size > |es.wasm_state.store.mem| then Trap + if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} es.wasm_state.store.mem[dst..dst+size] := data[0..size] .... @@ -4124,7 +4238,7 @@ ic0.msg_arg_data_size() : i32 = return |es.params.arg| ic0.msg_arg_data_copy(dst:i32, offset:i32, size:i32) = - copy_to_canister(dst, offset, size, es.param.arg) + copy_to_canister(dst, offset, size, es.params.arg) ic0.msg_caller_size() : i32 = return |es.params.caller| @@ -4142,21 +4256,21 @@ ic0.msg_reject_msg_copy(dst:i32, offset:i32, size:i32) : i32 = copy_to_canister(dst, offset, size, es.params.reject_msg) ic0.msg_reply_data_append(src : i32, size : i32) = - if es.response ≠ NoResponse then Trap + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.reply_params.arg := es.reply_params.arg · copy_from_canister(src, size) ic0.msg_reply() = - if es.response ≠ NoResponse then Trap + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.response := Reply (es.reply_params.arg) es.cycles_available := 0 ic0.msg_reject(src : i32, size : i32) = - if es.response ≠ NoResponse then Trap + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.response := Reject (CANISTER_REJECT, copy_from_canister(src, size)) es.cycles_available := 0 ic0.msg_cycles_available() : i64 = - if es.cycles_available >= 2^64^ then Trap + if es.cycles_available >= 2^64^ then Trap {cycles_used = es.cycles_used;} return es.cycles_available ic0.msg_cycles_available128(dst : i32) = @@ -4164,7 +4278,7 @@ ic0.msg_cycles_available128(dst : i32) = copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) ic0.msg_cycles_refunded() : i64 = - if es.params.cycles_refunded >= 2^64^ then Trap + if es.params.cycles_refunded >= 2^64^ then Trap {cycles_used = es.cycles_used;} return es.params.cycles_refunded ic0.msg_cycles_refunded128(dst : i32) = @@ -4172,7 +4286,7 @@ ic0.msg_cycles_refunded128(dst : i32) = copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) ic0.accept_message() = - if es.ingress_filter = Accept then Trap + if es.ingress_filter = Accept then Trap {cycles_used = es.cycles_used;} es.ingress_filter = Accept ic0.msg_method_name_size() : i32 = @@ -4203,7 +4317,7 @@ ic0.canister_self_copy(dst:i32, offset:i32, size:i32) = copy_to_canister(dst, offset, size, es.wasm_state.self_id) ic0.canister_cycle_balance() : i64 = - if es.balance >= 2^64^ then Trap + if es.balance >= 2^64^ then Trap {cycles_used = es.cycles_used;} return es.balance ic0.canister_cycles_balance128(dst : i32) = @@ -4228,17 +4342,17 @@ ic0.call_new( ) = discard_pending_call() - if es.balance < MAX_CYCLES_PER_RESPONSE then Trap + if es.balance < MAX_CYCLES_PER_RESPONSE then Trap {cycles_used = es.cycles_used;} es.balance := es.balance - MAX_CYCLES_PER_RESPONSE callee := copy_from_canister(callee_src, callee_size); method_name := copy_from_canister(name_src, name_size); - if reply_fun > |es.wasm_state.store.table| then Trap - if typeof(es.wasm_state.store.table[reply_fun]) ≠ func (anyref, i32) -> () then Trap + if reply_fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} + if typeof(es.wasm_state.store.table[reply_fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} - if reject_fun > |es.wasm_state.store.table| then Trap - if typeof(es.wasm_state.store.table[reject_fun]) ≠ func (anyref, i32) -> () then Trap + if reject_fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} + if typeof(es.wasm_state.store.table[reject_fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} es.pending_call = MethodCall { callee = callee; @@ -4253,33 +4367,33 @@ ic0.call_new( } ic0.call_data_append (src : i32, size : i32) = - if es.pending_call = NoPendingCall then Trap + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} es.pending_call.arg := es.pending_call.arg · copy_from_canister(src, size) ic0.call_on_cleanup (fun : i32, env : i32) = - if fun > |es.wasm_state.store.table| then Trap - if typeof(es.wasm_state.store.table[fun]) ≠ func (anyref, i32) -> () then Trap - if es.pending_call = NoPendingCall then Trap - if es.pending_call.callback.on_cleanup ≠ NoClosure then Trap + if fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} + if typeof(es.wasm_state.store.table[fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + if es.pending_call.callback.on_cleanup ≠ NoClosure then Trap {cycles_used = es.cycles_used;} es.pending_call.callback.on_cleanup := Closure { fun = fun; env = env} ic0.call_cycles_add(amount : i64) = - if es.pending_call = NoPendingCall then Trap - if es.balance < amount then Trap + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + if es.balance < amount then Trap {cycles_used = es.cycles_used;} es.balance := es.balance - amount es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount ic0.call_cycles_add128(amount_high : i64, amount_low : i64) = let amount = amount_high * 2^64^ + amount_low - if es.pending_call = NoPendingCall then Trap - if es.balance < amount then Trap + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + if es.balance < amount then Trap {cycles_used = es.cycles_used;} es.balance := es.balance - amount es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount ic0.call_peform() : ( err_code : i32 ) = - if es.pending_call = NoPendingCall then Trap + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} // are we below the threezing threshold? // Or maybe the system has other reasons to not perform this @@ -4299,12 +4413,12 @@ discard_pending_call() = es.pending_call := NoPendingCall ic0.stable_size() : (page_count : i32) = - if |es.wasm_state.store.mem| > 2^32^ then Trap + if |es.wasm_state.store.mem| > 2^32^ then Trap {cycles_used = es.cycles_used;} page_count := |es.wasm_state.stable_mem| / 64k return page_count ic0.stable_grow(new_pages : i32) : (old_page_count : i32) = - if |es.wasm_state.store.mem| > 2^32^ then Trap + if |es.wasm_state.store.mem| > 2^32^ then Trap {cycles_used = es.cycles_used;} if arbitrary() then return -1 else old_size := |es.wasm_state.stable_mem| / 64k @@ -4314,16 +4428,16 @@ ic0.stable_grow(new_pages : i32) : (old_page_count : i32) = return old_size ic0.stable_write(offset : i32, src : i32, size : i32) - if |es.wasm_state.store.mem| > 2^32^ then Trap - if src+size > |es.wasm_state.store.mem| then Trap - if offset+size > |es.wasm_state.stable_mem| then Trap + if |es.wasm_state.store.mem| > 2^32^ then Trap {cycles_used = es.cycles_used;} + if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] ic0.stable_read(dst : i32, offset : i32, size : i32) - if |es.wasm_state.store.mem| > 2^32^ then Trap - if offset+size > |es.wasm_state.stable_mem| then Trap - if dst+size > |es.wasm_state.store.mem| then Trap + if |es.wasm_state.store.mem| > 2^32^ then Trap {cycles_used = es.cycles_used;} + if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} + if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] @@ -4340,14 +4454,14 @@ ic0.stable64_grow(new_pages : i64) : (old_page_count : i64) = return old_size ic0.stable64_write(offset : i64, src : i64, size : i64) - if src+size > |es.wasm_state.store.mem| then Trap - if offset+size > |es.wasm_state.stable_mem| then Trap + if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] ic0.stable64_read(dst : i64, offset : i64, size : i64) - if offset+size > |es.wasm_state.stable_mem| then Trap - if dst+size > |es.wasm_state.store.mem| then Trap + if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} + if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] @@ -4363,19 +4477,21 @@ ic0.data_certificate_present() : i32 = else return 1 ic0.data_certificate_size() : i32 = - if es.params.sysenv.certificate = NoCertificate then Trap + if es.params.sysenv.certificate = NoCertificate then Trap {cycles_used = es.cycles_used;} return |es.params.sysenv.certificate| ic0.data_certificate_copy(dst: i32, offset: i32, size: i32) = - if es.params.sysenv.certificate = NoCertificate then Trap + if es.params.sysenv.certificate = NoCertificate then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.sysenv.certificate) +ic0.performance_counter(counter_type : i32) : i64 = + arbitrary() ic0.debug_print(src : i32, size : i32) = return ic0.trap(src : i32, size : i32) = - Trap + Trap {cycles_used = es.cycles_used;} .... diff --git a/spec/requests.cddl b/spec/requests.cddl index 21d1f9f28..f964bafad 100644 --- a/spec/requests.cddl +++ b/spec/requests.cddl @@ -70,6 +70,7 @@ query-response = tagged<{ status: "rejected" reject_code: unsigned reject_message: text + ? error_code: text }> call-reply = { diff --git a/theories/IC.thy b/theories/IC.thy new file mode 100644 index 000000000..4e542f128 --- /dev/null +++ b/theories/IC.thy @@ -0,0 +1,2844 @@ +theory IC + imports "HOL-Library.AList" +begin + +(* General helper lemmas *) + +lemma in_set_updD: "x \ set (xs[n := z]) \ x \ set xs \ x = z" + by (meson insert_iff set_update_subset_insert subsetD) + +(* Partial maps *) + +typedef ('a, 'b) list_map = "{f :: ('a \ 'b) list. distinct (map fst f)}" + by (auto intro: exI[of _ "[]"]) + +setup_lifting type_definition_list_map + +lift_bnf (dead 'k, set: 'v) list_map [wits: "[] :: ('k \ 'v) list"] for map: map rel: rel + by auto + +hide_const (open) map set rel + +lift_definition list_map_dom :: "('a, 'b) list_map \ 'a set" is + "set \ map fst" . + +lift_definition list_map_vals :: "('a, 'b) list_map \ 'b list" is + "map snd" . + +lift_definition list_map_range :: "('a, 'b) list_map \ 'b set" is + "set \ map snd" . + +lemma in_set_map_filter_vals: "z \ set (List.map_filter g (list_map_vals f)) \ \w \ list_map_range f. g w = Some z" + by transfer (force simp: List.map_filter_def) + +lift_definition list_map_sum_vals :: "('b \ nat) \ ('a, 'b) list_map \ nat" is + "\g. sum_list \ (map (g \ snd))" . + +lift_definition list_map_get :: "('a, 'b) list_map \ 'a \ 'b option" is + "map_of" . + +lemma list_map_get_dom[dest]: "x \ list_map_dom f \ list_map_get f x = None \ False" + by transfer auto + +lemma list_map_get_range: "list_map_get f x = Some y \ y \ list_map_range f" + by transfer force + +lift_definition list_map_set :: "('a, 'b) list_map \ 'a \ 'b \ ('a, 'b) list_map" is + "\f x y. AList.update x y f" + by (rule distinct_update) + +lemma list_map_get_set: "list_map_get (list_map_set f x y) z = (if x = z then Some y else list_map_get f z)" + by transfer (auto simp add: update_Some_unfold update_conv) + +lemma list_map_dom_set[simp]: "list_map_dom (list_map_set f x y) = list_map_dom f \ {x}" + by transfer (auto simp add: dom_update) + +lemma list_map_range_setD: "z \ list_map_range (list_map_set f x y) \ z \ list_map_range f \ z = y" + apply transfer + apply auto + apply (metis distinct_update image_iff map_of_eq_Some_iff snd_eqD update_Some_unfold) + done + +lift_definition list_map_del :: "('a, 'b) list_map \ 'a \ ('a, 'b) list_map" is + "\f x. AList.delete x f" + by (rule distinct_delete) + +lemma list_map_get_del: "list_map_get (list_map_del f x) z = (if x = z then None else list_map_get f z)" + by transfer (auto simp add: delete_conv') + +lemma list_map_dom_del[simp]: "list_map_dom (list_map_del f x) = list_map_dom f - {x}" + by transfer (simp add: delete_keys) + +lemma list_map_range_del: "z \ list_map_range (list_map_del f x) \ z \ list_map_range f" + apply transfer + apply auto + apply (metis Some_eq_map_of_iff delete_notin_dom distinct_delete fst_eqD imageI map_of_delete snd_eqD) + done + +lift_definition list_map_empty :: "('a, 'b) list_map" is "[]" + by auto + +lemma list_map_get_empty[simp]: "list_map_get list_map_empty x = None" + by transfer auto + +lemma list_map_empty_dom[simp]: "list_map_dom list_map_empty = {}" + by transfer auto + +lemma list_map_empty_range[simp]: "list_map_range list_map_empty = {}" + by transfer auto + +lift_definition list_map_init :: "('a \ 'b) list \ ('a, 'b) list_map" is + "\xys. AList.updates (map fst xys) (map snd xys) []" + using distinct_updates + by force + +lift_definition list_map_map :: "('b \ 'c) \ ('a, 'b) list_map \ ('a, 'c) list_map" is + "\f xs. map (\(k, v). (k, f v)) xs" + by (auto simp: comp_def case_prod_beta) + +lemma list_map_dom_map_map[simp]: "list_map_dom (list_map_map g f) = list_map_dom f" + by transfer force + +lemma list_map_range_map_map[simp]: "list_map_range (list_map_map g f) = g ` list_map_range f" + by transfer force + +lemma list_map_sum_vals_split: "(\ctxt. ctxt \ list_map_range xs \ f (g ctxt) \ f ctxt) \ list_map_sum_vals f xs = + list_map_sum_vals id + (list_map_map (\ctxt. if P ctxt then f ctxt - f (g ctxt) else 0) xs) + + list_map_sum_vals f + (list_map_map (\ctxt. if P ctxt then g ctxt else ctxt) xs)" + apply (transfer fixing: f g P) + subgoal for xs + by (induction xs) auto + done + +lemma list_map_sum_vals_filter: + assumes "\b. b \ list_map_range xs \ P b = None \ f b = 0" "\b y. b \ list_map_range xs \ P b = Some y \ f b = g y" + shows "list_map_sum_vals id (list_map_map f xs) = sum_list (map g (List.map_filter P (list_map_vals xs)))" + using assms + apply (transfer fixing: f g P) + subgoal for xs + by (induction xs) (auto simp: List.map_filter_def) + done + +lemma list_map_sum_in_ge_aux: + fixes g :: "'a \ nat" + shows "distinct (map fst f) \ map_of f x = Some y \ g y \ sum_list (map g (map snd f))" + by (induction f) (auto split: if_splits) + +lemma list_map_sum_in_ge: "list_map_get f x = Some y \ list_map_sum_vals g f \ g y" + apply transfer + using list_map_sum_in_ge_aux[OF _ map_of_is_SomeI] + by fastforce + +lemma list_map_sum_in_aux: fixes g :: "'a \ nat" + shows "distinct (map fst f) \ map_of f x = Some y \ + sum_list (map (g \ snd) (AList.update x y' f)) = sum_list (map (g \ snd) f) + g y' - g y" + apply (induction f) + apply auto[1] + subgoal for a f + using list_map_sum_in_ge_aux[OF _ map_of_is_SomeI, of f x y g] + by auto + done + +lemma list_map_sum_in: "list_map_get f x = Some y \ list_map_sum_vals g (list_map_set f x y') = list_map_sum_vals g f + g y' - g y" + apply transfer + using list_map_sum_in_aux + by fastforce + +lemma list_map_sum_out_aux: + "x \ set (map fst f) \ sum_list (map (g \ snd) (AList.update x y f)) = sum_list (map (g \ snd) f) + g y" + by (induction f) (auto simp: add.assoc) + +lemma list_map_sum_out: "x \ list_map_dom f \ list_map_sum_vals g (list_map_set f x y) = list_map_sum_vals g f + g y" + apply transfer + using list_map_sum_out_aux + by fastforce + +lemma list_map_del_sum_aux: + fixes g :: "'a \ nat" + shows "distinct (map fst f) \ map_of f x = Some y \ sum_list (map (g \ snd) f) = sum_list (map (g \ snd) (AList.delete x f)) + g y" + by (induction f) auto + +lemma list_map_del_sum: "list_map_get f x = Some y \ list_map_sum_vals g f = list_map_sum_vals g (list_map_del f x) + g y" + apply transfer + using list_map_del_sum_aux + by fastforce + +(* Abstract behaviour *) + +(* Abstract canisters *) + +type_synonym 's method_name = 's + +type_synonym 'b arg = 'b +type_synonym 'p caller_id = 'p + +type_synonym timestamp = nat +datatype status = Running | Stopping | Stopped +record ('b) env = + time :: timestamp + balance :: nat + freezing_limit :: nat + certificate :: "'b option" + status :: status + +type_synonym reject_code = nat +datatype ('b, 's) response = + Reply "'b" +| Reject reject_code 's +record ('p, 'canid, 's, 'b, 'c) method_call = + callee :: 'canid + method_name :: "'s method_name" + arg :: 'b + transferred_cycles :: nat + callback :: 'c + +record 'x cycles_return = + return :: 'x + cycles_used :: nat +type_synonym trap_return = "unit cycles_return" +record ('w, 'p, 'canid, 's, 'b, 'c) update_return = + new_state :: 'w + new_calls :: "('p, 'canid, 's, 'b, 'c) method_call list" + new_certified_data :: "'b option" + response :: "('b, 's) response option" + cycles_accepted :: nat + cycles_used :: nat +record ('b, 's) query_return = + response :: "('b, 's) response" + cycles_used :: nat +record 'w heartbeat_return = + new_state :: 'w + cycles_used :: nat +type_synonym ('w, 'p, 'canid, 's, 'b, 'c) update_func = "'w \ trap_return + ('w, 'p, 'canid, 's, 'b, 'c) update_return" +type_synonym ('w, 'b, 's) query_func = "'w \ trap_return + ('b, 's) query_return" +type_synonym 'w heartbeat_func = "'w \ trap_return + 'w heartbeat_return" + +type_synonym available_cycles = nat +type_synonym refunded_cycles = nat + +datatype inspect_method_result = Accept | Reject +record ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module_rec = + init :: "'canid \ 'b arg \ 'p caller_id \ 'b env \ trap_return + 'w cycles_return" + pre_upgrade :: "'w \ 'p \ 'b env \ trap_return + 'sm cycles_return" + post_upgrade :: "'canid \ 'sm \ 'b arg \ 'p caller_id \ 'b env \ trap_return + 'w cycles_return" + update_methods :: "('s method_name, ('b arg \ 'p caller_id \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func) list_map" + query_methods :: "('s method_name, ('b arg \ 'p caller_id \ 'b env) \ ('w, 'b, 's) query_func) list_map" + heartbeat :: "'b env \ 'w heartbeat_func" + callbacks :: "('c \ ('b, 's) response \ refunded_cycles \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" + inspect_message :: "('s method_name \ 'w \ 'b arg \ 'p caller_id \ 'b env) \ trap_return + inspect_method_result cycles_return" +typedef ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module = + "{m :: ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module_rec. list_map_dom (update_methods m) \ list_map_dom (query_methods m) = {}}" + by (auto intro: exI[of _ "\init = undefined, pre_upgrade = undefined, post_upgrade = undefined, + update_methods = list_map_empty, query_methods = list_map_empty, heartbeat = undefined, callbacks = undefined, + inspect_message = undefined\"]) + +setup_lifting type_definition_canister_module + +lift_definition canister_module_init :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'canid \ 'b arg \ 'p \ 'b env \ trap_return + 'w cycles_return" is "init" . +lift_definition canister_module_pre_upgrade :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'w \ 'p \ 'b env \ trap_return + 'sm cycles_return" is pre_upgrade . +lift_definition canister_module_post_upgrade :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'canid \ 'sm \ 'b arg \ 'p \ 'b env \ trap_return + 'w cycles_return" is post_upgrade . +lift_definition canister_module_update_methods :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('s, ('b arg \ 'p \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func) list_map" is update_methods . +lift_definition canister_module_query_methods :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('s, ('b arg \ 'p \ 'b env) \ ('w, 'b, 's) query_func) list_map" is query_methods . +lift_definition canister_module_heartbeat :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'b env \ 'w heartbeat_func" is heartbeat . +lift_definition canister_module_callbacks :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('c \ ('b, 's) response \ refunded_cycles \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" is callbacks . +lift_definition canister_module_inspect_message :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('s \ 'w \ 'b arg \ 'p \ 'b env) \ trap_return + inspect_method_result cycles_return" is inspect_message . + +lift_definition dispatch_method :: "'s \ ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ + ((('b arg \ 'p \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func) + (('b arg \ 'p \ 'b env) \ ('w, 'b, 's) query_func)) option" is + "\f m. case list_map_get (update_methods m) f of Some f' \ Some (Inl f') | None \ (case list_map_get (query_methods m) f of Some f' \ Some (Inr f') | None \ None)" . + +(* Call contexts *) + +record ('b, 'p, 'uid, 'canid, 's) request = + nonce :: 'b + ingress_expiry :: nat + sender :: 'uid + canister_id :: 'canid + method_name :: 's + arg :: 'b +datatype ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin = + From_user "('b, 'p, 'uid, 'canid, 's) request" +| From_canister "'cid" "'c" +| From_heartbeat +record ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt_rep = + canister :: 'canid + origin :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin" + needs_to_respond :: bool + deleted :: bool + available_cycles :: nat + +typedef ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt = "{ctxt :: ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt_rep. + (deleted ctxt \ \needs_to_respond ctxt) \ (\needs_to_respond ctxt \ available_cycles ctxt = 0)}" + by (auto intro: exI[of _ "\canister = undefined, origin = undefined, needs_to_respond = True, deleted = False, available_cycles = 0\"]) + +setup_lifting type_definition_call_ctxt + +lift_definition call_ctxt_canister :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ 'canid" is "canister" . +lift_definition call_ctxt_origin :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin" is "origin" . +lift_definition call_ctxt_needs_to_respond :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ bool" is needs_to_respond . +lift_definition call_ctxt_deleted :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ bool" is deleted . +lift_definition call_ctxt_available_cycles :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ nat" is available_cycles . + +lemma call_ctxt_not_needs_to_respond_available_cycles: "\call_ctxt_needs_to_respond x2 \ call_ctxt_available_cycles x2 = 0" + by transfer auto + +lift_definition call_ctxt_respond :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt" is + "\ctxt. ctxt\needs_to_respond := False, available_cycles := 0\" + by auto + +lemma call_ctxt_respond_origin[simp]: "call_ctxt_origin (call_ctxt_respond ctxt) = call_ctxt_origin ctxt" + by transfer auto + +lemma call_ctxt_respond_needs_to_respond[dest]: "call_ctxt_needs_to_respond (call_ctxt_respond ctxt) \ False" + by transfer auto + +lemma call_ctxt_respond_available_cycles[simp]: "call_ctxt_available_cycles (call_ctxt_respond ctxt) = 0" + by transfer auto + +lift_definition call_ctxt_deduct_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt" is + "\n ctxt. ctxt\available_cycles := available_cycles ctxt - n\" + by auto + +lemma call_ctxt_deduct_cycles_origin[simp]: "call_ctxt_origin (call_ctxt_deduct_cycles n ctxt) = call_ctxt_origin ctxt" + by transfer auto + +lemma call_ctxt_deduct_cycles_needs_to_respond[simp]: "call_ctxt_needs_to_respond (call_ctxt_deduct_cycles n ctxt) = call_ctxt_needs_to_respond ctxt" + by transfer auto + +lemma call_ctxt_deduct_cycles_available_cycles[simp]: "call_ctxt_available_cycles (call_ctxt_deduct_cycles n ctxt) = call_ctxt_available_cycles ctxt - n" + by transfer auto + +lift_definition call_ctxt_delete :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt" is + "\ctxt. ctxt\deleted := True, needs_to_respond := False, available_cycles := 0\" + by auto + +lemma call_ctxt_delete_needs_to_respond[simp]: "call_ctxt_needs_to_respond (call_ctxt_delete ctxt) = False" + by transfer auto + +(* Calls and Messages *) + +datatype 'canid queue_origin = System | Canister 'canid +datatype 'canid queue = Unordered | Queue "'canid queue_origin" 'canid +datatype ('s, 'p, 'b, 'c) entry_point = + Public_method "'s method_name" "'p" "'b" +| Callback "'c" "('b, 's) response" "refunded_cycles" +| Heartbeat + +datatype ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message = + Call_message "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin" 'p 'canid 's 'b nat "'canid queue" +| Func_message 'cid 'canid "('s, 'p, 'b, 'c) entry_point" "'canid queue" +| Response_message "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin" "('b, 's) response" nat + +(* API requests *) + +type_synonym 'b path = "'b list" +record ('b, 'uid) StateRead = + nonce :: 'b + ingress_expiry :: nat + sender :: 'uid + paths :: "'b path list" +record ('b, 'uid, 'canid, 's) CanisterQuery = + nonce :: 'b + ingress_expiry :: nat + sender :: 'uid + canister_id :: 'canid + method_name :: 's + arg :: 'b +type_synonym ('b, 'uid, 'canid, 's) APIReadRequest = "('b, 'uid) StateRead + ('b, 'uid, 'canid, 's) CanisterQuery" + +record ('p, 'canid, 'pk, 'sig) delegation = + pubkey :: 'pk + targets :: "'canid list option" + senders :: "'p list option" + expiration :: timestamp +record ('p, 'canid, 'pk, 'sig) signed_delegation = + delegation :: "('p, 'canid, 'pk, 'sig) delegation" + signature :: "'sig" + +record ('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope = + content :: "('b, 'p, 'uid, 'canid, 's) request + ('b, 'uid, 'canid, 's) APIReadRequest" + sender_pubkey :: "'pk option" + sender_sig :: "'sig option" + sender_delegation :: "('p, 'canid, 'pk, 'sig) delegation list" + +datatype ('b, 's) request_status = Received | Processing | Rejected reject_code 's | Replied 'b | Done + +(* The system state *) + +record ('p, 'canid, 'b, 'w, 'sm, 'c, 's) can_state_rec = + wasm_state :: 'w + module :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module" + raw_module :: 'b + public_custom_sections :: "('s, 'b) list_map" + private_custom_sections :: "('s, 'b) list_map" +type_synonym ('p, 'canid, 'b, 'w, 'sm, 'c, 's) can_state = "('p, 'canid, 'b, 'w, 'sm, 'c, 's) can_state_rec option" +datatype ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status = Running | Stopping "(('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin \ nat) list" | Stopped +record ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic = + requests :: "(('b, 'p, 'uid, 'canid, 's) request, ('b, 's) request_status) list_map" + canisters :: "('canid, ('p, 'canid, 'b, 'w, 'sm, 'c, 's) can_state) list_map" + controllers :: "('canid, 'p set) list_map" + freezing_threshold :: "('canid, nat) list_map" + canister_status :: "('canid, ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status) list_map" + time :: "('canid, timestamp) list_map" + balances :: "('canid, nat) list_map" + certified_data :: "('canid, 'b) list_map" + system_time :: timestamp + call_contexts :: "('cid, ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt) list_map" + messages :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message list" + root_key :: 'pk + +fun simple_status :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status \ status" where + "simple_status can_status.Running = status.Running" +| "simple_status (can_status.Stopping _) = status.Stopping" +| "simple_status can_status.Stopped = status.Stopped" + +(* Initial state *) + +definition initial_ic :: "nat \ 'pk \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "initial_ic t pk = \requests = list_map_empty, + canisters = list_map_empty, + controllers = list_map_empty, + freezing_threshold = list_map_empty, + canister_status = list_map_empty, + time = list_map_empty, + balances = list_map_empty, + certified_data = list_map_empty, + system_time = t, + call_contexts = list_map_empty, + messages = [], + root_key = pk\" + +(* Invariants *) + +definition ic_can_status_inv :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status set \ 'cid set \ bool" where + "ic_can_status_inv st c = (\can_status \ st. + case can_status of Stopping os \ \(orig, cycles) \ set os. (case orig of + From_canister ctxt_id _ \ ctxt_id \ c + | _ \ True) + | _ \ True)" + +lemma ic_can_status_inv_mono1: "ic_can_status_inv x y \ + z \ x \ {can_status.Running, can_status.Stopped} \ + ic_can_status_inv z y" + by (fastforce simp: ic_can_status_inv_def split: can_status.splits call_origin.splits) + +lemma ic_can_status_inv_mono2: "ic_can_status_inv x y \ + y \ z \ + ic_can_status_inv x z" + by (force simp: ic_can_status_inv_def split: can_status.splits call_origin.splits) + +lemma ic_can_status_inv_stopping: "ic_can_status_inv x y \ + (\ctxt_id c. os = From_canister ctxt_id c \ ctxt_id \ y) \ + z \ x \ {can_status.Stopping [(os, cyc)]} \ + ic_can_status_inv z y" + by (fastforce simp: ic_can_status_inv_def split: can_status.splits call_origin.splits) + +lemma ic_can_status_inv_stopping_app: "ic_can_status_inv x y \ + can_status.Stopping w \ x \ + (\ctxt_id c. os = From_canister ctxt_id c \ ctxt_id \ y) \ + z \ x \ {can_status.Stopping (w @ [(os, cyc)])} \ + ic_can_status_inv z y" + by (force simp: ic_can_status_inv_def split: can_status.splits call_origin.splits dest!: subsetD[where ?A=z]) + +lemma ic_can_status_inv_del: "ic_can_status_inv x z \ + (\os other_ctxt_id c cyc. Stopping os \ x \ (From_canister other_ctxt_id c, cyc) \ set os \ ctxt_id \ other_ctxt_id) \ + ic_can_status_inv x (z - {ctxt_id})" + by (fastforce simp: ic_can_status_inv_def split: can_status.splits call_origin.splits) + +definition ic_inv :: "('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_inv S = ((\msg \ set (messages S). case msg of + Call_message (From_canister ctxt_id _) _ _ _ _ _ _ \ ctxt_id \ list_map_dom (call_contexts S) + | Response_message (From_canister ctxt_id _) _ _ \ ctxt_id \ list_map_dom (call_contexts S) + | _ \ True) \ + (\ctxt \ list_map_range (call_contexts S). call_ctxt_needs_to_respond ctxt \ + (case call_ctxt_origin ctxt of From_canister ctxt_id _ \ ctxt_id \ list_map_dom (call_contexts S) + | _ \ True)) \ + ic_can_status_inv (list_map_range (canister_status S)) (list_map_dom (call_contexts S)))" + +lemma ic_initial_inv: "ic_inv (initial_ic t pk)" + by (auto simp: ic_inv_def ic_can_status_inv_def initial_ic_def split: call_origin.splits) + +(* Candid *) + +datatype ('s, 'b, 'p) candid = + Candid_nat nat + | Candid_text 's + | Candid_blob (candid_unwrap_blob: 'b) + | Candid_opt "('s, 'b, 'p) candid" + | Candid_vec "('s, 'b, 'p) candid list" + | Candid_record "('s, ('s, 'b, 'p) candid) list_map" + | Candid_null + | Candid_empty + +fun candid_is_blob :: "('s, 'b, 'p) candid \ bool" where + "candid_is_blob (Candid_blob b) = True" +| "candid_is_blob _ = False" + +fun candid_lookup :: "('s, 'b, 'p) candid \ 's \ ('s, 'b, 'p) candid option" where + "candid_lookup (Candid_record m) s = list_map_get m s" +| "candid_lookup _ _ = None" + +(* State transitions *) + +context fixes + CANISTER_ERROR :: reject_code + and CANISTER_REJECT :: reject_code + and SYS_FATAL :: reject_code + and SYS_TRANSIENT :: reject_code + and MAX_CYCLES_PER_MESSAGE :: nat + and MAX_CYCLES_PER_RESPONSE :: nat + and MAX_CANISTER_BALANCE :: nat + and ic_idle_cycles_burned_rate :: "('p :: linorder, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ 'canid \ nat" + and blob_length :: "'b \ nat" + and sha_256 :: "'b \ 'b" + and ic_principal :: 'canid + and blob_of_candid :: "('s, 'b, 'p) candid \ 'b" + and parse_candid :: "'b \ ('s, 'b, 'p) candid option" + and parse_principal :: "'b \ 'p option" + and blob_of_principal :: "'p \ 'b" + and empty_blob :: 'b + and is_system_assigned :: "'p \ bool" + and encode_string :: "string \ 's" + and principal_of_uid :: "'uid \ 'p" + and principal_of_canid :: "'canid \ 'p" + and canid_of_principal :: "'p \ 'canid option" + and parse_wasm_mod :: "'b \ ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module option" + and parse_public_custom_sections :: "'b \ ('s, 'b) list_map option" + and parse_private_custom_sections :: "'b \ ('s, 'b) list_map option" + and verify_envelope :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ 'p \ nat \ 'p set" (* TODO *) + and principal_list_of_set :: "'p set \ 'p list" +begin + +(* Type conversion functions *) + +definition canid_of_blob :: "'b \ 'canid option" where + "canid_of_blob b = (case parse_principal b of Some p \ canid_of_principal p | _ \ None)" + +definition blob_of_canid :: "'canid \ 'b" where + "blob_of_canid = blob_of_principal \ principal_of_canid" + +(* Candid helper functions *) + +definition candid_nested_lookup :: "'b \ 's list \ ('s, 'b, 'p) candid option" where + "candid_nested_lookup b = foldl (\c s. case c of Some c' \ candid_lookup c' s | _ \ None) (parse_candid b)" + +definition candid_parse_nat :: "'b \ 's list \ nat option" where + "candid_parse_nat b s = (case candid_nested_lookup b s of Some (Candid_nat n') \ Some n' | _ \ None)" + +definition candid_parse_text :: "'b \ 's list \ 's option" where + "candid_parse_text b s = (case candid_nested_lookup b s of Some (Candid_text t') \ Some t' | _ \ None)" + +definition candid_parse_blob :: "'b \ 's list \ 'b option" where + "candid_parse_blob b s = (case candid_nested_lookup b s of Some (Candid_blob b') \ Some b' | _ \ None)" + +definition candid_parse_cid :: "'b \ 'canid option" where + "candid_parse_cid b = (case candid_parse_blob b [encode_string ''canister_id''] of Some b' \ canid_of_blob b' | _ \ None)" + +definition candid_parse_controllers :: "'b \ 'p set option" where + "candid_parse_controllers b = (case candid_nested_lookup b [encode_string ''settings'', encode_string ''controllers''] of Some (Candid_vec xs) \ + if (\c'' \ set xs. candid_is_blob c'' \ parse_principal (candid_unwrap_blob c'') \ None) then + Some (the ` parse_principal ` candid_unwrap_blob ` set xs) + else None | _ \ None)" + +fun candid_of_status :: "status \ ('s, 'b, 'p) candid" where + "candid_of_status status.Running = Candid_text (encode_string ''Running'')" +| "candid_of_status status.Stopping = Candid_text (encode_string ''Stopping'')" +| "candid_of_status status.Stopped = Candid_text (encode_string ''Stopped'')" + +(* Cycles *) + +fun carried_cycles :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin \ nat" where + "carried_cycles (From_canister _ _) = MAX_CYCLES_PER_RESPONSE" +| "carried_cycles _ = 0" + +fun cycles_reserved :: "('s, 'p, 'b, 'c) entry_point \ nat" where + "cycles_reserved (entry_point.Public_method _ _ _) = MAX_CYCLES_PER_MESSAGE" +| "cycles_reserved (entry_point.Callback _ _ _) = MAX_CYCLES_PER_RESPONSE" +| "cycles_reserved (entry_point.Heartbeat) = MAX_CYCLES_PER_MESSAGE" + +fun message_cycles :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message \ nat" where + "message_cycles (Call_message orig _ _ _ _ trans_cycles q) = carried_cycles orig + trans_cycles" +| "message_cycles (Func_message _ _ ep _) = cycles_reserved ep" +| "message_cycles (Response_message orig _ ref_cycles) = carried_cycles orig + ref_cycles" + +fun status_cycles :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status \ nat" where + "status_cycles (Stopping os) = sum_list (map (carried_cycles \ fst) os) + sum_list (map snd os)" +| "status_cycles _ = 0" + +lift_definition call_ctxt_carried_cycles :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt \ nat" is + "\ctxt. (if needs_to_respond ctxt + then available_cycles ctxt + carried_cycles (origin ctxt) + else 0)" . + +lemma call_ctxt_respond_carried_cycles[simp]: "call_ctxt_carried_cycles (call_ctxt_respond ctxt) = 0" + by transfer auto + +lemma call_ctxt_carried_cycles: "call_ctxt_carried_cycles ctxt = (if call_ctxt_needs_to_respond ctxt + then call_ctxt_available_cycles ctxt + carried_cycles (call_ctxt_origin ctxt) else 0)" + by transfer auto + +lemma call_ctxt_delete_carried_cycles[simp]: "call_ctxt_carried_cycles (call_ctxt_delete ctxt) = 0" + by transfer auto + +definition total_cycles :: "('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "total_cycles ic = ( + let cycles_in_balances = list_map_sum_vals id (balances ic) in + let cycles_in_messages = sum_list (map message_cycles (messages ic)) in + let cycles_in_contexts = list_map_sum_vals call_ctxt_carried_cycles (call_contexts ic) in + let cycles_in_statuses = list_map_sum_vals status_cycles (canister_status ic) in + cycles_in_balances + cycles_in_messages + cycles_in_contexts + cycles_in_statuses)" + +(* Accessor functions *) + +fun calling_context :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin \ 'cid option" where + "calling_context (From_canister c _) = Some c" +| "calling_context _ = None" + +fun message_queue :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message \ 'canid queue option" where + "message_queue (Call_message _ _ _ _ _ _ q) = Some q" +| "message_queue (Func_message _ _ _ q) = Some q" +| "message_queue _ = None" + +(* Effective canister IDs *) + +definition is_effective_canister_id :: "('b, 'p, 'uid, 'canid, 's) request \ 'p \ bool" where + "is_effective_canister_id r p = (if request.canister_id r = ic_principal then + request.method_name r = encode_string ''provisional_create_canister_with_cycles'' \ + (case candid_parse_cid (request.arg r) of Some cid \ principal_of_canid cid = p | _ \ False) + else principal_of_canid (request.canister_id r) = p)" + + + +(* System transition: API Request submission [DONE] *) + +definition ic_freezing_limit :: "('p :: linorder, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ 'canid \ nat" where + "ic_freezing_limit S cid = ic_idle_cycles_burned_rate S cid * (the (list_map_get (freezing_threshold S) cid)) div (24 * 60 * 60)" + +definition request_submission_pre :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ 'p \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "request_submission_pre E ECID S = (case content E of Inl req \ + principal_of_canid (request.canister_id req) \ verify_envelope E (principal_of_uid (request.sender req)) (system_time S) \ + req \ list_map_dom (requests S) \ + system_time S \ request.ingress_expiry req \ + is_effective_canister_id req ECID \ + ( + ( + request.canister_id req = ic_principal \ + (case candid_parse_cid (request.arg req) of Some cid \ + (case list_map_get (controllers S) cid of Some ctrls \ + principal_of_uid (request.sender req) \ ctrls \ + request.method_name req \ {encode_string ''install_code'', encode_string ''uninstall_code'', encode_string ''update_settings'', + encode_string ''start_canister'', encode_string ''stop_canister'', + encode_string ''canister_status'', encode_string ''delete_canister'', + encode_string ''provisional_create_canister_with_cycles'', encode_string ''provisional_top_up_canister''} + | _ \ False) + | _ \ False) + ) + \ + ( + request.canister_id req \ ic_principal \ + (case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req)) of + (Some (Some can), Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status\ in + (case canister_module_inspect_message (module can) (request.method_name req, wasm_state can, request.arg req, principal_of_uid (request.sender req), env) of Inr ret \ + cycles_return.return ret = Accept \ cycles_return.cycles_used ret \ bal + | _ \ False) + | _ \ False) + ) + ) + | _ \ False)" + +definition request_submission_post :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ 'p \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "request_submission_post E ECID S = ( + let req = projl (content E); + cid = request.canister_id req; + balances = (if cid \ ic_principal then + (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of + (Some (Some can), Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + (case canister_module_inspect_message (module can) (request.method_name req, wasm_state can, request.arg req, principal_of_uid (request.sender req), env) of Inr ret \ + list_map_set (balances S) cid (bal - cycles_return.cycles_used ret))) + else balances S) in + S\requests := list_map_set (requests S) req Received, balances := balances\)" + +definition request_submission_burned_cycles :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ 'p \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "request_submission_burned_cycles E ECID S = ( + let req = projl (content E); + cid = request.canister_id req in + (if request.canister_id req \ ic_principal then + (case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req)) of + (Some (Some can), Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status\ in + (case canister_module_inspect_message (module can) (request.method_name req, wasm_state can, request.arg req, principal_of_uid (request.sender req), env) of Inr ret \ + cycles_return.cycles_used ret)) + else 0))" + +lemma request_submission_cycles_inv: + assumes "request_submission_pre E ECID S" + shows "total_cycles S = total_cycles (request_submission_post E ECID S) + request_submission_burned_cycles E ECID S" +proof - + obtain req where req_def: "content E = Inl req" + using assms + by (auto simp: request_submission_pre_def split: sum.splits) + { + assume "request.canister_id req \ ic_principal" + then have "(case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req)) of + (Some (Some can), Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status\ in + (case canister_module_inspect_message (module can) (request.method_name req, wasm_state can, request.arg req, principal_of_uid (request.sender req), env) of Inr ret \ + cycles_return.return ret = Accept \ cycles_return.cycles_used ret \ bal + | _ \ False) + | _ \ False)" + using assms + by (auto simp: request_submission_pre_def req_def split: option.splits sum.splits prod.splits) + then have ?thesis + using list_map_sum_in_ge[where ?f="balances S" and ?g=id and ?x="request.canister_id req"] + by (auto simp: request_submission_pre_def request_submission_post_def request_submission_burned_cycles_def total_cycles_def req_def list_map_sum_in[where ?f="balances S"] + split: option.splits sum.splits) + } + then show ?thesis + by (auto simp: request_submission_pre_def request_submission_post_def request_submission_burned_cycles_def total_cycles_def req_def) +qed + +lemma request_submission_ic_inv: + assumes "request_submission_pre E ECID S" "ic_inv S" + shows "ic_inv (request_submission_post E ECID S)" + using assms + by (auto simp: ic_inv_def request_submission_pre_def request_submission_post_def Let_def + split: sum.splits message.splits call_origin.splits) + + + +(* System transition: Request rejection [DONE] *) + +definition request_rejection_pre :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ ('b, 'p, 'uid, 'canid, 's) request \ reject_code \ 's \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "request_rejection_pre E req code msg S = (list_map_get (requests S) req = Some Received \ (code = SYS_FATAL \ code = SYS_TRANSIENT))" + +definition request_rejection_post :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope \ ('b, 'p, 'uid, 'canid, 's) request \ reject_code \ 's \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "request_rejection_post E req code msg S = S\requests := list_map_set (requests S) req (Rejected code msg)\" + +lemma request_rejection_cycles_inv: + assumes "request_rejection_pre E req code msg S" + shows "total_cycles S = total_cycles (request_rejection_post E req code msg S)" + by (auto simp: request_rejection_pre_def request_rejection_post_def total_cycles_def) + +lemma request_rejection_ic_inv: + assumes "request_rejection_pre E req code msg S" "ic_inv S" + shows "ic_inv (request_rejection_post E req code msg S)" + using assms + by (auto simp: ic_inv_def request_rejection_pre_def request_rejection_post_def Let_def + split: sum.splits message.splits call_origin.splits) + + + +(* System transition: Initiating canister calls [DONE] *) + +definition initiate_canister_call_pre :: "('b, 'p, 'uid, 'canid, 's) request \ + ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "initiate_canister_call_pre req S = (list_map_get (requests S) req = Some Received \ + system_time S \ request.ingress_expiry req \ + request.canister_id req \ list_map_dom (canisters S))" + +definition initiate_canister_call_post :: "('b, 'p, 'uid, 'canid, 's) request \ + ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ + ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "initiate_canister_call_post req S = + S\requests := list_map_set (requests S) req Processing, messages := + Call_message (From_user req) (principal_of_uid (request.sender req)) (request.canister_id req) (request.method_name req) + (request.arg req) 0 Unordered # messages S\" + +lemma initiate_canister_call_cycles_inv: + assumes "initiate_canister_call_pre R S" + shows "total_cycles S = total_cycles (initiate_canister_call_post R S)" + by (auto simp: initiate_canister_call_pre_def initiate_canister_call_post_def total_cycles_def) + +lemma initiate_canister_call_ic_inv: + assumes "initiate_canister_call_pre R S" "ic_inv S" + shows "ic_inv (initiate_canister_call_post R S)" + using assms + by (auto simp: ic_inv_def initiate_canister_call_pre_def initiate_canister_call_post_def Let_def + split: sum.splits message.splits call_origin.splits) + + + +(* System transition: Calls to stopped/stopping/frozen canisters are rejected [DONE] *) + +definition call_reject_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "call_reject_pre n S = (n < length (messages S) \ (case messages S ! n of + Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + (case list_map_get (canister_status S) cee of Some Stopped \ True + | Some (Stopping l) \ True + | _ \ (case list_map_get (balances S) cee of Some bal \ bal < ic_freezing_limit S cee | _ \ False)) + | _ \ False))" + +definition call_reject_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "call_reject_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + S\messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (response.Reject CANISTER_ERROR (encode_string ''canister not running'')) trans_cycles]\)" + +lemma call_reject_cycles_inv: + assumes "call_reject_pre n S" + shows "total_cycles S = total_cycles (call_reject_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: call_reject_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: call_reject_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: call_reject_pre_def call_reject_post_def total_cycles_def Let_def msgs split: message.splits option.splits) +qed + +lemma call_reject_ic_inv: + assumes "call_reject_pre n S" "ic_inv S" + shows "ic_inv (call_reject_post n S)" + using assms + by (auto simp: ic_inv_def call_reject_pre_def call_reject_post_def Let_def + split: sum.splits message.splits call_origin.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"]) + + + +(* System transition: Call context creation: Public entry points [DONE] *) + +definition call_context_create_pre :: "nat \ 'cid + \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "call_context_create_pre n ctxt_id S = (n < length (messages S) \ (case messages S ! n of + Call_message orig cer cee mn a trans_cycles q \ + (case list_map_get (canisters S) cee of Some (Some can) \ True | _ \ False) \ + list_map_get (canister_status S) cee = Some Running \ + (case list_map_get (balances S) cee of Some bal \ bal \ ic_freezing_limit S cee + MAX_CYCLES_PER_MESSAGE | None \ False) \ + ctxt_id \ list_map_dom (call_contexts S) + | _ \ False))" + +lift_definition create_call_ctxt :: "'canid \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin \ nat \ + ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt" is + "\cee orig trans_cycles. \canister = cee, origin = orig, needs_to_respond = True, deleted = False, available_cycles = trans_cycles\" + by auto + +lemma create_call_ctxt_origin[simp]: "call_ctxt_origin (create_call_ctxt cee orig trans_cycles) = orig" + by transfer auto + +lemma create_call_ctxt_carried_cycles[simp]: "call_ctxt_carried_cycles (create_call_ctxt cee orig trans_cycles) = carried_cycles orig + trans_cycles" + by transfer auto + +definition call_context_create_post :: "nat \ 'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "call_context_create_post n ctxt_id S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + case list_map_get (balances S) cee of Some bal \ + S\messages := list_update (messages S) n (Func_message ctxt_id cee (Public_method mn cer a) q), + call_contexts := list_map_set (call_contexts S) ctxt_id (create_call_ctxt cee orig trans_cycles), + balances := list_map_set (balances S) cee (bal - MAX_CYCLES_PER_MESSAGE)\)" + +lemma call_context_create_cycles_inv: + assumes "call_context_create_pre n ctxt_id S" + shows "total_cycles S = total_cycles (call_context_create_post n ctxt_id S)" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: call_context_create_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ m # younger) ! n = m" + and msgs_upd: "(older @ Call_message orig cer cee mn a trans_cycles q # younger)[n := m] = older @ m # younger" for m + using id_take_nth_drop[of n "messages S"] upd_conv_take_nth_drop[of n "messages S"] assms + by (auto simp: call_context_create_pre_def msg older_def younger_def nth_append) + show ?thesis + using assms list_map_sum_in_ge[of "balances S" cee, where ?g=id] + by (auto simp: call_context_create_pre_def call_context_create_post_def total_cycles_def + list_map_sum_in[where ?g=id, simplified] list_map_sum_out msgs msgs_upd split: option.splits) +qed + +lemma call_context_create_ic_inv: + assumes "call_context_create_pre n ctxt_id S" "ic_inv S" + shows "ic_inv (call_context_create_post n ctxt_id S)" + using assms + by (auto simp: ic_inv_def call_context_create_pre_def call_context_create_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD + intro: ic_can_status_inv_mono2) + + + +(* System transition: Call context creation: Heartbeat [DONE] *) + +definition call_context_heartbeat_pre :: "'canid \ 'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "call_context_heartbeat_pre cee ctxt_id S = ( + (case list_map_get (canisters S) cee of Some (Some can) \ True | _ \ False) \ + list_map_get (canister_status S) cee = Some Running \ + (case list_map_get (balances S) cee of Some bal \ bal \ ic_freezing_limit S cee + MAX_CYCLES_PER_MESSAGE | _ \ False) \ + ctxt_id \ list_map_dom (call_contexts S))" + +lift_definition create_call_ctxt_heartbeat :: "'canid \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt" is + "\cee. \canister = cee, origin = From_heartbeat, needs_to_respond = False, deleted = False, available_cycles = 0\" + by auto + +lemma create_call_ctxt_heartbeat_needs_to_respond[simp]: "call_ctxt_needs_to_respond (create_call_ctxt_heartbeat cee) = False" + by transfer auto + +lemma create_call_ctxt_heartbeat_carried_cycles[simp]: "call_ctxt_carried_cycles (create_call_ctxt_heartbeat cee) = 0" + by transfer auto + +definition call_context_heartbeat_post :: "'canid \ 'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "call_context_heartbeat_post cee ctxt_id S = + (case list_map_get (balances S) cee of Some bal \ + S\messages := Func_message ctxt_id cee Heartbeat (Queue System cee) # messages S, + call_contexts := list_map_set (call_contexts S) ctxt_id (create_call_ctxt_heartbeat cee), + balances := list_map_set (balances S) cee (bal - MAX_CYCLES_PER_MESSAGE)\)" + +lemma call_context_heartbeat_cycles_inv: + assumes "call_context_heartbeat_pre cee ctxt_id S" + shows "total_cycles S = total_cycles (call_context_heartbeat_post cee ctxt_id S)" + using assms list_map_sum_in_ge[of "balances S" cee, where ?g=id, simplified] + by (auto simp: call_context_heartbeat_pre_def call_context_heartbeat_post_def total_cycles_def + list_map_sum_in[where ?g=id, simplified] list_map_sum_out split: option.splits) + +lemma call_context_heartbeat_ic_inv: + assumes "call_context_heartbeat_pre cee ctxt_id S" "ic_inv S" + shows "ic_inv (call_context_heartbeat_post cee ctxt_id S)" + using assms + by (auto simp: ic_inv_def call_context_heartbeat_pre_def call_context_heartbeat_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD + intro: ic_can_status_inv_mono2) + + + +(* System transition: Message execution [DONE] *) + +fun query_as_update :: "(('b arg \ 'p \ 'b env) \ ('w, 'b, 's) query_func) \ 'b arg \ 'p \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" where + "query_as_update (f, a, e) = (\w. case f (a, e) w of Inl t \ Inl t | + Inr res \ Inr \update_return.new_state = w, update_return.new_calls = [], update_return.new_certified_data = None, + update_return.response = Some (query_return.response res), update_return.cycles_accepted = 0, update_return.cycles_used = query_return.cycles_used res\)" + +fun heartbeat_as_update :: "('b env \ 'w heartbeat_func) \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" where + "heartbeat_as_update (f, e) = (\w. case f e w of Inl t \ Inl t | + Inr res \ Inr \update_return.new_state = heartbeat_return.new_state res, update_return.new_calls = [], update_return.new_certified_data = None, + update_return.response = None, update_return.cycles_accepted = 0, update_return.cycles_used = heartbeat_return.cycles_used res\)" + +fun exec_function :: "('s, 'p, 'b, 'c) entry_point \ 'b env \ nat \ ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" where + "exec_function (entry_point.Public_method mn c a) e bal m = ( + case dispatch_method mn m of Some (Inl upd) \ upd (a, c, e, bal) + | Some (Inr query) \ query_as_update (query, a, c, e) + | None \ undefined + )" +| "exec_function (entry_point.Callback c resp ref_cycles) e bal m = + canister_module_callbacks m (c, resp, ref_cycles, e, bal)" +| "exec_function (entry_point.Heartbeat) e bal m = heartbeat_as_update ((canister_module_heartbeat m), e)" + +definition message_execution_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "message_execution_pre n S = + (n < length (messages S) \ (case messages S ! n of Func_message ctxt_id recv ep q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + (case (list_map_get (canisters S) recv, list_map_get (balances S) recv, list_map_get (canister_status S) recv, + list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id) of + (Some (Some can), Some bal, Some can_status, Some t, Some ctxt) \ True | _ \ False) + | _ \ False))" + +definition message_execution_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "message_execution_post n S = (case messages S ! n of Func_message ctxt_id recv ep q \ + (case (list_map_get (canisters S) recv, list_map_get (balances S) recv, list_map_get (canister_status S) recv, + list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id) of + (Some (Some can), Some bal, Some can_status, Some t, Some ctxt) \ ( + let Mod = module can; + Is_response = (case ep of Callback _ _ _ \ True | _ \ False); + Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\; + Available = call_ctxt_available_cycles ctxt; + F = exec_function ep Env Available Mod; + R = F (wasm_state can); + cyc_used = (case R of Inr res \ update_return.cycles_used res | Inl trap \ cycles_return.cycles_used trap); + (cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, update_return.new_calls res)); + New_balance = bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + - (cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res)); + no_response = (case R of Inr result \ update_return.response result = None) in + if \isl R \ cyc_used \ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + cycles_accepted_res \ Available \ + cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res) \ + bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + New_balance \ (if Is_response then 0 else ic_freezing_limit S recv) \ + (no_response \ call_ctxt_needs_to_respond ctxt) then + (let result = projr R; + new_call_to_message = (\call. Call_message (From_canister ctxt_id (method_call.callback call)) (principal_of_canid recv) + (method_call.callee call) (method_call.method_name call) (method_call.arg call) (method_call.transferred_cycles call) (Queue (Canister recv) (method_call.callee call))); + response_messages = (case update_return.response result of None \ [] + | Some resp \ [Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res)]); + messages = take n (messages S) @ drop (Suc n) (messages S) @ map new_call_to_message new_calls_res @ response_messages; + new_ctxt = (case update_return.response result of + None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt + | Some _ \ call_ctxt_respond ctxt); + certified_data = (case new_certified_data result of None \ certified_data S + | Some cd \ list_map_set (certified_data S) recv cd) + in S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), + messages := messages, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, + certified_data := certified_data, balances := list_map_set (balances S) recv New_balance\) + else S\messages := take n (messages S) @ drop (Suc n) (messages S), + balances := list_map_set (balances S) recv ((bal + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) + - min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))\)) + | _ \ undefined)" + +definition message_execution_burned_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "message_execution_burned_cycles n S = (case messages S ! n of Func_message ctxt_id recv ep q \ + (case (list_map_get (canisters S) recv, list_map_get (balances S) recv, list_map_get (canister_status S) recv, + list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id) of + (Some (Some can), Some bal, Some can_status, Some t, Some ctxt) \ ( + let Mod = module can; + Is_response = (case ep of Callback _ _ _ \ True | _ \ False); + Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\; + Available = call_ctxt_available_cycles ctxt; + F = exec_function ep Env Available Mod; + R = F (wasm_state can); + cyc_used = (case R of Inr res \ update_return.cycles_used res | Inl trap \ cycles_return.cycles_used trap) in + min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + )))" + +lemma message_execution_cycles_monotonic: + assumes pre: "message_execution_pre n S" + shows "total_cycles S = total_cycles (message_execution_post n S) + message_execution_burned_cycles n S" +proof - + obtain ctxt_id recv ep q can bal can_status t ctxt where msg: "messages S ! n = Func_message ctxt_id recv ep q" + and prod: "list_map_get (canisters S) recv = Some (Some can)" + "list_map_get (balances S) recv = Some bal" + "list_map_get (canister_status S) recv = Some can_status" + "list_map_get (time S) recv = Some t" + "list_map_get (call_contexts S) ctxt_id = Some ctxt" + using pre + by (auto simp: message_execution_pre_def split: message.splits option.splits) + define Mod where "Mod = can_state_rec.module can" + define Is_response where "Is_response = (case ep of Callback _ _ _ \ True | _ \ False)" + define Env :: "'b env" where "Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\" + define Available where "Available = call_ctxt_available_cycles ctxt" + define F where "F = exec_function ep Env Available Mod" + define R where "R = F (wasm_state can)" + define cyc_used where "cyc_used = (case R of Inr res \ update_return.cycles_used res | Inl trap \ cycles_return.cycles_used trap)" + obtain cycles_accepted_res new_calls_res where res: "(cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, new_calls res))" + by (cases "(case R of Inr res \ (update_return.cycles_accepted res, new_calls res))") auto + define New_balance where "New_balance = bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + - (cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res))" + define no_response where "no_response = (case R of Inr result \ update_return.response result = None)" + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Func_message ctxt_id recv ep q # younger" + "take n older = older" "drop (Suc n) older = []" + "take (n - length older) ws = []" "drop (Suc n - length older) (w # ws) = ws" + for w and ws :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message list" + using id_take_nth_drop[of n "messages S"] pre + by (auto simp: message_execution_pre_def msg older_def younger_def) + note lm = list_map_sum_in[OF prod(2), where ?g=id, simplified] list_map_sum_in_ge[OF prod(2), where ?g=id, simplified] + list_map_sum_in[OF prod(5), where ?g=call_ctxt_carried_cycles] list_map_sum_in_ge[OF prod(5), where ?g=call_ctxt_carried_cycles] + define S'' where "S'' = S\messages := take n (messages S) @ drop (Suc n) (messages S), + balances := list_map_set (balances S) recv ((bal + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) - min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))\" + define cond where "cond = (\isl R \ cyc_used \ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + cycles_accepted_res \ Available \ + cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res) \ + bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + New_balance \ (if Is_response then 0 else ic_freezing_limit S recv) \ + (no_response \ call_ctxt_needs_to_respond ctxt))" + have reserved: "(if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) = cycles_reserved ep" + by (auto simp: Is_response_def split: entry_point.splits) + show ?thesis + proof (cases cond) + case False + have "message_execution_post n S = S''" + "message_execution_burned_cycles n S = min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)" + using False + by (simp_all add: message_execution_post_def message_execution_burned_cycles_def Let_def msg prod + Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] cyc_used_def[symmetric] res[symmetric] + New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric] del: min_less_iff_conj split del: if_split) + then show ?thesis + using lm(2) + by (auto simp: total_cycles_def S''_def msgs lm(1) reserved) + next + case True + define result where "result = projr R" + have R_Inr: "R = Inr result" + using True + by (auto simp: cond_def result_def split: option.splits) + define response_messages where "response_messages = (case update_return.response result of None \ [] + | Some resp \ [Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res)])" + define new_call_to_message :: "(?'p, 'canid, 's, 'b, 'c) method_call \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message" where + "new_call_to_message = (\call. Call_message (From_canister ctxt_id (callback call)) (principal_of_canid recv) + (callee call) (method_call.method_name call) (method_call.arg call) (transferred_cycles call) (Queue (Canister recv) (callee call)))" + define messages where "messages = take n (ic.messages S) @ drop (Suc n) (ic.messages S) @ map new_call_to_message new_calls_res @ response_messages" + define new_ctxt where "new_ctxt = (case update_return.response result of + None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt + | Some _ \ call_ctxt_respond ctxt)" + define certified_data where "certified_data = (case new_certified_data result of None \ ic.certified_data S + | Some cd \ list_map_set (ic.certified_data S) recv cd)" + define S' where "S' = S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), + messages := messages, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, + certified_data := certified_data, balances := list_map_set (balances S) recv New_balance\" + have cycles_accepted_res_def: "cycles_accepted_res = update_return.cycles_accepted result" + and new_calls_res_def: "new_calls_res = new_calls result" + using res + by (auto simp: R_Inr) + have no_response: "no_response = (update_return.response result = None)" + by (auto simp: no_response_def R_Inr) + have msg_exec: "message_execution_post n S = S'" + and lost: "message_execution_burned_cycles n S = min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)" + using True + by (simp_all add: message_execution_post_def message_execution_burned_cycles_def Let_def msg prod + Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] cyc_used_def[symmetric] res[symmetric] + New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric] + messages_def[symmetric] new_ctxt_def[symmetric] certified_data_def[symmetric] S'_def[symmetric] + result_def[symmetric] response_messages_def[symmetric] new_call_to_message_def[symmetric] + del: min_less_iff_conj split del: if_split) + have "message_cycles \ new_call_to_message = (\c. MAX_CYCLES_PER_RESPONSE + transferred_cycles c)" for c :: "(?'p, 'canid, 's, 'b, 'c) method_call" + by (auto simp: new_call_to_message_def) + then have A1: "sum_list (map (message_cycles \ new_call_to_message) new_calls_res) = (\x\new_calls_res. MAX_CYCLES_PER_RESPONSE + transferred_cycles x)" + by auto + have A2: "sum_list (map local.message_cycles response_messages) = (case update_return.response result of None \ 0 + | _ \ carried_cycles (call_ctxt_origin ctxt) + (call_ctxt_available_cycles ctxt - update_return.cycles_accepted result))" + by (auto simp: response_messages_def Available_def cycles_accepted_res_def split: option.splits) + have A3: "call_ctxt_carried_cycles new_ctxt = (case update_return.response result of Some _ \ 0 + | _ \ if call_ctxt_needs_to_respond ctxt then carried_cycles (call_ctxt_origin ctxt) + (call_ctxt_available_cycles ctxt - update_return.cycles_accepted result) else 0)" + by (auto simp: new_ctxt_def Available_def cycles_accepted_res_def call_ctxt_carried_cycles split: option.splits) + have A4: "call_ctxt_carried_cycles ctxt = (if call_ctxt_needs_to_respond ctxt then carried_cycles (call_ctxt_origin ctxt) + call_ctxt_available_cycles ctxt else 0)" + using call_ctxt_carried_cycles + by auto + have reserve: "cycles_reserved ep = (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)" + by (auto simp: Is_response_def split: entry_point.splits) + have messages_msgs: "messages = older @ younger @ map new_call_to_message new_calls_res @ response_messages" + by (auto simp: messages_def older_def younger_def) + show ?thesis + using lm(2,4) True call_ctxt_not_needs_to_respond_available_cycles[of ctxt] + by (auto simp: cond_def msg_exec S'_def total_cycles_def lm(1,3) msgs messages_msgs A1 A2 A3 A4 New_balance_def + reserve cycles_accepted_res_def no_response_def R_Inr lost Available_def split: option.splits) + qed +qed + +lemma message_execution_ic_inv: + assumes "message_execution_pre n S" "ic_inv S" + shows "ic_inv (message_execution_post n S)" +proof - + obtain ctxt_id recv ep q can bal can_status t ctxt where msg: "messages S ! n = Func_message ctxt_id recv ep q" + and prod: "list_map_get (canisters S) recv = Some (Some can)" + "list_map_get (balances S) recv = Some bal" + "list_map_get (canister_status S) recv = Some can_status" + "list_map_get (time S) recv = Some t" + "list_map_get (call_contexts S) ctxt_id = Some ctxt" + using assms + by (auto simp: message_execution_pre_def split: message.splits option.splits) + define Mod where "Mod = can_state_rec.module can" + define Is_response where "Is_response = (case ep of Callback _ _ _ \ True | _ \ False)" + define Env :: "'b env" where "Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\" + define Available where "Available = call_ctxt_available_cycles ctxt" + define F where "F = exec_function ep Env Available Mod" + define R where "R = F (wasm_state can)" + define cyc_used where "cyc_used = (case R of Inr res \ update_return.cycles_used res | Inl trap \ cycles_return.cycles_used trap)" + obtain cycles_accepted_res new_calls_res where res: "(cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, new_calls res))" + by (cases "(case R of Inr res \ (update_return.cycles_accepted res, new_calls res))") auto + define New_balance where "New_balance = bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + - (cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res))" + define no_response where "no_response = (case R of Inr result \ update_return.response result = None)" + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Func_message ctxt_id recv ep q # younger" + "take n older = older" "drop (Suc n) older = []" + "take (n - length older) ws = []" "drop (Suc n - length older) (w # ws) = ws" + for w and ws :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message list" + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: message_execution_pre_def msg older_def younger_def) + define S'' where "S'' = S\messages := take n (messages S) @ drop (Suc n) (messages S), + balances := list_map_set (balances S) recv ((bal + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) - min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))\" + define cond where "cond = (\isl R \ cyc_used \ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + cycles_accepted_res \ Available \ + cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res) \ + bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + New_balance \ (if Is_response then 0 else ic_freezing_limit S recv) \ + (no_response \ call_ctxt_needs_to_respond ctxt))" + show ?thesis + proof (cases cond) + case False + have "message_execution_post n S = S''" + using False + by (simp_all add: message_execution_post_def Let_def msg prod + Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] cyc_used_def[symmetric] res[symmetric] + New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric]) + then show ?thesis + using assms(2) + apply (auto simp: ic_inv_def S''_def msgs split: message.splits call_origin.splits) + apply force + apply fast + apply force + apply fast + done + next + case True + define result where "result = projr R" + have R_Inr: "R = Inr result" + using True + by (auto simp: cond_def result_def split: option.splits) + define response_messages where "response_messages = (case update_return.response result of None \ [] + | Some resp \ [Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res)])" + define new_call_to_message :: "(?'p, 'canid, 's, 'b, 'c) method_call \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message" where + "new_call_to_message = (\call. Call_message (From_canister ctxt_id (callback call)) (principal_of_canid recv) + (callee call) (method_call.method_name call) (method_call.arg call) (transferred_cycles call) (Queue (Canister recv) (callee call)))" + define messages where "messages = take n (ic.messages S) @ drop (Suc n) (ic.messages S) @ map new_call_to_message new_calls_res @ response_messages" + define new_ctxt where "new_ctxt = (case update_return.response result of + None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt + | Some _ \ call_ctxt_respond ctxt)" + define certified_data where "certified_data = (case new_certified_data result of None \ ic.certified_data S + | Some cd \ list_map_set (ic.certified_data S) recv cd)" + define S' where "S' = S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), + messages := messages, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, + certified_data := certified_data, balances := list_map_set (balances S) recv New_balance\" + have cycles_accepted_res_def: "cycles_accepted_res = update_return.cycles_accepted result" + and new_calls_res_def: "new_calls_res = new_calls result" + using res + by (auto simp: R_Inr) + have no_response: "no_response = (update_return.response result = None)" + by (auto simp: no_response_def R_Inr) + have msg_exec: "message_execution_post n S = S'" + using True + by (simp_all add: message_execution_post_def Let_def msg prod + Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] cyc_used_def[symmetric] res[symmetric] + New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric] + messages_def[symmetric] new_ctxt_def[symmetric] certified_data_def[symmetric] S'_def[symmetric] + result_def[symmetric] response_messages_def[symmetric] new_call_to_message_def[symmetric]) + have messages_msgs: "messages = older @ younger @ map new_call_to_message new_calls_res @ response_messages" + by (auto simp: messages_def older_def younger_def) + have ctxt_in_range: "ctxt \ list_map_range (call_contexts S)" + using prod(5) + by (simp add: list_map_get_range) + have response_msgsD: "msg \ set response_messages \ update_return.response result \ None \ + (\resp. msg = Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res))" for msg + by (auto simp: response_messages_def) metis + have call_ctxt_origin_new_ctxt: "call_ctxt_origin new_ctxt = call_ctxt_origin ctxt" + by (auto simp: new_ctxt_def split: option.splits) + have call_ctxt_needs_to_respond_new_ctxtD: "call_ctxt_needs_to_respond new_ctxt \ call_ctxt_needs_to_respond ctxt" + by (auto simp: new_ctxt_def split: option.splits) + show ?thesis + using assms(2) ctxt_in_range True call_ctxt_not_needs_to_respond_available_cycles[of ctxt] + apply (auto simp: cond_def msg_exec S'_def ic_inv_def msgs messages_msgs new_call_to_message_def + no_response_def R_Inr call_ctxt_origin_new_ctxt + split: option.splits message.splits call_origin.splits + dest!: list_map_range_setD response_msgsD call_ctxt_needs_to_respond_new_ctxtD + intro: ic_can_status_inv_mono2) + apply blast + apply fast + apply blast + apply fast + apply blast + apply fast + apply blast + apply fast + done + qed +qed + + + +(* System transition: Call context starvation [DONE] *) + +definition call_context_starvation_pre :: "'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "call_context_starvation_pre ctxt_id S = + (case list_map_get (call_contexts S) ctxt_id of Some call_context \ + call_ctxt_needs_to_respond call_context \ + call_ctxt_origin call_context \ From_heartbeat \ + (\msg \ set (messages S). case msg of + Call_message orig _ _ _ _ _ _ \ calling_context orig \ Some ctxt_id + | Response_message orig _ _ \ calling_context orig \ Some ctxt_id + | _ \ True) \ + (\other_call_context \ list_map_range (call_contexts S). + call_ctxt_needs_to_respond other_call_context \ + calling_context (call_ctxt_origin other_call_context) \ Some ctxt_id) + | None \ False)" + +definition call_context_starvation_post :: "'cid \ + ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "call_context_starvation_post ctxt_id S = ( + case list_map_get (call_contexts S) ctxt_id of Some call_context \ + let msg = Response_message (call_ctxt_origin call_context) (response.Reject CANISTER_ERROR (encode_string ''starvation'')) (call_ctxt_available_cycles call_context) + in S\call_contexts := list_map_set (call_contexts S) ctxt_id (call_ctxt_respond call_context), + messages := messages S @ [msg]\)" + +lemma call_context_starvation_cycles_inv: + assumes "call_context_starvation_pre ctxt_id S" + shows "total_cycles S = total_cycles (call_context_starvation_post ctxt_id S)" + using assms list_map_sum_in_ge[where ?f="call_contexts S" and ?x=ctxt_id and ?g=call_ctxt_carried_cycles] + by (auto simp: call_context_starvation_pre_def call_context_starvation_post_def total_cycles_def + call_ctxt_carried_cycles list_map_sum_in[where ?g=call_ctxt_carried_cycles] split: option.splits) + +lemma call_context_starvation_ic_inv: + assumes "call_context_starvation_pre ctxt_id S" "ic_inv S" + shows "ic_inv (call_context_starvation_post ctxt_id S)" + using assms + by (auto simp: ic_inv_def call_context_starvation_pre_def call_context_starvation_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range + intro: ic_can_status_inv_mono2) + + + +(* System transition: Call context removal [DONE] *) + +definition call_context_removal_pre :: "'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "call_context_removal_pre ctxt_id S = ( + (case list_map_get (call_contexts S) ctxt_id of Some call_context \ + (\call_ctxt_needs_to_respond call_context \ + (call_ctxt_origin call_context = From_heartbeat \ + (\msg \ set (messages S). case msg of + Func_message other_ctxt_id _ _ _ \ other_ctxt_id \ ctxt_id + | _ \ True))) \ + (\msg \ set (messages S). case msg of + Call_message ctxt _ _ _ _ _ _ \ calling_context ctxt \ Some ctxt_id + | Response_message ctxt _ _ \ calling_context ctxt \ Some ctxt_id + | _ \ True) \ + (\other_call_context \ list_map_range (call_contexts S). + call_ctxt_needs_to_respond other_call_context \ + calling_context (call_ctxt_origin other_call_context) \ Some ctxt_id) \ + (\can_status \ list_map_range (canister_status S). case can_status of Stopping os \ + \(orig, cyc) \ set os. (case orig of From_canister other_ctxt_id _ \ ctxt_id \ other_ctxt_id | _ \ True) + | _ \ True) + | None \ False))" + +definition call_context_removal_post :: "'cid \ + ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "call_context_removal_post ctxt_id S = S\call_contexts := list_map_del (call_contexts S) ctxt_id\" + +definition call_context_removal_burned_cycles :: "'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "call_context_removal_burned_cycles ctxt_id S = call_ctxt_available_cycles (the (list_map_get (call_contexts S) ctxt_id))" + +lemma call_context_removal_cycles_monotonic: + assumes "call_context_removal_pre ctxt_id S" + shows "total_cycles S = total_cycles (call_context_removal_post ctxt_id S) + call_context_removal_burned_cycles ctxt_id S" + using assms call_ctxt_not_needs_to_respond_available_cycles + by (auto simp: call_context_removal_pre_def call_context_removal_post_def call_context_removal_burned_cycles_def total_cycles_def call_ctxt_carried_cycles list_map_del_sum split: option.splits) + +lemma call_context_removal_ic_inv: + assumes "call_context_removal_pre ctxt_id S" "ic_inv S" + shows "ic_inv (call_context_removal_post ctxt_id S)" + using assms + by (force simp: ic_inv_def call_context_removal_pre_def call_context_removal_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del + intro!: ic_can_status_inv_del[where ?z="list_map_dom (call_contexts S)"]) + + + +(* System transition: IC Management Canister: Canister creation [DONE] *) + +definition ic_canister_creation_pre :: "nat \ 'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_creation_pre n cid t S = (n < length (messages S) \ (case messages S ! n of + Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''create_canister'' \ + parse_candid a \ None \ + is_system_assigned (principal_of_canid cid) \ + cid \ list_map_dom (canisters S) \ + cid \ list_map_dom (time S) \ + cid \ list_map_dom (controllers S) \ + cid \ list_map_dom (balances S) \ + cid \ list_map_dom (certified_data S) \ + cid \ list_map_dom (canister_status S) + | _ \ False))" + +definition ic_canister_creation_post :: "nat \ 'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_creation_post n cid t S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let ctrls = (case candid_parse_controllers a of Some ctrls \ ctrls | _ \ {cer}) in + S\canisters := list_map_set (canisters S) cid None, + time := list_map_set (time S) cid t, + controllers := list_map_set (controllers S) cid ctrls, + freezing_threshold := list_map_set (freezing_threshold S) cid 2592000, + balances := list_map_set (balances S) cid trans_cycles, + certified_data := list_map_set (certified_data S) cid empty_blob, + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid + (Candid_record (list_map_init [(encode_string ''canister_id'', Candid_blob (blob_of_canid cid))])))) 0], + canister_status := list_map_set (canister_status S) cid Running\)" + +lemma ic_canister_creation_cycles_inv: + assumes "ic_canister_creation_pre n cid t S" + shows "total_cycles S = total_cycles (ic_canister_creation_post n cid t S)" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: ic_canister_creation_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_canister_creation_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_canister_creation_pre_def ic_canister_creation_post_def total_cycles_def Let_def msgs + list_map_sum_out[where ?g=id] list_map_sum_out[where ?g=status_cycles] split: message.splits option.splits) +qed + +lemma ic_canister_creation_ic_inv: + assumes "ic_canister_creation_pre n cid t S" "ic_inv S" + shows "ic_inv (ic_canister_creation_post n cid t S)" + using assms + by (auto simp: ic_inv_def ic_canister_creation_pre_def ic_canister_creation_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + intro: ic_can_status_inv_mono1) + + + +(* System transition: IC Management Canister: Changing settings [DONE] *) + +definition ic_update_settings_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_update_settings_pre n S = (n < length (messages S) \ (case messages S ! n of + Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''update_settings'' \ + (case candid_parse_cid a of Some cid \ + (case list_map_get (controllers S) cid of Some ctrls \ cer \ ctrls + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_update_settings_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_update_settings_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a); + ctrls = (case candid_parse_controllers a of Some ctrls \ list_map_set (controllers S) cid ctrls | _ \ controllers S); + freezing_thres = (case candid_nested_lookup a [encode_string '''settings'', encode_string ''freezing_threshold''] of Some (Candid_nat freeze) \ + list_map_set (freezing_threshold S) cid freeze | _ \ freezing_threshold S) in + S\controllers := ctrls, freezing_threshold := freezing_thres, messages := take n (messages S) @ drop (Suc n) (messages S) @ + [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)" + +lemma ic_update_settings_cycles_inv: + assumes "ic_update_settings_pre n S" + shows "total_cycles S = total_cycles (ic_update_settings_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: ic_update_settings_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_update_settings_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_update_settings_pre_def ic_update_settings_post_def total_cycles_def Let_def msgs split: message.splits option.splits) +qed + +lemma ic_update_settings_ic_inv: + assumes "ic_update_settings_pre n S" "ic_inv S" + shows "ic_inv (ic_update_settings_post n S)" + using assms + by (auto simp: ic_inv_def ic_update_settings_pre_def ic_update_settings_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del) + + + +(* System transition: IC Management Canister: Canister status [DONE] *) + +definition ic_canister_status_pre :: "nat \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_status_pre n m S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''canister_status'' \ + (case candid_parse_cid a of Some cid \ + cid \ list_map_dom (canisters S) \ + cid \ list_map_dom (canister_status S) \ + cid \ list_map_dom (balances S) \ + cid \ list_map_dom (freezing_threshold S) \ + (case list_map_get (controllers S) cid of Some ctrls \ cer \ ctrls | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_canister_status_post :: "nat \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_status_post n m S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a); + hash = (case the (list_map_get (canisters S) cid) of None \ Candid_null + | Some can \ Candid_opt (Candid_blob (sha_256 (raw_module can)))) in + S\messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid + (Candid_record (list_map_init [(encode_string ''status'', candid_of_status (simple_status (the (list_map_get (canister_status S) cid)))), + (encode_string ''module_hash'', hash), + (encode_string ''controllers'', Candid_vec (map (Candid_blob \ blob_of_principal) (principal_list_of_set (the (list_map_get (controllers S) cid))))), + (encode_string ''memory_size'', Candid_nat m), + (encode_string ''cycles'', Candid_nat (the (list_map_get (balances S) cid))), + (encode_string ''freezing_threshold'', Candid_nat (the (list_map_get (freezing_threshold S) cid))), + (encode_string ''idle_cycles_burned_per_day'', Candid_nat (ic_idle_cycles_burned_rate S cid))])))) trans_cycles]\)" + +lemma ic_canister_status_cycles_inv: + assumes "ic_canister_status_pre n m S" + shows "total_cycles S = total_cycles (ic_canister_status_post n m S)" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: ic_canister_status_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_canister_status_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_canister_status_pre_def ic_canister_status_post_def total_cycles_def Let_def msgs split: message.splits option.splits) +qed + +lemma ic_canister_status_ic_inv: + assumes "ic_canister_status_pre n m S" "ic_inv S" + shows "ic_inv (ic_canister_status_post n m S)" + using assms + by (auto simp: ic_inv_def ic_canister_status_pre_def ic_canister_status_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del) + + + +(* System transition: IC Management Canister: Code installation [DONE] *) + +definition ic_code_installation_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_code_installation_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''install_code'' \ + (case candid_parse_cid a of Some cid \ + (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of + (Some mode, Some w, Some ar) \ + (case parse_wasm_mod w of Some m \ + parse_public_custom_sections w \ None \ + parse_private_custom_sections w \ None \ + (case (list_map_get (controllers S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of + (Some ctrls, Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + ((mode = encode_string ''install'' \ (case list_map_get (canisters S) cid of Some None \ True | _ \ False)) \ mode = encode_string ''reinstall'') \ + cer \ ctrls \ + (case canister_module_init m (cid, ar, cer, env) of Inl _ \ False + | Inr ret \ cycles_return.cycles_used ret \ bal) \ + list_map_dom (canister_module_update_methods m) \ list_map_dom (canister_module_query_methods m) = {} + | _ \ False) | _ \ False) | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_code_installation_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_code_installation_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (case candid_parse_cid a of Some cid \ + (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of + (Some mode, Some w, Some a) \ + (case parse_wasm_mod w of Some m \ + (case (list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of + (Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\; + (new_state, cyc_used) = (case canister_module_init m (cid, a, cer, env) of Inr ret \ (cycles_return.return ret, cycles_return.cycles_used ret)) in + S\canisters := list_map_set (canisters S) cid (Some \wasm_state = new_state, module = m, raw_module = w, + public_custom_sections = the (parse_public_custom_sections w), private_custom_sections = the (parse_private_custom_sections w)\), + balances := list_map_set (balances S) cid (bal - cyc_used), + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)))))" + +definition ic_code_installation_burned_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "ic_code_installation_burned_cycles n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (case candid_parse_cid a of Some cid \ + (case (candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of + (Some w, Some a) \ + (case parse_wasm_mod w of Some m \ + (case (list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of + (Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + (case canister_module_init m (cid, a, cer, env) of Inr ret \ cycles_return.cycles_used ret))))))" + +lemma ic_code_installation_cycles_inv: + assumes "ic_code_installation_pre n S" + shows "total_cycles S = total_cycles (ic_code_installation_post n S) + ic_code_installation_burned_cycles n S" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: ic_code_installation_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_code_installation_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms list_map_sum_in_ge[where ?f="balances S" and ?g=id and ?x="the (candid_parse_cid a)"] + by (auto simp: ic_code_installation_pre_def ic_code_installation_post_def ic_code_installation_burned_cycles_def total_cycles_def Let_def msgs list_map_sum_in[where ?f="balances S"] split: message.splits option.splits sum.splits prod.splits) +qed + +lemma ic_code_installation_ic_inv: + assumes "ic_code_installation_pre n S" "ic_inv S" + shows "ic_inv (ic_code_installation_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid mode w ar m ctrls t bal can_status where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and parse: "candid_parse_cid a = Some cid" + "candid_parse_text a [encode_string ''mode''] = Some mode" + "candid_parse_blob a [encode_string ''wasm_module''] = Some w" + "candid_parse_blob a [encode_string ''arg''] = Some ar" + "parse_wasm_mod w = Some m" + and list_map_get: + "list_map_get (controllers S) cid = Some ctrls" + "list_map_get (time S) cid = Some t" + "list_map_get (balances S) cid = Some bal" + "list_map_get (canister_status S) cid = Some can_status" + using assms + by (auto simp: ic_code_installation_pre_def split: message.splits option.splits) + show ?thesis + using assms + by (auto simp: ic_inv_def ic_code_installation_pre_def ic_code_installation_post_def msg parse list_map_get Let_def + split: sum.splits call_origin.splits message.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del) +qed + + + +(* System transition: IC Management Canister: Code upgrade [DONE] *) + +definition ic_code_upgrade_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_code_upgrade_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''install_code'' \ + (case candid_parse_cid a of Some cid \ + (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of + (Some mode, Some w, Some ar) \ + (case parse_wasm_mod w of Some m \ + parse_public_custom_sections w \ None \ + parse_private_custom_sections w \ None \ + (case (list_map_get (canisters S) cid, list_map_get (controllers S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of + (Some (Some can), Some ctrls, Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + mode = encode_string ''upgrade'' \ + cer \ ctrls \ + (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env) of Inr pre_ret \ + (case canister_module_post_upgrade m (cid, cycles_return.return pre_ret, ar, cer, env) of Inr post_ret \ + cycles_return.cycles_used pre_ret + cycles_return.cycles_used post_ret \ bal + | _ \ False) | _ \ False) \ + list_map_dom (canister_module_update_methods m) \ list_map_dom (canister_module_query_methods m) = {} + | _ \ False) | _ \ False) | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_code_upgrade_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_code_upgrade_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (case candid_parse_cid a of Some cid \ + (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of + (Some mode, Some w, Some ar) \ + (case parse_wasm_mod w of Some m \ + (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of + (Some (Some can), Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env) of Inr pre_ret \ + (case canister_module_post_upgrade m (cid, cycles_return.return pre_ret, ar, cer, env) of Inr post_ret \ + S\canisters := list_map_set (canisters S) cid (Some \wasm_state = cycles_return.return post_ret, module = m, raw_module = w, + public_custom_sections = the (parse_public_custom_sections w), private_custom_sections = the (parse_private_custom_sections w)\), + balances := list_map_set (balances S) cid (bal - (cycles_return.cycles_used pre_ret + cycles_return.cycles_used post_ret)), + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)))))))" + +definition ic_code_upgrade_burned_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "ic_code_upgrade_burned_cycles n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (case candid_parse_cid a of Some cid \ + (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of + (Some mode, Some w, Some ar) \ + (case parse_wasm_mod w of Some m \ + (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of + (Some (Some can), Some t, Some bal, Some can_status) \ + let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env) of Inr pre_ret \ + (case canister_module_post_upgrade m (cid, cycles_return.return pre_ret, ar, cer, env) of Inr post_ret \ + cycles_return.cycles_used pre_ret + cycles_return.cycles_used post_ret)))))))" + +lemma ic_code_upgrade_cycles_inv: + assumes "ic_code_upgrade_pre n S" + shows "total_cycles S = total_cycles (ic_code_upgrade_post n S) + ic_code_upgrade_burned_cycles n S" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: ic_code_upgrade_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_code_upgrade_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms list_map_sum_in_ge[where ?f="balances S" and ?g=id and ?x="the (candid_parse_cid a)"] + by (auto simp: ic_code_upgrade_pre_def ic_code_upgrade_post_def ic_code_upgrade_burned_cycles_def total_cycles_def Let_def msgs list_map_sum_in[where ?f="balances S"] split: message.splits option.splits sum.splits) +qed + +lemma ic_code_upgrade_ic_inv: + assumes "ic_code_upgrade_pre n S" "ic_inv S" + shows "ic_inv (ic_code_upgrade_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid mode w ar m can ctrls t bal can_status where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and parse: "candid_parse_cid a = Some cid" + "candid_parse_text a [encode_string ''mode''] = Some mode" + "candid_parse_blob a [encode_string ''wasm_module''] = Some w" + "candid_parse_blob a [encode_string ''arg''] = Some ar" + "parse_wasm_mod w = Some m" + and list_map_get: + "list_map_get (canisters S) cid = Some (Some can)" + "list_map_get (controllers S) cid = Some ctrls" + "list_map_get (time S) cid = Some t" + "list_map_get (balances S) cid = Some bal" + "list_map_get (canister_status S) cid = Some can_status" + using assms + by (auto simp: ic_code_upgrade_pre_def split: message.splits option.splits) + show ?thesis + using assms + by (auto simp: ic_inv_def ic_code_upgrade_pre_def ic_code_upgrade_post_def msg parse list_map_get Let_def + split: sum.splits call_origin.splits message.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del) +qed + + + +(* System transition: IC Management Canister: Code uninstallation [DONE] *) + +definition ic_code_uninstallation_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_code_uninstallation_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''uninstall_code'' \ + (case candid_parse_cid a of Some cid \ + (case list_map_get (controllers S) cid of Some ctrls \ cer \ ctrls + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_code_uninstallation_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_code_uninstallation_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (case candid_parse_cid a of Some cid \ + let call_ctxt_to_msg = (\ctxt. + if call_ctxt_canister ctxt = cid \ call_ctxt_needs_to_respond ctxt then + Some (Response_message (call_ctxt_origin ctxt) (response.Reject CANISTER_REJECT (encode_string ''Canister has been uninstalled'')) (call_ctxt_available_cycles ctxt)) + else None); + call_ctxt_to_ctxt = (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt) in + S\canisters := list_map_set (canisters S) cid None, certified_data := list_map_set (certified_data S) cid empty_blob, + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles] @ + List.map_filter call_ctxt_to_msg (list_map_vals (call_contexts S)), + call_contexts := list_map_map call_ctxt_to_ctxt (call_contexts S)\))" + +lemma ic_code_uninstallation_cycles_inv: + assumes "ic_code_uninstallation_pre n S" + shows "total_cycles S = total_cycles (ic_code_uninstallation_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_code_uninstallation_pre_def split: message.splits option.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_code_uninstallation_pre_def msg younger_def older_def nth_append) + have F1: "list_map_sum_vals call_ctxt_carried_cycles (call_contexts S) = + list_map_sum_vals id + (list_map_map (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_carried_cycles ctxt else 0) (call_contexts S)) + + list_map_sum_vals call_ctxt_carried_cycles + (list_map_map (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt) (call_contexts S))" + using list_map_sum_vals_split[where ?f="call_ctxt_carried_cycles" and ?g="call_ctxt_delete", unfolded call_ctxt_delete_carried_cycles diff_zero] + by auto + show ?thesis + using assms + by (auto simp: ic_code_uninstallation_pre_def ic_code_uninstallation_post_def total_cycles_def call_ctxt_carried_cycles cid_def Let_def msgs F1 + split: message.splits option.splits sum.splits if_splits intro!: list_map_sum_vals_filter) +qed + +lemma ic_code_uninstallation_ic_inv: + assumes "ic_code_uninstallation_pre n S" "ic_inv S" + shows "ic_inv (ic_code_uninstallation_post n S)" + using assms + by (auto simp: ic_inv_def ic_code_uninstallation_pre_def ic_code_uninstallation_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: IC Management Canister: Stopping a canister (running) [DONE] *) + +definition ic_canister_stop_running_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_stop_running_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''stop_canister'' \ + (case candid_parse_cid a of Some cid \ + (case (list_map_get (canister_status S) cid, list_map_get (controllers S) cid) of (Some Running, Some ctrls) \ + cer \ ctrls + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_canister_stop_running_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_stop_running_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a) in + S\messages := take n (messages S) @ drop (Suc n) (messages S), + canister_status := list_map_set (canister_status S) cid (Stopping [(orig, trans_cycles)])\)" + +lemma ic_canister_stop_running_cycles_inv: + assumes "ic_canister_stop_running_pre n S" + shows "total_cycles S = total_cycles (ic_canister_stop_running_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_canister_stop_running_pre_def split: message.splits option.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_canister_stop_running_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_canister_stop_running_pre_def ic_canister_stop_running_post_def total_cycles_def call_ctxt_carried_cycles cid_def Let_def msgs + list_map_sum_in[where ?g=status_cycles] split: message.splits option.splits sum.splits can_status.splits) +qed + +lemma ic_canister_stop_running_ic_inv: + assumes "ic_canister_stop_running_pre n S" "ic_inv S" + shows "ic_inv (ic_canister_stop_running_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_canister_stop_running_pre_def split: message.splits option.splits) + show ?thesis + using assms + by (force simp: ic_inv_def ic_canister_stop_running_pre_def ic_canister_stop_running_post_def msg Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals intro!: ic_can_status_inv_stopping[where ?x="list_map_range (canister_status S)" and ?os=orig]) +qed + + + +(* System transition: IC Management Canister: Stopping a canister (stopping) [DONE] *) + +definition ic_canister_stop_stopping_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_stop_stopping_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''stop_canister'' \ + (case candid_parse_cid a of Some cid \ + (case (list_map_get (canister_status S) cid, list_map_get (controllers S) cid) of (Some (Stopping os), Some ctrls) \ + cer \ ctrls + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_canister_stop_stopping_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_stop_stopping_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a) in + (case list_map_get (canister_status S) cid of Some (Stopping os) \ + S\messages := take n (messages S) @ drop (Suc n) (messages S), + canister_status := list_map_set (canister_status S) cid (Stopping (os @ [(orig, trans_cycles)]))\))" + +lemma ic_canister_stop_stopping_cycles_inv: + assumes "ic_canister_stop_stopping_pre n S" + shows "total_cycles S = total_cycles (ic_canister_stop_stopping_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_canister_stop_stopping_pre_def split: message.splits option.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_canister_stop_stopping_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_canister_stop_stopping_pre_def ic_canister_stop_stopping_post_def total_cycles_def call_ctxt_carried_cycles cid_def Let_def msgs + list_map_sum_in[where ?g=status_cycles] split: message.splits option.splits sum.splits can_status.splits) +qed + +lemma ic_canister_stop_stopping_ic_inv: + assumes "ic_canister_stop_stopping_pre n S" "ic_inv S" + shows "ic_inv (ic_canister_stop_stopping_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_canister_stop_stopping_pre_def split: message.splits option.splits) + show ?thesis + using assms + by (force simp: ic_inv_def ic_canister_stop_stopping_pre_def ic_canister_stop_stopping_post_def msg Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals intro!: ic_can_status_inv_stopping_app[where ?x="list_map_range (canister_status S)" and ?os=orig]) +qed + + + +(* System transition: IC Management Canister: Stopping a canister (done stopping) [DONE] *) + +definition ic_canister_stop_done_stopping_pre :: "'canid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_stop_done_stopping_pre cid S = + (case list_map_get (canister_status S) cid of Some (Stopping os) \ + (\ctxt \ list_map_range (call_contexts S). call_ctxt_deleted ctxt \ call_ctxt_canister ctxt \ cid) + | _ \ False)" + +definition ic_canister_stop_done_stopping_post :: "'canid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_stop_done_stopping_post cid S = ( + let orig_cycles_to_msg = (\(or, cyc). Response_message or (Reply (blob_of_candid Candid_empty)) cyc) in + (case list_map_get (canister_status S) cid of Some (Stopping os) \ + S\canister_status := list_map_set (canister_status S) cid Stopped, + messages := messages S @ map orig_cycles_to_msg os\))" + +lemma ic_canister_stop_done_stopping_cycles_inv: + assumes "ic_canister_stop_done_stopping_pre cid S" + shows "total_cycles S = total_cycles (ic_canister_stop_done_stopping_post cid S)" +proof - + have F1: "(message_cycles \ (\(or, y). Response_message or (Reply (blob_of_candid Candid_empty)) y)) = (\(or, y). carried_cycles or + y)" + by auto + have F2: "(\(or, y)\xs. carried_cycles or + y) = sum_list (map (carried_cycles \ fst) xs) + sum_list (map snd xs)" for xs + by (induction xs) auto + show ?thesis + using assms list_map_sum_in_ge[where ?g=status_cycles and ?f="canister_status S" and ?x=cid] + by (auto simp: ic_canister_stop_done_stopping_pre_def ic_canister_stop_done_stopping_post_def total_cycles_def call_ctxt_carried_cycles Let_def + F1 F2 list_map_sum_in[where ?g=status_cycles] split: message.splits option.splits sum.splits can_status.splits) +qed + +lemma ic_canister_stop_done_stopping_ic_inv: + assumes "ic_canister_stop_done_stopping_pre cid S" "ic_inv S" + shows "ic_inv (ic_canister_stop_done_stopping_post cid S)" + using assms + by (force simp: ic_inv_def ic_can_status_inv_def ic_canister_stop_done_stopping_pre_def ic_canister_stop_done_stopping_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: IC Management Canister: Stopping a canister (stopped) [DONE] *) + +definition ic_canister_stop_stopped_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_stop_stopped_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''stop_canister'' \ + (case candid_parse_cid a of Some cid \ + (case (list_map_get (canister_status S) cid, list_map_get (controllers S) cid) of (Some Stopped, Some ctrls) \ + cer \ ctrls + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_canister_stop_stopped_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_stop_stopped_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + S\messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)" + +lemma ic_canister_stop_stopped_cycles_inv: + assumes "ic_canister_stop_stopped_pre n S" + shows "total_cycles S = total_cycles (ic_canister_stop_stopped_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_canister_stop_stopped_pre_def split: message.splits option.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_canister_stop_stopped_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_canister_stop_stopped_pre_def ic_canister_stop_stopped_post_def total_cycles_def call_ctxt_carried_cycles cid_def Let_def msgs + split: message.splits option.splits sum.splits can_status.splits) +qed + +lemma ic_canister_stop_stopped_ic_inv: + assumes "ic_canister_stop_stopped_pre n S" "ic_inv S" + shows "ic_inv (ic_canister_stop_stopped_post n S)" + using assms + by (auto simp: ic_inv_def ic_canister_stop_stopped_pre_def ic_canister_stop_stopped_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: IC Management Canister: Starting a canister (not stopping) [DONE] *) + +definition ic_canister_start_not_stopping_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_start_not_stopping_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''start_canister'' \ + (case candid_parse_cid a of Some cid \ + (case (list_map_get (canister_status S) cid, list_map_get (controllers S) cid) of (Some can_status, Some ctrls) \ + (can_status = Running \ can_status = Stopped) \ + cer \ ctrls + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_canister_start_not_stopping_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_start_not_stopping_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a) in + S\canister_status := list_map_set (canister_status S) cid Running, + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)" + +lemma ic_canister_start_not_stopping_cycles_inv: + assumes "ic_canister_start_not_stopping_pre n S" + shows "total_cycles S = total_cycles (ic_canister_start_not_stopping_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_canister_start_not_stopping_pre_def split: message.splits option.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_canister_start_not_stopping_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_canister_start_not_stopping_pre_def ic_canister_start_not_stopping_post_def total_cycles_def call_ctxt_carried_cycles cid_def Let_def msgs + list_map_sum_in[where ?g=status_cycles] split: message.splits option.splits sum.splits) +qed + +lemma ic_canister_start_not_stopping_ic_inv: + assumes "ic_canister_start_not_stopping_pre n S" "ic_inv S" + shows "ic_inv (ic_canister_start_not_stopping_post n S)" + using assms + by (auto simp: ic_inv_def ic_canister_start_not_stopping_pre_def ic_canister_start_not_stopping_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals intro: ic_can_status_inv_mono1) + + + +(* System transition: IC Management Canister: Starting a canister (stopping) [DONE] *) + +definition ic_canister_start_stopping_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_start_stopping_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''start_canister'' \ + (case candid_parse_cid a of Some cid \ + (case (list_map_get (canister_status S) cid, list_map_get (controllers S) cid) of (Some (Stopping _), Some ctrls) \ + cer \ ctrls + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_canister_start_stopping_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_start_stopping_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a); + orig_cycles_to_msg = (\(or, cyc). Response_message or (response.Reject CANISTER_REJECT (encode_string ''Canister has been restarted'')) cyc) in + (case list_map_get (canister_status S) cid of Some (Stopping os) \ + S\canister_status := list_map_set (canister_status S) cid Running, + messages := take n (messages S) @ drop (Suc n) (messages S) @ Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles # + map orig_cycles_to_msg os\))" + +lemma ic_canister_start_stopping_cycles_inv: + assumes "ic_canister_start_stopping_pre n S" + shows "total_cycles S = total_cycles (ic_canister_start_stopping_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_canister_start_stopping_pre_def split: message.splits option.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_canister_start_stopping_pre_def msg younger_def older_def nth_append) + have F1: "(message_cycles \ (\(or, y). Response_message or (response.Reject CANISTER_REJECT (encode_string ''Canister has been restarted'')) y)) = (\(or, y). carried_cycles or + y)" + by auto + have F2: "(\(or, y)\xs. carried_cycles or + y) = sum_list (map (carried_cycles \ fst) xs) + sum_list (map snd xs)" for xs + by (induction xs) auto + show ?thesis + using assms list_map_sum_in_ge[where ?g=status_cycles and ?f="canister_status S" and ?x=cid] + by (auto simp: ic_canister_start_stopping_pre_def ic_canister_start_stopping_post_def total_cycles_def call_ctxt_carried_cycles cid_def Let_def msgs F1 F2 + list_map_sum_in[where ?g=status_cycles and ?f="canister_status S"] + split: message.splits option.splits sum.splits can_status.splits) +qed + +lemma ic_canister_start_stopping_ic_inv: + assumes "ic_canister_start_stopping_pre n S" "ic_inv S" + shows "ic_inv (ic_canister_start_stopping_post n S)" + using assms + apply (auto simp: ic_inv_def ic_canister_start_stopping_pre_def ic_canister_start_stopping_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del in_set_map_filter_vals + intro: ic_can_status_inv_mono1) + apply (fastforce simp: ic_can_status_inv_def)+ + done + + + +(* System transition: IC Management Canister: Canister deletion [DONE] *) + +definition ic_canister_deletion_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_canister_deletion_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''delete_canister'' \ + (case candid_parse_cid a of Some cid \ + (case (list_map_get (canister_status S) cid, list_map_get (controllers S) cid, list_map_get (balances S) cid) of (Some Stopped, Some ctrls, Some bal) \ + cer \ ctrls + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_canister_deletion_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_canister_deletion_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a) in + S\canisters := list_map_del (canisters S) cid, + controllers := list_map_del (controllers S) cid, + freezing_threshold := list_map_del (freezing_threshold S) cid, + canister_status := list_map_del (canister_status S) cid, + time := list_map_del (time S) cid, + balances := list_map_del (balances S) cid, + certified_data := list_map_del (certified_data S) cid, + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)" + +definition ic_canister_deletion_burned_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "ic_canister_deletion_burned_cycles n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a) in the (list_map_get (balances S) cid))" + +lemma ic_canister_deletion_cycles_monotonic: + assumes "ic_canister_deletion_pre n S" + shows "total_cycles S = total_cycles (ic_canister_deletion_post n S) + ic_canister_deletion_burned_cycles n S" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_canister_deletion_pre_def split: message.splits option.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_canister_deletion_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_canister_deletion_pre_def ic_canister_deletion_post_def ic_canister_deletion_burned_cycles_def total_cycles_def call_ctxt_carried_cycles cid_def Let_def msgs + list_map_del_sum[where ?g=id and ?f="balances S"] list_map_del_sum[where ?g=status_cycles and ?f="canister_status S"] + split: message.splits option.splits sum.splits can_status.splits) +qed + +lemma ic_canister_deletion_ic_inv: + assumes "ic_canister_deletion_pre n S" "ic_inv S" + shows "ic_inv (ic_canister_deletion_post n S)" + using assms + by (auto simp: ic_inv_def ic_canister_deletion_pre_def ic_canister_deletion_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del in_set_map_filter_vals + intro: ic_can_status_inv_mono1) + + + +(* System transition: IC Management Canister: Depositing cycles [DONE] *) + +definition ic_depositing_cycles_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_depositing_cycles_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''deposit_cycles'' \ + (case candid_parse_cid a of Some cid \ + (case list_map_get (balances S) cid of Some bal \ + True + | _ \ False) | _ \ False) + | _ \ False))" + +definition ic_depositing_cycles_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_depositing_cycles_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a) in + (case list_map_get (balances S) cid of Some bal \ + S\balances := list_map_set (balances S) cid (bal + trans_cycles), + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) 0]\))" + +lemma ic_depositing_cycles_cycles_monotonic: + assumes "ic_depositing_cycles_pre n S" + shows "total_cycles S = total_cycles (ic_depositing_cycles_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q cid where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + and cid_def: "candid_parse_cid a = Some cid" + using assms + by (auto simp: ic_depositing_cycles_pre_def split: message.splits option.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_depositing_cycles_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms list_map_sum_in_ge[where ?g=id and ?f="balances S" and ?x=cid] + by (auto simp: ic_depositing_cycles_pre_def ic_depositing_cycles_post_def total_cycles_def call_ctxt_carried_cycles cid_def Let_def msgs + list_map_sum_in[where ?g=id and ?f="balances S"] min_def split: message.splits option.splits sum.splits) +qed + +lemma ic_depositing_cycles_ic_inv: + assumes "ic_depositing_cycles_pre n S" "ic_inv S" + shows "ic_inv (ic_depositing_cycles_post n S)" + using assms + by (auto simp: ic_inv_def ic_depositing_cycles_pre_def ic_depositing_cycles_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: IC Management Canister: Random numbers [DONE] *) + +definition ic_random_numbers_pre :: "nat \ 'b \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_random_numbers_pre n b S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''raw_rand'' \ + a = blob_of_candid Candid_empty \ + blob_length b = 32 + | _ \ False))" + +definition ic_random_numbers_post :: "nat \ 'b \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_random_numbers_post n b S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + S\messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid (Candid_blob b))) trans_cycles]\)" + +lemma ic_random_numbers_cycles_inv: + assumes "ic_random_numbers_pre n b S" + shows "total_cycles S = total_cycles (ic_random_numbers_post n b S)" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: ic_random_numbers_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_random_numbers_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_random_numbers_pre_def ic_random_numbers_post_def total_cycles_def call_ctxt_carried_cycles Let_def msgs) +qed + +lemma ic_random_numbers_ic_inv: + assumes "ic_random_numbers_pre n b S" "ic_inv S" + shows "ic_inv (ic_random_numbers_post n b S)" + using assms + by (auto simp: ic_inv_def ic_random_numbers_pre_def ic_random_numbers_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: IC Management Canister: Canister creation with cycles [DONE] *) + +definition ic_provisional_canister_creation_pre :: "nat \ 'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_provisional_canister_creation_pre n cid t S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (case candid_parse_nat a [encode_string ''amount''] of Some cyc \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''provisional_create_canister_with_cycles'' \ + is_system_assigned (principal_of_canid cid) \ + cid \ list_map_dom (canisters S) \ + cid \ list_map_dom (time S) \ + cid \ list_map_dom (controllers S) \ + cid \ list_map_dom (balances S) \ + cid \ list_map_dom (certified_data S) \ + cid \ list_map_dom (canister_status S) + | _ \ False) | _ \ False))" + +definition ic_provisional_canister_creation_post :: "nat \ 'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_provisional_canister_creation_post n cid t S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cyc = the (candid_parse_nat a [encode_string ''amount'']) in + S\canisters := list_map_set (canisters S) cid None, + time := list_map_set (time S) cid t, + controllers := list_map_set (controllers S) cid {cer}, + freezing_threshold := list_map_set (freezing_threshold S) cid 2592000, + balances := list_map_set (balances S) cid cyc, + certified_data := list_map_set (certified_data S) cid empty_blob, + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid + (Candid_record (list_map_init [(encode_string ''canister_id'', Candid_blob (blob_of_canid cid))])))) trans_cycles], + canister_status := list_map_set (canister_status S) cid Running\)" + +definition ic_provisional_canister_creation_minted_cycles :: "nat \ 'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "ic_provisional_canister_creation_minted_cycles n cid t S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + the (candid_parse_nat a [encode_string ''amount'']))" + +lemma ic_provisional_canister_creation_cycles_antimonotonic: + assumes "ic_provisional_canister_creation_pre n cid t S" + shows "total_cycles S + ic_provisional_canister_creation_minted_cycles n cid t S = total_cycles (ic_provisional_canister_creation_post n cid t S)" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: ic_provisional_canister_creation_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_provisional_canister_creation_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: ic_provisional_canister_creation_pre_def ic_provisional_canister_creation_post_def ic_provisional_canister_creation_minted_cycles_def total_cycles_def Let_def msgs + list_map_sum_out[where ?g=id] list_map_sum_out[where ?g=status_cycles] split: message.splits option.splits) +qed + +lemma ic_provisional_canister_creation_ic_inv: + assumes "ic_provisional_canister_creation_pre n cid t S" "ic_inv S" + shows "ic_inv (ic_provisional_canister_creation_post n cid t S)" + using assms + by (auto simp: ic_inv_def ic_provisional_canister_creation_pre_def ic_provisional_canister_creation_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del in_set_map_filter_vals + intro: ic_can_status_inv_mono1) + + + +(* System transition: IC Management Canister: Top up canister [DONE] *) + +definition ic_top_up_canister_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "ic_top_up_canister_pre n S = (n < length (messages S) \ (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + (case (candid_parse_cid a, candid_parse_nat a [encode_string ''amount'']) of (Some cid, Some cyc) \ + (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ + cee = ic_principal \ + mn = encode_string ''provisional_top_up_canister'' \ + cid \ list_map_dom (balances S) + | _ \ False) | _ \ False))" + +definition ic_top_up_canister_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "ic_top_up_canister_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a); + cyc = the (candid_parse_nat a [encode_string ''amount'']); + bal = the (list_map_get (balances S) cid) in + S\balances := list_map_set (balances S) cid (bal + cyc)\)" + +definition ic_top_up_canister_minted_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "ic_top_up_canister_minted_cycles n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + the (candid_parse_nat a [encode_string ''amount'']))" + +lemma ic_top_up_canister_cycles_antimonotonic: + assumes "ic_top_up_canister_pre n S" + shows "total_cycles S + ic_top_up_canister_minted_cycles n S = total_cycles (ic_top_up_canister_post n S)" +proof - + obtain orig cer cee mn a trans_cycles q where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + using assms + by (auto simp: ic_top_up_canister_pre_def split: message.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Call_message orig cer cee mn a trans_cycles q # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: ic_top_up_canister_pre_def msg younger_def older_def nth_append) + define cid where "cid = the (candid_parse_cid a)" + show ?thesis + using assms + by (cases "list_map_get (balances S) cid") + (auto simp: ic_top_up_canister_pre_def ic_top_up_canister_post_def ic_top_up_canister_minted_cycles_def total_cycles_def Let_def msgs cid_def + list_map_sum_in[where ?g=id] split: message.splits option.splits) +qed + +lemma ic_top_up_canister_ic_inv: + assumes "ic_top_up_canister_pre n S" "ic_inv S" + shows "ic_inv (ic_top_up_canister_post n S)" + using assms + by (auto simp: ic_inv_def ic_top_up_canister_pre_def ic_top_up_canister_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: Callback invocation (not deleted) [DONE] *) + +definition callback_invocation_not_deleted_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "callback_invocation_not_deleted_pre n S = (n < length (messages S) \ (case messages S ! n of Response_message (From_canister ctxt_id c) resp ref_cycles \ + (case list_map_get (call_contexts S) ctxt_id of Some ctxt \ + let cid = call_ctxt_canister ctxt in + \call_ctxt_deleted ctxt \ + (case list_map_get (balances S) cid of Some bal \ True + | _ \ False) + | _ \ False) + | _ \ False))" + +definition callback_invocation_not_deleted_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "callback_invocation_not_deleted_post n S = (case messages S ! n of Response_message (From_canister ctxt_id c) resp ref_cycles \ + (case list_map_get (call_contexts S) ctxt_id of Some ctxt \ + let cid = call_ctxt_canister ctxt in + (case list_map_get (balances S) cid of Some bal \ + S\balances := list_map_set (balances S) cid (bal + ref_cycles), + messages := list_update (messages S) n (Func_message ctxt_id cid (Callback c resp ref_cycles) Unordered)\)))" + +lemma callback_invocation_not_deleted_cycles_inv: + assumes "callback_invocation_not_deleted_pre n S" + shows "total_cycles S = total_cycles (callback_invocation_not_deleted_post n S)" +proof - + obtain ctxt_id c resp ref_cycles where msg: "messages S ! n = Response_message (From_canister ctxt_id c) resp ref_cycles" + using assms + by (auto simp: callback_invocation_not_deleted_pre_def split: message.splits option.splits call_origin.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Response_message (From_canister ctxt_id c) resp ref_cycles # younger" "(older @ w # younger) ! n = w" + and msgs_upd: "(older @ Response_message (From_canister ctxt_id c) resp ref_cycles # younger)[n := m] = older @ m # younger" for w m + using assms id_take_nth_drop[of n "messages S"] upd_conv_take_nth_drop[of n "messages S"] assms + by (auto simp: callback_invocation_not_deleted_pre_def msg older_def younger_def nth_append) + show ?thesis + using assms + by (auto simp: callback_invocation_not_deleted_pre_def callback_invocation_not_deleted_post_def total_cycles_def call_ctxt_carried_cycles Let_def msgs msgs_upd + list_map_sum_in[where ?g=id and ?f="balances S"] split: option.splits) +qed + +lemma callback_invocation_not_deleted_ic_inv: + assumes "callback_invocation_not_deleted_pre n S" "ic_inv S" + shows "ic_inv (callback_invocation_not_deleted_post n S)" + using assms + by (auto simp: ic_inv_def callback_invocation_not_deleted_pre_def callback_invocation_not_deleted_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD) + + + +(* System transition: Callback invocation (deleted) [DONE] *) + +definition callback_invocation_deleted_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "callback_invocation_deleted_pre n S = (n < length (messages S) \ (case messages S ! n of Response_message (From_canister ctxt_id c) resp ref_cycles \ + (case list_map_get (call_contexts S) ctxt_id of Some ctxt \ + let cid = call_ctxt_canister ctxt in + call_ctxt_deleted ctxt \ + (case list_map_get (balances S) cid of Some bal \ True + | _ \ False) + | _ \ False) + | _ \ False))" + +definition callback_invocation_deleted_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "callback_invocation_deleted_post n S = (case messages S ! n of Response_message (From_canister ctxt_id c) resp ref_cycles \ + (case list_map_get (call_contexts S) ctxt_id of Some ctxt \ + let cid = call_ctxt_canister ctxt in + (case list_map_get (balances S) cid of Some bal \ + S\balances := list_map_set (balances S) cid (bal + ref_cycles + MAX_CYCLES_PER_RESPONSE), + messages := take n (messages S) @ drop (Suc n) (messages S)\)))" + +lemma callback_invocation_deleted_cycles_inv: + assumes "callback_invocation_deleted_pre n S" + shows "total_cycles S = total_cycles (callback_invocation_deleted_post n S)" +proof - + obtain ctxt_id c resp ref_cycles where msg: "messages S ! n = Response_message (From_canister ctxt_id c) resp ref_cycles" + using assms + by (auto simp: callback_invocation_deleted_pre_def split: message.splits option.splits call_origin.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Response_message (From_canister ctxt_id c) resp ref_cycles # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: callback_invocation_deleted_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: callback_invocation_deleted_pre_def callback_invocation_deleted_post_def total_cycles_def call_ctxt_carried_cycles Let_def msgs + list_map_sum_in[where ?g=id and ?f="balances S"] split: option.splits) +qed + +lemma callback_invocation_deleted_ic_inv: + assumes "callback_invocation_deleted_pre n S" "ic_inv S" + shows "ic_inv (callback_invocation_deleted_post n S)" + using assms + by (auto simp: ic_inv_def callback_invocation_deleted_pre_def callback_invocation_deleted_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD) + + + +(* System transition: Respond to user request [DONE] *) + +definition respond_to_user_request_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "respond_to_user_request_pre n S = (n < length (messages S) \ (case messages S ! n of Response_message (From_user req) resp ref_cycles \ + (case list_map_get (requests S) req of Some Processing \ True + | _ \ False) + | _ \ False))" + +definition respond_to_user_request_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "respond_to_user_request_post n S = (case messages S ! n of Response_message (From_user req) resp ref_cycles \ + let req_resp = (case resp of Reply b \ Replied b | response.Reject c b \ Rejected c b) in + S\messages := take n (messages S) @ drop (Suc n) (messages S), + requests := list_map_set (requests S) req req_resp\)" + +definition respond_to_user_request_burned_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "respond_to_user_request_burned_cycles n S = (case messages S ! n of Response_message (From_user req) resp ref_cycles \ + ref_cycles)" + +lemma respond_to_user_request_cycles_monotonic: + assumes "respond_to_user_request_pre n S" + shows "total_cycles S = total_cycles (respond_to_user_request_post n S) + respond_to_user_request_burned_cycles n S" +proof - + obtain req resp ref_cycles where msg: "messages S ! n = Response_message (From_user req) resp ref_cycles" + using assms + by (auto simp: respond_to_user_request_pre_def split: message.splits option.splits call_origin.splits) + define older where "older = take n (messages S)" + define younger where "younger = drop (Suc n) (messages S)" + have msgs: "messages S = older @ Response_message (From_user req) resp ref_cycles # younger" "(older @ w # younger) ! n = w" + "take n older = older" "take (n - length older) ws = []" "drop (Suc n) older = []" + "drop (Suc n - length older) (w # ws) = ws" for w ws + using id_take_nth_drop[of n "messages S"] assms + by (auto simp: respond_to_user_request_pre_def msg younger_def older_def nth_append) + show ?thesis + using assms + by (auto simp: respond_to_user_request_pre_def respond_to_user_request_post_def respond_to_user_request_burned_cycles_def total_cycles_def call_ctxt_carried_cycles Let_def msgs + list_map_sum_in[where ?g=id and ?f="balances S"] split: option.splits request_status.splits) +qed + +lemma respond_to_user_request_ic_inv: + assumes "respond_to_user_request_pre n S" "ic_inv S" + shows "ic_inv (respond_to_user_request_post n S)" + using assms + by (auto simp: ic_inv_def respond_to_user_request_pre_def respond_to_user_request_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD) + + + +(* System transition: Request clean up [DONE] *) + +definition request_cleanup_pre :: "('b, 'p, 'uid, 'canid, 's) request \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "request_cleanup_pre req S = (case list_map_get (requests S) req of Some req_status \ + (case req_status of Replied _ \ True | Rejected _ _ \ True | _ \ False) + | _ \ False)" + +definition request_cleanup_post :: "('b, 'p, 'uid, 'canid, 's) request \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "request_cleanup_post req S = (S\requests := list_map_set (requests S) req Done\)" + +lemma request_cleanup_cycles_inv: + assumes "request_cleanup_pre n S" + shows "total_cycles S = total_cycles (request_cleanup_post n S)" + by (auto simp: request_cleanup_post_def total_cycles_def) + +lemma request_cleanup_ic_inv: + assumes "request_cleanup_pre req S" "ic_inv S" + shows "ic_inv (request_cleanup_post req S)" + using assms + by (auto simp: ic_inv_def request_cleanup_pre_def request_cleanup_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD in_set_updD) + + + +(* System transition: Request clean up (expired) [DONE] *) + +definition request_cleanup_expired_pre :: "('b, 'p, 'uid, 'canid, 's) request \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "request_cleanup_expired_pre req S = (case list_map_get (requests S) req of Some req_status \ + (case req_status of Replied _ \ True | Rejected _ _ \ True | Done \ True | _ \ False) \ + request.ingress_expiry req < system_time S + | _ \ False)" + +definition request_cleanup_expired_post :: "('b, 'p, 'uid, 'canid, 's) request \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "request_cleanup_expired_post req S = (S\requests := list_map_del (requests S) req\)" + +lemma request_cleanup_expired_cycles_inv: + assumes "request_cleanup_expired_pre n S" + shows "total_cycles S = total_cycles (request_cleanup_expired_post n S)" + by (auto simp: request_cleanup_expired_post_def total_cycles_def) + +lemma request_cleanup_expired_ic_inv: + assumes "request_cleanup_expired_pre req S" "ic_inv S" + shows "ic_inv (request_cleanup_expired_post req S)" + using assms + by (auto simp: ic_inv_def request_cleanup_expired_pre_def request_cleanup_expired_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD in_set_updD) + + + +(* System transition: Canister out of cycles [DONE] *) + +definition canister_out_of_cycles_pre :: "'canid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "canister_out_of_cycles_pre cid S = (case list_map_get (balances S) cid of Some 0 \ True + | _ \ False)" + +definition canister_out_of_cycles_post :: "'canid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "canister_out_of_cycles_post cid S = ( + let call_ctxt_to_msg = (\ctxt. + if call_ctxt_canister ctxt = cid \ call_ctxt_needs_to_respond ctxt then + Some (Response_message (call_ctxt_origin ctxt) (response.Reject CANISTER_REJECT (encode_string ''Canister has been uninstalled'')) (call_ctxt_available_cycles ctxt)) + else None); + call_ctxt_to_ctxt = (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt) in + S\canisters := list_map_set (canisters S) cid None, certified_data := list_map_set (certified_data S) cid empty_blob, + messages := messages S @ List.map_filter call_ctxt_to_msg (list_map_vals (call_contexts S)), + call_contexts := list_map_map call_ctxt_to_ctxt (call_contexts S)\)" + +lemma canister_out_of_cycles_cycles_inv: + assumes "canister_out_of_cycles_pre cid S" + shows "total_cycles S = total_cycles (canister_out_of_cycles_post cid S)" +proof - + have F1: "list_map_sum_vals call_ctxt_carried_cycles (call_contexts S) = + list_map_sum_vals id + (list_map_map (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_carried_cycles ctxt else 0) (call_contexts S)) + + list_map_sum_vals call_ctxt_carried_cycles + (list_map_map (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt) (call_contexts S))" + using list_map_sum_vals_split[where ?f="call_ctxt_carried_cycles" and ?g="call_ctxt_delete", unfolded call_ctxt_delete_carried_cycles diff_zero] + by auto + show ?thesis + using assms + by (auto simp: canister_out_of_cycles_pre_def canister_out_of_cycles_post_def total_cycles_def call_ctxt_carried_cycles Let_def F1 + split: message.splits option.splits sum.splits if_splits intro!: list_map_sum_vals_filter) +qed + +lemma canister_out_of_cycles_ic_inv: + assumes "canister_out_of_cycles_pre cid S" "ic_inv S" + shows "ic_inv (canister_out_of_cycles_post cid S)" + using assms + by (auto simp: ic_inv_def canister_out_of_cycles_pre_def canister_out_of_cycles_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: Time progressing and cycle consumption (canister time) [DONE] *) + +definition canister_time_progress_pre :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "canister_time_progress_pre cid t1 S = (case list_map_get (time S) cid of Some t0 \ + t0 < t1 + | _ \ False)" + +definition canister_time_progress_post :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "canister_time_progress_post cid t1 S = (S\time := list_map_set (time S) cid t1\)" + +lemma canister_time_progress_cycles_inv: + assumes "canister_time_progress_pre cid t1 S" + shows "total_cycles S = total_cycles (canister_time_progress_post cid t1 S)" + by (auto simp: canister_time_progress_post_def total_cycles_def) + +lemma canister_time_progress_ic_inv: + assumes "canister_time_progress_pre cid t1 S" "ic_inv S" + shows "ic_inv (canister_time_progress_post cid t1 S)" + using assms + by (auto simp: ic_inv_def canister_time_progress_pre_def canister_time_progress_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: Time progressing and cycle consumption (cycle consumption) [DONE] *) + +definition cycle_consumption_pre :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "cycle_consumption_pre cid b1 S = (case list_map_get (balances S) cid of Some b0 \ + 0 \ b1 \ b1 < b0 + | _ \ False)" + +definition cycle_consumption_post :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "cycle_consumption_post cid b1 S = (S\balances := list_map_set (balances S) cid b1\)" + +definition cycle_consumption_burned_cycles :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where + "cycle_consumption_burned_cycles cid b1 S = the (list_map_get (balances S) cid) - b1" + +lemma cycle_consumption_cycles_monotonic: + assumes "cycle_consumption_pre cid b1 S" + shows "total_cycles S = total_cycles (cycle_consumption_post cid b1 S) + cycle_consumption_burned_cycles cid b1 S" + using assms list_map_sum_in_ge[where ?g=id and ?f="balances S" and ?x=cid] + by (auto simp: cycle_consumption_pre_def cycle_consumption_post_def cycle_consumption_burned_cycles_def total_cycles_def + list_map_sum_in[where ?g=id and ?f="balances S"] split: option.splits) + +lemma cycle_consumption_ic_inv: + assumes "cycle_consumption_pre cid b1 S" "ic_inv S" + shows "ic_inv (cycle_consumption_post cid b1 S)" + using assms + by (auto simp: ic_inv_def cycle_consumption_pre_def cycle_consumption_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* System transition: Time progressing and cycle consumption (system time) [DONE] *) + +definition system_time_progress_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "system_time_progress_pre t1 S = (system_time S < t1)" + +definition system_time_progress_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "system_time_progress_post t1 S = (S\system_time := t1\)" + +lemma system_time_progress_cycles_inv: + assumes "system_time_progress_pre t1 S" + shows "total_cycles S = total_cycles (system_time_progress_post t1 S)" + by (auto simp: system_time_progress_post_def total_cycles_def) + +lemma system_time_progress_ic_inv: + assumes "system_time_progress_pre t1 S" "ic_inv S" + shows "ic_inv (system_time_progress_post t1 S)" + using assms + by (auto simp: ic_inv_def system_time_progress_pre_def system_time_progress_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + +(* State machine *) + +inductive ic_steps :: "'sig itself \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ + nat \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + ic_steps_refl: "ic_steps sig S 0 0 S" +| request_submission: "ic_steps sig S0 minted burned S \ request_submission_pre (E :: ('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope) ECID S \ ic_steps sig S0 minted (burned + request_submission_burned_cycles E ECID S) (request_submission_post E ECID S)" +| request_rejection: "ic_steps sig S0 minted burned S \ request_rejection_pre (E :: ('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) envelope) req code msg S \ ic_steps sig S0 minted burned (request_rejection_post E req code msg S)" +| initiate_canister_call: "ic_steps sig S0 minted burned S \ initiate_canister_call_pre req S \ ic_steps sig S0 minted burned (initiate_canister_call_post req S)" +| call_reject: "ic_steps sig S0 minted burned S \ call_reject_pre n S \ ic_steps sig S0 minted burned (call_reject_post n S)" +| call_context_create: "ic_steps sig S0 minted burned S \ call_context_create_pre n ctxt_id S \ ic_steps sig S0 minted burned (call_context_create_post n ctxt_id S)" +| call_context_heartbeat: "ic_steps sig S0 minted burned S \ call_context_heartbeat_pre cee ctxt_id S \ ic_steps sig S0 minted burned (call_context_heartbeat_post cee ctxt_id S)" +| message_execution: "ic_steps sig S0 minted burned S \ message_execution_pre n S \ ic_steps sig S0 minted (burned + message_execution_burned_cycles n S) (message_execution_post n S)" +| call_context_starvation: "ic_steps sig S0 minted burned S \ call_context_starvation_pre ctxt_id S \ ic_steps sig S0 minted burned (call_context_starvation_post ctxt_id S)" +| call_context_removal: "ic_steps sig S0 minted burned S \ call_context_removal_pre ctxt_id S \ ic_steps sig S0 minted (burned + call_context_removal_burned_cycles ctxt_id S) (call_context_removal_post ctxt_id S)" +| ic_canister_creation: "ic_steps sig S0 minted burned S \ ic_canister_creation_pre n cid t S \ ic_steps sig S0 minted burned (ic_canister_creation_post n cid t S)" +| ic_update_settings: "ic_steps sig S0 minted burned S \ ic_update_settings_pre n S \ ic_steps sig S0 minted burned (ic_update_settings_post n S)" +| ic_canister_status: "ic_steps sig S0 minted burned S \ ic_canister_status_pre n m S \ ic_steps sig S0 minted burned (ic_canister_status_post n m S)" +| ic_code_installation: "ic_steps sig S0 minted burned S \ ic_code_installation_pre n S \ ic_steps sig S0 minted (burned + ic_code_installation_burned_cycles n S) (ic_code_installation_post n S)" +| ic_code_upgrade: "ic_steps sig S0 minted burned S \ ic_code_upgrade_pre n S \ ic_steps sig S0 minted (burned + ic_code_upgrade_burned_cycles n S) (ic_code_upgrade_post n S)" +| ic_code_uninstallation: "ic_steps sig S0 minted burned S \ ic_code_uninstallation_pre n S \ ic_steps sig S0 minted burned (ic_code_uninstallation_post n S)" +| ic_canister_stop_running: "ic_steps sig S0 minted burned S \ ic_canister_stop_running_pre n S \ ic_steps sig S0 minted burned (ic_canister_stop_running_post n S)" +| ic_canister_stop_stopping: "ic_steps sig S0 minted burned S \ ic_canister_stop_stopping_pre n S \ ic_steps sig S0 minted burned (ic_canister_stop_stopping_post n S)" +| ic_canister_stop_done_stopping: "ic_steps sig S0 minted burned S \ ic_canister_stop_done_stopping_pre cid S \ ic_steps sig S0 minted burned (ic_canister_stop_done_stopping_post cid S)" +| ic_canister_stop_stopped: "ic_steps sig S0 minted burned S \ ic_canister_stop_stopped_pre n S \ ic_steps sig S0 minted burned (ic_canister_stop_stopped_post n S)" +| ic_canister_start_not_stopping: "ic_steps sig S0 minted burned S \ ic_canister_start_not_stopping_pre n S \ ic_steps sig S0 minted burned (ic_canister_start_not_stopping_post n S)" +| ic_canister_start_stopping: "ic_steps sig S0 minted burned S \ ic_canister_start_stopping_pre n S \ ic_steps sig S0 minted burned (ic_canister_start_stopping_post n S)" +| ic_canister_deletion: "ic_steps sig S0 minted burned S \ ic_canister_deletion_pre n S \ ic_steps sig S0 minted (burned + ic_canister_deletion_burned_cycles n S) (ic_canister_deletion_post n S)" +| ic_depositing_cycles: "ic_steps sig S0 minted burned S \ ic_depositing_cycles_pre n S \ ic_steps sig S0 minted burned (ic_depositing_cycles_post n S)" +| ic_random_numbers: "ic_steps sig S0 minted burned S \ ic_random_numbers_pre n b S \ ic_steps sig S0 minted burned (ic_random_numbers_post n b S)" +| ic_provisional_canister_creation: "ic_steps sig S0 minted burned S \ ic_provisional_canister_creation_pre n cid t S \ ic_steps sig S0 (minted + ic_provisional_canister_creation_minted_cycles n cid t S) burned (ic_provisional_canister_creation_post n cid t S)" +| ic_top_up_canister: "ic_steps sig S0 minted burned S \ ic_top_up_canister_pre n S \ ic_steps sig S0 (minted + ic_top_up_canister_minted_cycles n S) burned (ic_top_up_canister_post n S)" +| callback_invocation_not_deleted: "ic_steps sig S0 minted burned S \ callback_invocation_not_deleted_pre n S \ ic_steps sig S0 minted burned (callback_invocation_not_deleted_post n S)" +| callback_invocation_deleted: "ic_steps sig S0 minted burned S \ callback_invocation_deleted_pre n S \ ic_steps sig S0 minted burned (callback_invocation_deleted_post n S)" +| respond_to_user_request: "ic_steps sig S0 minted burned S \ respond_to_user_request_pre n S \ ic_steps sig S0 minted (burned + respond_to_user_request_burned_cycles n S) (respond_to_user_request_post n S)" +| request_cleanup: "ic_steps sig S0 minted burned S \ request_cleanup_pre req S \ ic_steps sig S0 minted burned (request_cleanup_post req S)" +| request_cleanup_expired: "ic_steps sig S0 minted burned S \ request_cleanup_expired_pre req S \ ic_steps sig S0 minted burned (request_cleanup_expired_post req S)" +| canister_out_of_cycles: "ic_steps sig S0 minted burned S \ canister_out_of_cycles_pre cid S \ ic_steps sig S0 minted burned (canister_out_of_cycles_post cid S)" +| canister_time_progress: "ic_steps sig S0 minted burned S \ canister_time_progress_pre cid t1 S \ ic_steps sig S0 minted burned (canister_time_progress_post cid t1 S)" +| cycle_consumption: "ic_steps sig S0 minted burned S \ cycle_consumption_pre cid b1 S \ ic_steps sig S0 minted (burned + cycle_consumption_burned_cycles cid b1 S) (cycle_consumption_post cid b1 S)" +| system_time_progress: "ic_steps sig S0 minted burned S \ system_time_progress_pre t1 S \ ic_steps sig S0 minted burned (system_time_progress_post t1 S)" + +lemma total_cycles: + assumes "ic_steps TYPE('sig) S0 minted burned S" + shows "total_cycles S0 + minted = total_cycles S + burned" + using assms + apply (induction "TYPE('sig)" S0 minted burned S rule: ic_steps.induct) + apply auto[1] + using request_submission_cycles_inv apply fastforce + using request_rejection_cycles_inv apply fastforce + using initiate_canister_call_cycles_inv apply fastforce + using call_reject_cycles_inv apply fastforce + using call_context_create_cycles_inv apply fastforce + using call_context_heartbeat_cycles_inv apply fastforce + using message_execution_cycles_monotonic apply fastforce + using call_context_starvation_cycles_inv apply fastforce + using call_context_removal_cycles_monotonic apply fastforce + using ic_canister_creation_cycles_inv apply fastforce + using ic_update_settings_cycles_inv apply fastforce + using ic_canister_status_cycles_inv apply fastforce + using ic_code_installation_cycles_inv apply fastforce + using ic_code_upgrade_cycles_inv apply fastforce + using ic_code_uninstallation_cycles_inv apply fastforce + using ic_canister_stop_running_cycles_inv apply fastforce + using ic_canister_stop_stopping_cycles_inv apply fastforce + using ic_canister_stop_done_stopping_cycles_inv apply fastforce + using ic_canister_stop_stopped_cycles_inv apply fastforce + using ic_canister_start_not_stopping_cycles_inv apply fastforce + using ic_canister_start_stopping_cycles_inv apply fastforce + using ic_canister_deletion_cycles_monotonic apply fastforce + using ic_depositing_cycles_cycles_monotonic apply fastforce + using ic_random_numbers_cycles_inv apply fastforce + using ic_provisional_canister_creation_cycles_antimonotonic apply fastforce + using ic_top_up_canister_cycles_antimonotonic apply fastforce + using callback_invocation_not_deleted_cycles_inv apply fastforce + using callback_invocation_deleted_cycles_inv apply fastforce + using respond_to_user_request_cycles_monotonic apply fastforce + using request_cleanup_cycles_inv apply fastforce + using request_cleanup_expired_cycles_inv apply fastforce + using canister_out_of_cycles_cycles_inv apply fastforce + using canister_time_progress_cycles_inv apply fastforce + using cycle_consumption_cycles_monotonic apply fastforce + using system_time_progress_cycles_inv apply fastforce + done + +lemma ic_inv: + assumes "ic_steps TYPE('sig) S0 minted burned S" + shows "ic_inv S0 \ ic_inv S" + using assms + apply (induction "TYPE('sig)" S0 minted burned S rule: ic_steps.induct) + apply auto[1] + using request_submission_ic_inv apply fastforce + using request_rejection_ic_inv apply fastforce + using initiate_canister_call_ic_inv apply fastforce + using call_reject_ic_inv apply fastforce + using call_context_create_ic_inv apply fastforce + using call_context_heartbeat_ic_inv apply fastforce + using message_execution_ic_inv apply fastforce + using call_context_starvation_ic_inv apply fastforce + using call_context_removal_ic_inv apply fastforce + using ic_canister_creation_ic_inv apply fastforce + using ic_update_settings_ic_inv apply fastforce + using ic_canister_status_ic_inv apply fastforce + using ic_code_installation_ic_inv apply fastforce + using ic_code_upgrade_ic_inv apply fastforce + using ic_code_uninstallation_ic_inv apply fastforce + using ic_canister_stop_running_ic_inv apply fastforce + using ic_canister_stop_stopping_ic_inv apply fastforce + using ic_canister_stop_done_stopping_ic_inv apply fastforce + using ic_canister_stop_stopped_ic_inv apply fastforce + using ic_canister_start_not_stopping_ic_inv apply fastforce + using ic_canister_start_stopping_ic_inv apply fastforce + using ic_canister_deletion_ic_inv apply fastforce + using ic_depositing_cycles_ic_inv apply fastforce + using ic_random_numbers_ic_inv apply fastforce + using ic_provisional_canister_creation_ic_inv apply fastforce + using ic_top_up_canister_ic_inv apply fastforce + using callback_invocation_not_deleted_ic_inv apply fastforce + using callback_invocation_deleted_ic_inv apply fastforce + using respond_to_user_request_ic_inv apply fastforce + using request_cleanup_ic_inv apply fastforce + using request_cleanup_expired_ic_inv apply fastforce + using canister_out_of_cycles_ic_inv apply fastforce + using canister_time_progress_ic_inv apply fastforce + using cycle_consumption_ic_inv apply fastforce + using system_time_progress_ic_inv apply fastforce + done + +end + +export_code request_submission_pre request_submission_post + request_rejection_pre request_rejection_post + initiate_canister_call_pre initiate_canister_call_post + call_reject_pre call_reject_post + call_context_create_pre call_context_create_post + call_context_heartbeat_pre call_context_heartbeat_post + message_execution_pre message_execution_post + call_context_starvation_pre call_context_starvation_post + call_context_removal_pre call_context_removal_post + ic_canister_creation_pre ic_canister_creation_post + ic_update_settings_pre ic_update_settings_post + ic_canister_status_pre ic_canister_status_post + ic_code_installation_pre ic_code_installation_post + ic_code_upgrade_pre ic_code_upgrade_post + ic_code_uninstallation_pre ic_code_uninstallation_post + ic_canister_stop_running_pre ic_canister_stop_running_post + ic_canister_stop_stopping_pre ic_canister_stop_stopping_post + ic_canister_stop_done_stopping_pre ic_canister_stop_done_stopping_post + ic_canister_stop_stopped_pre ic_canister_stop_stopped_post + ic_canister_start_not_stopping_pre ic_canister_start_not_stopping_post + ic_canister_start_stopping_pre ic_canister_start_stopping_post + ic_canister_deletion_pre ic_canister_deletion_post + ic_depositing_cycles_pre ic_depositing_cycles_post + ic_random_numbers_pre ic_random_numbers_post + ic_provisional_canister_creation_pre ic_provisional_canister_creation_post + ic_top_up_canister_pre ic_top_up_canister_post + callback_invocation_not_deleted_pre callback_invocation_not_deleted_post + callback_invocation_deleted_pre callback_invocation_deleted_post + respond_to_user_request_pre respond_to_user_request_post + request_cleanup_pre request_cleanup_post + request_cleanup_expired_pre request_cleanup_expired_post + canister_out_of_cycles_pre canister_out_of_cycles_post + canister_time_progress_pre canister_time_progress_post + cycle_consumption_pre cycle_consumption_post + system_time_progress_pre system_time_progress_post +in Haskell module_name IC file_prefix code + +end diff --git a/theories/ROOT b/theories/ROOT new file mode 100644 index 000000000..7db8ae08d --- /dev/null +++ b/theories/ROOT @@ -0,0 +1,8 @@ +chapter IC + +session Internet_Computer (IC) = "HOL-Library" + + options [timeout=600] + theories + IC +export_files (in ".") [2] + "Internet_Computer.IC:code/**" From 3c63f4ccbb6fb8dfc30ae34fea255249f61fda63 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Tue, 27 Sep 2022 11:05:15 +0200 Subject: [PATCH 036/102] Cut release 0.18.7 --- spec/changelog.adoc | 9 +++++++++ spec/ic.did | 18 ++++++++++++++++++ spec/index.adoc | 32 ++++++++++++++++++++++---------- 3 files changed, 49 insertions(+), 10 deletions(-) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 787c2af92..384e48fad 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,15 @@ [#changelog] == Changelog +[#unreleased] +=== ∞ (unreleased) +* Spec: User delegations include a principal scope + +[#0_18_7] +=== 0.18.7 (2022-09-27) +* HTTP request API +* Reserved principals + [#0_18_6] === 0.18.6 (2022-08-09) * Canister access to performance metrics diff --git a/spec/ic.did b/spec/ic.did index 8a4f91dc0..e0efb2d97 100644 --- a/spec/ic.did +++ b/spec/ic.did @@ -16,6 +16,14 @@ type definite_canister_settings = record { freezing_threshold : nat; }; +type http_header = record { name: text; value: text }; + +type http_response = record { + status: nat; + headers: vec http_header; + body: blob; +}; + type ecdsa_curve = variant { secp256k1; }; type satoshi = nat64; @@ -101,6 +109,16 @@ service ic : { delete_canister : (record {canister_id : canister_id}) -> (); deposit_cycles : (record {canister_id : canister_id}) -> (); raw_rand : () -> (blob); + http_request : (record { + url : text; + max_response_bytes: opt nat64; + method : variant { get; head; post }; + headers: vec http_header; + body : opt blob; + transform : opt variant { + function: func (http_response) -> (http_response) query + }; + }) -> (http_response); // Threshold ECDSA signature ecdsa_public_key : (record { diff --git a/spec/index.adoc b/spec/index.adoc index 55ee440f2..0084024dd 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,6 +1,6 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.6 +0.18.7 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ @@ -139,8 +139,13 @@ NOTE: Derived IDs are currently not explicitly used in this document, but they m + This has the form `0x04`, and is used for the anonymous caller. It can be used in call and query requests without a signature. -When the IC creates a _fresh_ id, it never creates a self-authenticating id, an anonymous id or an id derived from what could be a canister or user. +5. _Reserved ids_ ++ +These have the form of `blob · 0x7f`, `0 ≤ |blob| < 29`. ++ +These ids can be useful for applications that want to re-use the <> but want to indicate explicitly that the blob does not address any canisters or a user. +When the IC creates a _fresh_ id, it never creates a self-authenticating id, reserved id, an anonymous id or an id derived from what could be a canister or user. [#textual-ids] ==== Textual representation of principals @@ -681,7 +686,7 @@ The following encodings of field values as blobs are used [#request-id] === Request ids -When signing requests or querying the status of a request (see <>) in the state tree, the user identifies the request using a _request id_, which is the <> of the `content` map of the original request. +When signing requests or querying the status of a request (see <>) in the state tree, the user identifies the request using a _request id_, which is the <> of the `content` map of the original request. A request id must have length of 32 bytes. NOTE: The request id is independent of the representation of the request (currently only CBOR), and does not change if the specification adds further optional fields to a request type. @@ -1635,6 +1640,8 @@ Otherwise it will be rejected. [#ic-http_request] === IC method `http_request` +NOTE: The IC http_request API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. + This method makes an HTTP request to a given URL and returns the HTTP response, possibly after a transformation. The canister should aim to issue _idempotent_ requests, meaning that it must not change the state at the remote server, or the remote server has the means to identify duplicated requests. Otherwise, the risk of failure increases. @@ -1643,18 +1650,22 @@ The responses for all identical requests must match too. However, a web service For this reason, the calling canister can supply a transformation function, which the IC uses to let the canister sanitize the responses from such unique values. The transformation function is executed separately on the corresponding response received for a request. The final response will only be available to the calling canister. -Currently, only the `GET`, `HEAD`, and `POST` methods are supported for HTTP requests. Note that when using `POST`, the calling canister must make sure that the remote server is able to handle idempotent requests sent from multiple sources. This may require, for example, to set a certain request header to uniquely identify the request. +Currently, the `GET`, `HEAD`, and `POST` methods are supported for HTTP requests. + +It is important to note the following for the usage of the `POST` method: +- The calling canister must make sure that the remote server is able to handle idempotent requests sent from multiple sources. This may require, for example, to set a certain request header to uniquely identify the request. +- There are no confidentiality guarantees on the request content. There is no guarantee that all sent requests are as specified by the canister. If the canister receives a response, then at least one request that was sent matched the canister's request, and the response was to that request. For security reasons, only HTTPS connections are allowed (URLs must start with `https://`). The IC uses industry-standard root CA lists to validate certificates of remote web servers. Each request can specify the maximal expected size for the response from the remote HTTP server. The upper limit on the size of a response defaults to `2MiB` if no maximal size value is specified. -An error will be returned when the response is larger than the maximal size. +An error will be returned when the response is larger than the maximal size. The `2MiB` size limit also applies to the value returned by the `transform` function. The following parameters should be supplied for the call: -- `url` - the requested URL -- `max_response_bytes` - optional, specifies the maximal size of the response in bytes. The call will be charged based on this parameter. If not provided, the maximum of `2MiB` will be used. +- `url` - the requested URL. The URL may specify a custom port number. However, only ports 80, 443, and 20000-65535 can be used. +- `max_response_bytes` - optional, specifies the maximal size of the response in bytes. Any value less than or equal to `2MiB` is accepted. The call will be charged based on this parameter. If not provided, the maximum of `2MiB` will be used. - `method` - currently, only GET, HEAD, and POST are supported - `headers` - list of HTTP request headers and their corresponding values - `transform` - an optional function that transforms raw responses to sanitized responses. If provided, the calling canister itself must export this function. @@ -1794,7 +1805,7 @@ The transaction fees in the Bitcoin network change dynamically based on the numb pending transactions. It must be possible for a canister to determine an adequate fee when creating a Bitcoin transaction. -This function returns the 100 fee percentiles, measured in millisatoshi/byte (10^3 millisatoshi = 1 satoshi), over the last 10,000 transactions in the specified network, i.e., +This function returns the 100 fee percentiles, measured in millisatoshi/vbyte (1000 millisatoshi = 1 satoshi), over the last 10,000 transactions in the specified network, i.e., over the transactions in the last approximately 4-10 blocks. [#certification] @@ -1904,6 +1915,7 @@ find_label(l, _ · Labeled l1 t · _) | l == l1 = Found t find_label(l, _ · Labeled l1 _ · Labeled l2 _ · _) | l1 < l < l2 = Absent find_label(l, Labeled l2 _ · _) | l < l2 = Absent find_label(l, _ · Labeled l1 _ ) | l1 < l = Absent +find_label(l, [Leaf _]) = Absent find_label(l, []) = Absent find_label(l, _) = Unknown .... @@ -1916,7 +1928,7 @@ well_formed(tree) = well_formed_forest(trees) = strictly_increasing([l | Label l _ ∈ trees]) ∧ ∀ Label _ t ∈ trees. well_formed(t) ∧ - ∀ t ∈ trees ≠ Leaf _ + ∀ t ∈ trees. t ≠ Leaf _ .... [#certification-delegation] @@ -2482,7 +2494,7 @@ RequestStatus A `Path` may refer to a request by way of a _request id_, as specified in <>: .... -RequestId = Blob +RequestId = { b ∈ Blob | |b| = 32 } hash_of_map: Request -> RequestId .... From 2277fc9270a1e5562ba6541983e5da65a304a2c5 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Tue, 27 Sep 2022 11:25:26 +0200 Subject: [PATCH 037/102] Publish release 0.18.7 --- spec/changelog.adoc | 5 +++++ spec/ic.did | 18 ++++++++++++++++++ spec/index.adoc | 32 ++++++++++++++++++++++---------- 3 files changed, 45 insertions(+), 10 deletions(-) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 787c2af92..0f3c28535 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,11 @@ [#changelog] == Changelog +[#0_18_7] +=== 0.18.7 (2022-09-27) +* HTTP request API +* Reserved principals + [#0_18_6] === 0.18.6 (2022-08-09) * Canister access to performance metrics diff --git a/spec/ic.did b/spec/ic.did index 8a4f91dc0..e0efb2d97 100644 --- a/spec/ic.did +++ b/spec/ic.did @@ -16,6 +16,14 @@ type definite_canister_settings = record { freezing_threshold : nat; }; +type http_header = record { name: text; value: text }; + +type http_response = record { + status: nat; + headers: vec http_header; + body: blob; +}; + type ecdsa_curve = variant { secp256k1; }; type satoshi = nat64; @@ -101,6 +109,16 @@ service ic : { delete_canister : (record {canister_id : canister_id}) -> (); deposit_cycles : (record {canister_id : canister_id}) -> (); raw_rand : () -> (blob); + http_request : (record { + url : text; + max_response_bytes: opt nat64; + method : variant { get; head; post }; + headers: vec http_header; + body : opt blob; + transform : opt variant { + function: func (http_response) -> (http_response) query + }; + }) -> (http_response); // Threshold ECDSA signature ecdsa_public_key : (record { diff --git a/spec/index.adoc b/spec/index.adoc index 55ee440f2..0084024dd 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,6 +1,6 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.6 +0.18.7 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ @@ -139,8 +139,13 @@ NOTE: Derived IDs are currently not explicitly used in this document, but they m + This has the form `0x04`, and is used for the anonymous caller. It can be used in call and query requests without a signature. -When the IC creates a _fresh_ id, it never creates a self-authenticating id, an anonymous id or an id derived from what could be a canister or user. +5. _Reserved ids_ ++ +These have the form of `blob · 0x7f`, `0 ≤ |blob| < 29`. ++ +These ids can be useful for applications that want to re-use the <> but want to indicate explicitly that the blob does not address any canisters or a user. +When the IC creates a _fresh_ id, it never creates a self-authenticating id, reserved id, an anonymous id or an id derived from what could be a canister or user. [#textual-ids] ==== Textual representation of principals @@ -681,7 +686,7 @@ The following encodings of field values as blobs are used [#request-id] === Request ids -When signing requests or querying the status of a request (see <>) in the state tree, the user identifies the request using a _request id_, which is the <> of the `content` map of the original request. +When signing requests or querying the status of a request (see <>) in the state tree, the user identifies the request using a _request id_, which is the <> of the `content` map of the original request. A request id must have length of 32 bytes. NOTE: The request id is independent of the representation of the request (currently only CBOR), and does not change if the specification adds further optional fields to a request type. @@ -1635,6 +1640,8 @@ Otherwise it will be rejected. [#ic-http_request] === IC method `http_request` +NOTE: The IC http_request API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. + This method makes an HTTP request to a given URL and returns the HTTP response, possibly after a transformation. The canister should aim to issue _idempotent_ requests, meaning that it must not change the state at the remote server, or the remote server has the means to identify duplicated requests. Otherwise, the risk of failure increases. @@ -1643,18 +1650,22 @@ The responses for all identical requests must match too. However, a web service For this reason, the calling canister can supply a transformation function, which the IC uses to let the canister sanitize the responses from such unique values. The transformation function is executed separately on the corresponding response received for a request. The final response will only be available to the calling canister. -Currently, only the `GET`, `HEAD`, and `POST` methods are supported for HTTP requests. Note that when using `POST`, the calling canister must make sure that the remote server is able to handle idempotent requests sent from multiple sources. This may require, for example, to set a certain request header to uniquely identify the request. +Currently, the `GET`, `HEAD`, and `POST` methods are supported for HTTP requests. + +It is important to note the following for the usage of the `POST` method: +- The calling canister must make sure that the remote server is able to handle idempotent requests sent from multiple sources. This may require, for example, to set a certain request header to uniquely identify the request. +- There are no confidentiality guarantees on the request content. There is no guarantee that all sent requests are as specified by the canister. If the canister receives a response, then at least one request that was sent matched the canister's request, and the response was to that request. For security reasons, only HTTPS connections are allowed (URLs must start with `https://`). The IC uses industry-standard root CA lists to validate certificates of remote web servers. Each request can specify the maximal expected size for the response from the remote HTTP server. The upper limit on the size of a response defaults to `2MiB` if no maximal size value is specified. -An error will be returned when the response is larger than the maximal size. +An error will be returned when the response is larger than the maximal size. The `2MiB` size limit also applies to the value returned by the `transform` function. The following parameters should be supplied for the call: -- `url` - the requested URL -- `max_response_bytes` - optional, specifies the maximal size of the response in bytes. The call will be charged based on this parameter. If not provided, the maximum of `2MiB` will be used. +- `url` - the requested URL. The URL may specify a custom port number. However, only ports 80, 443, and 20000-65535 can be used. +- `max_response_bytes` - optional, specifies the maximal size of the response in bytes. Any value less than or equal to `2MiB` is accepted. The call will be charged based on this parameter. If not provided, the maximum of `2MiB` will be used. - `method` - currently, only GET, HEAD, and POST are supported - `headers` - list of HTTP request headers and their corresponding values - `transform` - an optional function that transforms raw responses to sanitized responses. If provided, the calling canister itself must export this function. @@ -1794,7 +1805,7 @@ The transaction fees in the Bitcoin network change dynamically based on the numb pending transactions. It must be possible for a canister to determine an adequate fee when creating a Bitcoin transaction. -This function returns the 100 fee percentiles, measured in millisatoshi/byte (10^3 millisatoshi = 1 satoshi), over the last 10,000 transactions in the specified network, i.e., +This function returns the 100 fee percentiles, measured in millisatoshi/vbyte (1000 millisatoshi = 1 satoshi), over the last 10,000 transactions in the specified network, i.e., over the transactions in the last approximately 4-10 blocks. [#certification] @@ -1904,6 +1915,7 @@ find_label(l, _ · Labeled l1 t · _) | l == l1 = Found t find_label(l, _ · Labeled l1 _ · Labeled l2 _ · _) | l1 < l < l2 = Absent find_label(l, Labeled l2 _ · _) | l < l2 = Absent find_label(l, _ · Labeled l1 _ ) | l1 < l = Absent +find_label(l, [Leaf _]) = Absent find_label(l, []) = Absent find_label(l, _) = Unknown .... @@ -1916,7 +1928,7 @@ well_formed(tree) = well_formed_forest(trees) = strictly_increasing([l | Label l _ ∈ trees]) ∧ ∀ Label _ t ∈ trees. well_formed(t) ∧ - ∀ t ∈ trees ≠ Leaf _ + ∀ t ∈ trees. t ≠ Leaf _ .... [#certification-delegation] @@ -2482,7 +2494,7 @@ RequestStatus A `Path` may refer to a request by way of a _request id_, as specified in <>: .... -RequestId = Blob +RequestId = { b ∈ Blob | |b| = 32 } hash_of_map: Request -> RequestId .... From 241c1ddbd88bf0c0bcf357c42ccb7d8a89d2c4d0 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Tue, 27 Sep 2022 11:27:02 +0200 Subject: [PATCH 038/102] Clean up changelog --- spec/changelog.adoc | 4 ---- 1 file changed, 4 deletions(-) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 384e48fad..0f3c28535 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,10 +1,6 @@ [#changelog] == Changelog -[#unreleased] -=== ∞ (unreleased) -* Spec: User delegations include a principal scope - [#0_18_7] === 0.18.7 (2022-09-27) * HTTP request API From 2f9b0b01c9406cd0d89c7ccf7d99519f5f4c264c Mon Sep 17 00:00:00 2001 From: ais <97464093+aisconnolly@users.noreply.github.com> Date: Wed, 28 Sep 2022 10:29:20 +0000 Subject: [PATCH 039/102] adding md conversion --- spec/md-spec/ic-interface-spec.md | 4624 ++++++++++++++++++++++ spec/md-spec/interface-spec-changelog.md | 258 ++ 2 files changed, 4882 insertions(+) create mode 100644 spec/md-spec/ic-interface-spec.md create mode 100644 spec/md-spec/interface-spec-changelog.md diff --git a/spec/md-spec/ic-interface-spec.md b/spec/md-spec/ic-interface-spec.md new file mode 100644 index 000000000..a01431743 --- /dev/null +++ b/spec/md-spec/ic-interface-spec.md @@ -0,0 +1,4624 @@ +import Changelog from './_attachments/interface-spec-changelog.md'; + +# The Internet Computer Interface Specification {#the_internet_computer_interface_specification} + +## Introduction {#introduction} + +Welcome to *the Internet Computer*! We speak of "the" Internet Computer, because although under the hood a large number of physical computers are working together in a blockchain protocol, in the end we have the appearance of a single, shared, secure and world-wide accessible computer. Developers who want to build decentralized applications (or *dapps* for short) that run on the Internet Computer blockchain and end-users who want to use those dapps need to know very little, if anything, about the underlying protocol. However, knowing some details about the interfaces that the Internet Computer exposes can allow interested developers and architects to take fuller advantages of the unique features that the Internet Computer provides. + +### Target audience {#target_audience} + +This document describes this *external* view of the Internet Computer, i.e. the low-level interfaces it provides to dapp developers and users, and what will happen when they use these interfaces. + +:::note +While this document describes the external interface and behavior of the Internet Computer, it is not intended as end-user or end-developer documentation. Most developers will interact with the Internet Computer through additional tooling like the SDK, Canister Development Kits and Motoko. Please see the [developer docs](../developer-docs/ic-overview.mdx) for suitable documentation. + +::: + +The target audience of this document are + +- those who use these low-level interfaces (e.g. implement agents, canister developments kits, emulators, other tooling). + +- those who implement these low-level interfaces (e.g. developers of the Internet Computer implementation) + +- those who want to understand the intricacies of the Internet Computer's behavior in great detail (e.g. to do a security analysis) + +:::note +This document is a rigorous, technically dense reference. It is not an introduction to the Internet Computer, and as such most useful to those who understand the high-level concepts. Please see more high-level documentation first. +::: + +### Scope of this document {#scope_of_this_document} + +If you think of the Internet Computer as a distributed engine that executes WebAssembly-based dapps, then this document describes exclusively the aspect of executing those dapps. To the extent possible, this document will *not* talk about consensus protocols, nodes, subnets, orthogonal persistence or governance. + +This document tries to be implementation agnostic: It would apply just as well to a (hypothetical) compatible reimplementation of the Internet Computer. This implies that this document does not cover interfaces towards those running the Internet Computer (e.g. data center operators, protocol developers, governance users), as topics like node update, monitoring, logging are inherently tied to the actual *implementation* and its architecture. + +### Overview of the Internet Computer {#overview_of_the_internet_computer} + +Dapps on the Internet Computer, or *IC* for short, are implemented as *canister smart contracts*, or *canisters* for short. If you want to build on the Internet Computer as a dapp developer, you first create a *canister module* that contains the WebAssembly code and configuration for your dapp, and deploy it using the [HTTPS interface](#http-interface). You can create canister modules using the Motoko language and the SDK, which is more convenient. If you want to use your own tooling, however, then this document describes [what a canister module looks like](#canister-module-format) and how the [WebAssembly code can interact with the IC](#system-api). + +Once your dapp is running on the Internet Computer, it is a canister smart contract, and users can interact with it. They can use the [HTTPS interface](#http-interface) to interact with the canister according to the [System API](#system-api). + +The user can also use the HTTPS interface to issue read-only queries, which are faster, but cannot change the state of a canister. + +```plantuml + actor Developer + actor User + participant "Internet Computer" as IC + participant "Canister 1" as Can1 + Developer -> IC : /submit create canister + create Can1 + IC -> Can1 : create + Developer <-- IC : canister-id=1 + Developer -> IC : /submit install module + IC -> Can1 : initialize + ||| + User -> IC : /submit call “hello” + IC -> Can1 : hello + return "Hello world!" + User <-- IC : "Hello World!" +``` +**A typical use of the Internet Computer. (This is a simplified view; some of the arrows represent multiple interaction steps or polling.)** + +Sections “[HTTPS Interface](#https-interface)” and “[Canister interface (System API)](#system-api)” describe these interfaces, together with a brief description of what they do. Afterwards, you will find a [more formal description](#abstract-behavior) of the Internet Computer that describes its abstract behavior with more rigor. + +### Nomenclature {#nomenclature} + +To get some consistency in this document, we try to use the following terms with precision: + +We avoid the term "client", as it could be the client of the Internet Computer or the client inside the distributed network that makes up the Internet Computer. Instead, we use the term *user* to denote the external entity interacting with the Internet Computer, even if in most cases it will be some code (sometimes called "agent") acting on behalf of a (human) user. + +The public entry points of canisters are called *methods*. Methods can be declared to be either *update methods* (state mutation is preserved) or *query methods* (state mutation is discarded, no further calls can be made). + +Methods can be *called*, from *caller* to *callee*, and will eventually incur a *response* which is either a *reply* or a *reject*. A method may have *parameters*, which are provided with concrete *arguments* in a method call. + +External calls can be update calls, which can call both kinds of methods, and query calls, which can *only* call query methods. Inter-canister calls can also call both kinds of methods. Note that calls from a canister to itself also count as \"inter-canister\". + +Internally, a call or a response is transmitted as a *message* from a *sender* to a *receiver*. Messages do not have a response. + +WebAssembly *functions* are exported by the WebAssembly module or provided by the System API. These are *invoked* and can either *trap* or *return*, possibly with a return value. Functions, too, have parameters and take arguments. + +External *users* interact with the Internet Computer by issuing *requests* on the HTTPS interface. Requests have responses which can either be replies or rejects. Some requests cause internal messages to be created. + +Canisters and users are identified by a *principal*, sometimes also called an *id*. + +## Pervasive concepts {#pervasive_concepts} + +Before going into the details of the four public interfaces described in this document (namely the agent-facing [HTTPS interface](#https-interface), the canister-facing [System API](#system-api), the [virtual Management canister](#the-ic-management-canister) and the [System State Tree](#the-system-state-tree)), this section introduces some concepts that transcend multiple interfaces. + +### Unspecified constants and limits {#unspecified_constants_and_limits} + +This specification may refer to certain constants and limits without specifying their concrete value (yet), i.e. they are implementation defined. Many are resource limits which are relevant only to specify the error-handling behavior of the IC (which, as mentioned above, is also not yet precisely described in this document). This list is not complete. + +- `MAX_CYCLES_PER_MESSAGE` + + Amount of cycles that a canister has to have before a message is attempted to be executed, which is deducted from the canister balance before message execution. See [Message execution](#rule-message-execution). + +- `MAX_CYCLES_PER_RESPONSE` + + Amount of cycles that the IC sets aside when a canister performs a call. This is used to pay for processing the response message, and unused cycles after the execution of the response are refunded. See [Message execution](#rule-message-execution). + +- `DEFAULT_PROVISIONAL_CYCLES_BALANCE` + + Amount of cycles allocated to a new canister by default, if not explicitly specified. See [IC method ](#icmethod-provisional_create_canister_with_cycles). + +### Principals {#principal} + +Principals are generic identifiers for canisters, users and possibly other concepts in the future. As far as most uses of the IC are concerned they are *opaque* binary blobs with a length between 0 and 29 bytes, and there is intentionally no mechanism to tell canister ids and user ids apart. + +There is, however, some structure to them to encode specific authentication and authorization behavior. + +#### Special forms of Principals {#id-classes} + +In this section, `H` denotes SHA-224, `·` denotes blob concatenation and `|p|` denotes the length of `p` in bytes, encoded as a single byte. + +There are several classes of ids: + +1. *Opaque ids*. + + These are always generated by the IC and have no structure of interest outside of it. + + :::note + Typically, these end with the byte `0x01`, but users of the IC should not need to care about that. + ::: + +2. *Self-authenticating ids*. + + These have the form `H(public_key) · 0x02` (29 bytes). + + An external user can use these ids as the `sender` of a request if they own the corresponding private key. The public key uses one of the encodings described in [Signatures](#signatures). + +3. *Derived ids* + + These have the form `H(|registering_principal| · registering_principal · derivation_nonce) · 0x03` (29 bytes). + + These ids are treated specially when an id needs to be registered. In such a request, whoever requests an id can provide a `derivation_nonce`. By hashing that together with the principal of the caller, every principal has a space of ids that only they can register ids from. + + :::note + Derived IDs are currently not explicitly used in this document, but they may be used internally or in the future. + ::: + +4. *Anonymous id* + + This has the form `0x04`, and is used for the anonymous caller. It can be used in call and query requests without a signature. + +5. *Reserved ids* + These have the form of `blob · 0x7f`, `0 ≤ |blob| < 29`. + + These ids can be useful for applications that want to re-use the [Textual representation of principals](#textual-ids) but want to indicate explicitly that the blob does not address any canisters or a user. + +When the IC creates a *fresh* id, it never creates a self-authenticating id, reserved id, an anonymous id or an id derived from what could be a canister or user. + +#### Textual representation of principals {#textual-ids} + +We specify a *canonical textual format* that is recommended whenever principals need to be printed or read in textual format, e.g. in log messages, transactions browser, command line tools, source code. + +The textual representation of a blob `b` is `Grouped(Base32(CRC32(b) · b))` where + +- `CRC32` is a four byte check sequence, calculated as defined by ISO 3309, ITU-T V.42, and [elsewhere](https://www.w3.org/TR/2003/REC-PNG-20031110/#5CRC-algorithm), and stored as big-endian, i.e., the most significant byte comes first and then the less significant bytes come in descending order of significance (MSB B2 B1 LSB). + +- `Base32` is the Base32 encoding as defined in [RFC 4648](https://tools.ietf.org/html/rfc4648#section-6), with no padding character added. + +- The middle dot denotes concatenation. + +- `Grouped` takes an ASCII string and inserts the separator `-` (dash) every 5 characters. The last group may contain less than 5 characters. A separator never appears at the beginning or end. + +The textual representation is conventionally printed with *lower case letters*, but parsed case-insensitively. + +Because the maximum size of a principal is 29 bytes, the textual representation will be no longer than 63 characters (10 times 5 plus 3 characters with 10 separators in between them). + +:::tip +The canister with id `0xABCD01` has check sequence `0x233FF206` ([online calculator](https://crccalc.com/?crc=ABCD01&method=crc32&datatype=hex&outtype=hex)); the final id is thus `em77e-bvlzu-aq`. + +Example encoding from hex, and decoding to hex, in bash (the following can be pasted into a terminal as is): + + function textual_encode() { + ( echo "$1" | xxd -r -p | /usr/bin/crc32 /dev/stdin; echo -n "$1" ) | + xxd -r -p | base32 | tr A-Z a-z | + tr -d = | fold -w5 | paste -sd'-' - + } + + function textual_decode() { + echo -n "$1" | tr -d - | tr a-z A-Z | + fold -w 8 | xargs -n1 printf '%-8s' | tr ' ' = | + base32 -d | xxd -p | tr -d '\n' | cut -b9- | tr a-z A-Z + } +::: + +### Canister lifecycle + +Dapps on the Internet Computer are called *canisters*. Conceptually, they consist of the following pieces of state: + +- A canister id (a [principal](#principal)) + +- Their *controllers* (a possibly empty list of [principal](#principal)) + +- A cycle balance + +- The *canister status*, which is one of `running`, `stopping` or `stopped`. + +- Resource reservations + +A canister can be *empty* (e.g. directly after creation) or *non-empty*. A non-empty canister also has + +- code, in the form of a canister module + +- state (memories, globals etc.) + +- possibly further data that is specific to the implementation of the IC (e.g. queues) + +Canisters are empty after creation and uninstallation, and become non-empty through [code installation](#ic-install_code). + +If an empty canister receives a response, that response is dropped, as if the canister trapped when processing the response. The cycles set aside for its processing and the cycles carried on the responses are added to the canister's *cycles* balance. + +#### Canister cycles + +The IC relies on *cycles*, a utility token, to manage its resources. A canister pays for the resources it uses from its *cycle balance*. The *cycle_balance* is stored as 128-bit unsigned integers and operations on them are saturating. In particular, if *cycles* are added to a canister that would bring its total balance beyond 2^128-1, then the balance will be capped at 2^128-1 and any additional cycles will be lost. + +When the cycle balance of a canister falls to zero, the canister is *deallocated*. This has the same effect as + +- uninstalling the canister (as described in [IC method ](#ic-uninstall_code)) + +- setting all resource reservations to zero + +Afterwards the canister is empty. It can be reinstalled after topping up its balance. + +:::note +Once the IC frees the resources of a canister, its id, *cycles* balance, and *controllers* are preserved on the IC for a minimum of 10 years. What happens to the canister after this period is currently unspecified. +::: + +#### Canister status + +The canister status can be used to control whether the canister is processing calls: + +- In status `running`, calls to the canister are processed as normal. + +- In status `stopping`, calls to the canister are rejected by the IC, but responses to the canister are processed as normal. + +- In status `stopped`, calls to the canister are rejected by the IC, and there are no outstanding responses to call contexts that are not marked as deleted. + +In all cases, calls to the [management canister](#the-ic-management-canister) are processed, regardless of the state of the managed canister. + +The controllers of the canister can initiate transitions between these states using [`stop_canister`](#ic-stop_canister) and [`start_canister`](#ic-start_canister), and query the state using [`canister_status`](#ic-canister_status). The canister itself can also query its state using [`ic0.canister_status`](#system-api-canister-status). + +:::note +This status is orthogonal to the question of whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected, but because the canister is empty. +::: + +### Signatures + +Digital signature schemes are used for authenticating messages in various parts of the IC infrastructure. Signatures are domain separated, which means that every message is prefixed with a byte string that is unique to the purpose of the signature. + +The IC supports multiple signature schemes, with details given in the following subsections. For each scheme, we specify the data encoded in the public key (which is always DER-encoded, and indicates the scheme to use) as well as the form of the signatures (which are opaque blobs for the purposes of the rest of this specification). + +In all cases, the signed *payload* is the concatenation of the domain separator and the message. All uses of signatures in this specification indicate a domain separator, to uniquely identify the purpose of the signature. The domain separators are prefix-free by construction, as their first byte indicates their length. + +#### Ed25519 and ECDSA signatures {#ecdsa} + +Plain signatures are supported for the schemes + +- [**Ed25519**](https://ed25519.cr.yp.to/index.html) or + +- [**ECDSA**](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf) on curve P-256 (also known as `secp256r1`), using SHA-256 as hash function, as well as on the Koblitz curve `secp256k1`. + +- Public keys must be valid for signature schemes Ed25519 or ECDSA and are encoded as DER. + + - See [RFC 8410](https://tools.ietf.org/html/rfc8410) for DER encoding of Ed25519 public keys. + + - See [RFC 5480](https://tools.ietf.org/rfc/rfc5480) for DER encoding of ECDSA public keys; the DER encoding must not specify a hash function. For curve `secp256k1`, the OID 1.3.132.0.10 is used. The points must be specified in uncompressed form (i.e. `0x04` followed by the big-endian 32-byte encodings of `x` and `y`). + +- The signatures are encoded as the concatenation of the 32-byte big endian encodings of the two values *r* and *s*. + +#### Web Authentication {#webauthn} + +The allowed signature schemes for web authentication are + +- [**ECDSA**](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf) on curve P-256 (also known as `secp256r1`), using SHA-256 as hash function. + +- [**RSA PKCS#1v1.5 (RSASSA-PKCS1-v1_5)**](https://datatracker.ietf.org/doc/html/rfc8017#section-8.2), using SHA-256 as hash function. + +The signature is calculated by using the payload as the challenge in the web authentication assertion. + +The signature is checked by verifying that the `challenge` field contains the [base64url encoding](https://tools.ietf.org/html/rfc4648#section-5) of the payload, and that `signature` verifies on `authenticatorData · SHA-256(utf8(clientDataJSON))`, as specified in the [WebAuthn w3c recommendation](https://www.w3.org/TR/webauthn/#op-get-assertion). + +- The public key is encoded as a DER-wrapped COSE key. + + It uses the `SubjectPublicKeyInfo` type used for other types of public keys (see, e.g., [RFC 8410, Section 4](https://tools.ietf.org/html/rfc8410#section-4)), with OID 1.3.6.1.4.1.56387.1.1 (iso.org.dod.internet.private.enterprise.dfinity.mechanisms.der-wrapped-cose). The `BIT STRING` field `subjectPublicKey` contains the COSE encoding. See [WebAuthn w3c recommendation](https://www.w3.org/TR/webauthn/#sctn-encoded-credPubKey-examples) or [RFC 8152](https://tools.ietf.org/html/rfc8152#section-13.1) for details on the COSE encoding. + + :::tip + A DER wrapping of a COSE key is shown below. It can be parsed via the command `sed "s/#.*//" | xxd -r -p | openssl asn1parse -inform der`. + + 30 5E # SEQUENCE of length 94 bytes + 30 0C # SEQUENCE of length 12 bytes + 06 0A 2B 06 01 04 01 83 B8 43 01 01 # OID 1.3.6.1.4.1.56387.1.1 + 03 4E 00 # BIT STRING encoding of length 78, + A501 0203 2620 0121 5820 7FFD 8363 2072 # length is at byte boundary + FD1B FEAF 3FBA A431 46E0 EF95 C3F5 5E39 # contents is a valid COSE key + 94A4 1BBF 2B51 74D7 71DA 2258 2032 497E # with ECDSA on curve P-256 + ED0A 7F6F 0009 2876 5B83 1816 2CFD 80A9 + 4E52 5A6A 368C 2363 063D 04E6 ED + + You can also view the wrapping in [an online ASN.1 JavaScript decoder](https://lapo.it/asn1js/#MF4wDAYKKwYBBAGDuEMBAQNOAKUBAgMmIAEhWCB__YNjIHL9G_6vP7qkMUbg75XD9V45lKQbvytRdNdx2iJYIDJJfu0Kf28ACSh2W4MYFiz9gKlOUlpqNowjYwY9BObt). + ::: + +- The signature is a CBOR value consisting of a data item with major type 6 ("Semantic tag") and tag value `55799` (see [Self-Describe CBOR](https://tools.ietf.org/html/rfc7049#section-2.4.5)), followed by a map with three mandatory fields: + + - `authenticator_data` (`blob`): WebAuthn authenticator data. + + - `client_data_json` (`text`): WebAuthn client data in JSON representation. + + - `signature` (`blob`): Signature as specified in the [WebAuthn w3c recommendation](https://www.w3.org/TR/webauthn/#signature-attestation-types), which means DER encoding in the case of an ECDSA signature. + +#### Canister signatures + +The IC also supports a scheme where a canister can sign a payload by declaring a special "certified variable". + +This section makes forward references to other concepts in this document, in particular the section [Certification](#certification). + +- The public key is a DER-wrapped structure that indicates the *signing canister*, and includes a freely choosable seed. Each choice of seed yields a distinct public key for the canister, and the canister can choose to encode information, such as a user id, in the seed. + + More concretely, it uses the `SubjectPublicKeyInfo` type used for other types of public keys (see, e.g., [RFC 8410, Section 4](https://tools.ietf.org/html/rfc8410#section-4)), with OID 1.3.6.1.4.1.56387.1.2 (iso.org.dod.internet.private.enterprise.dfinity.mechanisms.canister-signature). + + The `BIT STRING` field `subjectPublicKey` is the blob `|signing_canister_id| · signing_canister_id · seed`, where `|signing_canister_id|` is the one-byte encoding of the the length of the `signing_canister_id` and `·` denotes blob concatenation. + +- The signature is a CBOR value consisting of a data item with major type 6 ("Semantic tag") and tag value `55799` (see [Self-Describe CBOR](https://tools.ietf.org/html/rfc7049#section-2.4.5)), followed by a map with two mandatory fields: + + - `certificate` (`blob`): A CBOR-encoded certificate as per [Encoding of certificates](#certification-encoding). + + - `tree` (`hash-tree`): A hash tree as per [Encoding of certificates](#certification-encoding). + +- Given a payload together with public key and signature in the format described above the signature can be verified by checking the following two conditions: + + - The `certificate` must be a valid certificate as described in [Certification](#certification), with + + lookup(/canister//certified_data, certificate.tree) = Found (reconstruct(tree)) + + where `signing_canister_id` is the id of the signing canister and `reconstruct` is a function that computes a root-hash for the tree. + + - If the `certificate` includes subnet delegations (possibly nested), then the `signing_canister_id` must be included in each delegation's canister id range (see [Delegation](#certification-delegation)). + + - The `tree` must be a `well_formed` tree with + + lookup(/sig//, tree) = Found "" + + where `s` is the SHA-256 hash of the `seed` used in the public key and `m` is the SHA-256 hash of the payload. + +## The system state tree {#state-tree} + +Parts of the IC state are publicly exposed (e.g. via [Request: Read state](#http-read-state) or [Certified data](#system-api-certified-data)) in a verified way (see [Certification](#certification) for the machinery for certifying). This section describes the content of this system state abstractly. + +Conceptually, the system state is a tree with labeled children, and values in the leaves. Equivalently, the system state is a mapping from paths (sequences of labels) to values, where the domain is prefix-free. + +Labels are always blobs (but often with a human readable representation). In this document, paths are written suggestively with slashes as separators; the actual encoding is not actually using slashes as delimiters, and labels may contain the 0x2F byte (ASCII `/`) just fine. Values are either natural numbers, text values or blob values. + +This section specifies the publicly relevant paths in the tree. + +### Time {#state-tree-time} + +- `/time` (natural): + + All partial state trees include a timestamp, indicating the time at which the state is current. + +### Subnet information {#state-tree-subnet} + +The state tree contains information about the topology of the Internet Computer. + +- `/subnet//public_key` (blob) + + The public key of the subnet (a DER-encoded BLS key, see [Certification](#certification)) + +- `/subnet//canister_ranges` (blob) + + The set of canister ids assigned to this subnet, represented as a list of closed intervals of canister ids, ordered lexicographically, and encoded as CBOR according to this CDDL: + + canister_ranges = tagged<[*canister_range]> + canister_range = [principal principal] + principal = bytes .size (0..29) + tagged = #6.55799(t) ; the CBOR tag + + :::note + Because this uses the lexicographic ordering of princpials, and the byte distinguishing the various classes of ids is at the *end*, this range by construction conceptually includes principals of various classes. This specification needs to take care that the fact that principals that are not canisters may appear in these ranges does not cause confusion. + ::: + +### Request status {#state-tree-request-status} + +For each asynchronous request known to the Internet Computer, its status is in a subtree at `/request_status/`. Please see [Overview of canister calling](#http-call-overview) for more details on how asynchronous requests work. + +- `/request_status//status` (text) + + One of `received`, `processing`, `replied`, `rejected` or `done`, see [Overview of canister calling](#http-call-overview) for more details on what each status means. + +- `/request_status//reply` (blob) + + If the status is `replied`, then this path contains the reply blob, else it is not present. + +- `/request_status//reject_code` (natural) + + If the status is `rejected`, then this path contains the reject code (see [Reject codes](#reject-codes)), else it is not present. + +- `/request_status//reject_message` (text) + + If the status is `rejected`, then this path contains a textual diagnostic message, else it is not present. + +- `/request_status//error_code` (text) + + If the status is `rejected`, then this path might be present and contain an implementation-specific error code (see [Error codes](#error-codes)), else it is not present. + +:::note +Immediately after submitting a request, the request may not show up yet as the Internet Computer is still working on accepting the request as pending. +::: + +:::note +Request statuses will not actually be kept around indefinitely, and eventually the Internet Computer forgets about the request. This will happen no sooner than the request's expiry time, so that replay attacks are prevented. +::: + +### Certified data {#state-tree-certified-data} + +- `/canister//certified_data` (blob): + + The certified data of the canister with the given id, see [Certified data](#system-api-certified-data). + +### Canister information {#state-tree-canister-information} + +Users have the ability to learn about the hash of the canister's module, its current controllers, and metadata in a certified way. + +- `/canister//module_hash` (blob): + + If the canister is empty, this path does not exist. If the canister is not empty, it exists and contains the SHA256 hash of the currently installed canister module. Cf. [IC method ](#ic-canister_status). + +- `/canister//controllers` (blob): + + The current controllers of the canister. The value consists of a CBOR data item with major type 6 ("Semantic tag") and tag value `55799` (see [Self-Describe CBOR](https://tools.ietf.org/html/rfc7049#section-2.4.5)), followed by an array of principals in their binary form (CDDL `#6.55799([* bytes .size (0..29)])`). + +- `/canister//metadata/` (blob): + + If the canister has a [custom section](https://webassembly.github.io/spec/core/binary/modules.html#custom-section) called `icp:public ` or `icp:private `, this path contains the content of the custom section. Otherwise, this path does not exist. + + It is recommended for the canister to have a custom section called \"icp:public candid:service\", which contains the UTF-8 encoding of [the Candid interface](https://github.com/dfinity/candid/blob/master/spec/Candid.md#core-grammar) for the canister. + +## HTTPS Interface {#http-interface} + +The concrete mechanism that users use to send requests to the Internet Computer is via an HTTPS API, which exposes three endpoints to handle interactions, plus one for diagnostics: + +- At `/api/v2/canister//call` the user can submit (asynchronous, state-changing) calls. + +- At `/api/v2/canister//read_state` the user can read various information about the state of the Internet Computer. In particular, they can poll for the status of a call here. + +- At `/api/v2/canister//query` the user can perform (synchronous, non-state-changing) query calls. + +- At `/api/v2/status` the user can retrieve status information about the Internet Computer. + +In these paths, the `` is the [textual representation](#textual-ids) of the [*effective* canister id](#http-effective-canister-id). + +Requests to `/api/v2/canister//call`, `/api/v2/canister//read_state` and `/api/v2/canister//query` are POST requests with a CBOR-encoded request body, which consists of a authentication envelope (as per [Authentication](#authentication)) and request-specific content as described below. + +:::note +This document does not yet explain how to find the location and port of the Internet Computer. +::: + +### Overview of canister calling {#http-call-overview} + +Users interact with the Internet Computer by calling canisters. By the very nature of a blockchain protocol, they cannot be acted upon immediately, but only with a delay. Moreover, the actual node that the user talks to may not be honest or, for other reasons, may fail to get the request on the way. This implies the following high-level workflow: + +1. A user submits a call via the [HTTPS Interface](#http-interface). No useful information is returned in the immediate response (as such information cannot be trustworthy anyways). + +2. For a certain amount of time, the IC behaves as if it does not know about the call. + +3. The IC asks the targeted canister if it is willing to accept this message and be charged for the expense of processing it. This uses the [Ingress message inspection](#system-api-inspect-message) API for normal calls. For calls to the management canister, the rules in [The IC management canister](#ic-management-canister) apply. + +4. At some point, the IC may accept the call for processing and set its status to `received`. This indicates that the IC as a whole has received the call and plans on processing it (although it may still not get processed if the IC is under high load). Furthermore, the user should also be able to ask any endpoint about the status of the pending call. + +5. Once it is clear that the call will be acted upon (sufficient resources, call not yet expired), the status changes to `processing`. Now the user has the guarantee that the request will have an effect, e.g. it will reach the target canister. + +6. The IC is processing the call. For some calls this may be atomic, for others this involves multiple internal steps. + +7. Eventually, a response will be produced, and can be retrieved for a certain amount of time. The response is either a `reply`, indicating success, or a `reject`, indicating some form of error. + +8. In the case that the call has been retained for long enough, but the request has not expired yet, the IC can forget the response data and only remember the call as `done`, to prevent a replay attack. + +9. Once the expiry time is past, the IC can prune the call and its response, and completely forget about it. + +This yields the following interaction diagram: +```plantuml + (*) --> "User creates call" #DDDDDD + --> "Submitted to node\n(with 202 response)" as submit #DDDDDD + --> "received" + --> "processing" + if "" as X then + --> "replied" + --> "done" + else + --> "rejected (canister)" + --> "done" + + "X" --> "rejected (system)" + "received" --> "rejected (system)" + --> "done" + + "received" --> "pruned" #DDDDDD + "submit" --> "dropped" #DDDDDD + "done" --> "pruned" #DDDDDD + + endif +``` + +State transitions may be instantaneous and not always externally visible. For example, the state of a request may move from `received` via `processing` to `replied` in one go. Similarly, the IC may not implement the `done` state at all, and keep calls in state `replied`/`rejected` until they are pruned. + +All gray states are *not* explicitly represented in the state of the IC, and are indistinguishable from "call does not exist". + +The characteristic property of the `received` state is that the call has made it past the (potentially malicious) endpoint *into the state of the IC*. It is now pointless (but harmless) to submit the (identical) call again. Before reaching that state, submitting the identical call to further nodes might be a useful safeguard against a malicious or misbehaving node. + +The characteristic property of the `processing` state is that *the initial effect of the call has happened or will happen*. This is best explained by an example: Consider a counter canister. It exports a method `inc` that increases the counter. Assume that the canister is bug free, and is not going to be forcibly removed. A user submits a call to call `inc`. If the user sees request status `processing`, the state change is guaranteed to happen. The user can stop monitoring the status and does not have to retry submitting. + +A call may be rejected by the IC or the canister. In either case, there is no guarantee about how much processing of the call has happened. + +To avoid replay attacks, the transition from `done` or `received` to `pruned` must happen no earlier than the call's `ingress_expiry` field. + +Calls must stay in `replied` or `rejected` long enough for polling users to catch the response. + +When asking the IC about the state or call of a request, the user uses the request id (see [Request ids](#request-id)) to read the request status (see [Request status](#state-tree-request-status)) from the state tree (see [Request: Read state](#http-read-state)). + +### Request: Call {#http-call} + +In order to call a canister, the user makes a POST request to `/api/v2/canister//call`. The request body consists of an authentication envelope with a `content` map with the following fields: + +- `request_type` (`text`): Always `call` + +- `sender`, `nonce`, `ingress_expiry`: See [Authentication](#authentication) + +- `canister_id` (`blob`): The principal of the canister to call. + +- `method_name` (`text`): Name of the canister method to call + +- `arg` (`blob`): Argument to pass to the canister method + +The HTTP response to this request has an empty body and HTTP status 202, or a HTTP error (4xx or 5xx). Paranoid agents should not trust this response, and use [`read_state`](#http-read-state) to determine the status of the call. + +This request type can *also* be used to call a query method. A user may choose to go this way, instead of via the faster and cheaper [Request: Query call](#http-query) below, if they want to get a *certified* response. + +:::note +The functionality exposed via the [The IC management canister](#ic-management-canister) can be used this way. +::: + +### Request: Read state {#http-read-state} + +In order to read parts of the [The system state tree](#the-system-state-tree), the user makes a POST request to `/api/v2/canister//read_state`. The request body consists of an authentication envelope with a `content` map with the following fields: + +- `request_type` (`text`): Always `read_state` + +- `sender`, `nonce`, `ingress_expiry`: See [Authentication](#authentication) + +- `paths` (sequence of paths): A list of paths, where a path is itself a sequence of blobs. + +The HTTP response to this request consists of a CBOR map with the following fields: + +- `certificate` (`blob`): A certificate (see [Certification](#certification)). + + If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation's canister id range (see [Delegation](#certification-delegation)), unless + + - the `effective_canister_id` is that of the Management Canister (`aaaaa-aa`), + + - all requested paths have `/time` or `/request_status/` as prefix where the (single) original request referenced by `` is an update call to the Management Canister (`aaaaa-aa`) and the method name is `provisional_create_canister_with_cycles`, and + + - whenever the certificate contains the path `/request_status//reply`, then its value is a Candid-encoded record with a `canister_id` field of type `principal` and the `canister_id` must be included in each delegation's canister id range. + +The returned certificate reveals all values whose path is a suffix of the list of requested paths. It also always reveals `/time`, even if not explicitly requested. + +All requested paths must have one of the following paths as prefix: + +- `/time`. Can be requested by anyone. + +- `/subnet`. Can be requested by anyone. + +- `/request_status/`. Can only be requested by the same sender as the original request referenced by `` and if the effective canister id of the original request matches ``. + +- `/canisters//module_hash`. Can be requested by anyone if `` matches ``. + +- `/canisters//controllers`. Can be requested by anyone if `` matches ``. The order may vary depending on the implementation. + +- `/canisters//metadata/`. Can be requested by anyone if `` matches `` and `` is a public custom section. If `` is a private custom section, it can only be requested by the controllers of the canister. + +Note that the paths `/canisters//certified_data` are not accessible with this method; these paths are only exposed to the canister themselves via the System API (see [Certified data](#system-api-certified-data)). + +See [The system state tree](#the-system-state-tree) for details on the state tree. + +### Request: Query call {#http-query} + +A query call is a fast, but less secure way to call a canister. Only methods that are explicitly marked as "query methods" by the canister can be called this way. + +In order to make a query call to canister, the user makes a POST request to `/api/v2/canister//query`. The request body consists of an authentication envelope with a `content` map with the following fields: + +- `request_type` (`text`): Always `query` + +- `sender`, `nonce`, `ingress_expiry`: See [Authentication](#authentication) + +- `canister_id` (`blob`): The principal of the canister to call. + +- `method_name` (`text`): Name of the canister method to call + +- `arg` (`blob`): Argument to pass to the canister method + +If the call resulted in a reply, the response is a CBOR map with the following fields: + +- `status` (`text`): `replied` + +- `reply`: a CBOR map with the field `arg` (`blob`) which contains the reply data. + +If the call resulted in a reject, the response is a CBOR map with the following fields: + +- `status` (`text`): `rejected` + +- `reject_code` (`nat`): The reject code (see [Reject codes](#reject-codes)). + +- `reject_message` (`text`): a textual diagnostic message. + +- `error_code` (text): an optional implementation-specific textual error code (see [Error codes](#error-codes)). + +Canister methods that do not change the canister state can be executed more efficiently. This method provides that ability, and returns the canister's response directly within the HTTP response. + +### Effective canister id {#http-effective-canister-id} + +The `` in the URL paths of requests is the *effective* destination of the request. + +- If the request is an update call to the Management Canister (`aaaaa-aa`), then: + + - If the call is to the `provisional_create_canister_with_cycles` method, then any principal can be used as the effective canister id for this call. + + - Otherwise, if the `arg` is a Candid-encoded record with a `canister_id` field of type `principal`, then the effective canister id must be that principal. + + - Otherwise, the call is rejected by the system independently of the effective canister id. + +- If the request is an update call to a canister that is not the Management Canister (`aaaaa-aa`) or if the request is a query call, then the effective canister id must be the `canister_id` in the request. + + ::: note + The expectation is that user-side agent code shields users and developers from the notion of effective canister ID, in analogy to how the System API interface shields canister developers from worrying about routing. + + The Internet Computer blockchain mainnet rejects all requests whose effective canister id is in no subnet's canister ranges, independently of whether the remaining conditions on the effective canister id are satisfied. + + The Internet Computer blockchain mainnet does not support `provisional_create_canister_with_cycles` and thus all calls to this method are rejected independently of the effective canister id. + + In development instances of the Internet Computer Protocol (e.g. testnets), the effective canister id of a request submitted to a node must be a canister id from the canister ranges of the subnet to which the node belongs, unless the request is an update call to the Management Canister (`aaaaa-aa`), the method name is `provisional_create_canister_with_cycles`, and the effective canister id is that of the Management Canister (`aaaaa-aa`). + ::: + +### Authentication + +All requests coming in via the HTTPS interface need to be either *anonymous* or *authenticated* using a cryptographic signature. To that end, the following fields are present in the `content` map in all cases: + +- `nonce` (`blob`, optional): Arbitrary user-provided data, typically randomly generated. This can be used to create distinct requests with otherwise identical fields. + +- `ingress_expiry` (`nat`, required): An upper limit on the validity of the request, expressed in nanoseconds since 1970-01-01 (like [ic0.time()](#system-api-time)). This avoids replay attacks: The IC will not accept requests, or transition requests from status `received` to status `processing`, if their expiry date is in the past. The IC may refuse to accept requests with an ingress expiry date too far in the future. This applies to synchronous and asynchronous requests alike (and could have been called `request_expiry`). + +- `sender` (`Principal`, required): The user who issued the request. + +The envelope, i.e. the overall request, has the following keys: + +- `content` (`record`): the actual request content + +- `sender_pubkey` (`blob`, optional): Public key used to authenticate this request. Since a user may in the future have more than one key, this field tells the IC which key is used. + +- `sender_delegation` (`array` of maps, optional): a chain of delegations, starting with the one signed by `sender_pubkey` and ending with the one delegating to the key relating to `sender_sig`. + +- `sender_sig` (`blob`, optional): Signature to authenticate this request. + +The public key must authenticate the `sender` principal: + +- A public key can authenticate a principal if the latter is a self-authenticating id derived from that public key (see [Special forms of Principals](#id-classes)). + +- The fields `sender_pubkey`, `sender_sig`, and `sender_delegation` must be omitted if the `sender` field is the anonymous principal. The fields `sender_pubkey` and `sender_sig` must be set if the `sender` field is not the anonymous principal. + +The request id (see [Request ids](#request-id)) is calculated from the content record. This allows the signature to be based on the request id, and implies that signature and public key are not semantically relevant. + +The field `sender_pubkey` contains a public key supported by one of the schemes described in [Signatures](#signatures). + +Signing transactions can be delegated from one key to another one. If delegation is used, then the `sender_delegation` field contains an array of delegations, each of which is a map with the following fields: + +- `delegation` (`map`): Map with fields: + + - `pubkey` (`blob`): Public key as described in [Signatures](#signatures). + + - `expiration` (`nat`): Expiration of the delegation, in nanoseconds since 1970-01-01, analogously to the `ingress_expiry` field above. + + - `targets` (`array` of `CanisterId`, optional): If this field is set, the delegation only applies for requests sent to the canisters in the list. + +- `signature` (`blob`): Signature on the 32-byte [representation-independent hash](#hash-of-map) of the map contained in the `delegation` field as described in [Signatures](#signatures), using the 27 bytes `\x1Aic-request-auth-delegation` as the domain separator. + + For the first delegation in the array, this signature is created with the key corresponding to the public key from the `sender_pubkey` field, all subsequent delegations are signed with the key corresponding to the public key contained in the preceding delegation. + +The `sender_sig` field is calculated by signing the concatenation of the 11 bytes `\x0Aic-request` (the domain separator) and the 32 byte [request id](#request-id) with the secret key that belongs to the key specified in the last delegation or, if no delegations are present, the public key specified in `sender_pubkey`. + +The delegation field, if present, must not contain more than four delegations. + +### Representation-independent hashing of structured data {#hash-of-map} + +Structured data, such as (recursive) maps, are authenticated by signing a representation-independent hash of the data. This hash is computed as follows (using SHA256 in the steps below): + +1. For each field that is present in the map (i.e. omitted optional fields are indeed omitted): + + - concatenate the hash of the field's name (in ascii-encoding, without terminal `\x00`) and the hash of the value (with the encoding specified below). + +2. Sort these concatenations from low to high + +3. Concatenate the sorted elements, and hash the result. + +The resulting hash of 256 bits (32 bytes) is the representation-independent hash. + +The following encodings of field values as blobs are used + +- Binary blobs (`canister_id`, `arg`, `nonce`, `module`) are used as-is. + +- Strings (`request_type`, `method_name`) are encoded in UTF-8, without a terminal `\x00`. + +- Natural numbers (`compute_allocation`, `memory_allocation`, `ingress_expiry`) are encoded using the shortest form [Unsigned LEB128](https://en.wikipedia.org/wiki/LEB128#Unsigned_LEB128) encoding. For example, `0` should be encoded as a single zero byte `[0x00]` and `624485` should be encoded as byte sequence `[0xE5, 0x8E, 0x26]`. + +- Arrays (`paths`) are encoded as the concatenation of the hashes of the encodings of the array elements. + +- Maps (`sender_delegation`) are encoded by recursively computing the representation-independent hash. + +### Request ids {#request-id} + +When signing requests or querying the status of a request (see [Request status](#state-tree-request-status)) in the state tree, the user identifies the request using a *request id*, which is the [representation-independent hash](#hash-of-map) of the `content` map of the original request. A request id must have length of 32 bytes. + +:::note +The request id is independent of the representation of the request (currently only CBOR), and does not change if the specification adds further optional fields to a request type. +::: + +:::note +The recommended textual representation of a request id is a hexadecimal string with lower-case letters prefixed with \'0x\'. E.g., request id consisting of bytes `[00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 0A, 0B, 0C, 0D, 0E, 0F, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 1A, 1B, 1C, 1D, 1E, 1F]` should be displayed as `0x000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f`. +::: + +:::tip +Example calculation (where `H` denotes SHA-256 and `·` denotes blob concatenation): + + hash_of_map({ request_type: "call", canister_id: 0x00000000000004D2, method_name: "hello", arg: "DIDL\x00\xFD*"}) + = H(concat (sort + [ H("request_type") · H("call") + , H("canister_id") · H("\x00\x00\x00\x00\x00\x00\x04\xD2") + , H("method_name") · H("hello") + , H("arg") · H("DIDL\x00\xFD*") + ])) + = H(concat (sort + [ 769e6f87bdda39c859642b74ce9763cdd37cb1cd672733e8c54efaa33ab78af9 · 7edb360f06acaef2cc80dba16cf563f199d347db4443da04da0c8173e3f9e4ed + , 0a3eb2ba16702a387e6321066dd952db7a31f9b5cc92981e0a92dd56802d3df9 · 4d8c47c3c1c837964011441882d745f7e92d10a40cef0520447c63029eafe396 + , 293536232cf9231c86002f4ee293176a0179c002daa9fc24be9bb51acdd642b6 · 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 + , b25f03dedd69be07f356a06fe35c1b0ddc0de77dcd9066c4be0c6bbde14b23ff · 6c0b2ae49718f6995c02ac5700c9c789d7b7862a0d53e6d40a73f1fcd2f70189 + ])) + = H(concat + [ 0a3eb2ba16702a387e6321066dd952db7a31f9b5cc92981e0a92dd56802d3df9 · 4d8c47c3c1c837964011441882d745f7e92d10a40cef0520447c63029eafe396 + , 293536232cf9231c86002f4ee293176a0179c002daa9fc24be9bb51acdd642b6 · 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 + , 769e6f87bdda39c859642b74ce9763cdd37cb1cd672733e8c54efaa33ab78af9 · 7edb360f06acaef2cc80dba16cf563f199d347db4443da04da0c8173e3f9e4ed + , b25f03dedd69be07f356a06fe35c1b0ddc0de77dcd9066c4be0c6bbde14b23ff · 6c0b2ae49718f6995c02ac5700c9c789d7b7862a0d53e6d40a73f1fcd2f70189 + ]) + = 8781291c347db32a9d8c10eb62b710fce5a93be676474c42babc74c51858f94b +::: + +### Reject codes + +An API request or inter-canister call that is pending in the IC will eventually result in either a *reply* (indicating success, and carrying data) or a *reject* (indicating an error of some sorts). A reject contains a *rejection code* that classifies the error and a hopefully helpful *reject message* string. + +Rejection codes are member of the following enumeration: + +- `SYS_FATAL` (1): Fatal system error, retry unlikely to be useful. + +- `SYS_TRANSIENT` (2): Transient system error, retry might be possible. + +- `DESTINATION_INVALID` (3): Invalid destination (e.g. canister/account does not exist) + +- `CANISTER_REJECT` (4): Explicit reject by the canister. + +- `CANISTER_ERROR` (5): Canister error (e.g., trap, no response) + +The symbolic names of this enumeration are used throughout this specification, but on all interfaces (HTTPS API, System API), they are represented as positive numbers as given in the list above. + +The error message is guaranteed to be a string, i.e. not arbitrary binary data. + +When canisters explicitly reject a message (see [Public methods](#system-api-requests)), they can specify the reject message, but *not* the reject code; it is always `CANISTER_REJECT`. In this sense, the reject code is trustworthy: If the IC responds with a `SYS_FATAL` reject, then it really was the IC issuing this reject. + +### Error codes + +Implementations of the API can provide additional details for rejected messages in the form of a textual label identifying the error condition. API clients can use these labels to handle errors programmatically or suggest recovery paths to the user. The specification reserves error codes matching the regular expression `IC[0-9]+` (e.g., `IC502`) for the DFINITY implementation of the API. + +### Status endpoint {#api-status} + +Additionally, the Internet Computer provides an API endpoint to obtain various status fields at + + /api/v2/status + +For this endpoint, the user performs a GET request, and receives a CBOR value with the following fields. The IC may include additional implementation-specific fields. + +- `ic_api_version` (string, mandatory): Identifies the interface version supported, i.e. the version of the present document that the internet computer aims to support, e.g. `0.8.1`. The implementation may also return `unversioned` to indicate that it does *not* comply to a particular version, e.g. in between releases. + +- `impl_source` (string, optional): Identifies the implementation of the Internet Computer Protocol, by convention with the canonical location of the source code (e.g. `https://github.com/dfinity/ic`). + +- `impl_version` (string, optional): If the user is talking to a released version of an Internet Computer Protocol implementation, this is the version number. For non-released versions, output of `git describe` like `0.1.13-13-g2414721` would also be very suitable. + +- `impl_revision` (string, optional): The precise git revision of the Internet Computer Protocol implementation + +- `root_key` (blob, only in development instances): The public key (a DER-encoded BLS key) of the root key of this development instance of the Internet Computer Protocol. This *must* be present in short-lived development instances, to allow the agent to fetch the public key. For the Internet Computer, agents must have an independent trustworthy source for this data, and must not be tempted to fetch it from this insecure location. + +See [CBOR encoding of requests and responses](#api-cbor) for details on the precise CBOR encoding of this object. + +:::note +Future additions may include local time, geographic location, and other useful implementation-specific information such as blockheight. This data may possibly be signed by the node. +::: + +### CBOR encoding of requests and responses {#api-cbor} + +Requests and responses are specified here as records with named fields and using suggestive human readable syntax. The actual format in the body of the HTTP request or response, however, is [CBOR](https://en.wikipedia.org/wiki/CBOR). + +Concretely, it consists of a data item with major type 6 ("Semantic tag") and tag value `55799` (see [Self-Describe CBOR](https://tools.ietf.org/html/rfc7049#section-2.4.5)), followed by a record. + +Requests consist of an envelope record with keys `sender_sig` (a blob), `sender_pubkey` (a blob) and `content` (a record). The first two are metadata that are used for request authentication, while the last one is the actual content of the request. + +The following encodings are used: + +- Strings: Major type 3 ("Text string"). + +- Blobs: Major type 2 ("Byte string"). + +- Nats: Major type 0 ("Unsigned integer") if small enough to fit that type, else the [Bignum](https://tools.ietf.org/html/rfc7049#section-2.4.2) format is used. + +- Records: Major type 5 ("Map of pairs of data items"), followed by the fields, where keys are encoded with major type 3 ("Text string"). + +- Arrays: Major type 4 (\"Array of data items\"). + +As advised by [section "Creating CBOR-Based Protocols"](https://tools.ietf.org/html/rfc7049#section-3) of the CBOR spec, we clarify that: + +- Floating-point numbers may not be used to encode integers. + +- Duplicate keys are prohibited in CBOR maps. + +:::tip +A typical request would be (written in [CBOR diagnostic notation](https://tools.ietf.org/html/rfc7049#section-6), which can be checked and converted on [cbor.me](http://cbor.me/)): + + 55799({ + "content": { + "request_type": "call", + "canister_id": h'ABCD01', + "method_name": "say_hello", + "arg": h'0061736d01000000' + }, + "sender_sig": h'DEADBEEF', + "sender_pubkey": h'b7a3c12dc0c8c748ab07525b701122b88bd78f600c76342d27f25e5f92444cde' + }) +::: + +### CDDL description of requests and responses {#api-cddl} + +The [Concise Data Definition Language (CDDL)](https://tools.ietf.org/html/rfc8610) is a data description language for CBOR. This section summarizes the format of the CBOR data passed to and from the entry points described above. This [file](_attachments/requests.cddl) summarizes the format of the CBOR data passed to and from the entry points described above. + +### Ordering guarantees {#ordering_guarantees} + +The order in which the various messages between canisters are delivered and executed is not fully specified. The guarantee provided by the IC is that function calls between two canisters are executed in order, so that a canister that requires in-order execution need not wait for the response from an earlier message to a canister before sending a later message to that same canister. + +More precisely: + +- Method calls between any *two* canisters are delivered in order, as if they were communicating over a single simple FIFO queue. + +- If a WebAssembly function, within a single invocation, makes multiple calls to the same canister, they are queued in the order of invocations to `ic0.call_perform`. + +- Responses (including replies with `ic0.msg_reply`, explicit rejects with `ic0.msg_reject` and system-generated error responses) do *not* have any ordering guarantee relative to each other or to method calls. + +- There is no particular order guarantee for ingress messages submitted via the HTTPS interface. + +### Synchronicity across nodes {#synchronicity_across_nodes} + +This document describes the Internet Computer as having a single global state that can be modified and queried. In reality, it consists of many nodes, which may not be perfectly in sync. + +As long as you talk to one (honest) node only, the observed behavior is nicely sequential. If you issue an update (i.e. state-mutating) call to a canister (e.g. bump a counter), and node A indicates that the call has been executed, and you then issue a query call to node A, then A's response is guaranteed to include the effect of the update call (and you will receive the updated counter value). + +If you then (quickly) issue a read request to node B, it may be that B responds to your read query based on the old state of the canister (and you might receive the old counter value). + +A related problem is that query calls are not certified, and nodes may be dishonest in their response. In that case, the user might want to get more assurance by querying multiple nodes and comparing the result. However, it is (currently) not possible to query a *specific* state. + +:::note +Applications can work around these problems. For the first problem, the query result could be such that the user can tell if the update has been received or not. For the second problem, even if using [certified data](#system-api-certified-data) is not possible, if replies are monotonic in some sense the user can get assurance in their intersection (e.g. if the query returns a list of events that grows over time, then even if different nodes return different lists, the user can get assurance in those events that are reported by many nodes). +::: + +## Canister module format + +A canister module is simply a [WebAssembly module](https://webassembly.github.io/spec/core/index.html) in binary format (typically `.wasm`). + +## Canister interface (System API) {#system-api} + +The System API is the interface between the running canister and the Internet Computer. It allows the WebAssembly module of a canister to expose functionality to the users (method entry points) and the IC (e.g. initialization), and exposes functionality of the IC to the canister (e.g. calling other canisters). Because WebAssembly is rather low-level, it also explains how to express higher level concepts (e.g. binary blobs). + +We want to leverage advanced WebAssembly features, such as WebAssembly host references. But as they are not yet supported by all tools involved, this section describes an initial System API that does not rely on host references. In section [Outlook: Using Host References](#host-references), we outline some of the proposed uses of WebAssembly host references. + +### WebAssembly module requirements {#system-api-module} + +In order for a WebAssembly module to be usable as the code for the canister, it needs to conform to the following requirements: + +- If it imports a memory, it must import it from `env.memory`. In the following, "the Wasm memory" refers to this memory. + +- If it imports a table, it must import it from `env.table`. In the following, "the Wasm table" refers to this table. + +- It may only import a function if it is listed in [Overview of imports](#system-api-imports). + +- It may have a `(start)` function. + +- If it exports a function called `canister_init`, the function must have type `() -> ()`. + +- If it exports a function called `canister_inspect_message`, the function must have type `() -> ()`. + +- If it exports a function called `canister_heartbeat`, the function must have type `() -> ()`. + +- If it exports any functions called `canister_update ` or `canister_query ` for some `name`, the functions must have type `() -> ()`. + +- It may not export both `canister_update ` and `canister_query ` with the same `name`. + +- It may not export other methods the names of which start with the prefix `canister_` besides the methods allowed above. + +- It may not have both `icp:public ` and `icp:private ` with the same `name` as the custom section name. + +- It may not have other custom sections the names of which start with the prefix `icp:` besides the \`icp:public \` and \`icp:private \`. + +- The IC may reject WebAssembly modules that + declare more than 6000 functions, or + declare more than 200 globals, or + declare more than 16 exported custom sections (the custom section names with prefix `icp:`), or + the total size of the exported custom sections exceeds 1MiB + +### Interpretation of numbers {#interpretation_of_numbers} + +WebAssembly number types (`i32`, `i64`) do not indicate if the numbers are to be interpreted as signed or unsigned. Unless noted otherwise, whenever the System API interprets them as numbers (e.g. memory pointers, buffer offsets, array sizes), they are to be interpreted as unsigned. + +### Entry points + +The canister provides entry points which are invoked by the IC under various circumstances: + +- The canister may export a function with name `canister_init` and type `() -> ()`. + +- The canister may export a function with name `canister_pre_upgrade` and type `() -> ()`. + +- The canister may export a function with name `canister_post_upgrade` and type `() -> ()`. + +- The canister may export functions with name `canister_inspect_message` with type `() -> ()`. + +- The canister may export a function with name `canister_heartbeat` with type `() -> ()`. + +- The canister may export functions with name `canister_update ` and type `() -> ()`. + +- The canister may export functions with name `canister_query ` and type `() -> ()`. + +- The canister table may contain functions of type `(env : i32) -> ()` which may be used as callbacks for inter-canister calls. + +If the execution of any of these entry points traps for any reason, then all changes to the WebAssembly state, as well as the effect of any externally visible system call (like `ic0.msg_reply`, `ic0.msg_reject`, `ic0.call_perform`), are discarded. For upgrades, this transactional behavior applies to the `canister_pre_upgrade`/`canister_post_upgrade` sequence as a whole. + +#### Canister initialization {#system-api-init} + +If `canister_init` is present, then this is the first exported WebAssembly function invoked by the IC. The argument that was passed along with the canister initialization call (see [IC method ](#ic-install_code)) is available to the canister via `ic0.msg_arg_data_size/copy`. + +The IC assumes the canister to be fully instantiated if the `canister_init` method entry point returns. If the `canister_init` method entry point traps, then canister installation has failed, and the canister is reverted to its previous state (i.e. empty with `install`, or whatever it was for a `reinstall`). + +#### Canister upgrades {#system-api-upgrades} + +When a canister is upgraded to a new WebAssembly module, the IC: + +1. Invokes `canister_pre_upgrade` (if present) on the old instance, to give the canister a chance to clean up (e.g. move data to [stable memory](#system-api-stable-memory)). + +2. Instantiates the new module, including the execution of `(start)`, with a fresh WebAssembly state. + +3. Invokes `canister_post_upgrade` (if present) on the new instance, passing the `arg` provided in the `install_code` call ([IC method ](#ic-install_code)). + +The stable memory is preserved throughout the process; any other WebAssembly state is discarded. + +During these steps, no other entry point of the old or new canister is invoked. The `canister_init` function of the new canister is *not* invoked. + +These steps are atomic: If `canister_pre_upgrade` or `canister_post_upgrade` trap, the upgrade has failed, and the canister is reverted to the previous state. Otherwise, the upgrade has succeeded, and the old instance is discarded. + +#### Public methods {#system-api-requests} + +To define a public method of name `name`, a WebAssembly module exports a function with name `canister_update ` or `canister_query ` and type `() -> ()`. We call this the *method entry point*. The name of the exported function distinguishes update and query methods. + +:::note +The space in `canister_update ` resp. `canister_query ` is intentional. There is exactly one space between `canister_update/canister_query` and the ``. +::: + +The argument of the call (e.g. the content of the `arg` field in the [API request to call a canister method](#http-call)) is copied into the canister on demand using the System API functions shown below. + +Eventually, a method will want to send a response, using `ic0.reply` or `ic0.reject` + +#### Heartbeat {#heartbeat} + +For periodic or time-based execution, the WebAssembly module can export a function with name `canister_heartbeat`. The heartbeats scheduling algorithm is implementation-defined. + +`canister_heartbeat` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. + +:::note +While an implementation will likely try to keep the interval between `canister_heartbeat` invocations to within a few seconds, this is not formally part of this specification. +::: + +#### Callbacks {#callbacks} + +Callbacks are addressed by their table index (as a proxy for a Wasm `funcref`). + +In the reply callback of a [inter-canister method call](#system-api-call), the argument refers to the response to that call. In reject callbacks, no argument is available. + +### Overview of imports {#system-api-imports} + + +The following sections describe various System API functions, also referred to as system calls, which we summarize here. + + ic0.msg_arg_data_size : () -> i32; // I U Q Ry F + ic0.msg_arg_data_copy : (dst : i32, offset : i32, size : i32) -> (); // I U Q Ry F + ic0.msg_caller_size : () -> i32; // I G U Q F + ic0.msg_caller_copy : (dst : i32, offset: i32, size : i32) -> (); // I G U Q F + ic0.msg_reject_code : () -> i32; // Ry Rt + ic0.msg_reject_msg_size : () -> i32; // Rt + ic0.msg_reject_msg_copy : (dst : i32, offset : i32, size : i32) -> (); // Rt + + ic0.msg_reply_data_append : (src : i32, size : i32) -> (); // U Q Ry Rt + ic0.msg_reply : () -> (); // U Q Ry Rt + ic0.msg_reject : (src : i32, size : i32) -> (); // U Q Ry Rt + + ic0.msg_cycles_available : () -> i64; // U Rt Ry + ic0.msg_cycles_available128 : (dst : i32) -> (); // U Rt Ry + ic0.msg_cycles_refunded : () -> i64; // Rt Ry + ic0.msg_cycles_refunded128 : (dst : i32) -> (); // Rt Ry + ic0.msg_cycles_accept : (max_amount : i64) -> ( amount : i64 ); // U Rt Ry + ic0.msg_cycles_accept128 : (max_amount_high : i64, max_amount_low: i64, dst : i32) + -> (); // U Rt Ry + + ic0.canister_self_size : () -> i32; // * + ic0.canister_self_copy : (dst : i32, offset : i32, size : i32) -> (); // * + ic0.canister_cycle_balance : () -> i64; // * + ic0.canister_cycle_balance128 : (dst : i32) -> (); // * + ic0.canister_status : () -> i32; // * + + ic0.msg_method_name_size : () -> i32 // F + ic0.msg_method_name_copy : (dst : i32, offset : i32, size : i32) -> (); // F + ic0.accept_message : () -> (); // F + + ic0.call_new : // U Ry Rt H + ( callee_src : i32, + callee_size : i32, + name_src : i32, + name_size : i32, + reply_fun : i32, + reply_env : i32, + reject_fun : i32, + reject_env : i32 + ) -> (); + ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt H + ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt H + ic0.call_cycles_add : (amount : i64) -> (); // U Ry Rt H + ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt H + ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt H + + ic0.stable_size : () -> (page_count : i32); // * + ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32); // * + ic0.stable_write : (offset : i32, src : i32, size : i32) -> (); // * + ic0.stable_read : (dst : i32, offset : i32, size : i32) -> (); // * + ic0.stable64_size : () -> (page_count : i64); // * + ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64); // * + ic0.stable64_write : (offset : i64, src : i64, size : i64) -> (); // * + ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> (); // * + + ic0.certified_data_set : (src: i32, size: i32) -> () // I G U Ry Rt H + ic0.data_certificate_present : () -> i32 // * + ic0.data_certificate_size : () -> i32 // * + ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> () // * + + ic0.time : () -> (timestamp : i64); // * + ic0.performance_counter : (type : i32) -> (counter : i64); // * s + + ic0.debug_print : (src : i32, size : i32) -> (); // * s + ic0.trap : (src : i32, size : i32) -> (); // * s + +The comment after each function lists from where these functions may be invoked: + +- `I`: from `canister_init` or `canister_post_upgrade` + +- `G`: from `canister_pre_upgrade` + +- `U`: from `canister_update …` + +- `Q`: from `canister_query …` + +- `Ry`: from a reply callback + +- `Rt`: from a reject callback + +- `C`: from a cleanup callback + +- `s`: the `(start)` module initialization function + +- `F`: from `canister_inspect_message` + +- `H`: from `canister_heartbeat` + +- `*` = `I G U Q Ry Rt C F H` (NB: Not `(start)`) + +If the canister invokes a system call from somewhere else, it will trap. + +### Blob-typed arguments and results {#blob_typed_arguments_and_results} + +WebAssembly functions parameter and result types can only be primitive number types. To model functions that accept or return blobs or text values, the following idiom is used: + +To provide access to a string or blob `foo`, the System API provides two functions: + + ic0.foo_size : () -> i32 + ic0.foo_copy : (dst : i32, offset: i32, size : i32) -> () + +The `*_size` function indicates the size, in bytes, of `foo`. The `*_copy` function copies `size` bytes from `foo[offset..offset+size]` to `memory[dst..dst+size]`. This traps if `offset+size` is greater than the size of `foo`, or if `dst+size` exceeds the size of the Wasm memory. + +Dually, a System API function that conceptually takes a blob or string as a parameter `foo` has two parameters: + + ic0.set_foo : (src : i32, size: i32) -> … + +which copies, at the time of function invocation, the data referred to by `src`/`size` out of the canister. Unless otherwise noted, this traps if `src+size` exceeds the size of the WebAssembly memory. + +### Method arguments {#method_arguments} + +The canister can access an argument. For `canister_init`, `canister_post_upgrade` and method entry points, the argument is the argument of the call; in a reply callback, it refers to the received reply. So the lifetime of the argument data is a single WebAssembly function execution, not the whole method call tree. + +- ``` +ic0.msg_arg_data_size : () -> i32 +ic0.msg_arg_data_copy : (dst : i32, offset : i32, size : i32) -> () + ``` + +The message argument data. + +- ``` +ic0.msg_caller_size : () -> i32 +ic0.msg_caller_copy : (dst : i32, offset: i32, size : i32) -> () + ``` + +The identity of the caller, which may be a canister id or a user id. During canister installation or upgrade, this is the id of the user or canister requesting the installation or upgrade. + +- `ic0.msg_reject_code : () -> i32` + +Returns the reject code, if the current function is invoked as a reject callback. + +It returns the special "no error" code `0` if the callback is *not* invoked as a reject callback; this allows canisters to use a single entry point for both the reply and reject callback, if they choose to do so. + + +- ``` +ic0.msg_reject_msg_size : () -> i32 +ic0.msg_reject_msg_copy : (dst : i32, offset : i32, size : i32) -> () + ``` + + +The reject message. Traps if there is no reject message (i.e. if `reject_code` is `0`). + +### Responding + +Eventually, the canister will want to respond to the original call, either by replying (indicating success) or rejecting (signalling an error): + +- `ic0.msg_reply_data_append : (src : i32, size : i32) -> ()` + + Appends data it to the (initially empty) data reply. + + :::note + This can be invoked multiple times to build up the argument with data from various places on the Wasm heap. This way, the canister does not have to first copy all the pieces from various places into one location. + ::: + + This traps if the current call already has been or does not need to be responded to. + +- `ic0.msg_reply : () -> ()` + + Replies to the sender with the data assembled using `ic0.msg_reply_data_append`. + + This function can be called at most once (a second call will trap), and must be called exactly once to indicate success. + + See [Cycles](#system-api-cycles) for how this interacts with cycles available on this call. + +- `ic0.msg_reject : (src : i32, size : i32) -> ()` + + Rejects the call. The data referred to by `src`/`size` is used for the diagnostic message. + + This system call traps if `src+size` exceeds the size of the WebAssembly memory, or if the current call already has been or does not need to be responded to, or if the data referred to by `src`/`size` is not valid UTF8. + + The other end will receive this reject with reject code `CANISTER_REJECT`, see [Reject codes](#reject-codes). + + Possible reply data assembled using `ic0.msg_reply_data_append` is discarded. + + See [Cycles](#system-api-cycles) for how this interacts with cycles available on this call. + +### Ingress message inspection {#system-api-inspect-message} + +A canister can inspect ingress messages before executing them. When the IC receives an update call from a user, the IC will use the canister method `canister_inspect_message` to determine whether the message shall be accepted. If the canister is empty (i.e. does not have a Wasm module), then the ingress message will be rejected. If the canister is not empty and does not implement `canister_inspect_message`, then the ingress message will be accepted. + +In `canister_inspect_message`, the canister can accept the message by invoking `ic0.accept_message : () → ()`. This function traps if invoked twice. If the canister traps in `canister_inspect_message` or does not call `ic0.accept_message`, then the access is denied. + +:::note +The `canister_inspect_message` is *not* invoked for query calls, inter-canister calls or calls to the management canister. +::: + +### Self-identification {#system-api-canister-self} + +A canister can learn about its own identity: + +- ``` + ic0.canister_self_size : () -> i32 + ic0.canister_self_copy: (dst : i32, offset : i32, size : i32) -> () + ``` + +These functions allow the canister to query its own canister id (as a blob). + +### Canister status {#system-api-canister-status} + +This function allows a canister to find out if it is running, stopping or stopped (see [IC method ](#ic-canister_status) and [IC method ](#ic-stop_canister) for context). + +- `ic0.canister_status : () → i32` + + returns the current status of the canister: + + Status `1` indicates running, `2` indicates stopping, and `3` indicates stopped. + + Status `3` (stopped) can be observed, for example, in `canister_pre_upgrade` and can be used to prevent accidentally upgrading a canister that is not fully stopped. + +### Inter-canister method calls {#system-api-call} + +When handling an update call (or a callback), a canister can do further calls to another canister. Calls are assembled in a builder-like fashion, starting with `ic0.call_new`, adding more attributes using the `ic0.call_*` functions, and eventually performing the call with `ic0.call_perform`. + +- ``` + ic0.call_new : + ( callee_src : i32, + callee_size : i32, + name_src : i32, + name_size : i32, + reply_fun : i32, + reply_env : i32, + reject_fun : i32, + reject_env : i32, + ) -> () + ``` + + Begins assembling a call to the canister specified by `callee_src/_size` at method `name_src/_size`. + + The IC records two mandatory callback functions, represented by a table entry index `*_fun` and some additional value `*_env`. When the response comes back, the table is read at the corresponding index, expected to be a function of type `(env : i32) -> ()`, and passed the corresponding `*_env` value. + + The reply callback is executed upon successful completion of the method call, which can query the reply using `ic0.msg_arg_data_*`. + + The reject callback is executed if the method call fails asynchronously or the other canister explicitly rejects the call. The reject code and message can be queried using `ic0.msg_reject_code` and `ic0.msg_reject_msg_*`. + + This deducts `MAX_CYCLES_PER_RESPONSE` cycles from the canister balance and sets them aside for response processing. This will trap if not sufficient cycles are available. + + Subsequent calls to the following functions set further attributes of that call, until the call is concluded (with `ic0.call_perform`) or discarded (by returning without calling `ic0.call_perform` or by starting a new call with `ic0.call_new`.) + +- `ic0.call_on_cleanup : (fun : i32, env : i32) -> ()` + + If a cleanup callback (of type `(env : i32) -> ()`) is specified for this call, it is executed if and only if the `reply` or the `reject` callback was executed and trapped (for any reason). + + During the execution of the `cleanup` function, only a subset of the System API is available (namely `ic0.debug_print`, `ic0.trap` and the `ic0.stable_*` functions). The cleanup function is expected to run swiftly (within a fixed, yet to be specified cycle limit) and serves to free resources associated with the callback. + + If this traps (e.g. runs out of cycles), the state changes from the `cleanup` function are discarded, as usual, and no further actions are taken related to that call. Canisters likely want to avoid this from happening. + + There must be at most one call to `ic0.call_on_cleanup` between `ic0.call_new` and `ic0.call_perform`. + + +- `ic0.call_data_append : (src : i32, size : i32) -> ()` + + Appends the specified bytes to the argument of the call. Initially, the argument is empty. + + This may be called multiple times between `ic0.call_new` and `ic0.call_perform`. + +- `ic0.call_cycles_add : (amount : i64) -> ()` + + This adds cycles onto a call. See [Cycles](#system-api-cycles). + + This may be called multiple times between `ic0.call_new` and `ic0.call_perform`. + +- `ic0.call_cycles_add128 : (amount_high : i64, amount_low : i64) -> ()` + + This adds cycles onto a call. See [Cycles](#system-api-cycles). + + This may be called multiple times between `ic0.call_new` and `ic0.call_perform`. + +- `ic0.call_perform : () -> ( err_code : i32 )` + + This concludes assembling the call. It queues the call message to the given destination, but does not actually act on it until the current WebAssembly function returns without trapping. + + If the function returns `0` as the `err_code`, the IC was able to enqueue the call. In this case, the call will either be delivered, returned because the destination canister does not exist or returned because of an out of cycles condition. This also means that exactly one of the reply or reject callbacks will be executed. + + If the function returns a non-zero value, the call cannot (and will not be) performed. This can happen due to a lack of resources within the IC, but also if it would reduce the current cycle balance to a level below where the canister would be frozen. + + After `ic0.call_perform` and before the next call to `ic0.call_new`, all other `ic0.call_*` function calls trap. + +### Cycles {#system-api-cycles} + +Each canister maintains a balance of *cycles*, which are used to pay for platform usage. Cycles are represented by 128-bit values. + +:::note +This specification currently does not go into details about which actions cost how many cycles and/or when. In general, you must assume that the canister's cycle balance can change arbitrarily between method executions, and during each System API function call, unless explicitly mentioned otherwise. +::: + +- `ic0.canister_cycle_balance : () → i64` + + Indicates the current cycle balance of the canister. It is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add`. After execution of the message, the IC may add unused cycles from the reserve back to the balance. + + :::note + This call traps if the current balance does not fit into a 64-bit value. Canisters that need to deal with larger cycles balances should use `ic0.canister_cycles_balance128` instead. + ::: + +- `ic0.canister_cycle_balance128 : (dst : i32) → ()` + + Indicates the current cycle balance of the canister by copying the value at the location `dst` in the canister memory. It is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add128`. After execution of the message, the IC may add unused cycles from the reserve back to the balance. + + This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +- `ic0.msg_cycles_available : () → i64` + + Returns the amount of cycles that were transferred by the caller of the current call, and is still available in this message. + + Initially, in the update method entry point, this is the amount that the caller passed to the canister. When cycles are accepted (`ic0.msg_cycles_accept`), this reports fewer cycles accordingly. When the call is responded to (reply or reject), all available cycles are refunded to the caller, and this will return 0. + + :::note + This call traps if the amount of cycles available does not fit into a 64-bit value. Please use `ic0.msg_cycles_available128` instead. + ::: + +- `ic0.msg_cycles_available128 : (dst : i32) → ()` + + Indicates the number of cycles transferred by the caller of the current call, still available in this message. The amount of cycles is represented by a 128-bit value. This call copies this value starting at the location `dst` in the canister memory. + + Initially, in the update method entry point, this is the amount that the caller passed to the canister. When cycles are accepted (`ic0.msg_cycles_accept128`), this reports fewer cycles accordingly. When the call is responded to (reply or reject), all available cycles are refunded to the caller, and this will report 0 cycles. + + This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +- `ic0.msg_cycles_accept : (max_amount : i64) → (amount : i64)` + + This moves cycles from the call to the canister balance. It moves as many cycles as possible, up to these constraints: + + - It moves no more cycles than `max_amount`. + + - It moves no more cycles than available according to `ic0.msg_cycles_available`, and + + It can be called multiple times, each time possibly adding more cycles to the balance. + + The return value indicates how many cycles were actually moved. + + This system call does not trap. + + :::tip + Example: To accept all cycles provided in a call, invoke `ic0.msg_cycles_accept(ic0.msg_cycles_available())` in the method handler or a callback handler, *before* calling reply or reject. + ::: + +- `ic0.msg_cycles_accept128 : (max_amount_high : i64, max_amount_low : i64, dst : i32) → ()` + + This moves cycles from the call to the canister balance. It moves as many cycles as possible, up to these constraints: + + - It moves no more cycles than the amount obtained by combining `max_amount_high` and `max_amount_low`. Cycles are represented by 128-bit values. + + - It moves no more cycles than available according to `ic0.msg_cycles_available128`, and + + It can be called multiple times, each time possibly adding more cycles to the balance. + + This call also copies the amount of cycles that were actually moved starting at the location `dst` in the canister memory. + + This does not trap. + + This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +- `ic0.call_cycles_add : (amount : i64) → ()` + + This function moves cycles from the canister balance onto the call under construction, to be transferred with that call. + + The cycles are deducted from the balance as shown by `ic0.canister_cycle_balance` immediately, and moved back if the call cannot be performed (e.g. if `ic0.call_perform` signals an error, or if the canister invokes `ic0.call_new` or returns without calling `ic0.call_perform`). + + This system call traps if trying to transfer more cycles than are in the current balance of the canister. + +- `ic0.call_cycles_add128 : (amount_high : i64, amount_low : i64) → ()` + + This function moves cycles from the canister balance onto the call under construction, to be transferred with that call. + + The amount of cycles it moves is represented by a 128-bit value which can be obtained by combining the `amount_high` and `amount_low` parameters. + + The cycles are deducted from the balance as shown by `ic0.canister_cycles_balance128` immediately, and moved back if the call cannot be performed (e.g. if `ic0.call_perform` signals an error, or if the canister invokes `ic0.call_new` or returns without calling `ic0.call_perform`). + + This traps if trying to transfer more cycles than are in the current balance of the canister. + + This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +- `ic0.msg_cycles_refunded : () → i64` + + This function can only be used in a callback handler (reply or reject), and indicates the amount of cycles that came back with the response as a refund. The refund has already been added to the canister balance automatically. + +:::note +This call traps if the amount of cycles refunded does not fit into a 64-bit value. In general, it is recommended to use `ic0.msg_cycles_refunded128` instead. +::: + +- `ic0.msg_cycles_refunded128 : (dst : i32) → ()` + + This function can only be used in a callback handler (reply or reject), and indicates the amount of cycles that came back with the response as a refund. The refund has already been added to the canister balance automatically. + + This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +### Stable memory {#system-api-stable-memory} + +Canisters have the ability to store and retrieve data from a secondary memory. The purpose of this *stable memory* is to provide space to store data beyond upgrades. The interface mirrors roughly the memory-related instructions of WebAssembly, and tries to be forward compatible with exposing this feature as an additional memory. + +The stable memory is initially empty. + +- `ic0.stable_size : () → (page_count : i32)` + + returns the current size of the stable memory in WebAssembly pages. (One WebAssembly page is 64KiB) + + This system call traps if the size of the stable memory exceeds 2^32 bytes. + +- `ic0.stable_grow : (new_pages : i32) → (old_page_count : i32)` + + tries to grow the memory by `new_pages` many pages containing zeroes. + + This system call traps if the *previous* size of the memory exceeds 2^32 bytes. + + If the *new* size of the memory exceeds 2^32 bytes or growing is unsuccessful, then it returns `-1`. + + Otherwise, it grows the memory and returns the *previous* size of the memory in pages. + +- `ic0.stable_write : (offset : i32, src : i32, size : i32) → ()` + + copies the data referred to by `src`/`size` out of the canister and replaces the corresponding segment starting at `offset` in the stable memory. + + This system call traps if the size of the stable memory exceeds 2^32 bytes. + + It also traps if `src+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. + +- `ic0.stable_read : (dst : i32, offset : i32, size : i32) → ()` + + copies the data referred to by `offset`/`size` out of the stable memory and replaces the corresponding bytes starting at `dest` in the canister memory. + + This system call traps if the size of the stable memory exceeds 2^32 bytes. + + It also traps if `dst+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory + +- `ic0.stable64_size : () → (page_count : i64)` + + returns the current size of the stable memory in WebAssembly pages. (One WebAssembly page is 64KiB) + + This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +- `ic0.stable64_grow : (new_pages : i64) → (old_page_count : i64)` + + tries to grow the memory by `new_pages` many pages containing zeroes. + + If successful, returns the *previous* size of the memory (in pages). Otherwise, returns `-1`. + + This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +- `ic0.stable64_write : (offset : i64, src : i64, size : i64) → ()` + + Copies the data from location \[src, src+size) of the canister memory to location \[offset, offset+size) in the stable memory. + + This system call traps if `src+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. + + This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +- `ic0.stable64_read : (dst : i64, offset : i64, size : i64) → ()` + + Copies the data from location \[offset, offset+size) of the stable memory to the location \[dst, dst+size) in the canister memory. + + This system call traps if `dst+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. + + This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +### System time {#system-api-time} + +The canister can query the IC for the current time. + +`ic0.time : () -> i64` + +The time is given as nanoseconds since 1970-01-01. The IC guarantees that + +- the time, as observed by the canister, is monotonically increasing, even across canister upgrades. + +- within an invocation of one entry point, the time is constant. + +The times observed by different canisters are unrelated, and calls from one canister to another may appear to travel "backwards in time". + +:::note +While an implementation will likely try to keep the time returned by `ic0.time` close to the real time, this is not formally part of this specification. +::: + +### Performance counter {#system-api-performance-counter} + +The canister can query the \"performance counter\", which is a deterministic monotonically increasing integer approximating the amount of work the canister has done since the beginning of the current execution. + +`ic0.performance_counter : (counter_type : i32) -> i64` + +The argument `type` decides which performance counter to return: + +- 0 : instruction counter. The number of WebAssembly instructions the system has determined that the canister has executed. + +In the future, we might expose more performance counters. + +The system resets the counter at the beginning of each [Entry points](#entry-points) invocation. + +The main purpose of this counter is to facilitate in-canister performance profiling. + +### Certified data {#system-api-certified-data} + +For each canister, the IC keeps track of "certified data", a canister-defined blob. For fresh canisters, this blob is the empty blob (`""`). + +- `ic0.certified_data_set : (src: i32, size : i32) -> ()` + + The canister can update the certified data with this call. The passed data must be no larger than 32 bytes. This can be used any number of times. + +When executing a query method via a query call (i.e. in non-replicated state), the canister can fetch a certificate that authenticates to third parties the value last set via `ic0.certified_data_set`. + +- `ic0.data_certificate_present : () -> i32` + + returns `1` if a certificate is present, and `0` otherwise. + + This will return `1` when called from a query method when invoked via a query call. + + This will return `0` if the query method is executed within replicated execution (e.g. when invoked via an update call or inter-canister call). + +- ic0.data_certificate_size : () -> i32 + ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> () + + Copies the certificate for the current value of the certified data to the canister. + + The certificate is a blob as described in [Certification](#certification) that contains the values at path `/canister//certified_data` and at path `/time` of [The system state tree](#the-system-state-tree). + + If this `certificate` includes subnet delegations (possibly nested), then the id of the current canister will be included in each delegation's canister id range. + + This traps if `ic0.data_certificate_present()` returns `0`. + +### Debugging aids {#debugging_aids} + +In a local canister execution environment, the canister needs a way to emit textual trace messages. On the "real" network, these do not do anything. + +- `ic0.debug_print : (src : i32, size : i32) -> ()` + + When executing in an environment that supports debugging, this copies out the data specified by `src` and `size`, and logs, prints or stores it in an environment-appropriate way. The copied data may likely be a valid string in UTF8-encoding, but the environment should be prepared to handle binary data (e.g. by printing it in escaped form). The data does typically not include a terminating `\0` or `\n`. + + Semantically, this function is always a no-op, and never traps, even if the `src+size` exceeds the size of the memory, or if this function is executed from `(start)`. If the environment cannot perform the print, it just skips it. + +Similarly, the System API allows the canister to effectively trap, but give some indication about why it trapped: + +- `ic0.trap : (src : i32, size : i32) -> ()` + + This function always traps. + + The environment may copy out the data specified by `src` and `size`, and log, print or store it in an environment-appropriate way, or include it in system-generated reject messages where appropriate. The copied data may likely be a valid string in UTF8-encoding, but the environment should be prepared to handle binary data (e.g. by printing it in escaped form or substituting invalid characters). + +### Outlook: Using Host References {#host-references} + +The Internet Computer aims to make the most of the WebAssembly platform, and embraces WebAssembly features. With WebAssembly host references, we can make the platform more secure, the interfaces more abstract and more compositional. The above `ic0` System API does not yet use WebAssembly host references. Once they become available on our platform, a new version of the System API using host references will be available via the `ic` module. The changes will be, at least + +1. The introduction of a `api_nonce` reference, which models the capability to use the System API. It is passed as an argument to `canister_init`, `canister_update ` etc., and expected as an argument by almost all System API function calls. (The debugging aids remain unconstrained.) + +2. The use of references, instead of binary blobs, to address principals (user ids, canister ids), e.g. in `ic0.msg_caller` or in `ic0.call_new`. Additional functions will be provided to convert between the transparent binary representation of principals and references. + +3. Making the builder interface to create calls build calls identified by a reference, rather than having an implicit partial call in the background. + +A canister may only use the old *or* the new interface; the IC detects which interface the canister intends to use based on the names and types of its function imports and exports. + +## The IC management canister {#ic-management-canister} + +The interfaces above provide the fundamental ability for external users and canisters to contact other canisters. But the Internet Computer provides additional functionality, such as canister and user management. This functionality is exposed to external users and canisters via the *IC management canister*. + +:::note +The *IC management canister* is just a facade; it does not actually exist as a canister (with isolated state, Wasm code, etc.). +::: + +The IC management canister address is `aaaaa-aa` (i.e. the empty blob). + +It is possible to use the management canister via external requests (a.k.a. ingress messages). The cost of processing that request is charged to the canister that is being managed. Most methods only permit the controllers to call them. Calls to `raw_rand` and `deposit_cycles` are never accepted as ingress messages. + +### Interface overview {#ic-candid} + +The [interface description](_attachments/ic.did) below, in [Candid syntax](https://github.com/dfinity/candid/blob/master/spec/Candid.md), describes the available functionality. + +``` candid name= ic-interface file file=_attachments/ic.did +``` + +The binary encoding of arguments and results are as per Candid specification. + +### IC method `create_canister` {#ic-create_canister} + +Before deploying a canister, the administrator of the canister first has to register it with the IC, to get a canister id (with an empty canister behind it), and then separately install the code. + +The optional `settings` parameter can be used to set the following settings: + +- `controllers` (`vec principal`) + + A list of principals. Must be between 0 and 10 in size. This value is assigned to the *controllers* attribute of the canister. + + Default value: A list containing only the caller of the `create_canister` call. + +- `compute_allocation` (`nat`) + + Must be a number between 0 and 100, inclusively. It indicates how much compute power should be guaranteed to this canister, expressed as a percentage of the maximum compute power that a single canister can allocate. If the IC cannot provide the requested allocation, for example because it is oversubscribed, the call will be rejected. + + Default value: 0 + +- `memory_allocation` (`nat`) + + Must be a number between 0 and 2^48^ (i.e 256TB), inclusively. It indicates how much memory the canister is allowed to use in total. Any attempt to grow memory usage beyond this allocation will fail. If the IC cannot provide the requested allocation, for example because it is oversubscribed, the call will be rejected. If set to 0, then memory growth of the canister will be best-effort and subject to the available memory on the IC. + + Default value: 0 + +Until code is installed, the canister is `Empty` and behaves like a canister that has no public methods. + +- `freezing_threshold` (`nat`) + + Must be a number between 0 and 2^64^-1, inclusively, and indicates a length of time in seconds. + + A canister is considered frozen whenever the IC estimates that the canister would be depleted of cycles before `freezing_threshold` seconds pass, given the canister's current size and the IC's current cost for storage. + + Calls to a frozen canister will be rejected (like for a stopping canister). Additionally, a canister cannot perform calls if that would, due the cost of the call and transferred cycles, would push the balance into frozen territory; these calls fail with `ic0.call_perform` returning a non-zero error code. + + Default value: 2592000 (approximately 30 days). + +### IC method `update_settings` {#ic-update_settings} + +Only *controllers* of the canister can update settings. See [IC method ](#ic-create_canister) for a description of settings. + +Not including a setting in the `settings` record means not changing that field. The defaults described above are only relevant during canister creation. + +### IC method `install_code` {#ic-install_code} + +This method installs code into a canister. + +Only controllers of the canister can install code. + +- If `mode = install`, the canister must be empty before. This will instantiate the canister module and invoke its `canister_init` method (if present), as explained in Section "[Canister initialization](#system-api-init)", passing the `arg` to the canister. + +- If `mode = reinstall`, if the canister was not empty, its existing code and state is removed before proceeding as for `mode = install`. + + Note that this is different from `uninstall_code` followed by `install_code`, as that will forcibly reject all calls awaiting a response. + +- If `mode = upgrade`, this will perform an upgrade of a non-empty canister as described in [Canister upgrades](#system-api-upgrades), passing `arg` to the `canister_post_upgrade` method of the new instance. + +This is atomic: If the response to this request is a `reject`, then this call had no effect. + +:::note +Some canisters may not be able to make sense of callbacks after upgrades; these should be stopped first, to wait for all outstanding callbacks that are not marked as deleted, or be uninstalled first, to prevent outstanding callbacks from being invoked (by marking the corresponding call contexts as deleted). It is expected that the canister admin (or their tooling) does that separately. +::: + +The `wasm_module` field specifies the canister module to be installed. The system supports multiple encodings of the `wasm_module` field: + +- If the `wasm_module` starts with byte sequence `[0x00, 'a', 's', 'm']`, the system parses `wasm_module` as a raw WebAssembly binary. + +- If the `wasm_module` starts with byte sequence `[0x1f, 0x8b, 0x08]`, the system decompresses the contents of `wasm_module` as a gzip stream according to [RFC-1952](https://datatracker.ietf.org/doc/html/rfc1952.html) and then parses the output as a WebAssembly binary. + +### IC method `uninstall_code` {#ic-uninstall_code} + +This method removes a canister's code and state, making the canister *empty* again. + +Only controllers of the canister can uninstall code. + +Uninstalling a canister's code will reject all calls that the canister has not yet responded to, and drop the canister's code and state. Outstanding responses to the canister will not be processed, even if they arrive after code has been installed again. + +The canister is now [empty](#canister-lifecycle). In particular, any incoming or queued calls will be rejected. + +A canister after *uninstalling* retains its *cycles* balance, *controllers*, status, and allocations. + +### IC method `canister_status` {#ic-canister_status} + +Indicates various information about the canister. It contains: + +- The status of the canister. It could be one of `running`, `stopping` or `stopped`. + +- A SHA256 hash of the module installed on the canister. This is `null` if the canister is empty. + +- The controllers of the canister. + +- The memory size taken by the canister. + +- The cycle balance of the canister. + +Only the controllers of the canister can request its status. + +### IC method `stop_canister` {#ic-stop_canister} + +The controllers of a canister may stop a canister (e.g., to prepare for a canister upgrade). + +Stopping a canister is not an atomic action. The immediate effect is that the status of the canister is changed to `stopping` (unless the canister is already stopped). The IC will reject all calls to a stopping canister, indicating that the canister is stopping. Responses to a stopping canister are processed as usual. When all outstanding responses to call contexts that are not marked as deleted have been processed (so there are no open call contexts that are not marked as deleted), the canister status is changed to `stopped` and the management canister responds to the caller of the `stop_canister` request. + +### IC method `start_canister` {#ic-start_canister} + +A canister may be started by its controllers. + +If the canister status was `stopped` or `stopping` then the canister status is simply set to `running`. In the latter case all `stop_canister` calls which are processing fail (and are rejected). + +If the canister was already `running` then the status stays unchanged. + +### IC method `delete_canister` {#ic-delete_canister} + +This method deletes a canister from the IC. + +Only controllers of the canister can delete it and the canister must already be stopped. Deleting a canister cannot be undone, any state stored on the canister is permanently deleted and its cycles are discarded. Once a canister is deleted, its ID cannot be reused. + +### IC method `deposit_cycles` {#ic-deposit_cycles} + +This method deposits the cycles included in this call into the specified canister. + +There is no restriction on who can invoke this method. + +### IC method `raw_rand` {#ic-raw_rand} + +This method takes no input and returns 32 pseudo-random bytes to the caller. The return value is unknown to any part of the IC at time of the submission of this call. A new return value is generated for each call to this method. + +### IC method `ecdsa_public_key` {#ic-ecdsa_public_key} + +:::note +The ECDSA API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. +::: + +This method returns a [SEC1](https://www.secg.org/sec1-v2.pdf) encoded ECDSA public key for the given canister using the given derivation path. If the `canister_id` is unspecified, it will default to the canister id of the caller. The `derivation_path` is a vector of variable length byte strings. The `key_id` is a struct specifying both a curve and a name. The availability of a particular `key_id` depends on implementation. + +For curve `secp256k1`, the public key is derived using a generalization of BIP32 (see [ia.cr/2021/1330, Appendix D](https://ia.cr/2021/1330)). To derive (non-hardened) [BIP-0032](https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki)-compatible public keys, each byte string (`blob`) in the `derivation_path` must be a 4-byte big-endian encoding of an unsigned integer less than 2^31. + +The return result is an extended public key consisting of an ECDSA `public_key`, encoded in [SEC1](https://www.secg.org/sec2-v2.pdf) compressed form, and a `chain_code`, which can be used to deterministically derive child keys of the `public_key`. + +This call requires that the ECDSA feature is enabled, and the `canister_id` meets the requirement of a canister id. Otherwise it will be rejected. + +### IC method `sign_with_ecdsa` {#ic-sign_with_ecdsa} + +:::note +The ECDSA API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. +::: + +This method returns a new [ECDSA](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf) signature of the given `message_hash` that can be separately verified against a derived ECDSA public key. This public key can be obtained by calling `ecdsa_public_key` with the caller's `canister_id`, and the same `derivation_path` and `key_id` used here. + +The signatures are encoded as the concatenation of the [SEC1](https://www.secg.org/sec2-v2.pdf) encodings of the two values r and s. For curve `secp256k1`, this corresponds to 32-byte big-endian encoding. + +This call requires that the ECDSA feature is enabled, the caller is a canister, and `message_hash` is 32 bytes long. Otherwise it will be rejected. + + +### IC method `http_request` {#ic-http_request} + +:::note +The IC http_request API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. +::: + +This method makes an HTTP request to a given URL and returns the HTTP response, possibly after a transformation. + +The canister should aim to issue *idempotent* requests, meaning that it must not change the state at the remote server, or the remote server has the means to identify duplicated requests. Otherwise, the risk of failure increases. + +The responses for all identical requests must match too. However, a web service could return slightly different responses for identical idempotent requests. For example, it may include some unique identification or a timestamp that would vary across responses. + +For this reason, the calling canister can supply a transformation function, which the IC uses to let the canister sanitize the responses from such unique values. The transformation function is executed separately on the corresponding response received for a request. The final response will only be available to the calling canister. + +Currently, the `GET`, `HEAD`, and `POST` methods are supported for HTTP requests. + +It is important to note the following for the usage of the `POST` method: - The calling canister must make sure that the remote server is able to handle idempotent requests sent from multiple sources. This may require, for example, to set a certain request header to uniquely identify the request. - There are no confidentiality guarantees on the request content. There is no guarantee that all sent requests are as specified by the canister. If the canister receives a response, then at least one request that was sent matched the canister's request, and the response was to that request. + +For security reasons, only HTTPS connections are allowed (URLs must start with `https://`). The IC uses industry-standard root CA lists to validate certificates of remote web servers. + +Each request can specify the maximal expected size for the response from the remote HTTP server. The upper limit on the size of a response defaults to `2MiB` if no maximal size value is specified. An error will be returned when the response is larger than the maximal size. The `2MiB` size limit also applies to the value returned by the `transform` function. + +The following parameters should be supplied for the call: + +- `url` - the requested URL + +- `max_response_bytes` - optional, specifies the maximal size of the response in bytes. Any value less than or equal to `2MiB` is accepted. The call will be charged based on this parameter. If not provided, the maximum of `2MiB` will be used. + +- `method` - currently, only GET, HEAD, and POST are supported + +- `headers` - list of HTTP request headers and their corresponding values + +- `transform` - an optional function that transforms raw responses to sanitized responses. If provided, the calling canister itself must export this function. + +The returned response (and the response provided to the `transform` function, if specified) contains the following fields: + +- `status` - the response status (e.g., 200, 404) + +- `headers` - list of HTTP response headers and their corresponding values + +- `body` - the response's body + +The `transform` function may, for example, transform the body in any way, add or remove headers, modify headers, etc. When the transform function was invoked due to a canister HTTP request, the caller's identity is the principal of the management canister. + +### IC method `provisional_create_canister_with_cycles` {#ic-provisional_create_canister_with_cycles} + +As a provisional method on development instances, the `provisional_create_canister_with_cycles` method is provided. It behaves as `create_canister`, but initializes the canister's balance with `amount` fresh cycles (using `DEFAULT_PROVISIONAL_CYCLES_BALANCE` if `amount = null`). + +Cycles added to this call via `ic0.call_cycles_add128` are returned to the caller. + +This method is only available in local development instances. + +### IC method `provisional_top_up_canister` {#ic-provisional_top_up_canister} + +As a provisional method on development instances, the `provisional_top_up_canister` method is provided. It adds `amount` cycles to the balance of canister identified by `amount`. + +Cycles added to this call via `ic0.call_cycles_add128` are returned to the caller. + +Any user can top-up any canister this way. + +This method is only available in local development instances. + +## The IC Bitcoin API {#ic-bitcoin-api} + +:::note +The IC Bitcoin API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. +::: + +The Bitcoin functionality is exposed via the management canister. Information about Bitcoin can be found in the [Bitcoin developer guides](https://developer.bitcoin.org/devguide/). Invoking the functions of the Bitcoin API will cost cycles. The concrete costs for each function are yet to be determined. + +### IC method `bitcoin_get_utxos` {#ic-bitcoin_get_utxos} + +Given a `get_utxos_request`, which must specify a Bitcoin address and a Bitcoin network (`mainnet` or `testnet`), the function returns all unspent transaction outputs (UTXOs) associated with the provided address in the specified Bitcoin network based on the current view of the Bitcoin blockchain available to the Bitcoin component. The UTXOs are returned sorted by block height in descending order. + +The following address formats are supported: + +- Pay to public key hash (P2PKH) + +- Pay to script hash (P2SH) + +- Pay to witness public key hash (P2WPKH) + +- Pay to witness script hash (P2WSH) + +- Pay to taproot (P2TR) + +If the address is malformed, the call is rejected. + +The optional `filter` parameter can be used to restrict the set of returned UTXOs, either providing a minimum number of confirmations or a page reference when pagination is used for addresses with many UTXOs. In the first case, only UTXOs with at least the provided number of confirmations are returned, i.e., transactions with fewer than this number of confirmations are not considered. In other words, if the number of confirmations is `c`, an output is returned if it occurred in a transaction with at least `c` confirmations and there is no transaction that spends the same output with at least `c` confirmations. + +There is an upper bound of 144 on the minimum number of confirmations. If a larger minimum number of confirmations is specified, the call is rejected. Note that this is not a severe restriction as the minimum number of confirmations is typically set to a value around 6 in practice. + +It is important to note that the validity of transactions is not verified in the Bitcoin component. The Bitcoin component relies on the proof of work that goes into the blocks and the verification of the blocks in the Bitcoin network. For a newly discovered block, a regular Bitcoin (full) node therefore provides a higher level of security than the Bitcoin component, which implies that it is advisable to set the number of confirmations to a reasonably large value, such as 6, to gain confidence in the correctness of the returned UTXOs. + +There is an upper bound of 100,000 UTXOs that can be returned in a single request. For addresses that contain sufficiently large UTXOs, the partial set of the address\' UTXOs are returned along with a page reference. + +In the second case, a page reference (a series of bytes) must be provided, which instructs the Bitcoin component to collect UTXOs starting from the corresponding \"page\". + +A `get_utxos_request` without the optional `filter` results in a request that considers the full blockchain, which is equivalent to setting `min_confirmations` to 0. + +The recommended workflow is to issue a request with the desired number of confirmations. If the `next_page` field in the response is not empty, there are more UTXOs than in the returned vector. In that case, the `page` field should be set to the `next_page` bytes in the subsequent request to obtain the next batch of UTXOs. + +### IC method `bitcoin_get_balance` {#ic-bitcoin_get_balance} + +Given a `get_balance_request`, which must specify a Bitcoin address and a Bitcoin network (`mainnet` or `testnet`), the function returns the current balance of this address in `Satoshi` (10\^8 Satoshi = 1 Bitcoin) in the specified Bitcoin network. The same address formats as for [`bitcoin_get_utxos`](#ic-bitcoin_get_utxos) are supported. + +If the address is malformed, the call is rejected. + +The optional `min_confirmations` parameter can be used to limit the set of considered UTXOs for the calculation of the balance to those with at least the provided number of confirmations in the same manner as for the [`bitcoin_get_utxos`](#ic-bitcoin_get_utxos) call. + +Given an address and the optional `min_confirmations` parameter, `bitcoin_get_balance` iterates over all UTXOs, i.e., the same balance is returned as when calling [`bitcoin_get_utxos`](#ic-bitcoin_get_utxos) for the same address and the same number of confirmations and, if necessary, using pagination to get all UTXOs for the same tip hash. + +### IC method `bitcoin_send_transaction` {#ic-bitcoin_send_transaction} + +Given a `send_transaction_request`, which must specify a `blob` of a Bitcoin transaction and a Bitcoin network (`mainnet` or `testnet`), several checks are performed: + +- The transaction is well formed. + +- The transaction only consumes unspent outputs with respect to the current (longest) blockchain, i.e., there is no block on the (longest) chain that consumes any of these outputs. + +- There is a positive transaction fee. + +If at least one of these checks fails, the call is rejected. + +If the transaction passes these tests, the transaction is forwarded to the specified Bitcoin network. + +The Bitcoin component periodically forwards the transaction until the transaction appears in a block appended to the blockchain or the transaction times out after 24 hours. As soon as it appears in a block or expires, the Bitcoin component drops the transaction. It follows that the function does not provide any guarantees that a submitted transaction will ever appear in a block. + +### IC method `bitcoin_get_current_fee_percentiles` {#ic-bitcoin_get_current_fee_percentiles} + +The transaction fees in the Bitcoin network change dynamically based on the number of pending transactions. It must be possible for a canister to determine an adequate fee when creating a Bitcoin transaction. + +This function returns the 100 fee percentiles, measured in millisatoshi/vbyte (1000 millisatoshi = 1 satoshi), over the last 10,000 transactions in the specified network, i.e., over the transactions in the last approximately 4-10 blocks. + +## Certification + +Some parts of the IC state are exposed to users in a tamperproof way via certification: the IC can reveal a *partial state tree* which includes just the data of interest, together with a signature on the root hash of the state tree. This means that a user can be sure that the response is correct, even if the user happens to be communicating with a malicious node, or has received the certificate via some other untrusted way. + +To validate a value using a certificate, the user conceptually + +1. checks the validity of the partial tree using `verify_cert`, + +2. looks up the value in the certificate using `lookup` at a given path, which uses the subroutine `lookup_path` on the certificate's tree + +This mechanism is used in the `read_state` request type, and eventually also for other purposes. + +### Root of trust {#root_of_trust} + +The root of trust is the *root public key*, which must be known to the user a priori. In a local canister execution environment, the key can be fetched via the [`/api/v2/status`](#api-status) endpoint. + +### Certificate {#certificate} + +A certificate consists of + +- a tree + +- a signature on the tree root hash valid under some *public key* + +- an optional *delegation* that links that public key to *root public key*. + +The IC will certify states by issuing certificates where the tree is a partial state tree. The state tree can be pruned by replacing subtrees with their root hashes (yielding a new and potentially smaller but still valid certificate) to only include paths pertaining to relevant data but still preserving enough information to recover the *tree root hash*. + +More formally, a certificate is described by the following data structure: + + Certificate = { + tree : HashTree + signature : Signature + delegation : NoDelegation | Delegation + } + HashTree + = Empty + | Fork HashTree HashTree + | Labeled Label HashTree + | Leaf blob + | Pruned Hash + Label = Blob + Hash = Blob + Signature = Blob + +A certificate is validated with regard to the root of trust by the following algorithm (which uses `check_delegation` defined in [Delegation](#certification-delegation)): + + verify_cert(cert) = + let root_hash = reconstruct(cert.tree) + let der_key = check_delegation(cert.delegation) // see section Delegations below + bls_key = extract_der(der_key) + verify_bls_signature(bls_key, cert.signature, domain_sep("ic-state-root") · root_hash) + + reconstruct(Empty) = H(domain_sep("ic-hashtree-empty")) + reconstruct(Fork t1 t2) = H(domain_sep("ic-hashtree-fork") · reconstruct(t1) · reconstruct(t2)) + reconstruct(Labeled l t) = H(domain_sep("ic-hashtree-labeled") · l · reconstruct(t)) + reconstruct(Leaf v) = H(domain_sep("ic-hashtree-leaf") · v) + reconstruct(Pruned h) = h + + domain_sep(s) = byte(|s|) · s + +where `H` is the SHA-256 hash function, + + verify_bls_signature : PublicKey -> Signature -> Blob -> Bool + +is the [BLS signature verification function](https://tools.ietf.org/html/draft-irtf-cfrg-bls-signature-04#section-4), ciphersuite BLS_SIG_BLS12381G1_XMD:SHA-256_SSWU_RO_NUL\_. See that document also for details on the encoding of BLS public keys and signatures, and + + extract_der : Blob -> Blob + +implements DER decoding of the public key, following [RFC4580](https://tools.ietf.org/html/rfc5480) using OID 1.3.6.1.4.1.44668.5.3.1.2.1 for the algorithm and 1.3.6.1.4.1.44668.5.3.2.1 for the curve. + +All state trees include the time at path `/time` (see [Time](#state-tree-time)). Users that get a certificate with a state tree can look up the timestamp to guard against working on obsolete data. + +### Lookup {#lookup} + +Given a (verified) tree, the user can fetch the value at a given path, which is a sequence of labels (blobs). In this document, we write paths suggestively with slashes as separators; the actual encoding is not actually using slashes as delimiters. + +The following algorithm looks up a `path` in a certificate, and returns either + +- the value + +- `Absent`, if the value is guaranteed to be absent in the original state tree, + +- `Unknown`, if this partial view does not include information about this path, or + +- `Error`, if the path does not make sense for this certificate: + +```html +lookup(path, cert) = lookup_path(path, cert.tree) + +lookup_path([], Empty) = Absent +lookup_path([], Leaf v) = v +lookup_path([], Pruned _) = Unknown +lookup_path([], Labeled _ _) = Error +lookup_path([], Fork _ _) = Error + +lookup_path(l::ls, tree) = + match find_label(l, flatten_forks(tree)) with + | Absent -> Absent + | Unknown -> Unknown + | Error -> Error + | Found subtree -> lookup_path ls subtree + +flatten_forks(Empty) = [] +flatten_forks(Fork t1 t2) = flatten_forks(t1) · flatten_forks(t2) +flatten_forks(t) = [t] + +find_label(l, _ · Labeled l1 t · _) | l == l1 = Found t +find_label(l, _ · Labeled l1 _ · Labeled l2 _ · _) | l1 < l < l2 = Absent +find_label(l, Labeled l2 _ · _) | l < l2 = Absent +find_label(l, _ · Labeled l1 _ ) | l1 < l = Absent +find_label(l, [Leaf _]) = Absent +find_label(l, []) = Absent +find_label(l, _) = Unknown +``` + + +The IC will only produce well-formed state trees, and the above algorithm assumes well-formed trees. These have the property that labeled subtrees appear in strictly increasing order of labels, and are not mixed with leaves. More formally: + + well_formed(tree) = + (tree = Leaf _) ∨ (well_formed_forest(flatten_forks(tree))) + + well_formed_forest(trees) = + strictly_increasing([l | Label l _ ∈ trees]) ∧ + ∀ Label _ t ∈ trees. well_formed(t) ∧ + ∀ t ∈ trees. t ≠ Leaf _ + +### Delegation {#certification-delegation} + +The root key can delegate certification authority to other keys. + +A certificate by the root subnet does not have a delegation field. A certificate by other subnets include a delegation, which is itself a certificate that proves that the subnet is listed in the root subnet's state tree (see [Subnet information](#state-tree-subnet)), and reveals its public key. + +:::note +The nested certificate *typically* does not itself again contain a delegation, although there is no reason why agents should enforce that property. +::: + + Delegation = + Delegation { + subnet_id : Principal; + certificate : Certificate; + } + +A chain of delegations is verified using the following algorithm, which also returns the delegated key (a DER-encoded BLS key): + + check_delegation(NoDelegation) : public_bls_key = + return root_public_key + check_delegation(Delegation d) : public_bls_key = + verify_cert(d.certificate) + return lookup(["subnet",d.subnet_id,"public_key"],d.certificate) + +where `root_public_key` is the a priori known root key. + +Delegations are *scoped*, i.e., they indicate which set of canister principals the delegatee subnet may certify for. This set can be obtained from a delegation `d` using `lookup(["subnet",d.subnet_id,"canister_ranges"],d.certificate)`, which must be present, and is encoded as described in [Subnet information](#state-tree-subnet). The various applications of certificates describe if and how the subnet scope comes into play. + +### Encoding of certificates {#certification-encoding} + +The binary encoding of a certificate is a CBOR value according to the following CDDL. You can also [download the file](_attachments/certificates.cddl). + +The values in the [The system state tree](#the-system-state-tree) are encoded to blobs as follows: + +- natural numbers are leb128-encoded. + +- text values are UTF-8-encoded + +- blob values are encoded as is + +### Example {#example} + +Consider the following tree-shaped data (all single character strings denote labels, all other denote values) + + ─┬╴ "a" ─┬─ "x" ─╴"hello" + │ └╴ "y" ─╴"world" + ├╴ "b" ──╴ "good" + ├╴ "c" + └╴ "d" ──╴ "morning" + +A possible hash tree for this labeled tree might be, where `┬` denotes a fork. This is not a typical encoding (a fork with `Empty` on one side can be avoided), but it is valid. + + ─┬─┬╴"a" ─┬─┬╴"x" ─╴"hello" + │ │ │ └╴Empty + │ │ └╴ "y" ─╴"world" + │ └╴"b" ──╴"good" + └─┬╴"c" ──╴Empty + └╴"d" ──╴"morning" + +This tree has the following CBOR encoding + + 8301830183024161830183018302417882034568656c6c6f810083024179820345776f726c6483024162820344676f6f648301830241638100830241648203476d6f726e696e67 + +and the following root hash + + eb5c5b2195e62d996b84c9bcc8259d19a83786a2f59e0878cec84c811f669aa0 + +Pruning this tree with the following paths + + /a/y + /ax + /d + +would lead to this tree (with pruned subtree represented by their hash): + + ─┬─┬╴"a" ─┬─ 1B4FEFF9BEF8131788B0C9DC6DBAD6E81E524249C879E9F10F71CE3749F5A638 + │ │ └╴ "y" ─╴"world" + │ └╴"b" ──╴7B32AC0C6BA8CE35AC82C255FC7906F7FC130DAB2A090F80FE12F9C2CAE83BA6 + └─┬╴EC8324B8A1F1AC16BD2E806EDBA78006479C9877FED4EB464A25485465AF601D + └╴"d" ──╴"morning" + +Note that the `"b"` label is included (without content) to prove the absence of the `/ax` path. + +This tree encodes to CBOR as + + 83018301830241618301820458201b4feff9bef8131788b0c9dc6dbad6e81e524249c879e9f10f71ce3749f5a63883024179820345776f726c6483024162820458207b32ac0c6ba8ce35ac82c255fc7906f7fc130dab2a090f80fe12f9c2cae83ba6830182045820ec8324b8a1f1ac16bd2e806edba78006479c9877fed4eb464a25485465af601d830241648203476d6f726e696e67 + +and (obviously) the same root hash. + +In the pruned tree, the `lookup_path` function behaves as follows: + + lookup_path(["a", "a"], pruned_tree) = Unknown + lookup_path(["a", "y"], pruned_tree) = Found "world" + lookup_path(["aa"], pruned_tree) = Absent + lookup_path(["ax"], pruned_tree) = Absent + lookup_path(["b"], pruned_tree) = Unknown + lookup_path(["bb"], pruned_tree) = Unknown + lookup_path(["d"], pruned_tree) = Found "morning" + lookup_path(["e"], pruned_tree) = Absent + + +## The HTTP Gateway protocol {#http-gateway} + +This section specifies the *HTTP Gateway protocol*, which allows canisters to handle conventional HTTP requests. + +This feature involves the help of a *HTTP Gateway* that translates between HTTP requests and the IC protocol. Such a gateway could be a stand-alone proxy, it could be implemented in a web browsers (natively, via plugin or via a service worker) or in other ways. This document describes the interface and semantics of this protocol independent of a concrete Gateway, so that all Gateway implementations can be compatible. + +Conceptually, this protocol builds on top of the interface specified in the remainder of this document, and therefore is an "application-level" interface, not a feature of the core Internet Computer system described in the other sections, and could be a separate document. We nevertheless include this protocol in the Internet Computer Interface Specification because of its important role in the ecosystem and due to the importance of keeping multiple Gateway implementations in sync. + +### Overview {#_overview} + +A HTTP request by an HTTP client is handled by these steps: + +1. The Gateway resolves the Host of the request to a canister id. + +2. The Gateway Candid-encodes the HTTP request data. + +3. The Gateway invokes the canister via a query call to `http_request`. + +4. The canister handles the request and returns a HTTP response, encoded in Candid, together with additional metadata. + +5. If requested by the canister, the Gateway sends the request again via an update call to `http_request_update`. + +6. If applicable, the Gateway fetches further body data via streaming query calls. + +7. If applicable, the Gateway validates the certificate of the response. + +8. The Gateway sends the response to the HTTP client. + +### Candid interface {#http-gateway-interface} + +The following interface description, in [Candid syntax](https://github.com/dfinity/candid/blob/master/spec/Candid.md), describes the expected Canister interface. You can also [download the file](_attachments/http-gateway.did). + +``` candid name= ic-interface file file=_attachments/http-gateway.did +``` + + +Only canisters that use the "Upgrade to update calls" feature need to provide the `http_request_update` method. + +::: note +Canisters not using these features can completely leave out the `streaming_strategy` and/or `upgrade` fields in the `HttpResponse` they return, due to how Candid subtyping works. This might simplify their code. +::: + +### Canister resolution {#http-gateway-name-resolution} + +The Gateway needs to know the canister id of the canister to talk to, and obtains that information from the hostname as follows: + +1. Check that the hostname, taken from the `Host` field of the HTTP request, is of the form `.raw.ic0.app` or `.ic0.app`, or fail. + +2. If the `` is in the following table, use the given canister ids: + +|Canister hostname resolution +|-------------------------------------------- +| Hostname | Canister id +| `identity` | `rdmx6-jaaaa-aaaaa-aaadq-cai` +| `nns` | `qoctq-giaaa-aaaaa-aaaea-cai` +| `dscvr` | `h5aet-waaaa-aaaab-qaamq-cai` +| `personhood` | `g3wsl-eqaaa-aaaan-aaaaa-cai` +|-------------------------------------------- + +3. Else, if `` is a valid textual encoding of a principal, use that principal as the canister id. + +4. Else fail. + +If the hostname was of the form `.ic0.app`, it is a _safe_ hostname; if it was of the form `.raw.ic0.app` it is a _raw_ hostname. + +### Request encoding + +The HTTP request is encoded into the `HttpRequest` Candid structure. + +* The `method` field contains the HTTP method (e.g. `HTTP`), in upper case. + +* The `url` field contains the URL from the HTTP request line, i.e. without protocol or hostname, and including query parameters. + +* The `headers` field contains the headers of the HTTP request. + +* The `body` field contains the body of the HTTP request (without any content encodings processed by the Gateway). + +### Upgrade to update calls + +If the canister sets `upgrade = opt true` in the `HttpResponse` reply from `http_request`, then the Gateway ignores all other fields of the reply. The Gateway performs an *update* call to `http_request_update`, passing the same `HttpRequest` record as the argument, and uses that response instead. + +The value of the `upgrade` field returned from `http_request_update` is ignored. + +### Response decoding + +The Gateway assembles the HTTP response from the given `HttpResponse` record: + +* The HTTP response status code is taken from the `status_code` field. + +* The HTTP response headers are taken from the `headers` field. ++ +NOTE: Not all Gateway implementations may be able to pass on all forms of headers. In particular, Service Workers are unable to pass on the `Set-Cookie` header. ++ +::: note +HTTP Gateways may add additional headers. In particular, the following headers may be set: + access-control-allow-origin: * + access-control-allow-methods: GET, POST, HEAD, OPTIONS + access-control-allow-headers: DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Cookie + access-control-expose-headers: Content-Length,Content-Range + x-cache-status: MISS +::: + +* The HTTP response body is initialized with the value of the `body` field, and further assembled as per the [streaming protocol](#http-gateway-streaming). + + +### Response body streaming {#http-gateway-streaming} + +The HTTP Gateway protocol has provisions to transfer further chunks of the body data from the canister to the HTTP Gateway, to overcome the message limit of the Internet Computer. This streaming protocol is independent of any possible streaming of data between the HTTP Gateway and the HTTP client. The gateway may assemble the response in whole before passing it on, or pass the chunks on directly, on the TCP or HTTP level, as it sees fit. When the Gateway is [certifying the response](#http-gateway-certification), it must not pass on uncertified chunks. + +If the `streaming_strategy` field of the `HttpResponse` is set, the HTTP Gateway then uses further query calls to obtain further chunks to append to the body: + +1. If the function reference in the `callback` field of the `streaming_strategy` is not a method of the given canister, the Gateway fails the request. + +2. Else, it makes a query call to the given method, passing the `token` value given in the `streaming_strategy` as the argument. + +3. That query method returns a `StreamingCallbackHttpResponse`. The `body` therein is appended to the body of the HTTP response. This is repeated as long as the method returns some token in the `token` field, until that field is `null`. + +:::warn +The type of the `token` value is chosen by the canister; the HTTP Gateway obtains the Candid type of the encoded message from the canister, and uses it when passing the token back to the canister. This generic use of Candid is not covered by the Candid specification, and may not be possible in some cases (e.g. when using "future types"). Canister authors may have to use "simple" types. +::: + + +### Response certification {#http-gateway-certification} + +If the hostname was safe, the HTTP Gateway performs *certificate validation*: + +1. It searches for a response header called `Ic-Certificate` (case-insensitive). + +2. The value of the header must be a structured header according to RFC 8941 with fields `certificate` and `tree`, both being byte sequences. + +3. The `certificate` must be a valid certificate as per [Certification](#certification), signed by the root key. If the certificate contains a subnet delegation, the delegation must be valid for the given canister. The timestamp in `/time` must be recent. The subnet state tree in the certificate must reveal the canister's [certified data](#state-tree-certified-data). + +4. The `tree` must be a hash tree as per [Encoding of certificates](#certification-encoding). + +5. The root hash of that `tree` must match the canister's certified data. + +6. The path `["http_assets",]`, where `url` is the utf8-encoded `url` from the `HttpRequest` must exist and be a leaf. Else, if it does not exist, `["http_assets","/index.html"]` must exist and be a leaf. + +7. That leaf must contain the SHA-256 hash of the *decoded* body. +The decoded body is the body of the HTTP response (in particular, after assembling streaming chunks), decoded according to the `Content-Encoding` header, if present. Supported encodings for `Content-Encoding` are `gzip` and `deflate.` + +:::warn +The certification protocol only covers the mapping from request URL to response body. It completely ignores the request method and headers, and does not cover the response headers and status code. +::: + +## Abstract behavior + +The previous sections describe the interfaces, i.e. outer edges of the Internet Computer, but give only intuitive and vague information in prose about what these interfaces actually do. + +The present section aims to address that question with great precision, by describing the *abstract state* of the whole Internet Computer, and how this state can change in response to API function calls, or spontaneously (modeling asynchronous, distributed or non-deterministic execution). + +The design of this abstract specification (e.g. how and where pending messages are stored) are *not* to be understood to in any way prescribe a concrete implementation or software architecture. The goals here are formal precision and clarity, but not implementability, so this can lead to different ways of phrasing. + +### Notation {#notation} + +We specify the behavior of the Internet Computer using ad-hoc pseudocode. + +The manipulated values are primitive values (numbers, text, binary blobs), aggregate values (lists, unordered lists a.k.a. bags, partial maps, records with fixed fields, named constructors) and functions. + +We use a concatenation operator `·` with various types: to extend sets and maps, or to concatenate lists with lists or lists with elements. + +The shape of values is described using a hand-wavy type system. We use `Foo = Nat` to define type aliases; now `Foo` can be used instead of `Nat`. Often, the right-hand side is a more complex type here, e.g. a record, or multiple possible types separated by a vertical bar (`|`). Partial maps are written as `Key ↦ Value` and the function type as `Argument → Result`. + +:::note +All values are immutable! State change is specified by describing the new state, not by changing the existing state. +::: + +Record fields are accessed using dot-notation (e.g. `S.request_id > 0`). To create a new record from an existing record `R` with some fields changed, the syntax `R where field = new_value` is used. This syntax can also be used to create new records with some deeply nested field changed: `R where some_map[key].field = new_value`. + +In the state transitions, upper-case variables (`S`, `C`, `Req_id`) are free variables: The state transition may be taken for any possible value of these variables. `S` always refers to the previous state. A state transition often comes with a list of *conditions*, which may restrict the values of these free variables. The *state after* is usually described using the record update syntax by starting with `S where`. + +For example, the condition `S.messages = Older_messages · M · Younger_messages` says that `M` is some message in field `messages` of the record `S`, and that `Younger_messages` and `Older_messages` are the other messages in the state. If the "state after" specifies `S with messages = Older_messages · Younger_messages`, then the message `M` is removed from the state. + +### Abstract state {#abstract_state} + +In this specification, we describe the Internet Computer as a state machine. In particular, there is a single piece of data that describes the complete state of the IC, called `S`. + +Of course, this is a huge simplification: The real Internet Computer is distributed and has a multi-component architecture, and the state is spread over many different components, some physically separated. But this simplification allows us to have a concise description of the behavior, and to easily make global decisions (such as, "is there any pending message"), without having to specify the bookkeeping that allows such global decisions. + +#### Identifiers {#identifiers} + +Principals (canister ids and user ids) are blobs, but some of them have special form, as explained in [Special forms of Principals](#id-classes). + + type Principal = Blob + +The function + + mk_self_authenticating_id : PublicKey -> Principal + mk_self_authenticating_id pk = H(pk) · 0x02 + +calculates self-authenticating ids. + +The function + + mk_derived_id : Principal -> Blob -> Principal + mk_derived_id p nonce = H(|p| · p · nonce) · 0x03 + +calculates derived ids. With `|p|` we denote the length of the principal, in bytes, encoded as a single byte. + +The principal of the anonymous user is fixed: + + anonymous_id : Principal + anonymous_id = 0x04 + +The principal of the management canister is the empty blob (i.e. `aaaaa-aa`): + + ic_principal : Principal = "" + +These function domains and fixed values are mutually disjoint. + +Method names can be arbitrary pieces of text: + + MethodName = Text + +#### Abstract canisters + +The [WebAssembly System API](#system-api) is relatively low-level, and some of its details (e.g. that the argument data is queried using separate calls, and that closures are represented by a function pointer and a number, that method names need to be mangled) would clutter this section. Therefore, we abstract over the WebAssembly details as follows: + +- The state of a WebAssembly module (memory, tables, globals) is hidden behind an abstract `WasmState`. The `WasmState` contains the `StableMemory`, which can be extracted using `pre_upgrade` and passed to `post_upgrade`. + +- A canister module `CanisterModule` consists of an initial state, and a (pure) function that models function invocation. It either indicates that the canister function traps, or returns a new state together with a description of the invoked asynchronous System API calls. + + WasmState = (abstract) + StableMemory = (abstract) + Callback = (abstract) + + Arg = Blob; + CallerId = Principal; + + Timestamp = Nat; + Env = { + time : Timestamp + balance : Nat; + freezing_limit : Nat; + certificate : NoCertificate | Blob + status : Running | Stopping | Stopped + } + + RejectCode = Nat + Response = Reply Blob | Reject (RejectCode, Text) + MethodCall = { + callee : CanisterId; + method_name: MethodName; + arg: Blob; + transferred_cycles: Nat; + callback: Callback; + } + + UpdateFunc = WasmState -> Trap { cycles_used : Nat; } | Return { + new_state : WasmState; + new_calls : List MethodCall; + new_certified_data : NoCertifiedData | Blob + response : NoResponse | Response; + cycles_accepted : Nat; + cycles_used : Nat; + } + QueryFunc = WasmState -> Trap { cycles_used : Nat; } | Return { + response : Response; + cycles_used : Nat; + } + HeartbeatFunc = WasmState -> Trap { cycles_used : Nat; } | Return { + new_state : WasmState; + cycles_used : Nat; + } + + AvailableCycles = Nat + RefundedCycles = Nat + + CanisterModule = { + init : (CanisterId, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { + new_state : WasmState; + cycles_used : Nat; + } + pre_upgrade : (WasmState, Principal, Env) -> Trap { cycles_used : Nat; } | Return { + stable_memory : StableMemory; + cycles_used : Nat; + } + post_upgrade : (CanisterId, StableMemory, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { + new_state : WasmState; + cycles_used : Nat; + } + update_methods : MethodName ↦ ((Arg, CallerId, Env, AvailableCycles) -> UpdateFunc) + query_methods : MethodName ↦ ((Arg, CallerId, Env) -> QueryFunc) + heartbeat : (Env) -> HeartbeatFunc + callbacks : (Callback, Response, RefundedCycles, Env, AvailableCycles) -> UpdateFunc + inspect_message : (MethodName, WasmState, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { + status : Accept | Reject; + cycles_used : Nat; + } + } + +This high-level interface presents a pure, mathematical model of a canister, and hides the bookkeeping required to provide the System API as seen in Section [Canister interface (System API)](#system-api). + +The `CanisterId` parameter of `init` and `post_upgrade` is merely passed through to the canister, via the `canister.self` system call. + +The `Env` parameter provides synchronous read-only access to portions of the system state and canister metadata that are always available. + +The parsing of a blob to a canister module and its public and private custom sections is modelled via the (possibly implicitly failing) functions + + parse_wasm_mod : Blob -> CanisterModule + parse_public_custom_sections : Blob -> Text ↦ Blob + parse_private_custom_sections : Blob -> Text ↦ Blob + +The concrete mapping of this abstract `CanisterModule` to actual WebAssembly concepts and the System API is described separately in section [Abstract Canisters to System API](#concrete-canisters). + +#### Call contexts {#call_contexts} + +The Internet Computer provides certain messaging guarantees: If a user or a canister calls another canister, it will eventually get a single response (a reply or a rejection), even if some canister code along the way fails. + +To ensure that only one response is generated, and also to detect when no response can be generated any more, the IC maintains a *call context*. The `needs_to_respond` field is set to `false` once the call has received a response. Further attempts to respond will now fail. + + Request = { + nonce : Blob; + ingress_expiry : Nat; + sender : UserId; + canister_id : CanisterId; + method_name : Text; + arg : Blob; + } + CallId = (abstract) + CallOrigin + = FromUser { + request : Request; + } + | FromCanister { + calling_context : CallId; + callback: Callback + } + | FromHeartbeat + CallCtxt = { + canister : CanisterId; + origin : CallOrigin; + needs_to_respond : bool; + deleted : bool; + available_cycles : Nat; + } + +#### Calls and Messages {#calls_and_messages} + +Calls into and within the IC are implemented as messages passed between canisters. During their lifetime, messages change shape: they begin as a call to a public method, which is resolved to a WebAssembly function that is then executed, potentially generating a response which is then delivered. + +Therefore, a message can have different shapes: + + Queue = Unordered | Queue { from : System | CanisterId; to : CanisterId } + EntryPoint + = PublicMethod MethodName Principal Blob + | Callback Callback Response RefundedCycles + | Heartbeat + + Message + = CallMessage { + origin : CallOrigin; + caller : Principal; + callee : CanisterId; + method_name : Text; + arg : Blob; + transferred_cycles : Nat; + queue : Queue; + } + | FuncMessage { + call_context : CallId; + receiver : CanisterId; + entry_point : EntryPoint; + queue : Queue; + } + | ResponseMessage { + origin : CallOrigin; + response : Response; + refunded_cycles : Nat; + } + +The `queue` field is used to describe the message ordering behavior. Its concrete value is only used to determine when the relative order of two messages must be preserved, and is otherwise not interpreted. Response messages are not ordered, as explained above, so they have no `queue` field. + +A reference implementation would likely maintain a separate list of `messages` for each such queue to efficiently find eligible messages; this document uses a single global list for a simpler and more concise system state. + +#### API requests {#api_requests} + +We distinguish between the *asynchronous* API requests (type `Request`) passed to `/api/v2/…/call`, which may be present in the IC state, and the *synchronous* API requests passed to `/api/v2/…/read_state` and `/api/v2/…/query`, which are only ephemeral. + +These are the synchronous read messages: + + Path = List(Blob) + APIReadRequest + = StateRead = { + nonce : Blob; + ingress_expiry : Nat; + sender : UserId; + paths : List(Path); + } + | CanisterQuery = { + nonce : Blob; + ingress_expiry : Nat; + sender : UserId; + canister_id : CanisterId; + method_name : Text; + arg : Blob; + } + +Signed delegations contain the (unsigned) delegation data in a nested record, next to the signature of that data. + + PublicKey = Blob + Signature = Blob + SignedDelegation = { + delegation : { + pubkey : PublicKey; + targets : [CanisterId] | Unrestricted; + senders : [Principal] | Unrestricted; + expiration : Timestamp + }; + signature : Signature + } + +For the signatures in a `Request`, we assume that the following function implements signature verification as described in [Authentication](#authentication). This function picks the corresponding signature scheme according to the DER-encoded metadata in the public key. + + verify_signature : PublicKey -> Signature -> Blob -> Bool + + Envelope = { + content : Request | APIReadRequest; + sender_pubkey : PublicKey | NoPublicKey; + sender_sig : Signature | NoSignature; + sender_delegation: [SignedDelegation] + } + +The evolution of a `Request` goes through these states, as explained in [Overview of canister calling](#http-call-overview): + + RequestStatus + = Received + | Processing + | Rejected (RejectCode, Text) + | Replied Blob + | Done + +A `Path` may refer to a request by way of a *request id*, as specified in [Request ids](#request-id): + + RequestId = { b ∈ Blob | |b| = 32 } + hash_of_map: Request -> RequestId + +#### The system state {#_the_system_state} + +Finally, we can describe the state of the IC as a record having the following fields: + + CanState + = EmptyCanister | { + wasm_state : WasmState; + module : CanisterModule; + raw_module : Blob; + public_custom_sections: Text ↦ Blob; + private_custom_sections: Text ↦ Blob; + } + CanStatus + = Running + | Stopping (List (CallOrigin, Nat)) + | Stopped + S = { + requests : Request ↦ RequestStatus; + canisters : CanisterId ↦ CanState; + controllers : CanisterId ↦ Set Principal; + freezing_threshold : CanisterId ↦ Nat; + canister_status: CanisterId ↦ CanStatus; + time : CanisterId ↦ Timestamp; + balances: CanisterId ↦ Nat; + certified_data: CanisterId ↦ Blob; + system_time : Timestamp + call_contexts : CallId ↦ CallCtxt; + messages : List Message; // ordered! + root_key : PublicKey + } + +To convert `CanStatus` into `status : Running | Stopping | Stopped` from `Env`, we define the following conversion function: + + simple_status(Running) = Running + simple_status(Stopping _) = Stopping + simple_status(Stopped) = Stopped + +#### Initial state {#initial_state} + +The initial state of the IC is + + { + requests = (); + canisters = (); + controllers = (); + freezing_threshold = (); + canister_status = (); + time = (); + balances = (); + certified_data = (); + system_time = T; + call_contexts = (); + messages = []; + root_key = PublicKey; + } + +for some time stamp `T`, some DER-encoded BLS public key `PublicKey`, and using `()` to denote the empty map or bag. + +### Invariants {#invariants} + +The following is an incomplete list of invariants that should hold for the abstract state `S`, and are not already covered by the type annotations in this section. + +- No method name is the name of an update and query method in a CanisterModule at the same time: + + ∀ _ ↦ CanState ∈ S.canisters: + dom(CanState.module.update_methods) ∩ dom(CanState.module.query_methods) = ∅ + +- Deleted call contexts were not awaiting a response: + + ∀ Ctxt_id ↦ Ctxt ∈ S.call_contexts: + if Ctxt.deleted then Ctxt.needs_to_respond = false + +- Responded call contexts have no available_cycles left: + + ∀ Ctxt_id ↦ Ctxt ∈ S.call_contexts: + if Ctxt.needs_to_respond = false then Ctxt.available_cycles = 0 + +- Referenced call contexts exist: + + ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ∈ dom(S.call_contexts) + ∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ∈ dom(S.call_contexts) + ∀ _ ↦ {needs_to_respond = true, origin = FromCanister O, …} ∈ S.call_contexts: O.calling_context ∈ dom(S.call_contexts) + ∀ _ ↦ Stopping Origins ∈ S.canister_status: ∀(FromCanister O, _) ∈ Origins. O.calling_context ∈ dom(S.call_contexts) + +### State transitions {#state_transitions} + +Based on this abstract notion of the state, we can describe the behavior of the IC. There are three classes of behaviors: + +- Asynchronous API requests that are submitted via `/api/v2/…/call`. These transitions describe checks that the request must pass to be considered received. + +- Spontaneous transitions that model the internal behavior of the IC, by describing conditions on the state that allow the transition to happen, and the state after. + +- Responses to reads (i.e. `/api/v2/…/read_state` and `/api/v2/…/query`). By definition, these do *not* change the state of the IC, and merely describe the response based on the read request (or query, respectively) and the current state. + +The state transitions are not complete with regard to error handling. For example, the behavior of sending a request to a non-existent canister is not specified here. For now, we trust implementors to make sensible decisions there. + +We model the [The IC management canister](#ic-management-canister) with one state transition per method. There, we assume a function + + candid : Value -> Blob + +that represents Candid encoding; this is implicitly taking the method types, as declared in [Interface overview](#ic-candid), into account. We model the parsing of Candid values in the "Conditions" section using `candid` as well, by treating it as a non-deterministic function. + +#### Envelope Authentication {#envelope_authentication} + +The following predicate describes when an envelope `E` correctly signs the enclosed request with a key belonging to a user `U`, at time `T`: It returns which canister ids this envelope may be used at (as a set of principals). + + verify_envelope({ content = C }, U, T) + = { p : p is PrincipalId } if U = anonymous_id + verify_envelope({ content = C, sender_pubkey = PK, sender_sig = Sig, sender_delegation = DS}, U, T) + = TS if U = mk_self_authenticating_id E.sender_pubkey + ∧ (PK', TS) = verify_delegations(DS, PK, T, { p : p is PrincipalId }) + ∧ verify_signature ("\x0Aic-request" · hash_of_map(C), PK') + + verify_delegations([], PK, T, TS) = (PK, TS) + verify_delegations([D] · DS, PK, T, TS) + = verify_delegations(C, DS, D.pubkey, T, TS ∩ delegation_targets(DS)) + if verify_signature PK D.signature ("\x1Aic-request-auth-delegation" · hash_of_map(D.delegation)) + ∧ D.delegation.expiration ≥ T + + delegation_targets(DS) + = if D.targets = Unrestricted + then { p : p is PrincipalId } + else D.targets + +#### Effective canister ids {#effective_canister_ids} + +A `Request` has an effective canister id according to the rules in [Effective canister id](#http-effective-canister-id): + + is_effective_canister_id(Request {canister_id = ic_principal, method = provisional_create_canister_with_cycles, p) + is_effective_canister_id(Request {canister_id = ic_principal, arg = candid({canister_id = p, …}), …}, p) + is_effective_canister_id(Request {canister_id = p, …}, p), if p ≠ ic_principal + +#### API Request submission {#api_request_submission} + +After a node accepts a request via `/api/v2/canister//call`, the request gets added to the IC state as `Received`. + +This may only happen if the signature is valid and is created with a correct key. Due to this check, the envelope is discarded after this point. + +Requests that have expired are dropped here. + +Ingress message inspection is applied, and messages that are not accepted by the canister are dropped. + +The (unspecified) function `idle_cycles_burned_rate(S, cid)` determines the idle resource consumption rate in cycles per day of the canister with id `cid`, given its current memory footprint, compute and storage cost, and memory and compute allocation. The function `freezing_limit(S, cid)` determines the freezing threshold in cycles of the canister with id `cid`, given its current memory footprint, compute and storage cost, memory and compute allocation, and current `freezing_threshold` setting. The value `freezing_limit(S, cid)` is derived from `idle_cycles_burned_rate(S, cid)` and `freezing_threshold` as follows: + + freezing_limit(S, cid) = idle_cycles_burned_rate(S, cid) * S.freezing_threshold[cid] / (24 * 60 * 60) + +Submitted request + +`E : Envelope` + +Conditions + +```html +E.content.canister_id ∈ verify_envelope(E, E.content.sender, S.system_time) +E.content ∉ dom(S.requests) +S.system_time <= E.content.ingress_expiry +is_effective_canister_id(E.content, ECID) +( E.content.canister_id = ic_principal + E.content.arg = candid({canister_id = CanisterId, …}) + E.content.sender ∈ S.controllers[CanisterId] + E.content.method_name ∈ + { "install_code", "uninstall_code", "update_settings", "start_canister", "stop_canister", + "canister_status", "delete_canister", + "provisional_create_canister_with_cycles", "provisional_top_up_canister" } + ) ∨ ( + E.content.canister_id ≠ ic_principal + S.canisters[E.content.canister_id] ≠ EmptyCanister + Env = { + time = S.time[E.content.canister_id]; + balance = S.balances[E.content.canister_id] + freezing_limit = freezing_limit(S, E.content.canister_id); + certificate = NoCertificate; + status = simple_status(S.canister_status[E.content.canister_id]); + } + S.canisters[E.content.canister_id].module.inspect_message + (E.content.method_name, S.canisters[E.content.canister_id].wasm_state, E.content.arg, E.content.sender, Env) = Return {status = Accept; cycles_used = Cycles_used;} + Cycles_used ≤ S.balances[E.content.canister_id]) +``` + + +State after + +```html +S with + requests[E.content] = Received + if E.content.canister_id ≠ ic_principal then + balances[E.content.canister_id] = S.balances[E.content.canister_id] - Cycles_used +``` + +:::note +This is not instantaneous (the IC takes some time to agree it accepts the request) nor guaranteed (a node could just drop the request, or maybe it did not pass validation). But once the request has entered the IC state like this, it will be acted upon. +::: + +#### Request rejection {#request_rejection} + +The IC may reject a received message for internal reasons (high load, low resources) or expiry. The precise conditions are not specified here, but the reject code must indicate this to be a system error. + +Conditions: + +```html +S.requests[R] = Received +Code = SYS_FATAL or Code = SYS_TRANSIENT +``` + + +State after + +```html +S with + requests[R] = Rejected (Code, Msg) +``` + + +#### Initiating canister calls {#initiating_canister_calls} + +A first step in processing a canister update call is to create a `CallMessage` in the message queue. + +The `request` field of the `FromUser` origin establishes the connection to the API message. One could use the corresponding `hash_of_map` for this purpose, but this formulation is more abstract. + +The IC does not make any guarantees about the order of incoming messages. + +Conditions +```html +S.requests[R] = Received +S.system_time <= R.ingress_expiry +C = S.canisters[R.canister_id] +``` + + +State after +```html +S with + requests[R] = Processing + messages = + CallMessage { + origin = FromUser { request = R }; + caller = R.sender; + callee = R.canister_id; + method_name = R.method_name; + arg = R.arg; + transferred_cycles = 0; + queue = Unordered; + } · S.messages +``` +#### Calls to stopped/stopping/frozen canisters are rejected {#calls_to_stoppedstoppingfrozen_canisters_are_rejected} + +A call to a canister which is stopping, stopped, or frozen is automatically rejected. + +Conditions + +```html +S.messages = Older_messages · CallMessage CM · Younger_messages +(CM.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ CM.queue) +S.canister_status[CM.callee] = Stopped or S.canister_status[CM.callee] = Stopping _ or balances[CM.callee] < freezing_limit(S, CM.callee) +``` + + +State after + +```html +S.messages = Older_messages · Younger_messages · + ResponseMessage { + origin = CM.origin; + response = Reject (CANISTER_ERROR, "canister not running"); + refunded_cycles = CM.transferred_cycles; + } +``` + + +#### Call context creation {#call_context_creation} + +Before invoking a heartbeat or a message to a public entry point, a call context is created for bookkeeping purposes. For these invocations the canister must be running (so not stopped, or stopping). Additionally, these invocations only happen for \"real\" canisters, not the IC management canister. + +This "bookkeeping transition" must be immediately followed by the corresponding ["Message execution" transition](#rule-message-execution). + +- Call context creation: Public entry points + + For a message to a public entry point, the method is looked up in the list of exports. This happens for both ingress and inter-canister messages. + + The position of the message in the queue is unchanged. + + Conditions + ``` + S.messages = Older_messages · CallMessage CM · Younger_messages + S.canisters[CM.callee] ≠ EmptyCanister + S.canister_status[CM.callee] = Running + S.balances[CM.callee] ≥ freezing_limit(S, CM.callee) + MAX_CYCLES_PER_MESSAGE + Ctxt_id ∉ dom(S.call_contexts) + ``` + + State after + + + ``` + S with + messages = + Older_messages · + FuncMessage { + call_context = Ctxt_id; + receiver = CM.callee; + entry_point = PublicMethod CM.method_name CM.caller CM.arg; + queue = CM.queue; + } · + Younger_messages + call_contexts[Ctxt_id] = { + canister = CM.callee; + origin = CM.origin; + needs_to_respond = true; + deleted = false; + available_cycles = CM.transferred_cycles; + } + balances[CM.callee] = S.balances[CM.callee] - MAX_CYCLES_PER_MESSAGE + ``` + +- Call context creation: Heartbeat + + If canister `C` exports a method with name `canister_heartbeat`, the IC will create the corresponding call context. + + Conditions + + ``` + S.canisters[C] ≠ EmptyCanister + S.canister_status[C] = Running + balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE + Ctxt_id ∉ dom(S.call_contexts) + ``` + + State after + + ``` + S with + messages = + FuncMessage { + call_context = Ctxt_id; + receiver = C; + entry_point = Heartbeat; + queue = Queue { from = System; to = C }; + } + · S.messages + call_contexts[Ctxt_id] = { + canister = C; + origin = FromHeartbeat; + needs_to_respond = false; + deleted = false; + available_cycles = 0; + } + balances[C] = S.balances[C] - MAX_CYCLES_PER_MESSAGE + ``` + +The IC can execute any message that is at the head of its queue, i.e. there is no older message with the same abstract `queue` field. The actual message execution, if successful, may enqueue further messages and --- if the function returns a response --- record this response. The new call and response messages are enqueued at the end. + +Note that new messages are executed only if the canister is Running and is not frozen. + +#### Message execution {#rule-message-execution} + +The transition models the actual execution of a message, whether it is an initial call to a public method or a response. In either case, a call context already exists (see transition "Call context creation"). + +Conditions +```html +S.messages = Older_messages · FuncMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +S.canisters[M.receiver] ≠ EmptyCanister +Mod = S.canisters[M.receiver].module + +Is_response = M.entry_point == Callback _ _ _ + +Env = { + time = S.time[M.receiver]; + balance = S.balances[M.receiver] + freezing_limit = freezing_limit(S, M.receiver); + certificate = NoCertificate; + status = simple_status(S.canister_status[M.receiver]); +} + +Available = S.call_contexts[M.call_contexts].available_cycles + ( M.entry_point = PublicMethod Name Caller Arg + (F = Mod.update_methods[Name](Arg, Caller, Env, Available)) or (F = query_as_update(Mod.query_methods[Name], Arg, Caller, Env)) +) +or +( M.entry_point = Callback Callback Response RefundedCycles + F = Mod.callbacks(Callback, Response, RefundedCycles, Env, Available) +) +or +( M.entry_point = Heartbeat + F = heartbeat_as_update(Mod.heartbeat, Env) +) + +R = F(S.canisters[M.receiver].wasm_state) +``` + + +State after +```html + if + R = Return res + res.cycles_used ≤ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + res.cycles_accepted ≤ Available + (res.cycles_used + ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ]) ≤ + (S.balances[M.receiver] + res.cycles_accepted + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) + New_balance = + (S.balances[M.receiver] + res.cycles_accepted + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) + - (res.cycles_used + ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ]) + New_balance ≥ if Is_response then 0 else freezing_limit(S, M.receiver); + (res.response = NoResponse) or S.call_contexts[M.call_context].needs_to_respond +then + S with + canisters[M.receiver].wasm_state = res.new_state; + messages = + Older_messages · + Younger_messages · + [ CallMessage { + origin = FromCanister { + call_context = M.call_context; + callback = call.callback + }; + caller = M.receiver; + callee = call.callee; + method_name = call.method_name; + arg = call.arg; + transferred_cycles = call.transferred_cycles + queue = Queue { from = M.receiver; to = call.callee }; + } + | call ∈ res.new_calls ] · + [ ResponseMessage { + origin = S.call_contexts[M.call_context].origin + response = res.response; + refunded_cycles = Available - res.cycles_accepted; + } + | res.response ≠ NoResponse ] + + if res.response = NoResponse: + call_contexts[M.call_context].available_cycles = Available - res.cycles_accepted + else + call_contexts[M.call_context].needs_to_respond = false + call_contexts[M.call_context].available_cycles = 0 + + if res.new_certified_data ≠ NoCertifiedData: + certified_data[M.receiver] = res.new_certified_data + + balances[M.receiver] = New_balance +else + S with + messages = Older_messages · Younger_messages + balances[M.receiver] = + (S.balances[M.receiver] + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) + - min (R.cycles_used, (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) +``` + +Depending on whether this is a call message and a response messages, we have either set aside `MAX_CYCLES_PER_MESSAGE` or `MAX_CYCLES_PER_RESPONSE`, either in the call context creation rule or the Callback invocation rule. + +The cycle consumption of executing this message is modeled via the unspecified `Cycles_used` variable; the variable takes some value between 0 and `MAX_CYCLES_PER_MESSAGE`/`MAX_CYCLES_PER_RESPONSE` (for call execution and response execution, respectively). + +This transition detects certain behavior that will appear as a trap (and which an implementation may implement by trapping directly in a system call): + +- Responding if the present call context does not need to be responded to + +- Accepting more cycles than are available on the call context + +- Sending out more cycles than available to the canister + +- Consuming more cycles than allowed (and reserved) + +If message execution [*traps* (in the sense of a Wasm function)](#define-wasm-fn), the message gets dropped. No response is generated (as some other message may still fulfill this calling context). Any state mutation is discarded. If the message was a call, the associated cycles are held by its associated call context and will be refunded to the caller, see [Call context starvation](#rule-starvation). + +If message execution [*returns* (in the sense of a Wasm function)](#define-wasm-fn), the state is updated and possible outbound calls and responses are enqueued. + +Note that returning does *not* imply that the call associated with this message now *succeeds* in the sense defined in [section responding](#responding); that would require a (unique) call to `ic0.reply`. Note also that the state changes are persisted even when the IC is set to synthesize a [CANISTER_ERROR](#CANISTER_ERROR) reject immediately afterward (which happens when this returns without calling `ic0.reply` or `ic0.reject`, the corresponding call has not been responded to and there are no outstanding callbacks, see [Call context starvation](#rule-starvation)). + +The functions `query_as_update` and `heartbeat_as_update` turns a query function resp the heartbeat into an update function; this is merely a notational trick to simplify the rule: + + query_as_update(f, arg, env) = λ wasm_state → + match f(arg, env)(wasm_state) with + Trap trap → Trap trap + Return res → Return { + new_state = wasm_state; + new_calls = []; + new_certified_data = NoCertifiedData; + response = res.response; + cycles_accepted = 0; + cycles_used = res.cycles_used; + } + + heartbeat_as_update(f, env) = λ wasm_state → + match f(env)(wasm_state) with + Trap trap → Trap trap + Return res → Return { + new_state = res.new_state; + new_calls = []; + new_certified_data = NoCertifiedData; + response = NoResponse; + cycles_accepted = 0; + cycles_used = res.cycles_used; + } + +Note that by construction, a query function will either trap or return with a response; it will never send calls, and it will never change the state of the canister. + +#### Call context starvation {#rule-starvation} + +If the call context is not for heartbeat and there is no call, downstream calling context or response that could possibly fulfill a calling context, then a reject is synthesized. The error message below is *not* indicative. In particular, if the IC has an idea about *why* this starved, it can put that in there (e.g. the initial message handler trapped with an out-of-memory access). + +Conditions +```html + S.call_contexts[Ctxt_id].needs_to_respond = true + S.call_contexts[Ctxt_id].origin ≠ FromHeartbeat + ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id + ∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id + ∀ _ ↦ {needs_to_respond = true, origin = FromCanister O, …} ∈ S.call_contexts: O.calling_context ≠ Ctxt_id +``` + + +State after + +``` html +S with + call_contexts[Ctxt_id].needs_to_respond = false + call_contexts[Ctxt_id].available_cycles = 0 + messages = + S.messages · + ResponseMessage { + origin = S.call_contexts[Ctxt_id].origin; + response = Reject (CANISTER_ERROR, "starvation"); + refunded_cycles = S.call_contexts[Ctxt_id].available_cycles + } +``` + + +#### Call context removal {#call_context_removal} + +If there is no call, downstream calling context, or response that references a call context, and the call context has been replied to or the call context corresponds to a heartbeat that had already been executed, then the call context can be removed. + +Conditions + +```html +( + S.call_contexts[Ctxt_id].needs_to_respond = false +) or +( + S.call_contexts[Ctxt_id].origin = FromHeartbeat + ∀ FuncMessage M ∈ S.messages. M.call_context ≠ Ctxt_id +) + ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id + ∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id + ∀ _ ↦ {needs_to_respond = true, origin = FromCanister O, …} ∈ S.call_contexts: O.calling_context ≠ Ctxt_id + ∀ _ ↦ Stopping Origins ∈ S.canister_status: ∀(FromCanister O, _) ∈ Origins. O.calling_context ≠ Ctxt_id +``` + + +State after + +```html +S with + call_contexts[Ctxt_id] = (deleted) +``` + + +#### IC Management Canister: Canister creation {#ic_management_canister_canister_creation} + +The IC chooses an appropriate canister id and instantiates a new (empty) canister identified by this id. The *controllers* are set such that the sender of this request is the only controller, unless the `settings` say otherwise. All cycles on this call are now the canister's initial cycles. + +This is also when the System Time of the new canister starts ticking. + +The `compute_allocation` and `memory_allocation` settings are ignored in this abstract model of the Internet Computer, as it does not address questions of performance or scheduling. + +Conditions + +```html +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'create_canister' +M.arg = candid(A) +is_system_assigned CanisterId +CanisterId ∉ dom(S.canisters) +``` + + +State after + +```html +S with + canisters[CanisterId] = EmptyCanister + time[CanisterId] = CurrentTime + if A.settings.controllers is not null: + controllers[CanisterId] = A.settings.controllers + else: + controllers[CanisterId] = [M.caller] + freezing_threshold[CanisterId] = 2592000 + balances[CanisterId] = M.transferred_cycles + certified_data[CanisterId] = "" + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = Reply (candid({canister_id = CanisterId})) + refunded_cycles = 0 + } + canister_status[CanisterId] = Running +``` + + +This uses the predicate + + is_system_assigned : Principal -> Bool + +which characterizes all system-assigned ids. + +To avoid clashes with potential user ids or is derived from users or canisters, we require (somewhat handwavy) that + +- `is_system_assigned (mk_self_authenticating_id pk) = false` for possible public keys `pk` and + +- `is_system_assigned (mk_derived_id p dn) = false` for any `p` that could be a user id or canister id. + +- `is_system_assigned p = false` for `|p| > 29`. + +- `is_system_assigned ic_principal = false`. + +#### IC Management Canister: Changing settings {#ic_management_canister_changing_settings} + +Only the controllers of the given canister can update the canister settings. + +The `compute_allocation` and `memory_allocation` settings are ignored in this abstract model of the Internet Computer, as it does not address questions of performance or scheduling. + +Conditions + +```html +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'update_settings' +M.arg = candid(A) +M.caller ∈ S.controllers[A.canister_id] +``` + + +State after + +```html + S with + if A.settings.controllers is not null: + controllers[A.canister_id] = A.settings.controllers + if A.settings.freezing_threshold is not null: + freezing_threshold[A.canister_id] = A.settings.freezing_threshold + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = Reply (candid()) + refunded_cycles = M.transferred_cycles + } +``` + + +#### IC Management Canister: Canister status {#ic_management_canister_canister_status} + +The controllers of a canister can obtain information about the canister. + +The `Memory_size` is the (in this specification underspecified) total size of storage in bytes. + +The `idle_cycles_burned_per_day` is the idle consumption of resources in cycles per day. + +Conditions + +```html +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'canister_status' +M.arg = candid(A) +M.caller ∈ S.controllers[A.canister_id] +``` + + +State after + +```html + S with + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = candid({ + status = simple_status(S.canister_status[A.canister_id]); + module_hash = + if S.canisters[A.canister_id] = EmptyCanister + then null + else opt (SHA-256(S.canisters[A.canister_id].raw_module)); + controllers = S.controllers[A.canister_id]; + memory_size = Memory_size; + cycles = S.balances[A.canister_id]; + freezing_threshold = S.freezing_threshold[A.canister_id]; + idle_cycles_burned_per_day = idle_cycles_burned_rate(S, A.canister_id); + }) + refunded_cycles = M.transferred_cycles + } +``` + + +#### IC Management Canister: Code installation {#ic_management_canister_code_installation} + +Only the controllers of the given canister can install code. This transition installs new code over a canister. This involves invoking the `canister_init` method (see [Canister initialization](#system-api-init)), which must succeed. + +Conditions + +```html + S.messages = Older_messages · CallMessage M · Younger_messages + (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) + M.callee = ic_principal + M.method_name = 'install_code' + M.arg = candid(A) + Mod = parse_wasm_mod(A.wasm_module) + Public_custom_sections = parse_public_custom_sections(A.wasm_module); + Private_custom_sections = parse_private_custom_sections(A.wasm_module); + (A.mode = install and S.canisters[A.canister_id] = EmptyCanister) or A.mode = reinstall + M.caller ∈ S.controllers[A.canister_id] + Env = { + time = S.time[A.canister_id]; + balance = S.balances[A.canister_id]; + freezing_limit = freezing_limit(S, A.canister_id); + certificate = NoCertificate; + status = simple_status(S.canister_status[A.canister_id]); + } + Mod.init(A.canister_id, A.arg, M.caller, Env) = Return {new_state = New_state; cycles_used = Cycles_used;} + Cycles_used ≤ S.balances[A.canister_id] + dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ +``` + + +State after + +```html + S with + canisters[A.canister_id] = { + wasm_state = New_state; + module = Mod; + raw_module = A.wasm_module; + public_custom_sections = Public_custom_sections; + private_custom_sections = Private_custom_sections; + } + balances[A.canister_id] = S.balances[A.canister_id] - Cycles_used + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin; + response = Reply (candid()); + refunded_cycles = M.transferred_cycles; + } +``` + + +#### IC Management Canister: Code upgrade {#ic_management_canister_code_upgrade} + +Only the controllers of the given canister can install new code. This changes the code of an *existing* canister, preserving the state in the stable memory. This involves invoking the `canister_pre_upgrade` method on the old and `canister_post_upgrade` method on the new canister, which must succeed and must not invoke other methods. + +Conditions + +```html + S.messages = Older_messages · CallMessage M · Younger_messages + (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) + M.callee = ic_principal + M.method_name = 'install_code' + M.arg = candid(A) + Mod = parse_wasm_mod(A.wasm_module) + Public_custom_sections = parse_public_custom_sections(A.wasm_module) + Private_custom_sections = parse_private_custom_sections(A.wasm_module) + A.mode = upgrade + M.caller ∈ S.controllers[A.canister_id] + S.canisters[A.canister_id] = { wasm_state = Old_state; module = Old_module, …} + Env = { + time = S.time[A.canister_id]; + balance = S.balances[A.canister_id]; + freezing_limit = freezing_limit(S, A.canister_id); + certificate = NoCertificate; + status = simple_status(S.canister_status[A.canister_id]); + } + Old_module.pre_upgrade(Old_State, M.caller, Env) = Return {stable_memory = Stable_memory; cycles_used = Cycles_used;} + Mod.post_upgrade(A.canister_id, Stable_memory, A.arg, M.caller, Env) = Return {new_state = New_state; cycles_used = Cycles_used';} + Cycles_used + Cycles_used' ≤ S.balances[A.canister_id] + dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ +``` + + +State after + +```html + S with + canisters[A.canister_id] = { + wasm_state = New_state; + module = Mod; + raw_module = A.wasm_module; + public_custom_sections = Public_custom_sections; + private_custom_sections = Private_custom_sections; + } + balances[A.canister_id] = S.balances[A.canister_id] - (Cycles_used + Cycles_used'); + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin; + response = Reply (candid()); + refunded_cycles = M.transferred_cycles; + } +``` + + +#### IC Management Canister: Code uninstallation {#rule-uninstall} + +Upon uninstallation, the canister is reverted to an empty canister, and all outstanding call contexts are rejected and marked as deleted. + +Conditions + +```html +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'uninstall_code' +M.arg = candid(A) +M.caller ∈ S.controllers[A.canister_id] +``` + + +State after + +```html +S with + canisters[A.canister_id] = EmptyCanister + certified_data[A.canister_id] = "" + + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = Reply (candid()) + refunded_cycles = M.transferred_cycles + } · + [ ResponseMessage { + origin = Ctxt.origin + response = Reject (CANISTER_REJECT, 'Canister has been uninstalled') + refunded_cycles = Ctxt.available_cycles + } + | Ctxt_id ↦ Ctxt ∈ S.call_contexts + , Ctxt.canister = A.canister_id + , Ctxt.needs_to_respond = true + ] + + for Ctxt_id ↦ Ctxt ∈ S.call_contexts: + if Ctxt.canister = A.canister_id: + call_contexts[Ctxt_id].deleted := true + call_contexts[Ctxt_id].needs_to_respond := false + call_contexts[Ctxt_id].available_cycles := 0 +``` + + +#### IC Management Canister: Stopping a canister {#ic_management_canister_stopping_a_canister} + +The controllers of a canister can stop a canister. Stopping a canister goes through two steps. First, the status of the canister is set to `Stopping`; as explained above, a stopping canister rejects all incoming requests and continues processing outstanding responses. When a stopping canister has no more open call contexts that are not marked as deleted, its status is changed to `Stopped` and a response is generated. Note that when processing responses, a stopping canister can make calls to other canisters and thus create new call contexts. In addition, a canister which is stopped, or stopping will accept (and respond) to further `stop_canister` requests. + +We encode this behavior via three (types of) transitions: + +1. First, any `stop_canister` call sets the state of the canister to `Stopping`; we record in the status the origin (and cycles) of all `stop_canister` calls which arrive at the canister while it is stopping (or stopped). + +2. Next, when the canister has no open call contexts that are not marked as deleted (so, in particular, all outstanding responses to the canister have been processed or will be discared because the call context has been marked as deleted), the status of the canister is set to `Stopped`. + +3. Finally, each pending `stop_canister` call (which are encoded in the status) is responded to, to indicate that the canister is stopped. + +Conditions + +``` +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'stop_canister' +M.arg = candid(A) +S.canister_status[A.canister_id] = Running +M.caller ∈ S.controllers[A.canister_id] +``` + +State after + +``` +S with + messages = Older_messages · Younger_messages + canister_status[A.canister_id] = Stopping [(M.origin, M.transferred_cycles)] +``` + +The next two transitions record any additional \'stop_canister\' requests that arrive at a stopping (or stopped) canister in its status. + +Conditions + +```html +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'stop_canister' +M.arg = candid(A) +S.canister_status[A.canister_id] = Stopping Origins +M.caller ∈ S.controllers[A.canister_id] +``` + +State after + +```html +S with + messages = Older_messages · Younger_messages + canister_status[A.canister_id] = Stopping (Origins · [(M.origin, M.transferred_cycles)]) +``` + +The status of a stopping canister which has no open call contexts that are not marked as deleted is set to `Stopped`, and all pending `stop_canister` calls are replied to. + +Conditions + +```html + S.canister_status[Canister_id] = Stopping Origins + ∀ Ctxt_id. S.call_contexts[Ctxt_id].deleted or S.call_contexts[Ctxt_id].canister ≠ Canister_id +``` + + +State after + +```html + S with + canister_status[CanisterId] = Stopped + messages = S.Messages · + [ ResponseMessage { + origin = O + response = Reply (candid()) + refunded_cycles = C + } + | (O, C) ∈ Origins + ] +``` + + +:::note +Sending a `stop_canister` message to an already stopped canister is acknowledged (i.e. responded with success), but is otherwise a no-op: +::: + +Conditions + +```html +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'stop_canister' +M.arg = candid(A) +S.canister_status[A.canister_id] = Stopped +M.caller ∈ S.controllers[A.canister_id] +``` + + +State after + +```html + S with + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin; + response = Reply (candid()); + refunded_cycles = M.transferred_cycles; + } +``` + + +#### IC Management Canister: Starting a canister {#ic_management_canister_starting_a_canister} + +The controllers of a canister can start a `stopped` canister. If the canister is already running, the command has no effect on the canister. + +Conditions + +```html + +``` + S.messages = Older_messages · CallMessage M · Younger_messages + (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) + M.callee = ic_principal + M.method_name = 'start_canister' + M.arg = candid(A) + S.canister_status[A.canister_id] = Running or S.canister_status[A.canister_id] = Stopped + M.caller ∈ S.controllers[A.canister_id] + +State after + +```html + S with + canister_status[A.canister_id] = Running + messages = Older_messages · Younger_messages · + ResponseMessage{ + origin = M.origin + response = Reply (candid()) + refunded_cycles = M.transferred_cycles + } +``` + + +If the status of the canister was 'stopping', then the canister status is set to `running`. The pending `stop_canister` request(s) are rejected. + +Conditions + +```html +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'start_canister' +M.arg = candid(A) +S.canister_status[A.canister_id] = Stopping Origins +M.caller ∈ S.controllers[A.canister_id] +``` + + +State after + +```html + S with + canister_status[A.canister_id] = Running + messages = Older_messages · Younger_messages · + ResponseMessage{ + origin = M.origin + response = Reply (candid()) + refunded_cycles = M.transferred_cycles + } · + [ ResponseMessage { + origin = O + response = Reject (CANISTER_REJECT, 'Canister has been restarted') + refunded_cycles = C + } + | (O, C) ∈ Origins + ] +``` + + +#### IC Management Canister: Canister deletion {#ic_management_canister_canister_deletion} + +Conditions + +```html + S.messages = Older_messages · CallMessage M · Younger_messages + (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) + M.callee = ic_principal + M.method_name = 'delete_canister' + M.arg = candid(A) + S.canister_status[A.canister_id] = Stopped + M.caller ∈ S.controllers[A.canister_id] +``` + + +State after + +```html + S with + canisters[A.canister_id] = (deleted) + controllers[A.canister_id] = (deleted) + freezing_threshold[A.canister_id] = (deleted) + canister_status[A.canister_id] = (deleted) + time[A.canister_id] = (deleted) + balances[A.canister_id] = (deleted) + certified_data[A.canister_id] = (deleted) + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = Reply (candid()) + refunded_cycles = M.transferred_cycles + } +``` + + +#### IC Management Canister: Depositing cycles {#ic_management_canister_depositing_cycles} + +Conditions + +```html +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'deposit_cycles' +M.arg = candid(A) +A.canister_id ∈ dom(S.balances) +``` + + +State after + +```html +S with + balances[A.canister_id] = + S.balances[A.canister_id] + M.transferred_cycles + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = Reply (candid()) + refunded_cycles = 0 + } +``` + + +#### IC Management Canister: Random numbers {#ic_management_canister_random_numbers} + +The management canister can produce pseudo-random bytes. It always returns a 32-byte `blob`: + +The precise guarantees around the randomness, e.g. unpredictability, are not captured in this formal semantics. + +Conditions + +```html +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'raw_rand' +M.arg = candid() +|B| = 32 +``` + + +State after + +```html +S with + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = Reply (candid(B)) + refunded_cycles = M.transferred_cycles + } +``` + + +#### IC Management Canister: Canister creation with cycles {#ic_management_canister_canister_creation_with_cycles} + +This is a variant of `create_canister`, which sets the initial cycle balance based on the `amount` argument. + +Conditions + +```html +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'provisional_create_canister_with_cycles' +M.arg = candid(A) +is_system_assigned CanisterId +CanisterId ∉ dom(S.canisters) +``` + +State after + +```html +S with + canisters[CanisterId] = EmptyCanister + time[CanisterId] = CurrentTime + controllers[CanisterId] = [M.caller] + freezing_threshold[CanisterId] = 2592000 + balances[CanisterId] = A.amount + certified_data[CanisterId] = "" + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = Reply (candid({canister_id = CanisterId})) + transferred_cycles = M.transferred_cycles + } + canister_status[CanisterId] = Running +``` + + +#### IC Management Canister: Top up canister {#ic_management_canister_top_up_canister} + +Conditions + +```html +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'provisional_top_up_canister' +M.arg = candid(A) +A.canister_id ∈ dom(S.canisters) +``` + + +State after + +```html +S with + balances[A.canister_id] = S.balances[A.canister_id] + A.amount +``` + + +#### Callback invocation {#callback_invocation} + +When an inter-canister call has been responded to, we can queue the call to the callback. + +This "bookkeeping transition" must be immediately followed by the corresponding ["Message execution" transition](#rule-message-execution). + +Conditions + +```html +S.messages = Older_messages · ResponseMessage RM · Younger_messages +RM.origin = FromCanister { + call_context = Ctxt_id + callback = Callback + } +not S.call_contexts[Ctxt_id].deleted +S.call_contexts[Ctxt_id].canister ∈ dom(S.balances) +``` + +State after + +```html +S with + balances[S.call_contexts[Ctxt_id].canister] = + S.balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles + messages = + Older_messages · + FuncMessage { + call_context = Ctxt_id + receiver = S.call_contexts[Ctxt_id].canister + entry_point = Callback Callback RM.response RM.refunded_cycles + queue = Unordered + } · + Younger_messages +``` + + +If the responded call context does not exist anymore, because the canister has been uninstalled since, the refunded cycles are still added to the canister balance, but no function invocation is enqueued: + +Conditions + +```html +S.messages = Older_messages · ResponseMessage RM · Younger_messages +RM.origin = FromCanister { + call_context = Ctxt_id + callback = Callback + } +S.call_contexts[Ctxt_id].deleted +``` + + +State after + +```html +S with + balances[S.call_contexts[Ctxt_id].canister] = + S.balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles + MAX_CYCLES_PER_RESPONSE + messages = Older_messages · Younger_messages +``` + + +#### Respond to user request {#respond_to_user_request} + +When an ingress method call has been responded to, we can record the response in the list of queries. + +Conditions + +```html + +S.messages = Older_messages · ResponseMessage RM · Younger_messages +RM.origin = FromUser { request = M } +S.requests[M] = Processing +``` + + +State after + +```html +S with + messages = Older_messages · Younger_messages + requests[M] = + | Replied R if M.response = Reply R + | Rejected (c, R) if M.response = Reject (c, R) +``` + +NB: The refunded cycles, `RM.refunded_cycles` are, by construction, empty. + +#### Request clean up {#request_clean_up} + +The IC will keep the data for a completed or rejected request around for a certain, implementation defined amount of time, to allow users to poll for the data. After that time, the data of the request will be dropped: + +Conditions + +```html +(S.requests[M] = Replied _) or (S.requests[M] = Rejected _) +``` + + +State after + +```html +S with + requests[M] = Done +``` + + +At the same or some later point, the request will be removed from the state of the IC. This must happen no earlier than the ingress expiry time set in the request. + +Conditions + +```html +(S.requests[M] = Replied _) or (S.requests[M] = Rejected _) or (S.requests[M] = Done) +M.ingress_expiry < S.system_time +``` + + +State after + +```html +S with + requests[M] = (deleted) +``` + + +#### Canister out of cycles {#canister_out_of_cycles} + +Once a canister runs out of cycles, its code is uninstalled (cf. [IC Management Canister: Code uninstallation](#rule-uninstall)) and the allocations are set to zero (NB: allocations are currently not modeled in the formal model): + +Conditions + +```html +S.balances[CanisterId] = 0 +``` + + +State after + +```html +S with + canisters[CanisterId] = EmptyCanister + certified_data[Canister_id] = "" + + messages = S.messages · + [ ResponseMessage { + origin = Ctxt.origin + response = Reject (CANISTER_REJECT, 'Canister has been uninstalled') + refunded_cycles = Ctxt.available_cycles + } + | Ctxt_id ↦ Ctxt ∈ S.call_contexts + , Ctxt.canister = CanisterId + , Ctxt.needs_to_respond = true + ] + + for Ctxt_id ↦ Ctxt ∈ S.call_contexts: + if Ctxt.canister = CanisterId: + call_contexts[Ctxt_id].deleted := true + call_contexts[Ctxt_id].needs_to_respond := false + call_contexts[Ctxt_id].available_cycles := 0 +``` + + +#### Time progressing and cycle consumption {#time_progressing_and_cycle_consumption} + +Time progresses. Abstractly, it does so independently for each canister, and in unspecified intervals. + +Conditions + +```html +T0 = S.time[CanisterId] +T1 > T0 +``` + + +State after + +```html +S with + time[CanisterId] = T1 +``` + + +The canister cycle balance similarly depletes at an unspecified rate, but stays non-negative: + +Conditions + +```html +B0 = S.balances[CanisterId] +0 ≤ B1 < B0 +``` + + +State after + +```html +S with + balances[CanisterId] = B1 +``` + + +Similarly, the system time, used to expire requests, progresses: + +Conditions + +```html +T0 = S.system_time +T1 > T0 +``` + + +State after + +```html +S with + system_time = T1 +``` + + +#### Query call {#query_call} + +Canister query calls to `/api/v2/canister//query` can be executed directly. They can only be executed against canisters which have a status of `Running` and are also not frozen. + +During the execution of a query call, a certificate is provided to the canister that is valid, contains a current state tree (or "recent enough"; the specification is currently vague about how old the certificate may be) and reveals the canister's [Certified Data](#system-api-certified-data). + +Submitted request + +`E` + +Conditions + +```html +E.content = CanisterQuery Q +Q.canister_id ∈ verify_envelope(E, Q.sender, S.system_time) +is_effective_canister_id(E.content, ECID) +S.system_time <= Q.ingress_expiry +S.canisters[Q.canister_id] ≠ EmptyCanister +S.canister_status[Q.canister_id] = Running ∧ S.balances[Q.canister_id] >= freezing_limit(S, Q.canister_id) +C = S.canisters[Q.canister_id] +F = C.module.query_methods[Q.method_name] +verify_cert(Cert) +lookup(["canister",Q.canister_id,"certified_data"], Cert) = Found S.certified_data[Q.canister_id] +lookup(["time"], Cert) = Found S.system_time // or “recent enough” +Env = { + time = S.time[Q.receiver]; + balance = S.balances[Q.canister_id]; + freezing_limit = freezing_limit(S, Q.canister_id); + certificate = Cert; + status = simple_status(S.canister_status[Q.receiver]); +} +``` + + +Read response: + +- If `F(Q.Arg, Q.sender, Env) = Trap trap` then + + {status: rejected; reject_code: CANISTER_ERROR, reject_message: , error_code: } + +- Else if `F(Q.Arg, Q.sender, Env) = Return {response = Reject (code, msg); …}` then + + {status: rejected; reject_code: : reject_message: , error_code: } + +- Else if `F(Q.Arg, Q.sender, Env) = Return {response = Reply R; …}` then + + {status: success; reply: { arg : } } + +#### Certified state reads {#certified_state_reads} + +The user can read elements of the *state tree*, using a `read_state` request to `/api/v2/canister//read_state`. + +Submitted request + +`E` + +Conditions + +```html +E.content = ReadState RS +TS = verify_envelope(E, RS.sender, S.system_time) +S.system_time <= RS.ingress_expiry +∀ path ∈ RS.paths. may_read_path(S, R.sender, path) +∀ ["request_status", Rid] · _ ∈ RS.paths. ∃ R ∈ S.requests ∧ hash_of_map(R) = Rid ∧ R.canister_id ∈ TS +``` + + +Read response + + A record with + - `{certificate: C}` + +The predicate `may_read_path` is defined as follows, implementing the access control outlined in [Request: Read state](#http-read-state): + + may_read_path(S, _, ["time"]) = True + may_read_path(S, _, ["request_status", Rid] · _) = + if ∃ R ∈ S.requests ∧ hash_of_map(R) = Rid + then RS.sender == R.sender ∧ is_effective_canister_id(R, ECID) + else True + may_read_path(S, _, ["canister", cid, "module_hash"]) = cid == ECID + may_read_path(S, _, ["canister", cid, "controllers"]) = cid == ECID + may_read_path(S, _, ["canister", cid, "metadata", name]) = + if name ∈ dom(S.canisters[cid].public_custom_sections) + then cid == ECID + else if name ∈ dom(S.canisters[cid].private_custom_sections) + then cid == ECID ∧ RS.sender ∈ S.controllers[cid] + else False + may_read_path(S, _, _) = False + +The response is a certificate `cert`, as specified in [Certification](#certification), which passes `verify_cert` (assuming `S.root_key` as the root of trust), and where for every `path` documented in [The system state tree](#state-tree) that is a suffix of a path in `RS.paths` or of `["time"]`, we have + + lookup(path, cert) = lookup_in_tree(path, state_tree(S)) + +where `state_tree` constructs the a labeled tree from the IC state `S` and the (so far underspecified) set of subnets `subnets`, as per [The system state tree](#state-tree) + + state_tree(S) = { + "time": S.system_time; + "request_id": { request_id(R): request_status_tree(S) | (R ↦ S) ∈ S.requests }; + "canister": + { canister_id : + { "module_hash" : SHA256(C.raw_module) | if C ≠ EmptyCanister } ∪ + { "controllers" : CBOR(S.controllers[canister_id]) } ∪ + { "metadata": { name: blob | (name, blob) ∈ S.canisters[canister_id].public_custom_sections ∪ S.canisters[canister_id].private_custom_sections } } + | (canister_id, C) ∈ S.canisters }; + "subnet": { subnet_id : { "public_key" : pub } | (subnet_id, subnet_pk) ∈ subnets }; + } + + request_status_tree(Received) = + { "status": "received" } + request_status_tree(Processing) = + { "status": "processing" } + request_status_tree(Rejected (code,msg)) = + { "status": "rejected"; "reject_code": code; "reject_message": msg, error_code: } + request_status_tree(Replied arg) = + { "status": "replied"; "reply": arg } + request_status_tree(Done) = + { "status": "Done" } + +and where `lookup_in_tree` is a function that returns the value or `Absent` as appropriately. + +### Abstract Canisters to System API {#concrete-canisters} + +In Section [Abstract canisters](#abstract-canisters) we introduced an abstraction over the interface to a canister, to avoid cluttering the abstract specification of the Internet Computer from WebAssembly details. In this section, we will fill the gap and explain how the abstract canister interface maps to the [concrete System API](#system-api) and the WebAssembly concepts as defined in the [WebAssembly specification](https://webassembly.github.io/spec/core/index.html). + +#### The concrete `WasmState` {#the_concrete_wasmstate} + +The abstract `WasmState` above models the WebAssembly *store* `S`, which encompasses the functions, tables, memories and globals of the WebAssembly program, plus additional data maintained by the IC, such as the stable memory: + + WasmState = { + store : S; // a store as per WebAssembly spec + self_id : CanId; + stable_mem : Blob + } + +As explained in Section "[WebAssembly module requirements](#system-api-module)", the WebAssembly module imports at most *one* memory and at most *one* table; in the following, *the* memory (resp. table) and the fields `mem` and `table` of `S` refer to that. Any system call that accesses the memory (resp. table) will trap if the module does not import the memory (resp. table). + +We model `mem` as an array of bytes, and `table` as an array of execution functions. + +The abstract `Callback` type above models an entry point for responses: + + Closure = { + fun : i32, + env : i32, + } + + Callback = { + on_reply : Closure; + on_reject : Closure; + on_cleanup : Closure | NoClosure; + } + +#### The execution state {#the_execution_state} + +We can model the execution of WebAssembly functions as stateful functions that have access to the WebAssembly store. In order to also model the behavior of the system imports, which have access to additional data structures, we extend the state as follows: + + Params = { + data : NoData | Blob; + caller : NoCaller | Principal; + reject_code : 0 | SYS_FATAL | SYS_TRANSIENT | …; + reject_message : Text; + sysenv : Env; + cycles_refunded : Nat; + method_name : Text; + } + ExecutionState = { + wasm_state : WasmState; + params : Params; + response : NoResponse | Response; + cycles_accepted : Nat; + cycles_available : Nat; + cycles_used : Nat; + balance : Funds; + reply_params : { arg : Blob }; + pending_call : MethodCall | NoPendingCall; + calls : List MethodCall; + new_certified_data : NoCertifiedData | Blob; + ingress_filter : Accept | Reject; + } + +This allows us to model WebAssembly functions, including host-provided imports, as functions with implicit mutable access to an `ExecutionState`, dubbed *execution functions*. Syntactically, we express this using an implicit argument of type `ref ExecutionState` in angle brackets (e.g. `func(x)` for the invocation of a WebAssembly function with type `(x : i32) -> ()`). The lifetime of the `ExecutionState` data structure is that of one such function invocation. + +:::warn +It is nonsensical to pass to an execution function a WebAssembly store `S` that comes from a different WebAssembly module than one defining the function. +::: + +#### The concrete `CanisterModule` {#the_concrete_canistermodule} + +Finally we can specify the abstract `CanisterModule` that models a concrete WebAssembly module. + +- The `initial_wasm_store` mentioned below is the store of the WebAssembly module after *instantiation* (as per WebAssembly spec) of the WasmModule contained in the [canister module](#canister-module-format), including executing a potential `(start)` function. + +- For more convenience when creating a new `ExecutionState`, we define the following partial records: + + empty_params = { + arg = NoArg; + caller = NoCaller; + reject_code = 0; + reject_message = ""; + cycles_refunded = 0; + method_name = NoText; + } + + empty_execution_state = { + wasm_state = (undefined); + params = (undefined); + response = NoResponse; + cycles_accepted = 0; + cycles_available = 0; + cycles_used = 0; + balance = 0; + reply_params = { arg = "" }; + pending_call = NoPendingCall; + calls = []; + new_certified_data = NoCertifiedData; + ingress_filter = Reject; + } + +- The `init` field of the `CanisterModule` is defined as follows: + + If the WebAssembly module does not export a function called under the name `canister_init`, then the argument blob is ignored and the `initial_wasm_store` is returned: + + init = λ (self_id, arg, caller, sysenv) → + Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" }; cycles_used = 0;} + + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_init`, it is + + init = λ (self_id, arg, caller, sysenv) → + let es = ref {empty_execution_state with + wasm_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" } + params = empty_params with { arg = arg; caller = caller; sysenv } + balance = sysenv.balance + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} + Return {new_state = es.wasm_state; cycles_used = es.cycles_used;} + + This formulation checks afterwards that the system calls `call.perform` or `msg.reply` were not invoked; an implementation can of course trap as soon as these system calls are invoked. + +- The `pre_upgrade` field of the `CanisterModule` is defined as follows: + + If the WebAssembly module does not export a function called under the name `canister_pre_upgrade`, then it simply returns the stable memory: + + pre_upgrade = λ (old_state, caller, sysenv) → Return {stable_memory = old_state.stable_mem; cycles_used = 0;} + + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_pre_upgrade`, it is + + pre_upgrade = λ (old_state, caller, sysenv) → + let es = ref {empty_execution_state with + wasm_state = old_state + params = { empty_params with caller = caller; sysenv } + balance = sysenv.balance + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} + Return {stable_memory = es.wasm_state.stable_mem; cycles_used = es.cycles_used;} + +- The `post_upgrade` field of the `CanisterModule` is defined as follows: + + If the WebAssembly module does not export a function called under the name `canister_post_upgrade`, then the argument blob is ignored and the `initial_wasm_store` is returned: + + post_upgrade = λ (self_id, stable_mem, arg, caller, sysenv) → + Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem }; cycles_used = 0;} + + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_post_upgrade`, it is + + post_upgrade = λ (self_id, stable_mem, arg, caller, sysenv) → + let es = ref {empty_execution_state with + wasm_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem } + params = { empty_params with arg = arg; caller = caller; sysenv } + balance = sysenv.balance + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} + Return {new_state = es.wasm_state; cycles_used = es.cycles_used;} + +- The partial map `update_methods` of the `CanisterModule` is defined for all method names `method` for which the WebAssembly program exports a function `func` named `canister_update `, and has value + + update_methods[method] = λ (arg, caller, sysenv, available) → λ wasm_state → + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = empty_params with { arg = arg; caller = caller; sysenv } + balance = sysenv.balance + cycles_available = available; + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + new_calls = es.calls; + response = es.response; + cycles_accepted = es.cycles_accepted; + cycles_used = es.cycles_used; + new_certified_data = es.new_certified_data; + } + +- The partial map `query_methods` of the `CanisterModule` is defined for all method names `method` for which the WebAssembly program exports a function `func` named `canister_query `, and has value + + query_methods[method] = λ (arg, caller, sysenv) → λ wasm_state → + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = empty_params with { arg = arg; caller = caller; sysenv } + balance = sysenv.balance + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used;} + if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} + if es.response = NoResponse then Trap {cycles_used = es.cycles_used;} + Return { + response = es.response; + cycles_used = es.cycles_used; + } + + This formulation checks afterwards that the system call `ic0.call_perform` was not invoked; an implementation can of course trap already when these system calls have been invoked. + + By construction, the (possibly modified) `es.wasm_state` is discarded. + +- The function `heartbeat` of the `CanisterModule` is defined if the WebAssembly program exports a function `func` named `canister_heartbeat`, and has value + + heartbeat = λ (sysenv) → λ wasm_state → + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = empty_params with { arg = NoArg; caller = NoCaller; sysenv } + balance = sysenv.balance + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + cycles_used = es.cycles_used; + } + + otherwise it is + +```html +heartbeat = λ (sysenv) → λ wasm_state → Trap +``` + heartbeat = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} + +- The function `callbacks` of the `CanisterModule` is defined as follows + + callbacks = λ(callbacks, response, refunded_cycles, sysenv, available) → λ wasm_state → + let params0 = { empty_params with + sysenv + cycles_refunded = refund_cycles; + } + let (fun, env, params) = match response with + Reply data -> + (callbacks.on_reply.fun, callbacks.on_reply.env, + { params0 with data}) + Reject (reject_code, reject_message)-> + (callbacks.on_reject.fun, callbacks.on_reject.env, + { params0 with reject_code; reject_message}) + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = params; + balance = sysenv.balance; + cycles_available = available; + } + try + if fun > |es.wasm_state.store.table| then Trap + let func = es.wasm_state.store.table[fun] + if typeof(func) ≠ func (i32) -> () then Trap + + func(env) + Return { + new_state = es.wasm_state; + new_calls = es.calls; + response = es.response; + cycles_accepted = es.cycles_accepted; + cycles_used = es.cycles_used; + new_certified_data = es.certified_data; + } + with Trap + if callbacks.on_cleanup = NoClosure then Trap {cycles_used = es.cycles_used;} + if callbacks.on_cleanup.fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} + let func = es.wasm_state.store.table[callbacks.on_cleanup.fun] + if typeof(func) ≠ func (i32) -> () then Trap {cycles_used = es.cycles_used;} + + let es' = ref { empty_execution_state with + wasm_state = wasm_state; + } + try func(callbacks.on_cleanup.env) with Trap then Trap {cycles_used = es.cycles_used + es'.cycles_used;} + if es'.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used + es'.cycles_used;} + if es'.calls ≠ [] then Trap {cycles_used = es.cycles_used + es'.cycles_used;} + if es'.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used + es'.cycles_used;} + if es'.response ≠ NoResponse then Trap {cycles_used = es.cycles_used + es'.cycles_used;} + Return { + new_state = es'.wasm_state; + new_calls = []; + response = NoResponse; + cycles_accepted = 0; + cycles_used = es.cycles_used + es'.cycles_used; + } + + Note that if the initial callback handler traps, the cleanup callback (if present) is executed, and the canister has the chance to update its state. + +- The `inspect_message` field of the `CanisterModule` is defined as follows. + + If the WebAssembly module does not export a function called under the name `canister_inspect_message`, then access is always granted: + + inspect_message = λ (method_name, wasm_state, arg, caller, sysenv) → + Return {status = Accept; cycles_used = 0;} + + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_inspect_message`, it is + + inspect_message = λ (method_name, wasm_state, arg, caller, sysenv) → + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = empty_params with { + arg = arg; + caller = caller; + method_name = method_name; + sysenv + } + balance = sysenv.balance; + cycles_available = 0; // ingress requests have no funds + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + Return {status = es.ingress_filter; cycles_used = es.cycles_used;}; + +#### Helper functions {#helper_functions} + +In the following section, we use the these helper functions + + copy_to_canister(dst : i32, offset : i32, size : i32, data : blob) = + if offset+size > |data| then Trap {cycles_used = es.cycles_used;} + if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + es.wasm_state.store.mem[dst..dst+size] := data[offset..offset+size] + + copy_from_canister(src : i32, size : i32) blob = + if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + return es.wasm_state.store.mem[src..src+size] + +Cycles are represented by 128-bit values so they require 16 bytes of memory. + + copy_cycles_to_canister(dst : i32, data : blob) = + let size = 16; + if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + es.wasm_state.store.mem[dst..dst+size] := data[0..size] + +#### System imports {#system_imports} + +Upon *instantiation* of the WebAssembly module, we can provide the following functions as imports. + +The pseudo-code below does *not* explicitly enforce the restrictions of which imports are available in which contexts; for that the table in [Overview of imports](#system-api-imports) is authoritative, and is assumed to be part of the implementation. + + ic0.msg_arg_data_size() : i32 = + return |es.params.arg| + + ic0.msg_arg_data_copy(dst:i32, offset:i32, size:i32) = + copy_to_canister(dst, offset, size, es.params.arg) + + ic0.msg_caller_size() : i32 = + return |es.params.caller| + + ic0.msg_caller_copy(dst:i32, offset:i32, size:i32) : i32 = + copy_to_canister(dst, offset, size, es.params.caller) + + ic0.msg_reject_code() : i32 = + es.params.reject_code + + ic0.msg_reject_msg_size() : i32 = + return |es.params.reject_msg| + + ic0.msg_reject_msg_copy(dst:i32, offset:i32, size:i32) : i32 = + copy_to_canister(dst, offset, size, es.params.reject_msg) + + ic0.msg_reply_data_append(src : i32, size : i32) = + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + es.reply_params.arg := es.reply_params.arg · copy_from_canister(src, size) + + ic0.msg_reply() = + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + es.response := Reply (es.reply_params.arg) + es.cycles_available := 0 + + ic0.msg_reject(src : i32, size : i32) = + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + es.response := Reject (CANISTER_REJECT, copy_from_canister(src, size)) + es.cycles_available := 0 + + ic0.msg_cycles_available() : i64 = + if es.cycles_available >= 2^64 then Trap {cycles_used = es.cycles_used;} + return es.cycles_available + + ic0.msg_cycles_available128(dst : i32) = + let amount = es.cycles_available + copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) + + ic0.msg_cycles_refunded() : i64 = + if es.params.cycles_refunded >= 2^64 then Trap {cycles_used = es.cycles_used;} + return es.params.cycles_refunded + + ic0.msg_cycles_refunded128(dst : i32) = + let amount = es.params.cycles_refunded + copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) + + ic0.accept_message() = + if es.ingress_filter = Accept then Trap {cycles_used = es.cycles_used;} + es.ingress_filter = Accept + + ic0.msg_method_name_size() : i32 = + return |es.method_name| + + ic0.msg_method_name_copy(dst : i32, offset : i32, size : i32) : i32 = + copy_to_canister(dst, offset, size, es.params.method_name) + + ic0.msg_cycles_accept(max_amount : i64) : i64 = + let amount = min(max_amount, es.cycles_available) + es.cycles_available := es.cycles_available - amount + es.cycles_accepted := es.cycles_accepted + amount + es.balance := es.balance + amount + return amount + + ic0.msg_cycles_accept128(max_amount_high : i64, max_amount_low : i64, dst : i32) = + let max_amount = max_amount_high * 2^64 + max_amount_low + let amount = min(max_amount, es.cycles_available) + es.cycles_available := es.cycles_available - amount + es.cycles_accepted := es.cycles_accepted + amount + es.balance := es.balance + amount + copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) + + ic0.canister_self_size() : i32 = + return |es.wasm_state.self_id| + + ic0.canister_self_copy(dst:i32, offset:i32, size:i32) = + copy_to_canister(dst, offset, size, es.wasm_state.self_id) + + ic0.canister_cycle_balance() : i64 = + if es.balance >= 2^64 then Trap {cycles_used = es.cycles_used;} + return es.balance + + ic0.canister_cycles_balance128(dst : i32) = + let amount = es.balance + copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) + + ic0.canister_status() : i32 = + match es.params.sysenv.canister_status with + Running -> return 1 + Stopping -> return 2 + Stopped -> return 3 + + ic0.call_new( + callee_src : i32, + callee_size : i32, + name_src : i32, + name_size : i32, + reply_fun : i32, + reply_env : i32, + reject_fun : i32, + reject_env : i32, + ) = + discard_pending_call() + + if es.balance < MAX_CYCLES_PER_RESPONSE then Trap {cycles_used = es.cycles_used;} + es.balance := es.balance - MAX_CYCLES_PER_RESPONSE + + callee := copy_from_canister(callee_src, callee_size); + method_name := copy_from_canister(name_src, name_size); + + if reply_fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} + if typeof(es.wasm_state.store.table[reply_fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} + + if reject_fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} + if typeof(es.wasm_state.store.table[reject_fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} + + es.pending_call = MethodCall { + callee = callee; + method_name = callee; + arg = ""; + transferred_cycles = 0; + callback = Callback { + on_reply = Closure { fun = reply_fun; env = reply_env } + on_reject = Closure { fun = reject_fun; env = reject_env } + on_cleanup = NoClosure + }; + } + + ic0.call_data_append (src : i32, size : i32) = + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + es.pending_call.arg := es.pending_call.arg · copy_from_canister(src, size) + + ic0.call_on_cleanup (fun : i32, env : i32) = + if fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} + if typeof(es.wasm_state.store.table[fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + if es.pending_call.callback.on_cleanup ≠ NoClosure then Trap {cycles_used = es.cycles_used;} + es.pending_call.callback.on_cleanup := Closure { fun = fun; env = env} + + ic0.call_cycles_add(amount : i64) = + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + if es.balance < amount then Trap {cycles_used = es.cycles_used;} + + es.balance := es.balance - amount + es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount + + ic0.call_cycles_add128(amount_high : i64, amount_low : i64) = + let amount = amount_high * 2^64 + amount_low + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + if es.balance < amount then Trap {cycles_used = es.cycles_used;} + + es.balance := es.balance - amount + es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount + + ic0.call_peform() : ( err_code : i32 ) = + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + + // are we below the threezing threshold? + // Or maybe the system has other reasons to not perform this + if es.balance < es.env.freezing_limit or system_cannot_do_this_call_now() + then + discard_pending_call() + return 1 + or + es.calls := es.calls · es.pending_call + es.pending_call := NoPendingCall + return 0 + + // helper function + discard_pending_call() = + if es.pending_call ≠ NoPendingCall then + es.balance := es.balance + MAX_CYCLES_PER_RESPONSE + es.pending_call.transferred_cycles + es.pending_call := NoPendingCall + + ic0.stable_size() : (page_count : i32) = + if |es.wasm_state.store.mem| > 2^32 then Trap {cycles_used = es.cycles_used;} + page_count := |es.wasm_state.stable_mem| / 64k + return page_count + + ic0.stable_grow(new_pages : i32) : (old_page_count : i32) = + if |es.wasm_state.store.mem| > 2^32 then Trap {cycles_used = es.cycles_used;} + if arbitrary() then return -1 + else + old_size := |es.wasm_state.stable_mem| / 64k + if old_size + new_pages > 2^16 then return -1 + es.wasm_state.stable_mem := + es.wasm_state.stable_mem · repeat(0x00, new_pages * 64k) + return old_size + + ic0.stable_write(offset : i32, src : i32, size : i32) + if |es.wasm_state.store.mem| > 2^32 then Trap {cycles_used = es.cycles_used;} + if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} + + es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] + + ic0.stable_read(dst : i32, offset : i32, size : i32) + if |es.wasm_state.store.mem| > 2^32 then Trap {cycles_used = es.cycles_used;} + if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} + if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + + es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] + + ic0.stable64_size() : (page_count : i64) = + return |es.wasm_state.stable_mem| / 64k + + ic0.stable64_grow(new_pages : i64) : (old_page_count : i64) = + if arbitrary() + then return -1 + else + old_size := |es.wasm_state.stable_mem| / 64k + es.wasm_state.stable_mem := + es.wasm_state.stable_mem · repeat(0x00, new_pages * 64k) + return old_size + + ic0.stable64_write(offset : i64, src : i64, size : i64) + if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} + + es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] + + ic0.stable64_read(dst : i64, offset : i64, size : i64) + if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} + if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + + es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] + + ic0.time() : i32 = + return es.params.time + + ic0.certified_data_set(src: i32, size: i32) = + es.new_certified_data := es.wasm_state[src..src+size] + + ic0.data_certificate_present() : i32 = + if es.params.sysenv.certificate = NoCertificate + then return 0 + else return 1 + + ic0.data_certificate_size() : i32 = + if es.params.sysenv.certificate = NoCertificate then Trap {cycles_used = es.cycles_used;} + return |es.params.sysenv.certificate| + + ic0.data_certificate_copy(dst: i32, offset: i32, size: i32) = + if es.params.sysenv.certificate = NoCertificate then Trap {cycles_used = es.cycles_used;} + copy_to_canister(dst, offset, size, es.params.sysenv.certificate) + + ic0.performance_counter(counter_type : i32) : i64 = + arbitrary() + + ic0.debug_print(src : i32, size : i32) = + return + + ic0.trap(src : i32, size : i32) = + Trap {cycles_used = es.cycles_used;} + + diff --git a/spec/md-spec/interface-spec-changelog.md b/spec/md-spec/interface-spec-changelog.md new file mode 100644 index 000000000..4e67c5699 --- /dev/null +++ b/spec/md-spec/interface-spec-changelog.md @@ -0,0 +1,258 @@ +## Changelog {#changelog} + +## 0.18.7 (2022-09-27) {#0_18_7} +* HTTP request API +* Reserved principals + +## 0.18.6 (2022-08-09) {#0_18_6} +* Canister access to performance metrics +* Query calls are rejected when the canister is frozen +* Support for implementation-specific error codes for requests +* Deleted call contexts do not prevent canister from reaching Stopped state +* Update effective canister id checks in certificate delegations +* Formal model in Isabelle + +## 0.18.5 (2022-07-08) {#0_18_5} +* Idle consumption of resources in cycles per day can be obtain via `canister_status` method of the management canister +* Include the HTTP Gateway Protocol in this spec +* Clarifications in definition of cycles consumption + +## 0.18.4 (2022-06-20) {#0_18_4} + +* Canister cycle balances are represented by 128 bits, and no system-defined upper limit exists anymore +* Canister modules can be gzip-encoded +* Expose Wasm custom sections in the state tree +* EXPERIMENTAL: Canister API for accessing Bitcoin transactions +* EXPERIMENTAL: Canister API for threshold ECDSA signatures + +### 0.18.3 (2022-01-10) {#0_18_3} + +* New System API which uses 128-bit values to represent the amount of cycles +* Subnet delegations include a canister id scope + +### 0.18.2 (2021-09-29) {#0_18_2} + +* Canister heartbeat +* Terminology changes +* Support for 64-bit stable memory + + +### 0.18.1 (2021-08-04) {#0_18_1} + +* Support RSA PKCS#1 v1.5 signatures in web authentication +* Spec clarification: Fix various typos and improve textual clarity + + +### 0.18.0 (2021-05-18) {#0_18_0} + +* A canister has a set of controllers, instead of always one + + +### 0.17.0 (2021-04-22) {#0_17_0} + +* Canister Signatures are introduced +* Spec clarification: the signature in the WebAuthn scheme is prefixed by the CBOR self-identifying tag +* Cycle-depleted canisters are forcibly uninstalled +* Canister settings in `create_canister` and `update_settings`. `install_code` no longer takes allocation settings. +* A freezing threshold can be configured via the canister settings + +### 0.16.1 (2021-04-14) {#0_16_1} +* The cleanup callback is introduced + +### 0.16.0 (2021-03-25) {#0_16_0} + +* New http v2 API that allows for stateless boundary nodes + +### 0.15.6 (2021-03-25) {#0_15_6} + +* The system may impose limits on the number of globals and functions +* No ingress messages towards empty canisters are accepted +* No ingress messages towards `raw_rand` and `deposit_cycles` are accepted +* A memory allocation of `0` means “best effort” + + +### 0.15.5 (2021-03-11) {#0_15_5} + +* deposit_cycles(): any caller allowed + + +### 0.15.4 (2021-03-04) {#0_15_4} + +* Ingress message filtering +* Add ECDSA signatures on curve secp256k1 +* Clarify that the `ic0.data_certificate_present` system function may be + called in all contexts. + + +### 0.15.3 (2021-02-26) {#0_15_3} + +* Expose module hash and controller via `read_state` + + +### 0.15.2 (2021-02-09) {#0_15_2} + +* The document is renamed to “Internet Computer Interface Spec” + + +### 0.15.0 (2020-12-17) {#0_15_0} + +* Support for raw Ed25519 keys is removed + + +### 0.14.1 (2020-12-08) {#0_14_1} + +* The default `memory_allocation` becomes unspecified + + +### 0.14.0 (2020-11-18) {#0_14_0} + +* Support for funds is scaled back to only support cycles +* The `ic0.msg_cycles_accept` system call now returns the actually accepted + cycles +* The `provisional_` management calls are introduced + + +### 0.13.2 (2020-11-12) {#0_13_2} + +* The `ic0.canister_status` system call + + +### 0.13.1 (2020-11-06) {#0_13_1} + +* Delegation between user public keys + + +### 0.13.0 (2020-10-19) {#0_13_0} + +* Certification (also removes “request-status” request) + + +### 0.12.2 (2020-10-23) {#0_12_2} + +* User authentication method based on WebAuthn is introduced +* User authentication can use ECDSA +* Public keys are DER-encoded + + +### 0.12.1 (2020-10-16) {#0_12_1} + +* Return more information in the `canister_status` management call + + +### 0.12.0 (2020-10-13) {#0_12_0} + +* Anonymous requests must have the sender field set + + +### 0.11.1 (2020-10-01) {#0_11_1} + +* The `deposit_funds` call + + +### 0.11.0 (2020-09-23) {#0_11_0} + +* Inter-canister calls are now performed using a builder-like API +* Support for funds (balances and transfers) + + +### 0.10.3 (2020-09-21) {#v0_10_3} + +* The anonymous user is introduced + + +### 0.10.1 (2020-09-01) {#v0_10_1} + +* Forward-port changes from 0.9.3 + + +### 0.10.0 (2020-08-06) {#v0_10_0} + +* Users can set/update a memory allocation when installing/upgrading a canister. +* The `expiry` field is added to requests + + +### 0.9.3 (2020-09-01) {#v0_9_3} + +* The management canister supports the `raw_rand` method + + +### 0.9.2 (2020-08-05) {#v0_9_2} + +* Canister controllers can stop/start canisters and can query their status. +* Canister controllers can delete canisters + + +### 0.9.1 (2020-07-20) {#v0_9_1} + +* Forward-port changes from 0.8.2 + + +### 0.9.0 (2020-07-15) {#v0_9_0} + +* Introduction of a domain separator (again) +* The calculation of “derived ids” has changed +* The self-authenticating and derived id forms use a truncated hash +* The textual representation of principals has changed + + +### 0.8.2 (2020-07-17) {#v0_8_2} + +* Installing code via `reinstall` works also on the empty canister + + +### 0.8.1 (2020-07-10) {#v0_8_1} + +* Reflect refined process in README and intro. +* `ic0.time` added + + +### 0.8.0 (2020-06-23) {#v0_8_0} + +* Revert the introduction of a domain separator + + +### 0.6.2 (2020-06-23) {#v0_6_2} + +* Fix meaning-changing typos in `ic.did` + + +### 0.6.0 (2020-06-08) {#v0_6_0} + +* Make all canister ids system-chosen +* HTTP requests for management features are removed + + +### 0.4.0 (2020-05-25) {#v0_4_0} + +* (editorial) the term “principal” is now used for the _id_ of a canister or + user, not the canister or user itself +* The signature of a request needs to be calculated using a domain separator +* Describe the `controller` attribute, add a request to change it +* The IC management canister is introduced + + +### 0.2.16 (2020-05-29) {#v0_2_16} + +* More tests about calls from query methods + + +### 0.2.14 (2020-05-14) {#v0_2_14} + +* Bugfix: Mode should be `reinstall`, not `replace` + + +### 0.2.8 (2020-04-23) {#v0_2_8} + +* Include section with CDDL description + + +### 0.2.4 (2020-03-23) {#v0_2_4} + +* simplify versioning (only three components), skip 0.2.2 to avoid confusion with 0.2.0.2 +* Clarification: `reply` field is always present +* General cleanup based on front-to-back reading + + +### 0.2.0.0 (2020-03-11) {#v0_2_0_0} + +* This is the first release. Subsequent releases will include a changelog. From 923bfa075918ac893635c7c9301582d92d92aac2 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Wed, 9 Nov 2022 12:08:55 +0100 Subject: [PATCH 040/102] Update changelog --- spec/changelog.adoc | 6 ++ spec/ic.did | 5 +- spec/ic0.txt | 2 +- spec/index.adoc | 137 +++++++++++++++++++++++++------------------- theories/IC.thy | 95 ++++++++++++++---------------- 5 files changed, 131 insertions(+), 114 deletions(-) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 0f3c28535..c423c3dcd 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,12 @@ [#changelog] == Changelog +[#0_18_8] +=== 0.18.8 (2022-11-09) +* Updated HTTP request API +* Canister status available to canister +* 64-bit stable memory is no longer experimental + [#0_18_7] === 0.18.7 (2022-09-27) * HTTP request API diff --git a/spec/ic.did b/spec/ic.did index e0efb2d97..a43bcbf2f 100644 --- a/spec/ic.did +++ b/spec/ic.did @@ -115,8 +115,9 @@ service ic : { method : variant { get; head; post }; headers: vec http_header; body : opt blob; - transform : opt variant { - function: func (http_response) -> (http_response) query + transform : opt record { + function : func (record {response : http_response; context : blob}) -> (http_response) query; + context : blob }; }) -> (http_response); diff --git a/spec/ic0.txt b/spec/ic0.txt index 53e9d2716..a37d2f8ff 100644 --- a/spec/ic0.txt +++ b/spec/ic0.txt @@ -62,4 +62,4 @@ ic0.time : () -> (timestamp : i64); // * ic0.performance_counter : (counter_type : i32) -> (counter : i64); // * s ic0.debug_print : (src : i32, size : i32) -> (); // * s -ic0.trap : (src : i32, size : i32) -> (); // * s \ No newline at end of file +ic0.trap : (src : i32, size : i32) -> (); // * s diff --git a/spec/index.adoc b/spec/index.adoc index 0084024dd..b192b8fc7 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -230,9 +230,9 @@ The canister status can be used to control whether the canister is processing ca In all cases, calls to the <> are processed, regardless of the state of the managed canister. -The controllers of the canister can initiate transitions between these states using <> and <>, and query the state using <>. The canister itself can also query its state using <>. +The controllers of the canister can initiate transitions between these states using <> and <>, and query the state using <> (NB: this call returns additional information, such as the cycle balance of the canister). The canister itself can also query its state using <>. -NOTE: This status is orthogonal to the question of whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected, but because the canister is empty. +NOTE: This status is orthogonal to whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected, but because the canister is empty. [#signatures] === Signatures @@ -548,24 +548,36 @@ The HTTP response to this request consists of a CBOR map with the following fiel * `certificate` (`blob`): A certificate (see <>). + -If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation’s canister id range (see <>), unless +If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation’s canister id range (see <>). -- the `effective_canister_id` is that of the Management Canister (`aaaaa-aa`), -- all requested paths have `/time` or `/request_status/` as prefix where the (single) original request referenced by `` is an update call to the Management Canister (`aaaaa-aa`) and the method name is `provisional_create_canister_with_cycles`, and -- whenever the certificate contains the path `/request_status//reply`, then its value is a Candid-encoded record with a `canister_id` field of type `principal` and the `canister_id` must be included in each delegation’s canister id range. - -The returned certificate reveals all values whose path is a suffix of the list of requested paths. It also always reveals `/time`, even if not explicitly requested. +The returned certificate reveals all values whose path is a suffix of a requested path. It also always reveals `/time`, even if not explicitly requested. All requested paths must have one of the following paths as prefix: - * `/time`. Can be requested by anyone. - * `/subnet`. Can be requested by anyone. - * `/request_status/`. Can only be requested by the same sender as the original request referenced by `` and if the effective canister id of the original request matches ``. - * `/canisters//module_hash`. Can be requested by anyone if `` matches ``. - * `/canisters//controllers`. Can be requested by anyone if `` matches ``. The order may vary depending on the implementation. - * `/canisters//metadata/`. Can be requested by anyone if `` matches `` and `` is a public custom section. If `` is a private custom section, it can only be requested by the controllers of the canister. + * `/time`. Can always be requested. + * `/subnet`. Can always be requested. + * `/request_status/`. Can be requested if no path with such a prefix exists in the state tree or + - the sender of the original request referenced by `` is the same as the sender of the read state request and + - the effective canister id of the original request referenced by `` matches ``. + * `/canisters//module_hash`. Can be requested if `` matches ``. + * `/canisters//controllers`. Can be requested if `` matches ``. The order of controllers in the value at this path may vary depending on the implementation. + * `/canisters//metadata/`. Can be requested if `` matches ``, `` is encoded in UTF-8, and + - canister with canister id `` does not exist or + - canister with canister id `` is empty or + - canister with canister id `` does not have `` as its custom section or + - `` is a public custom section or + - `` is a private custom section and the sender of the read state request is a controller of the canister. + +If a path cannot be requested, then the HTTP response to the read state request is undefined. + +Read state requests containing many requested paths might be rejected with a 4xx HTTP status code. + +[NOTE] +==== +The Internet Computer blockchain mainnet might reject read state requests that request more than 1 path with prefix `/request_status/`. +==== -Note that the paths `/canisters//certified_data` are not accessible with this method; these paths are only exposed to the canister themselves via the System API (see <>). +Note that the paths `/canisters//certified_data` are not accessible with this method; these paths are only exposed to the canisters themselves via the System API (see <>). See <> for details on the state tree. @@ -617,7 +629,7 @@ The Internet Computer blockchain mainnet rejects all requests whose effective ca The Internet Computer blockchain mainnet does not support `provisional_create_canister_with_cycles` and thus all calls to this method are rejected independently of the effective canister id. -In development instances of the Internet Computer Protocol (e.g. testnets), the effective canister id of a request submitted to a node must be a canister id from the canister ranges of the subnet to which the node belongs, unless the request is an update call to the Management Canister (`aaaaa-aa`), the method name is `provisional_create_canister_with_cycles`, and the effective canister id is that of the Management Canister (`aaaaa-aa`). +In development instances of the Internet Computer Protocol (e.g. testnets), the effective canister id of a request submitted to a node must be a canister id from the canister ranges of the subnet to which the node belongs. ==== [#authentication] @@ -851,7 +863,8 @@ NOTE: Applications can work around these problems. For the first problem, the qu [#canister-module-format] == Canister module format -A canister module is simply a https://webassembly.github.io/spec/core/index.html[WebAssembly module] in binary format (typically `.wasm`). +A canister module is a https://webassembly.github.io/spec/core/index.html[WebAssembly module] that is either in binary format (typically `.wasm`) or gzip-compressed (typically `.wasm.gz`). +If the module starts with byte sequence `[0x1f, 0x8b, 0x08]`, then the system decompresses the contents as a gzip stream according to https://datatracker.ietf.org/doc/html/rfc1952.html[RFC-1952] and then parses the output as a WebAssembly binary. [#system-api] == Canister interface (System API) @@ -1291,7 +1304,7 @@ This system call is experimental. It may be changed or removed in the future. Ca Canisters have the ability to store and retrieve data from a secondary memory. The purpose of this _stable memory_ is to provide space to store data beyond upgrades. The interface mirrors roughly the memory-related instructions of WebAssembly, and tries to be forward compatible with exposing this feature as an additional memory. -The stable memory is initially empty. +The stable memory is initially empty and can be grown up to 32 GiB (provided the subnet has capacity). * `ic0.stable_size : () -> (page_count : i32)` + @@ -1328,32 +1341,24 @@ It also traps if `dst+size` exceeds the size of the WebAssembly memory or `offse * `ic0.stable64_size : () -> (page_count : i64)` + returns the current size of the stable memory in WebAssembly pages. (One WebAssembly page is 64KiB) -+ -This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. * `ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64)` + tries to grow the memory by `new_pages` many pages containing zeroes. + If successful, returns the _previous_ size of the memory (in pages). Otherwise, returns `-1`. -+ -This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. * `ic0.stable64_write : (offset : i64, src : i64, size : i64) -> ()` + Copies the data from location [src, src+size) of the canister memory to location [offset, offset+size) in the stable memory. + This system call traps if `src+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. -+ -This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. * `ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> ()` + Copies the data from location [offset, offset+size) of the stable memory to the location [dst, dst+size) in the canister memory. + This system call traps if `dst+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. -+ -This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. [#system-api-time] === System time @@ -1538,10 +1543,10 @@ This is atomic: If the response to this request is a `reject`, then this call ha NOTE: Some canisters may not be able to make sense of callbacks after upgrades; these should be stopped first, to wait for all outstanding callbacks that are not marked as deleted, or be uninstalled first, to prevent outstanding callbacks from being invoked (by marking the corresponding call contexts as deleted). It is expected that the canister admin (or their tooling) does that separately. The `wasm_module` field specifies the canister module to be installed. -The system supports multiple encodings of the `wasm_module` field: +The system supports multiple encodings of the `wasm_module` field, as described in <>: * If the `wasm_module` starts with byte sequence `[0x00, 'a', 's', 'm']`, the system parses `wasm_module` as a raw WebAssembly binary. - * If the `wasm_module` starts with byte sequence `[0x1f, 0x8b, 0x08]`, the system decompresses the contents of `wasm_module` as a gzip stream according to https://datatracker.ietf.org/doc/html/rfc1952.html[RFC-1952] and then parses the output as a WebAssembly binary. + * If the `wasm_module` starts with byte sequence `[0x1f, 0x8b, 0x08]`, the system parses `wasm_module` as a gzip-compressed WebAssembly binary. [#ic-uninstall_code] === IC method `uninstall_code` @@ -1567,7 +1572,7 @@ Indicates various information about the canister. It contains: * The memory size taken by the canister. * The cycle balance of the canister. -Only the controllers of the canister can request its status. +Only the controllers of the canister or the canister itself can request its status. [#ic-stop_canister] === IC method `stop_canister` @@ -1664,11 +1669,12 @@ The `2MiB` size limit also applies to the value returned by the `transform` func The following parameters should be supplied for the call: -- `url` - the requested URL. The URL may specify a custom port number. However, only ports 80, 443, and 20000-65535 can be used. +- `url` - the requested URL. The URL may specify a custom port number. - `max_response_bytes` - optional, specifies the maximal size of the response in bytes. Any value less than or equal to `2MiB` is accepted. The call will be charged based on this parameter. If not provided, the maximum of `2MiB` will be used. - `method` - currently, only GET, HEAD, and POST are supported - `headers` - list of HTTP request headers and their corresponding values -- `transform` - an optional function that transforms raw responses to sanitized responses. If provided, the calling canister itself must export this function. +- `body` - optional, the content of the request's body +- `transform` - an optional record that includes a function that transforms raw responses to sanitized responses, and a byte-encoded context that is provided to the function upon invocation, along with the response to be sanitized. If provided, the calling canister itself must export this function. The returned response (and the response provided to the `transform` function, if specified) contains the following fields: @@ -1679,6 +1685,8 @@ The returned response (and the response provided to the `transform` function, if The `transform` function may, for example, transform the body in any way, add or remove headers, modify headers, etc. When the transform function was invoked due to a canister HTTP request, the caller's identity is the principal of the management canister. +NOTE: The identity of the caller for a valid invocation of the `transform` function is `aaaaa-aa`. This information can be used by developers to implement access control mechanism for this function. + [#ic-provisional_create_canister_with_cycles] === IC method `provisional_create_canister_with_cycles` @@ -2761,7 +2769,7 @@ Conditions:: State after:: .... - S.messages = Older_messages · Younger_messages · + messages = Older_messages · Younger_messages · ResponseMessage { origin = CM.origin; response = Reject (CANISTER_ERROR, "canister not running"); @@ -3078,7 +3086,10 @@ S with controllers[CanisterId] = A.settings.controllers else: controllers[CanisterId] = [M.caller] - freezing_threshold[CanisterId] = 2592000 + if A.settings.freezing_threshold is not null: + freezing_threshold[CanisterId] = A.settings.freezing_threshold + else: + freezing_threshold[CanisterId] = 2592000 balances[CanisterId] = M.transferred_cycles certified_data[CanisterId] = "" messages = Older_messages · Younger_messages · @@ -3148,7 +3159,7 @@ Conditions:: M.callee = ic_principal M.method_name = 'canister_status' M.arg = candid(A) - M.caller ∈ S.controllers[A.canister_id] + M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id} .... State after:: .... @@ -3361,8 +3372,8 @@ The status of a stopping canister which has no open call contexts that are not m Conditions:: .... - S.canister_status[Canister_id] = Stopping Origins - ∀ Ctxt_id. S.call_contexts[Ctxt_id].deleted or S.call_contexts[Ctxt_id].canister ≠ Canister_id + S.canister_status[CanisterId] = Stopping Origins + ∀ Ctxt_id. S.call_contexts[Ctxt_id].deleted or S.call_contexts[Ctxt_id].canister ≠ CanisterId .... State after:: .... @@ -3558,15 +3569,24 @@ State after:: S with canisters[CanisterId] = EmptyCanister time[CanisterId] = CurrentTime - controllers[CanisterId] = [M.caller] - freezing_threshold[CanisterId] = 2592000 - balances[CanisterId] = A.amount + if A.settings.controllers is not null: + controllers[CanisterId] = A.settings.controllers + else: + controllers[CanisterId] = [M.caller] + if A.settings.freezing_threshold is not null: + freezing_threshold[CanisterId] = A.settings.freezing_threshold + else: + freezing_threshold[CanisterId] = 2592000 + if A.amount is not null: + balances[CanisterId] = A.amount + else: + balances[CanisterId] = DEFAULT_PROVISIONAL_CYCLES_BALANCE certified_data[CanisterId] = "" messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin response = Reply (candid({canister_id = CanisterId})) - transferred_cycles = M.transferred_cycles + refunded_cycles = M.transferred_cycles } canister_status[CanisterId] = Running .... @@ -3820,7 +3840,7 @@ Conditions:: TS = verify_envelope(E, RS.sender, S.system_time) S.system_time <= RS.ingress_expiry ∀ path ∈ RS.paths. may_read_path(S, R.sender, path) - ∀ ["request_status", Rid] · _ ∈ RS.paths. ∃ R ∈ S.requests ∧ hash_of_map(R) = Rid ∧ R.canister_id ∈ TS + ∀ ["request_status", Rid] · _ ∈ RS.paths. ∀ R ∈ dom(S.requests). hash_of_map(R) = Rid => R.canister_id ∈ TS .... Read response:: A record with @@ -3828,50 +3848,51 @@ A record with The predicate `may_read_path` is defined as follows, implementing the access control outlined in <>: .... -may_read_path(S, _, ["time"]) = True +may_read_path(S, _, ["time"] · _) = True +may_read_path(S, _, ["subnet"] · _) = True may_read_path(S, _, ["request_status", Rid] · _) = - if ∃ R ∈ S.requests ∧ hash_of_map(R) = Rid - then RS.sender == R.sender ∧ is_effective_canister_id(R, ECID) - else True -may_read_path(S, _, ["canister", cid, "module_hash"]) = cid == ECID -may_read_path(S, _, ["canister", cid, "controllers"]) = cid == ECID -may_read_path(S, _, ["canister", cid, "metadata", name]) = - if name ∈ dom(S.canisters[cid].public_custom_sections) - then cid == ECID - else if name ∈ dom(S.canisters[cid].private_custom_sections) - then cid == ECID ∧ RS.sender ∈ S.controllers[cid] - else False + ∀ (R ↦ (_, ECID')) ∈ dom(S.requests). hash_of_map(R) = Rid => RS.sender == R.sender ∧ ECID == ECID' +may_read_path(S, _, ["canister", cid, "module_hash"] · _) = cid == ECID +may_read_path(S, _, ["canister", cid, "controllers"] · _) = cid == ECID +may_read_path(S, _, ["canister", cid, "metadata", name] · _) = cid == ECID ∧ UTF8(name) ∧ + (cid ∉ dom(S.canisters[cid]) ∨ + S.canisters[cid] = EmptyCanister ∨ + name ∉ (dom(S.canisters[cid].public_custom_sections) ∪ dom(S.canisters[cid].private_custom_sections)) ∨ + name ∈ dom(S.canisters[cid].public_custom_sections) ∨ + (name ∈ dom(S.canisters[cid].private_custom_sections) ∧ RS.sender ∈ S.controllers[cid]) + ) may_read_path(S, _, _) = False .... +where `UTF8(name)` holds if `name` is encoded in UTF-8. The response is a certificate `cert`, as specified in <>, which passes `verify_cert` (assuming `S.root_key` as the root of trust), and where for every `path` documented in <> that is a suffix of a path in `RS.paths` or of `["time"]`, we have .... lookup(path, cert) = lookup_in_tree(path, state_tree(S)) .... -where `state_tree` constructs the a labeled tree from the IC state `S` and the (so far underspecified) set of subnets `subnets`, as per <> +where `state_tree` constructs a labeled tree from the IC state `S` and the (so far underspecified) set of subnets `subnets`, as per <> .... state_tree(S) = { "time": S.system_time; - "request_id": { request_id(R): request_status_tree(S) | (R ↦ S) ∈ S.requests }; + "subnet": { subnet_id : { "public_key" : subnet_pk, "canister_ranges" : subnet_ranges } | (subnet_id, subnet_pk, subnet_ranges) ∈ subnets }; + "request_status": { request_id(R): request_status_tree(T) | (R ↦ (T, _)) ∈ S.requests }; "canister": { canister_id : { "module_hash" : SHA256(C.raw_module) | if C ≠ EmptyCanister } ∪ { "controllers" : CBOR(S.controllers[canister_id]) } ∪ { "metadata": { name: blob | (name, blob) ∈ S.canisters[canister_id].public_custom_sections ∪ S.canisters[canister_id].private_custom_sections } } | (canister_id, C) ∈ S.canisters }; - "subnet": { subnet_id : { "public_key" : pub } | (subnet_id, subnet_pk) ∈ subnets }; } request_status_tree(Received) = { "status": "received" } request_status_tree(Processing) = { "status": "processing" } -request_status_tree(Rejected (code,msg)) = +request_status_tree(Rejected (code, msg)) = { "status": "rejected"; "reject_code": code; "reject_message": msg, error_code: } request_status_tree(Replied arg) = { "status": "replied"; "reply": arg } request_status_tree(Done) = - { "status": "Done" } + { "status": "done" } .... and where `lookup_in_tree` is a function that returns the value or `Absent` as appropriately. diff --git a/theories/IC.thy b/theories/IC.thy index 4e542f128..73f422e94 100644 --- a/theories/IC.thy +++ b/theories/IC.thy @@ -1138,12 +1138,6 @@ proof - define no_response where "no_response = (case R of Inr result \ update_return.response result = None)" define older where "older = take n (messages S)" define younger where "younger = drop (Suc n) (messages S)" - have msgs: "messages S = older @ Func_message ctxt_id recv ep q # younger" - "take n older = older" "drop (Suc n) older = []" - "take (n - length older) ws = []" "drop (Suc n - length older) (w # ws) = ws" - for w and ws :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message list" - using id_take_nth_drop[of n "messages S"] assms - by (auto simp: message_execution_pre_def msg older_def younger_def) define S'' where "S'' = S\messages := take n (messages S) @ drop (Suc n) (messages S), balances := list_map_set (balances S) recv ((bal + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) - min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))\" define cond where "cond = (\isl R \ cyc_used \ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ @@ -1161,19 +1155,11 @@ proof - Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] cyc_used_def[symmetric] res[symmetric] New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric]) then show ?thesis - using assms(2) - apply (auto simp: ic_inv_def S''_def msgs split: message.splits call_origin.splits) - apply force - apply fast - apply force - apply fast - done + using assms(3)[OF msg prod Mod_def Is_response_def Env_def Available_def F_def R_def cyc_used_def res New_balance_def no_response_def, folded cond_def S''_def] False + by auto next case True define result where "result = projr R" - have R_Inr: "R = Inr result" - using True - by (auto simp: cond_def result_def split: option.splits) define response_messages where "response_messages = (case update_return.response result of None \ [] | Some resp \ [Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res)])" define new_call_to_message :: "(?'p, 'canid, 's, 'b, 'c) method_call \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message" where @@ -1188,12 +1174,6 @@ proof - define S' where "S' = S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), messages := messages, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, certified_data := certified_data, balances := list_map_set (balances S) recv New_balance\" - have cycles_accepted_res_def: "cycles_accepted_res = update_return.cycles_accepted result" - and new_calls_res_def: "new_calls_res = new_calls result" - using res - by (auto simp: R_Inr) - have no_response: "no_response = (update_return.response result = None)" - by (auto simp: no_response_def R_Inr) have msg_exec: "message_execution_post n S = S'" using True by (simp_all add: message_execution_post_def Let_def msg prod @@ -1201,37 +1181,45 @@ proof - New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric] messages_def[symmetric] new_ctxt_def[symmetric] certified_data_def[symmetric] S'_def[symmetric] result_def[symmetric] response_messages_def[symmetric] new_call_to_message_def[symmetric]) - have messages_msgs: "messages = older @ younger @ map new_call_to_message new_calls_res @ response_messages" - by (auto simp: messages_def older_def younger_def) - have ctxt_in_range: "ctxt \ list_map_range (call_contexts S)" - using prod(5) - by (simp add: list_map_get_range) - have response_msgsD: "msg \ set response_messages \ update_return.response result \ None \ - (\resp. msg = Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res))" for msg - by (auto simp: response_messages_def) metis - have call_ctxt_origin_new_ctxt: "call_ctxt_origin new_ctxt = call_ctxt_origin ctxt" - by (auto simp: new_ctxt_def split: option.splits) - have call_ctxt_needs_to_respond_new_ctxtD: "call_ctxt_needs_to_respond new_ctxt \ call_ctxt_needs_to_respond ctxt" - by (auto simp: new_ctxt_def split: option.splits) show ?thesis - using assms(2) ctxt_in_range True call_ctxt_not_needs_to_respond_available_cycles[of ctxt] - apply (auto simp: cond_def msg_exec S'_def ic_inv_def msgs messages_msgs new_call_to_message_def - no_response_def R_Inr call_ctxt_origin_new_ctxt - split: option.splits message.splits call_origin.splits - dest!: list_map_range_setD response_msgsD call_ctxt_needs_to_respond_new_ctxtD - intro: ic_can_status_inv_mono2) - apply blast - apply fast - apply blast - apply fast - apply blast - apply fast - apply blast - apply fast - done + using assms(2)[OF msg prod Mod_def Is_response_def Env_def Available_def F_def R_def cyc_used_def res New_balance_def no_response_def _ _ _ _ _ _ result_def + new_call_to_message_def response_messages_def messages_def new_ctxt_def certified_data_def, folded S'_def] True + by (auto simp: cond_def msg_exec) qed qed +lemma message_execution_ic_inv: + assumes "message_execution_pre n S" "ic_inv S" + shows "ic_inv (message_execution_post n S)" +proof (rule message_execution_cases[OF assms(1)]) + fix recv msgs ctxt_id new_ctxt cert_data New_balance new_calls_res response_messages ctxt Available cycles_accepted_res no_response idx + fix can :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) can_state_rec" + fix result :: "('w, 'p, 'canid, 's, 'b, 'c) update_return" + fix new_call_to_message :: "('p, 'canid, 's, 'b, 'c) method_call \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message" + fix R :: "unit cycles_return + ('w, 'p, 'canid, 's, 'b, 'c) update_return" + assume ctxt: "list_map_get (call_contexts S) ctxt_id = Some ctxt" + assume "msgs = take n (messages S) @ drop (Suc n) (messages S) @ map new_call_to_message new_calls_res @ response_messages" + "new_call_to_message = (\call. Call_message (From_canister ctxt_id (callback call)) (principal_of_canid recv) (callee call) (method_call.method_name call) (method_call.arg call) (transferred_cycles call) + (Queue (Canister recv) (callee call)))" + "response_messages = (case update_return.response result of None \ [] | Some resp \ [Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res)])" + "no_response \ call_ctxt_needs_to_respond ctxt" + "no_response = (case R of Inr result \ update_return.response result = None)" + "\ isl R" "result = projr R" + "new_ctxt = (case update_return.response result of None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt | Some x \ call_ctxt_respond ctxt)" + then show "ic_inv (S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), messages := msgs, + call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, certified_data := cert_data, balances := list_map_set (balances S) recv New_balance\)" + using assms(2) list_map_get_range[OF ctxt] + by (cases R) + (force simp: ic_inv_def ic_can_status_inv_def split: message.splits call_origin.splits can_status.splits dest!: in_set_takeD in_set_dropD list_map_range_setD)+ +next + fix recv bal Is_response cyc_used idx + show "ic_inv (S\messages := take n (messages S) @ drop (Suc n) (messages S), + balances := list_map_set (balances S) recv (bal + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + - min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))\)" + using assms(2) + by (auto simp: ic_inv_def split: message.splits call_origin.splits can_status.splits dest!: in_set_takeD in_set_dropD) +qed + (* System transition: Call context starvation [DONE] *) @@ -1447,7 +1435,7 @@ definition ic_canister_status_pre :: "nat \ nat \ ('p, ' cid \ list_map_dom (canister_status S) \ cid \ list_map_dom (balances S) \ cid \ list_map_dom (freezing_threshold S) \ - (case list_map_get (controllers S) cid of Some ctrls \ cer \ ctrls | _ \ False) | _ \ False) + (case list_map_get (controllers S) cid of Some ctrls \ cer \ ctrls \ {principal_of_canid cid} | _ \ False) | _ \ False) | _ \ False))" definition ic_canister_status_post :: "nat \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where @@ -1566,7 +1554,7 @@ lemma ic_code_installation_ic_inv: assumes "ic_code_installation_pre n S" "ic_inv S" shows "ic_inv (ic_code_installation_post n S)" proof - - obtain orig cer cee mn a trans_cycles q cid mode w ar m ctrls t bal can_status where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + obtain orig cer cee mn a trans_cycles q cid mode w ar m ctrls t bal can_status idx where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" and parse: "candid_parse_cid a = Some cid" "candid_parse_text a [encode_string ''mode''] = Some mode" "candid_parse_blob a [encode_string ''wasm_module''] = Some w" @@ -1666,7 +1654,7 @@ lemma ic_code_upgrade_ic_inv: assumes "ic_code_upgrade_pre n S" "ic_inv S" shows "ic_inv (ic_code_upgrade_post n S)" proof - - obtain orig cer cee mn a trans_cycles q cid mode w ar m can ctrls t bal can_status where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + obtain orig cer cee mn a trans_cycles q cid mode w ar m can ctrls t bal can_status idx where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" and parse: "candid_parse_cid a = Some cid" "candid_parse_text a [encode_string ''mode''] = Some mode" "candid_parse_blob a [encode_string ''wasm_module''] = Some w" @@ -1925,6 +1913,7 @@ definition ic_canister_stop_stopped_pre :: "nat \ ('p, 'uid, 'canid, definition ic_canister_stop_stopped_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where "ic_canister_stop_stopped_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a) in S\messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)" lemma ic_canister_stop_stopped_cycles_inv: @@ -2628,7 +2617,7 @@ definition cycle_consumption_pre :: "'canid \ nat \ ('p, | _ \ False)" definition cycle_consumption_post :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where - "cycle_consumption_post cid b1 S = (S\balances := list_map_set (balances S) cid b1\)" + "cycle_consumption_post cid b1 S = S\balances := list_map_set (balances S) cid b1\" definition cycle_consumption_burned_cycles :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where "cycle_consumption_burned_cycles cid b1 S = the (list_map_get (balances S) cid) - b1" From 0cc5d352bf621a7785c0c93a882902965ffe3ae2 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Wed, 9 Nov 2022 12:09:12 +0100 Subject: [PATCH 041/102] Bump version --- spec/index.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/index.adoc b/spec/index.adoc index b192b8fc7..4cb1f6820 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,6 +1,6 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.7 +0.18.8 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ From a45a2a1e3a398368c1a407f56ccbc9107a3e5972 Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Wed, 9 Nov 2022 14:57:25 +0100 Subject: [PATCH 042/102] fix formal model --- theories/IC.thy | 71 +++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 66 insertions(+), 5 deletions(-) diff --git a/theories/IC.thy b/theories/IC.thy index 73f422e94..d026c3f3e 100644 --- a/theories/IC.thy +++ b/theories/IC.thy @@ -1112,17 +1112,78 @@ proof - qed qed -lemma message_execution_ic_inv: - assumes "message_execution_pre n S" "ic_inv S" - shows "ic_inv (message_execution_post n S)" -proof - +lemma message_execution_cases: + assumes "message_execution_pre n S" + "\ctxt_id recv ep q can bal can_status t ctxt Mod Is_response Env Available F R cyc_used cycles_accepted_res new_calls_res New_balance no_response result new_call_to_message response_messages msgs new_ctxt cert_data idx. + messages S ! n = Func_message ctxt_id recv ep q \ list_map_get (canisters S) recv = Some (Some can) \ list_map_get (balances S) recv = Some bal \ + list_map_get (canister_status S) recv = Some can_status \ + list_map_get (time S) recv = Some t \ + list_map_get (call_contexts S) ctxt_id = Some ctxt \ + Mod = module can \ + Is_response = (case ep of Callback _ _ _ \ True | _ \ False) \ + Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\ \ + Available = call_ctxt_available_cycles ctxt \ + F = exec_function ep Env Available Mod \ + R = F (wasm_state can) \ + cyc_used = (case R of Inr res \ update_return.cycles_used res | Inl trap \ cycles_return.cycles_used trap) \ + (cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, update_return.new_calls res)) \ + New_balance = bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + - (cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res)) \ + no_response = (case R of Inr result \ update_return.response result = None) \ + \isl R \ cyc_used \ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + cycles_accepted_res \ Available \ + cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res) \ + bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + New_balance \ (if Is_response then 0 else ic_freezing_limit S recv) \ + (no_response \ call_ctxt_needs_to_respond ctxt) \ + result = projr R \ + new_call_to_message = (\call. Call_message (From_canister ctxt_id (method_call.callback call)) (principal_of_canid recv) + (method_call.callee call) (method_call.method_name call) (method_call.arg call) (method_call.transferred_cycles call) (Queue (Canister recv) (method_call.callee call))) \ + response_messages = (case update_return.response result of None \ [] + | Some resp \ [Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res)]) \ + msgs = take n (messages S) @ drop (Suc n) (messages S) @ map new_call_to_message new_calls_res @ response_messages \ + new_ctxt = (case update_return.response result of + None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt + | Some _ \ call_ctxt_respond ctxt) \ + cert_data = (case new_certified_data result of None \ certified_data S + | Some cd \ list_map_set (certified_data S) recv cd) \ + P n S (S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), + messages := msgs, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, + certified_data := cert_data, balances := list_map_set (balances S) recv New_balance\)" + "\ctxt_id recv ep q can bal can_status t ctxt Mod Is_response Env Available F R cyc_used cycles_accepted_res new_calls_res New_balance no_response idx. + messages S ! n = Func_message ctxt_id recv ep q \ list_map_get (canisters S) recv = Some (Some can) \ list_map_get (balances S) recv = Some bal \ + list_map_get (canister_status S) recv = Some can_status \ + list_map_get (time S) recv = Some t \ + list_map_get (call_contexts S) ctxt_id = Some ctxt \ + Mod = module can \ + Is_response = (case ep of Callback _ _ _ \ True | _ \ False) \ + Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\ \ + Available = call_ctxt_available_cycles ctxt \ + F = exec_function ep Env Available Mod \ + R = F (wasm_state can) \ + cyc_used = (case R of Inr res \ update_return.cycles_used res | Inl trap \ cycles_return.cycles_used trap) \ + (cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, update_return.new_calls res)) \ + New_balance = bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + - (cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res)) \ + no_response = (case R of Inr result \ update_return.response result = None) \ + \(\isl R \ cyc_used \ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + cycles_accepted_res \ Available \ + cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res) \ + bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + New_balance \ (if Is_response then 0 else ic_freezing_limit S recv) \ + (no_response \ call_ctxt_needs_to_respond ctxt)) \ + P n S (S\messages := take n (messages S) @ drop (Suc n) (messages S), + balances := list_map_set (balances S) recv ((bal + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) + - min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))\)" + shows "P n S (message_execution_post n S)" + proof - obtain ctxt_id recv ep q can bal can_status t ctxt where msg: "messages S ! n = Func_message ctxt_id recv ep q" and prod: "list_map_get (canisters S) recv = Some (Some can)" "list_map_get (balances S) recv = Some bal" "list_map_get (canister_status S) recv = Some can_status" "list_map_get (time S) recv = Some t" "list_map_get (call_contexts S) ctxt_id = Some ctxt" - using assms + using assms(1) by (auto simp: message_execution_pre_def split: message.splits option.splits) define Mod where "Mod = can_state_rec.module can" define Is_response where "Is_response = (case ep of Callback _ _ _ \ True | _ \ False)" From cc9188d1d964ae1c7bcaba98c56aa43b117c6cce Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Fri, 11 Nov 2022 09:47:33 +0100 Subject: [PATCH 043/102] Apply changes from release 0.18.8 --- spec/changelog.adoc | 6 ++ spec/ic.did | 5 +- spec/ic0.txt | 2 +- spec/index.adoc | 139 +++++++++++++++++++++---------------- theories/IC.thy | 166 ++++++++++++++++++++++++++++---------------- 5 files changed, 198 insertions(+), 120 deletions(-) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 0f3c28535..c423c3dcd 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,12 @@ [#changelog] == Changelog +[#0_18_8] +=== 0.18.8 (2022-11-09) +* Updated HTTP request API +* Canister status available to canister +* 64-bit stable memory is no longer experimental + [#0_18_7] === 0.18.7 (2022-09-27) * HTTP request API diff --git a/spec/ic.did b/spec/ic.did index e0efb2d97..a43bcbf2f 100644 --- a/spec/ic.did +++ b/spec/ic.did @@ -115,8 +115,9 @@ service ic : { method : variant { get; head; post }; headers: vec http_header; body : opt blob; - transform : opt variant { - function: func (http_response) -> (http_response) query + transform : opt record { + function : func (record {response : http_response; context : blob}) -> (http_response) query; + context : blob }; }) -> (http_response); diff --git a/spec/ic0.txt b/spec/ic0.txt index 53e9d2716..a37d2f8ff 100644 --- a/spec/ic0.txt +++ b/spec/ic0.txt @@ -62,4 +62,4 @@ ic0.time : () -> (timestamp : i64); // * ic0.performance_counter : (counter_type : i32) -> (counter : i64); // * s ic0.debug_print : (src : i32, size : i32) -> (); // * s -ic0.trap : (src : i32, size : i32) -> (); // * s \ No newline at end of file +ic0.trap : (src : i32, size : i32) -> (); // * s diff --git a/spec/index.adoc b/spec/index.adoc index 0084024dd..4cb1f6820 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,6 +1,6 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.7 +0.18.8 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ @@ -230,9 +230,9 @@ The canister status can be used to control whether the canister is processing ca In all cases, calls to the <> are processed, regardless of the state of the managed canister. -The controllers of the canister can initiate transitions between these states using <> and <>, and query the state using <>. The canister itself can also query its state using <>. +The controllers of the canister can initiate transitions between these states using <> and <>, and query the state using <> (NB: this call returns additional information, such as the cycle balance of the canister). The canister itself can also query its state using <>. -NOTE: This status is orthogonal to the question of whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected, but because the canister is empty. +NOTE: This status is orthogonal to whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected, but because the canister is empty. [#signatures] === Signatures @@ -548,24 +548,36 @@ The HTTP response to this request consists of a CBOR map with the following fiel * `certificate` (`blob`): A certificate (see <>). + -If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation’s canister id range (see <>), unless +If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation’s canister id range (see <>). -- the `effective_canister_id` is that of the Management Canister (`aaaaa-aa`), -- all requested paths have `/time` or `/request_status/` as prefix where the (single) original request referenced by `` is an update call to the Management Canister (`aaaaa-aa`) and the method name is `provisional_create_canister_with_cycles`, and -- whenever the certificate contains the path `/request_status//reply`, then its value is a Candid-encoded record with a `canister_id` field of type `principal` and the `canister_id` must be included in each delegation’s canister id range. - -The returned certificate reveals all values whose path is a suffix of the list of requested paths. It also always reveals `/time`, even if not explicitly requested. +The returned certificate reveals all values whose path is a suffix of a requested path. It also always reveals `/time`, even if not explicitly requested. All requested paths must have one of the following paths as prefix: - * `/time`. Can be requested by anyone. - * `/subnet`. Can be requested by anyone. - * `/request_status/`. Can only be requested by the same sender as the original request referenced by `` and if the effective canister id of the original request matches ``. - * `/canisters//module_hash`. Can be requested by anyone if `` matches ``. - * `/canisters//controllers`. Can be requested by anyone if `` matches ``. The order may vary depending on the implementation. - * `/canisters//metadata/`. Can be requested by anyone if `` matches `` and `` is a public custom section. If `` is a private custom section, it can only be requested by the controllers of the canister. + * `/time`. Can always be requested. + * `/subnet`. Can always be requested. + * `/request_status/`. Can be requested if no path with such a prefix exists in the state tree or + - the sender of the original request referenced by `` is the same as the sender of the read state request and + - the effective canister id of the original request referenced by `` matches ``. + * `/canisters//module_hash`. Can be requested if `` matches ``. + * `/canisters//controllers`. Can be requested if `` matches ``. The order of controllers in the value at this path may vary depending on the implementation. + * `/canisters//metadata/`. Can be requested if `` matches ``, `` is encoded in UTF-8, and + - canister with canister id `` does not exist or + - canister with canister id `` is empty or + - canister with canister id `` does not have `` as its custom section or + - `` is a public custom section or + - `` is a private custom section and the sender of the read state request is a controller of the canister. + +If a path cannot be requested, then the HTTP response to the read state request is undefined. + +Read state requests containing many requested paths might be rejected with a 4xx HTTP status code. + +[NOTE] +==== +The Internet Computer blockchain mainnet might reject read state requests that request more than 1 path with prefix `/request_status/`. +==== -Note that the paths `/canisters//certified_data` are not accessible with this method; these paths are only exposed to the canister themselves via the System API (see <>). +Note that the paths `/canisters//certified_data` are not accessible with this method; these paths are only exposed to the canisters themselves via the System API (see <>). See <> for details on the state tree. @@ -617,7 +629,7 @@ The Internet Computer blockchain mainnet rejects all requests whose effective ca The Internet Computer blockchain mainnet does not support `provisional_create_canister_with_cycles` and thus all calls to this method are rejected independently of the effective canister id. -In development instances of the Internet Computer Protocol (e.g. testnets), the effective canister id of a request submitted to a node must be a canister id from the canister ranges of the subnet to which the node belongs, unless the request is an update call to the Management Canister (`aaaaa-aa`), the method name is `provisional_create_canister_with_cycles`, and the effective canister id is that of the Management Canister (`aaaaa-aa`). +In development instances of the Internet Computer Protocol (e.g. testnets), the effective canister id of a request submitted to a node must be a canister id from the canister ranges of the subnet to which the node belongs. ==== [#authentication] @@ -851,7 +863,8 @@ NOTE: Applications can work around these problems. For the first problem, the qu [#canister-module-format] == Canister module format -A canister module is simply a https://webassembly.github.io/spec/core/index.html[WebAssembly module] in binary format (typically `.wasm`). +A canister module is a https://webassembly.github.io/spec/core/index.html[WebAssembly module] that is either in binary format (typically `.wasm`) or gzip-compressed (typically `.wasm.gz`). +If the module starts with byte sequence `[0x1f, 0x8b, 0x08]`, then the system decompresses the contents as a gzip stream according to https://datatracker.ietf.org/doc/html/rfc1952.html[RFC-1952] and then parses the output as a WebAssembly binary. [#system-api] == Canister interface (System API) @@ -1291,7 +1304,7 @@ This system call is experimental. It may be changed or removed in the future. Ca Canisters have the ability to store and retrieve data from a secondary memory. The purpose of this _stable memory_ is to provide space to store data beyond upgrades. The interface mirrors roughly the memory-related instructions of WebAssembly, and tries to be forward compatible with exposing this feature as an additional memory. -The stable memory is initially empty. +The stable memory is initially empty and can be grown up to 32 GiB (provided the subnet has capacity). * `ic0.stable_size : () -> (page_count : i32)` + @@ -1328,32 +1341,24 @@ It also traps if `dst+size` exceeds the size of the WebAssembly memory or `offse * `ic0.stable64_size : () -> (page_count : i64)` + returns the current size of the stable memory in WebAssembly pages. (One WebAssembly page is 64KiB) -+ -This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. * `ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64)` + tries to grow the memory by `new_pages` many pages containing zeroes. + If successful, returns the _previous_ size of the memory (in pages). Otherwise, returns `-1`. -+ -This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. * `ic0.stable64_write : (offset : i64, src : i64, size : i64) -> ()` + Copies the data from location [src, src+size) of the canister memory to location [offset, offset+size) in the stable memory. + This system call traps if `src+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. -+ -This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. * `ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> ()` + Copies the data from location [offset, offset+size) of the stable memory to the location [dst, dst+size) in the canister memory. + This system call traps if `dst+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. -+ -This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. [#system-api-time] === System time @@ -1538,10 +1543,10 @@ This is atomic: If the response to this request is a `reject`, then this call ha NOTE: Some canisters may not be able to make sense of callbacks after upgrades; these should be stopped first, to wait for all outstanding callbacks that are not marked as deleted, or be uninstalled first, to prevent outstanding callbacks from being invoked (by marking the corresponding call contexts as deleted). It is expected that the canister admin (or their tooling) does that separately. The `wasm_module` field specifies the canister module to be installed. -The system supports multiple encodings of the `wasm_module` field: +The system supports multiple encodings of the `wasm_module` field, as described in <>: * If the `wasm_module` starts with byte sequence `[0x00, 'a', 's', 'm']`, the system parses `wasm_module` as a raw WebAssembly binary. - * If the `wasm_module` starts with byte sequence `[0x1f, 0x8b, 0x08]`, the system decompresses the contents of `wasm_module` as a gzip stream according to https://datatracker.ietf.org/doc/html/rfc1952.html[RFC-1952] and then parses the output as a WebAssembly binary. + * If the `wasm_module` starts with byte sequence `[0x1f, 0x8b, 0x08]`, the system parses `wasm_module` as a gzip-compressed WebAssembly binary. [#ic-uninstall_code] === IC method `uninstall_code` @@ -1567,7 +1572,7 @@ Indicates various information about the canister. It contains: * The memory size taken by the canister. * The cycle balance of the canister. -Only the controllers of the canister can request its status. +Only the controllers of the canister or the canister itself can request its status. [#ic-stop_canister] === IC method `stop_canister` @@ -1664,11 +1669,12 @@ The `2MiB` size limit also applies to the value returned by the `transform` func The following parameters should be supplied for the call: -- `url` - the requested URL. The URL may specify a custom port number. However, only ports 80, 443, and 20000-65535 can be used. +- `url` - the requested URL. The URL may specify a custom port number. - `max_response_bytes` - optional, specifies the maximal size of the response in bytes. Any value less than or equal to `2MiB` is accepted. The call will be charged based on this parameter. If not provided, the maximum of `2MiB` will be used. - `method` - currently, only GET, HEAD, and POST are supported - `headers` - list of HTTP request headers and their corresponding values -- `transform` - an optional function that transforms raw responses to sanitized responses. If provided, the calling canister itself must export this function. +- `body` - optional, the content of the request's body +- `transform` - an optional record that includes a function that transforms raw responses to sanitized responses, and a byte-encoded context that is provided to the function upon invocation, along with the response to be sanitized. If provided, the calling canister itself must export this function. The returned response (and the response provided to the `transform` function, if specified) contains the following fields: @@ -1679,6 +1685,8 @@ The returned response (and the response provided to the `transform` function, if The `transform` function may, for example, transform the body in any way, add or remove headers, modify headers, etc. When the transform function was invoked due to a canister HTTP request, the caller's identity is the principal of the management canister. +NOTE: The identity of the caller for a valid invocation of the `transform` function is `aaaaa-aa`. This information can be used by developers to implement access control mechanism for this function. + [#ic-provisional_create_canister_with_cycles] === IC method `provisional_create_canister_with_cycles` @@ -2761,7 +2769,7 @@ Conditions:: State after:: .... - S.messages = Older_messages · Younger_messages · + messages = Older_messages · Younger_messages · ResponseMessage { origin = CM.origin; response = Reject (CANISTER_ERROR, "canister not running"); @@ -3078,7 +3086,10 @@ S with controllers[CanisterId] = A.settings.controllers else: controllers[CanisterId] = [M.caller] - freezing_threshold[CanisterId] = 2592000 + if A.settings.freezing_threshold is not null: + freezing_threshold[CanisterId] = A.settings.freezing_threshold + else: + freezing_threshold[CanisterId] = 2592000 balances[CanisterId] = M.transferred_cycles certified_data[CanisterId] = "" messages = Older_messages · Younger_messages · @@ -3148,7 +3159,7 @@ Conditions:: M.callee = ic_principal M.method_name = 'canister_status' M.arg = candid(A) - M.caller ∈ S.controllers[A.canister_id] + M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id} .... State after:: .... @@ -3361,8 +3372,8 @@ The status of a stopping canister which has no open call contexts that are not m Conditions:: .... - S.canister_status[Canister_id] = Stopping Origins - ∀ Ctxt_id. S.call_contexts[Ctxt_id].deleted or S.call_contexts[Ctxt_id].canister ≠ Canister_id + S.canister_status[CanisterId] = Stopping Origins + ∀ Ctxt_id. S.call_contexts[Ctxt_id].deleted or S.call_contexts[Ctxt_id].canister ≠ CanisterId .... State after:: .... @@ -3558,15 +3569,24 @@ State after:: S with canisters[CanisterId] = EmptyCanister time[CanisterId] = CurrentTime - controllers[CanisterId] = [M.caller] - freezing_threshold[CanisterId] = 2592000 - balances[CanisterId] = A.amount + if A.settings.controllers is not null: + controllers[CanisterId] = A.settings.controllers + else: + controllers[CanisterId] = [M.caller] + if A.settings.freezing_threshold is not null: + freezing_threshold[CanisterId] = A.settings.freezing_threshold + else: + freezing_threshold[CanisterId] = 2592000 + if A.amount is not null: + balances[CanisterId] = A.amount + else: + balances[CanisterId] = DEFAULT_PROVISIONAL_CYCLES_BALANCE certified_data[CanisterId] = "" messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin response = Reply (candid({canister_id = CanisterId})) - transferred_cycles = M.transferred_cycles + refunded_cycles = M.transferred_cycles } canister_status[CanisterId] = Running .... @@ -3820,7 +3840,7 @@ Conditions:: TS = verify_envelope(E, RS.sender, S.system_time) S.system_time <= RS.ingress_expiry ∀ path ∈ RS.paths. may_read_path(S, R.sender, path) - ∀ ["request_status", Rid] · _ ∈ RS.paths. ∃ R ∈ S.requests ∧ hash_of_map(R) = Rid ∧ R.canister_id ∈ TS + ∀ ["request_status", Rid] · _ ∈ RS.paths. ∀ R ∈ dom(S.requests). hash_of_map(R) = Rid => R.canister_id ∈ TS .... Read response:: A record with @@ -3828,50 +3848,51 @@ A record with The predicate `may_read_path` is defined as follows, implementing the access control outlined in <>: .... -may_read_path(S, _, ["time"]) = True +may_read_path(S, _, ["time"] · _) = True +may_read_path(S, _, ["subnet"] · _) = True may_read_path(S, _, ["request_status", Rid] · _) = - if ∃ R ∈ S.requests ∧ hash_of_map(R) = Rid - then RS.sender == R.sender ∧ is_effective_canister_id(R, ECID) - else True -may_read_path(S, _, ["canister", cid, "module_hash"]) = cid == ECID -may_read_path(S, _, ["canister", cid, "controllers"]) = cid == ECID -may_read_path(S, _, ["canister", cid, "metadata", name]) = - if name ∈ dom(S.canisters[cid].public_custom_sections) - then cid == ECID - else if name ∈ dom(S.canisters[cid].private_custom_sections) - then cid == ECID ∧ RS.sender ∈ S.controllers[cid] - else False + ∀ (R ↦ (_, ECID')) ∈ dom(S.requests). hash_of_map(R) = Rid => RS.sender == R.sender ∧ ECID == ECID' +may_read_path(S, _, ["canister", cid, "module_hash"] · _) = cid == ECID +may_read_path(S, _, ["canister", cid, "controllers"] · _) = cid == ECID +may_read_path(S, _, ["canister", cid, "metadata", name] · _) = cid == ECID ∧ UTF8(name) ∧ + (cid ∉ dom(S.canisters[cid]) ∨ + S.canisters[cid] = EmptyCanister ∨ + name ∉ (dom(S.canisters[cid].public_custom_sections) ∪ dom(S.canisters[cid].private_custom_sections)) ∨ + name ∈ dom(S.canisters[cid].public_custom_sections) ∨ + (name ∈ dom(S.canisters[cid].private_custom_sections) ∧ RS.sender ∈ S.controllers[cid]) + ) may_read_path(S, _, _) = False .... +where `UTF8(name)` holds if `name` is encoded in UTF-8. The response is a certificate `cert`, as specified in <>, which passes `verify_cert` (assuming `S.root_key` as the root of trust), and where for every `path` documented in <> that is a suffix of a path in `RS.paths` or of `["time"]`, we have .... lookup(path, cert) = lookup_in_tree(path, state_tree(S)) .... -where `state_tree` constructs the a labeled tree from the IC state `S` and the (so far underspecified) set of subnets `subnets`, as per <> +where `state_tree` constructs a labeled tree from the IC state `S` and the (so far underspecified) set of subnets `subnets`, as per <> .... state_tree(S) = { "time": S.system_time; - "request_id": { request_id(R): request_status_tree(S) | (R ↦ S) ∈ S.requests }; + "subnet": { subnet_id : { "public_key" : subnet_pk, "canister_ranges" : subnet_ranges } | (subnet_id, subnet_pk, subnet_ranges) ∈ subnets }; + "request_status": { request_id(R): request_status_tree(T) | (R ↦ (T, _)) ∈ S.requests }; "canister": { canister_id : { "module_hash" : SHA256(C.raw_module) | if C ≠ EmptyCanister } ∪ { "controllers" : CBOR(S.controllers[canister_id]) } ∪ { "metadata": { name: blob | (name, blob) ∈ S.canisters[canister_id].public_custom_sections ∪ S.canisters[canister_id].private_custom_sections } } | (canister_id, C) ∈ S.canisters }; - "subnet": { subnet_id : { "public_key" : pub } | (subnet_id, subnet_pk) ∈ subnets }; } request_status_tree(Received) = { "status": "received" } request_status_tree(Processing) = { "status": "processing" } -request_status_tree(Rejected (code,msg)) = +request_status_tree(Rejected (code, msg)) = { "status": "rejected"; "reject_code": code; "reject_message": msg, error_code: } request_status_tree(Replied arg) = { "status": "replied"; "reply": arg } request_status_tree(Done) = - { "status": "Done" } + { "status": "done" } .... and where `lookup_in_tree` is a function that returns the value or `Absent` as appropriately. diff --git a/theories/IC.thy b/theories/IC.thy index 4e542f128..d026c3f3e 100644 --- a/theories/IC.thy +++ b/theories/IC.thy @@ -1112,17 +1112,78 @@ proof - qed qed -lemma message_execution_ic_inv: - assumes "message_execution_pre n S" "ic_inv S" - shows "ic_inv (message_execution_post n S)" -proof - +lemma message_execution_cases: + assumes "message_execution_pre n S" + "\ctxt_id recv ep q can bal can_status t ctxt Mod Is_response Env Available F R cyc_used cycles_accepted_res new_calls_res New_balance no_response result new_call_to_message response_messages msgs new_ctxt cert_data idx. + messages S ! n = Func_message ctxt_id recv ep q \ list_map_get (canisters S) recv = Some (Some can) \ list_map_get (balances S) recv = Some bal \ + list_map_get (canister_status S) recv = Some can_status \ + list_map_get (time S) recv = Some t \ + list_map_get (call_contexts S) ctxt_id = Some ctxt \ + Mod = module can \ + Is_response = (case ep of Callback _ _ _ \ True | _ \ False) \ + Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\ \ + Available = call_ctxt_available_cycles ctxt \ + F = exec_function ep Env Available Mod \ + R = F (wasm_state can) \ + cyc_used = (case R of Inr res \ update_return.cycles_used res | Inl trap \ cycles_return.cycles_used trap) \ + (cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, update_return.new_calls res)) \ + New_balance = bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + - (cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res)) \ + no_response = (case R of Inr result \ update_return.response result = None) \ + \isl R \ cyc_used \ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + cycles_accepted_res \ Available \ + cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res) \ + bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + New_balance \ (if Is_response then 0 else ic_freezing_limit S recv) \ + (no_response \ call_ctxt_needs_to_respond ctxt) \ + result = projr R \ + new_call_to_message = (\call. Call_message (From_canister ctxt_id (method_call.callback call)) (principal_of_canid recv) + (method_call.callee call) (method_call.method_name call) (method_call.arg call) (method_call.transferred_cycles call) (Queue (Canister recv) (method_call.callee call))) \ + response_messages = (case update_return.response result of None \ [] + | Some resp \ [Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res)]) \ + msgs = take n (messages S) @ drop (Suc n) (messages S) @ map new_call_to_message new_calls_res @ response_messages \ + new_ctxt = (case update_return.response result of + None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt + | Some _ \ call_ctxt_respond ctxt) \ + cert_data = (case new_certified_data result of None \ certified_data S + | Some cd \ list_map_set (certified_data S) recv cd) \ + P n S (S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), + messages := msgs, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, + certified_data := cert_data, balances := list_map_set (balances S) recv New_balance\)" + "\ctxt_id recv ep q can bal can_status t ctxt Mod Is_response Env Available F R cyc_used cycles_accepted_res new_calls_res New_balance no_response idx. + messages S ! n = Func_message ctxt_id recv ep q \ list_map_get (canisters S) recv = Some (Some can) \ list_map_get (balances S) recv = Some bal \ + list_map_get (canister_status S) recv = Some can_status \ + list_map_get (time S) recv = Some t \ + list_map_get (call_contexts S) ctxt_id = Some ctxt \ + Mod = module can \ + Is_response = (case ep of Callback _ _ _ \ True | _ \ False) \ + Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\ \ + Available = call_ctxt_available_cycles ctxt \ + F = exec_function ep Env Available Mod \ + R = F (wasm_state can) \ + cyc_used = (case R of Inr res \ update_return.cycles_used res | Inl trap \ cycles_return.cycles_used trap) \ + (cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, update_return.new_calls res)) \ + New_balance = bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + - (cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res)) \ + no_response = (case R of Inr result \ update_return.response result = None) \ + \(\isl R \ cyc_used \ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + cycles_accepted_res \ Available \ + cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res) \ + bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ + New_balance \ (if Is_response then 0 else ic_freezing_limit S recv) \ + (no_response \ call_ctxt_needs_to_respond ctxt)) \ + P n S (S\messages := take n (messages S) @ drop (Suc n) (messages S), + balances := list_map_set (balances S) recv ((bal + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) + - min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))\)" + shows "P n S (message_execution_post n S)" + proof - obtain ctxt_id recv ep q can bal can_status t ctxt where msg: "messages S ! n = Func_message ctxt_id recv ep q" and prod: "list_map_get (canisters S) recv = Some (Some can)" "list_map_get (balances S) recv = Some bal" "list_map_get (canister_status S) recv = Some can_status" "list_map_get (time S) recv = Some t" "list_map_get (call_contexts S) ctxt_id = Some ctxt" - using assms + using assms(1) by (auto simp: message_execution_pre_def split: message.splits option.splits) define Mod where "Mod = can_state_rec.module can" define Is_response where "Is_response = (case ep of Callback _ _ _ \ True | _ \ False)" @@ -1138,12 +1199,6 @@ proof - define no_response where "no_response = (case R of Inr result \ update_return.response result = None)" define older where "older = take n (messages S)" define younger where "younger = drop (Suc n) (messages S)" - have msgs: "messages S = older @ Func_message ctxt_id recv ep q # younger" - "take n older = older" "drop (Suc n) older = []" - "take (n - length older) ws = []" "drop (Suc n - length older) (w # ws) = ws" - for w and ws :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message list" - using id_take_nth_drop[of n "messages S"] assms - by (auto simp: message_execution_pre_def msg older_def younger_def) define S'' where "S'' = S\messages := take n (messages S) @ drop (Suc n) (messages S), balances := list_map_set (balances S) recv ((bal + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) - min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))\" define cond where "cond = (\isl R \ cyc_used \ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) \ @@ -1161,19 +1216,11 @@ proof - Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] cyc_used_def[symmetric] res[symmetric] New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric]) then show ?thesis - using assms(2) - apply (auto simp: ic_inv_def S''_def msgs split: message.splits call_origin.splits) - apply force - apply fast - apply force - apply fast - done + using assms(3)[OF msg prod Mod_def Is_response_def Env_def Available_def F_def R_def cyc_used_def res New_balance_def no_response_def, folded cond_def S''_def] False + by auto next case True define result where "result = projr R" - have R_Inr: "R = Inr result" - using True - by (auto simp: cond_def result_def split: option.splits) define response_messages where "response_messages = (case update_return.response result of None \ [] | Some resp \ [Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res)])" define new_call_to_message :: "(?'p, 'canid, 's, 'b, 'c) method_call \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message" where @@ -1188,12 +1235,6 @@ proof - define S' where "S' = S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), messages := messages, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, certified_data := certified_data, balances := list_map_set (balances S) recv New_balance\" - have cycles_accepted_res_def: "cycles_accepted_res = update_return.cycles_accepted result" - and new_calls_res_def: "new_calls_res = new_calls result" - using res - by (auto simp: R_Inr) - have no_response: "no_response = (update_return.response result = None)" - by (auto simp: no_response_def R_Inr) have msg_exec: "message_execution_post n S = S'" using True by (simp_all add: message_execution_post_def Let_def msg prod @@ -1201,37 +1242,45 @@ proof - New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric] messages_def[symmetric] new_ctxt_def[symmetric] certified_data_def[symmetric] S'_def[symmetric] result_def[symmetric] response_messages_def[symmetric] new_call_to_message_def[symmetric]) - have messages_msgs: "messages = older @ younger @ map new_call_to_message new_calls_res @ response_messages" - by (auto simp: messages_def older_def younger_def) - have ctxt_in_range: "ctxt \ list_map_range (call_contexts S)" - using prod(5) - by (simp add: list_map_get_range) - have response_msgsD: "msg \ set response_messages \ update_return.response result \ None \ - (\resp. msg = Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res))" for msg - by (auto simp: response_messages_def) metis - have call_ctxt_origin_new_ctxt: "call_ctxt_origin new_ctxt = call_ctxt_origin ctxt" - by (auto simp: new_ctxt_def split: option.splits) - have call_ctxt_needs_to_respond_new_ctxtD: "call_ctxt_needs_to_respond new_ctxt \ call_ctxt_needs_to_respond ctxt" - by (auto simp: new_ctxt_def split: option.splits) show ?thesis - using assms(2) ctxt_in_range True call_ctxt_not_needs_to_respond_available_cycles[of ctxt] - apply (auto simp: cond_def msg_exec S'_def ic_inv_def msgs messages_msgs new_call_to_message_def - no_response_def R_Inr call_ctxt_origin_new_ctxt - split: option.splits message.splits call_origin.splits - dest!: list_map_range_setD response_msgsD call_ctxt_needs_to_respond_new_ctxtD - intro: ic_can_status_inv_mono2) - apply blast - apply fast - apply blast - apply fast - apply blast - apply fast - apply blast - apply fast - done + using assms(2)[OF msg prod Mod_def Is_response_def Env_def Available_def F_def R_def cyc_used_def res New_balance_def no_response_def _ _ _ _ _ _ result_def + new_call_to_message_def response_messages_def messages_def new_ctxt_def certified_data_def, folded S'_def] True + by (auto simp: cond_def msg_exec) qed qed +lemma message_execution_ic_inv: + assumes "message_execution_pre n S" "ic_inv S" + shows "ic_inv (message_execution_post n S)" +proof (rule message_execution_cases[OF assms(1)]) + fix recv msgs ctxt_id new_ctxt cert_data New_balance new_calls_res response_messages ctxt Available cycles_accepted_res no_response idx + fix can :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) can_state_rec" + fix result :: "('w, 'p, 'canid, 's, 'b, 'c) update_return" + fix new_call_to_message :: "('p, 'canid, 's, 'b, 'c) method_call \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message" + fix R :: "unit cycles_return + ('w, 'p, 'canid, 's, 'b, 'c) update_return" + assume ctxt: "list_map_get (call_contexts S) ctxt_id = Some ctxt" + assume "msgs = take n (messages S) @ drop (Suc n) (messages S) @ map new_call_to_message new_calls_res @ response_messages" + "new_call_to_message = (\call. Call_message (From_canister ctxt_id (callback call)) (principal_of_canid recv) (callee call) (method_call.method_name call) (method_call.arg call) (transferred_cycles call) + (Queue (Canister recv) (callee call)))" + "response_messages = (case update_return.response result of None \ [] | Some resp \ [Response_message (call_ctxt_origin ctxt) resp (Available - cycles_accepted_res)])" + "no_response \ call_ctxt_needs_to_respond ctxt" + "no_response = (case R of Inr result \ update_return.response result = None)" + "\ isl R" "result = projr R" + "new_ctxt = (case update_return.response result of None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt | Some x \ call_ctxt_respond ctxt)" + then show "ic_inv (S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), messages := msgs, + call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, certified_data := cert_data, balances := list_map_set (balances S) recv New_balance\)" + using assms(2) list_map_get_range[OF ctxt] + by (cases R) + (force simp: ic_inv_def ic_can_status_inv_def split: message.splits call_origin.splits can_status.splits dest!: in_set_takeD in_set_dropD list_map_range_setD)+ +next + fix recv bal Is_response cyc_used idx + show "ic_inv (S\messages := take n (messages S) @ drop (Suc n) (messages S), + balances := list_map_set (balances S) recv (bal + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + - min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))\)" + using assms(2) + by (auto simp: ic_inv_def split: message.splits call_origin.splits can_status.splits dest!: in_set_takeD in_set_dropD) +qed + (* System transition: Call context starvation [DONE] *) @@ -1447,7 +1496,7 @@ definition ic_canister_status_pre :: "nat \ nat \ ('p, ' cid \ list_map_dom (canister_status S) \ cid \ list_map_dom (balances S) \ cid \ list_map_dom (freezing_threshold S) \ - (case list_map_get (controllers S) cid of Some ctrls \ cer \ ctrls | _ \ False) | _ \ False) + (case list_map_get (controllers S) cid of Some ctrls \ cer \ ctrls \ {principal_of_canid cid} | _ \ False) | _ \ False) | _ \ False))" definition ic_canister_status_post :: "nat \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where @@ -1566,7 +1615,7 @@ lemma ic_code_installation_ic_inv: assumes "ic_code_installation_pre n S" "ic_inv S" shows "ic_inv (ic_code_installation_post n S)" proof - - obtain orig cer cee mn a trans_cycles q cid mode w ar m ctrls t bal can_status where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + obtain orig cer cee mn a trans_cycles q cid mode w ar m ctrls t bal can_status idx where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" and parse: "candid_parse_cid a = Some cid" "candid_parse_text a [encode_string ''mode''] = Some mode" "candid_parse_blob a [encode_string ''wasm_module''] = Some w" @@ -1666,7 +1715,7 @@ lemma ic_code_upgrade_ic_inv: assumes "ic_code_upgrade_pre n S" "ic_inv S" shows "ic_inv (ic_code_upgrade_post n S)" proof - - obtain orig cer cee mn a trans_cycles q cid mode w ar m can ctrls t bal can_status where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + obtain orig cer cee mn a trans_cycles q cid mode w ar m can ctrls t bal can_status idx where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" and parse: "candid_parse_cid a = Some cid" "candid_parse_text a [encode_string ''mode''] = Some mode" "candid_parse_blob a [encode_string ''wasm_module''] = Some w" @@ -1925,6 +1974,7 @@ definition ic_canister_stop_stopped_pre :: "nat \ ('p, 'uid, 'canid, definition ic_canister_stop_stopped_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where "ic_canister_stop_stopped_post n S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ + let cid = the (candid_parse_cid a) in S\messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)" lemma ic_canister_stop_stopped_cycles_inv: @@ -2628,7 +2678,7 @@ definition cycle_consumption_pre :: "'canid \ nat \ ('p, | _ \ False)" definition cycle_consumption_post :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where - "cycle_consumption_post cid b1 S = (S\balances := list_map_set (balances S) cid b1\)" + "cycle_consumption_post cid b1 S = S\balances := list_map_set (balances S) cid b1\" definition cycle_consumption_burned_cycles :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where "cycle_consumption_burned_cycles cid b1 S = the (list_map_get (balances S) cid) - b1" From 6f9ee0f483b6532126859385e1c5d4070aff5ed0 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Fri, 11 Nov 2022 15:51:32 +0100 Subject: [PATCH 044/102] Convert adoc to md --- spec/md-spec/ic-interface-spec.md | 134 ++++++++++++++--------- spec/md-spec/interface-spec-changelog.md | 5 + 2 files changed, 87 insertions(+), 52 deletions(-) diff --git a/spec/md-spec/ic-interface-spec.md b/spec/md-spec/ic-interface-spec.md index a01431743..bc716168b 100644 --- a/spec/md-spec/ic-interface-spec.md +++ b/spec/md-spec/ic-interface-spec.md @@ -239,7 +239,8 @@ The canister status can be used to control whether the canister is processing ca In all cases, calls to the [management canister](#the-ic-management-canister) are processed, regardless of the state of the managed canister. -The controllers of the canister can initiate transitions between these states using [`stop_canister`](#ic-stop_canister) and [`start_canister`](#ic-start_canister), and query the state using [`canister_status`](#ic-canister_status). The canister itself can also query its state using [`ic0.canister_status`](#system-api-canister-status). +The controllers of the canister can initiate transitions between these states using [`stop_canister`](#ic-stop_canister) and [`start_canister`](#ic-start_canister), and query the state using [`canister_status`](#ic-canister_status). The canister itself can also query its state using [`ic0.canister_status`](#system-api-canister-status). (NB: this call returns additional information, such as the cycle balance of the canister). The canister itself can also query its state using [`ic0.canister_status`](#system-api-canister-status). + :::note This status is orthogonal to the question of whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected, but because the canister is empty. @@ -554,31 +555,47 @@ The HTTP response to this request consists of a CBOR map with the following fiel - `certificate` (`blob`): A certificate (see [Certification](#certification)). - If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation's canister id range (see [Delegation](#certification-delegation)), unless + If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation's canister id range (see [Delegation](#certification-delegation)). - - the `effective_canister_id` is that of the Management Canister (`aaaaa-aa`), +The returned certificate reveals all values whose path is a suffix of a requested path. It also always reveals `/time`, even if not explicitly requested. - - all requested paths have `/time` or `/request_status/` as prefix where the (single) original request referenced by `` is an update call to the Management Canister (`aaaaa-aa`) and the method name is `provisional_create_canister_with_cycles`, and +All requested paths must have one of the following paths as prefix: - - whenever the certificate contains the path `/request_status//reply`, then its value is a Candid-encoded record with a `canister_id` field of type `principal` and the `canister_id` must be included in each delegation's canister id range. +- `/time`. Can always be requested. -The returned certificate reveals all values whose path is a suffix of the list of requested paths. It also always reveals `/time`, even if not explicitly requested. +- `/subnet`. Can always be requested. -All requested paths must have one of the following paths as prefix: +- `/request_status/`. Can be requested if no path with such a prefix exists in the state tree or + + - the sender of the original request referenced by `` is the same as the sender of the read state request and + + - the effective canister id of the original request referenced by `` matches ``. -- `/time`. Can be requested by anyone. +-`/canisters//module_hash`. Can be requested if `` matches ``. -- `/subnet`. Can be requested by anyone. +-`/canisters//controllers`. Can be requested if `` matches ``. The order of controllers in the value at this path may vary depending on the implementation. -- `/request_status/`. Can only be requested by the same sender as the original request referenced by `` and if the effective canister id of the original request matches ``. +-`/canisters//metadata/`. Can be requested if `` matches ``, `` is encoded in UTF-8, and -- `/canisters//module_hash`. Can be requested by anyone if `` matches ``. + - canister with canister id `` does not exist or -- `/canisters//controllers`. Can be requested by anyone if `` matches ``. The order may vary depending on the implementation. + - canister with canister id `` is empty or -- `/canisters//metadata/`. Can be requested by anyone if `` matches `` and `` is a public custom section. If `` is a private custom section, it can only be requested by the controllers of the canister. + - canister with canister id `` does not have `` as its custom section or -Note that the paths `/canisters//certified_data` are not accessible with this method; these paths are only exposed to the canister themselves via the System API (see [Certified data](#system-api-certified-data)). + - `` is a public custom section or + + - `` is a private custom section and the sender of the read state request is a controller of the canister. + +If a path cannot be requested, then the HTTP response to the read state request is undefined. + +Read state requests containing many requested paths might be rejected with a 4xx HTTP status code. + +:::note +The Internet Computer blockchain mainnet might reject read state requests that request more than 1 path with prefix `/request_status/`. +::: + +Note that the paths `/canisters//certified_data` are not accessible with this method; these paths are only exposed to the canisters themselves via the System API (see [Certified data](#system-api-certified-data)). See [The system state tree](#the-system-state-tree) for details on the state tree. @@ -637,7 +654,7 @@ The `` in the URL paths of requests is the *effective* de The Internet Computer blockchain mainnet does not support `provisional_create_canister_with_cycles` and thus all calls to this method are rejected independently of the effective canister id. - In development instances of the Internet Computer Protocol (e.g. testnets), the effective canister id of a request submitted to a node must be a canister id from the canister ranges of the subnet to which the node belongs, unless the request is an update call to the Management Canister (`aaaaa-aa`), the method name is `provisional_create_canister_with_cycles`, and the effective canister id is that of the Management Canister (`aaaaa-aa`). + In development instances of the Internet Computer Protocol (e.g. testnets), the effective canister id of a request submitted to a node must be a canister id from the canister ranges of the subnet to which the node belongs. ::: ### Authentication @@ -877,6 +894,7 @@ Applications can work around these problems. For the first problem, the query re ## Canister module format A canister module is simply a [WebAssembly module](https://webassembly.github.io/spec/core/index.html) in binary format (typically `.wasm`). +If the module starts with byte sequence `[0x1f, 0x8b, 0x08]`, then the system decompresses the contents as a gzip stream according to [RFC-1952](https://datatracker.ietf.org/doc/html/rfc1952.html) and then parses the output as a WebAssembly binary. ## Canister interface (System API) {#system-api} @@ -1422,32 +1440,24 @@ The stable memory is initially empty. returns the current size of the stable memory in WebAssembly pages. (One WebAssembly page is 64KiB) - This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. - - `ic0.stable64_grow : (new_pages : i64) → (old_page_count : i64)` tries to grow the memory by `new_pages` many pages containing zeroes. If successful, returns the *previous* size of the memory (in pages). Otherwise, returns `-1`. - This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. - - `ic0.stable64_write : (offset : i64, src : i64, size : i64) → ()` Copies the data from location \[src, src+size) of the canister memory to location \[offset, offset+size) in the stable memory. This system call traps if `src+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. - This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. - - `ic0.stable64_read : (dst : i64, offset : i64, size : i64) → ()` Copies the data from location \[offset, offset+size) of the stable memory to the location \[dst, dst+size) in the canister memory. This system call traps if `dst+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. - This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. - ### System time {#system-api-time} The canister can query the IC for the current time. @@ -1624,7 +1634,7 @@ This is atomic: If the response to this request is a `reject`, then this call ha Some canisters may not be able to make sense of callbacks after upgrades; these should be stopped first, to wait for all outstanding callbacks that are not marked as deleted, or be uninstalled first, to prevent outstanding callbacks from being invoked (by marking the corresponding call contexts as deleted). It is expected that the canister admin (or their tooling) does that separately. ::: -The `wasm_module` field specifies the canister module to be installed. The system supports multiple encodings of the `wasm_module` field: +The `wasm_module` field specifies the canister module to be installed. The system supports multiple encodings of the `wasm_module` field, as described in [Canister module format](#canister-module-format): - If the `wasm_module` starts with byte sequence `[0x00, 'a', 's', 'm']`, the system parses `wasm_module` as a raw WebAssembly binary. @@ -1656,7 +1666,7 @@ Indicates various information about the canister. It contains: - The cycle balance of the canister. -Only the controllers of the canister can request its status. +Only the controllers of the canister or the canister itself can request its status. ### IC method `stop_canister` {#ic-stop_canister} @@ -1739,7 +1749,7 @@ Each request can specify the maximal expected size for the response from the rem The following parameters should be supplied for the call: -- `url` - the requested URL +- `url` - the requested URL. The URL may specify a custom port number. - `max_response_bytes` - optional, specifies the maximal size of the response in bytes. Any value less than or equal to `2MiB` is accepted. The call will be charged based on this parameter. If not provided, the maximum of `2MiB` will be used. @@ -1747,7 +1757,9 @@ The following parameters should be supplied for the call: - `headers` - list of HTTP request headers and their corresponding values -- `transform` - an optional function that transforms raw responses to sanitized responses. If provided, the calling canister itself must export this function. +- `body` - optional, the content of the request's body + +- `transform` - an optional function that transforms raw responses to sanitized responses, and a byte-encoded context that is provided to the function upon invocation, along with the response to be sanitized. If provided, the calling canister itself must export this function. The returned response (and the response provided to the `transform` function, if specified) contains the following fields: @@ -1759,6 +1771,10 @@ The returned response (and the response provided to the `transform` function, if The `transform` function may, for example, transform the body in any way, add or remove headers, modify headers, etc. When the transform function was invoked due to a canister HTTP request, the caller's identity is the principal of the management canister. +:::note +The identity of the caller for a valid invocation of the `transform` function is `aaaaa-aa`. This information can be used by developers to implement access control mechanism for this function. +::: + ### IC method `provisional_create_canister_with_cycles` {#ic-provisional_create_canister_with_cycles} As a provisional method on development instances, the `provisional_create_canister_with_cycles` method is provided. It behaves as `create_canister`, but initializes the canister's balance with `amount` fresh cycles (using `DEFAULT_PROVISIONAL_CYCLES_BALANCE` if `amount = null`). @@ -2774,7 +2790,7 @@ S.canister_status[CM.callee] = Stopped or S.canister_status[CM.callee] = Stoppin State after ```html -S.messages = Older_messages · Younger_messages · +messages = Older_messages · Younger_messages · ResponseMessage { origin = CM.origin; response = Reject (CANISTER_ERROR, "canister not running"); @@ -3100,7 +3116,10 @@ S with controllers[CanisterId] = A.settings.controllers else: controllers[CanisterId] = [M.caller] - freezing_threshold[CanisterId] = 2592000 + if A.settings.freezing_threshold is not null: + freezing_threshold[CanisterId] = A.settings.freezing_threshold + else: + freezing_threshold[CanisterId] = 2592000 balances[CanisterId] = M.transferred_cycles certified_data[CanisterId] = "" messages = Older_messages · Younger_messages · @@ -3180,7 +3199,7 @@ S.messages = Older_messages · CallMessage M · Younger_messages M.callee = ic_principal M.method_name = 'canister_status' M.arg = candid(A) -M.caller ∈ S.controllers[A.canister_id] +M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id} ``` @@ -3419,7 +3438,7 @@ Conditions ```html S.canister_status[Canister_id] = Stopping Origins - ∀ Ctxt_id. S.call_contexts[Ctxt_id].deleted or S.call_contexts[Ctxt_id].canister ≠ Canister_id + ∀ Ctxt_id. S.call_contexts[Ctxt_id].deleted or S.call_contexts[Ctxt_id].canister ≠ CanisterId ``` @@ -3653,9 +3672,18 @@ State after S with canisters[CanisterId] = EmptyCanister time[CanisterId] = CurrentTime - controllers[CanisterId] = [M.caller] - freezing_threshold[CanisterId] = 2592000 - balances[CanisterId] = A.amount + if A.settings.controllers is not null: + controllers[CanisterId] = A.settings.controllers + else: + controllers[CanisterId] = [M.caller] + if A.settings.freezing_threshold is not null: + freezing_threshold[CanisterId] = A.settings.freezing_threshold + else: + freezing_threshold[CanisterId] = 2592000 + if A.amount is not null: + balances[CanisterId] = A.amount + else: + balances[CanisterId] = DEFAULT_PROVISIONAL_CYCLES_BALANCE certified_data[CanisterId] = "" messages = Older_messages · Younger_messages · ResponseMessage { @@ -3968,7 +3996,7 @@ E.content = ReadState RS TS = verify_envelope(E, RS.sender, S.system_time) S.system_time <= RS.ingress_expiry ∀ path ∈ RS.paths. may_read_path(S, R.sender, path) -∀ ["request_status", Rid] · _ ∈ RS.paths. ∃ R ∈ S.requests ∧ hash_of_map(R) = Rid ∧ R.canister_id ∈ TS +∀ ["request_status", Rid] · _ ∈ RS.paths. ∃ R ∈ dom(S.requests) ∧ hash_of_map(R) = Rid ∧ R.canister_id ∈ TS ``` @@ -3979,49 +4007,51 @@ Read response The predicate `may_read_path` is defined as follows, implementing the access control outlined in [Request: Read state](#http-read-state): - may_read_path(S, _, ["time"]) = True + may_read_path(S, _, ["time"] · _) = True + may_read_path(S, _, ["subnet"] · _) = True + ∀ (R ↦ (_, ECID')) ∈ dom(S.requests). hash_of_map(R) = Rid => RS.sender == R.sender ∧ ECID == ECID' + may_read_path(S, _, ["canister", cid, "module_hash"] · _) = cid == ECID + may_read_path(S, _, ["canister", cid, "controllers"] · _) = cid == ECID + may_read_path(S, _, ["canister", cid, "metadata", name] · _) = cid == ECID ∧ UTF8(name) ∧ + (cid ∉ dom(S.canisters[cid]) ∨ + S.canisters[cid] = EmptyCanister ∨ + name ∉ (dom(S.canisters[cid].public_custom_sections) ∪ dom(S.canisters[cid].private_custom_sections)) ∨ + name ∈ dom(S.canisters[cid].public_custom_sections) ∨ + (name ∈ dom(S.canisters[cid].private_custom_sections) ∧ RS.sender ∈ S.controllers[cid]) + ) may_read_path(S, _, ["request_status", Rid] · _) = - if ∃ R ∈ S.requests ∧ hash_of_map(R) = Rid - then RS.sender == R.sender ∧ is_effective_canister_id(R, ECID) - else True - may_read_path(S, _, ["canister", cid, "module_hash"]) = cid == ECID - may_read_path(S, _, ["canister", cid, "controllers"]) = cid == ECID - may_read_path(S, _, ["canister", cid, "metadata", name]) = - if name ∈ dom(S.canisters[cid].public_custom_sections) - then cid == ECID - else if name ∈ dom(S.canisters[cid].private_custom_sections) - then cid == ECID ∧ RS.sender ∈ S.controllers[cid] - else False may_read_path(S, _, _) = False +where `UTF8(name)` holds if `name` is encoded in UTF-8. + The response is a certificate `cert`, as specified in [Certification](#certification), which passes `verify_cert` (assuming `S.root_key` as the root of trust), and where for every `path` documented in [The system state tree](#state-tree) that is a suffix of a path in `RS.paths` or of `["time"]`, we have lookup(path, cert) = lookup_in_tree(path, state_tree(S)) -where `state_tree` constructs the a labeled tree from the IC state `S` and the (so far underspecified) set of subnets `subnets`, as per [The system state tree](#state-tree) +where `state_tree` constructs a labeled tree from the IC state `S` and the (so far underspecified) set of subnets `subnets`, as per [The system state tree](#state-tree) state_tree(S) = { "time": S.system_time; - "request_id": { request_id(R): request_status_tree(S) | (R ↦ S) ∈ S.requests }; + "subnet": { subnet_id : { "public_key" : subnet_pk, "canister_ranges" : subnet_ranges } | (subnet_id, subnet_pk, subnet_ranges) ∈ subnets }; + "request_status": { request_id(R): request_status_tree(T) | (R ↦ (T, _)) ∈ S.requests }; "canister": { canister_id : { "module_hash" : SHA256(C.raw_module) | if C ≠ EmptyCanister } ∪ { "controllers" : CBOR(S.controllers[canister_id]) } ∪ { "metadata": { name: blob | (name, blob) ∈ S.canisters[canister_id].public_custom_sections ∪ S.canisters[canister_id].private_custom_sections } } | (canister_id, C) ∈ S.canisters }; - "subnet": { subnet_id : { "public_key" : pub } | (subnet_id, subnet_pk) ∈ subnets }; } request_status_tree(Received) = { "status": "received" } request_status_tree(Processing) = { "status": "processing" } - request_status_tree(Rejected (code,msg)) = + request_status_tree(Rejected (code, msg)) = { "status": "rejected"; "reject_code": code; "reject_message": msg, error_code: } request_status_tree(Replied arg) = { "status": "replied"; "reply": arg } request_status_tree(Done) = - { "status": "Done" } + { "status": "done" } and where `lookup_in_tree` is a function that returns the value or `Absent` as appropriately. diff --git a/spec/md-spec/interface-spec-changelog.md b/spec/md-spec/interface-spec-changelog.md index 4e67c5699..25bafe578 100644 --- a/spec/md-spec/interface-spec-changelog.md +++ b/spec/md-spec/interface-spec-changelog.md @@ -1,5 +1,10 @@ ## Changelog {#changelog} +## 0.18.8 (2022-11-09) {#0_18_8} +* Updated HTTP request API +* Canister status available to canister +* 64-bit stable memory is no longer experimental + ## 0.18.7 (2022-09-27) {#0_18_7} * HTTP request API * Reserved principals From 18bc24e152e3eae4e1d4595dd896dc766be32433 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Tue, 6 Dec 2022 16:41:06 +0100 Subject: [PATCH 045/102] new release --- spec/changelog.adoc | 6 + spec/ic0.txt | 15 +- spec/index.adoc | 430 ++++++++++++++++++++++++++++++++++---------- 3 files changed, 348 insertions(+), 103 deletions(-) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index c423c3dcd..366abc2f2 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,12 @@ [#changelog] == Changelog +[#0_18_9] +=== 0.18.9 (2022-12-06) +* Global timers +* Canister version +* Clarifications for HTTP requests & Bitcoin integration costs + [#0_18_8] === 0.18.8 (2022-11-09) * Updated HTTP request API diff --git a/spec/ic0.txt b/spec/ic0.txt index a37d2f8ff..a553089d3 100644 --- a/spec/ic0.txt +++ b/spec/ic0.txt @@ -28,7 +28,7 @@ ic0.msg_method_name_size : () -> i32; // F ic0.msg_method_name_copy : (dst : i32, offset : i32, size : i32) -> (); // F ic0.accept_message : () -> (); // F -ic0.call_new : // U Ry Rt H +ic0.call_new : // U Ry Rt T ( callee_src : i32, callee_size : i32, name_src : i32, @@ -38,11 +38,11 @@ ic0.call_new : // U reject_fun : i32, reject_env : i32 ) -> (); -ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt H -ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt H -ic0.call_cycles_add : (amount : i64) -> (); // U Ry Rt H -ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt H -ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt H +ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt T +ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt T +ic0.call_cycles_add : (amount : i64) -> (); // U Ry Rt T +ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt T +ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt T ic0.stable_size : () -> (page_count : i32); // * ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32); // * @@ -53,12 +53,13 @@ ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64); // * ic0.stable64_write : (offset : i64, src : i64, size : i64) -> (); // * ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> (); // * -ic0.certified_data_set : (src: i32, size: i32) -> (); // I G U Ry Rt H +ic0.certified_data_set : (src: i32, size: i32) -> (); // I G U Ry Rt T ic0.data_certificate_present : () -> i32; // * ic0.data_certificate_size : () -> i32; // * ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> (); // * ic0.time : () -> (timestamp : i64); // * +ic0.global_timer_set : (timestamp : i64) -> i64; // I U Ry Rt C T ic0.performance_counter : (counter_type : i32) -> (counter : i64); // * s ic0.debug_print : (src : i32, size : i32) -> (); // * s diff --git a/spec/index.adoc b/spec/index.adoc index 4cb1f6820..b120d2266 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,6 +1,6 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.8 +0.18.9 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ @@ -225,14 +225,14 @@ NOTE: Once the IC frees the resources of a canister, its id, _cycles_ balance, a The canister status can be used to control whether the canister is processing calls: * In status `running`, calls to the canister are processed as normal. -* In status `stopping`, calls to the canister are rejected by the IC, but responses to the canister are processed as normal. -* In status `stopped`, calls to the canister are rejected by the IC, and there are no outstanding responses to call contexts that are not marked as deleted. +* In status `stopping`, calls to the canister are rejected by the IC with reject code `CANISTER_ERROR` (5), but responses to the canister are processed as normal. +* In status `stopped`, calls to the canister are rejected by the IC with reject code `CANISTER_ERROR` (5), and there are no outstanding responses to call contexts that are not marked as deleted. In all cases, calls to the <> are processed, regardless of the state of the managed canister. The controllers of the canister can initiate transitions between these states using <> and <>, and query the state using <> (NB: this call returns additional information, such as the cycle balance of the canister). The canister itself can also query its state using <>. -NOTE: This status is orthogonal to whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected, but because the canister is empty. +NOTE: This status is orthogonal to whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected by the IC with an HTTP error, but because the canister is empty. [#signatures] === Signatures @@ -886,6 +886,7 @@ In order for a WebAssembly module to be usable as the code for the canister, it * If it exports a function called `canister_init`, the function must have type `+() -> ()+`. * If it exports a function called `canister_inspect_message`, the function must have type `+() -> ()+`. * If it exports a function called `canister_heartbeat`, the function must have type `+() -> ()+`. +* If it exports a function called `canister_global_timer`, the function must have type `+() -> ()+`. * If it exports any functions called `canister_update ` or `canister_query ` for some `name`, the functions must have type `+() -> ()+`. * It may not export both `canister_update ` and `canister_query ` with the same `name`. * It may not export other methods the names of which start with the prefix `canister_` besides the methods allowed above. @@ -911,6 +912,7 @@ The canister provides entry points which are invoked by the IC under various cir * The canister may export a function with name `canister_post_upgrade` and type `+() -> ()+`. * The canister may export functions with name `canister_inspect_message` with type `+() -> ()+`. * The canister may export a function with name `canister_heartbeat` with type `+() -> ()+`. +* The canister may export a function with name `canister_global_timer` with type `+() -> ()+`. * The canister may export functions with name `canister_update ` and type `+() -> ()+`. * The canister may export functions with name `canister_query ` and type `+() -> ()+`. * The canister table may contain functions of type `+(env : i32) -> ()+` which may be used as callbacks for inter-canister calls. @@ -955,10 +957,20 @@ Eventually, a method will want to send a response, using `ic0.reply` or `ic0.rej For periodic or time-based execution, the WebAssembly module can export a function with name `canister_heartbeat`. The heartbeats scheduling algorithm is implementation-defined. -`canister_heartbeat` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. +`canister_heartbeat` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. Still, the function `canister_heartbeat` can initiate new calls. NOTE: While an implementation will likely try to keep the interval between `canister_heartbeat` invocations to within a few seconds, this is not formally part of this specification. +==== Global timer + +For time-based execution, the WebAssembly module can export a function with name `canister_global_timer`. This function is called if the canister has set its global timer (using the System API function `ic0.global_timer_set`) and the current time (as returned by the System API function `ic0.time`) has passed the value of the global timer. + +Once the function `canister_global_timer` is scheduled, the canister's global timer is deactivated. The global timer is also deactivated upon changes to the canister's Wasm module (calling `install_code`, `uninstall_code` methods of the management canister or if the canister runs out of cycles). In particular, the function `canister_global_timer` won't be scheduled again unless the canister sets the global timer again (using the System API function `ic0.global_timer_set`). The global timer scheduling algorithm is implementation-defined. + +`canister_global_timer` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. Still, the function `canister_global_timer` can initiate new calls. + +NOTE: While an implementation will likely try to keep the interval between the value of the global timer and the time-stamp of the `canister_global_timer` invocation within a few seconds, this is not formally part of this specification. + ==== Callbacks Callbacks are addressed by their table index (as a proxy for a Wasm `funcref`). @@ -987,8 +999,8 @@ The comment after each function lists from where these functions may be invoked: * `C`: from a cleanup callback * `s`: the `(start)` module initialization function * `F`: from `canister_inspect_message` -* `H`: from `canister_heartbeat` -* `*` = `I G U Q Ry Rt C F H` (NB: Not `(start)`) +* `T`: from _system task_ (`canister_heartbeat` or `canister_global_timer`) +* `*` = `I G U Q Ry Rt C F T` (NB: Not `(start)`) If the canister invokes a system call from somewhere else, it will trap. @@ -1113,6 +1125,17 @@ Status `1` indicates running, `2` indicates stopping, and `3` indicates stopped. + Status `3` (stopped) can be observed, for example, in `canister_pre_upgrade` and can be used to prevent accidentally upgrading a canister that is not fully stopped. +[#system-api-canister-version] +=== Canister version + +For each canister, the system maintains a _canister version_. Upon canister creation, it is set to 0, and it is *guaranteed* to be incremented upon every change of the canister's code or settings, i.e., upon every successful management canister call of methods `update_settings`, `install_code`, and `uninstall_code` on that canister and code uninstallation due to that canister running out of cycles. The system can arbitrarily increment the canister version also if the canister's code and settings do not change. + +* `ic0.canister_version : () -> i64` ++ +returns the current canister version. + +During the canister upgrade process, `canister_pre_upgrade` sees the old counter value, and `canister_post_upgrade` sees the new counter value. + [#system-api-call] === Inter-canister method calls @@ -1376,6 +1399,17 @@ The times observed by different canisters are unrelated, and calls from one cani NOTE: While an implementation will likely try to keep the time returned by `ic0.time` close to the real time, this is not formally part of this specification. +[#global-timer] +=== Global timer + +The canister can set a global timer to make the system schedule a call to the exported `canister_global_timer` Wasm method after the specified time. The time must be provided as nanoseconds since 1970-01-01. + +`+ic0.global_timer_set : (timestamp : i64) -> i64+` + +The function returns the previous value of the timer. If no timer is set before invoking the function, then the function returns zero. + +Passing zero as an argument to the function deactivates the timer and thus prevents the system from scheduling calls to the canister's `canister_global_timer` Wasm method. + [#system-api-performance-counter] === Performance counter @@ -1663,29 +1697,36 @@ It is important to note the following for the usage of the `POST` method: For security reasons, only HTTPS connections are allowed (URLs must start with `https://`). The IC uses industry-standard root CA lists to validate certificates of remote web servers. -Each request can specify the maximal expected size for the response from the remote HTTP server. The upper limit on the size of a response defaults to `2MiB` if no maximal size value is specified. -An error will be returned when the response is larger than the maximal size. -The `2MiB` size limit also applies to the value returned by the `transform` function. +The *size* of an HTTP request from the canister or an HTTP response from the remote HTTP server is the total number of bytes representing the names and values of HTTP headers and the HTTP body. The maximal size for the request from the canister is `2MB` (`2,000,000B`). Each request can specify a maximal size for the response from the remote HTTP server. The upper limit on the maximal size for the response is `2MB` (`2,000,000B`) and this value also applies if no maximal size value is specified. +An error will be returned when the request or response is larger than the maximal size. The following parameters should be supplied for the call: -- `url` - the requested URL. The URL may specify a custom port number. -- `max_response_bytes` - optional, specifies the maximal size of the response in bytes. Any value less than or equal to `2MiB` is accepted. The call will be charged based on this parameter. If not provided, the maximum of `2MiB` will be used. +- `url` - the requested URL. The URL must be valid according to https://www.ietf.org/rfc/rfc3986.txt[RFC-3986] and its length must not exceed `8192`. The URL may specify a custom port number. +- `max_response_bytes` - optional, specifies the maximal size of the response in bytes. If provided, the value must not exceed `2MB` (`2,000,000B`). The call will be charged based on this parameter. If not provided, the maximum of `2MB` will be used. - `method` - currently, only GET, HEAD, and POST are supported - `headers` - list of HTTP request headers and their corresponding values - `body` - optional, the content of the request's body - `transform` - an optional record that includes a function that transforms raw responses to sanitized responses, and a byte-encoded context that is provided to the function upon invocation, along with the response to be sanitized. If provided, the calling canister itself must export this function. +Cycles to pay for the call must be explicitly transferred with the call, i.e., they are not deducted from the caller's balance implicitly (e.g., as for inter-canister calls). + The returned response (and the response provided to the `transform` function, if specified) contains the following fields: - `status` - the response status (e.g., 200, 404) - `headers` - list of HTTP response headers and their corresponding values - `body` - the response's body -The `transform` function may, for example, transform the body in any way, add or remove headers, modify headers, etc. -When the transform function was invoked due to a canister HTTP request, the caller's identity is the principal of the management canister. +The `transform` function may, for example, transform the body in any way, add or remove headers, modify headers, etc. The maximal number of bytes representing the response produced by the `transform` function is `2MB` (`2,000,000B`). Note that the number of bytes representing the response produced by the `transform` function includes the serialization overhead of the encoding produced by the canister. + +When the transform function is invoked by the system due to a canister HTTP request, the caller's identity is the principal of the management canister. This information can be used by developers to implement access control mechanism for this function. -NOTE: The identity of the caller for a valid invocation of the `transform` function is `aaaaa-aa`. This information can be used by developers to implement access control mechanism for this function. +The following additional limits apply to HTTP requests and HTTP responses from the remote sever: +- the number of headers must not exceed `64`, +- the number of bytes representing a header name or value must not exceed `8KiB`, and +- the total number of bytes representing the header names and values must not exceed `48KiB`. + +NOTE: Currently, the Internet Computer mainnet only supports URLs that resolve to IPv6 destinations (i.e., the domain has a `AAAA` DNS record) in HTTP requests. [#ic-provisional_create_canister_with_cycles] === IC method `provisional_create_canister_with_cycles` @@ -1715,7 +1756,7 @@ NOTE: The IC Bitcoin API is considered EXPERIMENTAL. Canister developers must be The Bitcoin functionality is exposed via the management canister. Information about Bitcoin can be found in the https://developer.bitcoin.org/devguide/[Bitcoin developer guides]. -Invoking the functions of the Bitcoin API will cost cycles. The concrete costs for each function are yet to be determined. +Invoking the functions of the Bitcoin API will cost cycles. We refer the reader to the [Bitcoin documentation](https://internetcomputer.org/docs/current/developer-docs/integrations/bitcoin/bitcoin-how-it-works) for further relevant information and the [IC pricing page](https://internetcomputer.org/docs/current/developer-docs/deploy/computation-and-storage-costs) for information on pricing for the Bitcoin mainnet and testnet. [#ic-bitcoin_get_utxos] === IC method `bitcoin_get_utxos` @@ -1753,8 +1794,8 @@ For a newly discovered block, a regular Bitcoin (full) node therefore provides a than the Bitcoin component, which implies that it is advisable to set the number of confirmations to a reasonably large value, such as 6, to gain confidence in the correctness of the returned UTXOs. -There is an upper bound of 100,000 UTXOs that can be returned in a single request. For addresses that contain sufficiently large UTXOs, -the partial set of the address' UTXOs are returned along with a page reference. +There is an upper bound of 10,000 UTXOs that can be returned in a single request. For addresses that contain sufficiently many UTXOs, +a partial set of the address's UTXOs are returned along with a page reference. In the second case, a page reference (a series of bytes) must be provided, which instructs the Bitcoin component to collect UTXOs starting from the corresponding "page". @@ -1798,12 +1839,7 @@ If at least one of these checks fails, the call is rejected. If the transaction passes these tests, the transaction is forwarded to the specified Bitcoin network. - -The Bitcoin component periodically forwards the transaction -until the transaction appears in a block appended to the blockchain or the transaction -times out after 24 hours. As soon as it appears in a block or expires, -the Bitcoin component drops the transaction. -It follows that the function does not provide any guarantees that a submitted +Note that the function does not provide any guarantees that a submitted transaction will ever appear in a block. [#ic-bitcoin_get_current_fee_percentiles] @@ -1813,9 +1849,12 @@ The transaction fees in the Bitcoin network change dynamically based on the numb pending transactions. It must be possible for a canister to determine an adequate fee when creating a Bitcoin transaction. -This function returns the 100 fee percentiles, measured in millisatoshi/vbyte (1000 millisatoshi = 1 satoshi), over the last 10,000 transactions in the specified network, i.e., +This function returns fee percentiles, measured in millisatoshi/vbyte (1000 millisatoshi = 1 satoshi), over the last 10,000 transactions in the specified network, i.e., over the transactions in the last approximately 4-10 blocks. +The https://en.wikipedia.org/wiki/Percentile#The_nearest-rank_method[standard nearest-rank estimation method], inclusive, with the addition of a 0th percentile is used. +Concretely, for any i from 1 to 100, the ith percentile is the fee with rank `⌈i * 100⌉`. The 0th percentile is defined as the smallest fee (excluding coinbase transactions). + [#certification] == Certification @@ -2285,12 +2324,15 @@ Arg = Blob; CallerId = Principal; Timestamp = Nat; +CanisterVersion = Nat; Env = { - time : Timestamp + time : Timestamp; + global_timer : Nat; balance : Nat; freezing_limit : Nat; - certificate : NoCertificate | Blob - status : Running | Stopping | Stopped + certificate : NoCertificate | Blob; + status : Running | Stopping | Stopped; + canister_version : CanisterVersion; } RejectCode = Nat @@ -2306,7 +2348,8 @@ MethodCall = { UpdateFunc = WasmState -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; new_calls : List MethodCall; - new_certified_data : NoCertifiedData | Blob + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; response : NoResponse | Response; cycles_accepted : Nat; cycles_used : Nat; @@ -2315,8 +2358,11 @@ QueryFunc = WasmState -> Trap { cycles_used : Nat; } | Return { response : Response; cycles_used : Nat; } -HeartbeatFunc = WasmState -> Trap { cycles_used : Nat; } | Return { +SystemTaskFunc = WasmState -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; + new_calls : List MethodCall; + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; cycles_used : Nat; } @@ -2326,19 +2372,25 @@ RefundedCycles = Nat CanisterModule = { init : (CanisterId, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; cycles_used : Nat; } pre_upgrade : (WasmState, Principal, Env) -> Trap { cycles_used : Nat; } | Return { stable_memory : StableMemory; + new_certified_data : NoCertifiedData | Blob; cycles_used : Nat; } post_upgrade : (CanisterId, StableMemory, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; cycles_used : Nat; } update_methods : MethodName ↦ ((Arg, CallerId, Env, AvailableCycles) -> UpdateFunc) query_methods : MethodName ↦ ((Arg, CallerId, Env) -> QueryFunc) - heartbeat : (Env) -> HeartbeatFunc + heartbeat : (Env) -> SystemTaskFunc + global_timer : (Env) -> SystemTaskFunc callbacks : (Callback, Response, RefundedCycles, Env, AvailableCycles) -> UpdateFunc inspect_message : (MethodName, WasmState, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { status : Accept | Reject; @@ -2387,7 +2439,7 @@ CallOrigin calling_context : CallId; callback: Callback } - | FromHeartbeat + | FromSystem CallCtxt = { canister : CanisterId; origin : CallOrigin; @@ -2408,6 +2460,7 @@ EntryPoint = PublicMethod MethodName Principal Blob | Callback Callback Response RefundedCycles | Heartbeat + | GlobalTimer Message = CallMessage { @@ -2524,12 +2577,14 @@ CanStatus | Stopping (List (CallOrigin, Nat)) | Stopped S = { - requests : Request ↦ RequestStatus; + requests : Request ↦ (RequestStatus, Principal); canisters : CanisterId ↦ CanState; controllers : CanisterId ↦ Set Principal; freezing_threshold : CanisterId ↦ Nat; canister_status: CanisterId ↦ CanStatus; + canister_version: CanisterId ↦ CanisterVersion; time : CanisterId ↦ Timestamp; + global_timer : CanisterId ↦ Timestamp; balances: CanisterId ↦ Nat; certified_data: CanisterId ↦ Blob; system_time : Timestamp @@ -2556,7 +2611,9 @@ The initial state of the IC is controllers = (); freezing_threshold = (); canister_status = (); + canister_version = (); time = (); + global_timer = (); balances = (); certified_data = (); system_time = T; @@ -2690,10 +2747,12 @@ Conditions:: S.canisters[E.content.canister_id] ≠ EmptyCanister Env = { time = S.time[E.content.canister_id]; + global_timer = S.global_timer[E.content.canister_id]; balance = S.balances[E.content.canister_id] freezing_limit = freezing_limit(S, E.content.canister_id); certificate = NoCertificate; status = simple_status(S.canister_status[E.content.canister_id]); + canister_version = S.canister_version[E.content.canister_id]; } S.canisters[E.content.canister_id].module.inspect_message (E.content.method_name, S.canisters[E.content.canister_id].wasm_state, E.content.arg, E.content.sender, Env) = Return {status = Accept; cycles_used = Cycles_used;} @@ -2703,7 +2762,7 @@ Conditions:: State after:: .... S with - requests[E.content] = Received + requests[E.content] = (Received, ECID) if E.content.canister_id ≠ ic_principal then balances[E.content.canister_id] = S.balances[E.content.canister_id] - Cycles_used .... @@ -2716,13 +2775,13 @@ The IC may reject a received message for internal reasons (high load, low resour Conditions:: .... - S.requests[R] = Received + S.requests[R] = (Received, ECID) Code = SYS_FATAL or Code = SYS_TRANSIENT .... State after:: .... S with - requests[R] = Rejected (Code, Msg) + requests[R] = (Rejected (Code, Msg), ECID) .... ==== Initiating canister calls @@ -2735,14 +2794,14 @@ The IC does not make any guarantees about the order of incoming messages. Conditions:: .... - S.requests[R] = Received + S.requests[R] = (Received, ECID) S.system_time <= R.ingress_expiry C = S.canisters[R.canister_id] .... State after:: .... S with - requests[R] = Processing + requests[R] = (Processing, ECID) messages = CallMessage { origin = FromUser { request = R }; @@ -2780,7 +2839,7 @@ State after:: ==== Call context creation -Before invoking a heartbeat or a message to a public entry point, a call context is created for bookkeeping purposes. +Before invoking a heartbeat, a global timer, or a message to a public entry point, a call context is created for bookkeeping purposes. For these invocations the canister must be running (so not stopped, or stopping). Additionally, these invocations only happen for "real" canisters, not the IC management canister. @@ -2834,7 +2893,7 @@ Conditions:: .... S.canisters[C] ≠ EmptyCanister S.canister_status[C] = Running - balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE + S.balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE Ctxt_id ∉ dom(S.call_contexts) .... + @@ -2852,11 +2911,49 @@ S with · S.messages call_contexts[Ctxt_id] = { canister = C; - origin = FromHeartbeat; + origin = FromSystem; + needs_to_respond = false; + deleted = false; + available_cycles = 0; + } + balances[C] = S.balances[C] - MAX_CYCLES_PER_MESSAGE +.... + +* Call context creation: Global timer ++ +If canister `C` exports a method with name `canister_global_timer`, the global timer of canister `C` is set, and the current time for canister `C` has passed the value of the global timer, the IC will create the corresponding call context and deactivate the global timer. ++ +Conditions:: ++ +.... + S.canisters[C] ≠ EmptyCanister + S.canister_status[C] = Running + S.global_timer[C] ≠ 0 + S.time[C] ≥ S.global_timer[C] + S.balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE + Ctxt_id ∉ dom(S.call_contexts) +.... ++ +State after:: ++ +.... +S with + messages = + FuncMessage { + call_context = Ctxt_id; + receiver = C; + entry_point = GlobalTimer; + queue = Queue { from = System; to = C }; + } + · S.messages + call_contexts[Ctxt_id] = { + canister = C; + origin = FromSystem; needs_to_respond = false; deleted = false; available_cycles = 0; } + global_timer[C] = 0 balances[C] = S.balances[C] - MAX_CYCLES_PER_MESSAGE .... @@ -2880,10 +2977,12 @@ Conditions:: Env = { time = S.time[M.receiver]; + global_timer = S.global_timer[M.receiver]; balance = S.balances[M.receiver] freezing_limit = freezing_limit(S, M.receiver); certificate = NoCertificate; status = simple_status(S.canister_status[M.receiver]); + canister_version = S.canister_version[M.receiver]; } Available = S.call_contexts[M.call_contexts].available_cycles @@ -2896,7 +2995,11 @@ Conditions:: ) or ( M.entry_point = Heartbeat - F = heartbeat_as_update(Mod.heartbeat, Env) + F = system_task_as_update(Mod.heartbeat, Env) + ) + or + ( M.entry_point = GlobalTimer + F = system_task_as_update(Mod.global_timer, Env) ) R = F(S.canisters[M.receiver].wasm_state) @@ -2949,6 +3052,9 @@ then if res.new_certified_data ≠ NoCertifiedData: certified_data[M.receiver] = res.new_certified_data + if res.new_global_timer ≠ NoGlobalTimer: + global_timer[M.receiver] = res.new_global_timer + balances[M.receiver] = New_balance else S with @@ -2978,7 +3084,7 @@ If message execution <>; that would require a (unique) call to `ic0.reply`. Note also that the state changes are persisted even when the IC is set to synthesize a <> reject immediately afterward (which happens when this returns without calling `ic0.reply` or `ic0.reject`, the corresponding call has not been responded to and there are no outstanding callbacks, see <>). -The functions `query_as_update` and `heartbeat_as_update` turns a query function resp the heartbeat into an update function; this is merely a notational trick to simplify the rule: +The functions `query_as_update` and `system_task_as_update` turns a query function resp the heartbeat or global timer into an update function; this is merely a notational trick to simplify the rule: .... query_as_update(f, arg, env) = λ wasm_state → match f(arg, env)(wasm_state) with @@ -2987,18 +3093,20 @@ query_as_update(f, arg, env) = λ wasm_state → new_state = wasm_state; new_calls = []; new_certified_data = NoCertifiedData; + new_global_timer = NoGlobalTimer; response = res.response; cycles_accepted = 0; cycles_used = res.cycles_used; } -heartbeat_as_update(f, env) = λ wasm_state → +system_task_as_update(f, env) = λ wasm_state → match f(env)(wasm_state) with Trap trap → Trap trap Return res → Return { new_state = res.new_state; - new_calls = []; - new_certified_data = NoCertifiedData; + new_calls = res.new_calls; + new_certified_data = res.new_certified_data; + new_global_timer = res.new_global_timer; response = NoResponse; cycles_accepted = 0; cycles_used = res.cycles_used; @@ -3009,12 +3117,12 @@ Note that by construction, a query function will either trap or return with a re [#rule-starvation] ==== Call context starvation -If the call context is not for heartbeat and there is no call, downstream calling context or response that could possibly fulfill a calling context, then a reject is synthesized. The error message below is _not_ indicative. In particular, if the IC has an idea about _why_ this starved, it can put that in there (e.g. the initial message handler trapped with an out-of-memory access). +If the call context is not for heartbeat or global timer and there is no call, downstream calling context or response that could possibly fulfill a calling context, then a reject is synthesized. The error message below is _not_ indicative. In particular, if the IC has an idea about _why_ this starved, it can put that in there (e.g. the initial message handler trapped with an out-of-memory access). Conditions:: .... S.call_contexts[Ctxt_id].needs_to_respond = true - S.call_contexts[Ctxt_id].origin ≠ FromHeartbeat + S.call_contexts[Ctxt_id].origin ≠ FromSystem ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id ∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id ∀ _ ↦ {needs_to_respond = true, origin = FromCanister O, …} ∈ S.call_contexts: O.calling_context ≠ Ctxt_id @@ -3035,7 +3143,7 @@ S with ==== Call context removal -If there is no call, downstream calling context, or response that references a call context, and the call context has been replied to or the call context corresponds to a heartbeat that had already been executed, then the call context can be removed. +If there is no call, downstream calling context, or response that references a call context, and the call context has been replied to or the call context corresponds to a heartbeat or global timer that had already been executed, then the call context can be removed. Conditions:: .... @@ -3043,7 +3151,7 @@ Conditions:: S.call_contexts[Ctxt_id].needs_to_respond = false ) or ( - S.call_contexts[Ctxt_id].origin = FromHeartbeat + S.call_contexts[Ctxt_id].origin = FromSystem ∀ FuncMessage M ∈ S.messages. M.call_context ≠ Ctxt_id ) ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id @@ -3082,6 +3190,7 @@ State after:: S with canisters[CanisterId] = EmptyCanister time[CanisterId] = CurrentTime + global_timer[CanisterId] = 0 if A.settings.controllers is not null: controllers[CanisterId] = A.settings.controllers else: @@ -3099,6 +3208,7 @@ S with refunded_cycles = 0 } canister_status[CanisterId] = Running + canister_version[CanisterId] = 0 .... This uses the predicate @@ -3136,6 +3246,7 @@ S with controllers[A.canister_id] = A.settings.controllers if A.settings.freezing_threshold is not null: freezing_threshold[A.canister_id] = A.settings.freezing_threshold + canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin @@ -3201,12 +3312,14 @@ Conditions:: M.caller ∈ S.controllers[A.canister_id] Env = { time = S.time[A.canister_id]; + global_timer = S.global_timer[A.canister_id]; balance = S.balances[A.canister_id]; freezing_limit = freezing_limit(S, A.canister_id); certificate = NoCertificate; status = simple_status(S.canister_status[A.canister_id]); + canister_version = S.canister_version[A.canister_id] + 1; } - Mod.init(A.canister_id, A.arg, M.caller, Env) = Return {new_state = New_state; cycles_used = Cycles_used;} + Mod.init(A.canister_id, A.arg, M.caller, Env) = Return {new_state = New_state; new_certified_data = New_certified_data; new_global_timer = New_global_timer; cycles_used = Cycles_used;} Cycles_used ≤ S.balances[A.canister_id] dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ @@ -3221,6 +3334,13 @@ S with public_custom_sections = Public_custom_sections; private_custom_sections = Private_custom_sections; } + if New_certified_data ≠ NoCertifiedData: + certified_data[A.canister_id] = New_certified_data + if New_global_timer ≠ NoGlobalTimer: + global_timer[A.canister_id] = New_global_timer + else: + global_timer[A.canister_id] = 0 + canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 balances[A.canister_id] = S.balances[A.canister_id] - Cycles_used messages = Older_messages · Younger_messages · ResponseMessage { @@ -3254,8 +3374,16 @@ Conditions:: certificate = NoCertificate; status = simple_status(S.canister_status[A.canister_id]); } - Old_module.pre_upgrade(Old_State, M.caller, Env) = Return {stable_memory = Stable_memory; cycles_used = Cycles_used;} - Mod.post_upgrade(A.canister_id, Stable_memory, A.arg, M.caller, Env) = Return {new_state = New_state; cycles_used = Cycles_used';} + Env1 = Env with { + global_timer = S.global_timer[A.canister_id]; + canister_version = S.canister_version[A.canister_id]; + } + Old_module.pre_upgrade(Old_State, M.caller, Env1) = Return {stable_memory = Stable_memory; new_certified_data = New_certified_data; cycles_used = Cycles_used;} + Env2 = Env with { + global_timer = 0; + canister_version = S.canister_version[A.canister_id] + 1; + } + Mod.post_upgrade(A.canister_id, Stable_memory, A.arg, M.caller, Env2) = Return {new_state = New_state; new_certified_data = New_certified_data'; new_global_timer = New_global_timer; cycles_used = Cycles_used';} Cycles_used + Cycles_used' ≤ S.balances[A.canister_id] dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ .... @@ -3269,6 +3397,15 @@ S with public_custom_sections = Public_custom_sections; private_custom_sections = Private_custom_sections; } + if New_certified_data' ≠ NoCertifiedData: + certified_data[A.canister_id] = New_certified_data' + else if New_certified_data ≠ NoCertifiedData: + certified_data[A.canister_id] = New_certified_data + if New_global_timer ≠ NoGlobalTimer: + global_timer[A.canister_id] = New_global_timer + else: + global_timer[A.canister_id] = 0 + canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 balances[A.canister_id] = S.balances[A.canister_id] - (Cycles_used + Cycles_used'); messages = Older_messages · Younger_messages · ResponseMessage { @@ -3297,6 +3434,8 @@ State after:: S with canisters[A.canister_id] = EmptyCanister certified_data[A.canister_id] = "" + canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 + global_timer[A.canister_id] = 0 messages = Older_messages · Younger_messages · ResponseMessage { @@ -3489,7 +3628,9 @@ S with controllers[A.canister_id] = (deleted) freezing_threshold[A.canister_id] = (deleted) canister_status[A.canister_id] = (deleted) + canister_version[A.canister_id] = (deleted) time[A.canister_id] = (deleted) + global_timer[A.canister_id] = (deleted) balances[A.canister_id] = (deleted) certified_data[A.canister_id] = (deleted) messages = Older_messages · Younger_messages · @@ -3569,6 +3710,7 @@ State after:: S with canisters[CanisterId] = EmptyCanister time[CanisterId] = CurrentTime + global_timer[CanisterId] = 0 if A.settings.controllers is not null: controllers[CanisterId] = A.settings.controllers else: @@ -3589,6 +3731,7 @@ S with refunded_cycles = M.transferred_cycles } canister_status[CanisterId] = Running + canister_version[CanisterId] = 0 .... ==== IC Management Canister: Top up canister @@ -3669,15 +3812,15 @@ Conditions:: .... S.messages = Older_messages · ResponseMessage RM · Younger_messages RM.origin = FromUser { request = M } - S.requests[M] = Processing + S.requests[M] = (Processing, ECID) .... State after:: .... S with messages = Older_messages · Younger_messages requests[M] = - | Replied R if M.response = Reply R - | Rejected (c, R) if M.response = Reject (c, R) + | (Replied R, ECID) if M.response = Reply R + | (Rejected (c, R), ECID) if M.response = Reject (c, R) .... NB: The refunded cycles, `RM.refunded_cycles` are, by construction, empty. @@ -3688,12 +3831,12 @@ The IC will keep the data for a completed or rejected request around for a certa Conditions:: .... - (S.requests[M] = Replied _) or (S.requests[M] = Rejected _) + (S.requests[M] = (Replied _, ECID)) or (S.requests[M] = (Rejected _, ECID)) .... State after:: .... S with - requests[M] = Done + requests[M] = (Done, ECID) .... @@ -3701,7 +3844,7 @@ At the same or some later point, the request will be removed from the state of t Conditions:: .... - (S.requests[M] = Replied _) or (S.requests[M] = Rejected _) or (S.requests[M] = Done) + (S.requests[M] = (Replied _, _)) or (S.requests[M] = (Rejected _, _)) or (S.requests[M] = (Done, _)) M.ingress_expiry < S.system_time .... State after:: @@ -3722,7 +3865,9 @@ State after:: .... S with canisters[CanisterId] = EmptyCanister - certified_data[Canister_id] = "" + certified_data[CanisterId] = "" + canister_version[CanisterId] = S.canister_version[CanisterId] + 1 + global_timer[CanisterId] = 0 messages = S.messages · [ ResponseMessage { @@ -3743,7 +3888,7 @@ S with .... -==== Time progressing and cycle consumption +==== Time progressing, cycle consumption, and canister version increments Time progresses. Abstractly, it does so independently for each canister, and in unspecified intervals. @@ -3784,6 +3929,19 @@ S with system_time = T1 .... +Finally, the canister version can be incremented arbitrarily: + +Conditions:: +.... + N0 = S.canister_version[CanisterId] + N1 > N0 +.... +State after:: +.... +S with + canister_version[CanisterId] = N1 +.... + ==== Query call Canister query calls to `/api/v2/canister//query` can be executed directly. They can only be executed against canisters which have a status of `Running` and are also not frozen. @@ -3806,10 +3964,12 @@ Conditions:: lookup(["time"], Cert) = Found S.system_time // or “recent enough” Env = { time = S.time[Q.receiver]; + global_timer = S.global_timer[Q.receiver]; balance = S.balances[Q.canister_id]; freezing_limit = freezing_limit(S, Q.canister_id); certificate = Cert; status = simple_status(S.canister_status[Q.receiver]); + canister_version = S.canister_version[Q.receiver]; } .... Read response:: @@ -3955,7 +4115,9 @@ ExecutionState = { pending_call : MethodCall | NoPendingCall; calls : List MethodCall; new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; ingress_filter : Accept | Reject; + context : I | G | U | Q | Ry | Rt | C | F | T; } .... @@ -3994,7 +4156,9 @@ empty_execution_state = { pending_call = NoPendingCall; calls = []; new_certified_data = NoCertifiedData; + new_global_timer = NoGlobalTimer; ingress_filter = Reject; + context = (undefined); } .... @@ -4005,7 +4169,12 @@ If the WebAssembly module does not export a function called under the name `cani + .... init = λ (self_id, arg, caller, sysenv) → - Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" }; cycles_used = 0;} + Return { + new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" }; + new_certified_data = NoCertifiedData; + new_global_timer = NoGlobalTimer; + cycles_used = 0; + } .... + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_init`, it is @@ -4016,12 +4185,15 @@ init = λ (self_id, arg, caller, sysenv) → wasm_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" } params = empty_params with { arg = arg; caller = caller; sysenv } balance = sysenv.balance + context = I } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} - if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} - Return {new_state = es.wasm_state; cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + new_certified_data = es.new_certified_data; + new_global_timer = es.new_global_timer; + cycles_used = es.cycles_used; + } .... + This formulation checks afterwards that the system calls `call.perform` or `msg.reply` were not invoked; an implementation can of course trap as soon as these system calls are invoked. @@ -4031,7 +4203,7 @@ This formulation checks afterwards that the system calls `call.perform` or `msg. If the WebAssembly module does not export a function called under the name `canister_pre_upgrade`, then it simply returns the stable memory: + .... -pre_upgrade = λ (old_state, caller, sysenv) → Return {stable_memory = old_state.stable_mem; cycles_used = 0;} +pre_upgrade = λ (old_state, caller, sysenv) → Return {stable_memory = old_state.stable_mem; new_certified_data = NoCertifiedData; cycles_used = 0;} .... + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_pre_upgrade`, it is @@ -4042,12 +4214,14 @@ pre_upgrade = λ (old_state, caller, sysenv) → wasm_state = old_state params = { empty_params with caller = caller; sysenv } balance = sysenv.balance + context = G } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} - if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} - Return {stable_memory = es.wasm_state.stable_mem; cycles_used = es.cycles_used;} + Return { + stable_memory = es.wasm_state.stable_mem; + new_certified_data = es.new_certified_data; + cycles_used = es.cycles_used; + } .... @@ -4057,7 +4231,7 @@ If the WebAssembly module does not export a function called under the name `cani + .... post_upgrade = λ (self_id, stable_mem, arg, caller, sysenv) → - Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem }; cycles_used = 0;} + Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem }; new_certified_data = NoCertifiedData; new_global_timer = NoGlobalTimer; cycles_used = 0;} .... + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_post_upgrade`, it is @@ -4068,12 +4242,15 @@ post_upgrade = λ (self_id, stable_mem, arg, caller, sysenv) → wasm_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem } params = { empty_params with arg = arg; caller = caller; sysenv } balance = sysenv.balance + context = I } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} - if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} - Return {new_state = es.wasm_state; cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + new_certified_data = es.new_certified_data; + new_global_timer = es.new_global_timer; + cycles_used = es.cycles_used; + } .... * The partial map `update_methods` of the `CanisterModule` is defined for all method names `method` for which the WebAssembly program exports a function `func` named `canister_update `, and has value @@ -4085,16 +4262,17 @@ update_methods[method] = λ (arg, caller, sysenv, available) → λ wasm_state params = empty_params with { arg = arg; caller = caller; sysenv } balance = sysenv.balance cycles_available = available; + context = U } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} Return { new_state = es.wasm_state; new_calls = es.calls; + new_certified_data = es.new_certified_data; + new_global_timer = es.new_global_timer; response = es.response; cycles_accepted = es.cycles_accepted; cycles_used = es.cycles_used; - new_certified_data = es.new_certified_data; } .... @@ -4106,12 +4284,9 @@ query_methods[method] = λ (arg, caller, sysenv) → λ wasm_state → wasm_state = wasm_state; params = empty_params with { arg = arg; caller = caller; sysenv } balance = sysenv.balance + context = Q } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used;} - if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} - if es.response = NoResponse then Trap {cycles_used = es.cycles_used;} Return { response = es.response; cycles_used = es.cycles_used; @@ -4130,13 +4305,14 @@ heartbeat = λ (sysenv) → λ wasm_state → wasm_state = wasm_state; params = empty_params with { arg = NoArg; caller = NoCaller; sysenv } balance = sysenv.balance + context = T } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} - if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} Return { new_state = es.wasm_state; + new_calls = es.calls; + new_certified_data = es.certified_data; + new_global_timer = es.new_global_timer; cycles_used = es.cycles_used; } .... @@ -4145,6 +4321,30 @@ otherwise it is heartbeat = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} .... +* The function `global_timer` of the `CanisterModule` is defined if the WebAssembly program exports a function `func` named `canister_global_timer`, and has value ++ +.... +global_timer = λ (sysenv) → λ wasm_state → + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = empty_params with { arg = NoArg; caller = NoCaller; sysenv } + balance = sysenv.balance + context = T + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + new_calls = es.calls; + new_certified_data = es.certified_data; + new_global_timer = es.new_global_timer; + cycles_used = es.cycles_used; + } +.... +otherwise it is +.... +global_timer = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} +.... + * The function `callbacks` of the `CanisterModule` is defined as follows + .... @@ -4153,18 +4353,19 @@ callbacks = λ(callbacks, response, refunded_cycles, sysenv, available) → λ w sysenv cycles_refunded = refund_cycles; } - let (fun, env, params) = match response with + let (fun, env, params, context) = match response with Reply data -> (callbacks.on_reply.fun, callbacks.on_reply.env, - { params0 with data}) + { params0 with data}, Ry) Reject (reject_code, reject_message)-> (callbacks.on_reject.fun, callbacks.on_reject.env, - { params0 with reject_code; reject_message}) + { params0 with reject_code; reject_message}, Rt) let es = ref {empty_execution_state with wasm_state = wasm_state; params = params; balance = sysenv.balance; cycles_available = available; + context = context; } try if fun > |es.wasm_state.store.table| then Trap @@ -4175,10 +4376,11 @@ callbacks = λ(callbacks, response, refunded_cycles, sysenv, available) → λ w Return { new_state = es.wasm_state; new_calls = es.calls; + new_certified_data = es.certified_data; + new_global_timer = es.new_global_timer; response = es.response; cycles_accepted = es.cycles_accepted; cycles_used = es.cycles_used; - new_certified_data = es.certified_data; } with Trap if callbacks.on_cleanup = NoClosure then Trap {cycles_used = es.cycles_used;} @@ -4188,15 +4390,14 @@ callbacks = λ(callbacks, response, refunded_cycles, sysenv, available) → λ w let es' = ref { empty_execution_state with wasm_state = wasm_state; + context = C; } try func(callbacks.on_cleanup.env) with Trap then Trap {cycles_used = es.cycles_used + es'.cycles_used;} - if es'.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used + es'.cycles_used;} - if es'.calls ≠ [] then Trap {cycles_used = es.cycles_used + es'.cycles_used;} - if es'.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used + es'.cycles_used;} - if es'.response ≠ NoResponse then Trap {cycles_used = es.cycles_used + es'.cycles_used;} Return { new_state = es'.wasm_state; new_calls = []; + new_certified_data = NoCertifiedData; + new_global_timer = es'.new_global_timer; response = NoResponse; cycles_accepted = 0; cycles_used = es.cycles_used + es'.cycles_used; @@ -4228,10 +4429,9 @@ inspect_message = λ (method_name, wasm_state, arg, caller, sysenv) → } balance = sysenv.balance; cycles_available = 0; // ingress requests have no funds + context = F; } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} - if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} Return {status = es.ingress_filter; cycles_used = es.cycles_used;}; .... @@ -4268,67 +4468,85 @@ The pseudo-code below does _not_ explicitly enforce the restrictions of which im .... ic0.msg_arg_data_size() : i32 = + if es.context ∉ {I, U, Q, Ry, F} then Trap {cycles_used = es.cycles_used;} return |es.params.arg| ic0.msg_arg_data_copy(dst:i32, offset:i32, size:i32) = + if es.context ∉ {I, U, Q, Ry, F} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.arg) ic0.msg_caller_size() : i32 = + if es.context ∉ {I, G, U, Q, F} then Trap {cycles_used = es.cycles_used;} return |es.params.caller| ic0.msg_caller_copy(dst:i32, offset:i32, size:i32) : i32 = + if es.context ∉ {I, G, U, Q, F} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.caller) ic0.msg_reject_code() : i32 = + if es.context ∉ {Ry, Rt} then Trap {cycles_used = es.cycles_used;} es.params.reject_code ic0.msg_reject_msg_size() : i32 = + if es.context ∉ {Rt} then Trap {cycles_used = es.cycles_used;} return |es.params.reject_msg| ic0.msg_reject_msg_copy(dst:i32, offset:i32, size:i32) : i32 = + if es.context ∉ {Rt} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.reject_msg) ic0.msg_reply_data_append(src : i32, size : i32) = + if es.context ∉ {U, Q, Ry, Rt} then Trap {cycles_used = es.cycles_used;} if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.reply_params.arg := es.reply_params.arg · copy_from_canister(src, size) ic0.msg_reply() = + if es.context ∉ {U, Q, Ry, Rt} then Trap {cycles_used = es.cycles_used;} if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.response := Reply (es.reply_params.arg) es.cycles_available := 0 ic0.msg_reject(src : i32, size : i32) = + if es.context ∉ {U, Q, Ry, Rt} then Trap {cycles_used = es.cycles_used;} if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.response := Reject (CANISTER_REJECT, copy_from_canister(src, size)) es.cycles_available := 0 ic0.msg_cycles_available() : i64 = + if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} if es.cycles_available >= 2^64^ then Trap {cycles_used = es.cycles_used;} return es.cycles_available ic0.msg_cycles_available128(dst : i32) = + if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} let amount = es.cycles_available copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) ic0.msg_cycles_refunded() : i64 = + if es.context ∉ {Rt, Ry} then Trap {cycles_used = es.cycles_used;} if es.params.cycles_refunded >= 2^64^ then Trap {cycles_used = es.cycles_used;} return es.params.cycles_refunded ic0.msg_cycles_refunded128(dst : i32) = + if es.context ∉ {Rt, Ry} then Trap {cycles_used = es.cycles_used;} let amount = es.params.cycles_refunded copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) ic0.accept_message() = + if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} if es.ingress_filter = Accept then Trap {cycles_used = es.cycles_used;} es.ingress_filter = Accept ic0.msg_method_name_size() : i32 = + if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} return |es.method_name| ic0.msg_method_name_copy(dst : i32, offset : i32, size : i32) : i32 = + if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.method_name) ic0.msg_cycles_accept(max_amount : i64) : i64 = + if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} let amount = min(max_amount, es.cycles_available) es.cycles_available := es.cycles_available - amount es.cycles_accepted := es.cycles_accepted + amount @@ -4336,6 +4554,7 @@ ic0.msg_cycles_accept(max_amount : i64) : i64 = return amount ic0.msg_cycles_accept128(max_amount_high : i64, max_amount_low : i64, dst : i32) = + if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} let max_amount = max_amount_high * 2^64^ + max_amount_low let amount = min(max_amount, es.cycles_available) es.cycles_available := es.cycles_available - amount @@ -4363,6 +4582,9 @@ ic0.canister_status() : i32 = Stopping -> return 2 Stopped -> return 3 +ic0.canister_version() : i64 = + return es.params.sysenv.canister_version + ic0.call_new( callee_src : i32, callee_size : i32, @@ -4373,6 +4595,8 @@ ic0.call_new( reject_fun : i32, reject_env : i32, ) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} + discard_pending_call() if es.balance < MAX_CYCLES_PER_RESPONSE then Trap {cycles_used = es.cycles_used;} @@ -4400,10 +4624,12 @@ ic0.call_new( } ic0.call_data_append (src : i32, size : i32) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} es.pending_call.arg := es.pending_call.arg · copy_from_canister(src, size) ic0.call_on_cleanup (fun : i32, env : i32) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} if fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} if typeof(es.wasm_state.store.table[fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} @@ -4411,6 +4637,7 @@ ic0.call_on_cleanup (fun : i32, env : i32) = es.pending_call.callback.on_cleanup := Closure { fun = fun; env = env} ic0.call_cycles_add(amount : i64) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} if es.balance < amount then Trap {cycles_used = es.cycles_used;} @@ -4418,6 +4645,7 @@ ic0.call_cycles_add(amount : i64) = es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount ic0.call_cycles_add128(amount_high : i64, amount_low : i64) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} let amount = amount_high * 2^64^ + amount_low if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} if es.balance < amount then Trap {cycles_used = es.cycles_used;} @@ -4426,6 +4654,7 @@ ic0.call_cycles_add128(amount_high : i64, amount_low : i64) = es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount ic0.call_peform() : ( err_code : i32 ) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} // are we below the threezing threshold? @@ -4499,9 +4728,18 @@ ic0.stable64_read(dst : i64, offset : i64, size : i64) es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] ic0.time() : i32 = - return es.params.time + return es.params.sysenv.time + +ic0.global_timer_set(timestamp: i64) : i64 = + if es.context ∉ {I, U, Ry, Rt, C, T} then Trap {cycles_used = es.cycles_used;} + let prev_global_timer = es.new_global_timer + es.new_global_timer := timestamp + if prev_global_timer = NoGlobalTimer + then return es.params.sysenv.global_timer + else return prev_global_timer ic0.certified_data_set(src: i32, size: i32) = + if es.context ∉ {I, G, U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} es.new_certified_data := es.wasm_state[src..src+size] ic0.data_certificate_present() : i32 = From 7422f79ec37540653fe9cd3b7bccc5615a81cc8d Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Tue, 6 Dec 2022 16:48:19 +0100 Subject: [PATCH 046/102] new release --- spec/ic0.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/spec/ic0.txt b/spec/ic0.txt index a553089d3..454ff815f 100644 --- a/spec/ic0.txt +++ b/spec/ic0.txt @@ -23,6 +23,7 @@ ic0.canister_self_copy : (dst : i32, offset : i32, size : i32) -> (); // * ic0.canister_cycle_balance : () -> i64; // * ic0.canister_cycle_balance128 : (dst : i32) -> (); // * ic0.canister_status : () -> i32; // * +ic0.canister_version : () -> i64; // * ic0.msg_method_name_size : () -> i32; // F ic0.msg_method_name_copy : (dst : i32, offset : i32, size : i32) -> (); // F From 73903a86b2d1e2fc8ea38d33744a87d8fad709d1 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Tue, 6 Dec 2022 16:48:47 +0100 Subject: [PATCH 047/102] new release --- theories/IC.thy | 394 ++++++++++++++++++++++++++++++++---------------- 1 file changed, 268 insertions(+), 126 deletions(-) diff --git a/theories/IC.thy b/theories/IC.thy index d026c3f3e..ada3a64db 100644 --- a/theories/IC.thy +++ b/theories/IC.thy @@ -175,13 +175,16 @@ type_synonym 'b arg = 'b type_synonym 'p caller_id = 'p type_synonym timestamp = nat +type_synonym canister_version = nat datatype status = Running | Stopping | Stopped record ('b) env = time :: timestamp + global_timer :: nat balance :: nat freezing_limit :: nat certificate :: "'b option" status :: status + canister_version :: canister_version type_synonym reject_code = nat datatype ('b, 's) response = @@ -197,51 +200,66 @@ record ('p, 'canid, 's, 'b, 'c) method_call = record 'x cycles_return = return :: 'x cycles_used :: nat +record ('w, 'b) init_return = + new_state :: 'w + new_certified_data :: "'b option" + new_global_timer :: "nat option" + cycles_used :: nat +record ('sm, 'b) pre_upgrade_return = + stable_memory :: 'sm + new_certified_data :: "'b option" + cycles_used :: nat type_synonym trap_return = "unit cycles_return" record ('w, 'p, 'canid, 's, 'b, 'c) update_return = new_state :: 'w new_calls :: "('p, 'canid, 's, 'b, 'c) method_call list" new_certified_data :: "'b option" + new_global_timer :: "nat option" response :: "('b, 's) response option" cycles_accepted :: nat cycles_used :: nat record ('b, 's) query_return = response :: "('b, 's) response" cycles_used :: nat -record 'w heartbeat_return = +record ('w, 'p, 'canid, 's, 'b, 'c) system_task_return = new_state :: 'w + new_calls :: "('p, 'canid, 's, 'b, 'c) method_call list" + new_certified_data :: "'b option" + new_global_timer :: "nat option" cycles_used :: nat type_synonym ('w, 'p, 'canid, 's, 'b, 'c) update_func = "'w \ trap_return + ('w, 'p, 'canid, 's, 'b, 'c) update_return" type_synonym ('w, 'b, 's) query_func = "'w \ trap_return + ('b, 's) query_return" -type_synonym 'w heartbeat_func = "'w \ trap_return + 'w heartbeat_return" +type_synonym ('w, 'p, 'canid, 's, 'b, 'c) system_task_func = "'w \ trap_return + ('w, 'p, 'canid, 's, 'b, 'c) system_task_return" type_synonym available_cycles = nat type_synonym refunded_cycles = nat datatype inspect_method_result = Accept | Reject record ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module_rec = - init :: "'canid \ 'b arg \ 'p caller_id \ 'b env \ trap_return + 'w cycles_return" - pre_upgrade :: "'w \ 'p \ 'b env \ trap_return + 'sm cycles_return" - post_upgrade :: "'canid \ 'sm \ 'b arg \ 'p caller_id \ 'b env \ trap_return + 'w cycles_return" + init :: "'canid \ 'b arg \ 'p caller_id \ 'b env \ trap_return + ('w, 'b) init_return" + pre_upgrade :: "'w \ 'p \ 'b env \ trap_return + ('sm, 'b) pre_upgrade_return" + post_upgrade :: "'canid \ 'sm \ 'b arg \ 'p caller_id \ 'b env \ trap_return + ('w, 'b) init_return" update_methods :: "('s method_name, ('b arg \ 'p caller_id \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func) list_map" query_methods :: "('s method_name, ('b arg \ 'p caller_id \ 'b env) \ ('w, 'b, 's) query_func) list_map" - heartbeat :: "'b env \ 'w heartbeat_func" + heartbeat :: "'b env \ ('w, 'p, 'canid, 's, 'b, 'c) system_task_func" + global_timer :: "'b env \ ('w, 'p, 'canid, 's, 'b, 'c) system_task_func" callbacks :: "('c \ ('b, 's) response \ refunded_cycles \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" inspect_message :: "('s method_name \ 'w \ 'b arg \ 'p caller_id \ 'b env) \ trap_return + inspect_method_result cycles_return" typedef ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module = "{m :: ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module_rec. list_map_dom (update_methods m) \ list_map_dom (query_methods m) = {}}" by (auto intro: exI[of _ "\init = undefined, pre_upgrade = undefined, post_upgrade = undefined, - update_methods = list_map_empty, query_methods = list_map_empty, heartbeat = undefined, callbacks = undefined, + update_methods = list_map_empty, query_methods = list_map_empty, heartbeat = undefined, global_timer = undefined, callbacks = undefined, inspect_message = undefined\"]) setup_lifting type_definition_canister_module -lift_definition canister_module_init :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'canid \ 'b arg \ 'p \ 'b env \ trap_return + 'w cycles_return" is "init" . -lift_definition canister_module_pre_upgrade :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'w \ 'p \ 'b env \ trap_return + 'sm cycles_return" is pre_upgrade . -lift_definition canister_module_post_upgrade :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'canid \ 'sm \ 'b arg \ 'p \ 'b env \ trap_return + 'w cycles_return" is post_upgrade . +lift_definition canister_module_init :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'canid \ 'b arg \ 'p \ 'b env \ trap_return + ('w, 'b) init_return" is "init" . +lift_definition canister_module_pre_upgrade :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'w \ 'p \ 'b env \ trap_return + ('sm, 'b) pre_upgrade_return" is pre_upgrade . +lift_definition canister_module_post_upgrade :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'canid \ 'sm \ 'b arg \ 'p \ 'b env \ trap_return + ('w, 'b) init_return" is post_upgrade . lift_definition canister_module_update_methods :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('s, ('b arg \ 'p \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func) list_map" is update_methods . lift_definition canister_module_query_methods :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('s, ('b arg \ 'p \ 'b env) \ ('w, 'b, 's) query_func) list_map" is query_methods . -lift_definition canister_module_heartbeat :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'b env \ 'w heartbeat_func" is heartbeat . +lift_definition canister_module_heartbeat :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c) system_task_func" is heartbeat . +lift_definition canister_module_global_timer :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c) system_task_func" is global_timer . lift_definition canister_module_callbacks :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('c \ ('b, 's) response \ refunded_cycles \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" is callbacks . lift_definition canister_module_inspect_message :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('s \ 'w \ 'b arg \ 'p \ 'b env) \ trap_return + inspect_method_result cycles_return" is inspect_message . @@ -261,7 +279,7 @@ record ('b, 'p, 'uid, 'canid, 's) request = datatype ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin = From_user "('b, 'p, 'uid, 'canid, 's) request" | From_canister "'cid" "'c" -| From_heartbeat +| From_system record ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt_rep = canister :: 'canid origin :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin" @@ -325,6 +343,7 @@ datatype ('s, 'p, 'b, 'c) entry_point = Public_method "'s method_name" "'p" "'b" | Callback "'c" "('b, 's) response" "refunded_cycles" | Heartbeat +| Global_timer datatype ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message = Call_message "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin" 'p 'canid 's 'b nat "'canid queue" @@ -381,7 +400,9 @@ record ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic = controllers :: "('canid, 'p set) list_map" freezing_threshold :: "('canid, nat) list_map" canister_status :: "('canid, ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status) list_map" + canister_version :: "('canid, canister_version) list_map" time :: "('canid, timestamp) list_map" + global_timer :: "('canid, nat) list_map" balances :: "('canid, nat) list_map" certified_data :: "('canid, 'b) list_map" system_time :: timestamp @@ -402,7 +423,9 @@ definition initial_ic :: "nat \ 'pk \ ('p, 'uid, 'canid, controllers = list_map_empty, freezing_threshold = list_map_empty, canister_status = list_map_empty, + canister_version = list_map_empty, time = list_map_empty, + global_timer = list_map_empty, balances = list_map_empty, certified_data = list_map_empty, system_time = t, @@ -557,6 +580,7 @@ fun cycles_reserved :: "('s, 'p, 'b, 'c) entry_point \ nat" where "cycles_reserved (entry_point.Public_method _ _ _) = MAX_CYCLES_PER_MESSAGE" | "cycles_reserved (entry_point.Callback _ _ _) = MAX_CYCLES_PER_RESPONSE" | "cycles_reserved (entry_point.Heartbeat) = MAX_CYCLES_PER_MESSAGE" +| "cycles_reserved (entry_point.Global_timer) = MAX_CYCLES_PER_MESSAGE" fun message_cycles :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message \ nat" where "message_cycles (Call_message orig _ _ _ _ trans_cycles q) = carried_cycles orig + trans_cycles" @@ -638,9 +662,9 @@ definition request_submission_pre :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) enve \ ( request.canister_id req \ ic_principal \ - (case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req)) of - (Some (Some can), Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status\ in + (case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req), list_map_get (canister_version S) (request.canister_id req), list_map_get (global_timer S) (request.canister_id req)) of + (Some (Some can), Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status, canister_version = idx\ in (case canister_module_inspect_message (module can) (request.method_name req, wasm_state can, request.arg req, principal_of_uid (request.sender req), env) of Inr ret \ cycles_return.return ret = Accept \ cycles_return.cycles_used ret \ bal | _ \ False) @@ -654,9 +678,9 @@ definition request_submission_post :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) env let req = projl (content E); cid = request.canister_id req; balances = (if cid \ ic_principal then - (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of - (Some (Some can), Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) (request.canister_id req), list_map_get (global_timer S) (request.canister_id req)) of + (Some (Some can), Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = idx\ in (case canister_module_inspect_message (module can) (request.method_name req, wasm_state can, request.arg req, principal_of_uid (request.sender req), env) of Inr ret \ list_map_set (balances S) cid (bal - cycles_return.cycles_used ret))) else balances S) in @@ -667,9 +691,9 @@ definition request_submission_burned_cycles :: "('b, 'p, 'uid, 'canid, 's, 'pk, let req = projl (content E); cid = request.canister_id req in (if request.canister_id req \ ic_principal then - (case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req)) of - (Some (Some can), Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status\ in + (case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req), list_map_get (canister_version S) (request.canister_id req), list_map_get (global_timer S) (request.canister_id req)) of + (Some (Some can), Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status, canister_version = idx\ in (case canister_module_inspect_message (module can) (request.method_name req, wasm_state can, request.arg req, principal_of_uid (request.sender req), env) of Inr ret \ cycles_return.cycles_used ret)) else 0))" @@ -683,9 +707,9 @@ proof - by (auto simp: request_submission_pre_def split: sum.splits) { assume "request.canister_id req \ ic_principal" - then have "(case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req)) of - (Some (Some can), Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status\ in + then have "(case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req), list_map_get (canister_version S) (request.canister_id req), list_map_get (global_timer S) (request.canister_id req)) of + (Some (Some can), Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status, canister_version = idx\ in (case canister_module_inspect_message (module can) (request.method_name req, wasm_state can, request.arg req, principal_of_uid (request.sender req), env) of Inr ret \ cycles_return.return ret = Accept \ cycles_return.cycles_used ret \ bal | _ \ False) @@ -706,7 +730,7 @@ lemma request_submission_ic_inv: shows "ic_inv (request_submission_post E ECID S)" using assms by (auto simp: ic_inv_def request_submission_pre_def request_submission_post_def Let_def - split: sum.splits message.splits call_origin.splits) + split: sum.splits message.splits call_origin.splits prod.splits) @@ -868,6 +892,16 @@ lemma call_context_create_ic_inv: (* System transition: Call context creation: Heartbeat [DONE] *) +lift_definition create_call_ctxt_system_task :: "'canid \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt" is + "\cee. \canister = cee, origin = From_system, needs_to_respond = False, deleted = False, available_cycles = 0\" + by auto + +lemma create_call_ctxt_system_task_needs_to_respond[simp]: "call_ctxt_needs_to_respond (create_call_ctxt_system_task cee) = False" + by transfer auto + +lemma create_call_ctxt_system_task_carried_cycles[simp]: "call_ctxt_carried_cycles (create_call_ctxt_system_task cee) = 0" + by transfer auto + definition call_context_heartbeat_pre :: "'canid \ 'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "call_context_heartbeat_pre cee ctxt_id S = ( (case list_map_get (canisters S) cee of Some (Some can) \ True | _ \ False) \ @@ -875,21 +909,11 @@ definition call_context_heartbeat_pre :: "'canid \ 'cid \ bal \ ic_freezing_limit S cee + MAX_CYCLES_PER_MESSAGE | _ \ False) \ ctxt_id \ list_map_dom (call_contexts S))" -lift_definition create_call_ctxt_heartbeat :: "'canid \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt" is - "\cee. \canister = cee, origin = From_heartbeat, needs_to_respond = False, deleted = False, available_cycles = 0\" - by auto - -lemma create_call_ctxt_heartbeat_needs_to_respond[simp]: "call_ctxt_needs_to_respond (create_call_ctxt_heartbeat cee) = False" - by transfer auto - -lemma create_call_ctxt_heartbeat_carried_cycles[simp]: "call_ctxt_carried_cycles (create_call_ctxt_heartbeat cee) = 0" - by transfer auto - definition call_context_heartbeat_post :: "'canid \ 'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where "call_context_heartbeat_post cee ctxt_id S = (case list_map_get (balances S) cee of Some bal \ S\messages := Func_message ctxt_id cee Heartbeat (Queue System cee) # messages S, - call_contexts := list_map_set (call_contexts S) ctxt_id (create_call_ctxt_heartbeat cee), + call_contexts := list_map_set (call_contexts S) ctxt_id (create_call_ctxt_system_task cee), balances := list_map_set (balances S) cee (bal - MAX_CYCLES_PER_MESSAGE)\)" lemma call_context_heartbeat_cycles_inv: @@ -910,17 +934,54 @@ lemma call_context_heartbeat_ic_inv: +(* System transition: Call context creation: Global timer [DONE] *) + +definition call_context_global_timer_pre :: "'canid \ 'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "call_context_global_timer_pre cee ctxt_id S = ( + (case list_map_get (canisters S) cee of Some (Some can) \ True | _ \ False) \ + list_map_get (canister_status S) cee = Some Running \ + (case (list_map_get (time S) cee, list_map_get (global_timer S) cee) of (Some t, Some timer) \ timer \ 0 \ t \ timer | _ \ False) \ + (case list_map_get (balances S) cee of Some bal \ bal \ ic_freezing_limit S cee + MAX_CYCLES_PER_MESSAGE | _ \ False) \ + ctxt_id \ list_map_dom (call_contexts S))" + +definition call_context_global_timer_post :: "'canid \ 'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "call_context_global_timer_post cee ctxt_id S = + (case list_map_get (balances S) cee of Some bal \ + S\messages := Func_message ctxt_id cee Global_timer (Queue System cee) # messages S, + call_contexts := list_map_set (call_contexts S) ctxt_id (create_call_ctxt_system_task cee), + global_timer := list_map_set (global_timer S) cee 0, + balances := list_map_set (balances S) cee (bal - MAX_CYCLES_PER_MESSAGE)\)" + +lemma call_context_global_timer_cycles_inv: + assumes "call_context_global_timer_pre cee ctxt_id S" + shows "total_cycles S = total_cycles (call_context_global_timer_post cee ctxt_id S)" + using assms list_map_sum_in_ge[of "balances S" cee, where ?g=id, simplified] + by (auto simp: call_context_global_timer_pre_def call_context_global_timer_post_def total_cycles_def + list_map_sum_in[where ?g=id, simplified] list_map_sum_out split: option.splits) + +lemma call_context_global_timer_ic_inv: + assumes "call_context_global_timer_pre cee ctxt_id S" "ic_inv S" + shows "ic_inv (call_context_global_timer_post cee ctxt_id S)" + using assms + by (auto simp: ic_inv_def call_context_global_timer_pre_def call_context_global_timer_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD + intro: ic_can_status_inv_mono2) + + + (* System transition: Message execution [DONE] *) fun query_as_update :: "(('b arg \ 'p \ 'b env) \ ('w, 'b, 's) query_func) \ 'b arg \ 'p \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" where "query_as_update (f, a, e) = (\w. case f (a, e) w of Inl t \ Inl t | - Inr res \ Inr \update_return.new_state = w, update_return.new_calls = [], update_return.new_certified_data = None, + Inr res \ Inr \update_return.new_state = w, update_return.new_calls = [], update_return.new_certified_data = None, update_return.new_global_timer = None, update_return.response = Some (query_return.response res), update_return.cycles_accepted = 0, update_return.cycles_used = query_return.cycles_used res\)" -fun heartbeat_as_update :: "('b env \ 'w heartbeat_func) \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" where - "heartbeat_as_update (f, e) = (\w. case f e w of Inl t \ Inl t | - Inr res \ Inr \update_return.new_state = heartbeat_return.new_state res, update_return.new_calls = [], update_return.new_certified_data = None, - update_return.response = None, update_return.cycles_accepted = 0, update_return.cycles_used = heartbeat_return.cycles_used res\)" +fun system_task_as_update :: "('b env \ ('w, 'p, 'canid, 's, 'b, 'c) system_task_func) \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" where + "system_task_as_update (f, e) = (\w. case f e w of Inl t \ Inl t | + Inr res \ Inr \update_return.new_state = system_task_return.new_state res, update_return.new_calls = system_task_return.new_calls res, + update_return.new_certified_data = system_task_return.new_certified_data res, update_return.new_global_timer = system_task_return.new_global_timer res, + update_return.response = None, update_return.cycles_accepted = 0, update_return.cycles_used = system_task_return.cycles_used res\)" fun exec_function :: "('s, 'p, 'b, 'c) entry_point \ 'b env \ nat \ ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" where "exec_function (entry_point.Public_method mn c a) e bal m = ( @@ -930,25 +991,26 @@ fun exec_function :: "('s, 'p, 'b, 'c) entry_point \ 'b env \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "message_execution_pre n S = (n < length (messages S) \ (case messages S ! n of Func_message ctxt_id recv ep q \ (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ (case (list_map_get (canisters S) recv, list_map_get (balances S) recv, list_map_get (canister_status S) recv, - list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id) of - (Some (Some can), Some bal, Some can_status, Some t, Some ctxt) \ True | _ \ False) + list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id, list_map_get (canister_version S) recv, list_map_get (global_timer S) recv) of + (Some (Some can), Some bal, Some can_status, Some t, Some ctxt, Some idx, Some timer) \ True | _ \ False) | _ \ False))" definition message_execution_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where "message_execution_post n S = (case messages S ! n of Func_message ctxt_id recv ep q \ (case (list_map_get (canisters S) recv, list_map_get (balances S) recv, list_map_get (canister_status S) recv, - list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id) of - (Some (Some can), Some bal, Some can_status, Some t, Some ctxt) \ ( + list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id, list_map_get (canister_version S) recv, list_map_get (global_timer S) recv) of + (Some (Some can), Some bal, Some can_status, Some t, Some ctxt, Some idx, Some timer) \ ( let Mod = module can; Is_response = (case ep of Callback _ _ _ \ True | _ \ False); - Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\; + Env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status, canister_version = idx\; Available = call_ctxt_available_cycles ctxt; F = exec_function ep Env Available Mod; R = F (wasm_state can); @@ -972,11 +1034,13 @@ definition message_execution_post :: "nat \ ('p, 'uid, 'canid, 'b, ' new_ctxt = (case update_return.response result of None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt | Some _ \ call_ctxt_respond ctxt); - certified_data = (case new_certified_data result of None \ certified_data S - | Some cd \ list_map_set (certified_data S) recv cd) + certified_data = (case update_return.new_certified_data result of None \ certified_data S + | Some cd \ list_map_set (certified_data S) recv cd); + global_timer = (case update_return.new_global_timer result of None \ global_timer S + | Some new_timer \ list_map_set (global_timer S) recv new_timer) in S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), messages := messages, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, - certified_data := certified_data, balances := list_map_set (balances S) recv New_balance\) + certified_data := certified_data, global_timer := global_timer, balances := list_map_set (balances S) recv New_balance\) else S\messages := take n (messages S) @ drop (Suc n) (messages S), balances := list_map_set (balances S) recv ((bal + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) - min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))\)) @@ -985,11 +1049,11 @@ definition message_execution_post :: "nat \ ('p, 'uid, 'canid, 'b, ' definition message_execution_burned_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where "message_execution_burned_cycles n S = (case messages S ! n of Func_message ctxt_id recv ep q \ (case (list_map_get (canisters S) recv, list_map_get (balances S) recv, list_map_get (canister_status S) recv, - list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id) of - (Some (Some can), Some bal, Some can_status, Some t, Some ctxt) \ ( + list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id, list_map_get (canister_version S) recv, list_map_get (global_timer S) recv) of + (Some (Some can), Some bal, Some can_status, Some t, Some ctxt, Some idx, Some timer) \ ( let Mod = module can; Is_response = (case ep of Callback _ _ _ \ True | _ \ False); - Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\; + Env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status, canister_version = idx\; Available = call_ctxt_available_cycles ctxt; F = exec_function ep Env Available Mod; R = F (wasm_state can); @@ -1001,23 +1065,25 @@ lemma message_execution_cycles_monotonic: assumes pre: "message_execution_pre n S" shows "total_cycles S = total_cycles (message_execution_post n S) + message_execution_burned_cycles n S" proof - - obtain ctxt_id recv ep q can bal can_status t ctxt where msg: "messages S ! n = Func_message ctxt_id recv ep q" + obtain ctxt_id recv ep q can bal can_status t ctxt idx timer where msg: "messages S ! n = Func_message ctxt_id recv ep q" and prod: "list_map_get (canisters S) recv = Some (Some can)" "list_map_get (balances S) recv = Some bal" "list_map_get (canister_status S) recv = Some can_status" "list_map_get (time S) recv = Some t" "list_map_get (call_contexts S) ctxt_id = Some ctxt" + "list_map_get (canister_version S) recv = Some idx" + "list_map_get (global_timer S) recv = Some timer" using pre by (auto simp: message_execution_pre_def split: message.splits option.splits) define Mod where "Mod = can_state_rec.module can" define Is_response where "Is_response = (case ep of Callback _ _ _ \ True | _ \ False)" - define Env :: "'b env" where "Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\" + define Env :: "'b env" where "Env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status, canister_version = idx\" define Available where "Available = call_ctxt_available_cycles ctxt" define F where "F = exec_function ep Env Available Mod" define R where "R = F (wasm_state can)" define cyc_used where "cyc_used = (case R of Inr res \ update_return.cycles_used res | Inl trap \ cycles_return.cycles_used trap)" - obtain cycles_accepted_res new_calls_res where res: "(cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, new_calls res))" - by (cases "(case R of Inr res \ (update_return.cycles_accepted res, new_calls res))") auto + obtain cycles_accepted_res new_calls_res where res: "(cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, update_return.new_calls res))" + by (cases "(case R of Inr res \ (update_return.cycles_accepted res, update_return.new_calls res))") auto define New_balance where "New_balance = bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - (cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res))" define no_response where "no_response = (case R of Inr result \ update_return.response result = None)" @@ -1068,13 +1134,15 @@ proof - define new_ctxt where "new_ctxt = (case update_return.response result of None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt | Some _ \ call_ctxt_respond ctxt)" - define certified_data where "certified_data = (case new_certified_data result of None \ ic.certified_data S + define certified_data where "certified_data = (case update_return.new_certified_data result of None \ ic.certified_data S | Some cd \ list_map_set (ic.certified_data S) recv cd)" + define global_timer where "global_timer = (case update_return.new_global_timer result of None \ ic.global_timer S + | Some new_timer \ list_map_set (ic.global_timer S) recv new_timer)" define S' where "S' = S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), messages := messages, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, - certified_data := certified_data, balances := list_map_set (balances S) recv New_balance\" + certified_data := certified_data, global_timer := global_timer, balances := list_map_set (balances S) recv New_balance\" have cycles_accepted_res_def: "cycles_accepted_res = update_return.cycles_accepted result" - and new_calls_res_def: "new_calls_res = new_calls result" + and new_calls_res_def: "new_calls_res = update_return.new_calls result" using res by (auto simp: R_Inr) have no_response: "no_response = (update_return.response result = None)" @@ -1085,7 +1153,7 @@ proof - by (simp_all add: message_execution_post_def message_execution_burned_cycles_def Let_def msg prod Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] cyc_used_def[symmetric] res[symmetric] New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric] - messages_def[symmetric] new_ctxt_def[symmetric] certified_data_def[symmetric] S'_def[symmetric] + messages_def[symmetric] new_ctxt_def[symmetric] certified_data_def[symmetric] global_timer_def[symmetric] S'_def[symmetric] result_def[symmetric] response_messages_def[symmetric] new_call_to_message_def[symmetric] del: min_less_iff_conj split del: if_split) have "message_cycles \ new_call_to_message = (\c. MAX_CYCLES_PER_RESPONSE + transferred_cycles c)" for c :: "(?'p, 'canid, 's, 'b, 'c) method_call" @@ -1114,14 +1182,16 @@ qed lemma message_execution_cases: assumes "message_execution_pre n S" - "\ctxt_id recv ep q can bal can_status t ctxt Mod Is_response Env Available F R cyc_used cycles_accepted_res new_calls_res New_balance no_response result new_call_to_message response_messages msgs new_ctxt cert_data idx. + "\ctxt_id recv ep q can bal can_status t ctxt Mod Is_response Env Available F R cyc_used cycles_accepted_res new_calls_res New_balance no_response result new_call_to_message response_messages msgs new_ctxt cert_data idx timer glob_timer. messages S ! n = Func_message ctxt_id recv ep q \ list_map_get (canisters S) recv = Some (Some can) \ list_map_get (balances S) recv = Some bal \ list_map_get (canister_status S) recv = Some can_status \ list_map_get (time S) recv = Some t \ list_map_get (call_contexts S) ctxt_id = Some ctxt \ + list_map_get (canister_version S) recv = Some idx \ + list_map_get (global_timer S) recv = Some timer \ Mod = module can \ Is_response = (case ep of Callback _ _ _ \ True | _ \ False) \ - Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\ \ + Env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status, canister_version = idx\ \ Available = call_ctxt_available_cycles ctxt \ F = exec_function ep Env Available Mod \ R = F (wasm_state can) \ @@ -1145,19 +1215,23 @@ lemma message_execution_cases: new_ctxt = (case update_return.response result of None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt | Some _ \ call_ctxt_respond ctxt) \ - cert_data = (case new_certified_data result of None \ certified_data S + cert_data = (case update_return.new_certified_data result of None \ certified_data S | Some cd \ list_map_set (certified_data S) recv cd) \ + glob_timer = (case update_return.new_global_timer result of None \ global_timer S + | Some new_timer \ list_map_set (global_timer S) recv new_timer) \ P n S (S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), messages := msgs, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, - certified_data := cert_data, balances := list_map_set (balances S) recv New_balance\)" - "\ctxt_id recv ep q can bal can_status t ctxt Mod Is_response Env Available F R cyc_used cycles_accepted_res new_calls_res New_balance no_response idx. + certified_data := cert_data, global_timer := glob_timer, balances := list_map_set (balances S) recv New_balance\)" + "\ctxt_id recv ep q can bal can_status t ctxt Mod Is_response Env Available F R cyc_used cycles_accepted_res new_calls_res New_balance no_response idx timer glob_timer. messages S ! n = Func_message ctxt_id recv ep q \ list_map_get (canisters S) recv = Some (Some can) \ list_map_get (balances S) recv = Some bal \ list_map_get (canister_status S) recv = Some can_status \ list_map_get (time S) recv = Some t \ list_map_get (call_contexts S) ctxt_id = Some ctxt \ + list_map_get (canister_version S) recv = Some idx \ + list_map_get (global_timer S) recv = Some timer \ Mod = module can \ Is_response = (case ep of Callback _ _ _ \ True | _ \ False) \ - Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\ \ + Env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status, canister_version = idx\ \ Available = call_ctxt_available_cycles ctxt \ F = exec_function ep Env Available Mod \ R = F (wasm_state can) \ @@ -1177,23 +1251,25 @@ lemma message_execution_cases: - min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))\)" shows "P n S (message_execution_post n S)" proof - - obtain ctxt_id recv ep q can bal can_status t ctxt where msg: "messages S ! n = Func_message ctxt_id recv ep q" + obtain ctxt_id recv ep q can bal can_status t ctxt idx timer where msg: "messages S ! n = Func_message ctxt_id recv ep q" and prod: "list_map_get (canisters S) recv = Some (Some can)" "list_map_get (balances S) recv = Some bal" "list_map_get (canister_status S) recv = Some can_status" "list_map_get (time S) recv = Some t" "list_map_get (call_contexts S) ctxt_id = Some ctxt" + "list_map_get (canister_version S) recv = Some idx" + "list_map_get (global_timer S) recv = Some timer" using assms(1) by (auto simp: message_execution_pre_def split: message.splits option.splits) define Mod where "Mod = can_state_rec.module can" define Is_response where "Is_response = (case ep of Callback _ _ _ \ True | _ \ False)" - define Env :: "'b env" where "Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\" + define Env :: "'b env" where "Env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status, canister_version = idx\" define Available where "Available = call_ctxt_available_cycles ctxt" define F where "F = exec_function ep Env Available Mod" define R where "R = F (wasm_state can)" define cyc_used where "cyc_used = (case R of Inr res \ update_return.cycles_used res | Inl trap \ cycles_return.cycles_used trap)" - obtain cycles_accepted_res new_calls_res where res: "(cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, new_calls res))" - by (cases "(case R of Inr res \ (update_return.cycles_accepted res, new_calls res))") auto + obtain cycles_accepted_res new_calls_res where res: "(cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, update_return.new_calls res))" + by (cases "(case R of Inr res \ (update_return.cycles_accepted res, update_return.new_calls res))") auto define New_balance where "New_balance = bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - (cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res))" define no_response where "no_response = (case R of Inr result \ update_return.response result = None)" @@ -1230,21 +1306,23 @@ lemma message_execution_cases: define new_ctxt where "new_ctxt = (case update_return.response result of None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt | Some _ \ call_ctxt_respond ctxt)" - define certified_data where "certified_data = (case new_certified_data result of None \ ic.certified_data S + define certified_data where "certified_data = (case update_return.new_certified_data result of None \ ic.certified_data S | Some cd \ list_map_set (ic.certified_data S) recv cd)" + define global_timer where "global_timer = (case update_return.new_global_timer result of None \ ic.global_timer S + | Some new_timer \ list_map_set (ic.global_timer S) recv new_timer)" define S' where "S' = S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), messages := messages, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, - certified_data := certified_data, balances := list_map_set (balances S) recv New_balance\" + certified_data := certified_data, global_timer := global_timer, balances := list_map_set (balances S) recv New_balance\" have msg_exec: "message_execution_post n S = S'" using True by (simp_all add: message_execution_post_def Let_def msg prod Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] cyc_used_def[symmetric] res[symmetric] New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric] - messages_def[symmetric] new_ctxt_def[symmetric] certified_data_def[symmetric] S'_def[symmetric] + messages_def[symmetric] new_ctxt_def[symmetric] certified_data_def[symmetric] global_timer_def[symmetric] S'_def[symmetric] result_def[symmetric] response_messages_def[symmetric] new_call_to_message_def[symmetric]) show ?thesis using assms(2)[OF msg prod Mod_def Is_response_def Env_def Available_def F_def R_def cyc_used_def res New_balance_def no_response_def _ _ _ _ _ _ result_def - new_call_to_message_def response_messages_def messages_def new_ctxt_def certified_data_def, folded S'_def] True + new_call_to_message_def response_messages_def messages_def new_ctxt_def certified_data_def global_timer_def, folded S'_def] True by (auto simp: cond_def msg_exec) qed qed @@ -1253,7 +1331,7 @@ lemma message_execution_ic_inv: assumes "message_execution_pre n S" "ic_inv S" shows "ic_inv (message_execution_post n S)" proof (rule message_execution_cases[OF assms(1)]) - fix recv msgs ctxt_id new_ctxt cert_data New_balance new_calls_res response_messages ctxt Available cycles_accepted_res no_response idx + fix recv msgs ctxt_id new_ctxt cert_data New_balance new_calls_res response_messages ctxt Available cycles_accepted_res no_response idx glob_timer fix can :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) can_state_rec" fix result :: "('w, 'p, 'canid, 's, 'b, 'c) update_return" fix new_call_to_message :: "('p, 'canid, 's, 'b, 'c) method_call \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message" @@ -1268,7 +1346,7 @@ proof (rule message_execution_cases[OF assms(1)]) "\ isl R" "result = projr R" "new_ctxt = (case update_return.response result of None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt | Some x \ call_ctxt_respond ctxt)" then show "ic_inv (S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), messages := msgs, - call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, certified_data := cert_data, balances := list_map_set (balances S) recv New_balance\)" + call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, certified_data := cert_data, global_timer := glob_timer, balances := list_map_set (balances S) recv New_balance\)" using assms(2) list_map_get_range[OF ctxt] by (cases R) (force simp: ic_inv_def ic_can_status_inv_def split: message.splits call_origin.splits can_status.splits dest!: in_set_takeD in_set_dropD list_map_range_setD)+ @@ -1289,7 +1367,7 @@ definition call_context_starvation_pre :: "'cid \ ('p, 'uid, 'canid, "call_context_starvation_pre ctxt_id S = (case list_map_get (call_contexts S) ctxt_id of Some call_context \ call_ctxt_needs_to_respond call_context \ - call_ctxt_origin call_context \ From_heartbeat \ + call_ctxt_origin call_context \ From_system \ (\msg \ set (messages S). case msg of Call_message orig _ _ _ _ _ _ \ calling_context orig \ Some ctxt_id | Response_message orig _ _ \ calling_context orig \ Some ctxt_id @@ -1331,7 +1409,7 @@ definition call_context_removal_pre :: "'cid \ ('p, 'uid, 'canid, 'b "call_context_removal_pre ctxt_id S = ( (case list_map_get (call_contexts S) ctxt_id of Some call_context \ (\call_ctxt_needs_to_respond call_context \ - (call_ctxt_origin call_context = From_heartbeat \ + (call_ctxt_origin call_context = From_system \ (\msg \ set (messages S). case msg of Func_message other_ctxt_id _ _ _ \ other_ctxt_id \ ctxt_id | _ \ True))) \ @@ -1383,10 +1461,12 @@ definition ic_canister_creation_pre :: "nat \ 'canid \ n is_system_assigned (principal_of_canid cid) \ cid \ list_map_dom (canisters S) \ cid \ list_map_dom (time S) \ + cid \ list_map_dom (global_timer S) \ cid \ list_map_dom (controllers S) \ cid \ list_map_dom (balances S) \ cid \ list_map_dom (certified_data S) \ - cid \ list_map_dom (canister_status S) + cid \ list_map_dom (canister_status S) \ + cid \ list_map_dom (canister_version S) | _ \ False))" definition ic_canister_creation_post :: "nat \ 'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where @@ -1394,13 +1474,15 @@ definition ic_canister_creation_post :: "nat \ 'canid \ let ctrls = (case candid_parse_controllers a of Some ctrls \ ctrls | _ \ {cer}) in S\canisters := list_map_set (canisters S) cid None, time := list_map_set (time S) cid t, + global_timer := list_map_set (global_timer S) cid 0, controllers := list_map_set (controllers S) cid ctrls, freezing_threshold := list_map_set (freezing_threshold S) cid 2592000, balances := list_map_set (balances S) cid trans_cycles, certified_data := list_map_set (certified_data S) cid empty_blob, messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid (Candid_record (list_map_init [(encode_string ''canister_id'', Candid_blob (blob_of_canid cid))])))) 0], - canister_status := list_map_set (canister_status S) cid Running\)" + canister_status := list_map_set (canister_status S) cid Running, + canister_version := list_map_set (canister_version S) cid 0\)" lemma ic_canister_creation_cycles_inv: assumes "ic_canister_creation_pre n cid t S" @@ -1442,7 +1524,7 @@ definition ic_update_settings_pre :: "nat \ ('p, 'uid, 'canid, 'b, ' cee = ic_principal \ mn = encode_string ''update_settings'' \ (case candid_parse_cid a of Some cid \ - (case list_map_get (controllers S) cid of Some ctrls \ cer \ ctrls + (case (list_map_get (controllers S) cid, list_map_get (canister_version S) cid) of (Some ctrls, Some idx) \ cer \ ctrls | _ \ False) | _ \ False) | _ \ False))" @@ -1451,8 +1533,10 @@ definition ic_update_settings_post :: "nat \ ('p, 'uid, 'canid, 'b, let cid = the (candid_parse_cid a); ctrls = (case candid_parse_controllers a of Some ctrls \ list_map_set (controllers S) cid ctrls | _ \ controllers S); freezing_thres = (case candid_nested_lookup a [encode_string '''settings'', encode_string ''freezing_threshold''] of Some (Candid_nat freeze) \ - list_map_set (freezing_threshold S) cid freeze | _ \ freezing_threshold S) in - S\controllers := ctrls, freezing_threshold := freezing_thres, messages := take n (messages S) @ drop (Suc n) (messages S) @ + list_map_set (freezing_threshold S) cid freeze | _ \ freezing_threshold S); + idx = the (list_map_get (canister_version S) cid) in + S\controllers := ctrls, freezing_threshold := freezing_thres, canister_version := list_map_set (canister_version S) cid (Suc idx), + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)" lemma ic_update_settings_cycles_inv: @@ -1555,13 +1639,13 @@ definition ic_code_installation_pre :: "nat \ ('p, 'uid, 'canid, 'b, (case parse_wasm_mod w of Some m \ parse_public_custom_sections w \ None \ parse_private_custom_sections w \ None \ - (case (list_map_get (controllers S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of - (Some ctrls, Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + (case (list_map_get (controllers S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid, list_map_get (global_timer S) cid) of + (Some ctrls, Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\ in ((mode = encode_string ''install'' \ (case list_map_get (canisters S) cid of Some None \ True | _ \ False)) \ mode = encode_string ''reinstall'') \ cer \ ctrls \ (case canister_module_init m (cid, ar, cer, env) of Inl _ \ False - | Inr ret \ cycles_return.cycles_used ret \ bal) \ + | Inr ret \ init_return.cycles_used ret \ bal) \ list_map_dom (canister_module_update_methods m) \ list_map_dom (canister_module_query_methods m) = {} | _ \ False) | _ \ False) | _ \ False) | _ \ False) | _ \ False))" @@ -1572,12 +1656,16 @@ definition ic_code_installation_post :: "nat \ ('p, 'uid, 'canid, 'b (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of (Some mode, Some w, Some a) \ (case parse_wasm_mod w of Some m \ - (case (list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of - (Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\; - (new_state, cyc_used) = (case canister_module_init m (cid, a, cer, env) of Inr ret \ (cycles_return.return ret, cycles_return.cycles_used ret)) in + (case (list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid, list_map_get (global_timer S) cid) of + (Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\; + (new_state, new_certified_data, new_global_timer, cyc_used) = (case canister_module_init m (cid, a, cer, env) of Inr ret \ + (init_return.new_state ret, init_return.new_certified_data ret, init_return.new_global_timer ret, init_return.cycles_used ret)) in S\canisters := list_map_set (canisters S) cid (Some \wasm_state = new_state, module = m, raw_module = w, public_custom_sections = the (parse_public_custom_sections w), private_custom_sections = the (parse_private_custom_sections w)\), + certified_data := (case new_certified_data of None \ certified_data S | Some cd \ list_map_set (certified_data S) cid cd), + global_timer := list_map_set (global_timer S) cid (case new_global_timer of None \ 0 | Some new_timer \ new_timer), + canister_version := list_map_set (canister_version S) cid (Suc idx), balances := list_map_set (balances S) cid (bal - cyc_used), messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)))))" @@ -1587,10 +1675,10 @@ definition ic_code_installation_burned_cycles :: "nat \ ('p, 'uid, ' (case (candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of (Some w, Some a) \ (case parse_wasm_mod w of Some m \ - (case (list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of - (Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in - (case canister_module_init m (cid, a, cer, env) of Inr ret \ cycles_return.cycles_used ret))))))" + (case (list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid, list_map_get (global_timer S) cid) of + (Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\ in + (case canister_module_init m (cid, a, cer, env) of Inr ret \ init_return.cycles_used ret))))))" lemma ic_code_installation_cycles_inv: assumes "ic_code_installation_pre n S" @@ -1615,7 +1703,7 @@ lemma ic_code_installation_ic_inv: assumes "ic_code_installation_pre n S" "ic_inv S" shows "ic_inv (ic_code_installation_post n S)" proof - - obtain orig cer cee mn a trans_cycles q cid mode w ar m ctrls t bal can_status idx where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + obtain orig cer cee mn a trans_cycles q cid mode w ar m ctrls t bal can_status idx timer where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" and parse: "candid_parse_cid a = Some cid" "candid_parse_text a [encode_string ''mode''] = Some mode" "candid_parse_blob a [encode_string ''wasm_module''] = Some w" @@ -1626,6 +1714,8 @@ proof - "list_map_get (time S) cid = Some t" "list_map_get (balances S) cid = Some bal" "list_map_get (canister_status S) cid = Some can_status" + "list_map_get (canister_version S) cid = Some idx" + "list_map_get (global_timer S) cid = Some timer" using assms by (auto simp: ic_code_installation_pre_def split: message.splits option.splits) show ?thesis @@ -1650,14 +1740,15 @@ definition ic_code_upgrade_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, (case parse_wasm_mod w of Some m \ parse_public_custom_sections w \ None \ parse_private_custom_sections w \ None \ - (case (list_map_get (canisters S) cid, list_map_get (controllers S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of - (Some (Some can), Some ctrls, Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + (case (list_map_get (canisters S) cid, list_map_get (controllers S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid, list_map_get (global_timer S) cid) of + (Some (Some can), Some ctrls, Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env1 = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = idx\; + env2 = \env.time = t, env.global_timer = 0, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\ in mode = encode_string ''upgrade'' \ cer \ ctrls \ - (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env) of Inr pre_ret \ - (case canister_module_post_upgrade m (cid, cycles_return.return pre_ret, ar, cer, env) of Inr post_ret \ - cycles_return.cycles_used pre_ret + cycles_return.cycles_used post_ret \ bal + (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env1) of Inr pre_ret \ + (case canister_module_post_upgrade m (cid, pre_upgrade_return.stable_memory pre_ret, ar, cer, env2) of Inr post_ret \ + pre_upgrade_return.cycles_used pre_ret + init_return.cycles_used post_ret \ bal | _ \ False) | _ \ False) \ list_map_dom (canister_module_update_methods m) \ list_map_dom (canister_module_query_methods m) = {} | _ \ False) | _ \ False) | _ \ False) | _ \ False) @@ -1669,14 +1760,20 @@ definition ic_code_upgrade_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of (Some mode, Some w, Some ar) \ (case parse_wasm_mod w of Some m \ - (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of - (Some (Some can), Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in - (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env) of Inr pre_ret \ - (case canister_module_post_upgrade m (cid, cycles_return.return pre_ret, ar, cer, env) of Inr post_ret \ - S\canisters := list_map_set (canisters S) cid (Some \wasm_state = cycles_return.return post_ret, module = m, raw_module = w, + (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid, list_map_get (global_timer S) cid) of + (Some (Some can), Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env1 = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = idx\; + env2 = \env.time = t, env.global_timer = 0, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\ in + (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env1) of Inr pre_ret \ + (case canister_module_post_upgrade m (cid, pre_upgrade_return.stable_memory pre_ret, ar, cer, env2) of Inr post_ret \ + S\canisters := list_map_set (canisters S) cid (Some \wasm_state = init_return.new_state post_ret, module = m, raw_module = w, public_custom_sections = the (parse_public_custom_sections w), private_custom_sections = the (parse_private_custom_sections w)\), - balances := list_map_set (balances S) cid (bal - (cycles_return.cycles_used pre_ret + cycles_return.cycles_used post_ret)), + certified_data := (case init_return.new_certified_data post_ret of Some cd' \ list_map_set (certified_data S) cid cd' + | None \ (case pre_upgrade_return.new_certified_data pre_ret of Some cd \ list_map_set (certified_data S) cid cd + | None \ certified_data S)), + global_timer := list_map_set (global_timer S) cid (case init_return.new_global_timer post_ret of None \ 0 | Some new_timer \ new_timer), + canister_version := list_map_set (canister_version S) cid (Suc idx), + balances := list_map_set (balances S) cid (bal - (pre_upgrade_return.cycles_used pre_ret + init_return.cycles_used post_ret)), messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)))))))" definition ic_code_upgrade_burned_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where @@ -1685,12 +1782,13 @@ definition ic_code_upgrade_burned_cycles :: "nat \ ('p, 'uid, 'canid (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of (Some mode, Some w, Some ar) \ (case parse_wasm_mod w of Some m \ - (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of - (Some (Some can), Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in - (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env) of Inr pre_ret \ - (case canister_module_post_upgrade m (cid, cycles_return.return pre_ret, ar, cer, env) of Inr post_ret \ - cycles_return.cycles_used pre_ret + cycles_return.cycles_used post_ret)))))))" + (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid, list_map_get (global_timer S) cid) of + (Some (Some can), Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env1 = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = idx\; + env2 = \env.time = t, env.global_timer = 0, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\ in + (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env1) of Inr pre_ret \ + (case canister_module_post_upgrade m (cid, pre_upgrade_return.stable_memory pre_ret, ar, cer, env2) of Inr post_ret \ + pre_upgrade_return.cycles_used pre_ret + init_return.cycles_used post_ret)))))))" lemma ic_code_upgrade_cycles_inv: assumes "ic_code_upgrade_pre n S" @@ -1715,7 +1813,7 @@ lemma ic_code_upgrade_ic_inv: assumes "ic_code_upgrade_pre n S" "ic_inv S" shows "ic_inv (ic_code_upgrade_post n S)" proof - - obtain orig cer cee mn a trans_cycles q cid mode w ar m can ctrls t bal can_status idx where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + obtain orig cer cee mn a trans_cycles q cid mode w ar m can ctrls t bal can_status idx timer where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" and parse: "candid_parse_cid a = Some cid" "candid_parse_text a [encode_string ''mode''] = Some mode" "candid_parse_blob a [encode_string ''wasm_module''] = Some w" @@ -1727,6 +1825,8 @@ proof - "list_map_get (time S) cid = Some t" "list_map_get (balances S) cid = Some bal" "list_map_get (canister_status S) cid = Some can_status" + "list_map_get (canister_version S) cid = Some idx" + "list_map_get (global_timer S) cid = Some timer" using assms by (auto simp: ic_code_upgrade_pre_def split: message.splits option.splits) show ?thesis @@ -1746,7 +1846,7 @@ definition ic_code_uninstallation_pre :: "nat \ ('p, 'uid, 'canid, ' cee = ic_principal \ mn = encode_string ''uninstall_code'' \ (case candid_parse_cid a of Some cid \ - (case list_map_get (controllers S) cid of Some ctrls \ cer \ ctrls + (case (list_map_get (controllers S) cid, list_map_get (canister_version S) cid) of (Some ctrls, Some idx) \ cer \ ctrls | _ \ False) | _ \ False) | _ \ False))" @@ -1757,8 +1857,11 @@ definition ic_code_uninstallation_post :: "nat \ ('p, 'uid, 'canid, if call_ctxt_canister ctxt = cid \ call_ctxt_needs_to_respond ctxt then Some (Response_message (call_ctxt_origin ctxt) (response.Reject CANISTER_REJECT (encode_string ''Canister has been uninstalled'')) (call_ctxt_available_cycles ctxt)) else None); - call_ctxt_to_ctxt = (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt) in + call_ctxt_to_ctxt = (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt); + idx = the (list_map_get (canister_version S) cid) in S\canisters := list_map_set (canisters S) cid None, certified_data := list_map_set (certified_data S) cid empty_blob, + canister_version := list_map_set (canister_version S) cid (Suc idx), + global_timer := list_map_set (global_timer S) cid 0, messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles] @ List.map_filter call_ctxt_to_msg (list_map_vals (call_contexts S)), call_contexts := list_map_map call_ctxt_to_ctxt (call_contexts S)\))" @@ -2142,7 +2245,9 @@ definition ic_canister_deletion_post :: "nat \ ('p, 'uid, 'canid, 'b controllers := list_map_del (controllers S) cid, freezing_threshold := list_map_del (freezing_threshold S) cid, canister_status := list_map_del (canister_status S) cid, + canister_version := list_map_del (canister_version S) cid, time := list_map_del (time S) cid, + global_timer := list_map_del (global_timer S) cid, balances := list_map_del (balances S) cid, certified_data := list_map_del (certified_data S) cid, messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)" @@ -2292,10 +2397,12 @@ definition ic_provisional_canister_creation_pre :: "nat \ 'canid \ cid \ list_map_dom (canisters S) \ cid \ list_map_dom (time S) \ + cid \ list_map_dom (global_timer S) \ cid \ list_map_dom (controllers S) \ cid \ list_map_dom (balances S) \ cid \ list_map_dom (certified_data S) \ - cid \ list_map_dom (canister_status S) + cid \ list_map_dom (canister_status S) \ + cid \ list_map_dom (canister_version S) | _ \ False) | _ \ False))" definition ic_provisional_canister_creation_post :: "nat \ 'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where @@ -2303,13 +2410,15 @@ definition ic_provisional_canister_creation_post :: "nat \ 'canid \< let cyc = the (candid_parse_nat a [encode_string ''amount'']) in S\canisters := list_map_set (canisters S) cid None, time := list_map_set (time S) cid t, + global_timer := list_map_set (global_timer S) cid 0, controllers := list_map_set (controllers S) cid {cer}, freezing_threshold := list_map_set (freezing_threshold S) cid 2592000, balances := list_map_set (balances S) cid cyc, certified_data := list_map_set (certified_data S) cid empty_blob, messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid (Candid_record (list_map_init [(encode_string ''canister_id'', Candid_blob (blob_of_canid cid))])))) trans_cycles], - canister_status := list_map_set (canister_status S) cid Running\)" + canister_status := list_map_set (canister_status S) cid Running, + canister_version := list_map_set (canister_version S) cid 0\)" definition ic_provisional_canister_creation_minted_cycles :: "nat \ 'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where "ic_provisional_canister_creation_minted_cycles n cid t S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ @@ -2602,7 +2711,7 @@ lemma request_cleanup_expired_ic_inv: (* System transition: Canister out of cycles [DONE] *) definition canister_out_of_cycles_pre :: "'canid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where - "canister_out_of_cycles_pre cid S = (case list_map_get (balances S) cid of Some 0 \ True + "canister_out_of_cycles_pre cid S = (case (list_map_get (balances S) cid, list_map_get (canister_version S) cid) of (Some 0, Some idx) \ True | _ \ False)" definition canister_out_of_cycles_post :: "'canid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where @@ -2611,8 +2720,11 @@ definition canister_out_of_cycles_post :: "'canid \ ('p, 'uid, 'cani if call_ctxt_canister ctxt = cid \ call_ctxt_needs_to_respond ctxt then Some (Response_message (call_ctxt_origin ctxt) (response.Reject CANISTER_REJECT (encode_string ''Canister has been uninstalled'')) (call_ctxt_available_cycles ctxt)) else None); - call_ctxt_to_ctxt = (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt) in + call_ctxt_to_ctxt = (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt); + idx = the (list_map_get (canister_version S) cid) in S\canisters := list_map_set (canisters S) cid None, certified_data := list_map_set (certified_data S) cid empty_blob, + canister_version := list_map_set (canister_version S) cid (Suc idx), + global_timer := list_map_set (global_timer S) cid 0, messages := messages S @ List.map_filter call_ctxt_to_msg (list_map_vals (call_contexts S)), call_contexts := list_map_map call_ctxt_to_ctxt (call_contexts S)\)" @@ -2644,7 +2756,7 @@ lemma canister_out_of_cycles_ic_inv: -(* System transition: Time progressing and cycle consumption (canister time) [DONE] *) +(* System transition: Time progressing, cycle consumption, and canister version increments (canister time) [DONE] *) definition canister_time_progress_pre :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "canister_time_progress_pre cid t1 S = (case list_map_get (time S) cid of Some t0 \ @@ -2670,7 +2782,7 @@ lemma canister_time_progress_ic_inv: -(* System transition: Time progressing and cycle consumption (cycle consumption) [DONE] *) +(* System transition: Time progressing, cycle consumption, and canister version increments (cycle consumption) [DONE] *) definition cycle_consumption_pre :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "cycle_consumption_pre cid b1 S = (case list_map_get (balances S) cid of Some b0 \ @@ -2701,7 +2813,7 @@ lemma cycle_consumption_ic_inv: -(* System transition: Time progressing and cycle consumption (system time) [DONE] *) +(* System transition: Time progressing, cycle consumption, and canister version increments (system time) [DONE] *) definition system_time_progress_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "system_time_progress_pre t1 S = (system_time S < t1)" @@ -2725,6 +2837,32 @@ lemma system_time_progress_ic_inv: +(* System transition: Time progressing, cycle consumption, and canister version increments (canister version) [DONE] *) + +definition canister_version_progress_pre :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "canister_version_progress_pre cid n1 S = (case list_map_get (canister_version S) cid of Some n0 \ + n0 < n1 + | _ \ False)" + +definition canister_version_progress_post :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "canister_version_progress_post cid n1 S = (S\canister_version := list_map_set (canister_version S) cid n1\)" + +lemma canister_version_progress_cycles_inv: + assumes "canister_version_progress_pre cid n1 S" + shows "total_cycles S = total_cycles (canister_version_progress_post cid n1 S)" + by (auto simp: canister_version_progress_post_def total_cycles_def) + +lemma canister_version_progress_ic_inv: + assumes "canister_version_progress_pre cid n1 S" "ic_inv S" + shows "ic_inv (canister_version_progress_post cid n1 S)" + using assms + by (auto simp: ic_inv_def canister_version_progress_pre_def canister_version_progress_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + (* State machine *) inductive ic_steps :: "'sig itself \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ @@ -2765,6 +2903,7 @@ inductive ic_steps :: "'sig itself \ ('p, 'uid, 'canid, 'b, 'w, 'sm, | canister_time_progress: "ic_steps sig S0 minted burned S \ canister_time_progress_pre cid t1 S \ ic_steps sig S0 minted burned (canister_time_progress_post cid t1 S)" | cycle_consumption: "ic_steps sig S0 minted burned S \ cycle_consumption_pre cid b1 S \ ic_steps sig S0 minted (burned + cycle_consumption_burned_cycles cid b1 S) (cycle_consumption_post cid b1 S)" | system_time_progress: "ic_steps sig S0 minted burned S \ system_time_progress_pre t1 S \ ic_steps sig S0 minted burned (system_time_progress_post t1 S)" +| canister_version_progress: "ic_steps sig S0 minted burned S \ canister_version_progress_pre cid n1 S \ ic_steps sig S0 minted burned (canister_version_progress_post cid n1 S)" lemma total_cycles: assumes "ic_steps TYPE('sig) S0 minted burned S" @@ -2807,6 +2946,7 @@ lemma total_cycles: using canister_time_progress_cycles_inv apply fastforce using cycle_consumption_cycles_monotonic apply fastforce using system_time_progress_cycles_inv apply fastforce + using canister_version_progress_cycles_inv apply fastforce done lemma ic_inv: @@ -2850,6 +2990,7 @@ lemma ic_inv: using canister_time_progress_ic_inv apply fastforce using cycle_consumption_ic_inv apply fastforce using system_time_progress_ic_inv apply fastforce + using canister_version_progress_ic_inv apply fastforce done end @@ -2889,6 +3030,7 @@ export_code request_submission_pre request_submission_post canister_time_progress_pre canister_time_progress_post cycle_consumption_pre cycle_consumption_post system_time_progress_pre system_time_progress_post + canister_version_progress_pre canister_version_progress_post in Haskell module_name IC file_prefix code end From 05dc3dd11f725790364597d0caa846ca4a090b41 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Wed, 7 Dec 2022 18:58:19 +0100 Subject: [PATCH 048/102] updated .md to version 0.8.19 --- spec/md-spec/ic-interface-spec.md | 523 +++++++++++++++++------ spec/md-spec/interface-spec-changelog.md | 6 + 2 files changed, 391 insertions(+), 138 deletions(-) diff --git a/spec/md-spec/ic-interface-spec.md b/spec/md-spec/ic-interface-spec.md index bc716168b..25fe0656d 100644 --- a/spec/md-spec/ic-interface-spec.md +++ b/spec/md-spec/ic-interface-spec.md @@ -233,9 +233,9 @@ The canister status can be used to control whether the canister is processing ca - In status `running`, calls to the canister are processed as normal. -- In status `stopping`, calls to the canister are rejected by the IC, but responses to the canister are processed as normal. +- In status `stopping`, calls to the canister are rejected by the IC with reject code `CANISTER_ERROR` (5), but responses to the canister are processed as normal. -- In status `stopped`, calls to the canister are rejected by the IC, and there are no outstanding responses to call contexts that are not marked as deleted. +- In status `stopped`, calls to the canister are rejected by the IC with reject code `CANISTER_ERROR` (5), and there are no outstanding responses to call contexts that are not marked as deleted. In all cases, calls to the [management canister](#the-ic-management-canister) are processed, regardless of the state of the managed canister. @@ -243,7 +243,7 @@ The controllers of the canister can initiate transitions between these states us :::note -This status is orthogonal to the question of whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected, but because the canister is empty. +This status is orthogonal to whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected by the IC with an HTTP error, but because the canister is empty. ::: ### Signatures @@ -920,6 +920,8 @@ In order for a WebAssembly module to be usable as the code for the canister, it - If it exports a function called `canister_heartbeat`, the function must have type `() -> ()`. +- If it exports a function called `canister_global_timer`, the function must have type `+() -> ()+`. + - If it exports any functions called `canister_update ` or `canister_query ` for some `name`, the functions must have type `() -> ()`. - It may not export both `canister_update ` and `canister_query ` with the same `name`. @@ -950,6 +952,8 @@ The canister provides entry points which are invoked by the IC under various cir - The canister may export a function with name `canister_heartbeat` with type `() -> ()`. +- The canister may export a function with name `canister_global_timer` with type `+() -> ()+`. + - The canister may export functions with name `canister_update ` and type `() -> ()`. - The canister may export functions with name `canister_query ` and type `() -> ()`. @@ -996,12 +1000,24 @@ Eventually, a method will want to send a response, using `ic0.reply` or `ic0.rej For periodic or time-based execution, the WebAssembly module can export a function with name `canister_heartbeat`. The heartbeats scheduling algorithm is implementation-defined. -`canister_heartbeat` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. +`canister_heartbeat` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. Still, the function `canister_heartbeat` can initiate new calls. :::note While an implementation will likely try to keep the interval between `canister_heartbeat` invocations to within a few seconds, this is not formally part of this specification. ::: +#### Global timer {#timer} + +For time-based execution, the WebAssembly module can export a function with name `canister_global_timer`. This function is called if the canister has set its global timer (using the System API function `ic0.global_timer_set`) and the current time (as returned by the System API function `ic0.time`) has passed the value of the global timer. + +Once the function `canister_global_timer` is scheduled, the canister's global timer is deactivated. The global timer is also deactivated upon changes to the canister's Wasm module (calling `install_code`, `uninstall_code` methods of the management canister or if the canister runs out of cycles). In particular, the function `canister_global_timer` won't be scheduled again unless the canister sets the global timer again (using the System API function `ic0.global_timer_set`). The global timer scheduling algorithm is implementation-defined. + +`canister_global_timer` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. Still, the function `canister_global_timer` can initiate new calls. ++ +:::note +While an implementation will likely try to keep the interval between the value of the global timer and the time-stamp of the `canister_global_timer` invocation within a few seconds, this is not formally part of this specification. +::: + #### Callbacks {#callbacks} Callbacks are addressed by their table index (as a proxy for a Wasm `funcref`). @@ -1038,12 +1054,12 @@ The following sections describe various System API functions, also referred to a ic0.canister_cycle_balance : () -> i64; // * ic0.canister_cycle_balance128 : (dst : i32) -> (); // * ic0.canister_status : () -> i32; // * - + ic0.canister_version : () -> i64; // * ic0.msg_method_name_size : () -> i32 // F ic0.msg_method_name_copy : (dst : i32, offset : i32, size : i32) -> (); // F ic0.accept_message : () -> (); // F - ic0.call_new : // U Ry Rt H + ic0.call_new : // U Ry Rt T ( callee_src : i32, callee_size : i32, name_src : i32, @@ -1053,11 +1069,11 @@ The following sections describe various System API functions, also referred to a reject_fun : i32, reject_env : i32 ) -> (); - ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt H - ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt H - ic0.call_cycles_add : (amount : i64) -> (); // U Ry Rt H - ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt H - ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt H + ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt T + ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt T + ic0.call_cycles_add : (amount : i64) -> (); // U Ry Rt T + ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt T + ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt T ic0.stable_size : () -> (page_count : i32); // * ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32); // * @@ -1068,12 +1084,13 @@ The following sections describe various System API functions, also referred to a ic0.stable64_write : (offset : i64, src : i64, size : i64) -> (); // * ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> (); // * - ic0.certified_data_set : (src: i32, size: i32) -> () // I G U Ry Rt H + ic0.certified_data_set : (src: i32, size: i32) -> () // I G U Ry Rt T ic0.data_certificate_present : () -> i32 // * ic0.data_certificate_size : () -> i32 // * ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> () // * ic0.time : () -> (timestamp : i64); // * + ic0.global_timer_set : (timestamp : i64) -> i64; // I U Ry Rt C T ic0.performance_counter : (type : i32) -> (counter : i64); // * s ic0.debug_print : (src : i32, size : i32) -> (); // * s @@ -1099,9 +1116,9 @@ The comment after each function lists from where these functions may be invoked: - `F`: from `canister_inspect_message` -- `H`: from `canister_heartbeat` +- `T`: from _system task_ (`canister_heartbeat` or `canister_global_timer`) -- `*` = `I G U Q Ry Rt C F H` (NB: Not `(start)`) +- `*` = `I G U Q Ry Rt C F T` (NB: Not `(start)`) If the canister invokes a system call from somewhere else, it will trap. @@ -1222,6 +1239,18 @@ This function allows a canister to find out if it is running, stopping or stoppe Status `3` (stopped) can be observed, for example, in `canister_pre_upgrade` and can be used to prevent accidentally upgrading a canister that is not fully stopped. + +### Canister version {#system-api-canister-version} + +For each canister, the system maintains a _canister version_. Upon canister creation, it is set to 0, and it is *guaranteed* to be incremented upon every change of the canister's code or settings, i.e., upon every successful management canister call of methods `update_settings`, `install_code`, and `uninstall_code` on that canister and code uninstallation due to that canister running out of cycles. The system can arbitrarily increment the canister version also if the canister's code and settings do not change. + +* `ic0.canister_version : () -> i64` + +returns the current canister version. + +During the canister upgrade process, `canister_pre_upgrade` sees the old counter value, and `canister_post_upgrade` sees the new counter value. + + ### Inter-canister method calls {#system-api-call} When handling an update call (or a callback), a canister can do further calls to another canister. Calls are assembled in a builder-like fashion, starting with `ic0.call_new`, adding more attributes using the `ic0.call_*` functions, and eventually performing the call with `ic0.call_perform`. @@ -1476,6 +1505,19 @@ The times observed by different canisters are unrelated, and calls from one cani While an implementation will likely try to keep the time returned by `ic0.time` close to the real time, this is not formally part of this specification. ::: +### Global timer {#global-timer} + + +The canister can set a global timer to make the system schedule a call to the exported `canister_global_timer` Wasm method after the specified time. The time must be provided as nanoseconds since 1970-01-01. + +`+ic0.global_timer_set : (timestamp : i64) -> i64+` + +The function returns the previous value of the timer. If no timer is set before invoking the function, then the function returns zero. + +Passing zero as an argument to the function deactivates the timer and thus prevents the system from scheduling calls to the canister's `canister_global_timer` Wasm method. + + + ### Performance counter {#system-api-performance-counter} The canister can query the \"performance counter\", which is a deterministic monotonically increasing integer approximating the amount of work the canister has done since the beginning of the current execution. @@ -1745,13 +1787,14 @@ It is important to note the following for the usage of the `POST` method: - The For security reasons, only HTTPS connections are allowed (URLs must start with `https://`). The IC uses industry-standard root CA lists to validate certificates of remote web servers. -Each request can specify the maximal expected size for the response from the remote HTTP server. The upper limit on the size of a response defaults to `2MiB` if no maximal size value is specified. An error will be returned when the response is larger than the maximal size. The `2MiB` size limit also applies to the value returned by the `transform` function. +The *size* of an HTTP request from the canister or an HTTP response from the remote HTTP server is the total number of bytes representing the names and values of HTTP headers and the HTTP body. The maximal size for the request from the canister is `2MB` (`2,000,000B`). Each request can specify a maximal size for the response from the remote HTTP server. The upper limit on the maximal size for the response is `2MB` (`2,000,000B`) and this value also applies if no maximal size value is specified. +An error will be returned when the request or response is larger than the maximal size. The following parameters should be supplied for the call: -- `url` - the requested URL. The URL may specify a custom port number. +- `url` - the requested URL. The URL must be valid according to https://www.ietf.org/rfc/rfc3986.txt[RFC-3986] and its length must not exceed `8192`. The URL may specify a custom port number. -- `max_response_bytes` - optional, specifies the maximal size of the response in bytes. Any value less than or equal to `2MiB` is accepted. The call will be charged based on this parameter. If not provided, the maximum of `2MiB` will be used. +- `max_response_bytes` - optional, specifies the maximal size of the response in bytes. If provided, the value must not exceed `2MB` (`2,000,000B`). The call will be charged based on this parameter. If not provided, the maximum of `2MB` will be used. - `method` - currently, only GET, HEAD, and POST are supported @@ -1761,6 +1804,8 @@ The following parameters should be supplied for the call: - `transform` - an optional function that transforms raw responses to sanitized responses, and a byte-encoded context that is provided to the function upon invocation, along with the response to be sanitized. If provided, the calling canister itself must export this function. +Cycles to pay for the call must be explicitly transferred with the call, i.e., they are not deducted from the caller's balance implicitly (e.g., as for inter-canister calls). + The returned response (and the response provided to the `transform` function, if specified) contains the following fields: - `status` - the response status (e.g., 200, 404) @@ -1769,7 +1814,19 @@ The returned response (and the response provided to the `transform` function, if - `body` - the response's body -The `transform` function may, for example, transform the body in any way, add or remove headers, modify headers, etc. When the transform function was invoked due to a canister HTTP request, the caller's identity is the principal of the management canister. +The `transform` function may, for example, transform the body in any way, add or remove headers, modify headers, etc. The maximal number of bytes representing the response produced by the `transform` function is `2MB` (`2,000,000B`). Note that the number of bytes representing the response produced by the `transform` function includes the serialization overhead of the encoding produced by the canister. + +When the transform function is invoked by the system due to a canister HTTP request, the caller's identity is the principal of the management canister. This information can be used by developers to implement access control mechanism for this function. + +The following additional limits apply to HTTP requests and HTTP responses from the remote sever: +- the number of headers must not exceed `64`, +- the number of bytes representing a header name or value must not exceed `8KiB`, and +- the total number of bytes representing the header names and values must not exceed `48KiB`. + +:::note +Currently, the Internet Computer mainnet only supports URLs that resolve to IPv6 destinations (i.e., the domain has a `AAAA` DNS record) in HTTP requests. +::: + :::note The identity of the caller for a valid invocation of the `transform` function is `aaaaa-aa`. This information can be used by developers to implement access control mechanism for this function. @@ -1799,7 +1856,7 @@ This method is only available in local development instances. The IC Bitcoin API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. ::: -The Bitcoin functionality is exposed via the management canister. Information about Bitcoin can be found in the [Bitcoin developer guides](https://developer.bitcoin.org/devguide/). Invoking the functions of the Bitcoin API will cost cycles. The concrete costs for each function are yet to be determined. +The Bitcoin functionality is exposed via the management canister. Information about Bitcoin can be found in the [Bitcoin developer guides](https://developer.bitcoin.org/devguide/). Invoking the functions of the Bitcoin API will cost cycles. We refer the reader to the [Bitcoin documentation](https://internetcomputer.org/docs/current/developer-docs/integrations/bitcoin/bitcoin-how-it-works) for further relevant information and the [IC pricing page](https://internetcomputer.org/docs/current/developer-docs/deploy/computation-and-storage-costs) for information on pricing for the Bitcoin mainnet and testnet. ### IC method `bitcoin_get_utxos` {#ic-bitcoin_get_utxos} @@ -1825,7 +1882,7 @@ There is an upper bound of 144 on the minimum number of confirmations. If a larg It is important to note that the validity of transactions is not verified in the Bitcoin component. The Bitcoin component relies on the proof of work that goes into the blocks and the verification of the blocks in the Bitcoin network. For a newly discovered block, a regular Bitcoin (full) node therefore provides a higher level of security than the Bitcoin component, which implies that it is advisable to set the number of confirmations to a reasonably large value, such as 6, to gain confidence in the correctness of the returned UTXOs. -There is an upper bound of 100,000 UTXOs that can be returned in a single request. For addresses that contain sufficiently large UTXOs, the partial set of the address\' UTXOs are returned along with a page reference. +There is an upper bound of 10,000 UTXOs that can be returned in a single request. For addresses that contain sufficiently many UTXOs, a partial set of the address's UTXOs are returned along with a page reference. In the second case, a page reference (a series of bytes) must be provided, which instructs the Bitcoin component to collect UTXOs starting from the corresponding \"page\". @@ -1857,13 +1914,16 @@ If at least one of these checks fails, the call is rejected. If the transaction passes these tests, the transaction is forwarded to the specified Bitcoin network. -The Bitcoin component periodically forwards the transaction until the transaction appears in a block appended to the blockchain or the transaction times out after 24 hours. As soon as it appears in a block or expires, the Bitcoin component drops the transaction. It follows that the function does not provide any guarantees that a submitted transaction will ever appear in a block. +Note that the function does not provide any guarantees that a submitted that a submitted transaction will ever appear in a block. ### IC method `bitcoin_get_current_fee_percentiles` {#ic-bitcoin_get_current_fee_percentiles} The transaction fees in the Bitcoin network change dynamically based on the number of pending transactions. It must be possible for a canister to determine an adequate fee when creating a Bitcoin transaction. -This function returns the 100 fee percentiles, measured in millisatoshi/vbyte (1000 millisatoshi = 1 satoshi), over the last 10,000 transactions in the specified network, i.e., over the transactions in the last approximately 4-10 blocks. +This function returns fee percentiles, measured in millisatoshi/vbyte (1000 millisatoshi = 1 satoshi), over the last 10,000 transactions in the specified network, i.e., over the transactions in the last approximately 4-10 blocks. + +The https://en.wikipedia.org/wiki/Percentile#The_nearest-rank_method[standard nearest-rank estimation method], inclusive, with the addition of a 0th percentile is used. +Concretely, for any i from 1 to 100, the ith percentile is the fee with rank `⌈i * 100⌉`. The 0th percentile is defined as the smallest fee (excluding coinbase transactions). ## Certification @@ -2323,16 +2383,19 @@ The [WebAssembly System API](#system-api) is relatively low-level, and some of i CallerId = Principal; Timestamp = Nat; + CanisterVersion = Nat; Env = { - time : Timestamp + time : Timestamp; + global_timer : Nat; balance : Nat; freezing_limit : Nat; - certificate : NoCertificate | Blob - status : Running | Stopping | Stopped + certificate : NoCertificate | Blob; + status : Running | Stopping | Stopped; + canister_version : CanisterVersion; } - RejectCode = Nat - Response = Reply Blob | Reject (RejectCode, Text) + RejectCode = Nat; + Response = Reply Blob | Reject (RejectCode, Text); MethodCall = { callee : CanisterId; method_name: MethodName; @@ -2344,7 +2407,8 @@ The [WebAssembly System API](#system-api) is relatively low-level, and some of i UpdateFunc = WasmState -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; new_calls : List MethodCall; - new_certified_data : NoCertifiedData | Blob + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; response : NoResponse | Response; cycles_accepted : Nat; cycles_used : Nat; @@ -2353,8 +2417,11 @@ The [WebAssembly System API](#system-api) is relatively low-level, and some of i response : Response; cycles_used : Nat; } - HeartbeatFunc = WasmState -> Trap { cycles_used : Nat; } | Return { + SystemTaskFunc = WasmState -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; + new_calls : List MethodCall; + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; cycles_used : Nat; } @@ -2364,19 +2431,25 @@ The [WebAssembly System API](#system-api) is relatively low-level, and some of i CanisterModule = { init : (CanisterId, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; cycles_used : Nat; } pre_upgrade : (WasmState, Principal, Env) -> Trap { cycles_used : Nat; } | Return { stable_memory : StableMemory; + new_certified_data : NoCertifiedData | Blob; cycles_used : Nat; } post_upgrade : (CanisterId, StableMemory, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; cycles_used : Nat; } update_methods : MethodName ↦ ((Arg, CallerId, Env, AvailableCycles) -> UpdateFunc) query_methods : MethodName ↦ ((Arg, CallerId, Env) -> QueryFunc) - heartbeat : (Env) -> HeartbeatFunc + heartbeat : (Env) -> SystemTaskFunc + global_timer : (Env) -> SystemTaskFunc callbacks : (Callback, Response, RefundedCycles, Env, AvailableCycles) -> UpdateFunc inspect_message : (MethodName, WasmState, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { status : Accept | Reject; @@ -2421,7 +2494,7 @@ To ensure that only one response is generated, and also to detect when no respon calling_context : CallId; callback: Callback } - | FromHeartbeat + | FromSystem CallCtxt = { canister : CanisterId; origin : CallOrigin; @@ -2441,6 +2514,7 @@ Therefore, a message can have different shapes: = PublicMethod MethodName Principal Blob | Callback Callback Response RefundedCycles | Heartbeat + | GlobalTimer Message = CallMessage { @@ -2547,12 +2621,14 @@ Finally, we can describe the state of the IC as a record having the following fi | Stopping (List (CallOrigin, Nat)) | Stopped S = { - requests : Request ↦ RequestStatus; + requests : Request ↦ (RequestStatus, Principal); canisters : CanisterId ↦ CanState; controllers : CanisterId ↦ Set Principal; freezing_threshold : CanisterId ↦ Nat; canister_status: CanisterId ↦ CanStatus; + canister_version: CanisterId ↦ CanisterVersion; time : CanisterId ↦ Timestamp; + global_timer : CanisterId ↦ Timestamp; balances: CanisterId ↦ Nat; certified_data: CanisterId ↦ Blob; system_time : Timestamp @@ -2577,7 +2653,9 @@ The initial state of the IC is controllers = (); freezing_threshold = (); canister_status = (); + canister_version = (); time = (); + global_timer = (); balances = (); certified_data = (); system_time = T; @@ -2699,10 +2777,12 @@ is_effective_canister_id(E.content, ECID) S.canisters[E.content.canister_id] ≠ EmptyCanister Env = { time = S.time[E.content.canister_id]; + global_timer = S.global_timer[E.content.canister_id]; balance = S.balances[E.content.canister_id] freezing_limit = freezing_limit(S, E.content.canister_id); certificate = NoCertificate; status = simple_status(S.canister_status[E.content.canister_id]); + canister_version = S.canister_version[E.content.canister_id]; } S.canisters[E.content.canister_id].module.inspect_message (E.content.method_name, S.canisters[E.content.canister_id].wasm_state, E.content.arg, E.content.sender, Env) = Return {status = Accept; cycles_used = Cycles_used;} @@ -2714,7 +2794,7 @@ State after ```html S with - requests[E.content] = Received + requests[E.content] = (Received, ECID) if E.content.canister_id ≠ ic_principal then balances[E.content.canister_id] = S.balances[E.content.canister_id] - Cycles_used ``` @@ -2730,7 +2810,7 @@ The IC may reject a received message for internal reasons (high load, low resour Conditions: ```html -S.requests[R] = Received +S.requests[R] = (Received, ECID) Code = SYS_FATAL or Code = SYS_TRANSIENT ``` @@ -2739,7 +2819,7 @@ State after ```html S with - requests[R] = Rejected (Code, Msg) + requests[R] = (Rejected (Code, Msg), ECID) ``` @@ -2753,7 +2833,7 @@ The IC does not make any guarantees about the order of incoming messages. Conditions ```html -S.requests[R] = Received +S.requests[R] = (Received, ECID) S.system_time <= R.ingress_expiry C = S.canisters[R.canister_id] ``` @@ -2762,7 +2842,7 @@ C = S.canisters[R.canister_id] State after ```html S with - requests[R] = Processing + requests[R] = (Processing, ECID) messages = CallMessage { origin = FromUser { request = R }; @@ -2801,7 +2881,7 @@ messages = Older_messages · Younger_messages · #### Call context creation {#call_context_creation} -Before invoking a heartbeat or a message to a public entry point, a call context is created for bookkeeping purposes. For these invocations the canister must be running (so not stopped, or stopping). Additionally, these invocations only happen for \"real\" canisters, not the IC management canister. +Before invoking a heartbeat, a global timer, or a message to a public entry point, a call context is created for bookkeeping purposes. For these invocations the canister must be running (so not stopped, or stopping). Additionally, these invocations only happen for \"real\" canisters, not the IC management canister. This "bookkeeping transition" must be immediately followed by the corresponding ["Message execution" transition](#rule-message-execution). @@ -2812,18 +2892,18 @@ This "bookkeeping transition" must be immediately followed by the corresponding The position of the message in the queue is unchanged. Conditions - ``` + ```html S.messages = Older_messages · CallMessage CM · Younger_messages S.canisters[CM.callee] ≠ EmptyCanister S.canister_status[CM.callee] = Running S.balances[CM.callee] ≥ freezing_limit(S, CM.callee) + MAX_CYCLES_PER_MESSAGE Ctxt_id ∉ dom(S.call_contexts) - ``` + ``` State after - - ``` + + ``` html S with messages = Older_messages · @@ -2842,7 +2922,7 @@ This "bookkeeping transition" must be immediately followed by the corresponding available_cycles = CM.transferred_cycles; } balances[CM.callee] = S.balances[CM.callee] - MAX_CYCLES_PER_MESSAGE - ``` + ``` - Call context creation: Heartbeat @@ -2850,16 +2930,16 @@ This "bookkeeping transition" must be immediately followed by the corresponding Conditions - ``` +```html S.canisters[C] ≠ EmptyCanister S.canister_status[C] = Running - balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE + S.balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE Ctxt_id ∉ dom(S.call_contexts) - ``` +``` State after - ``` +```html S with messages = FuncMessage { @@ -2871,13 +2951,52 @@ This "bookkeeping transition" must be immediately followed by the corresponding · S.messages call_contexts[Ctxt_id] = { canister = C; - origin = FromHeartbeat; + origin = FromSystem; needs_to_respond = false; deleted = false; available_cycles = 0; } balances[C] = S.balances[C] - MAX_CYCLES_PER_MESSAGE - ``` +``` + + +- Call context creation: Global timer +If canister `C` exports a method with name `canister_global_timer`, the global timer of canister `C` is set, and the current time for canister `C` has passed the value of the global timer, the IC will create the corresponding call context and deactivate the global timer. + +Conditions:: + +```html + S.canisters[C] ≠ EmptyCanister + S.canister_status[C] = Running + S.global_timer[C] ≠ 0 + S.time[C] ≥ S.global_timer[C] + S.balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE + Ctxt_id ∉ dom(S.call_contexts) +``` + +State after:: + + ``` html +S with + messages = + FuncMessage { + call_context = Ctxt_id; + receiver = C; + entry_point = GlobalTimer; + queue = Queue { from = System; to = C }; + } + · S.messages + call_contexts[Ctxt_id] = { + canister = C; + origin = FromSystem; + needs_to_respond = false; + deleted = false; + available_cycles = 0; + } + global_timer[C] = 0 + balances[C] = S.balances[C] - MAX_CYCLES_PER_MESSAGE +``` + The IC can execute any message that is at the head of its queue, i.e. there is no older message with the same abstract `queue` field. The actual message execution, if successful, may enqueue further messages and --- if the function returns a response --- record this response. The new call and response messages are enqueued at the end. @@ -2898,10 +3017,12 @@ Is_response = M.entry_point == Callback _ _ _ Env = { time = S.time[M.receiver]; + global_timer = S.global_timer[M.receiver]; balance = S.balances[M.receiver] freezing_limit = freezing_limit(S, M.receiver); certificate = NoCertificate; status = simple_status(S.canister_status[M.receiver]); + canister_version = S.canister_version[M.receiver]; } Available = S.call_contexts[M.call_contexts].available_cycles @@ -2914,9 +3035,12 @@ or ) or ( M.entry_point = Heartbeat - F = heartbeat_as_update(Mod.heartbeat, Env) + F = system_task_as_update(Mod.heartbeat, Env) +) +or +( M.entry_point = GlobalTimer + F = system_task_as_update(Mod.global_timer, Env) ) - R = F(S.canisters[M.receiver].wasm_state) ``` @@ -2969,6 +3093,9 @@ then if res.new_certified_data ≠ NoCertifiedData: certified_data[M.receiver] = res.new_certified_data + if res.new_global_timer ≠ NoGlobalTimer: + global_timer[M.receiver] = res.new_global_timer + balances[M.receiver] = New_balance else S with @@ -2998,7 +3125,7 @@ If message execution [*returns* (in the sense of a Wasm function)](#define-wasm- Note that returning does *not* imply that the call associated with this message now *succeeds* in the sense defined in [section responding](#responding); that would require a (unique) call to `ic0.reply`. Note also that the state changes are persisted even when the IC is set to synthesize a [CANISTER_ERROR](#CANISTER_ERROR) reject immediately afterward (which happens when this returns without calling `ic0.reply` or `ic0.reject`, the corresponding call has not been responded to and there are no outstanding callbacks, see [Call context starvation](#rule-starvation)). -The functions `query_as_update` and `heartbeat_as_update` turns a query function resp the heartbeat into an update function; this is merely a notational trick to simplify the rule: +The functions `query_as_update` and `system_task_as_update` turns a query function resp the heartbeat or global timer into an update function; this is merely a notational trick to simplify the rule: query_as_update(f, arg, env) = λ wasm_state → match f(arg, env)(wasm_state) with @@ -3007,18 +3134,20 @@ The functions `query_as_update` and `heartbeat_as_update` turns a query function new_state = wasm_state; new_calls = []; new_certified_data = NoCertifiedData; + new_global_timer = NoGlobalTimer; response = res.response; cycles_accepted = 0; cycles_used = res.cycles_used; } - heartbeat_as_update(f, env) = λ wasm_state → + system_task_as_update(f, env) = λ wasm_state → match f(env)(wasm_state) with Trap trap → Trap trap Return res → Return { new_state = res.new_state; - new_calls = []; - new_certified_data = NoCertifiedData; + new_calls = res.new_calls; + new_certified_data = res.new_certified_data; + new_global_timer = res.new_global_timer; response = NoResponse; cycles_accepted = 0; cycles_used = res.cycles_used; @@ -3028,12 +3157,12 @@ Note that by construction, a query function will either trap or return with a re #### Call context starvation {#rule-starvation} -If the call context is not for heartbeat and there is no call, downstream calling context or response that could possibly fulfill a calling context, then a reject is synthesized. The error message below is *not* indicative. In particular, if the IC has an idea about *why* this starved, it can put that in there (e.g. the initial message handler trapped with an out-of-memory access). +If the call context is not for heartbeat or global timer and there is no call, downstream calling context or response that could possibly fulfill a calling context, then a reject is synthesized. The error message below is _not_ indicative. In particular, if the IC has an idea about _why_ this starved, it can put that in there (e.g. the initial message handler trapped with an out-of-memory access). Conditions ```html S.call_contexts[Ctxt_id].needs_to_respond = true - S.call_contexts[Ctxt_id].origin ≠ FromHeartbeat + S.call_contexts[Ctxt_id].origin ≠ FromSystem ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id ∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id ∀ _ ↦ {needs_to_respond = true, origin = FromCanister O, …} ∈ S.call_contexts: O.calling_context ≠ Ctxt_id @@ -3058,7 +3187,7 @@ S with #### Call context removal {#call_context_removal} -If there is no call, downstream calling context, or response that references a call context, and the call context has been replied to or the call context corresponds to a heartbeat that had already been executed, then the call context can be removed. +If there is no call, downstream calling context, or response that references a call context, and the call context has been replied to or the call context corresponds to a heartbeat or global timer that had already been executed, then the call context can be removed. Conditions @@ -3067,7 +3196,7 @@ Conditions S.call_contexts[Ctxt_id].needs_to_respond = false ) or ( - S.call_contexts[Ctxt_id].origin = FromHeartbeat + S.call_contexts[Ctxt_id].origin = FromHSystem ∀ FuncMessage M ∈ S.messages. M.call_context ≠ Ctxt_id ) ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id @@ -3112,6 +3241,7 @@ State after S with canisters[CanisterId] = EmptyCanister time[CanisterId] = CurrentTime + global_timer[CanisterId] = 0 if A.settings.controllers is not null: controllers[CanisterId] = A.settings.controllers else: @@ -3129,6 +3259,7 @@ S with refunded_cycles = 0 } canister_status[CanisterId] = Running + canister_version[CanisterId] = 0 ``` @@ -3174,6 +3305,7 @@ State after controllers[A.canister_id] = A.settings.controllers if A.settings.freezing_threshold is not null: freezing_threshold[A.canister_id] = A.settings.freezing_threshold + canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin @@ -3246,12 +3378,14 @@ Conditions M.caller ∈ S.controllers[A.canister_id] Env = { time = S.time[A.canister_id]; + global_timer = S.global_timer[A.canister_id]; balance = S.balances[A.canister_id]; freezing_limit = freezing_limit(S, A.canister_id); certificate = NoCertificate; status = simple_status(S.canister_status[A.canister_id]); + canister_version = S.canister_version[A.canister_id] } - Mod.init(A.canister_id, A.arg, M.caller, Env) = Return {new_state = New_state; cycles_used = Cycles_used;} + Mod.init(A.canister_id, A.arg, M.caller, Env) = Return {new_state = New_state; new_certified_data = New_certified_data; new_global_timer = New_global_timer; cycles_used = Cycles_used;} Cycles_used ≤ S.balances[A.canister_id] dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ ``` @@ -3268,6 +3402,13 @@ State after public_custom_sections = Public_custom_sections; private_custom_sections = Private_custom_sections; } + if New_certified_data ≠ NoCertifiedData: + certified_data[A.canister_id] = New_certified_data + if New_global_timer ≠ NoGlobalTimer: + global_timer[A.canister_id] = New_global_timer + else: + global_timer[A.canister_id] = 0 + canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 balances[A.canister_id] = S.balances[A.canister_id] - Cycles_used messages = Older_messages · Younger_messages · ResponseMessage { @@ -3303,8 +3444,16 @@ Conditions certificate = NoCertificate; status = simple_status(S.canister_status[A.canister_id]); } - Old_module.pre_upgrade(Old_State, M.caller, Env) = Return {stable_memory = Stable_memory; cycles_used = Cycles_used;} - Mod.post_upgrade(A.canister_id, Stable_memory, A.arg, M.caller, Env) = Return {new_state = New_state; cycles_used = Cycles_used';} + Env1 = Env with { + global_timer = S.global_timer[A.canister_id]; + canister_version = S.canister_version[A.canister_id]; + } + Old_module.pre_upgrade(Old_State, M.caller, Env1) = Return {stable_memory = Stable_memory; new_certified_data = New_certified_data; cycles_used = Cycles_used;} + Env2 = Env with { + global_timer = 0; + canister_version = S.canister_version[A.canister_id] + 1; + } + Mod.post_upgrade(A.canister_id, Stable_memory, A.arg, M.caller, Env2) = Return {new_state = New_state; new_certified_data = New_certified_data'; new_global_timer = New_global_timer; cycles_used = Cycles_used';} Cycles_used + Cycles_used' ≤ S.balances[A.canister_id] dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ ``` @@ -3321,6 +3470,15 @@ State after public_custom_sections = Public_custom_sections; private_custom_sections = Private_custom_sections; } + if New_certified_data' ≠ NoCertifiedData: + certified_data[A.canister_id] = New_certified_data' + else if New_certified_data ≠ NoCertifiedData: + certified_data[A.canister_id] = New_certified_data + if New_global_timer ≠ NoGlobalTimer: + global_timer[A.canister_id] = New_global_timer + else: + global_timer[A.canister_id] = 0 + canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 balances[A.canister_id] = S.balances[A.canister_id] - (Cycles_used + Cycles_used'); messages = Older_messages · Younger_messages · ResponseMessage { @@ -3353,6 +3511,8 @@ State after S with canisters[A.canister_id] = EmptyCanister certified_data[A.canister_id] = "" + canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 + global_timer[A.canister_id] = 0 messages = Older_messages · Younger_messages · ResponseMessage { @@ -3578,7 +3738,9 @@ State after controllers[A.canister_id] = (deleted) freezing_threshold[A.canister_id] = (deleted) canister_status[A.canister_id] = (deleted) + canister_version[A.canister_id] = (deleted) time[A.canister_id] = (deleted) + global_timer[A.canister_id] = (deleted) balances[A.canister_id] = (deleted) certified_data[A.canister_id] = (deleted) messages = Older_messages · Younger_messages · @@ -3671,6 +3833,7 @@ State after ```html S with canisters[CanisterId] = EmptyCanister + global_timer[CanisterId] = 0 time[CanisterId] = CurrentTime if A.settings.controllers is not null: controllers[CanisterId] = A.settings.controllers @@ -3692,6 +3855,7 @@ S with transferred_cycles = M.transferred_cycles } canister_status[CanisterId] = Running + canister_version[CanisterId] = 0 ``` @@ -3787,7 +3951,7 @@ Conditions S.messages = Older_messages · ResponseMessage RM · Younger_messages RM.origin = FromUser { request = M } -S.requests[M] = Processing +S.requests[M] = (Processing, ECID) ``` @@ -3797,8 +3961,8 @@ State after S with messages = Older_messages · Younger_messages requests[M] = - | Replied R if M.response = Reply R - | Rejected (c, R) if M.response = Reject (c, R) + | (Replied R, ECID) if M.response = Reply R + | (Rejected (c, R), ECID) if M.response = Reject (c, R) ``` NB: The refunded cycles, `RM.refunded_cycles` are, by construction, empty. @@ -3810,7 +3974,7 @@ The IC will keep the data for a completed or rejected request around for a certa Conditions ```html -(S.requests[M] = Replied _) or (S.requests[M] = Rejected _) +(S.requests[M] = (Replied _, ECID)) or (S.requests[M] = (Rejected _, ECID)) ``` @@ -3818,7 +3982,7 @@ State after ```html S with - requests[M] = Done + requests[M] = (Done, ECID) ``` @@ -3827,7 +3991,7 @@ At the same or some later point, the request will be removed from the state of t Conditions ```html -(S.requests[M] = Replied _) or (S.requests[M] = Rejected _) or (S.requests[M] = Done) +(S.requests[M] = (Replied _, _)) or (S.requests[M] = (Rejected _, _)) or (S.requests[M] = (Done, _)) M.ingress_expiry < S.system_time ``` @@ -3856,7 +4020,9 @@ State after ```html S with canisters[CanisterId] = EmptyCanister - certified_data[Canister_id] = "" + certified_data[CanisterId] = "" + canister_version[CanisterId] = S.canister_version[CanisterId] + 1 + global_timer[CanisterId] = 0 messages = S.messages · [ ResponseMessage { @@ -3877,7 +4043,7 @@ S with ``` -#### Time progressing and cycle consumption {#time_progressing_and_cycle_consumption} +#### Time progressing, cycle consumption, and canister version increments {#time_progressing_and_cycle_consumption} Time progresses. Abstractly, it does so independently for each canister, and in unspecified intervals. @@ -3932,6 +4098,23 @@ S with system_time = T1 ``` +Finally, the canister version can be incremented arbitrarily: + +Conditions + +```html + N0 = S.canister_version[CanisterId] + N1 > N0 +``` + State after + +```html + + S with + canister_version[CanisterId] = N1 + +``` + #### Query call {#query_call} @@ -3959,6 +4142,7 @@ lookup(["canister",Q.canister_id,"certified_data"], Cert) = Found S.certified_da lookup(["time"], Cert) = Found S.system_time // or “recent enough” Env = { time = S.time[Q.receiver]; + global_timer = S.global_timer[Q.receiver]; balance = S.balances[Q.canister_id]; freezing_limit = freezing_limit(S, Q.canister_id); certificate = Cert; @@ -4111,7 +4295,9 @@ We can model the execution of WebAssembly functions as stateful functions that h pending_call : MethodCall | NoPendingCall; calls : List MethodCall; new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; ingress_filter : Accept | Reject; + context : I | G | U | Q | Ry | Rt | C | F | T; } This allows us to model WebAssembly functions, including host-provided imports, as functions with implicit mutable access to an `ExecutionState`, dubbed *execution functions*. Syntactically, we express this using an implicit argument of type `ref ExecutionState` in angle brackets (e.g. `func(x)` for the invocation of a WebAssembly function with type `(x : i32) -> ()`). The lifetime of the `ExecutionState` data structure is that of one such function invocation. @@ -4122,7 +4308,7 @@ It is nonsensical to pass to an execution function a WebAssembly store `S` that #### The concrete `CanisterModule` {#the_concrete_canistermodule} -Finally we can specify the abstract `CanisterModule` that models a concrete WebAssembly module. +Finally, we can specify the abstract `CanisterModule` that models a concrete WebAssembly module. - The `initial_wasm_store` mentioned below is the store of the WebAssembly module after *instantiation* (as per WebAssembly spec) of the WasmModule contained in the [canister module](#canister-module-format), including executing a potential `(start)` function. @@ -4149,7 +4335,9 @@ Finally we can specify the abstract `CanisterModule` that models a concrete WebA pending_call = NoPendingCall; calls = []; new_certified_data = NoCertifiedData; + new_global_timer = NoGlobalTimer; ingress_filter = Reject; + context = (undefined); } - The `init` field of the `CanisterModule` is defined as follows: @@ -4157,50 +4345,49 @@ Finally we can specify the abstract `CanisterModule` that models a concrete WebA If the WebAssembly module does not export a function called under the name `canister_init`, then the argument blob is ignored and the `initial_wasm_store` is returned: init = λ (self_id, arg, caller, sysenv) → - Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" }; cycles_used = 0;} + Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" }; new_certified_data = NoCertifiedData; new_global_timer = NoGlobalTimer; cycles_used = 0;} + - - Otherwise, if the WebAssembly module exports a function `func` under the name `canister_init`, it is + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_init`, it is - init = λ (self_id, arg, caller, sysenv) → - let es = ref {empty_execution_state with - wasm_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" } - params = empty_params with { arg = arg; caller = caller; sysenv } - balance = sysenv.balance - } - try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} - if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} - Return {new_state = es.wasm_state; cycles_used = es.cycles_used;} - - This formulation checks afterwards that the system calls `call.perform` or `msg.reply` were not invoked; an implementation can of course trap as soon as these system calls are invoked. + init = λ (self_id, arg, caller, sysenv) → + let es = ref {empty_execution_state with + wasm_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" } + params = empty_params with { arg = arg; caller = caller; sysenv } + balance = sysenv.balance + context = I + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + Return {new_state = es.wasm_state; new_certified_data = es.new_certified_data; new_global_timer = new_global_timer; cycles_used = es.cycles_used; + } + + This formulation checks afterwards that the system calls `call.perform` or `msg.reply` were not invoked; an implementation can of course trap as soon as these system calls are invoked. -- The `pre_upgrade` field of the `CanisterModule` is defined as follows: + - The `pre_upgrade` field of the `CanisterModule` is defined as follows: - If the WebAssembly module does not export a function called under the name `canister_pre_upgrade`, then it simply returns the stable memory: + If the WebAssembly module does not export a function called under the name `canister_pre_upgrade`, then it simply returns the stable memory: - pre_upgrade = λ (old_state, caller, sysenv) → Return {stable_memory = old_state.stable_mem; cycles_used = 0;} + pre_upgrade = λ (old_state, caller, sysenv) → Return {stable_memory = old_state.stable_mem; new_certified_data = NoCertifiedData; cycles_used = 0;} - Otherwise, if the WebAssembly module exports a function `func` under the name `canister_pre_upgrade`, it is + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_pre_upgrade`, it is - pre_upgrade = λ (old_state, caller, sysenv) → - let es = ref {empty_execution_state with - wasm_state = old_state - params = { empty_params with caller = caller; sysenv } - balance = sysenv.balance - } - try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} - if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} - Return {stable_memory = es.wasm_state.stable_mem; cycles_used = es.cycles_used;} + pre_upgrade = λ (old_state, caller, sysenv) → + let es = ref {empty_execution_state with + wasm_state = old_state + params = { empty_params with caller = caller; sysenv } + balance = sysenv.balance + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + Return { new_state = es.wasm_state; new_certified_data = es.new_certified_data; cycles_used = es.cycles_used; + } + - The `post_upgrade` field of the `CanisterModule` is defined as follows: If the WebAssembly module does not export a function called under the name `canister_post_upgrade`, then the argument blob is ignored and the `initial_wasm_store` is returned: post_upgrade = λ (self_id, stable_mem, arg, caller, sysenv) → - Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem }; cycles_used = 0;} + Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem }; new_certified_data = NoCertifiedData; new_global_timer = NoGlobalTimer; cycles_used = 0;} Otherwise, if the WebAssembly module exports a function `func` under the name `canister_post_upgrade`, it is @@ -4209,12 +4396,11 @@ Finally we can specify the abstract `CanisterModule` that models a concrete WebA wasm_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem } params = { empty_params with arg = arg; caller = caller; sysenv } balance = sysenv.balance + contex = I } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} - if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} - Return {new_state = es.wasm_state; cycles_used = es.cycles_used;} + Return {new_state = es.wasm_state; new_certified_data = es.new_certified_data; new_global_timer = es.new_global_timer; cycles_used = es.cycles_used; + } - The partial map `update_methods` of the `CanisterModule` is defined for all method names `method` for which the WebAssembly program exports a function `func` named `canister_update `, and has value @@ -4224,17 +4410,18 @@ Finally we can specify the abstract `CanisterModule` that models a concrete WebA params = empty_params with { arg = arg; caller = caller; sysenv } balance = sysenv.balance cycles_available = available; + context = U + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + new_calls = es.calls; + new_certified_data = es.new_certified_data; + new_global_timer = es.new_global_timer; + response = es.response; + cycles_accepted = es.cycles_accepted; + cycles_used = es.cycles_used; } - try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} - Return { - new_state = es.wasm_state; - new_calls = es.calls; - response = es.response; - cycles_accepted = es.cycles_accepted; - cycles_used = es.cycles_used; - new_certified_data = es.new_certified_data; - } - The partial map `query_methods` of the `CanisterModule` is defined for all method names `method` for which the WebAssembly program exports a function `func` named `canister_query `, and has value @@ -4243,12 +4430,9 @@ Finally we can specify the abstract `CanisterModule` that models a concrete WebA wasm_state = wasm_state; params = empty_params with { arg = arg; caller = caller; sysenv } balance = sysenv.balance + context = Q } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used;} - if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} - if es.response = NoResponse then Trap {cycles_used = es.cycles_used;} Return { response = es.response; cycles_used = es.cycles_used; @@ -4265,22 +4449,47 @@ Finally we can specify the abstract `CanisterModule` that models a concrete WebA wasm_state = wasm_state; params = empty_params with { arg = NoArg; caller = NoCaller; sysenv } balance = sysenv.balance + context = T } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} - if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} Return { new_state = es.wasm_state; + new_calls = es.calls; + new_certified_data = es.certified_data; + new_global_timer = es.new_global_timer; cycles_used = es.cycles_used; } otherwise it is ```html -heartbeat = λ (sysenv) → λ wasm_state → Trap -``` heartbeat = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} +``` + +- The function `global_timer` of the `CanisterModule` is defined if the WebAssembly program exports a function `func` named `canister_global_timer`, and has value + +```html +global_timer = λ (sysenv) → λ wasm_state → + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = empty_params with { arg = NoArg; caller = NoCaller; sysenv } + balance = sysenv.balance + context = T + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + new_calls = es.calls; + new_certified_data = es.certified_data; + new_global_timer = es.new_global_timer; + cycles_used = es.cycles_used; + } +``` + otherwise it is +```html + global_timer = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} + +``` - The function `callbacks` of the `CanisterModule` is defined as follows @@ -4289,18 +4498,19 @@ heartbeat = λ (sysenv) → λ wasm_state → Trap sysenv cycles_refunded = refund_cycles; } - let (fun, env, params) = match response with + let (fun, env, params, context) = match response with Reply data -> (callbacks.on_reply.fun, callbacks.on_reply.env, - { params0 with data}) + { params0 with data}, Ry) Reject (reject_code, reject_message)-> (callbacks.on_reject.fun, callbacks.on_reject.env, - { params0 with reject_code; reject_message}) + { params0 with reject_code; reject_message}, Rt) let es = ref {empty_execution_state with wasm_state = wasm_state; params = params; balance = sysenv.balance; cycles_available = available; + context = context; } try if fun > |es.wasm_state.store.table| then Trap @@ -4311,10 +4521,11 @@ heartbeat = λ (sysenv) → λ wasm_state → Trap Return { new_state = es.wasm_state; new_calls = es.calls; + new_certified_data = es.certified_data; + new_global_timer = es.new_global_timer; response = es.response; cycles_accepted = es.cycles_accepted; cycles_used = es.cycles_used; - new_certified_data = es.certified_data; } with Trap if callbacks.on_cleanup = NoClosure then Trap {cycles_used = es.cycles_used;} @@ -4324,15 +4535,14 @@ heartbeat = λ (sysenv) → λ wasm_state → Trap let es' = ref { empty_execution_state with wasm_state = wasm_state; + context = C } try func(callbacks.on_cleanup.env) with Trap then Trap {cycles_used = es.cycles_used + es'.cycles_used;} - if es'.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used + es'.cycles_used;} - if es'.calls ≠ [] then Trap {cycles_used = es.cycles_used + es'.cycles_used;} - if es'.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used + es'.cycles_used;} - if es'.response ≠ NoResponse then Trap {cycles_used = es.cycles_used + es'.cycles_used;} Return { new_state = es'.wasm_state; new_calls = []; + new_certified_data = NoCertifiedData; + new_global_timer = es'.new_global_timer; response = NoResponse; cycles_accepted = 0; cycles_used = es.cycles_used + es'.cycles_used; @@ -4345,7 +4555,7 @@ heartbeat = λ (sysenv) → λ wasm_state → Trap If the WebAssembly module does not export a function called under the name `canister_inspect_message`, then access is always granted: inspect_message = λ (method_name, wasm_state, arg, caller, sysenv) → - Return {status = Accept; cycles_used = 0;} + Return {status = Accept; cycles_used = 0; context = F; } Otherwise, if the WebAssembly module exports a function `func` under the name `canister_inspect_message`, it is @@ -4362,8 +4572,6 @@ heartbeat = λ (sysenv) → λ wasm_state → Trap cycles_available = 0; // ingress requests have no funds } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} - if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} Return {status = es.ingress_filter; cycles_used = es.cycles_used;}; #### Helper functions {#helper_functions} @@ -4393,67 +4601,85 @@ Upon *instantiation* of the WebAssembly module, we can provide the following fun The pseudo-code below does *not* explicitly enforce the restrictions of which imports are available in which contexts; for that the table in [Overview of imports](#system-api-imports) is authoritative, and is assumed to be part of the implementation. ic0.msg_arg_data_size() : i32 = + if es.context ∉ {I, U, Q, Ry, F} then Trap {cycles_used = es.cycles_used;} return |es.params.arg| ic0.msg_arg_data_copy(dst:i32, offset:i32, size:i32) = + if es.context ∉ {I, U, Q, Ry, F} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.arg) ic0.msg_caller_size() : i32 = + if es.context ∉ {I, G, U, Q, F} then Trap {cycles_used = es.cycles_used;} return |es.params.caller| ic0.msg_caller_copy(dst:i32, offset:i32, size:i32) : i32 = + if es.context ∉ {I, G, U, Q, F} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.caller) ic0.msg_reject_code() : i32 = + if es.context ∉ {Ry, Rt} then Trap {cycles_used = es.cycles_used;} es.params.reject_code ic0.msg_reject_msg_size() : i32 = + if es.context ∉ {Rt} then Trap {cycles_used = es.cycles_used;} return |es.params.reject_msg| ic0.msg_reject_msg_copy(dst:i32, offset:i32, size:i32) : i32 = + if es.context ∉ {Rt} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.reject_msg) ic0.msg_reply_data_append(src : i32, size : i32) = + if es.context ∉ {U, Q, Ry, Rt} then Trap {cycles_used = es.cycles_used;} if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.reply_params.arg := es.reply_params.arg · copy_from_canister(src, size) ic0.msg_reply() = + if es.context ∉ {U, Q, Ry, Rt} then Trap {cycles_used = es.cycles_used;} if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.response := Reply (es.reply_params.arg) es.cycles_available := 0 ic0.msg_reject(src : i32, size : i32) = + if es.context ∉ {U, Q, Rt, Ry} then Trap {cycles_used = es.cycles_used;} if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.response := Reject (CANISTER_REJECT, copy_from_canister(src, size)) es.cycles_available := 0 ic0.msg_cycles_available() : i64 = + if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} if es.cycles_available >= 2^64 then Trap {cycles_used = es.cycles_used;} return es.cycles_available ic0.msg_cycles_available128(dst : i32) = + if es.context ∉ {Rt, Ry} then Trap {cycles_used = es.cycles_used;} let amount = es.cycles_available copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) ic0.msg_cycles_refunded() : i64 = + if es.context ∉ {Rt, Ry} then Trap {cycles_used = es.cycles_used;} if es.params.cycles_refunded >= 2^64 then Trap {cycles_used = es.cycles_used;} return es.params.cycles_refunded ic0.msg_cycles_refunded128(dst : i32) = + if es.context ∉ {Rt, Ry} then Trap {cycles_used = es.cycles_used;} let amount = es.params.cycles_refunded copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) ic0.accept_message() = + if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} if es.ingress_filter = Accept then Trap {cycles_used = es.cycles_used;} es.ingress_filter = Accept ic0.msg_method_name_size() : i32 = + if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} return |es.method_name| ic0.msg_method_name_copy(dst : i32, offset : i32, size : i32) : i32 = + if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.method_name) ic0.msg_cycles_accept(max_amount : i64) : i64 = + if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} let amount = min(max_amount, es.cycles_available) es.cycles_available := es.cycles_available - amount es.cycles_accepted := es.cycles_accepted + amount @@ -4461,6 +4687,7 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im return amount ic0.msg_cycles_accept128(max_amount_high : i64, max_amount_low : i64, dst : i32) = + if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} let max_amount = max_amount_high * 2^64 + max_amount_low let amount = min(max_amount, es.cycles_available) es.cycles_available := es.cycles_available - amount @@ -4488,6 +4715,10 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im Stopping -> return 2 Stopped -> return 3 + ic0.canister_version() : i64 = + return es.params.sysenv.canister_version + + ic0.call_new( callee_src : i32, callee_size : i32, @@ -4498,7 +4729,8 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im reject_fun : i32, reject_env : i32, ) = - discard_pending_call() + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} + discard_pending_call() if es.balance < MAX_CYCLES_PER_RESPONSE then Trap {cycles_used = es.cycles_used;} es.balance := es.balance - MAX_CYCLES_PER_RESPONSE @@ -4525,10 +4757,12 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im } ic0.call_data_append (src : i32, size : i32) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} es.pending_call.arg := es.pending_call.arg · copy_from_canister(src, size) ic0.call_on_cleanup (fun : i32, env : i32) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} if fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} if typeof(es.wasm_state.store.table[fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} @@ -4536,6 +4770,7 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im es.pending_call.callback.on_cleanup := Closure { fun = fun; env = env} ic0.call_cycles_add(amount : i64) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} if es.balance < amount then Trap {cycles_used = es.cycles_used;} @@ -4543,6 +4778,7 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount ic0.call_cycles_add128(amount_high : i64, amount_low : i64) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} let amount = amount_high * 2^64 + amount_low if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} if es.balance < amount then Trap {cycles_used = es.cycles_used;} @@ -4551,6 +4787,7 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount ic0.call_peform() : ( err_code : i32 ) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} // are we below the threezing threshold? @@ -4624,9 +4861,19 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] ic0.time() : i32 = - return es.params.time + return es.params.sysnev.time + + ic0.global_timer_set(timestamp: i64) : i64 = + if es.context ∉ {I, U, Ry, Rt, C, T} then Trap {cycles_used = es.cycles_used;} + let prev_global_timer = es.new_global_timer + es.new_global_timer := timestamp + if prev_global_timer = NoGlobalTimer + then return es.params.sysenv.global_timer + else return prev_global_timer + ic0.certified_data_set(src: i32, size: i32) = + if es.context ∉ {I, G, U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} es.new_certified_data := es.wasm_state[src..src+size] ic0.data_certificate_present() : i32 = diff --git a/spec/md-spec/interface-spec-changelog.md b/spec/md-spec/interface-spec-changelog.md index 25bafe578..200d54e8e 100644 --- a/spec/md-spec/interface-spec-changelog.md +++ b/spec/md-spec/interface-spec-changelog.md @@ -1,5 +1,11 @@ ## Changelog {#changelog} +## 0_18_9 (2022-11-06) {#0_18_9} +* Global timers +* Canister version +* Clarifications for HTTP requests & Bitcoin integration costs ++ + ## 0.18.8 (2022-11-09) {#0_18_8} * Updated HTTP request API * Canister status available to canister From 641920c404eaac07b6d40a1255a692e8a30087a5 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Fri, 9 Dec 2022 10:54:08 +0100 Subject: [PATCH 049/102] changes in adoc/isabelle --- spec/changelog.adoc | 6 + spec/ic0.txt | 16 +- spec/index.adoc | 430 ++++++++++++++++++++++++++++++++++---------- theories/IC.thy | 394 +++++++++++++++++++++++++++------------- 4 files changed, 617 insertions(+), 229 deletions(-) diff --git a/spec/changelog.adoc b/spec/changelog.adoc index c423c3dcd..366abc2f2 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,12 @@ [#changelog] == Changelog +[#0_18_9] +=== 0.18.9 (2022-12-06) +* Global timers +* Canister version +* Clarifications for HTTP requests & Bitcoin integration costs + [#0_18_8] === 0.18.8 (2022-11-09) * Updated HTTP request API diff --git a/spec/ic0.txt b/spec/ic0.txt index a37d2f8ff..454ff815f 100644 --- a/spec/ic0.txt +++ b/spec/ic0.txt @@ -23,12 +23,13 @@ ic0.canister_self_copy : (dst : i32, offset : i32, size : i32) -> (); // * ic0.canister_cycle_balance : () -> i64; // * ic0.canister_cycle_balance128 : (dst : i32) -> (); // * ic0.canister_status : () -> i32; // * +ic0.canister_version : () -> i64; // * ic0.msg_method_name_size : () -> i32; // F ic0.msg_method_name_copy : (dst : i32, offset : i32, size : i32) -> (); // F ic0.accept_message : () -> (); // F -ic0.call_new : // U Ry Rt H +ic0.call_new : // U Ry Rt T ( callee_src : i32, callee_size : i32, name_src : i32, @@ -38,11 +39,11 @@ ic0.call_new : // U reject_fun : i32, reject_env : i32 ) -> (); -ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt H -ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt H -ic0.call_cycles_add : (amount : i64) -> (); // U Ry Rt H -ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt H -ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt H +ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt T +ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt T +ic0.call_cycles_add : (amount : i64) -> (); // U Ry Rt T +ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt T +ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt T ic0.stable_size : () -> (page_count : i32); // * ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32); // * @@ -53,12 +54,13 @@ ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64); // * ic0.stable64_write : (offset : i64, src : i64, size : i64) -> (); // * ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> (); // * -ic0.certified_data_set : (src: i32, size: i32) -> (); // I G U Ry Rt H +ic0.certified_data_set : (src: i32, size: i32) -> (); // I G U Ry Rt T ic0.data_certificate_present : () -> i32; // * ic0.data_certificate_size : () -> i32; // * ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> (); // * ic0.time : () -> (timestamp : i64); // * +ic0.global_timer_set : (timestamp : i64) -> i64; // I U Ry Rt C T ic0.performance_counter : (counter_type : i32) -> (counter : i64); // * s ic0.debug_print : (src : i32, size : i32) -> (); // * s diff --git a/spec/index.adoc b/spec/index.adoc index 4cb1f6820..b120d2266 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,6 +1,6 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.8 +0.18.9 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ @@ -225,14 +225,14 @@ NOTE: Once the IC frees the resources of a canister, its id, _cycles_ balance, a The canister status can be used to control whether the canister is processing calls: * In status `running`, calls to the canister are processed as normal. -* In status `stopping`, calls to the canister are rejected by the IC, but responses to the canister are processed as normal. -* In status `stopped`, calls to the canister are rejected by the IC, and there are no outstanding responses to call contexts that are not marked as deleted. +* In status `stopping`, calls to the canister are rejected by the IC with reject code `CANISTER_ERROR` (5), but responses to the canister are processed as normal. +* In status `stopped`, calls to the canister are rejected by the IC with reject code `CANISTER_ERROR` (5), and there are no outstanding responses to call contexts that are not marked as deleted. In all cases, calls to the <> are processed, regardless of the state of the managed canister. The controllers of the canister can initiate transitions between these states using <> and <>, and query the state using <> (NB: this call returns additional information, such as the cycle balance of the canister). The canister itself can also query its state using <>. -NOTE: This status is orthogonal to whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected, but because the canister is empty. +NOTE: This status is orthogonal to whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected by the IC with an HTTP error, but because the canister is empty. [#signatures] === Signatures @@ -886,6 +886,7 @@ In order for a WebAssembly module to be usable as the code for the canister, it * If it exports a function called `canister_init`, the function must have type `+() -> ()+`. * If it exports a function called `canister_inspect_message`, the function must have type `+() -> ()+`. * If it exports a function called `canister_heartbeat`, the function must have type `+() -> ()+`. +* If it exports a function called `canister_global_timer`, the function must have type `+() -> ()+`. * If it exports any functions called `canister_update ` or `canister_query ` for some `name`, the functions must have type `+() -> ()+`. * It may not export both `canister_update ` and `canister_query ` with the same `name`. * It may not export other methods the names of which start with the prefix `canister_` besides the methods allowed above. @@ -911,6 +912,7 @@ The canister provides entry points which are invoked by the IC under various cir * The canister may export a function with name `canister_post_upgrade` and type `+() -> ()+`. * The canister may export functions with name `canister_inspect_message` with type `+() -> ()+`. * The canister may export a function with name `canister_heartbeat` with type `+() -> ()+`. +* The canister may export a function with name `canister_global_timer` with type `+() -> ()+`. * The canister may export functions with name `canister_update ` and type `+() -> ()+`. * The canister may export functions with name `canister_query ` and type `+() -> ()+`. * The canister table may contain functions of type `+(env : i32) -> ()+` which may be used as callbacks for inter-canister calls. @@ -955,10 +957,20 @@ Eventually, a method will want to send a response, using `ic0.reply` or `ic0.rej For periodic or time-based execution, the WebAssembly module can export a function with name `canister_heartbeat`. The heartbeats scheduling algorithm is implementation-defined. -`canister_heartbeat` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. +`canister_heartbeat` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. Still, the function `canister_heartbeat` can initiate new calls. NOTE: While an implementation will likely try to keep the interval between `canister_heartbeat` invocations to within a few seconds, this is not formally part of this specification. +==== Global timer + +For time-based execution, the WebAssembly module can export a function with name `canister_global_timer`. This function is called if the canister has set its global timer (using the System API function `ic0.global_timer_set`) and the current time (as returned by the System API function `ic0.time`) has passed the value of the global timer. + +Once the function `canister_global_timer` is scheduled, the canister's global timer is deactivated. The global timer is also deactivated upon changes to the canister's Wasm module (calling `install_code`, `uninstall_code` methods of the management canister or if the canister runs out of cycles). In particular, the function `canister_global_timer` won't be scheduled again unless the canister sets the global timer again (using the System API function `ic0.global_timer_set`). The global timer scheduling algorithm is implementation-defined. + +`canister_global_timer` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. Still, the function `canister_global_timer` can initiate new calls. + +NOTE: While an implementation will likely try to keep the interval between the value of the global timer and the time-stamp of the `canister_global_timer` invocation within a few seconds, this is not formally part of this specification. + ==== Callbacks Callbacks are addressed by their table index (as a proxy for a Wasm `funcref`). @@ -987,8 +999,8 @@ The comment after each function lists from where these functions may be invoked: * `C`: from a cleanup callback * `s`: the `(start)` module initialization function * `F`: from `canister_inspect_message` -* `H`: from `canister_heartbeat` -* `*` = `I G U Q Ry Rt C F H` (NB: Not `(start)`) +* `T`: from _system task_ (`canister_heartbeat` or `canister_global_timer`) +* `*` = `I G U Q Ry Rt C F T` (NB: Not `(start)`) If the canister invokes a system call from somewhere else, it will trap. @@ -1113,6 +1125,17 @@ Status `1` indicates running, `2` indicates stopping, and `3` indicates stopped. + Status `3` (stopped) can be observed, for example, in `canister_pre_upgrade` and can be used to prevent accidentally upgrading a canister that is not fully stopped. +[#system-api-canister-version] +=== Canister version + +For each canister, the system maintains a _canister version_. Upon canister creation, it is set to 0, and it is *guaranteed* to be incremented upon every change of the canister's code or settings, i.e., upon every successful management canister call of methods `update_settings`, `install_code`, and `uninstall_code` on that canister and code uninstallation due to that canister running out of cycles. The system can arbitrarily increment the canister version also if the canister's code and settings do not change. + +* `ic0.canister_version : () -> i64` ++ +returns the current canister version. + +During the canister upgrade process, `canister_pre_upgrade` sees the old counter value, and `canister_post_upgrade` sees the new counter value. + [#system-api-call] === Inter-canister method calls @@ -1376,6 +1399,17 @@ The times observed by different canisters are unrelated, and calls from one cani NOTE: While an implementation will likely try to keep the time returned by `ic0.time` close to the real time, this is not formally part of this specification. +[#global-timer] +=== Global timer + +The canister can set a global timer to make the system schedule a call to the exported `canister_global_timer` Wasm method after the specified time. The time must be provided as nanoseconds since 1970-01-01. + +`+ic0.global_timer_set : (timestamp : i64) -> i64+` + +The function returns the previous value of the timer. If no timer is set before invoking the function, then the function returns zero. + +Passing zero as an argument to the function deactivates the timer and thus prevents the system from scheduling calls to the canister's `canister_global_timer` Wasm method. + [#system-api-performance-counter] === Performance counter @@ -1663,29 +1697,36 @@ It is important to note the following for the usage of the `POST` method: For security reasons, only HTTPS connections are allowed (URLs must start with `https://`). The IC uses industry-standard root CA lists to validate certificates of remote web servers. -Each request can specify the maximal expected size for the response from the remote HTTP server. The upper limit on the size of a response defaults to `2MiB` if no maximal size value is specified. -An error will be returned when the response is larger than the maximal size. -The `2MiB` size limit also applies to the value returned by the `transform` function. +The *size* of an HTTP request from the canister or an HTTP response from the remote HTTP server is the total number of bytes representing the names and values of HTTP headers and the HTTP body. The maximal size for the request from the canister is `2MB` (`2,000,000B`). Each request can specify a maximal size for the response from the remote HTTP server. The upper limit on the maximal size for the response is `2MB` (`2,000,000B`) and this value also applies if no maximal size value is specified. +An error will be returned when the request or response is larger than the maximal size. The following parameters should be supplied for the call: -- `url` - the requested URL. The URL may specify a custom port number. -- `max_response_bytes` - optional, specifies the maximal size of the response in bytes. Any value less than or equal to `2MiB` is accepted. The call will be charged based on this parameter. If not provided, the maximum of `2MiB` will be used. +- `url` - the requested URL. The URL must be valid according to https://www.ietf.org/rfc/rfc3986.txt[RFC-3986] and its length must not exceed `8192`. The URL may specify a custom port number. +- `max_response_bytes` - optional, specifies the maximal size of the response in bytes. If provided, the value must not exceed `2MB` (`2,000,000B`). The call will be charged based on this parameter. If not provided, the maximum of `2MB` will be used. - `method` - currently, only GET, HEAD, and POST are supported - `headers` - list of HTTP request headers and their corresponding values - `body` - optional, the content of the request's body - `transform` - an optional record that includes a function that transforms raw responses to sanitized responses, and a byte-encoded context that is provided to the function upon invocation, along with the response to be sanitized. If provided, the calling canister itself must export this function. +Cycles to pay for the call must be explicitly transferred with the call, i.e., they are not deducted from the caller's balance implicitly (e.g., as for inter-canister calls). + The returned response (and the response provided to the `transform` function, if specified) contains the following fields: - `status` - the response status (e.g., 200, 404) - `headers` - list of HTTP response headers and their corresponding values - `body` - the response's body -The `transform` function may, for example, transform the body in any way, add or remove headers, modify headers, etc. -When the transform function was invoked due to a canister HTTP request, the caller's identity is the principal of the management canister. +The `transform` function may, for example, transform the body in any way, add or remove headers, modify headers, etc. The maximal number of bytes representing the response produced by the `transform` function is `2MB` (`2,000,000B`). Note that the number of bytes representing the response produced by the `transform` function includes the serialization overhead of the encoding produced by the canister. + +When the transform function is invoked by the system due to a canister HTTP request, the caller's identity is the principal of the management canister. This information can be used by developers to implement access control mechanism for this function. -NOTE: The identity of the caller for a valid invocation of the `transform` function is `aaaaa-aa`. This information can be used by developers to implement access control mechanism for this function. +The following additional limits apply to HTTP requests and HTTP responses from the remote sever: +- the number of headers must not exceed `64`, +- the number of bytes representing a header name or value must not exceed `8KiB`, and +- the total number of bytes representing the header names and values must not exceed `48KiB`. + +NOTE: Currently, the Internet Computer mainnet only supports URLs that resolve to IPv6 destinations (i.e., the domain has a `AAAA` DNS record) in HTTP requests. [#ic-provisional_create_canister_with_cycles] === IC method `provisional_create_canister_with_cycles` @@ -1715,7 +1756,7 @@ NOTE: The IC Bitcoin API is considered EXPERIMENTAL. Canister developers must be The Bitcoin functionality is exposed via the management canister. Information about Bitcoin can be found in the https://developer.bitcoin.org/devguide/[Bitcoin developer guides]. -Invoking the functions of the Bitcoin API will cost cycles. The concrete costs for each function are yet to be determined. +Invoking the functions of the Bitcoin API will cost cycles. We refer the reader to the [Bitcoin documentation](https://internetcomputer.org/docs/current/developer-docs/integrations/bitcoin/bitcoin-how-it-works) for further relevant information and the [IC pricing page](https://internetcomputer.org/docs/current/developer-docs/deploy/computation-and-storage-costs) for information on pricing for the Bitcoin mainnet and testnet. [#ic-bitcoin_get_utxos] === IC method `bitcoin_get_utxos` @@ -1753,8 +1794,8 @@ For a newly discovered block, a regular Bitcoin (full) node therefore provides a than the Bitcoin component, which implies that it is advisable to set the number of confirmations to a reasonably large value, such as 6, to gain confidence in the correctness of the returned UTXOs. -There is an upper bound of 100,000 UTXOs that can be returned in a single request. For addresses that contain sufficiently large UTXOs, -the partial set of the address' UTXOs are returned along with a page reference. +There is an upper bound of 10,000 UTXOs that can be returned in a single request. For addresses that contain sufficiently many UTXOs, +a partial set of the address's UTXOs are returned along with a page reference. In the second case, a page reference (a series of bytes) must be provided, which instructs the Bitcoin component to collect UTXOs starting from the corresponding "page". @@ -1798,12 +1839,7 @@ If at least one of these checks fails, the call is rejected. If the transaction passes these tests, the transaction is forwarded to the specified Bitcoin network. - -The Bitcoin component periodically forwards the transaction -until the transaction appears in a block appended to the blockchain or the transaction -times out after 24 hours. As soon as it appears in a block or expires, -the Bitcoin component drops the transaction. -It follows that the function does not provide any guarantees that a submitted +Note that the function does not provide any guarantees that a submitted transaction will ever appear in a block. [#ic-bitcoin_get_current_fee_percentiles] @@ -1813,9 +1849,12 @@ The transaction fees in the Bitcoin network change dynamically based on the numb pending transactions. It must be possible for a canister to determine an adequate fee when creating a Bitcoin transaction. -This function returns the 100 fee percentiles, measured in millisatoshi/vbyte (1000 millisatoshi = 1 satoshi), over the last 10,000 transactions in the specified network, i.e., +This function returns fee percentiles, measured in millisatoshi/vbyte (1000 millisatoshi = 1 satoshi), over the last 10,000 transactions in the specified network, i.e., over the transactions in the last approximately 4-10 blocks. +The https://en.wikipedia.org/wiki/Percentile#The_nearest-rank_method[standard nearest-rank estimation method], inclusive, with the addition of a 0th percentile is used. +Concretely, for any i from 1 to 100, the ith percentile is the fee with rank `⌈i * 100⌉`. The 0th percentile is defined as the smallest fee (excluding coinbase transactions). + [#certification] == Certification @@ -2285,12 +2324,15 @@ Arg = Blob; CallerId = Principal; Timestamp = Nat; +CanisterVersion = Nat; Env = { - time : Timestamp + time : Timestamp; + global_timer : Nat; balance : Nat; freezing_limit : Nat; - certificate : NoCertificate | Blob - status : Running | Stopping | Stopped + certificate : NoCertificate | Blob; + status : Running | Stopping | Stopped; + canister_version : CanisterVersion; } RejectCode = Nat @@ -2306,7 +2348,8 @@ MethodCall = { UpdateFunc = WasmState -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; new_calls : List MethodCall; - new_certified_data : NoCertifiedData | Blob + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; response : NoResponse | Response; cycles_accepted : Nat; cycles_used : Nat; @@ -2315,8 +2358,11 @@ QueryFunc = WasmState -> Trap { cycles_used : Nat; } | Return { response : Response; cycles_used : Nat; } -HeartbeatFunc = WasmState -> Trap { cycles_used : Nat; } | Return { +SystemTaskFunc = WasmState -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; + new_calls : List MethodCall; + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; cycles_used : Nat; } @@ -2326,19 +2372,25 @@ RefundedCycles = Nat CanisterModule = { init : (CanisterId, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; cycles_used : Nat; } pre_upgrade : (WasmState, Principal, Env) -> Trap { cycles_used : Nat; } | Return { stable_memory : StableMemory; + new_certified_data : NoCertifiedData | Blob; cycles_used : Nat; } post_upgrade : (CanisterId, StableMemory, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; cycles_used : Nat; } update_methods : MethodName ↦ ((Arg, CallerId, Env, AvailableCycles) -> UpdateFunc) query_methods : MethodName ↦ ((Arg, CallerId, Env) -> QueryFunc) - heartbeat : (Env) -> HeartbeatFunc + heartbeat : (Env) -> SystemTaskFunc + global_timer : (Env) -> SystemTaskFunc callbacks : (Callback, Response, RefundedCycles, Env, AvailableCycles) -> UpdateFunc inspect_message : (MethodName, WasmState, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { status : Accept | Reject; @@ -2387,7 +2439,7 @@ CallOrigin calling_context : CallId; callback: Callback } - | FromHeartbeat + | FromSystem CallCtxt = { canister : CanisterId; origin : CallOrigin; @@ -2408,6 +2460,7 @@ EntryPoint = PublicMethod MethodName Principal Blob | Callback Callback Response RefundedCycles | Heartbeat + | GlobalTimer Message = CallMessage { @@ -2524,12 +2577,14 @@ CanStatus | Stopping (List (CallOrigin, Nat)) | Stopped S = { - requests : Request ↦ RequestStatus; + requests : Request ↦ (RequestStatus, Principal); canisters : CanisterId ↦ CanState; controllers : CanisterId ↦ Set Principal; freezing_threshold : CanisterId ↦ Nat; canister_status: CanisterId ↦ CanStatus; + canister_version: CanisterId ↦ CanisterVersion; time : CanisterId ↦ Timestamp; + global_timer : CanisterId ↦ Timestamp; balances: CanisterId ↦ Nat; certified_data: CanisterId ↦ Blob; system_time : Timestamp @@ -2556,7 +2611,9 @@ The initial state of the IC is controllers = (); freezing_threshold = (); canister_status = (); + canister_version = (); time = (); + global_timer = (); balances = (); certified_data = (); system_time = T; @@ -2690,10 +2747,12 @@ Conditions:: S.canisters[E.content.canister_id] ≠ EmptyCanister Env = { time = S.time[E.content.canister_id]; + global_timer = S.global_timer[E.content.canister_id]; balance = S.balances[E.content.canister_id] freezing_limit = freezing_limit(S, E.content.canister_id); certificate = NoCertificate; status = simple_status(S.canister_status[E.content.canister_id]); + canister_version = S.canister_version[E.content.canister_id]; } S.canisters[E.content.canister_id].module.inspect_message (E.content.method_name, S.canisters[E.content.canister_id].wasm_state, E.content.arg, E.content.sender, Env) = Return {status = Accept; cycles_used = Cycles_used;} @@ -2703,7 +2762,7 @@ Conditions:: State after:: .... S with - requests[E.content] = Received + requests[E.content] = (Received, ECID) if E.content.canister_id ≠ ic_principal then balances[E.content.canister_id] = S.balances[E.content.canister_id] - Cycles_used .... @@ -2716,13 +2775,13 @@ The IC may reject a received message for internal reasons (high load, low resour Conditions:: .... - S.requests[R] = Received + S.requests[R] = (Received, ECID) Code = SYS_FATAL or Code = SYS_TRANSIENT .... State after:: .... S with - requests[R] = Rejected (Code, Msg) + requests[R] = (Rejected (Code, Msg), ECID) .... ==== Initiating canister calls @@ -2735,14 +2794,14 @@ The IC does not make any guarantees about the order of incoming messages. Conditions:: .... - S.requests[R] = Received + S.requests[R] = (Received, ECID) S.system_time <= R.ingress_expiry C = S.canisters[R.canister_id] .... State after:: .... S with - requests[R] = Processing + requests[R] = (Processing, ECID) messages = CallMessage { origin = FromUser { request = R }; @@ -2780,7 +2839,7 @@ State after:: ==== Call context creation -Before invoking a heartbeat or a message to a public entry point, a call context is created for bookkeeping purposes. +Before invoking a heartbeat, a global timer, or a message to a public entry point, a call context is created for bookkeeping purposes. For these invocations the canister must be running (so not stopped, or stopping). Additionally, these invocations only happen for "real" canisters, not the IC management canister. @@ -2834,7 +2893,7 @@ Conditions:: .... S.canisters[C] ≠ EmptyCanister S.canister_status[C] = Running - balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE + S.balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE Ctxt_id ∉ dom(S.call_contexts) .... + @@ -2852,11 +2911,49 @@ S with · S.messages call_contexts[Ctxt_id] = { canister = C; - origin = FromHeartbeat; + origin = FromSystem; + needs_to_respond = false; + deleted = false; + available_cycles = 0; + } + balances[C] = S.balances[C] - MAX_CYCLES_PER_MESSAGE +.... + +* Call context creation: Global timer ++ +If canister `C` exports a method with name `canister_global_timer`, the global timer of canister `C` is set, and the current time for canister `C` has passed the value of the global timer, the IC will create the corresponding call context and deactivate the global timer. ++ +Conditions:: ++ +.... + S.canisters[C] ≠ EmptyCanister + S.canister_status[C] = Running + S.global_timer[C] ≠ 0 + S.time[C] ≥ S.global_timer[C] + S.balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE + Ctxt_id ∉ dom(S.call_contexts) +.... ++ +State after:: ++ +.... +S with + messages = + FuncMessage { + call_context = Ctxt_id; + receiver = C; + entry_point = GlobalTimer; + queue = Queue { from = System; to = C }; + } + · S.messages + call_contexts[Ctxt_id] = { + canister = C; + origin = FromSystem; needs_to_respond = false; deleted = false; available_cycles = 0; } + global_timer[C] = 0 balances[C] = S.balances[C] - MAX_CYCLES_PER_MESSAGE .... @@ -2880,10 +2977,12 @@ Conditions:: Env = { time = S.time[M.receiver]; + global_timer = S.global_timer[M.receiver]; balance = S.balances[M.receiver] freezing_limit = freezing_limit(S, M.receiver); certificate = NoCertificate; status = simple_status(S.canister_status[M.receiver]); + canister_version = S.canister_version[M.receiver]; } Available = S.call_contexts[M.call_contexts].available_cycles @@ -2896,7 +2995,11 @@ Conditions:: ) or ( M.entry_point = Heartbeat - F = heartbeat_as_update(Mod.heartbeat, Env) + F = system_task_as_update(Mod.heartbeat, Env) + ) + or + ( M.entry_point = GlobalTimer + F = system_task_as_update(Mod.global_timer, Env) ) R = F(S.canisters[M.receiver].wasm_state) @@ -2949,6 +3052,9 @@ then if res.new_certified_data ≠ NoCertifiedData: certified_data[M.receiver] = res.new_certified_data + if res.new_global_timer ≠ NoGlobalTimer: + global_timer[M.receiver] = res.new_global_timer + balances[M.receiver] = New_balance else S with @@ -2978,7 +3084,7 @@ If message execution <>; that would require a (unique) call to `ic0.reply`. Note also that the state changes are persisted even when the IC is set to synthesize a <> reject immediately afterward (which happens when this returns without calling `ic0.reply` or `ic0.reject`, the corresponding call has not been responded to and there are no outstanding callbacks, see <>). -The functions `query_as_update` and `heartbeat_as_update` turns a query function resp the heartbeat into an update function; this is merely a notational trick to simplify the rule: +The functions `query_as_update` and `system_task_as_update` turns a query function resp the heartbeat or global timer into an update function; this is merely a notational trick to simplify the rule: .... query_as_update(f, arg, env) = λ wasm_state → match f(arg, env)(wasm_state) with @@ -2987,18 +3093,20 @@ query_as_update(f, arg, env) = λ wasm_state → new_state = wasm_state; new_calls = []; new_certified_data = NoCertifiedData; + new_global_timer = NoGlobalTimer; response = res.response; cycles_accepted = 0; cycles_used = res.cycles_used; } -heartbeat_as_update(f, env) = λ wasm_state → +system_task_as_update(f, env) = λ wasm_state → match f(env)(wasm_state) with Trap trap → Trap trap Return res → Return { new_state = res.new_state; - new_calls = []; - new_certified_data = NoCertifiedData; + new_calls = res.new_calls; + new_certified_data = res.new_certified_data; + new_global_timer = res.new_global_timer; response = NoResponse; cycles_accepted = 0; cycles_used = res.cycles_used; @@ -3009,12 +3117,12 @@ Note that by construction, a query function will either trap or return with a re [#rule-starvation] ==== Call context starvation -If the call context is not for heartbeat and there is no call, downstream calling context or response that could possibly fulfill a calling context, then a reject is synthesized. The error message below is _not_ indicative. In particular, if the IC has an idea about _why_ this starved, it can put that in there (e.g. the initial message handler trapped with an out-of-memory access). +If the call context is not for heartbeat or global timer and there is no call, downstream calling context or response that could possibly fulfill a calling context, then a reject is synthesized. The error message below is _not_ indicative. In particular, if the IC has an idea about _why_ this starved, it can put that in there (e.g. the initial message handler trapped with an out-of-memory access). Conditions:: .... S.call_contexts[Ctxt_id].needs_to_respond = true - S.call_contexts[Ctxt_id].origin ≠ FromHeartbeat + S.call_contexts[Ctxt_id].origin ≠ FromSystem ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id ∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id ∀ _ ↦ {needs_to_respond = true, origin = FromCanister O, …} ∈ S.call_contexts: O.calling_context ≠ Ctxt_id @@ -3035,7 +3143,7 @@ S with ==== Call context removal -If there is no call, downstream calling context, or response that references a call context, and the call context has been replied to or the call context corresponds to a heartbeat that had already been executed, then the call context can be removed. +If there is no call, downstream calling context, or response that references a call context, and the call context has been replied to or the call context corresponds to a heartbeat or global timer that had already been executed, then the call context can be removed. Conditions:: .... @@ -3043,7 +3151,7 @@ Conditions:: S.call_contexts[Ctxt_id].needs_to_respond = false ) or ( - S.call_contexts[Ctxt_id].origin = FromHeartbeat + S.call_contexts[Ctxt_id].origin = FromSystem ∀ FuncMessage M ∈ S.messages. M.call_context ≠ Ctxt_id ) ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id @@ -3082,6 +3190,7 @@ State after:: S with canisters[CanisterId] = EmptyCanister time[CanisterId] = CurrentTime + global_timer[CanisterId] = 0 if A.settings.controllers is not null: controllers[CanisterId] = A.settings.controllers else: @@ -3099,6 +3208,7 @@ S with refunded_cycles = 0 } canister_status[CanisterId] = Running + canister_version[CanisterId] = 0 .... This uses the predicate @@ -3136,6 +3246,7 @@ S with controllers[A.canister_id] = A.settings.controllers if A.settings.freezing_threshold is not null: freezing_threshold[A.canister_id] = A.settings.freezing_threshold + canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin @@ -3201,12 +3312,14 @@ Conditions:: M.caller ∈ S.controllers[A.canister_id] Env = { time = S.time[A.canister_id]; + global_timer = S.global_timer[A.canister_id]; balance = S.balances[A.canister_id]; freezing_limit = freezing_limit(S, A.canister_id); certificate = NoCertificate; status = simple_status(S.canister_status[A.canister_id]); + canister_version = S.canister_version[A.canister_id] + 1; } - Mod.init(A.canister_id, A.arg, M.caller, Env) = Return {new_state = New_state; cycles_used = Cycles_used;} + Mod.init(A.canister_id, A.arg, M.caller, Env) = Return {new_state = New_state; new_certified_data = New_certified_data; new_global_timer = New_global_timer; cycles_used = Cycles_used;} Cycles_used ≤ S.balances[A.canister_id] dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ @@ -3221,6 +3334,13 @@ S with public_custom_sections = Public_custom_sections; private_custom_sections = Private_custom_sections; } + if New_certified_data ≠ NoCertifiedData: + certified_data[A.canister_id] = New_certified_data + if New_global_timer ≠ NoGlobalTimer: + global_timer[A.canister_id] = New_global_timer + else: + global_timer[A.canister_id] = 0 + canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 balances[A.canister_id] = S.balances[A.canister_id] - Cycles_used messages = Older_messages · Younger_messages · ResponseMessage { @@ -3254,8 +3374,16 @@ Conditions:: certificate = NoCertificate; status = simple_status(S.canister_status[A.canister_id]); } - Old_module.pre_upgrade(Old_State, M.caller, Env) = Return {stable_memory = Stable_memory; cycles_used = Cycles_used;} - Mod.post_upgrade(A.canister_id, Stable_memory, A.arg, M.caller, Env) = Return {new_state = New_state; cycles_used = Cycles_used';} + Env1 = Env with { + global_timer = S.global_timer[A.canister_id]; + canister_version = S.canister_version[A.canister_id]; + } + Old_module.pre_upgrade(Old_State, M.caller, Env1) = Return {stable_memory = Stable_memory; new_certified_data = New_certified_data; cycles_used = Cycles_used;} + Env2 = Env with { + global_timer = 0; + canister_version = S.canister_version[A.canister_id] + 1; + } + Mod.post_upgrade(A.canister_id, Stable_memory, A.arg, M.caller, Env2) = Return {new_state = New_state; new_certified_data = New_certified_data'; new_global_timer = New_global_timer; cycles_used = Cycles_used';} Cycles_used + Cycles_used' ≤ S.balances[A.canister_id] dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ .... @@ -3269,6 +3397,15 @@ S with public_custom_sections = Public_custom_sections; private_custom_sections = Private_custom_sections; } + if New_certified_data' ≠ NoCertifiedData: + certified_data[A.canister_id] = New_certified_data' + else if New_certified_data ≠ NoCertifiedData: + certified_data[A.canister_id] = New_certified_data + if New_global_timer ≠ NoGlobalTimer: + global_timer[A.canister_id] = New_global_timer + else: + global_timer[A.canister_id] = 0 + canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 balances[A.canister_id] = S.balances[A.canister_id] - (Cycles_used + Cycles_used'); messages = Older_messages · Younger_messages · ResponseMessage { @@ -3297,6 +3434,8 @@ State after:: S with canisters[A.canister_id] = EmptyCanister certified_data[A.canister_id] = "" + canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 + global_timer[A.canister_id] = 0 messages = Older_messages · Younger_messages · ResponseMessage { @@ -3489,7 +3628,9 @@ S with controllers[A.canister_id] = (deleted) freezing_threshold[A.canister_id] = (deleted) canister_status[A.canister_id] = (deleted) + canister_version[A.canister_id] = (deleted) time[A.canister_id] = (deleted) + global_timer[A.canister_id] = (deleted) balances[A.canister_id] = (deleted) certified_data[A.canister_id] = (deleted) messages = Older_messages · Younger_messages · @@ -3569,6 +3710,7 @@ State after:: S with canisters[CanisterId] = EmptyCanister time[CanisterId] = CurrentTime + global_timer[CanisterId] = 0 if A.settings.controllers is not null: controllers[CanisterId] = A.settings.controllers else: @@ -3589,6 +3731,7 @@ S with refunded_cycles = M.transferred_cycles } canister_status[CanisterId] = Running + canister_version[CanisterId] = 0 .... ==== IC Management Canister: Top up canister @@ -3669,15 +3812,15 @@ Conditions:: .... S.messages = Older_messages · ResponseMessage RM · Younger_messages RM.origin = FromUser { request = M } - S.requests[M] = Processing + S.requests[M] = (Processing, ECID) .... State after:: .... S with messages = Older_messages · Younger_messages requests[M] = - | Replied R if M.response = Reply R - | Rejected (c, R) if M.response = Reject (c, R) + | (Replied R, ECID) if M.response = Reply R + | (Rejected (c, R), ECID) if M.response = Reject (c, R) .... NB: The refunded cycles, `RM.refunded_cycles` are, by construction, empty. @@ -3688,12 +3831,12 @@ The IC will keep the data for a completed or rejected request around for a certa Conditions:: .... - (S.requests[M] = Replied _) or (S.requests[M] = Rejected _) + (S.requests[M] = (Replied _, ECID)) or (S.requests[M] = (Rejected _, ECID)) .... State after:: .... S with - requests[M] = Done + requests[M] = (Done, ECID) .... @@ -3701,7 +3844,7 @@ At the same or some later point, the request will be removed from the state of t Conditions:: .... - (S.requests[M] = Replied _) or (S.requests[M] = Rejected _) or (S.requests[M] = Done) + (S.requests[M] = (Replied _, _)) or (S.requests[M] = (Rejected _, _)) or (S.requests[M] = (Done, _)) M.ingress_expiry < S.system_time .... State after:: @@ -3722,7 +3865,9 @@ State after:: .... S with canisters[CanisterId] = EmptyCanister - certified_data[Canister_id] = "" + certified_data[CanisterId] = "" + canister_version[CanisterId] = S.canister_version[CanisterId] + 1 + global_timer[CanisterId] = 0 messages = S.messages · [ ResponseMessage { @@ -3743,7 +3888,7 @@ S with .... -==== Time progressing and cycle consumption +==== Time progressing, cycle consumption, and canister version increments Time progresses. Abstractly, it does so independently for each canister, and in unspecified intervals. @@ -3784,6 +3929,19 @@ S with system_time = T1 .... +Finally, the canister version can be incremented arbitrarily: + +Conditions:: +.... + N0 = S.canister_version[CanisterId] + N1 > N0 +.... +State after:: +.... +S with + canister_version[CanisterId] = N1 +.... + ==== Query call Canister query calls to `/api/v2/canister//query` can be executed directly. They can only be executed against canisters which have a status of `Running` and are also not frozen. @@ -3806,10 +3964,12 @@ Conditions:: lookup(["time"], Cert) = Found S.system_time // or “recent enough” Env = { time = S.time[Q.receiver]; + global_timer = S.global_timer[Q.receiver]; balance = S.balances[Q.canister_id]; freezing_limit = freezing_limit(S, Q.canister_id); certificate = Cert; status = simple_status(S.canister_status[Q.receiver]); + canister_version = S.canister_version[Q.receiver]; } .... Read response:: @@ -3955,7 +4115,9 @@ ExecutionState = { pending_call : MethodCall | NoPendingCall; calls : List MethodCall; new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; ingress_filter : Accept | Reject; + context : I | G | U | Q | Ry | Rt | C | F | T; } .... @@ -3994,7 +4156,9 @@ empty_execution_state = { pending_call = NoPendingCall; calls = []; new_certified_data = NoCertifiedData; + new_global_timer = NoGlobalTimer; ingress_filter = Reject; + context = (undefined); } .... @@ -4005,7 +4169,12 @@ If the WebAssembly module does not export a function called under the name `cani + .... init = λ (self_id, arg, caller, sysenv) → - Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" }; cycles_used = 0;} + Return { + new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" }; + new_certified_data = NoCertifiedData; + new_global_timer = NoGlobalTimer; + cycles_used = 0; + } .... + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_init`, it is @@ -4016,12 +4185,15 @@ init = λ (self_id, arg, caller, sysenv) → wasm_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" } params = empty_params with { arg = arg; caller = caller; sysenv } balance = sysenv.balance + context = I } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} - if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} - Return {new_state = es.wasm_state; cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + new_certified_data = es.new_certified_data; + new_global_timer = es.new_global_timer; + cycles_used = es.cycles_used; + } .... + This formulation checks afterwards that the system calls `call.perform` or `msg.reply` were not invoked; an implementation can of course trap as soon as these system calls are invoked. @@ -4031,7 +4203,7 @@ This formulation checks afterwards that the system calls `call.perform` or `msg. If the WebAssembly module does not export a function called under the name `canister_pre_upgrade`, then it simply returns the stable memory: + .... -pre_upgrade = λ (old_state, caller, sysenv) → Return {stable_memory = old_state.stable_mem; cycles_used = 0;} +pre_upgrade = λ (old_state, caller, sysenv) → Return {stable_memory = old_state.stable_mem; new_certified_data = NoCertifiedData; cycles_used = 0;} .... + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_pre_upgrade`, it is @@ -4042,12 +4214,14 @@ pre_upgrade = λ (old_state, caller, sysenv) → wasm_state = old_state params = { empty_params with caller = caller; sysenv } balance = sysenv.balance + context = G } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} - if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} - Return {stable_memory = es.wasm_state.stable_mem; cycles_used = es.cycles_used;} + Return { + stable_memory = es.wasm_state.stable_mem; + new_certified_data = es.new_certified_data; + cycles_used = es.cycles_used; + } .... @@ -4057,7 +4231,7 @@ If the WebAssembly module does not export a function called under the name `cani + .... post_upgrade = λ (self_id, stable_mem, arg, caller, sysenv) → - Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem }; cycles_used = 0;} + Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem }; new_certified_data = NoCertifiedData; new_global_timer = NoGlobalTimer; cycles_used = 0;} .... + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_post_upgrade`, it is @@ -4068,12 +4242,15 @@ post_upgrade = λ (self_id, stable_mem, arg, caller, sysenv) → wasm_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem } params = { empty_params with arg = arg; caller = caller; sysenv } balance = sysenv.balance + context = I } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} - if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} - Return {new_state = es.wasm_state; cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + new_certified_data = es.new_certified_data; + new_global_timer = es.new_global_timer; + cycles_used = es.cycles_used; + } .... * The partial map `update_methods` of the `CanisterModule` is defined for all method names `method` for which the WebAssembly program exports a function `func` named `canister_update `, and has value @@ -4085,16 +4262,17 @@ update_methods[method] = λ (arg, caller, sysenv, available) → λ wasm_state params = empty_params with { arg = arg; caller = caller; sysenv } balance = sysenv.balance cycles_available = available; + context = U } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} Return { new_state = es.wasm_state; new_calls = es.calls; + new_certified_data = es.new_certified_data; + new_global_timer = es.new_global_timer; response = es.response; cycles_accepted = es.cycles_accepted; cycles_used = es.cycles_used; - new_certified_data = es.new_certified_data; } .... @@ -4106,12 +4284,9 @@ query_methods[method] = λ (arg, caller, sysenv) → λ wasm_state → wasm_state = wasm_state; params = empty_params with { arg = arg; caller = caller; sysenv } balance = sysenv.balance + context = Q } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used;} - if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} - if es.response = NoResponse then Trap {cycles_used = es.cycles_used;} Return { response = es.response; cycles_used = es.cycles_used; @@ -4130,13 +4305,14 @@ heartbeat = λ (sysenv) → λ wasm_state → wasm_state = wasm_state; params = empty_params with { arg = NoArg; caller = NoCaller; sysenv } balance = sysenv.balance + context = T } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used;} - if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} Return { new_state = es.wasm_state; + new_calls = es.calls; + new_certified_data = es.certified_data; + new_global_timer = es.new_global_timer; cycles_used = es.cycles_used; } .... @@ -4145,6 +4321,30 @@ otherwise it is heartbeat = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} .... +* The function `global_timer` of the `CanisterModule` is defined if the WebAssembly program exports a function `func` named `canister_global_timer`, and has value ++ +.... +global_timer = λ (sysenv) → λ wasm_state → + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = empty_params with { arg = NoArg; caller = NoCaller; sysenv } + balance = sysenv.balance + context = T + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + new_calls = es.calls; + new_certified_data = es.certified_data; + new_global_timer = es.new_global_timer; + cycles_used = es.cycles_used; + } +.... +otherwise it is +.... +global_timer = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} +.... + * The function `callbacks` of the `CanisterModule` is defined as follows + .... @@ -4153,18 +4353,19 @@ callbacks = λ(callbacks, response, refunded_cycles, sysenv, available) → λ w sysenv cycles_refunded = refund_cycles; } - let (fun, env, params) = match response with + let (fun, env, params, context) = match response with Reply data -> (callbacks.on_reply.fun, callbacks.on_reply.env, - { params0 with data}) + { params0 with data}, Ry) Reject (reject_code, reject_message)-> (callbacks.on_reject.fun, callbacks.on_reject.env, - { params0 with reject_code; reject_message}) + { params0 with reject_code; reject_message}, Rt) let es = ref {empty_execution_state with wasm_state = wasm_state; params = params; balance = sysenv.balance; cycles_available = available; + context = context; } try if fun > |es.wasm_state.store.table| then Trap @@ -4175,10 +4376,11 @@ callbacks = λ(callbacks, response, refunded_cycles, sysenv, available) → λ w Return { new_state = es.wasm_state; new_calls = es.calls; + new_certified_data = es.certified_data; + new_global_timer = es.new_global_timer; response = es.response; cycles_accepted = es.cycles_accepted; cycles_used = es.cycles_used; - new_certified_data = es.certified_data; } with Trap if callbacks.on_cleanup = NoClosure then Trap {cycles_used = es.cycles_used;} @@ -4188,15 +4390,14 @@ callbacks = λ(callbacks, response, refunded_cycles, sysenv, available) → λ w let es' = ref { empty_execution_state with wasm_state = wasm_state; + context = C; } try func(callbacks.on_cleanup.env) with Trap then Trap {cycles_used = es.cycles_used + es'.cycles_used;} - if es'.cycles_accepted ≠ 0 then Trap {cycles_used = es.cycles_used + es'.cycles_used;} - if es'.calls ≠ [] then Trap {cycles_used = es.cycles_used + es'.cycles_used;} - if es'.ingress_filter ≠ Reject then Trap {cycles_used = es.cycles_used + es'.cycles_used;} - if es'.response ≠ NoResponse then Trap {cycles_used = es.cycles_used + es'.cycles_used;} Return { new_state = es'.wasm_state; new_calls = []; + new_certified_data = NoCertifiedData; + new_global_timer = es'.new_global_timer; response = NoResponse; cycles_accepted = 0; cycles_used = es.cycles_used + es'.cycles_used; @@ -4228,10 +4429,9 @@ inspect_message = λ (method_name, wasm_state, arg, caller, sysenv) → } balance = sysenv.balance; cycles_available = 0; // ingress requests have no funds + context = F; } try func() with Trap then Trap {cycles_used = es.cycles_used;} - if es.calls ≠ [] then Trap {cycles_used = es.cycles_used;} - if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} Return {status = es.ingress_filter; cycles_used = es.cycles_used;}; .... @@ -4268,67 +4468,85 @@ The pseudo-code below does _not_ explicitly enforce the restrictions of which im .... ic0.msg_arg_data_size() : i32 = + if es.context ∉ {I, U, Q, Ry, F} then Trap {cycles_used = es.cycles_used;} return |es.params.arg| ic0.msg_arg_data_copy(dst:i32, offset:i32, size:i32) = + if es.context ∉ {I, U, Q, Ry, F} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.arg) ic0.msg_caller_size() : i32 = + if es.context ∉ {I, G, U, Q, F} then Trap {cycles_used = es.cycles_used;} return |es.params.caller| ic0.msg_caller_copy(dst:i32, offset:i32, size:i32) : i32 = + if es.context ∉ {I, G, U, Q, F} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.caller) ic0.msg_reject_code() : i32 = + if es.context ∉ {Ry, Rt} then Trap {cycles_used = es.cycles_used;} es.params.reject_code ic0.msg_reject_msg_size() : i32 = + if es.context ∉ {Rt} then Trap {cycles_used = es.cycles_used;} return |es.params.reject_msg| ic0.msg_reject_msg_copy(dst:i32, offset:i32, size:i32) : i32 = + if es.context ∉ {Rt} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.reject_msg) ic0.msg_reply_data_append(src : i32, size : i32) = + if es.context ∉ {U, Q, Ry, Rt} then Trap {cycles_used = es.cycles_used;} if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.reply_params.arg := es.reply_params.arg · copy_from_canister(src, size) ic0.msg_reply() = + if es.context ∉ {U, Q, Ry, Rt} then Trap {cycles_used = es.cycles_used;} if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.response := Reply (es.reply_params.arg) es.cycles_available := 0 ic0.msg_reject(src : i32, size : i32) = + if es.context ∉ {U, Q, Ry, Rt} then Trap {cycles_used = es.cycles_used;} if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.response := Reject (CANISTER_REJECT, copy_from_canister(src, size)) es.cycles_available := 0 ic0.msg_cycles_available() : i64 = + if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} if es.cycles_available >= 2^64^ then Trap {cycles_used = es.cycles_used;} return es.cycles_available ic0.msg_cycles_available128(dst : i32) = + if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} let amount = es.cycles_available copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) ic0.msg_cycles_refunded() : i64 = + if es.context ∉ {Rt, Ry} then Trap {cycles_used = es.cycles_used;} if es.params.cycles_refunded >= 2^64^ then Trap {cycles_used = es.cycles_used;} return es.params.cycles_refunded ic0.msg_cycles_refunded128(dst : i32) = + if es.context ∉ {Rt, Ry} then Trap {cycles_used = es.cycles_used;} let amount = es.params.cycles_refunded copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) ic0.accept_message() = + if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} if es.ingress_filter = Accept then Trap {cycles_used = es.cycles_used;} es.ingress_filter = Accept ic0.msg_method_name_size() : i32 = + if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} return |es.method_name| ic0.msg_method_name_copy(dst : i32, offset : i32, size : i32) : i32 = + if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.method_name) ic0.msg_cycles_accept(max_amount : i64) : i64 = + if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} let amount = min(max_amount, es.cycles_available) es.cycles_available := es.cycles_available - amount es.cycles_accepted := es.cycles_accepted + amount @@ -4336,6 +4554,7 @@ ic0.msg_cycles_accept(max_amount : i64) : i64 = return amount ic0.msg_cycles_accept128(max_amount_high : i64, max_amount_low : i64, dst : i32) = + if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} let max_amount = max_amount_high * 2^64^ + max_amount_low let amount = min(max_amount, es.cycles_available) es.cycles_available := es.cycles_available - amount @@ -4363,6 +4582,9 @@ ic0.canister_status() : i32 = Stopping -> return 2 Stopped -> return 3 +ic0.canister_version() : i64 = + return es.params.sysenv.canister_version + ic0.call_new( callee_src : i32, callee_size : i32, @@ -4373,6 +4595,8 @@ ic0.call_new( reject_fun : i32, reject_env : i32, ) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} + discard_pending_call() if es.balance < MAX_CYCLES_PER_RESPONSE then Trap {cycles_used = es.cycles_used;} @@ -4400,10 +4624,12 @@ ic0.call_new( } ic0.call_data_append (src : i32, size : i32) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} es.pending_call.arg := es.pending_call.arg · copy_from_canister(src, size) ic0.call_on_cleanup (fun : i32, env : i32) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} if fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} if typeof(es.wasm_state.store.table[fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} @@ -4411,6 +4637,7 @@ ic0.call_on_cleanup (fun : i32, env : i32) = es.pending_call.callback.on_cleanup := Closure { fun = fun; env = env} ic0.call_cycles_add(amount : i64) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} if es.balance < amount then Trap {cycles_used = es.cycles_used;} @@ -4418,6 +4645,7 @@ ic0.call_cycles_add(amount : i64) = es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount ic0.call_cycles_add128(amount_high : i64, amount_low : i64) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} let amount = amount_high * 2^64^ + amount_low if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} if es.balance < amount then Trap {cycles_used = es.cycles_used;} @@ -4426,6 +4654,7 @@ ic0.call_cycles_add128(amount_high : i64, amount_low : i64) = es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount ic0.call_peform() : ( err_code : i32 ) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} // are we below the threezing threshold? @@ -4499,9 +4728,18 @@ ic0.stable64_read(dst : i64, offset : i64, size : i64) es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] ic0.time() : i32 = - return es.params.time + return es.params.sysenv.time + +ic0.global_timer_set(timestamp: i64) : i64 = + if es.context ∉ {I, U, Ry, Rt, C, T} then Trap {cycles_used = es.cycles_used;} + let prev_global_timer = es.new_global_timer + es.new_global_timer := timestamp + if prev_global_timer = NoGlobalTimer + then return es.params.sysenv.global_timer + else return prev_global_timer ic0.certified_data_set(src: i32, size: i32) = + if es.context ∉ {I, G, U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} es.new_certified_data := es.wasm_state[src..src+size] ic0.data_certificate_present() : i32 = diff --git a/theories/IC.thy b/theories/IC.thy index d026c3f3e..ada3a64db 100644 --- a/theories/IC.thy +++ b/theories/IC.thy @@ -175,13 +175,16 @@ type_synonym 'b arg = 'b type_synonym 'p caller_id = 'p type_synonym timestamp = nat +type_synonym canister_version = nat datatype status = Running | Stopping | Stopped record ('b) env = time :: timestamp + global_timer :: nat balance :: nat freezing_limit :: nat certificate :: "'b option" status :: status + canister_version :: canister_version type_synonym reject_code = nat datatype ('b, 's) response = @@ -197,51 +200,66 @@ record ('p, 'canid, 's, 'b, 'c) method_call = record 'x cycles_return = return :: 'x cycles_used :: nat +record ('w, 'b) init_return = + new_state :: 'w + new_certified_data :: "'b option" + new_global_timer :: "nat option" + cycles_used :: nat +record ('sm, 'b) pre_upgrade_return = + stable_memory :: 'sm + new_certified_data :: "'b option" + cycles_used :: nat type_synonym trap_return = "unit cycles_return" record ('w, 'p, 'canid, 's, 'b, 'c) update_return = new_state :: 'w new_calls :: "('p, 'canid, 's, 'b, 'c) method_call list" new_certified_data :: "'b option" + new_global_timer :: "nat option" response :: "('b, 's) response option" cycles_accepted :: nat cycles_used :: nat record ('b, 's) query_return = response :: "('b, 's) response" cycles_used :: nat -record 'w heartbeat_return = +record ('w, 'p, 'canid, 's, 'b, 'c) system_task_return = new_state :: 'w + new_calls :: "('p, 'canid, 's, 'b, 'c) method_call list" + new_certified_data :: "'b option" + new_global_timer :: "nat option" cycles_used :: nat type_synonym ('w, 'p, 'canid, 's, 'b, 'c) update_func = "'w \ trap_return + ('w, 'p, 'canid, 's, 'b, 'c) update_return" type_synonym ('w, 'b, 's) query_func = "'w \ trap_return + ('b, 's) query_return" -type_synonym 'w heartbeat_func = "'w \ trap_return + 'w heartbeat_return" +type_synonym ('w, 'p, 'canid, 's, 'b, 'c) system_task_func = "'w \ trap_return + ('w, 'p, 'canid, 's, 'b, 'c) system_task_return" type_synonym available_cycles = nat type_synonym refunded_cycles = nat datatype inspect_method_result = Accept | Reject record ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module_rec = - init :: "'canid \ 'b arg \ 'p caller_id \ 'b env \ trap_return + 'w cycles_return" - pre_upgrade :: "'w \ 'p \ 'b env \ trap_return + 'sm cycles_return" - post_upgrade :: "'canid \ 'sm \ 'b arg \ 'p caller_id \ 'b env \ trap_return + 'w cycles_return" + init :: "'canid \ 'b arg \ 'p caller_id \ 'b env \ trap_return + ('w, 'b) init_return" + pre_upgrade :: "'w \ 'p \ 'b env \ trap_return + ('sm, 'b) pre_upgrade_return" + post_upgrade :: "'canid \ 'sm \ 'b arg \ 'p caller_id \ 'b env \ trap_return + ('w, 'b) init_return" update_methods :: "('s method_name, ('b arg \ 'p caller_id \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func) list_map" query_methods :: "('s method_name, ('b arg \ 'p caller_id \ 'b env) \ ('w, 'b, 's) query_func) list_map" - heartbeat :: "'b env \ 'w heartbeat_func" + heartbeat :: "'b env \ ('w, 'p, 'canid, 's, 'b, 'c) system_task_func" + global_timer :: "'b env \ ('w, 'p, 'canid, 's, 'b, 'c) system_task_func" callbacks :: "('c \ ('b, 's) response \ refunded_cycles \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" inspect_message :: "('s method_name \ 'w \ 'b arg \ 'p caller_id \ 'b env) \ trap_return + inspect_method_result cycles_return" typedef ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module = "{m :: ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module_rec. list_map_dom (update_methods m) \ list_map_dom (query_methods m) = {}}" by (auto intro: exI[of _ "\init = undefined, pre_upgrade = undefined, post_upgrade = undefined, - update_methods = list_map_empty, query_methods = list_map_empty, heartbeat = undefined, callbacks = undefined, + update_methods = list_map_empty, query_methods = list_map_empty, heartbeat = undefined, global_timer = undefined, callbacks = undefined, inspect_message = undefined\"]) setup_lifting type_definition_canister_module -lift_definition canister_module_init :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'canid \ 'b arg \ 'p \ 'b env \ trap_return + 'w cycles_return" is "init" . -lift_definition canister_module_pre_upgrade :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'w \ 'p \ 'b env \ trap_return + 'sm cycles_return" is pre_upgrade . -lift_definition canister_module_post_upgrade :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'canid \ 'sm \ 'b arg \ 'p \ 'b env \ trap_return + 'w cycles_return" is post_upgrade . +lift_definition canister_module_init :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'canid \ 'b arg \ 'p \ 'b env \ trap_return + ('w, 'b) init_return" is "init" . +lift_definition canister_module_pre_upgrade :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'w \ 'p \ 'b env \ trap_return + ('sm, 'b) pre_upgrade_return" is pre_upgrade . +lift_definition canister_module_post_upgrade :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'canid \ 'sm \ 'b arg \ 'p \ 'b env \ trap_return + ('w, 'b) init_return" is post_upgrade . lift_definition canister_module_update_methods :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('s, ('b arg \ 'p \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func) list_map" is update_methods . lift_definition canister_module_query_methods :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('s, ('b arg \ 'p \ 'b env) \ ('w, 'b, 's) query_func) list_map" is query_methods . -lift_definition canister_module_heartbeat :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'b env \ 'w heartbeat_func" is heartbeat . +lift_definition canister_module_heartbeat :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c) system_task_func" is heartbeat . +lift_definition canister_module_global_timer :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c) system_task_func" is global_timer . lift_definition canister_module_callbacks :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('c \ ('b, 's) response \ refunded_cycles \ 'b env \ available_cycles) \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" is callbacks . lift_definition canister_module_inspect_message :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('s \ 'w \ 'b arg \ 'p \ 'b env) \ trap_return + inspect_method_result cycles_return" is inspect_message . @@ -261,7 +279,7 @@ record ('b, 'p, 'uid, 'canid, 's) request = datatype ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin = From_user "('b, 'p, 'uid, 'canid, 's) request" | From_canister "'cid" "'c" -| From_heartbeat +| From_system record ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt_rep = canister :: 'canid origin :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin" @@ -325,6 +343,7 @@ datatype ('s, 'p, 'b, 'c) entry_point = Public_method "'s method_name" "'p" "'b" | Callback "'c" "('b, 's) response" "refunded_cycles" | Heartbeat +| Global_timer datatype ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message = Call_message "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) call_origin" 'p 'canid 's 'b nat "'canid queue" @@ -381,7 +400,9 @@ record ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic = controllers :: "('canid, 'p set) list_map" freezing_threshold :: "('canid, nat) list_map" canister_status :: "('canid, ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status) list_map" + canister_version :: "('canid, canister_version) list_map" time :: "('canid, timestamp) list_map" + global_timer :: "('canid, nat) list_map" balances :: "('canid, nat) list_map" certified_data :: "('canid, 'b) list_map" system_time :: timestamp @@ -402,7 +423,9 @@ definition initial_ic :: "nat \ 'pk \ ('p, 'uid, 'canid, controllers = list_map_empty, freezing_threshold = list_map_empty, canister_status = list_map_empty, + canister_version = list_map_empty, time = list_map_empty, + global_timer = list_map_empty, balances = list_map_empty, certified_data = list_map_empty, system_time = t, @@ -557,6 +580,7 @@ fun cycles_reserved :: "('s, 'p, 'b, 'c) entry_point \ nat" where "cycles_reserved (entry_point.Public_method _ _ _) = MAX_CYCLES_PER_MESSAGE" | "cycles_reserved (entry_point.Callback _ _ _) = MAX_CYCLES_PER_RESPONSE" | "cycles_reserved (entry_point.Heartbeat) = MAX_CYCLES_PER_MESSAGE" +| "cycles_reserved (entry_point.Global_timer) = MAX_CYCLES_PER_MESSAGE" fun message_cycles :: "('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message \ nat" where "message_cycles (Call_message orig _ _ _ _ trans_cycles q) = carried_cycles orig + trans_cycles" @@ -638,9 +662,9 @@ definition request_submission_pre :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) enve \ ( request.canister_id req \ ic_principal \ - (case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req)) of - (Some (Some can), Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status\ in + (case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req), list_map_get (canister_version S) (request.canister_id req), list_map_get (global_timer S) (request.canister_id req)) of + (Some (Some can), Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status, canister_version = idx\ in (case canister_module_inspect_message (module can) (request.method_name req, wasm_state can, request.arg req, principal_of_uid (request.sender req), env) of Inr ret \ cycles_return.return ret = Accept \ cycles_return.cycles_used ret \ bal | _ \ False) @@ -654,9 +678,9 @@ definition request_submission_post :: "('b, 'p, 'uid, 'canid, 's, 'pk, 'sig) env let req = projl (content E); cid = request.canister_id req; balances = (if cid \ ic_principal then - (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of - (Some (Some can), Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) (request.canister_id req), list_map_get (global_timer S) (request.canister_id req)) of + (Some (Some can), Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = idx\ in (case canister_module_inspect_message (module can) (request.method_name req, wasm_state can, request.arg req, principal_of_uid (request.sender req), env) of Inr ret \ list_map_set (balances S) cid (bal - cycles_return.cycles_used ret))) else balances S) in @@ -667,9 +691,9 @@ definition request_submission_burned_cycles :: "('b, 'p, 'uid, 'canid, 's, 'pk, let req = projl (content E); cid = request.canister_id req in (if request.canister_id req \ ic_principal then - (case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req)) of - (Some (Some can), Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status\ in + (case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req), list_map_get (canister_version S) (request.canister_id req), list_map_get (global_timer S) (request.canister_id req)) of + (Some (Some can), Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status, canister_version = idx\ in (case canister_module_inspect_message (module can) (request.method_name req, wasm_state can, request.arg req, principal_of_uid (request.sender req), env) of Inr ret \ cycles_return.cycles_used ret)) else 0))" @@ -683,9 +707,9 @@ proof - by (auto simp: request_submission_pre_def split: sum.splits) { assume "request.canister_id req \ ic_principal" - then have "(case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req)) of - (Some (Some can), Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status\ in + then have "(case (list_map_get (canisters S) (request.canister_id req), list_map_get (time S) (request.canister_id req), list_map_get (balances S) (request.canister_id req), list_map_get (canister_status S) (request.canister_id req), list_map_get (canister_version S) (request.canister_id req), list_map_get (global_timer S) (request.canister_id req)) of + (Some (Some can), Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S (request.canister_id req), certificate = None, status = simple_status can_status, canister_version = idx\ in (case canister_module_inspect_message (module can) (request.method_name req, wasm_state can, request.arg req, principal_of_uid (request.sender req), env) of Inr ret \ cycles_return.return ret = Accept \ cycles_return.cycles_used ret \ bal | _ \ False) @@ -706,7 +730,7 @@ lemma request_submission_ic_inv: shows "ic_inv (request_submission_post E ECID S)" using assms by (auto simp: ic_inv_def request_submission_pre_def request_submission_post_def Let_def - split: sum.splits message.splits call_origin.splits) + split: sum.splits message.splits call_origin.splits prod.splits) @@ -868,6 +892,16 @@ lemma call_context_create_ic_inv: (* System transition: Call context creation: Heartbeat [DONE] *) +lift_definition create_call_ctxt_system_task :: "'canid \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt" is + "\cee. \canister = cee, origin = From_system, needs_to_respond = False, deleted = False, available_cycles = 0\" + by auto + +lemma create_call_ctxt_system_task_needs_to_respond[simp]: "call_ctxt_needs_to_respond (create_call_ctxt_system_task cee) = False" + by transfer auto + +lemma create_call_ctxt_system_task_carried_cycles[simp]: "call_ctxt_carried_cycles (create_call_ctxt_system_task cee) = 0" + by transfer auto + definition call_context_heartbeat_pre :: "'canid \ 'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "call_context_heartbeat_pre cee ctxt_id S = ( (case list_map_get (canisters S) cee of Some (Some can) \ True | _ \ False) \ @@ -875,21 +909,11 @@ definition call_context_heartbeat_pre :: "'canid \ 'cid \ bal \ ic_freezing_limit S cee + MAX_CYCLES_PER_MESSAGE | _ \ False) \ ctxt_id \ list_map_dom (call_contexts S))" -lift_definition create_call_ctxt_heartbeat :: "'canid \ ('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt" is - "\cee. \canister = cee, origin = From_heartbeat, needs_to_respond = False, deleted = False, available_cycles = 0\" - by auto - -lemma create_call_ctxt_heartbeat_needs_to_respond[simp]: "call_ctxt_needs_to_respond (create_call_ctxt_heartbeat cee) = False" - by transfer auto - -lemma create_call_ctxt_heartbeat_carried_cycles[simp]: "call_ctxt_carried_cycles (create_call_ctxt_heartbeat cee) = 0" - by transfer auto - definition call_context_heartbeat_post :: "'canid \ 'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where "call_context_heartbeat_post cee ctxt_id S = (case list_map_get (balances S) cee of Some bal \ S\messages := Func_message ctxt_id cee Heartbeat (Queue System cee) # messages S, - call_contexts := list_map_set (call_contexts S) ctxt_id (create_call_ctxt_heartbeat cee), + call_contexts := list_map_set (call_contexts S) ctxt_id (create_call_ctxt_system_task cee), balances := list_map_set (balances S) cee (bal - MAX_CYCLES_PER_MESSAGE)\)" lemma call_context_heartbeat_cycles_inv: @@ -910,17 +934,54 @@ lemma call_context_heartbeat_ic_inv: +(* System transition: Call context creation: Global timer [DONE] *) + +definition call_context_global_timer_pre :: "'canid \ 'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "call_context_global_timer_pre cee ctxt_id S = ( + (case list_map_get (canisters S) cee of Some (Some can) \ True | _ \ False) \ + list_map_get (canister_status S) cee = Some Running \ + (case (list_map_get (time S) cee, list_map_get (global_timer S) cee) of (Some t, Some timer) \ timer \ 0 \ t \ timer | _ \ False) \ + (case list_map_get (balances S) cee of Some bal \ bal \ ic_freezing_limit S cee + MAX_CYCLES_PER_MESSAGE | _ \ False) \ + ctxt_id \ list_map_dom (call_contexts S))" + +definition call_context_global_timer_post :: "'canid \ 'cid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "call_context_global_timer_post cee ctxt_id S = + (case list_map_get (balances S) cee of Some bal \ + S\messages := Func_message ctxt_id cee Global_timer (Queue System cee) # messages S, + call_contexts := list_map_set (call_contexts S) ctxt_id (create_call_ctxt_system_task cee), + global_timer := list_map_set (global_timer S) cee 0, + balances := list_map_set (balances S) cee (bal - MAX_CYCLES_PER_MESSAGE)\)" + +lemma call_context_global_timer_cycles_inv: + assumes "call_context_global_timer_pre cee ctxt_id S" + shows "total_cycles S = total_cycles (call_context_global_timer_post cee ctxt_id S)" + using assms list_map_sum_in_ge[of "balances S" cee, where ?g=id, simplified] + by (auto simp: call_context_global_timer_pre_def call_context_global_timer_post_def total_cycles_def + list_map_sum_in[where ?g=id, simplified] list_map_sum_out split: option.splits) + +lemma call_context_global_timer_ic_inv: + assumes "call_context_global_timer_pre cee ctxt_id S" "ic_inv S" + shows "ic_inv (call_context_global_timer_post cee ctxt_id S)" + using assms + by (auto simp: ic_inv_def call_context_global_timer_pre_def call_context_global_timer_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD + intro: ic_can_status_inv_mono2) + + + (* System transition: Message execution [DONE] *) fun query_as_update :: "(('b arg \ 'p \ 'b env) \ ('w, 'b, 's) query_func) \ 'b arg \ 'p \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" where "query_as_update (f, a, e) = (\w. case f (a, e) w of Inl t \ Inl t | - Inr res \ Inr \update_return.new_state = w, update_return.new_calls = [], update_return.new_certified_data = None, + Inr res \ Inr \update_return.new_state = w, update_return.new_calls = [], update_return.new_certified_data = None, update_return.new_global_timer = None, update_return.response = Some (query_return.response res), update_return.cycles_accepted = 0, update_return.cycles_used = query_return.cycles_used res\)" -fun heartbeat_as_update :: "('b env \ 'w heartbeat_func) \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" where - "heartbeat_as_update (f, e) = (\w. case f e w of Inl t \ Inl t | - Inr res \ Inr \update_return.new_state = heartbeat_return.new_state res, update_return.new_calls = [], update_return.new_certified_data = None, - update_return.response = None, update_return.cycles_accepted = 0, update_return.cycles_used = heartbeat_return.cycles_used res\)" +fun system_task_as_update :: "('b env \ ('w, 'p, 'canid, 's, 'b, 'c) system_task_func) \ 'b env \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" where + "system_task_as_update (f, e) = (\w. case f e w of Inl t \ Inl t | + Inr res \ Inr \update_return.new_state = system_task_return.new_state res, update_return.new_calls = system_task_return.new_calls res, + update_return.new_certified_data = system_task_return.new_certified_data res, update_return.new_global_timer = system_task_return.new_global_timer res, + update_return.response = None, update_return.cycles_accepted = 0, update_return.cycles_used = system_task_return.cycles_used res\)" fun exec_function :: "('s, 'p, 'b, 'c) entry_point \ 'b env \ nat \ ('p, 'canid, 'b, 'w, 'sm, 'c, 's) canister_module \ ('w, 'p, 'canid, 's, 'b, 'c) update_func" where "exec_function (entry_point.Public_method mn c a) e bal m = ( @@ -930,25 +991,26 @@ fun exec_function :: "('s, 'p, 'b, 'c) entry_point \ 'b env \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "message_execution_pre n S = (n < length (messages S) \ (case messages S ! n of Func_message ctxt_id recv ep q \ (q = Unordered \ (\j < n. message_queue (messages S ! j) \ Some q)) \ (case (list_map_get (canisters S) recv, list_map_get (balances S) recv, list_map_get (canister_status S) recv, - list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id) of - (Some (Some can), Some bal, Some can_status, Some t, Some ctxt) \ True | _ \ False) + list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id, list_map_get (canister_version S) recv, list_map_get (global_timer S) recv) of + (Some (Some can), Some bal, Some can_status, Some t, Some ctxt, Some idx, Some timer) \ True | _ \ False) | _ \ False))" definition message_execution_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where "message_execution_post n S = (case messages S ! n of Func_message ctxt_id recv ep q \ (case (list_map_get (canisters S) recv, list_map_get (balances S) recv, list_map_get (canister_status S) recv, - list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id) of - (Some (Some can), Some bal, Some can_status, Some t, Some ctxt) \ ( + list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id, list_map_get (canister_version S) recv, list_map_get (global_timer S) recv) of + (Some (Some can), Some bal, Some can_status, Some t, Some ctxt, Some idx, Some timer) \ ( let Mod = module can; Is_response = (case ep of Callback _ _ _ \ True | _ \ False); - Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\; + Env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status, canister_version = idx\; Available = call_ctxt_available_cycles ctxt; F = exec_function ep Env Available Mod; R = F (wasm_state can); @@ -972,11 +1034,13 @@ definition message_execution_post :: "nat \ ('p, 'uid, 'canid, 'b, ' new_ctxt = (case update_return.response result of None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt | Some _ \ call_ctxt_respond ctxt); - certified_data = (case new_certified_data result of None \ certified_data S - | Some cd \ list_map_set (certified_data S) recv cd) + certified_data = (case update_return.new_certified_data result of None \ certified_data S + | Some cd \ list_map_set (certified_data S) recv cd); + global_timer = (case update_return.new_global_timer result of None \ global_timer S + | Some new_timer \ list_map_set (global_timer S) recv new_timer) in S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), messages := messages, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, - certified_data := certified_data, balances := list_map_set (balances S) recv New_balance\) + certified_data := certified_data, global_timer := global_timer, balances := list_map_set (balances S) recv New_balance\) else S\messages := take n (messages S) @ drop (Suc n) (messages S), balances := list_map_set (balances S) recv ((bal + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) - min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))\)) @@ -985,11 +1049,11 @@ definition message_execution_post :: "nat \ ('p, 'uid, 'canid, 'b, ' definition message_execution_burned_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where "message_execution_burned_cycles n S = (case messages S ! n of Func_message ctxt_id recv ep q \ (case (list_map_get (canisters S) recv, list_map_get (balances S) recv, list_map_get (canister_status S) recv, - list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id) of - (Some (Some can), Some bal, Some can_status, Some t, Some ctxt) \ ( + list_map_get (time S) recv, list_map_get (call_contexts S) ctxt_id, list_map_get (canister_version S) recv, list_map_get (global_timer S) recv) of + (Some (Some can), Some bal, Some can_status, Some t, Some ctxt, Some idx, Some timer) \ ( let Mod = module can; Is_response = (case ep of Callback _ _ _ \ True | _ \ False); - Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\; + Env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status, canister_version = idx\; Available = call_ctxt_available_cycles ctxt; F = exec_function ep Env Available Mod; R = F (wasm_state can); @@ -1001,23 +1065,25 @@ lemma message_execution_cycles_monotonic: assumes pre: "message_execution_pre n S" shows "total_cycles S = total_cycles (message_execution_post n S) + message_execution_burned_cycles n S" proof - - obtain ctxt_id recv ep q can bal can_status t ctxt where msg: "messages S ! n = Func_message ctxt_id recv ep q" + obtain ctxt_id recv ep q can bal can_status t ctxt idx timer where msg: "messages S ! n = Func_message ctxt_id recv ep q" and prod: "list_map_get (canisters S) recv = Some (Some can)" "list_map_get (balances S) recv = Some bal" "list_map_get (canister_status S) recv = Some can_status" "list_map_get (time S) recv = Some t" "list_map_get (call_contexts S) ctxt_id = Some ctxt" + "list_map_get (canister_version S) recv = Some idx" + "list_map_get (global_timer S) recv = Some timer" using pre by (auto simp: message_execution_pre_def split: message.splits option.splits) define Mod where "Mod = can_state_rec.module can" define Is_response where "Is_response = (case ep of Callback _ _ _ \ True | _ \ False)" - define Env :: "'b env" where "Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\" + define Env :: "'b env" where "Env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status, canister_version = idx\" define Available where "Available = call_ctxt_available_cycles ctxt" define F where "F = exec_function ep Env Available Mod" define R where "R = F (wasm_state can)" define cyc_used where "cyc_used = (case R of Inr res \ update_return.cycles_used res | Inl trap \ cycles_return.cycles_used trap)" - obtain cycles_accepted_res new_calls_res where res: "(cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, new_calls res))" - by (cases "(case R of Inr res \ (update_return.cycles_accepted res, new_calls res))") auto + obtain cycles_accepted_res new_calls_res where res: "(cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, update_return.new_calls res))" + by (cases "(case R of Inr res \ (update_return.cycles_accepted res, update_return.new_calls res))") auto define New_balance where "New_balance = bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - (cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res))" define no_response where "no_response = (case R of Inr result \ update_return.response result = None)" @@ -1068,13 +1134,15 @@ proof - define new_ctxt where "new_ctxt = (case update_return.response result of None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt | Some _ \ call_ctxt_respond ctxt)" - define certified_data where "certified_data = (case new_certified_data result of None \ ic.certified_data S + define certified_data where "certified_data = (case update_return.new_certified_data result of None \ ic.certified_data S | Some cd \ list_map_set (ic.certified_data S) recv cd)" + define global_timer where "global_timer = (case update_return.new_global_timer result of None \ ic.global_timer S + | Some new_timer \ list_map_set (ic.global_timer S) recv new_timer)" define S' where "S' = S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), messages := messages, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, - certified_data := certified_data, balances := list_map_set (balances S) recv New_balance\" + certified_data := certified_data, global_timer := global_timer, balances := list_map_set (balances S) recv New_balance\" have cycles_accepted_res_def: "cycles_accepted_res = update_return.cycles_accepted result" - and new_calls_res_def: "new_calls_res = new_calls result" + and new_calls_res_def: "new_calls_res = update_return.new_calls result" using res by (auto simp: R_Inr) have no_response: "no_response = (update_return.response result = None)" @@ -1085,7 +1153,7 @@ proof - by (simp_all add: message_execution_post_def message_execution_burned_cycles_def Let_def msg prod Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] cyc_used_def[symmetric] res[symmetric] New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric] - messages_def[symmetric] new_ctxt_def[symmetric] certified_data_def[symmetric] S'_def[symmetric] + messages_def[symmetric] new_ctxt_def[symmetric] certified_data_def[symmetric] global_timer_def[symmetric] S'_def[symmetric] result_def[symmetric] response_messages_def[symmetric] new_call_to_message_def[symmetric] del: min_less_iff_conj split del: if_split) have "message_cycles \ new_call_to_message = (\c. MAX_CYCLES_PER_RESPONSE + transferred_cycles c)" for c :: "(?'p, 'canid, 's, 'b, 'c) method_call" @@ -1114,14 +1182,16 @@ qed lemma message_execution_cases: assumes "message_execution_pre n S" - "\ctxt_id recv ep q can bal can_status t ctxt Mod Is_response Env Available F R cyc_used cycles_accepted_res new_calls_res New_balance no_response result new_call_to_message response_messages msgs new_ctxt cert_data idx. + "\ctxt_id recv ep q can bal can_status t ctxt Mod Is_response Env Available F R cyc_used cycles_accepted_res new_calls_res New_balance no_response result new_call_to_message response_messages msgs new_ctxt cert_data idx timer glob_timer. messages S ! n = Func_message ctxt_id recv ep q \ list_map_get (canisters S) recv = Some (Some can) \ list_map_get (balances S) recv = Some bal \ list_map_get (canister_status S) recv = Some can_status \ list_map_get (time S) recv = Some t \ list_map_get (call_contexts S) ctxt_id = Some ctxt \ + list_map_get (canister_version S) recv = Some idx \ + list_map_get (global_timer S) recv = Some timer \ Mod = module can \ Is_response = (case ep of Callback _ _ _ \ True | _ \ False) \ - Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\ \ + Env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status, canister_version = idx\ \ Available = call_ctxt_available_cycles ctxt \ F = exec_function ep Env Available Mod \ R = F (wasm_state can) \ @@ -1145,19 +1215,23 @@ lemma message_execution_cases: new_ctxt = (case update_return.response result of None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt | Some _ \ call_ctxt_respond ctxt) \ - cert_data = (case new_certified_data result of None \ certified_data S + cert_data = (case update_return.new_certified_data result of None \ certified_data S | Some cd \ list_map_set (certified_data S) recv cd) \ + glob_timer = (case update_return.new_global_timer result of None \ global_timer S + | Some new_timer \ list_map_set (global_timer S) recv new_timer) \ P n S (S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), messages := msgs, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, - certified_data := cert_data, balances := list_map_set (balances S) recv New_balance\)" - "\ctxt_id recv ep q can bal can_status t ctxt Mod Is_response Env Available F R cyc_used cycles_accepted_res new_calls_res New_balance no_response idx. + certified_data := cert_data, global_timer := glob_timer, balances := list_map_set (balances S) recv New_balance\)" + "\ctxt_id recv ep q can bal can_status t ctxt Mod Is_response Env Available F R cyc_used cycles_accepted_res new_calls_res New_balance no_response idx timer glob_timer. messages S ! n = Func_message ctxt_id recv ep q \ list_map_get (canisters S) recv = Some (Some can) \ list_map_get (balances S) recv = Some bal \ list_map_get (canister_status S) recv = Some can_status \ list_map_get (time S) recv = Some t \ list_map_get (call_contexts S) ctxt_id = Some ctxt \ + list_map_get (canister_version S) recv = Some idx \ + list_map_get (global_timer S) recv = Some timer \ Mod = module can \ Is_response = (case ep of Callback _ _ _ \ True | _ \ False) \ - Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\ \ + Env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status, canister_version = idx\ \ Available = call_ctxt_available_cycles ctxt \ F = exec_function ep Env Available Mod \ R = F (wasm_state can) \ @@ -1177,23 +1251,25 @@ lemma message_execution_cases: - min cyc_used (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE))\)" shows "P n S (message_execution_post n S)" proof - - obtain ctxt_id recv ep q can bal can_status t ctxt where msg: "messages S ! n = Func_message ctxt_id recv ep q" + obtain ctxt_id recv ep q can bal can_status t ctxt idx timer where msg: "messages S ! n = Func_message ctxt_id recv ep q" and prod: "list_map_get (canisters S) recv = Some (Some can)" "list_map_get (balances S) recv = Some bal" "list_map_get (canister_status S) recv = Some can_status" "list_map_get (time S) recv = Some t" "list_map_get (call_contexts S) ctxt_id = Some ctxt" + "list_map_get (canister_version S) recv = Some idx" + "list_map_get (global_timer S) recv = Some timer" using assms(1) by (auto simp: message_execution_pre_def split: message.splits option.splits) define Mod where "Mod = can_state_rec.module can" define Is_response where "Is_response = (case ep of Callback _ _ _ \ True | _ \ False)" - define Env :: "'b env" where "Env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status\" + define Env :: "'b env" where "Env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S recv, certificate = None, status = simple_status can_status, canister_version = idx\" define Available where "Available = call_ctxt_available_cycles ctxt" define F where "F = exec_function ep Env Available Mod" define R where "R = F (wasm_state can)" define cyc_used where "cyc_used = (case R of Inr res \ update_return.cycles_used res | Inl trap \ cycles_return.cycles_used trap)" - obtain cycles_accepted_res new_calls_res where res: "(cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, new_calls res))" - by (cases "(case R of Inr res \ (update_return.cycles_accepted res, new_calls res))") auto + obtain cycles_accepted_res new_calls_res where res: "(cycles_accepted_res, new_calls_res) = (case R of Inr res \ (update_return.cycles_accepted res, update_return.new_calls res))" + by (cases "(case R of Inr res \ (update_return.cycles_accepted res, update_return.new_calls res))") auto define New_balance where "New_balance = bal + cycles_accepted_res + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) - (cyc_used + sum_list (map (\x. MAX_CYCLES_PER_RESPONSE + transferred_cycles x) new_calls_res))" define no_response where "no_response = (case R of Inr result \ update_return.response result = None)" @@ -1230,21 +1306,23 @@ lemma message_execution_cases: define new_ctxt where "new_ctxt = (case update_return.response result of None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt | Some _ \ call_ctxt_respond ctxt)" - define certified_data where "certified_data = (case new_certified_data result of None \ ic.certified_data S + define certified_data where "certified_data = (case update_return.new_certified_data result of None \ ic.certified_data S | Some cd \ list_map_set (ic.certified_data S) recv cd)" + define global_timer where "global_timer = (case update_return.new_global_timer result of None \ ic.global_timer S + | Some new_timer \ list_map_set (ic.global_timer S) recv new_timer)" define S' where "S' = S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), messages := messages, call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, - certified_data := certified_data, balances := list_map_set (balances S) recv New_balance\" + certified_data := certified_data, global_timer := global_timer, balances := list_map_set (balances S) recv New_balance\" have msg_exec: "message_execution_post n S = S'" using True by (simp_all add: message_execution_post_def Let_def msg prod Mod_def[symmetric] Is_response_def[symmetric] Env_def[symmetric] Available_def[symmetric] F_def[symmetric] R_def[symmetric] cyc_used_def[symmetric] res[symmetric] New_balance_def[symmetric] no_response_def[symmetric] S''_def[symmetric] cond_def[symmetric] - messages_def[symmetric] new_ctxt_def[symmetric] certified_data_def[symmetric] S'_def[symmetric] + messages_def[symmetric] new_ctxt_def[symmetric] certified_data_def[symmetric] global_timer_def[symmetric] S'_def[symmetric] result_def[symmetric] response_messages_def[symmetric] new_call_to_message_def[symmetric]) show ?thesis using assms(2)[OF msg prod Mod_def Is_response_def Env_def Available_def F_def R_def cyc_used_def res New_balance_def no_response_def _ _ _ _ _ _ result_def - new_call_to_message_def response_messages_def messages_def new_ctxt_def certified_data_def, folded S'_def] True + new_call_to_message_def response_messages_def messages_def new_ctxt_def certified_data_def global_timer_def, folded S'_def] True by (auto simp: cond_def msg_exec) qed qed @@ -1253,7 +1331,7 @@ lemma message_execution_ic_inv: assumes "message_execution_pre n S" "ic_inv S" shows "ic_inv (message_execution_post n S)" proof (rule message_execution_cases[OF assms(1)]) - fix recv msgs ctxt_id new_ctxt cert_data New_balance new_calls_res response_messages ctxt Available cycles_accepted_res no_response idx + fix recv msgs ctxt_id new_ctxt cert_data New_balance new_calls_res response_messages ctxt Available cycles_accepted_res no_response idx glob_timer fix can :: "('p, 'canid, 'b, 'w, 'sm, 'c, 's) can_state_rec" fix result :: "('w, 'p, 'canid, 's, 'b, 'c) update_return" fix new_call_to_message :: "('p, 'canid, 's, 'b, 'c) method_call \ ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) message" @@ -1268,7 +1346,7 @@ proof (rule message_execution_cases[OF assms(1)]) "\ isl R" "result = projr R" "new_ctxt = (case update_return.response result of None \ call_ctxt_deduct_cycles cycles_accepted_res ctxt | Some x \ call_ctxt_respond ctxt)" then show "ic_inv (S\canisters := list_map_set (canisters S) recv (Some (can\wasm_state := update_return.new_state result\)), messages := msgs, - call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, certified_data := cert_data, balances := list_map_set (balances S) recv New_balance\)" + call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, certified_data := cert_data, global_timer := glob_timer, balances := list_map_set (balances S) recv New_balance\)" using assms(2) list_map_get_range[OF ctxt] by (cases R) (force simp: ic_inv_def ic_can_status_inv_def split: message.splits call_origin.splits can_status.splits dest!: in_set_takeD in_set_dropD list_map_range_setD)+ @@ -1289,7 +1367,7 @@ definition call_context_starvation_pre :: "'cid \ ('p, 'uid, 'canid, "call_context_starvation_pre ctxt_id S = (case list_map_get (call_contexts S) ctxt_id of Some call_context \ call_ctxt_needs_to_respond call_context \ - call_ctxt_origin call_context \ From_heartbeat \ + call_ctxt_origin call_context \ From_system \ (\msg \ set (messages S). case msg of Call_message orig _ _ _ _ _ _ \ calling_context orig \ Some ctxt_id | Response_message orig _ _ \ calling_context orig \ Some ctxt_id @@ -1331,7 +1409,7 @@ definition call_context_removal_pre :: "'cid \ ('p, 'uid, 'canid, 'b "call_context_removal_pre ctxt_id S = ( (case list_map_get (call_contexts S) ctxt_id of Some call_context \ (\call_ctxt_needs_to_respond call_context \ - (call_ctxt_origin call_context = From_heartbeat \ + (call_ctxt_origin call_context = From_system \ (\msg \ set (messages S). case msg of Func_message other_ctxt_id _ _ _ \ other_ctxt_id \ ctxt_id | _ \ True))) \ @@ -1383,10 +1461,12 @@ definition ic_canister_creation_pre :: "nat \ 'canid \ n is_system_assigned (principal_of_canid cid) \ cid \ list_map_dom (canisters S) \ cid \ list_map_dom (time S) \ + cid \ list_map_dom (global_timer S) \ cid \ list_map_dom (controllers S) \ cid \ list_map_dom (balances S) \ cid \ list_map_dom (certified_data S) \ - cid \ list_map_dom (canister_status S) + cid \ list_map_dom (canister_status S) \ + cid \ list_map_dom (canister_version S) | _ \ False))" definition ic_canister_creation_post :: "nat \ 'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where @@ -1394,13 +1474,15 @@ definition ic_canister_creation_post :: "nat \ 'canid \ let ctrls = (case candid_parse_controllers a of Some ctrls \ ctrls | _ \ {cer}) in S\canisters := list_map_set (canisters S) cid None, time := list_map_set (time S) cid t, + global_timer := list_map_set (global_timer S) cid 0, controllers := list_map_set (controllers S) cid ctrls, freezing_threshold := list_map_set (freezing_threshold S) cid 2592000, balances := list_map_set (balances S) cid trans_cycles, certified_data := list_map_set (certified_data S) cid empty_blob, messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid (Candid_record (list_map_init [(encode_string ''canister_id'', Candid_blob (blob_of_canid cid))])))) 0], - canister_status := list_map_set (canister_status S) cid Running\)" + canister_status := list_map_set (canister_status S) cid Running, + canister_version := list_map_set (canister_version S) cid 0\)" lemma ic_canister_creation_cycles_inv: assumes "ic_canister_creation_pre n cid t S" @@ -1442,7 +1524,7 @@ definition ic_update_settings_pre :: "nat \ ('p, 'uid, 'canid, 'b, ' cee = ic_principal \ mn = encode_string ''update_settings'' \ (case candid_parse_cid a of Some cid \ - (case list_map_get (controllers S) cid of Some ctrls \ cer \ ctrls + (case (list_map_get (controllers S) cid, list_map_get (canister_version S) cid) of (Some ctrls, Some idx) \ cer \ ctrls | _ \ False) | _ \ False) | _ \ False))" @@ -1451,8 +1533,10 @@ definition ic_update_settings_post :: "nat \ ('p, 'uid, 'canid, 'b, let cid = the (candid_parse_cid a); ctrls = (case candid_parse_controllers a of Some ctrls \ list_map_set (controllers S) cid ctrls | _ \ controllers S); freezing_thres = (case candid_nested_lookup a [encode_string '''settings'', encode_string ''freezing_threshold''] of Some (Candid_nat freeze) \ - list_map_set (freezing_threshold S) cid freeze | _ \ freezing_threshold S) in - S\controllers := ctrls, freezing_threshold := freezing_thres, messages := take n (messages S) @ drop (Suc n) (messages S) @ + list_map_set (freezing_threshold S) cid freeze | _ \ freezing_threshold S); + idx = the (list_map_get (canister_version S) cid) in + S\controllers := ctrls, freezing_threshold := freezing_thres, canister_version := list_map_set (canister_version S) cid (Suc idx), + messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)" lemma ic_update_settings_cycles_inv: @@ -1555,13 +1639,13 @@ definition ic_code_installation_pre :: "nat \ ('p, 'uid, 'canid, 'b, (case parse_wasm_mod w of Some m \ parse_public_custom_sections w \ None \ parse_private_custom_sections w \ None \ - (case (list_map_get (controllers S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of - (Some ctrls, Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + (case (list_map_get (controllers S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid, list_map_get (global_timer S) cid) of + (Some ctrls, Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\ in ((mode = encode_string ''install'' \ (case list_map_get (canisters S) cid of Some None \ True | _ \ False)) \ mode = encode_string ''reinstall'') \ cer \ ctrls \ (case canister_module_init m (cid, ar, cer, env) of Inl _ \ False - | Inr ret \ cycles_return.cycles_used ret \ bal) \ + | Inr ret \ init_return.cycles_used ret \ bal) \ list_map_dom (canister_module_update_methods m) \ list_map_dom (canister_module_query_methods m) = {} | _ \ False) | _ \ False) | _ \ False) | _ \ False) | _ \ False))" @@ -1572,12 +1656,16 @@ definition ic_code_installation_post :: "nat \ ('p, 'uid, 'canid, 'b (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of (Some mode, Some w, Some a) \ (case parse_wasm_mod w of Some m \ - (case (list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of - (Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\; - (new_state, cyc_used) = (case canister_module_init m (cid, a, cer, env) of Inr ret \ (cycles_return.return ret, cycles_return.cycles_used ret)) in + (case (list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid, list_map_get (global_timer S) cid) of + (Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\; + (new_state, new_certified_data, new_global_timer, cyc_used) = (case canister_module_init m (cid, a, cer, env) of Inr ret \ + (init_return.new_state ret, init_return.new_certified_data ret, init_return.new_global_timer ret, init_return.cycles_used ret)) in S\canisters := list_map_set (canisters S) cid (Some \wasm_state = new_state, module = m, raw_module = w, public_custom_sections = the (parse_public_custom_sections w), private_custom_sections = the (parse_private_custom_sections w)\), + certified_data := (case new_certified_data of None \ certified_data S | Some cd \ list_map_set (certified_data S) cid cd), + global_timer := list_map_set (global_timer S) cid (case new_global_timer of None \ 0 | Some new_timer \ new_timer), + canister_version := list_map_set (canister_version S) cid (Suc idx), balances := list_map_set (balances S) cid (bal - cyc_used), messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)))))" @@ -1587,10 +1675,10 @@ definition ic_code_installation_burned_cycles :: "nat \ ('p, 'uid, ' (case (candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of (Some w, Some a) \ (case parse_wasm_mod w of Some m \ - (case (list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of - (Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in - (case canister_module_init m (cid, a, cer, env) of Inr ret \ cycles_return.cycles_used ret))))))" + (case (list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid, list_map_get (global_timer S) cid) of + (Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\ in + (case canister_module_init m (cid, a, cer, env) of Inr ret \ init_return.cycles_used ret))))))" lemma ic_code_installation_cycles_inv: assumes "ic_code_installation_pre n S" @@ -1615,7 +1703,7 @@ lemma ic_code_installation_ic_inv: assumes "ic_code_installation_pre n S" "ic_inv S" shows "ic_inv (ic_code_installation_post n S)" proof - - obtain orig cer cee mn a trans_cycles q cid mode w ar m ctrls t bal can_status idx where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + obtain orig cer cee mn a trans_cycles q cid mode w ar m ctrls t bal can_status idx timer where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" and parse: "candid_parse_cid a = Some cid" "candid_parse_text a [encode_string ''mode''] = Some mode" "candid_parse_blob a [encode_string ''wasm_module''] = Some w" @@ -1626,6 +1714,8 @@ proof - "list_map_get (time S) cid = Some t" "list_map_get (balances S) cid = Some bal" "list_map_get (canister_status S) cid = Some can_status" + "list_map_get (canister_version S) cid = Some idx" + "list_map_get (global_timer S) cid = Some timer" using assms by (auto simp: ic_code_installation_pre_def split: message.splits option.splits) show ?thesis @@ -1650,14 +1740,15 @@ definition ic_code_upgrade_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, (case parse_wasm_mod w of Some m \ parse_public_custom_sections w \ None \ parse_private_custom_sections w \ None \ - (case (list_map_get (canisters S) cid, list_map_get (controllers S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of - (Some (Some can), Some ctrls, Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in + (case (list_map_get (canisters S) cid, list_map_get (controllers S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid, list_map_get (global_timer S) cid) of + (Some (Some can), Some ctrls, Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env1 = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = idx\; + env2 = \env.time = t, env.global_timer = 0, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\ in mode = encode_string ''upgrade'' \ cer \ ctrls \ - (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env) of Inr pre_ret \ - (case canister_module_post_upgrade m (cid, cycles_return.return pre_ret, ar, cer, env) of Inr post_ret \ - cycles_return.cycles_used pre_ret + cycles_return.cycles_used post_ret \ bal + (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env1) of Inr pre_ret \ + (case canister_module_post_upgrade m (cid, pre_upgrade_return.stable_memory pre_ret, ar, cer, env2) of Inr post_ret \ + pre_upgrade_return.cycles_used pre_ret + init_return.cycles_used post_ret \ bal | _ \ False) | _ \ False) \ list_map_dom (canister_module_update_methods m) \ list_map_dom (canister_module_query_methods m) = {} | _ \ False) | _ \ False) | _ \ False) | _ \ False) @@ -1669,14 +1760,20 @@ definition ic_code_upgrade_post :: "nat \ ('p, 'uid, 'canid, 'b, 'w, (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of (Some mode, Some w, Some ar) \ (case parse_wasm_mod w of Some m \ - (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of - (Some (Some can), Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in - (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env) of Inr pre_ret \ - (case canister_module_post_upgrade m (cid, cycles_return.return pre_ret, ar, cer, env) of Inr post_ret \ - S\canisters := list_map_set (canisters S) cid (Some \wasm_state = cycles_return.return post_ret, module = m, raw_module = w, + (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid, list_map_get (global_timer S) cid) of + (Some (Some can), Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env1 = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = idx\; + env2 = \env.time = t, env.global_timer = 0, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\ in + (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env1) of Inr pre_ret \ + (case canister_module_post_upgrade m (cid, pre_upgrade_return.stable_memory pre_ret, ar, cer, env2) of Inr post_ret \ + S\canisters := list_map_set (canisters S) cid (Some \wasm_state = init_return.new_state post_ret, module = m, raw_module = w, public_custom_sections = the (parse_public_custom_sections w), private_custom_sections = the (parse_private_custom_sections w)\), - balances := list_map_set (balances S) cid (bal - (cycles_return.cycles_used pre_ret + cycles_return.cycles_used post_ret)), + certified_data := (case init_return.new_certified_data post_ret of Some cd' \ list_map_set (certified_data S) cid cd' + | None \ (case pre_upgrade_return.new_certified_data pre_ret of Some cd \ list_map_set (certified_data S) cid cd + | None \ certified_data S)), + global_timer := list_map_set (global_timer S) cid (case init_return.new_global_timer post_ret of None \ 0 | Some new_timer \ new_timer), + canister_version := list_map_set (canister_version S) cid (Suc idx), + balances := list_map_set (balances S) cid (bal - (pre_upgrade_return.cycles_used pre_ret + init_return.cycles_used post_ret)), messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)))))))" definition ic_code_upgrade_burned_cycles :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where @@ -1685,12 +1782,13 @@ definition ic_code_upgrade_burned_cycles :: "nat \ ('p, 'uid, 'canid (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of (Some mode, Some w, Some ar) \ (case parse_wasm_mod w of Some m \ - (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid) of - (Some (Some can), Some t, Some bal, Some can_status) \ - let env = \env.time = t, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status\ in - (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env) of Inr pre_ret \ - (case canister_module_post_upgrade m (cid, cycles_return.return pre_ret, ar, cer, env) of Inr post_ret \ - cycles_return.cycles_used pre_ret + cycles_return.cycles_used post_ret)))))))" + (case (list_map_get (canisters S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid, list_map_get (global_timer S) cid) of + (Some (Some can), Some t, Some bal, Some can_status, Some idx, Some timer) \ + let env1 = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = idx\; + env2 = \env.time = t, env.global_timer = 0, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\ in + (case canister_module_pre_upgrade (module can) (wasm_state can, cer, env1) of Inr pre_ret \ + (case canister_module_post_upgrade m (cid, pre_upgrade_return.stable_memory pre_ret, ar, cer, env2) of Inr post_ret \ + pre_upgrade_return.cycles_used pre_ret + init_return.cycles_used post_ret)))))))" lemma ic_code_upgrade_cycles_inv: assumes "ic_code_upgrade_pre n S" @@ -1715,7 +1813,7 @@ lemma ic_code_upgrade_ic_inv: assumes "ic_code_upgrade_pre n S" "ic_inv S" shows "ic_inv (ic_code_upgrade_post n S)" proof - - obtain orig cer cee mn a trans_cycles q cid mode w ar m can ctrls t bal can_status idx where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + obtain orig cer cee mn a trans_cycles q cid mode w ar m can ctrls t bal can_status idx timer where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" and parse: "candid_parse_cid a = Some cid" "candid_parse_text a [encode_string ''mode''] = Some mode" "candid_parse_blob a [encode_string ''wasm_module''] = Some w" @@ -1727,6 +1825,8 @@ proof - "list_map_get (time S) cid = Some t" "list_map_get (balances S) cid = Some bal" "list_map_get (canister_status S) cid = Some can_status" + "list_map_get (canister_version S) cid = Some idx" + "list_map_get (global_timer S) cid = Some timer" using assms by (auto simp: ic_code_upgrade_pre_def split: message.splits option.splits) show ?thesis @@ -1746,7 +1846,7 @@ definition ic_code_uninstallation_pre :: "nat \ ('p, 'uid, 'canid, ' cee = ic_principal \ mn = encode_string ''uninstall_code'' \ (case candid_parse_cid a of Some cid \ - (case list_map_get (controllers S) cid of Some ctrls \ cer \ ctrls + (case (list_map_get (controllers S) cid, list_map_get (canister_version S) cid) of (Some ctrls, Some idx) \ cer \ ctrls | _ \ False) | _ \ False) | _ \ False))" @@ -1757,8 +1857,11 @@ definition ic_code_uninstallation_post :: "nat \ ('p, 'uid, 'canid, if call_ctxt_canister ctxt = cid \ call_ctxt_needs_to_respond ctxt then Some (Response_message (call_ctxt_origin ctxt) (response.Reject CANISTER_REJECT (encode_string ''Canister has been uninstalled'')) (call_ctxt_available_cycles ctxt)) else None); - call_ctxt_to_ctxt = (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt) in + call_ctxt_to_ctxt = (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt); + idx = the (list_map_get (canister_version S) cid) in S\canisters := list_map_set (canisters S) cid None, certified_data := list_map_set (certified_data S) cid empty_blob, + canister_version := list_map_set (canister_version S) cid (Suc idx), + global_timer := list_map_set (global_timer S) cid 0, messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles] @ List.map_filter call_ctxt_to_msg (list_map_vals (call_contexts S)), call_contexts := list_map_map call_ctxt_to_ctxt (call_contexts S)\))" @@ -2142,7 +2245,9 @@ definition ic_canister_deletion_post :: "nat \ ('p, 'uid, 'canid, 'b controllers := list_map_del (controllers S) cid, freezing_threshold := list_map_del (freezing_threshold S) cid, canister_status := list_map_del (canister_status S) cid, + canister_version := list_map_del (canister_version S) cid, time := list_map_del (time S) cid, + global_timer := list_map_del (global_timer S) cid, balances := list_map_del (balances S) cid, certified_data := list_map_del (certified_data S) cid, messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid Candid_empty)) trans_cycles]\)" @@ -2292,10 +2397,12 @@ definition ic_provisional_canister_creation_pre :: "nat \ 'canid \ cid \ list_map_dom (canisters S) \ cid \ list_map_dom (time S) \ + cid \ list_map_dom (global_timer S) \ cid \ list_map_dom (controllers S) \ cid \ list_map_dom (balances S) \ cid \ list_map_dom (certified_data S) \ - cid \ list_map_dom (canister_status S) + cid \ list_map_dom (canister_status S) \ + cid \ list_map_dom (canister_version S) | _ \ False) | _ \ False))" definition ic_provisional_canister_creation_post :: "nat \ 'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where @@ -2303,13 +2410,15 @@ definition ic_provisional_canister_creation_post :: "nat \ 'canid \< let cyc = the (candid_parse_nat a [encode_string ''amount'']) in S\canisters := list_map_set (canisters S) cid None, time := list_map_set (time S) cid t, + global_timer := list_map_set (global_timer S) cid 0, controllers := list_map_set (controllers S) cid {cer}, freezing_threshold := list_map_set (freezing_threshold S) cid 2592000, balances := list_map_set (balances S) cid cyc, certified_data := list_map_set (certified_data S) cid empty_blob, messages := take n (messages S) @ drop (Suc n) (messages S) @ [Response_message orig (Reply (blob_of_candid (Candid_record (list_map_init [(encode_string ''canister_id'', Candid_blob (blob_of_canid cid))])))) trans_cycles], - canister_status := list_map_set (canister_status S) cid Running\)" + canister_status := list_map_set (canister_status S) cid Running, + canister_version := list_map_set (canister_version S) cid 0\)" definition ic_provisional_canister_creation_minted_cycles :: "nat \ 'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ nat" where "ic_provisional_canister_creation_minted_cycles n cid t S = (case messages S ! n of Call_message orig cer cee mn a trans_cycles q \ @@ -2602,7 +2711,7 @@ lemma request_cleanup_expired_ic_inv: (* System transition: Canister out of cycles [DONE] *) definition canister_out_of_cycles_pre :: "'canid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where - "canister_out_of_cycles_pre cid S = (case list_map_get (balances S) cid of Some 0 \ True + "canister_out_of_cycles_pre cid S = (case (list_map_get (balances S) cid, list_map_get (canister_version S) cid) of (Some 0, Some idx) \ True | _ \ False)" definition canister_out_of_cycles_post :: "'canid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where @@ -2611,8 +2720,11 @@ definition canister_out_of_cycles_post :: "'canid \ ('p, 'uid, 'cani if call_ctxt_canister ctxt = cid \ call_ctxt_needs_to_respond ctxt then Some (Response_message (call_ctxt_origin ctxt) (response.Reject CANISTER_REJECT (encode_string ''Canister has been uninstalled'')) (call_ctxt_available_cycles ctxt)) else None); - call_ctxt_to_ctxt = (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt) in + call_ctxt_to_ctxt = (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt); + idx = the (list_map_get (canister_version S) cid) in S\canisters := list_map_set (canisters S) cid None, certified_data := list_map_set (certified_data S) cid empty_blob, + canister_version := list_map_set (canister_version S) cid (Suc idx), + global_timer := list_map_set (global_timer S) cid 0, messages := messages S @ List.map_filter call_ctxt_to_msg (list_map_vals (call_contexts S)), call_contexts := list_map_map call_ctxt_to_ctxt (call_contexts S)\)" @@ -2644,7 +2756,7 @@ lemma canister_out_of_cycles_ic_inv: -(* System transition: Time progressing and cycle consumption (canister time) [DONE] *) +(* System transition: Time progressing, cycle consumption, and canister version increments (canister time) [DONE] *) definition canister_time_progress_pre :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "canister_time_progress_pre cid t1 S = (case list_map_get (time S) cid of Some t0 \ @@ -2670,7 +2782,7 @@ lemma canister_time_progress_ic_inv: -(* System transition: Time progressing and cycle consumption (cycle consumption) [DONE] *) +(* System transition: Time progressing, cycle consumption, and canister version increments (cycle consumption) [DONE] *) definition cycle_consumption_pre :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "cycle_consumption_pre cid b1 S = (case list_map_get (balances S) cid of Some b0 \ @@ -2701,7 +2813,7 @@ lemma cycle_consumption_ic_inv: -(* System transition: Time progressing and cycle consumption (system time) [DONE] *) +(* System transition: Time progressing, cycle consumption, and canister version increments (system time) [DONE] *) definition system_time_progress_pre :: "nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "system_time_progress_pre t1 S = (system_time S < t1)" @@ -2725,6 +2837,32 @@ lemma system_time_progress_ic_inv: +(* System transition: Time progressing, cycle consumption, and canister version increments (canister version) [DONE] *) + +definition canister_version_progress_pre :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where + "canister_version_progress_pre cid n1 S = (case list_map_get (canister_version S) cid of Some n0 \ + n0 < n1 + | _ \ False)" + +definition canister_version_progress_post :: "'canid \ nat \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where + "canister_version_progress_post cid n1 S = (S\canister_version := list_map_set (canister_version S) cid n1\)" + +lemma canister_version_progress_cycles_inv: + assumes "canister_version_progress_pre cid n1 S" + shows "total_cycles S = total_cycles (canister_version_progress_post cid n1 S)" + by (auto simp: canister_version_progress_post_def total_cycles_def) + +lemma canister_version_progress_ic_inv: + assumes "canister_version_progress_pre cid n1 S" "ic_inv S" + shows "ic_inv (canister_version_progress_post cid n1 S)" + using assms + by (auto simp: ic_inv_def canister_version_progress_pre_def canister_version_progress_post_def Let_def + split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits + dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del + in_set_map_filter_vals) + + + (* State machine *) inductive ic_steps :: "'sig itself \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ @@ -2765,6 +2903,7 @@ inductive ic_steps :: "'sig itself \ ('p, 'uid, 'canid, 'b, 'w, 'sm, | canister_time_progress: "ic_steps sig S0 minted burned S \ canister_time_progress_pre cid t1 S \ ic_steps sig S0 minted burned (canister_time_progress_post cid t1 S)" | cycle_consumption: "ic_steps sig S0 minted burned S \ cycle_consumption_pre cid b1 S \ ic_steps sig S0 minted (burned + cycle_consumption_burned_cycles cid b1 S) (cycle_consumption_post cid b1 S)" | system_time_progress: "ic_steps sig S0 minted burned S \ system_time_progress_pre t1 S \ ic_steps sig S0 minted burned (system_time_progress_post t1 S)" +| canister_version_progress: "ic_steps sig S0 minted burned S \ canister_version_progress_pre cid n1 S \ ic_steps sig S0 minted burned (canister_version_progress_post cid n1 S)" lemma total_cycles: assumes "ic_steps TYPE('sig) S0 minted burned S" @@ -2807,6 +2946,7 @@ lemma total_cycles: using canister_time_progress_cycles_inv apply fastforce using cycle_consumption_cycles_monotonic apply fastforce using system_time_progress_cycles_inv apply fastforce + using canister_version_progress_cycles_inv apply fastforce done lemma ic_inv: @@ -2850,6 +2990,7 @@ lemma ic_inv: using canister_time_progress_ic_inv apply fastforce using cycle_consumption_ic_inv apply fastforce using system_time_progress_ic_inv apply fastforce + using canister_version_progress_ic_inv apply fastforce done end @@ -2889,6 +3030,7 @@ export_code request_submission_pre request_submission_post canister_time_progress_pre canister_time_progress_post cycle_consumption_pre cycle_consumption_post system_time_progress_pre system_time_progress_post + canister_version_progress_pre canister_version_progress_post in Haskell module_name IC file_prefix code end From a082f073fdbfdac27aafeb1679b3e60175172bd0 Mon Sep 17 00:00:00 2001 From: Bjoern Tackmann Date: Mon, 19 Dec 2022 14:16:32 +0100 Subject: [PATCH 050/102] Fix typo in changelog --- spec/md-spec/interface-spec-changelog.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/md-spec/interface-spec-changelog.md b/spec/md-spec/interface-spec-changelog.md index 200d54e8e..6d3c1dfb6 100644 --- a/spec/md-spec/interface-spec-changelog.md +++ b/spec/md-spec/interface-spec-changelog.md @@ -1,6 +1,6 @@ ## Changelog {#changelog} -## 0_18_9 (2022-11-06) {#0_18_9} +## 0.18.9 (2022-11-06) {#0_18_9} * Global timers * Canister version * Clarifications for HTTP requests & Bitcoin integration costs From c3ff773d694cc7ab11d2d2d40f489fe2974c2e40 Mon Sep 17 00:00:00 2001 From: ais <97464093+ais-dfn@users.noreply.github.com> Date: Thu, 9 Mar 2023 13:42:15 +0100 Subject: [PATCH 051/102] fixing broken link to docs after reshuffle --- spec/md-spec/ic-interface-spec.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/md-spec/ic-interface-spec.md b/spec/md-spec/ic-interface-spec.md index 25fe0656d..a13d2e220 100644 --- a/spec/md-spec/ic-interface-spec.md +++ b/spec/md-spec/ic-interface-spec.md @@ -11,7 +11,7 @@ Welcome to *the Internet Computer*! We speak of "the" Internet Computer, because This document describes this *external* view of the Internet Computer, i.e. the low-level interfaces it provides to dapp developers and users, and what will happen when they use these interfaces. :::note -While this document describes the external interface and behavior of the Internet Computer, it is not intended as end-user or end-developer documentation. Most developers will interact with the Internet Computer through additional tooling like the SDK, Canister Development Kits and Motoko. Please see the [developer docs](../developer-docs/ic-overview.mdx) for suitable documentation. +While this document describes the external interface and behavior of the Internet Computer, it is not intended as end-user or end-developer documentation. Most developers will interact with the Internet Computer through additional tooling like the SDK, Canister Development Kits and Motoko. Please see the [developer docs](../home.mdx) for suitable documentation. ::: From 987f3fab329d43b6d003444fcbedefefbdab619b Mon Sep 17 00:00:00 2001 From: Severin Siffert Date: Mon, 3 Apr 2023 11:33:21 +0200 Subject: [PATCH 052/102] fix markup --- spec/md-spec/ic-interface-spec.md | 111 ++++++++++++++++++++++++++++-- 1 file changed, 105 insertions(+), 6 deletions(-) diff --git a/spec/md-spec/ic-interface-spec.md b/spec/md-spec/ic-interface-spec.md index a13d2e220..22999a2b8 100644 --- a/spec/md-spec/ic-interface-spec.md +++ b/spec/md-spec/ic-interface-spec.md @@ -11,6 +11,7 @@ Welcome to *the Internet Computer*! We speak of "the" Internet Computer, because This document describes this *external* view of the Internet Computer, i.e. the low-level interfaces it provides to dapp developers and users, and what will happen when they use these interfaces. :::note + While this document describes the external interface and behavior of the Internet Computer, it is not intended as end-user or end-developer documentation. Most developers will interact with the Internet Computer through additional tooling like the SDK, Canister Development Kits and Motoko. Please see the [developer docs](../home.mdx) for suitable documentation. ::: @@ -24,7 +25,9 @@ The target audience of this document are - those who want to understand the intricacies of the Internet Computer's behavior in great detail (e.g. to do a security analysis) :::note + This document is a rigorous, technically dense reference. It is not an introduction to the Internet Computer, and as such most useful to those who understand the high-level concepts. Please see more high-level documentation first. + ::: ### Scope of this document {#scope_of_this_document} @@ -119,7 +122,9 @@ There are several classes of ids: These are always generated by the IC and have no structure of interest outside of it. :::note + Typically, these end with the byte `0x01`, but users of the IC should not need to care about that. + ::: 2. *Self-authenticating ids*. @@ -135,7 +140,9 @@ There are several classes of ids: These ids are treated specially when an id needs to be registered. In such a request, whoever requests an id can provide a `derivation_nonce`. By hashing that together with the principal of the caller, every principal has a space of ids that only they can register ids from. :::note + Derived IDs are currently not explicitly used in this document, but they may be used internally or in the future. + ::: 4. *Anonymous id* @@ -168,6 +175,7 @@ The textual representation is conventionally printed with *lower case letters*, Because the maximum size of a principal is 29 bytes, the textual representation will be no longer than 63 characters (10 times 5 plus 3 characters with 10 separators in between them). :::tip + The canister with id `0xABCD01` has check sequence `0x233FF206` ([online calculator](https://crccalc.com/?crc=ABCD01&method=crc32&datatype=hex&outtype=hex)); the final id is thus `em77e-bvlzu-aq`. Example encoding from hex, and decoding to hex, in bash (the following can be pasted into a terminal as is): @@ -183,6 +191,7 @@ Example encoding from hex, and decoding to hex, in bash (the following can be pa fold -w 8 | xargs -n1 printf '%-8s' | tr ' ' = | base32 -d | xxd -p | tr -d '\n' | cut -b9- | tr a-z A-Z } + ::: ### Canister lifecycle @@ -224,7 +233,9 @@ When the cycle balance of a canister falls to zero, the canister is *deallocated Afterwards the canister is empty. It can be reinstalled after topping up its balance. :::note + Once the IC frees the resources of a canister, its id, *cycles* balance, and *controllers* are preserved on the IC for a minimum of 10 years. What happens to the canister after this period is currently unspecified. + ::: #### Canister status @@ -243,7 +254,9 @@ The controllers of the canister can initiate transitions between these states us :::note + This status is orthogonal to whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected by the IC with an HTTP error, but because the canister is empty. + ::: ### Signatures @@ -287,6 +300,7 @@ The signature is checked by verifying that the `challenge` field contains the [b It uses the `SubjectPublicKeyInfo` type used for other types of public keys (see, e.g., [RFC 8410, Section 4](https://tools.ietf.org/html/rfc8410#section-4)), with OID 1.3.6.1.4.1.56387.1.1 (iso.org.dod.internet.private.enterprise.dfinity.mechanisms.der-wrapped-cose). The `BIT STRING` field `subjectPublicKey` contains the COSE encoding. See [WebAuthn w3c recommendation](https://www.w3.org/TR/webauthn/#sctn-encoded-credPubKey-examples) or [RFC 8152](https://tools.ietf.org/html/rfc8152#section-13.1) for details on the COSE encoding. :::tip + A DER wrapping of a COSE key is shown below. It can be parsed via the command `sed "s/#.*//" | xxd -r -p | openssl asn1parse -inform der`. 30 5E # SEQUENCE of length 94 bytes @@ -300,6 +314,7 @@ The signature is checked by verifying that the `challenge` field contains the [b 4E52 5A6A 368C 2363 063D 04E6 ED You can also view the wrapping in [an online ASN.1 JavaScript decoder](https://lapo.it/asn1js/#MF4wDAYKKwYBBAGDuEMBAQNOAKUBAgMmIAEhWCB__YNjIHL9G_6vP7qkMUbg75XD9V45lKQbvytRdNdx2iJYIDJJfu0Kf28ACSh2W4MYFiz9gKlOUlpqNowjYwY9BObt). + ::: - The signature is a CBOR value consisting of a data item with major type 6 ("Semantic tag") and tag value `55799` (see [Self-Describe CBOR](https://tools.ietf.org/html/rfc7049#section-2.4.5)), followed by a map with three mandatory fields: @@ -378,7 +393,9 @@ The state tree contains information about the topology of the Internet Computer. tagged = #6.55799(t) ; the CBOR tag :::note + Because this uses the lexicographic ordering of princpials, and the byte distinguishing the various classes of ids is at the *end*, this range by construction conceptually includes principals of various classes. This specification needs to take care that the fact that principals that are not canisters may appear in these ranges does not cause confusion. + ::: ### Request status {#state-tree-request-status} @@ -406,11 +423,15 @@ For each asynchronous request known to the Internet Computer, its status is in a If the status is `rejected`, then this path might be present and contain an implementation-specific error code (see [Error codes](#error-codes)), else it is not present. :::note + Immediately after submitting a request, the request may not show up yet as the Internet Computer is still working on accepting the request as pending. + ::: :::note + Request statuses will not actually be kept around indefinitely, and eventually the Internet Computer forgets about the request. This will happen no sooner than the request's expiry time, so that replay attacks are prevented. + ::: ### Certified data {#state-tree-certified-data} @@ -454,7 +475,9 @@ In these paths, the `` is the [textual representation](#t Requests to `/api/v2/canister//call`, `/api/v2/canister//read_state` and `/api/v2/canister//query` are POST requests with a CBOR-encoded request body, which consists of a authentication envelope (as per [Authentication](#authentication)) and request-specific content as described below. :::note + This document does not yet explain how to find the location and port of the Internet Computer. + ::: ### Overview of canister calling {#http-call-overview} @@ -538,7 +561,9 @@ The HTTP response to this request has an empty body and HTTP status 202, or a HT This request type can *also* be used to call a query method. A user may choose to go this way, instead of via the faster and cheaper [Request: Query call](#http-query) below, if they want to get a *certified* response. :::note + The functionality exposed via the [The IC management canister](#ic-management-canister) can be used this way. + ::: ### Request: Read state {#http-read-state} @@ -592,7 +617,9 @@ If a path cannot be requested, then the HTTP response to the read state request Read state requests containing many requested paths might be rejected with a 4xx HTTP status code. :::note + The Internet Computer blockchain mainnet might reject read state requests that request more than 1 path with prefix `/request_status/`. + ::: Note that the paths `/canisters//certified_data` are not accessible with this method; these paths are only exposed to the canisters themselves via the System API (see [Certified data](#system-api-certified-data)). @@ -647,7 +674,8 @@ The `` in the URL paths of requests is the *effective* de - If the request is an update call to a canister that is not the Management Canister (`aaaaa-aa`) or if the request is a query call, then the effective canister id must be the `canister_id` in the request. - ::: note + :::note + The expectation is that user-side agent code shields users and developers from the notion of effective canister ID, in analogy to how the System API interface shields canister developers from worrying about routing. The Internet Computer blockchain mainnet rejects all requests whose effective canister id is in no subnet's canister ranges, independently of whether the remaining conditions on the effective canister id are satisfied. @@ -655,6 +683,7 @@ The `` in the URL paths of requests is the *effective* de The Internet Computer blockchain mainnet does not support `provisional_create_canister_with_cycles` and thus all calls to this method are rejected independently of the effective canister id. In development instances of the Internet Computer Protocol (e.g. testnets), the effective canister id of a request submitted to a node must be a canister id from the canister ranges of the subnet to which the node belongs. + ::: ### Authentication @@ -736,14 +765,19 @@ The following encodings of field values as blobs are used When signing requests or querying the status of a request (see [Request status](#state-tree-request-status)) in the state tree, the user identifies the request using a *request id*, which is the [representation-independent hash](#hash-of-map) of the `content` map of the original request. A request id must have length of 32 bytes. :::note + The request id is independent of the representation of the request (currently only CBOR), and does not change if the specification adds further optional fields to a request type. + ::: :::note + The recommended textual representation of a request id is a hexadecimal string with lower-case letters prefixed with \'0x\'. E.g., request id consisting of bytes `[00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 0A, 0B, 0C, 0D, 0E, 0F, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 1A, 1B, 1C, 1D, 1E, 1F]` should be displayed as `0x000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f`. + ::: :::tip + Example calculation (where `H` denotes SHA-256 and `·` denotes blob concatenation): hash_of_map({ request_type: "call", canister_id: 0x00000000000004D2, method_name: "hello", arg: "DIDL\x00\xFD*"}) @@ -766,6 +800,7 @@ Example calculation (where `H` denotes SHA-256 and `·` denotes blob concatenati , b25f03dedd69be07f356a06fe35c1b0ddc0de77dcd9066c4be0c6bbde14b23ff · 6c0b2ae49718f6995c02ac5700c9c789d7b7862a0d53e6d40a73f1fcd2f70189 ]) = 8781291c347db32a9d8c10eb62b710fce5a93be676474c42babc74c51858f94b + ::: ### Reject codes @@ -815,7 +850,9 @@ For this endpoint, the user performs a GET request, and receives a CBOR value wi See [CBOR encoding of requests and responses](#api-cbor) for details on the precise CBOR encoding of this object. :::note + Future additions may include local time, geographic location, and other useful implementation-specific information such as blockheight. This data may possibly be signed by the node. + ::: ### CBOR encoding of requests and responses {#api-cbor} @@ -845,6 +882,7 @@ As advised by [section "Creating CBOR-Based Protocols"](https://tools.ietf.org/h - Duplicate keys are prohibited in CBOR maps. :::tip + A typical request would be (written in [CBOR diagnostic notation](https://tools.ietf.org/html/rfc7049#section-6), which can be checked and converted on [cbor.me](http://cbor.me/)): 55799({ @@ -857,6 +895,7 @@ A typical request would be (written in [CBOR diagnostic notation](https://tools. "sender_sig": h'DEADBEEF', "sender_pubkey": h'b7a3c12dc0c8c748ab07525b701122b88bd78f600c76342d27f25e5f92444cde' }) + ::: ### CDDL description of requests and responses {#api-cddl} @@ -888,7 +927,9 @@ If you then (quickly) issue a read request to node B, it may be that B responds A related problem is that query calls are not certified, and nodes may be dishonest in their response. In that case, the user might want to get more assurance by querying multiple nodes and comparing the result. However, it is (currently) not possible to query a *specific* state. :::note + Applications can work around these problems. For the first problem, the query result could be such that the user can tell if the update has been received or not. For the second problem, even if using [certified data](#system-api-certified-data) is not possible, if replies are monotonic in some sense the user can get assurance in their intersection (e.g. if the query returns a list of events that grows over time, then even if different nodes return different lists, the user can get assurance in those events that are reported by many nodes). + ::: ## Canister module format @@ -989,7 +1030,9 @@ These steps are atomic: If `canister_pre_upgrade` or `canister_post_upgrade` tra To define a public method of name `name`, a WebAssembly module exports a function with name `canister_update ` or `canister_query ` and type `() -> ()`. We call this the *method entry point*. The name of the exported function distinguishes update and query methods. :::note + The space in `canister_update ` resp. `canister_query ` is intentional. There is exactly one space between `canister_update/canister_query` and the ``. + ::: The argument of the call (e.g. the content of the `arg` field in the [API request to call a canister method](#http-call)) is copied into the canister on demand using the System API functions shown below. @@ -1003,7 +1046,9 @@ For periodic or time-based execution, the WebAssembly module can export a functi `canister_heartbeat` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. Still, the function `canister_heartbeat` can initiate new calls. :::note + While an implementation will likely try to keep the interval between `canister_heartbeat` invocations to within a few seconds, this is not formally part of this specification. + ::: #### Global timer {#timer} @@ -1014,8 +1059,11 @@ Once the function `canister_global_timer` is scheduled, the canister's global ti `canister_global_timer` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. Still, the function `canister_global_timer` can initiate new calls. + + :::note + While an implementation will likely try to keep the interval between the value of the global timer and the time-stamp of the `canister_global_timer` invocation within a few seconds, this is not formally part of this specification. + ::: #### Callbacks {#callbacks} @@ -1181,7 +1229,9 @@ Eventually, the canister will want to respond to the original call, either by re Appends data it to the (initially empty) data reply. :::note + This can be invoked multiple times to build up the argument with data from various places on the Wasm heap. This way, the canister does not have to first copy all the pieces from various places into one location. + ::: This traps if the current call already has been or does not need to be responded to. @@ -1213,7 +1263,9 @@ A canister can inspect ingress messages before executing them. When the IC recei In `canister_inspect_message`, the canister can accept the message by invoking `ic0.accept_message : () → ()`. This function traps if invoked twice. If the canister traps in `canister_inspect_message` or does not call `ic0.accept_message`, then the access is denied. :::note + The `canister_inspect_message` is *not* invoked for query calls, inter-canister calls or calls to the management canister. + ::: ### Self-identification {#system-api-canister-self} @@ -1324,7 +1376,9 @@ When handling an update call (or a callback), a canister can do further calls to Each canister maintains a balance of *cycles*, which are used to pay for platform usage. Cycles are represented by 128-bit values. :::note + This specification currently does not go into details about which actions cost how many cycles and/or when. In general, you must assume that the canister's cycle balance can change arbitrarily between method executions, and during each System API function call, unless explicitly mentioned otherwise. + ::: - `ic0.canister_cycle_balance : () → i64` @@ -1332,7 +1386,9 @@ This specification currently does not go into details about which actions cost h Indicates the current cycle balance of the canister. It is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add`. After execution of the message, the IC may add unused cycles from the reserve back to the balance. :::note + This call traps if the current balance does not fit into a 64-bit value. Canisters that need to deal with larger cycles balances should use `ic0.canister_cycles_balance128` instead. + ::: - `ic0.canister_cycle_balance128 : (dst : i32) → ()` @@ -1348,7 +1404,9 @@ This specification currently does not go into details about which actions cost h Initially, in the update method entry point, this is the amount that the caller passed to the canister. When cycles are accepted (`ic0.msg_cycles_accept`), this reports fewer cycles accordingly. When the call is responded to (reply or reject), all available cycles are refunded to the caller, and this will return 0. :::note + This call traps if the amount of cycles available does not fit into a 64-bit value. Please use `ic0.msg_cycles_available128` instead. + ::: - `ic0.msg_cycles_available128 : (dst : i32) → ()` @@ -1374,7 +1432,9 @@ This specification currently does not go into details about which actions cost h This system call does not trap. :::tip + Example: To accept all cycles provided in a call, invoke `ic0.msg_cycles_accept(ic0.msg_cycles_available())` in the method handler or a callback handler, *before* calling reply or reject. + ::: - `ic0.msg_cycles_accept128 : (max_amount_high : i64, max_amount_low : i64, dst : i32) → ()` @@ -1418,7 +1478,9 @@ This specification currently does not go into details about which actions cost h This function can only be used in a callback handler (reply or reject), and indicates the amount of cycles that came back with the response as a refund. The refund has already been added to the canister balance automatically. :::note + This call traps if the amount of cycles refunded does not fit into a 64-bit value. In general, it is recommended to use `ic0.msg_cycles_refunded128` instead. + ::: - `ic0.msg_cycles_refunded128 : (dst : i32) → ()` @@ -1502,7 +1564,9 @@ The time is given as nanoseconds since 1970-01-01. The IC guarantees that The times observed by different canisters are unrelated, and calls from one canister to another may appear to travel "backwards in time". :::note + While an implementation will likely try to keep the time returned by `ic0.time` close to the real time, this is not formally part of this specification. + ::: ### Global timer {#global-timer} @@ -1598,7 +1662,9 @@ A canister may only use the old *or* the new interface; the IC detects which int The interfaces above provide the fundamental ability for external users and canisters to contact other canisters. But the Internet Computer provides additional functionality, such as canister and user management. This functionality is exposed to external users and canisters via the *IC management canister*. :::note + The *IC management canister* is just a facade; it does not actually exist as a canister (with isolated state, Wasm code, etc.). + ::: The IC management canister address is `aaaaa-aa` (i.e. the empty blob). @@ -1673,7 +1739,9 @@ Only controllers of the canister can install code. This is atomic: If the response to this request is a `reject`, then this call had no effect. :::note + Some canisters may not be able to make sense of callbacks after upgrades; these should be stopped first, to wait for all outstanding callbacks that are not marked as deleted, or be uninstalled first, to prevent outstanding callbacks from being invoked (by marking the corresponding call contexts as deleted). It is expected that the canister admin (or their tooling) does that separately. + ::: The `wasm_module` field specifies the canister module to be installed. The system supports multiple encodings of the `wasm_module` field, as described in [Canister module format](#canister-module-format): @@ -1743,7 +1811,9 @@ This method takes no input and returns 32 pseudo-random bytes to the caller. The ### IC method `ecdsa_public_key` {#ic-ecdsa_public_key} :::note + The ECDSA API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. + ::: This method returns a [SEC1](https://www.secg.org/sec1-v2.pdf) encoded ECDSA public key for the given canister using the given derivation path. If the `canister_id` is unspecified, it will default to the canister id of the caller. The `derivation_path` is a vector of variable length byte strings. The `key_id` is a struct specifying both a curve and a name. The availability of a particular `key_id` depends on implementation. @@ -1757,7 +1827,9 @@ This call requires that the ECDSA feature is enabled, and the `canister_id` meet ### IC method `sign_with_ecdsa` {#ic-sign_with_ecdsa} :::note + The ECDSA API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. + ::: This method returns a new [ECDSA](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf) signature of the given `message_hash` that can be separately verified against a derived ECDSA public key. This public key can be obtained by calling `ecdsa_public_key` with the caller's `canister_id`, and the same `derivation_path` and `key_id` used here. @@ -1770,7 +1842,9 @@ This call requires that the ECDSA feature is enabled, the caller is a canister, ### IC method `http_request` {#ic-http_request} :::note + The IC http_request API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. + ::: This method makes an HTTP request to a given URL and returns the HTTP response, possibly after a transformation. @@ -1824,12 +1898,16 @@ The following additional limits apply to HTTP requests and HTTP responses from t - the total number of bytes representing the header names and values must not exceed `48KiB`. :::note + Currently, the Internet Computer mainnet only supports URLs that resolve to IPv6 destinations (i.e., the domain has a `AAAA` DNS record) in HTTP requests. + ::: :::note + The identity of the caller for a valid invocation of the `transform` function is `aaaaa-aa`. This information can be used by developers to implement access control mechanism for this function. + ::: ### IC method `provisional_create_canister_with_cycles` {#ic-provisional_create_canister_with_cycles} @@ -1853,7 +1931,9 @@ This method is only available in local development instances. ## The IC Bitcoin API {#ic-bitcoin-api} :::note + The IC Bitcoin API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. + ::: The Bitcoin functionality is exposed via the management canister. Information about Bitcoin can be found in the [Bitcoin developer guides](https://developer.bitcoin.org/devguide/). Invoking the functions of the Bitcoin API will cost cycles. We refer the reader to the [Bitcoin documentation](https://internetcomputer.org/docs/current/developer-docs/integrations/bitcoin/bitcoin-how-it-works) for further relevant information and the [IC pricing page](https://internetcomputer.org/docs/current/developer-docs/deploy/computation-and-storage-costs) for information on pricing for the Bitcoin mainnet and testnet. @@ -2059,7 +2139,9 @@ The root key can delegate certification authority to other keys. A certificate by the root subnet does not have a delegation field. A certificate by other subnets include a delegation, which is itself a certificate that proves that the subnet is listed in the root subnet's state tree (see [Subnet information](#state-tree-subnet)), and reveals its public key. :::note + The nested certificate *typically* does not itself again contain a delegation, although there is no reason why agents should enforce that property. + ::: Delegation = @@ -2191,8 +2273,10 @@ The following interface description, in [Candid syntax](https://github.com/dfini Only canisters that use the "Upgrade to update calls" feature need to provide the `http_request_update` method. -::: note +:::note + Canisters not using these features can completely leave out the `streaming_strategy` and/or `upgrade` fields in the `HttpResponse` they return, due to how Candid subtyping works. This might simplify their code. + ::: ### Canister resolution {#http-gateway-name-resolution} @@ -2246,13 +2330,16 @@ The Gateway assembles the HTTP response from the given `HttpResponse` record: + NOTE: Not all Gateway implementations may be able to pass on all forms of headers. In particular, Service Workers are unable to pass on the `Set-Cookie` header. + -::: note + +:::note + HTTP Gateways may add additional headers. In particular, the following headers may be set: access-control-allow-origin: * access-control-allow-methods: GET, POST, HEAD, OPTIONS access-control-allow-headers: DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Cookie access-control-expose-headers: Content-Length,Content-Range x-cache-status: MISS + ::: * The HTTP response body is initialized with the value of the `body` field, and further assembled as per the [streaming protocol](#http-gateway-streaming). @@ -2270,8 +2357,10 @@ If the `streaming_strategy` field of the `HttpResponse` is set, the HTTP Gateway 3. That query method returns a `StreamingCallbackHttpResponse`. The `body` therein is appended to the body of the HTTP response. This is repeated as long as the method returns some token in the `token` field, until that field is `null`. -:::warn +:::warning + The type of the `token` value is chosen by the canister; the HTTP Gateway obtains the Candid type of the encoded message from the canister, and uses it when passing the token back to the canister. This generic use of Candid is not covered by the Candid specification, and may not be possible in some cases (e.g. when using "future types"). Canister authors may have to use "simple" types. + ::: @@ -2294,8 +2383,10 @@ If the hostname was safe, the HTTP Gateway performs *certificate validation*: 7. That leaf must contain the SHA-256 hash of the *decoded* body. The decoded body is the body of the HTTP response (in particular, after assembling streaming chunks), decoded according to the `Content-Encoding` header, if present. Supported encodings for `Content-Encoding` are `gzip` and `deflate.` -:::warn +:::warning + The certification protocol only covers the mapping from request URL to response body. It completely ignores the request method and headers, and does not cover the response headers and status code. + ::: ## Abstract behavior @@ -2317,7 +2408,9 @@ We use a concatenation operator `·` with various types: to extend sets and maps The shape of values is described using a hand-wavy type system. We use `Foo = Nat` to define type aliases; now `Foo` can be used instead of `Nat`. Often, the right-hand side is a more complex type here, e.g. a record, or multiple possible types separated by a vertical bar (`|`). Partial maps are written as `Key ↦ Value` and the function type as `Argument → Result`. :::note + All values are immutable! State change is specified by describing the new state, not by changing the existing state. + ::: Record fields are accessed using dot-notation (e.g. `S.request_id > 0`). To create a new record from an existing record `R` with some fields changed, the syntax `R where field = new_value` is used. This syntax can also be used to create new records with some deeply nested field changed: `R where some_map[key].field = new_value`. @@ -2800,7 +2893,9 @@ S with ``` :::note + This is not instantaneous (the IC takes some time to agree it accepts the request) nor guaranteed (a node could just drop the request, or maybe it did not pass validation). But once the request has entered the IC state like this, it will be acted upon. + ::: #### Request rejection {#request_rejection} @@ -3619,7 +3714,9 @@ State after :::note + Sending a `stop_canister` message to an already stopped canister is acknowledged (i.e. responded with success), but is otherwise a no-op: + ::: Conditions @@ -4302,8 +4399,10 @@ We can model the execution of WebAssembly functions as stateful functions that h This allows us to model WebAssembly functions, including host-provided imports, as functions with implicit mutable access to an `ExecutionState`, dubbed *execution functions*. Syntactically, we express this using an implicit argument of type `ref ExecutionState` in angle brackets (e.g. `func(x)` for the invocation of a WebAssembly function with type `(x : i32) -> ()`). The lifetime of the `ExecutionState` data structure is that of one such function invocation. -:::warn +:::warning + It is nonsensical to pass to an execution function a WebAssembly store `S` that comes from a different WebAssembly module than one defining the function. + ::: #### The concrete `CanisterModule` {#the_concrete_canistermodule} From 38fad864d16d2cc3b27133a21472bf0f3041cfc0 Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Thu, 8 Jun 2023 10:54:59 +0200 Subject: [PATCH 053/102] new release 0.19 --- spec/changelog.adoc | 18 + spec/changelog.md | 285 +++ spec/index.adoc | 930 +++++--- spec/index.md | 5337 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 6210 insertions(+), 360 deletions(-) create mode 100644 spec/changelog.md create mode 100644 spec/index.md diff --git a/spec/changelog.adoc b/spec/changelog.adoc index 366abc2f2..afb5b4ffb 100644 --- a/spec/changelog.adoc +++ b/spec/changelog.adoc @@ -1,6 +1,24 @@ [#changelog] == Changelog +[#0_19_0] +=== 0.19.0 (2023-06-08) +* canister version can be specified in some management canister calls (canister creation, canister code changes, canister settings changes) +* IC records canister history (canister creation, canister code changes, and canister controllers changes) +* added a new `canister_info` management canister call returning current module hash, current controllers, and canister history +* added a new system API call `ic0.is_controller` (checking if a principal is a controller of the canister) +* stable memory System API calls can be invoked in the WebAssembly module `(start)` function +* the system API call `ic0.global_timer_set` can be invoked in canister pre-upgrade +* added modeling WASM start function in the concrete `CanisterModule` specification +* WebAssembly module requirements have been revised (relaxed max number of declared functions and globals, added conditions on exported functions) +* certified variables are cleared if a canister is reinstalled +* a canister having an open call context marked as deleted cannot reach Stopped state +* a desired canister ID of the canister created by `provisional_create_canister_with_cycles` (in testing environments) can be specified using `specified_id` +* conditions on envelope delegations have been revised (relaxed max number of delegations, restricted max number of targets per delegation, forbidden cycles in the delegation chain) +* added a new optional field `senders` in envelope delegations (restricting users to which a delegation applies) +* all `/request_status/` paths must refer to the same `request_id` in a `read_state` request +* IC protocol execution error conditions (such as failing `inspect_message` method of a canister) are returned as 202 HTTP responses with a cbor body describing the error (instead of 4xx or 5xx HTTP responses) + [#0_18_9] === 0.18.9 (2022-12-06) * Global timers diff --git a/spec/changelog.md b/spec/changelog.md new file mode 100644 index 000000000..ca5868347 --- /dev/null +++ b/spec/changelog.md @@ -0,0 +1,285 @@ +## Changelog {#changelog} + +### 0.19.0 (2023-06-08) {#0_19_0} +* canister version can be specified in some management canister calls (canister creation, canister code changes, canister settings changes) +* IC records canister history (canister creation, canister code changes, and canister controllers changes) +* added a new `canister_info` management canister call returning current module hash, current controllers, and canister history +* added a new system API call `ic0.is_controller` (checking if a principal is a controller of the canister) +* stable memory System API calls can be invoked in the WebAssembly module `(start)` function +* the system API call `ic0.global_timer_set` can be invoked in canister pre-upgrade +* added modeling WASM start function in the concrete `CanisterModule` specification +* WebAssembly module requirements have been revised (relaxed max number of declared functions and globals, added conditions on exported functions) +* certified variables are cleared if a canister is reinstalled +* a canister having an open call context marked as deleted cannot reach Stopped state +* a desired canister ID of the canister created by `provisional_create_canister_with_cycles` (in testing environments) can be specified using `specified_id` +* conditions on envelope delegations have been revised (relaxed max number of delegations, restricted max number of targets per delegation, forbidden cycles in the delegation chain) +* added a new optional field `senders` in envelope delegations (restricting users to which a delegation applies) +* all `/request_status/` paths must refer to the same `request_id` in a `read_state` request +* IC protocol execution error conditions (such as failing `inspect_message` method of a canister) are returned as 202 HTTP responses with a cbor body describing the error (instead of 4xx or 5xx HTTP responses) + +### 0.18.9 (2022-11-06) {#0_18_9} +* Global timers +* Canister version +* Clarifications for HTTP requests & Bitcoin integration costs + +### 0.18.8 (2022-11-09) {#0_18_8} +* Updated HTTP request API +* Canister status available to canister +* 64-bit stable memory is no longer experimental + +### 0.18.7 (2022-09-27) {#0_18_7} +* HTTP request API +* Reserved principals + +### 0.18.6 (2022-08-09) {#0_18_6} +* Canister access to performance metrics +* Query calls are rejected when the canister is frozen +* Support for implementation-specific error codes for requests +* Deleted call contexts do not prevent canister from reaching Stopped state +* Update effective canister id checks in certificate delegations +* Formal model in Isabelle + +### 0.18.5 (2022-07-08) {#0_18_5} +* Idle consumption of resources in cycles per day can be obtain via `canister_status` method of the management canister +* Include the HTTP Gateway Protocol in this spec +* Clarifications in definition of cycles consumption + +### 0.18.4 (2022-06-20) {#0_18_4} + +* Canister cycle balances are represented by 128 bits, and no system-defined upper limit exists anymore +* Canister modules can be gzip-encoded +* Expose Wasm custom sections in the state tree +* EXPERIMENTAL: Canister API for accessing Bitcoin transactions +* EXPERIMENTAL: Canister API for threshold ECDSA signatures + +### 0.18.3 (2022-01-10) {#0_18_3} + +* New System API which uses 128-bit values to represent the amount of cycles +* Subnet delegations include a canister id scope + +### 0.18.2 (2021-09-29) {#0_18_2} + +* Canister heartbeat +* Terminology changes +* Support for 64-bit stable memory + + +### 0.18.1 (2021-08-04) {#0_18_1} + +* Support RSA PKCS#1 v1.5 signatures in web authentication +* Spec clarification: Fix various typos and improve textual clarity + + +### 0.18.0 (2021-05-18) {#0_18_0} + +* A canister has a set of controllers, instead of always one + + +### 0.17.0 (2021-04-22) {#0_17_0} + +* Canister Signatures are introduced +* Spec clarification: the signature in the WebAuthn scheme is prefixed by the CBOR self-identifying tag +* Cycle-depleted canisters are forcibly uninstalled +* Canister settings in `create_canister` and `update_settings`. `install_code` no longer takes allocation settings. +* A freezing threshold can be configured via the canister settings + +### 0.16.1 (2021-04-14) {#0_16_1} +* The cleanup callback is introduced + +### 0.16.0 (2021-03-25) {#0_16_0} + +* New http v2 API that allows for stateless boundary nodes + +### 0.15.6 (2021-03-25) {#0_15_6} + +* The system may impose limits on the number of globals and functions +* No ingress messages towards empty canisters are accepted +* No ingress messages towards `raw_rand` and `deposit_cycles` are accepted +* A memory allocation of `0` means “best effort” + + +### 0.15.5 (2021-03-11) {#0_15_5} + +* deposit_cycles(): any caller allowed + + +### 0.15.4 (2021-03-04) {#0_15_4} + +* Ingress message filtering +* Add ECDSA signatures on curve secp256k1 +* Clarify that the `ic0.data_certificate_present` system function may be + called in all contexts. + + +### 0.15.3 (2021-02-26) {#0_15_3} + +* Expose module hash and controller via `read_state` + + +### 0.15.2 (2021-02-09) {#0_15_2} + +* The document is renamed to “Internet Computer Interface Spec” + + +### 0.15.0 (2020-12-17) {#0_15_0} + +* Support for raw Ed25519 keys is removed + + +### 0.14.1 (2020-12-08) {#0_14_1} + +* The default `memory_allocation` becomes unspecified + + +### 0.14.0 (2020-11-18) {#0_14_0} + +* Support for funds is scaled back to only support cycles +* The `ic0.msg_cycles_accept` system call now returns the actually accepted + cycles +* The `provisional_` management calls are introduced + + +### 0.13.2 (2020-11-12) {#0_13_2} + +* The `ic0.canister_status` system call + + +### 0.13.1 (2020-11-06) {#0_13_1} + +* Delegation between user public keys + + +### 0.13.0 (2020-10-19) {#0_13_0} + +* Certification (also removes “request-status” request) + + +### 0.12.2 (2020-10-23) {#0_12_2} + +* User authentication method based on WebAuthn is introduced +* User authentication can use ECDSA +* Public keys are DER-encoded + + +### 0.12.1 (2020-10-16) {#0_12_1} + +* Return more information in the `canister_status` management call + + +### 0.12.0 (2020-10-13) {#0_12_0} + +* Anonymous requests must have the sender field set + + +### 0.11.1 (2020-10-01) {#0_11_1} + +* The `deposit_funds` call + + +### 0.11.0 (2020-09-23) {#0_11_0} + +* Inter-canister calls are now performed using a builder-like API +* Support for funds (balances and transfers) + + +### 0.10.3 (2020-09-21) {#v0_10_3} + +* The anonymous user is introduced + + +### 0.10.1 (2020-09-01) {#v0_10_1} + +* Forward-port changes from 0.9.3 + + +### 0.10.0 (2020-08-06) {#v0_10_0} + +* Users can set/update a memory allocation when installing/upgrading a canister. +* The `expiry` field is added to requests + + +### 0.9.3 (2020-09-01) {#v0_9_3} + +* The management canister supports the `raw_rand` method + + +### 0.9.2 (2020-08-05) {#v0_9_2} + +* Canister controllers can stop/start canisters and can query their status. +* Canister controllers can delete canisters + + +### 0.9.1 (2020-07-20) {#v0_9_1} + +* Forward-port changes from 0.8.2 + + +### 0.9.0 (2020-07-15) {#v0_9_0} + +* Introduction of a domain separator (again) +* The calculation of “derived ids” has changed +* The self-authenticating and derived id forms use a truncated hash +* The textual representation of principals has changed + + +### 0.8.2 (2020-07-17) {#v0_8_2} + +* Installing code via `reinstall` works also on the empty canister + + +### 0.8.1 (2020-07-10) {#v0_8_1} + +* Reflect refined process in README and intro. +* `ic0.time` added + + +### 0.8.0 (2020-06-23) {#v0_8_0} + +* Revert the introduction of a domain separator + + +### 0.6.2 (2020-06-23) {#v0_6_2} + +* Fix meaning-changing typos in `ic.did` + + +### 0.6.0 (2020-06-08) {#v0_6_0} + +* Make all canister ids system-chosen +* HTTP requests for management features are removed + + +### 0.4.0 (2020-05-25) {#v0_4_0} + +* (editorial) the term “principal” is now used for the _id_ of a canister or + user, not the canister or user itself +* The signature of a request needs to be calculated using a domain separator +* Describe the `controller` attribute, add a request to change it +* The IC management canister is introduced + + +### 0.2.16 (2020-05-29) {#v0_2_16} + +* More tests about calls from query methods + + +### 0.2.14 (2020-05-14) {#v0_2_14} + +* Bugfix: Mode should be `reinstall`, not `replace` + + +### 0.2.8 (2020-04-23) {#v0_2_8} + +* Include section with CDDL description + + +### 0.2.4 (2020-03-23) {#v0_2_4} + +* simplify versioning (only three components), skip 0.2.2 to avoid confusion with 0.2.0.2 +* Clarification: `reply` field is always present +* General cleanup based on front-to-back reading + + +### 0.2.0.0 (2020-03-11) {#v0_2_0_0} + +* This is the first release. Subsequent releases will include a changelog. diff --git a/spec/index.adoc b/spec/index.adoc index b120d2266..ae7e7d928 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1,6 +1,6 @@ = The Internet Computer Interface Specification DFINITY Foundation -0.18.9 +0.19.0 // the following are ways to make this document work both in antora and stand alone :example: example$ :partial: partial$ @@ -217,7 +217,7 @@ When the cycle balance of a canister falls to zero, the canister is _deallocated Afterwards the canister is empty. It can be reinstalled after topping up its balance. -NOTE: Once the IC frees the resources of a canister, its id, _cycles_ balance, and _controllers_ are preserved on the IC for a minimum of 10 years. What happens to the canister after this period is currently unspecified. +NOTE: Once the IC frees the resources of a canister, its id, _cycles_ balance, _controllers_, canister _version_, and the total number of canister changes are preserved on the IC for a minimum of 10 years. What happens to the canister after this period is currently unspecified. [#canister-status] ==== Canister status @@ -226,7 +226,7 @@ The canister status can be used to control whether the canister is processing ca * In status `running`, calls to the canister are processed as normal. * In status `stopping`, calls to the canister are rejected by the IC with reject code `CANISTER_ERROR` (5), but responses to the canister are processed as normal. -* In status `stopped`, calls to the canister are rejected by the IC with reject code `CANISTER_ERROR` (5), and there are no outstanding responses to call contexts that are not marked as deleted. +* In status `stopped`, calls to the canister are rejected by the IC with reject code `CANISTER_ERROR` (5), and there are no outstanding responses. In all cases, calls to the <> are processed, regardless of the state of the managed canister. @@ -277,7 +277,6 @@ The signature is checked by verifying that the `challenge` field contains the ht * The public key is encoded as a DER-wrapped COSE key. + -- -[#oid-der-wrapped-cose] It uses the `SubjectPublicKeyInfo` type used for other types of public keys (see, e.g., https://tools.ietf.org/html/rfc8410#section-4[RFC 8410, Section 4]), with OID 1.3.6.1.4.1.56387.1.1 (iso.org.dod.internet.private.enterprise.dfinity.mechanisms.der-wrapped-cose). The `BIT STRING` field `subjectPublicKey` contains the COSE encoding. See https://www.w3.org/TR/webauthn/#sctn-encoded-credPubKey-examples[WebAuthn w3c recommendation] or https://tools.ietf.org/html/rfc8152#section-13.1[RFC 8152] for details on the COSE encoding. @@ -299,7 +298,7 @@ You can also view the wrapping in https://lapo.it/asn1js/#MF4wDAYKKwYBBAGDuEMBAQ ==== -- -* The signature is a CBOR value consisting of a data item with major type 6 (“Semantic tag”) and tag value `55799` (see https://tools.ietf.org/html/rfc7049#section-2.4.5[Self-Describe CBOR]), followed by a map with three mandatory fields: +* The signature is a CBOR (see <>) value consisting of a data item with major type 6 (“Semantic tag”) and tag value `55799`, followed by a map with three mandatory fields: - `authenticator_data` (`blob`): WebAuthn authenticator data. - `client_data_json` (`text`): WebAuthn client data in JSON representation. - `signature` (`blob`): Signature as specified in the https://www.w3.org/TR/webauthn/#signature-attestation-types[WebAuthn w3c recommendation], which means DER encoding in the case of an ECDSA signature. @@ -317,7 +316,7 @@ More concretely, it uses the `SubjectPublicKeyInfo` type used for other types of + The `BIT STRING` field `subjectPublicKey` is the blob `|signing_canister_id| · signing_canister_id · seed`, where `|signing_canister_id|` is the one-byte encoding of the the length of the `signing_canister_id` and `·` denotes blob concatenation. -* The signature is a CBOR value consisting of a data item with major type 6 (“Semantic tag”) and tag value `55799` (see https://tools.ietf.org/html/rfc7049#section-2.4.5[Self-Describe CBOR]), followed by a map with two mandatory fields: +* The signature is a CBOR (see <>) value consisting of a data item with major type 6 (“Semantic tag”) and tag value `55799`, followed by a map with two mandatory fields: + -- - `certificate` (`blob`): A CBOR-encoded certificate as per <>. @@ -342,6 +341,36 @@ where `signing_canister_id` is the id of the signing canister and `reconstruct` where `s` is the SHA-256 hash of the `seed` used in the public key and `m` is the SHA-256 hash of the payload. -- +[#supplementary-technologies] +=== Supplementary Technologies + +[#cbor] +==== CBOR + +https://www.rfc-editor.org/rfc/rfc8949[Concise Binary Object Representation (CBOR)] is a data format with a small code footprint, small message size and an extensible interface. CBOR is used extensively throughout the Internet Computer as the primary format for data exchange between components within the system. + +https://cbor.io[cbor.io] and https://en.wikipedia.org/wiki/CBOR[wikipedia.org] contain a lot of helpful background information and relevant tools. https://cbor.me[cbor.me] in particular, is very helpful for converting between CBOR hex and diagnostic information. + +For example, the following CBOR hex: +.... +82 61 61 a1 61 62 61 63 +.... + +Can be converted into the following CBOR diagnostic format: +.... +["a", {"b": "c"}] +.... + +Particular concepts to note from the spec are: + +* https://www.rfc-editor.org/rfc/rfc8949#name-specification-of-the-cbor-e[Specification of the CBOR Encoding] +* https://www.rfc-editor.org/rfc/rfc8949#name-major-types[CBOR Major Types] +* https://www.rfc-editor.org/rfc/rfc8949#self-describe[CBOR Self-Describe] + +[#cddl] +==== CDDL + +The https://tools.ietf.org/html/rfc8610[Concise Data Definition Language (CDDL)] is a data description language for CBOR. It is used at various points throughout this document to describe how certain data structures are encoded with CBOR. [#state-tree] == The system state tree @@ -372,7 +401,7 @@ The public key of the subnet (a DER-encoded BLS key, see <>) * `/subnet//canister_ranges` (blob) + -The set of canister ids assigned to this subnet, represented as a list of closed intervals of canister ids, ordered lexicographically, and encoded as CBOR according to this CDDL: +The set of canister ids assigned to this subnet, represented as a list of closed intervals of canister ids, ordered lexicographically, and encoded as CBOR (see <>) according to this CDDL (see <>): + .... canister_ranges = tagged<[*canister_range]> @@ -433,7 +462,7 @@ If the canister is not empty, it exists and contains the SHA256 hash of the curr * `/canister//controllers` (blob): + -The current controllers of the canister. The value consists of a CBOR data item with major type 6 (“Semantic tag”) and tag value `55799` (see https://tools.ietf.org/html/rfc7049#section-2.4.5[Self-Describe CBOR]), followed by an array of principals in their binary form (CDDL `#6.55799([* bytes .size (0..29)])`). +The current controllers of the canister. The value consists of a CBOR (see <>) data item with major type 6 (“Semantic tag”) and tag value `55799`, followed by an array of principals in their binary form (CDDL `#6.55799([* bytes .size (0..29)])`, see <>). * `/canister//metadata/` (blob): + @@ -446,7 +475,7 @@ It is recommended for the canister to have a custom section called "icp:public c The concrete mechanism that users use to send requests to the Internet Computer is via an HTTPS API, which exposes three endpoints to handle interactions, plus one for diagnostics: -* At `/api/v2/canister//call` the user can submit (asynchronous, state-changing) calls. +* At `/api/v2/canister//call` the user can submit (asynchronous, potentially state-changing) calls. * At `/api/v2/canister//read_state` the user can read various information about the state of the Internet Computer. In particular, they can poll for the status of a call here. * At `/api/v2/canister//query` the user can perform (synchronous, non-state-changing) query calls. * At `/api/v2/status` the user can retrieve status information about the Internet Computer. @@ -528,9 +557,17 @@ In order to call a canister, the user makes a POST request to `/api/v2/canister/ * `method_name` (`text`): Name of the canister method to call * `arg` (`blob`): Argument to pass to the canister method -The HTTP response to this request has an empty body and HTTP status 202, or a HTTP error (4xx or 5xx). Paranoid agents should not trust this response, and use <> to determine the status of the call. +The HTTP response to this request can have the following responses: + +* 202 HTTP status with empty body. Implying the request was accepted by the IC for further processing. Users should use <> to determine the status of the call. +* 200 HTTP status with non-empty body. Implying an execution pre-processing error occurred. The body of the response contains more information about the IC specific error encountered. The body is a CBOR map with the following fields: +** `reject_code` (`nat`): The reject code (see <>). +** `reject_message` (`text`): a textual diagnostic message. +** `error_code` (`text`): an optional implementation-specific textual error code (see <>). +* 4xx HTTP status for client errors (e.g. malformed request). Except for 429 HTTP status, retrying the request will likely have the same outcome. +* 5xx HTTP status when the server has encountered an error or is otherwise incapable of performing the request. The request might succeed if retried at a later time. -This request type can _also_ be used to call a query method. A user may choose to go this way, instead of via the faster and cheaper <> below, if they want to get a _certified_ response. +This request type can _also_ be used to call a query method. A user may choose to go this way, instead of via the faster and cheaper <> below, if they want to get a _certified_ response. Note that the canister state will not be changed by sending a call request type for a query method. NOTE: The functionality exposed via the <> can be used this way. @@ -544,7 +581,7 @@ In order to read parts of the <>, the user makes a POST request to ` * `sender`, `nonce`, `ingress_expiry`: See <> * `paths` (sequence of paths): A list of paths, where a path is itself a sequence of blobs. -The HTTP response to this request consists of a CBOR map with the following fields: +The HTTP response to this request consists of a CBOR (see <>) map with the following fields: * `certificate` (`blob`): A certificate (see <>). + @@ -568,14 +605,9 @@ All requested paths must have one of the following paths as prefix: - `` is a public custom section or - `` is a private custom section and the sender of the read state request is a controller of the canister. -If a path cannot be requested, then the HTTP response to the read state request is undefined. - -Read state requests containing many requested paths might be rejected with a 4xx HTTP status code. +Moreover, all paths with prefix `/request_status/` must refer to the same request ID ``. -[NOTE] -==== -The Internet Computer blockchain mainnet might reject read state requests that request more than 1 path with prefix `/request_status/`. -==== +If a path cannot be requested, then the HTTP response to the read state request is undefined. Note that the paths `/canisters//certified_data` are not accessible with this method; these paths are only exposed to the canisters themselves via the System API (see <>). @@ -594,7 +626,7 @@ In order to make a query call to canister, the user makes a POST request to `/ap * `method_name` (`text`): Name of the canister method to call * `arg` (`blob`): Argument to pass to the canister method -If the call resulted in a reply, the response is a CBOR map with the following fields: +If the call resulted in a reply, the response is a CBOR (see <>) map with the following fields: * `status` (`text`): `replied` * `reply`: a CBOR map with the field `arg` (`blob`) which contains the reply data. @@ -645,7 +677,7 @@ The envelope, i.e. the overall request, has the following keys: * `content` (`record`): the actual request content * `sender_pubkey` (`blob`, optional): Public key used to authenticate this request. Since a user may in the future have more than one key, this field tells the IC which key is used. -* `sender_delegation` (`array` of maps, optional): a chain of delegations, starting with the one signed by `sender_pubkey` and ending with the one delegating to the key relating to `sender_sig`. +* `sender_delegation` (`array` of maps, optional): a chain of delegations, starting with the one signed by `sender_pubkey` and ending with the one delegating to the key relating to `sender_sig`. Every public key in the chain of delegations should appear exactly once: cycles (a public key delegates to another public key that already previously appeared in the chain) or self-signed delegations (a public key delegates to itself) are not allowed and such requests will be refused by the IC. * `sender_sig` (`blob`, optional): Signature to authenticate this request. The public key must authenticate the `sender` principal: @@ -664,14 +696,15 @@ If delegation is used, then the `sender_delegation` field contains an array of d * `delegation` (`map`): Map with fields: - `pubkey` (`blob`): Public key as described in <>. - `expiration` (`nat`): Expiration of the delegation, in nanoseconds since 1970-01-01, analogously to the `ingress_expiry` field above. - - `targets` (`array` of `CanisterId`, optional): If this field is set, the delegation only applies for requests sent to the canisters in the list. + - `targets` (`array` of `CanisterId`, optional): If this field is set, the delegation only applies for requests sent to the canisters in the list. The list must contain no more than 1000 elements; otherwise, the request will not be accepted by the IC. + - `senders` (`array` of `Principal`, optional): If this field is set, the delegation only applies for requests originating from the principals in the list. * `signature` (`blob`): Signature on the 32-byte <> of the map contained in the `delegation` field as described in <>, using the 27 bytes `\x1Aic-request-auth-delegation` as the domain separator. + For the first delegation in the array, this signature is created with the key corresponding to the public key from the `sender_pubkey` field, all subsequent delegations are signed with the key corresponding to the public key contained in the preceding delegation. The `sender_sig` field is calculated by signing the concatenation of the 11 bytes `\x0Aic-request` (the domain separator) and the 32 byte <> with the secret key that belongs to the key specified in the last delegation or, if no delegations are present, the public key specified in `sender_pubkey`. -The delegation field, if present, must not contain more than four delegations. +The delegation field, if present, must not contain more than 20 delegations. [#hash-of-map] === Representation-independent hashing of structured data @@ -692,6 +725,8 @@ The following encodings of field values as blobs are used * Strings (`request_type`, `method_name`) are encoded in UTF-8, without a terminal `\x00`. * Natural numbers (`compute_allocation`, `memory_allocation`, `ingress_expiry`) are encoded using the shortest form https://en.wikipedia.org/wiki/LEB128#Unsigned_LEB128[Unsigned LEB128] encoding. For example, `0` should be encoded as a single zero byte `[0x00]` and `624485` should be encoded as byte sequence `[0xE5, 0x8E, 0x26]`. +* Integers are encoded using the shortest form https://en.wikipedia.org/wiki/LEB128#Signed_LEB128[Signed LEB128] encoding. + For example, `0` should be encoded as a single zero byte `[0x00]` and `-123456` should be encoded as byte sequence `[0xC0, 0xBB, 0x78]`. * Arrays (`paths`) are encoded as the concatenation of the hashes of the encodings of the array elements. * Maps (`sender_delegation`) are encoded by recursively computing the representation-independent hash. @@ -700,37 +735,43 @@ The following encodings of field values as blobs are used When signing requests or querying the status of a request (see <>) in the state tree, the user identifies the request using a _request id_, which is the <> of the `content` map of the original request. A request id must have length of 32 bytes. -NOTE: The request id is independent of the representation of the request (currently only CBOR), and does not change if the specification adds further optional fields to a request type. +NOTE: The request id is independent of the representation of the request (currently only CBOR, see <>), and does not change if the specification adds further optional fields to a request type. NOTE: The recommended textual representation of a request id is a hexadecimal string with lower-case letters prefixed with '0x'. E.g., request id consisting of bytes `[00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 0A, 0B, 0C, 0D, 0E, 0F, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 1A, 1B, 1C, 1D, 1E, 1F]` should be displayed as `0x000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f`. [TIP] ==== -Example calculation (where `H` denotes SHA-256 and `·` denotes blob concatenation): +Example calculation (where `H` denotes SHA-256 and `·` denotes blob concatenation) in which we assume that the optional nonce is not provided and thus omitted: [source,,options="nowrap"] ---- -hash_of_map({ request_type: "call", canister_id: 0x00000000000004D2, method_name: "hello", arg: "DIDL\x00\xFD*"}) +hash_of_map({ request_type: "call", sender: 0x04, ingress_expiry: 1685570400000000000, canister_id: 0x00000000000004D2, method_name: "hello", arg: "DIDL\x00\xFD*"}) = H(concat (sort [ H("request_type") · H("call") + , H("sender") · H("0x04") + , H("ingress_expiry") · H(1685570400000000000) , H("canister_id") · H("\x00\x00\x00\x00\x00\x00\x04\xD2") , H("method_name") · H("hello") , H("arg") · H("DIDL\x00\xFD*") ])) = H(concat (sort [ 769e6f87bdda39c859642b74ce9763cdd37cb1cd672733e8c54efaa33ab78af9 · 7edb360f06acaef2cc80dba16cf563f199d347db4443da04da0c8173e3f9e4ed + , 0a367b92cf0b037dfd89960ee832d56f7fc151681bb41e53690e776f5786998a · e52d9c508c502347344d8c07ad91cbd6068afc75ff6292f062a09ca381c89e71 + , 26cec6b6a9248a96ab24305b61b9d27e203af14a580a5b1ff2f67575cab4a868 · db8e57abc8cda1525d45fdd2637af091bc1f28b35819a40df71517d1501f2c76 , 0a3eb2ba16702a387e6321066dd952db7a31f9b5cc92981e0a92dd56802d3df9 · 4d8c47c3c1c837964011441882d745f7e92d10a40cef0520447c63029eafe396 , 293536232cf9231c86002f4ee293176a0179c002daa9fc24be9bb51acdd642b6 · 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 , b25f03dedd69be07f356a06fe35c1b0ddc0de77dcd9066c4be0c6bbde14b23ff · 6c0b2ae49718f6995c02ac5700c9c789d7b7862a0d53e6d40a73f1fcd2f70189 ])) = H(concat - [ 0a3eb2ba16702a387e6321066dd952db7a31f9b5cc92981e0a92dd56802d3df9 · 4d8c47c3c1c837964011441882d745f7e92d10a40cef0520447c63029eafe396 + [ 0a367b92cf0b037dfd89960ee832d56f7fc151681bb41e53690e776f5786998a · e52d9c508c502347344d8c07ad91cbd6068afc75ff6292f062a09ca381c89e71 + , 0a3eb2ba16702a387e6321066dd952db7a31f9b5cc92981e0a92dd56802d3df9 · 4d8c47c3c1c837964011441882d745f7e92d10a40cef0520447c63029eafe396 + , 26cec6b6a9248a96ab24305b61b9d27e203af14a580a5b1ff2f67575cab4a868 · db8e57abc8cda1525d45fdd2637af091bc1f28b35819a40df71517d1501f2c76 , 293536232cf9231c86002f4ee293176a0179c002daa9fc24be9bb51acdd642b6 · 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 , 769e6f87bdda39c859642b74ce9763cdd37cb1cd672733e8c54efaa33ab78af9 · 7edb360f06acaef2cc80dba16cf563f199d347db4443da04da0c8173e3f9e4ed , b25f03dedd69be07f356a06fe35c1b0ddc0de77dcd9066c4be0c6bbde14b23ff · 6c0b2ae49718f6995c02ac5700c9c789d7b7862a0d53e6d40a73f1fcd2f70189 ]) - = 8781291c347db32a9d8c10eb62b710fce5a93be676474c42babc74c51858f94b + = 1d1091364d6bb8a6c16b203ee75467d59ead468f523eb058880ae8ec80e2b101 ---- ==== @@ -768,9 +809,9 @@ Additionally, the Internet Computer provides an API endpoint to obtain various s /api/v2/status .... -For this endpoint, the user performs a GET request, and receives a CBOR value with the following fields. The IC may include additional implementation-specific fields. +For this endpoint, the user performs a GET request, and receives a CBOR (see <>) value with the following fields. The IC may include additional implementation-specific fields. -* `ic_api_version` (string, mandatory): Identifies the interface version supported, i.e. the version of the present document that the internet computer aims to support, e.g. `0.8.1`. The implementation may also return `unversioned` to indicate that it does _not_ comply to a particular version, e.g. in between releases. +* `ic_api_version` (string, mandatory): Identifies the interface version supported, i.e. the version of the present document that the Internet Computer aims to support, e.g. `0.8.1`. The implementation may also return `unversioned` to indicate that it does _not_ comply to a particular version, e.g. in between releases. * `impl_source` (string, optional): Identifies the implementation of the Internet Computer Protocol, by convention with the canonical location of the source code (e.g. `https://github.com/dfinity/ic`). * `impl_version` (string, optional): If the user is talking to a released version of an Internet Computer Protocol implementation, this is the version number. For non-released versions, output of `git describe` like `0.1.13-13-g2414721` would also be very suitable. * `impl_revision` (string, optional): The precise git revision of the Internet Computer Protocol implementation @@ -785,9 +826,9 @@ possibly be signed by the node. [#api-cbor] === CBOR encoding of requests and responses -Requests and responses are specified here as records with named fields and using suggestive human readable syntax. The actual format in the body of the HTTP request or response, however, is https://en.wikipedia.org/wiki/CBOR[CBOR]. +Requests and responses are specified here as records with named fields and using suggestive human readable syntax. The actual format in the body of the HTTP request or response, however, is CBOR (see <>). -Concretely, it consists of a data item with major type 6 (“Semantic tag”) and tag value `55799` (see https://tools.ietf.org/html/rfc7049#section-2.4.5[Self-Describe CBOR]), followed by a record. +Concretely, it consists of a data item with major type 6 (“Semantic tag”) and tag value `55799`, followed by a record. Requests consist of an envelope record with keys `sender_sig` (a blob), `sender_pubkey` (a blob) and `content` (a record). The first two are metadata that are used for request authentication, while the last one is the actual content of the request. @@ -795,18 +836,18 @@ The following encodings are used: * Strings: Major type 3 (“Text string”). * Blobs: Major type 2 (“Byte string”). -* Nats: Major type 0 (“Unsigned integer”) if small enough to fit that type, else the https://tools.ietf.org/html/rfc7049#section-2.4.2[Bignum] format is used. +* Nats: Major type 0 (“Unsigned integer”) if small enough to fit that type, else the https://www.rfc-editor.org/rfc/rfc8949#name-bignums[Bignum] format is used. * Records: Major type 5 (“Map of pairs of data items”), followed by the fields, where keys are encoded with major type 3 (“Text string”). * Arrays: Major type 4 ("Array of data items"). -As advised by https://tools.ietf.org/html/rfc7049#section-3[section “Creating CBOR-Based Protocols”] of the CBOR spec, we clarify that: +As advised by https://www.rfc-editor.org/rfc/rfc8949#name-creating-cbor-based-protoco[section “Creating CBOR-Based Protocols”] of the CBOR spec, we clarify that: * Floating-point numbers may not be used to encode integers. * Duplicate keys are prohibited in CBOR maps. [TIP] ==== -A typical request would be (written in https://tools.ietf.org/html/rfc7049#section-6[CBOR diagnostic notation], which can be checked and converted on http://cbor.me/[cbor.me]): +A typical request would be (written in https://www.rfc-editor.org/rfc/rfc8949#name-diagnostic-notation[CBOR diagnostic notation], which can be checked and converted on http://cbor.me/[cbor.me]): .... 55799({ "content": { @@ -825,7 +866,7 @@ A typical request would be (written in https://tools.ietf.org/html/rfc7049#secti [#api-cddl] === CDDL description of requests and responses -The https://tools.ietf.org/html/rfc8610[Concise Data Definition Language (CDDL)] is a data description language for CBOR. This section summarizes the format of the CBOR data passed to and from the entry points described above. You can also link:{attachmentsdir}/requests.cddl[download the file]. +This section summarizes the format of the CBOR data passed to and from the entry points described above. You can also link:{attachmentsdir}/requests.cddl[download the file] and see <> for more information. [source,bash] ---- @@ -893,10 +934,12 @@ In order for a WebAssembly module to be usable as the code for the canister, it * It may not have both `icp:public ` and `icp:private ` with the same `name` as the custom section name. * It may not have other custom sections the names of which start with the prefix `icp:` besides the `icp:public ` and `icp:private `. * The IC may reject WebAssembly modules that - + declare more than 6000 functions, or - + declare more than 200 globals, or - + declare more than 16 exported custom sections (the custom section names with prefix `icp:`), or - + the total size of the exported custom sections exceeds 1MiB + - declare more than 50,000 functions, or + - declare more than 300 globals, or + - declare more than 16 exported custom sections (the custom section names with prefix `icp:`), or + - the number of all exported functions called `canister_update ` or `canister_query ` exceeds 1,000, or + - the sum of `` lengths in all exported functions called `canister_update ` or `canister_query ` exceeds 20,000, or + - the total size of the exported custom sections exceeds 1MiB. === Interpretation of numbers @@ -1026,30 +1069,21 @@ which copies, at the time of function invocation, the data referred to by `src`/ The canister can access an argument. For `canister_init`, `canister_post_upgrade` and method entry points, the argument is the argument of the call; in a reply callback, it refers to the received reply. So the lifetime of the argument data is a single WebAssembly function execution, not the whole method call tree. -* {blank} -+ - ic0.msg_arg_data_size : () -> i32 - ic0.msg_arg_data_copy : (dst : i32, offset : i32, size : i32) -> () +* `ic0.msg_arg_data_size : () -> i32` and `ic0.msg_arg_data_copy : (dst : i32, offset : i32, size : i32) -> ()` + The message argument data. -* {blank} -+ - ic0.msg_caller_size : () -> i32 - ic0.msg_caller_copy : (dst : i32, offset: i32, size : i32) -> () +* `ic0.msg_caller_size : () -> i32` and `ic0.msg_caller_copy : (dst : i32, offset: i32, size : i32) -> ()` + The identity of the caller, which may be a canister id or a user id. During canister installation or upgrade, this is the id of the user or canister requesting the installation or upgrade. -* `+ic0.msg_reject_code : () -> i32+` +* `ic0.msg_reject_code : () -> i32` + Returns the reject code, if the current function is invoked as a reject callback. + It returns the special “no error” code `0` if the callback is _not_ invoked as a reject callback; this allows canisters to use a single entry point for both the reply and reject callback, if they choose to do so. + -* {blank} -+ - ic0.msg_reject_msg_size : () -> i32 - ic0.msg_reject_msg_copy : (dst : i32, offset : i32, size : i32) -> () +* `ic0.msg_reject_msg_size : () -> i32` and `ic0.msg_reject_msg_copy : (dst : i32, offset : i32, size : i32) -> ()` + The reject message. Traps if there is no reject message (i.e. if `reject_code` is `0`). @@ -1058,15 +1092,17 @@ The reject message. Traps if there is no reject message (i.e. if `reject_code` i Eventually, the canister will want to respond to the original call, either by replying (indicating success) or rejecting (signalling an error): -* `+ic0.msg_reply_data_append : (src : i32, size : i32) -> ()+` +* `ic0.msg_reply_data_append : (src : i32, size : i32) -> ()` + Appends data it to the (initially empty) data reply. + -NOTE: This can be invoked multiple times to build up the argument with data from various places on the Wasm heap. This way, the canister does not have to first copy all the pieces from various places into one location. -+ This traps if the current call already has been or does not need to be responded to. ++ +Any data assembled, but not replied using `ic0.msg_reply`, gets discarded at the end of the current message execution. In particular, the reply buffer gets reset when the canister yields control without calling `ic0.msg_reply`. ++ +NOTE: This can be invoked multiple times within the same message execution to build up the argument with data from various places on the Wasm heap. This way, the canister does not have to first copy all the pieces from various places into one location. -* `+ic0.msg_reply : () -> ()+` +* `ic0.msg_reply : () -> ()` + Replies to the sender with the data assembled using `ic0.msg_reply_data_append`. + @@ -1074,7 +1110,7 @@ This function can be called at most once (a second call will trap), and must be + See <> for how this interacts with cycles available on this call. -* `+ic0.msg_reject : (src : i32, size : i32) -> ()+` +* `ic0.msg_reject : (src : i32, size : i32) -> ()` + Rejects the call. The data referred to by `src`/`size` is used for the diagnostic message. + @@ -1105,10 +1141,7 @@ NOTE: The `canister_inspect_message` is _not_ invoked for query calls, inter-can A canister can learn about its own identity: -* {blank} -+ - ic0.canister_self_size : () -> i32 - ic0.canister_self_copy: (dst : i32, offset : i32, size : i32) -> () +* `ic0.canister_self_size : () -> i32` and `ic0.canister_self_copy: (dst : i32, offset : i32, size : i32) -> ()` + These functions allow the canister to query its own canister id (as a blob). @@ -1141,10 +1174,7 @@ During the canister upgrade process, `canister_pre_upgrade` sees the old counter When handling an update call (or a callback), a canister can do further calls to another canister. Calls are assembled in a builder-like fashion, starting with `ic0.call_new`, adding more attributes using the `ic0.call_*` functions, and eventually performing the call with `ic0.call_perform`. -* {blank} -+ --- - ic0.call_new : +* `ic0.call_new : ( callee_src : i32, callee_size : i32, name_src : i32, @@ -1153,7 +1183,7 @@ When handling an update call (or a callback), a canister can do further calls to reply_env : i32, reject_fun : i32, reject_env : i32, - ) -> () + ) -> ()` Begins assembling a call to the canister specified by `callee_src/_size` at method `name_src/_size`. @@ -1166,12 +1196,8 @@ The reject callback is executed if the method call fails asynchronously or the o This deducts `MAX_CYCLES_PER_RESPONSE` cycles from the canister balance and sets them aside for response processing. This will trap if not sufficient cycles are available. Subsequent calls to the following functions set further attributes of that call, until the call is concluded (with `ic0.call_perform`) or discarded (by returning without calling `ic0.call_perform` or by starting a new call with `ic0.call_new`.) --- -* {blank} -+ --- - ic0.call_on_cleanup : (fun : i32, env : i32) -> () +* `ic0.call_on_cleanup : (fun : i32, env : i32) -> ()` If a cleanup callback (of type `+(env : i32) -> ()+`) is specified for this call, it is executed if and only if the `reply` or the `reject` callback was executed and trapped (for any reason). @@ -1180,7 +1206,6 @@ During the execution of the `cleanup` function, only a subset of the System API If this traps (e.g. runs out of cycles), the state changes from the `cleanup` function are discarded, as usual, and no further actions are taken related to that call. Canisters likely want to avoid this from happening. There must be at most one call to `ic0.call_on_cleanup` between `ic0.call_new` and `ic0.call_perform`. --- * `+ic0.call_data_append : (src : i32, size : i32) -> ()+` + @@ -1254,38 +1279,34 @@ This system call is experimental. It may be changed or removed in the future. Ca * `ic0.msg_cycles_accept : (max_amount : i64) -> (amount : i64)` + --- This moves cycles from the call to the canister balance. It moves as many cycles as possible, up to these constraints: - -* It moves no more cycles than `max_amount`. - -* It moves no more cycles than available according to `ic0.msg_cycles_available`, and - ++ +It moves no more cycles than `max_amount`. ++ +It moves no more cycles than available according to `ic0.msg_cycles_available`, and ++ It can be called multiple times, each time possibly adding more cycles to the balance. - ++ The return value indicates how many cycles were actually moved. - ++ This system call does not trap. - ++ [TIP] ===== Example: To accept all cycles provided in a call, invoke `ic0.msg_cycles_accept(ic0.msg_cycles_available())` in the method handler or a callback handler, _before_ calling reply or reject. ===== --- * `ic0.msg_cycles_accept128 : (max_amount_high : i64, max_amount_low : i64, dst : i32) -> ()` + --- This moves cycles from the call to the canister balance. It moves as many cycles as possible, up to these constraints: - -* It moves no more cycles than the amount obtained by combining `max_amount_high` and `max_amount_low`. Cycles are represented by 128-bit values. - -* It moves no more cycles than available according to `ic0.msg_cycles_available128`, and - ++ +It moves no more cycles than the amount obtained by combining `max_amount_high` and `max_amount_low`. Cycles are represented by 128-bit values. ++ +It moves no more cycles than available according to `ic0.msg_cycles_available128`, and ++ It can be called multiple times, each time possibly adding more cycles to the balance. - ++ This call also copies the amount of cycles that were actually moved starting at the location `dst` in the canister memory. --- + This does not trap. + @@ -1427,10 +1448,21 @@ The system resets the counter at the beginning of each <> invocati The main purpose of this counter is to facilitate in-canister performance profiling. +[#system-api-controller-check] +=== Controller check + +The canister can check whether a given principal is one of its controllers. + +`+ic0.is_controller : (src : i32, size: i32) -> (result: i32)+` + +Checks whether the principal identified by `src`/`size` is one of the controllers of the canister. If yes, then a value of 1 is returned, otherwise a 0 is returned. It can be called multiple times. + +This system call traps if `src+size` exceeds the size of the WebAssembly memory or the principal identified by `src`/`size` is not a valid binary encoding of a principal. + [#system-api-certified-data] === Certified data -For each canister, the IC keeps track of “certified data”, a canister-defined blob. For fresh canisters, this blob is the empty blob (`""`). +For each canister, the IC keeps track of “certified data”, a canister-defined blob. For fresh canisters (upon install or reinstall), this blob is the empty blob (`""`). * `+ic0.certified_data_set : (src: i32, size : i32) -> ()+` + @@ -1446,10 +1478,7 @@ This will return `1` when called from a query method when invoked via a query ca + This will return `0` if the query method is executed within replicated execution (e.g. when invoked via an update call or inter-canister call). -* {blank} -+ - ic0.data_certificate_size : () -> i32 - ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> () +* `ic0.data_certificate_size : () -> i32` and `ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> ()` + Copies the certificate for the current value of the certified data to the canister. + @@ -1537,8 +1566,6 @@ If set to 0, then memory growth of the canister will be best-effort and subject + Default value: 0 -Until code is installed, the canister is `Empty` and behaves like a canister that has no public methods. - * `freezing_threshold` (`nat`) + Must be a number between 0 and 2^64^-1, inclusively, and indicates a length of time in seconds. @@ -1549,6 +1576,9 @@ Calls to a frozen canister will be rejected (like for a stopping canister). Addi + Default value: 2592000 (approximately 30 days). +The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. + +Until code is installed, the canister is `Empty` and behaves like a canister that has no public methods. [#ic-update_settings] === IC method `update_settings` @@ -1557,6 +1587,8 @@ Only _controllers_ of the canister can update settings. See <>: @@ -1582,6 +1614,8 @@ The system supports multiple encodings of the `wasm_module` field, as described * If the `wasm_module` starts with byte sequence `[0x00, 'a', 's', 'm']`, the system parses `wasm_module` as a raw WebAssembly binary. * If the `wasm_module` starts with byte sequence `[0x1f, 0x8b, 0x08]`, the system parses `wasm_module` as a gzip-compressed WebAssembly binary. +The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. + [#ic-uninstall_code] === IC method `uninstall_code` @@ -1595,6 +1629,8 @@ The canister is now <>. In particular, any incoming or A canister after _uninstalling_ retains its _cycles_ balance, _controllers_, status, and allocations. +The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. + [#ic-canister_status] === IC method `canister_status` @@ -1608,6 +1644,27 @@ Indicates various information about the canister. It contains: Only the controllers of the canister or the canister itself can request its status. +[#ic-canister-info] +=== IC method `canister_info` + +Provides the history of the canister, its current module SHA-256 hash, and its current controllers. Every canister can call this method on every other canister (including itself). Users cannot call this method. + +The canister history consists of a list of canister changes (canister creation, code uninstallation, code deployment, or controllers change). Every canister change consists of the system timestamp at which the change was performed, the canister version after performing the change, the change's origin (a user or a canister), and its details. The change origin includes the principal (called _originator_ in the following) that initiated the change and, if the originator is a canister, the originator's canister version when the originator initiated the change (if available). Code deployments are described by their mode (code install, code reinstall, code upgrade) and the SHA-256 hash of the newly deployed canister module. Canister creations and controllers changes are described by the full new set of the canister controllers after the change. + +The system can drop the oldest canister changes from the list to keep its length bounded (at least `20` changes are guaranteed to remain in the list). The system also drops all canister changes if the canister runs out of cycles. + +The following parameters should be supplied for the call: + +* `canister_id`: the canister ID of the canister to retrieve information about. +* `num_requested_changes`: optional, specifies the number of requested canister changes. If not provided, the default value of `0` will be used. + +The returned response contains the following fields: + +* `total_num_changes`: the total number of canister changes that have been ever recorded in the history. This value does not change if the system drops the oldest canister changes from the list of changes. +* `recent_changes`: the list containing the most recent canister changes. If `num_requested_changes` is provided, then this list contains that number of changes or, if more changes are requested than available in the history, then this list contains all changes available in the history. If `num_requested_changes` is not specified, then this list is empty. +* `module_hash`: the SHA-256 hash of the currently installed canister module (or `null` if the canister is empty). +* `controllers`: the current set of canister controllers. + [#ic-stop_canister] === IC method `stop_canister` @@ -1616,7 +1673,7 @@ The controllers of a canister may stop a canister (e.g., to prepare for a canist Stopping a canister is not an atomic action. The immediate effect is that the status of the canister is changed to `stopping` (unless the canister is already stopped). The IC will reject all calls to a stopping canister, indicating that the canister is stopping. Responses to a stopping canister are processed as usual. -When all outstanding responses to call contexts that are not marked as deleted have been processed (so there are no open call contexts that are not marked as deleted), the canister status is changed to `stopped` and the management canister responds to the caller of the `stop_canister` request. +When all outstanding responses have been processed (so there are no open call contexts), the canister status is changed to `stopped` and the management canister responds to the caller of the `stop_canister` request. [#ic-start_canister] === IC method `start_canister` @@ -1652,7 +1709,7 @@ This method takes no input and returns 32 pseudo-random bytes to the caller. The NOTE: The ECDSA API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. This method returns a https://www.secg.org/sec1-v2.pdf[SEC1] encoded ECDSA public key for the given canister using the given derivation path. If the `canister_id` is unspecified, it will default to the canister id of the caller. -The `derivation_path` is a vector of variable length byte strings. +The `derivation_path` is a vector of variable length byte strings. Each byte string may be of arbitrary length, including empty. The total number of strings in `derivation_path` can be at most 255. The `key_id` is a struct specifying both a curve and a name. The availability of a particular `key_id` depends on implementation. @@ -1722,16 +1779,21 @@ The `transform` function may, for example, transform the body in any way, add or When the transform function is invoked by the system due to a canister HTTP request, the caller's identity is the principal of the management canister. This information can be used by developers to implement access control mechanism for this function. The following additional limits apply to HTTP requests and HTTP responses from the remote sever: + - the number of headers must not exceed `64`, - the number of bytes representing a header name or value must not exceed `8KiB`, and - the total number of bytes representing the header names and values must not exceed `48KiB`. NOTE: Currently, the Internet Computer mainnet only supports URLs that resolve to IPv6 destinations (i.e., the domain has a `AAAA` DNS record) in HTTP requests. +WARNING: If you do not specify the `max_response_bytes` parameter, the maximum of a `2MB` response will be charged for, which is expensive in terms of cycles. Always set the parameter to a reasonable upper bound of the expected network response size to not incur unnecessary cycles costs for your request. + [#ic-provisional_create_canister_with_cycles] === IC method `provisional_create_canister_with_cycles` -As a provisional method on development instances, the `provisional_create_canister_with_cycles` method is provided. It behaves as `create_canister`, but initializes the canister’s balance with `amount` fresh cycles (using `DEFAULT_PROVISIONAL_CYCLES_BALANCE` if `amount = null`). +As a provisional method on development instances, the `provisional_create_canister_with_cycles` method is provided. It behaves as `create_canister`, but initializes the canister’s balance with `amount` fresh cycles (using `DEFAULT_PROVISIONAL_CYCLES_BALANCE` if `amount = null`). If `specified_id` is provided, the canister is created under this id. Note that canister creation using `create_canister` or `provisional_create_canister_with_cycles` with `specified_id = null` can fail after calling `provisional_create_canister_with_cycles` with provided `specified_id`. In that case, canister creation should be retried. + +The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. Cycles added to this call via `ic0.call_cycles_add128` are returned to the caller. @@ -1839,8 +1901,8 @@ If at least one of these checks fails, the call is rejected. If the transaction passes these tests, the transaction is forwarded to the specified Bitcoin network. -Note that the function does not provide any guarantees that a submitted -transaction will ever appear in a block. +Note that the function does not provide any guarantees that the transaction will make it into the mempool or that +the transaction will ever appear in a block. [#ic-bitcoin_get_current_fee_percentiles] === IC method `bitcoin_get_current_fee_percentiles` @@ -2011,7 +2073,7 @@ Delegations are _scoped_, i.e., they indicate which set of canister principals t [#certification-encoding] === Encoding of certificates -The binary encoding of a certificate is a CBOR value according to the following CDDL. You can also link:{attachmentsdir}/certificates.cddl[download the file]. +The binary encoding of a certificate is a CBOR (see <>) value according to the following CDDL (see <>). You can also link:{attachmentsdir}/certificates.cddl[download the file]. [source,bash] ---- @@ -2050,7 +2112,7 @@ avoided), but it is valid. // The following is checked in `impl/src/IC/Test/HashTree.hs`, so please keep // in sync -This tree has the following CBOR encoding +This tree has the following CBOR (see <>) encoding .... 8301830183024161830183018302417882034568656c6c6f810083024179820345776f726c6483024162820344676f6f648301830241638100830241648203476d6f726e696e67 .... @@ -2096,143 +2158,7 @@ lookup_path(["e"], pruned_tree) = Absent [#http-gateway] == The HTTP Gateway protocol -This section specifies the _HTTP Gateway protocol_, which allows canisters to handle conventional HTTP requests. - -This feature involves the help of a _HTTP Gateway_ that translates between HTTP requests and the IC protocol. Such a gateway could be a stand-alone proxy, it could be implemented in a web browsers (natively, via plugin or via a service worker) or in other ways. This document describes the interface and semantics of this protocol independent of a concrete Gateway, so that all Gateway implementations can be compatible. - -Conceptually, this protocol builds on top of the interface specified in the remainder of this document, and therefore is an “application-level” interface, not a feature of the core Internet Computer system described in the other sections, and could be a separate document. We nevertheless include this protocol in the Internet Computer Interface Specification because of its important role in the ecosystem and due to the importance of keeping multiple Gateway implementations in sync. - -=== Overview - -A HTTP request by an HTTP client is handled by these steps: - -1. The Gateway resolves the Host of the request to a canister id. -2. The Gateway Candid-encodes the HTTP request data. -3. The Gateway invokes the canister via a query call to `http_request`. -4. The canister handles the request and returns a HTTP response, encoded in Candid, together with additional metadata. -5. If requested by the canister, the Gateway sends the request again via an update call to `http_request_update`. -6. If applicable, the Gateway fetches further body data via streaming query calls. -7. If applicable, the Gateway validates the certificate of the response. -8. The Gateway sends the response to the HTTP client. - -[#http-gateway-interface] -=== Candid interface - -The following interface description, in https://github.com/dfinity/candid/blob/master/spec/Candid.md[Candid syntax], describes the expected Canister interface. You can also link:{attachmentsdir}/http-gateway.did[download the file]. ----- -include::{example}http-gateway.did[] ----- - -Only canisters that use the “Upgrade to update calls” feature need to provide the `http_request_update` method. - -NOTE: Canisters not using these features can completely leave out the `streaming_strategy` and/or `upgrade` fields in the `HttpResponse` they return, due to how Candid subtyping works. This might simplify their code. - -[#http-gateway-name-resolution] -=== Canister resolution - -The Gateway needs to know the canister id of the canister to talk to, and obtains that information from the hostname as follows: - -1. Check that the hostname, taken from the `Host` field of the HTTP request, is of the form `.raw.ic0.app` or `.ic0.app`, or fail. - -2. If the `` is in the following table, use the given canister ids: -+ -.Canister hostname resolution -|============================================ -| Hostname | Canister id -| `identity` | `rdmx6-jaaaa-aaaaa-aaadq-cai` -| `nns` | `qoctq-giaaa-aaaaa-aaaea-cai` -| `dscvr` | `h5aet-waaaa-aaaab-qaamq-cai` -| `personhood` | `g3wsl-eqaaa-aaaan-aaaaa-cai` -|============================================ - -3. Else, if `` is a valid textual encoding of a principal, use that principal as the canister id. - -4. Else fail. - -If the hostname was of the form `.ic0.app`, it is a _safe_ hostname; if it was of the form `.raw.ic0.app` it is a _raw_ hostname. - -=== Request encoding - -The HTTP request is encoded into the `HttpRequest` Candid structure. - -* The `method` field contains the HTTP method (e.g. `HTTP`), in upper case. - -* The `url` field contains the URL from the HTTP request line, i.e. without protocol or hostname, and including query parameters. - -* The `headers` field contains the headers of the HTTP request. - -* The `body` field contains the body of the HTTP request (without any content encodings processed by the Gateway). - -=== Upgrade to update calls - -If the canister sets `upgrade = opt true` in the `HttpResponse` reply from `http_request`, then the Gateway ignores all other fields of the reply. The Gateway performs an _update_ call to `http_request_update`, passing the same `HttpRequest` record as the argument, and uses that response instead. - -The value of the `upgrade` field returned from `http_request_update` is ignored. - -=== Response decoding - -The Gateway assembles the HTTP response from the given `HttpResponse` record: - -* The HTTP response status code is taken from the `status_code` field. - -* The HTTP response headers are taken from the `headers` field. -+ -NOTE: Not all Gateway implementations may be able to pass on all forms of headers. In particular, Service Workers are unable to pass on the `Set-Cookie` header. -+ -[NOTE] -==== -HTTP Gateways may add additional headers. In particular, the following headers may be set: - -.... -access-control-allow-origin: * -access-control-allow-methods: GET, POST, HEAD, OPTIONS -access-control-allow-headers: DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Cookie -access-control-expose-headers: Content-Length,Content-Range -x-cache-status: MISS -.... -==== - -* The HTTP response body is initialized with the value of the `body` field, and further assembled as per the <>. - -[#http-gateway-streaming] -=== Response body streaming - -The HTTP Gateway protocol has provisions to transfer further chunks of the body data from the canister to the HTTP Gateway, to overcome the message limit of the Internet Computer. This streaming protocol is independent of any possible streaming of data between the HTTP Gateway and the HTTP client. The gateway may assemble the response in whole before passing it on, or pass the chunks on directly, on the TCP or HTTP level, as it sees fit. When the Gateway is <>, it must not pass on uncertified chunks. - -If the `streaming_strategy` field of the `HttpResponse` is set, the HTTP Gateway then uses further query calls to obtain further chunks to append to the body: - -1. If the function reference in the `callback` field of the `streaming_strategy` is not a method of the given canister, the Gateway fails the request. - -2. Else, it makes a query call to the given method, passing the `token` value given in the `streaming_strategy` as the argument. - -3. That query method returns a `StreamingCallbackHttpResponse`. The `body` therein is appended to the body of the HTTP response. This is repeated as long as the method returns some token in the `token` field, until that field is `null`. - -WARNING: The type of the `token` value is chosen by the canister; the HTTP Gateway obtains the Candid type of the encoded message from the canister, and uses it when passing the token back to the canister. This generic use of Candid is not covered by the Candid specification, and may not be possible in some cases (e.g. when using “future types”). Canister authors may have to use “simple” types. - - -[#http-gateway-certification] -=== Response certification - -If the hostname was safe, the HTTP Gateway performs _certificate validation_: - -1. It searches for a response header called `Ic-Certificate` (case-insensitive). - -2. The value of the header must be a structured header according to RFC 8941 with fields `certificate` and `tree`, both being byte sequences. - -3. The `certificate` must be a valid certificate as per <>, signed by the root key. If the certificate contains a subnet delegation, the delegation must be valid for the given canister. The timestamp in `/time` must be recent. The subnet state tree in the certificate must reveal the canister’s <>. - -4. The `tree` must be a hash tree as per <>. - -5. The root hash of that `tree` must match the canister’s certified data. - -6. The path `["http_assets",]`, where `url` is the utf8-encoded `url` from the `HttpRequest` must exist and be a leaf. Else, if it does not exist, `["http_assets","/index.html"]` must exist and be a leaf. - -7. That leaf must contain the SHA-256 hash of the _decoded_ body. -+ -The decoded body is the body of the HTTP response (in particular, after assembling streaming chunks), decoded according to the `Content-Encoding` header, if present. Supported encodings for `Content-Encoding` are `gzip` and `deflate.` - -WARNING: The certification protocol only covers the mapping from request URL to response body. It completely ignores the request method and headers, and does not cover the response headers and status code. - +The HTTP Gateway Protocol has been moved into its own https://internetcomputer.org/docs/current/references/http-gateway-protocol-spec[specification]. [#abstract-behavior] == Abstract behavior @@ -2327,6 +2253,7 @@ Timestamp = Nat; CanisterVersion = Nat; Env = { time : Timestamp; + controllers : List Principal; global_timer : Nat; balance : Nat; freezing_limit : Nat; @@ -2437,9 +2364,9 @@ CallOrigin } | FromCanister { calling_context : CallId; - callback: Callback + callback: Callback; } - | FromSystem + | FromSystemTask CallCtxt = { canister : CanisterId; origin : CallOrigin; @@ -2576,6 +2503,40 @@ CanStatus = Running | Stopping (List (CallOrigin, Nat)) | Stopped +ChangeOrigin + = FromUser { + user_id : PrincipalId; + } + | FromCanister { + canister_id : PrincipalId; + canister_version : CanisterVersion | NoCanisterVersion; + } +CodeDeploymentMode + = Install + | Reinstall + | Upgrade +ChangeDetails + = Creation { + controllers : [PrincipalId]; + } + | CodeUninstall + | CodeDeployment { + mode : CodeDeploymentMode; + module_hash : Blob; + } + | ControllersChange { + controllers : [PrincipalId]; + } +Change = { + timestamp_nanos : Timestamp; + canister_version : CanisterVersion; + origin : ChangeOrigin; + details : ChangeDetails; +} +CanisterHistory = { + total_num_changes : Nat; + recent_changes : [Change]; +} S = { requests : Request ↦ (RequestStatus, Principal); canisters : CanisterId ↦ CanState; @@ -2587,6 +2548,7 @@ S = { global_timer : CanisterId ↦ Timestamp; balances: CanisterId ↦ Nat; certified_data: CanisterId ↦ Blob; + canister_history: CanisterId ↦ CanisterHistory; system_time : Timestamp call_contexts : CallId ↦ CallCtxt; messages : List Message; // ordered! @@ -2601,6 +2563,21 @@ simple_status(Stopping _) = Stopping simple_status(Stopped) = Stopped .... +To convert `CallOrigin` into `ChangeOrigin`, we define the following conversion function: +.... +change_origin(principal, _, FromUser { … }) = FromUser { + user_id = principal + } +change_origin(principal, sender_canister_version, FromCanister { … }) = FromCanister { + canister_id = principal + canister_version = sender_canister_version + } +change_origin(principal, sender_canister_version, FromSystemTask) = FromCanister { + canister_id = principal + canister_version = sender_canister_version + } +.... + ==== Initial state The initial state of the IC is @@ -2649,6 +2626,13 @@ The following is an incomplete list of invariants that should hold for the abstr if Ctxt.needs_to_respond = false then Ctxt.available_cycles = 0 .... +* A stopped canister does not have any call contexts (in particular, a stopped canister does not have any call contexts marked as deleted): ++ +.... +∀ Ctxt_id ↦ Ctxt ∈ S.call_contexts: + S.canister_status[Ctxt.canister] ≠ Stopped +.... + * Referenced call contexts exist: + .... @@ -2680,22 +2664,28 @@ The following predicate describes when an envelope `E` correctly signs the enclo It returns which canister ids this envelope may be used at (as a set of principals). .... verify_envelope({ content = C }, U, T) - = { p : p is PrincipalId } if U = anonymous_id + = { p : p is CanisterID } if U = anonymous_id verify_envelope({ content = C, sender_pubkey = PK, sender_sig = Sig, sender_delegation = DS}, U, T) = TS if U = mk_self_authenticating_id E.sender_pubkey - ∧ (PK', TS) = verify_delegations(DS, PK, T, { p : p is PrincipalId }) - ∧ verify_signature ("\x0Aic-request" · hash_of_map(C), PK') + ∧ (PK', TS) = verify_delegations(DS, PK, T, { p : p is CanisterId }, U) + ∧ verify_signature PK' Sig ("\x0Aic-request" · hash_of_map(C)) -verify_delegations([], PK, T, TS) = (PK, TS) -verify_delegations([D] · DS, PK, T, TS) - = verify_delegations(C, DS, D.pubkey, T, TS ∩ delegation_targets(DS)) +verify_delegations([], PK, T, TS, U) = (PK, TS) +verify_delegations([D] · DS, PK, T, TS, U) + = verify_delegations(DS, D.pubkey, T, TS ∩ delegation_targets(D), U) if verify_signature PK D.signature ("\x1Aic-request-auth-delegation" · hash_of_map(D.delegation)) ∧ D.delegation.expiration ≥ T + ∧ U ∈ delegated_senders(D) -delegation_targets(DS) +delegation_targets(D) = if D.targets = Unrestricted - then { p : p is PrincipalId } + then { p : p is CanisterId } else D.targets + +delegated_senders(D) + = if D.senders = Unrestricted + then { p : p is Principal } + else D.senders .... ==== Effective canister ids @@ -2703,7 +2693,7 @@ delegation_targets(DS) A `Request` has an effective canister id according to the rules in <<#http-effective-canister-id>>: .... -is_effective_canister_id(Request {canister_id = ic_principal, method = provisional_create_canister_with_cycles, p) +is_effective_canister_id(Request {canister_id = ic_principal, method = provisional_create_canister_with_cycles, …}, p) is_effective_canister_id(Request {canister_id = ic_principal, arg = candid({canister_id = p, …}), …}, p) is_effective_canister_id(Request {canister_id = p, …}, p), if p ≠ ic_principal .... @@ -2747,6 +2737,7 @@ Conditions:: S.canisters[E.content.canister_id] ≠ EmptyCanister Env = { time = S.time[E.content.canister_id]; + controllers = S.controllers[E.content.canister_id]; global_timer = S.global_timer[E.content.canister_id]; balance = S.balances[E.content.canister_id] freezing_limit = freezing_limit(S, E.content.canister_id); @@ -2840,19 +2831,18 @@ State after:: ==== Call context creation Before invoking a heartbeat, a global timer, or a message to a public entry point, a call context is created for bookkeeping purposes. -For these invocations the canister must be running (so not stopped, or stopping). +For these invocations the canister must be running (so not stopped or stopping). Additionally, these invocations only happen for "real" canisters, not the IC management canister. This “bookkeeping transition” must be immediately followed by the corresponding <>. -* Call context creation: Public entry points -+ +_Call context creation: Public entry points_ + For a message to a public entry point, the method is looked up in the list of exports. This happens for both ingress and inter-canister messages. -+ + The position of the message in the queue is unchanged. -+ + Conditions:: -+ .... S.messages = Older_messages · CallMessage CM · Younger_messages S.canisters[CM.callee] ≠ EmptyCanister @@ -2860,9 +2850,8 @@ Conditions:: S.balances[CM.callee] ≥ freezing_limit(S, CM.callee) + MAX_CYCLES_PER_MESSAGE Ctxt_id ∉ dom(S.call_contexts) .... -+ + State after:: -+ .... S with messages = @@ -2884,21 +2873,19 @@ S with balances[CM.callee] = S.balances[CM.callee] - MAX_CYCLES_PER_MESSAGE .... -* Call context creation: Heartbeat -+ +_Call context creation: Heartbeat_ + If canister `C` exports a method with name `canister_heartbeat`, the IC will create the corresponding call context. -+ + Conditions:: -+ .... S.canisters[C] ≠ EmptyCanister S.canister_status[C] = Running S.balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE Ctxt_id ∉ dom(S.call_contexts) .... -+ + State after:: -+ .... S with messages = @@ -2911,7 +2898,7 @@ S with · S.messages call_contexts[Ctxt_id] = { canister = C; - origin = FromSystem; + origin = FromSystemTask; needs_to_respond = false; deleted = false; available_cycles = 0; @@ -2919,12 +2906,11 @@ S with balances[C] = S.balances[C] - MAX_CYCLES_PER_MESSAGE .... -* Call context creation: Global timer -+ +_Call context creation: Global timer_ + If canister `C` exports a method with name `canister_global_timer`, the global timer of canister `C` is set, and the current time for canister `C` has passed the value of the global timer, the IC will create the corresponding call context and deactivate the global timer. -+ + Conditions:: -+ .... S.canisters[C] ≠ EmptyCanister S.canister_status[C] = Running @@ -2933,9 +2919,8 @@ Conditions:: S.balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE Ctxt_id ∉ dom(S.call_contexts) .... -+ + State after:: -+ .... S with messages = @@ -2948,7 +2933,7 @@ S with · S.messages call_contexts[Ctxt_id] = { canister = C; - origin = FromSystem; + origin = FromSystemTask; needs_to_respond = false; deleted = false; available_cycles = 0; @@ -2977,6 +2962,7 @@ Conditions:: Env = { time = S.time[M.receiver]; + controllers = S.controllers[M.receiver]; global_timer = S.global_timer[M.receiver]; balance = S.balances[M.receiver] freezing_limit = freezing_limit(S, M.receiver); @@ -3008,6 +2994,7 @@ State after:: .... if R = Return res + validate_sender_canister_version(res.new_calls, S.canister_version[M.receiver]) res.cycles_used ≤ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) res.cycles_accepted ≤ Available (res.cycles_used + ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ]) ≤ @@ -3026,7 +3013,7 @@ then [ CallMessage { origin = FromCanister { call_context = M.call_context; - callback = call.callback + callback = call.callback; }; caller = M.receiver; callee = call.callee; @@ -3084,6 +3071,12 @@ If message execution <>; that would require a (unique) call to `ic0.reply`. Note also that the state changes are persisted even when the IC is set to synthesize a <> reject immediately afterward (which happens when this returns without calling `ic0.reply` or `ic0.reject`, the corresponding call has not been responded to and there are no outstanding callbacks, see <>). +The function `validate_sender_canister_version` checks that `sender_canister_version` matches the actual canister version of the sender in all calls to the methods of the management canister that take `sender_canister_version`: +.... +validate_sender_canister_version(new_calls, canister_version_from_system) = + ∀ call ∈ new_calls. (call.callee = ic_principal and (call.method = 'create_canister' or call.method = 'update_settings' or call.method = 'install_code' or call.method = 'uninstall_code' or call.method = 'provisional_create_canister_with_cycles') and call.arg = candid(A) and A.sender_canister_version = n) => n = canister_version_from_system +.... + The functions `query_as_update` and `system_task_as_update` turns a query function resp the heartbeat or global timer into an update function; this is merely a notational trick to simplify the rule: .... query_as_update(f, arg, env) = λ wasm_state → @@ -3122,7 +3115,7 @@ If the call context is not for heartbeat or global timer and there is no call, d Conditions:: .... S.call_contexts[Ctxt_id].needs_to_respond = true - S.call_contexts[Ctxt_id].origin ≠ FromSystem + S.call_contexts[Ctxt_id].origin ≠ FromSystemTask ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id ∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id ∀ _ ↦ {needs_to_respond = true, origin = FromCanister O, …} ∈ S.call_contexts: O.calling_context ≠ Ctxt_id @@ -3151,7 +3144,7 @@ Conditions:: S.call_contexts[Ctxt_id].needs_to_respond = false ) or ( - S.call_contexts[Ctxt_id].origin = FromSystem + S.call_contexts[Ctxt_id].origin = FromSystemTask ∀ FuncMessage M ∈ S.messages. M.call_context ≠ Ctxt_id ) ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id @@ -3184,6 +3177,10 @@ Conditions:: M.arg = candid(A) is_system_assigned CanisterId CanisterId ∉ dom(S.canisters) + if A.settings.controllers is not null: + New_controllers = A.settings.controllers + else: + New_controllers = [M.caller] .... State after:: .... @@ -3191,16 +3188,24 @@ S with canisters[CanisterId] = EmptyCanister time[CanisterId] = CurrentTime global_timer[CanisterId] = 0 - if A.settings.controllers is not null: - controllers[CanisterId] = A.settings.controllers - else: - controllers[CanisterId] = [M.caller] + controllers[CanisterId] = New_controllers if A.settings.freezing_threshold is not null: freezing_threshold[CanisterId] = A.settings.freezing_threshold else: freezing_threshold[CanisterId] = 2592000 balances[CanisterId] = M.transferred_cycles certified_data[CanisterId] = "" + canister_history[CanisterId] = { + total_num_changes = 1 + recent_changes = { + timestamp_nanos = CurrentTime + canister_version = 0 + origin = change_origin(M.caller, A.sender_canister_version, M.origin) + details = Creation { + controllers = New_controllers + } + } + } messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin @@ -3238,12 +3243,27 @@ Conditions:: M.method_name = 'update_settings' M.arg = candid(A) M.caller ∈ S.controllers[A.canister_id] + S.canister_history[A.canister_id] = { + total_num_changes = N; + recent_changes = H; + } .... State after:: .... S with if A.settings.controllers is not null: controllers[A.canister_id] = A.settings.controllers + canister_history[A.canister_id] = { + total_num_changes = N + 1; + recent_changes = H · { + timestamp_nanos = S.time[A.canister_id]; + canister_version = S.canister_version[A.canister_id] + 1; + origin = change_origin(M.caller, A.sender_canister_version, M.origin); + details = ControllersChange { + controllers = A.settings.controllers; + }; + }; + } if A.settings.freezing_threshold is not null: freezing_threshold[A.canister_id] = A.settings.freezing_threshold canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 @@ -3294,6 +3314,40 @@ S with } .... +==== IC Management Canister: Canister information + +Every canister can retrieve the canister history, current module hash, and current controllers of every other canister (including itself). + +Conditions:: +.... + S.messages = Older_messages · CallMessage M · Younger_messages + (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) + M.callee = ic_principal + M.method_name = 'canister_info' + M.arg = candid(A) + if A.num_requested_changes = null then From = |S.canister_history[A.canister_id].recent_changes| + else From = max(0, |S.canister_history[A.canister_id].recent_changes| - A.num_requested_changes) + End = |S.canister_history[A.canister_id].recent_changes| - 1 +.... +State after:: +.... +S with + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = candid({ + total_num_changes = S.canister_history[A.canister_id].total_num_changes; + recent_changes = S.canister_history[A.canister_id].recent_changes[From..End]; + module_hash = + if S.canisters[A.canister_id] = EmptyCanister + then null + else opt (SHA-256(S.canisters[A.canister_id].raw_module)); + controllers = S.controllers[A.canister_id]; + }) + refunded_cycles = M.transferred_cycles + } +.... + ==== IC Management Canister: Code installation Only the controllers of the given canister can install code. This transition installs new code over a canister. This involves invoking the `canister_init` method (see <>), which must succeed. @@ -3312,7 +3366,8 @@ Conditions:: M.caller ∈ S.controllers[A.canister_id] Env = { time = S.time[A.canister_id]; - global_timer = S.global_timer[A.canister_id]; + controllers = S.controllers[A.canister_id]; + global_timer = 0; balance = S.balances[A.canister_id]; freezing_limit = freezing_limit(S, A.canister_id); certificate = NoCertificate; @@ -3322,7 +3377,10 @@ Conditions:: Mod.init(A.canister_id, A.arg, M.caller, Env) = Return {new_state = New_state; new_certified_data = New_certified_data; new_global_timer = New_global_timer; cycles_used = Cycles_used;} Cycles_used ≤ S.balances[A.canister_id] dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ - + S.canister_history[A.canister_id] = { + total_num_changes = N; + recent_changes = H; + } .... State after:: .... @@ -3334,14 +3392,25 @@ S with public_custom_sections = Public_custom_sections; private_custom_sections = Private_custom_sections; } - if New_certified_data ≠ NoCertifiedData: - certified_data[A.canister_id] = New_certified_data + certified_data[A.canister_id] = New_certified_data if New_global_timer ≠ NoGlobalTimer: global_timer[A.canister_id] = New_global_timer else: global_timer[A.canister_id] = 0 canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 balances[A.canister_id] = S.balances[A.canister_id] - Cycles_used + canister_history[A.canister_id] = { + total_num_changes = N + 1; + recent_changes = H · { + timestamp_nanos = S.time[A.canister_id]; + canister_version = S.canister_version[A.canister_id] + 1 + origin = change_origin(M.caller, A.sender_canister_version, M.origin); + details = CodeDeployment { + mode = A.mode; + module_hash = SHA-256(A.wasm_module); + }; + }; + } messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin; @@ -3369,6 +3438,7 @@ Conditions:: S.canisters[A.canister_id] = { wasm_state = Old_state; module = Old_module, …} Env = { time = S.time[A.canister_id]; + controllers = S.controllers[A.canister_id]; balance = S.balances[A.canister_id]; freezing_limit = freezing_limit(S, A.canister_id); certificate = NoCertificate; @@ -3386,6 +3456,10 @@ Conditions:: Mod.post_upgrade(A.canister_id, Stable_memory, A.arg, M.caller, Env2) = Return {new_state = New_state; new_certified_data = New_certified_data'; new_global_timer = New_global_timer; cycles_used = Cycles_used';} Cycles_used + Cycles_used' ≤ S.balances[A.canister_id] dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ + S.canister_history[A.canister_id] = { + total_num_changes = N; + recent_changes = H; + } .... State after:: .... @@ -3407,6 +3481,18 @@ S with global_timer[A.canister_id] = 0 canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 balances[A.canister_id] = S.balances[A.canister_id] - (Cycles_used + Cycles_used'); + canister_history[A.canister_id] = { + total_num_changes = N + 1; + recent_changes = H · { + timestamp_nanos = S.time[A.canister_id]; + canister_version = S.canister_version[A.canister_id] + 1 + origin = change_origin(M.caller, A.sender_canister_version, M.origin); + details = CodeDeployment { + mode = Upgrade; + module_hash = SHA-256(A.wasm_module); + }; + }; + } messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin; @@ -3428,12 +3514,25 @@ Conditions:: M.method_name = 'uninstall_code' M.arg = candid(A) M.caller ∈ S.controllers[A.canister_id] + S.canister_history[A.canister_id] = { + total_num_changes = N; + recent_changes = H; + } .... State after:: .... S with canisters[A.canister_id] = EmptyCanister certified_data[A.canister_id] = "" + canister_history[A.canister_id] = { + total_num_changes = N + 1; + recent_changes = H · { + timestamp_nanos = S.time[A.canister_id]; + canister_version = S.canister_version[A.canister_id] + 1 + origin = change_origin(M.caller, A.sender_canister_version, M.origin); + details = CodeUninstall; + }; + } canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 global_timer[A.canister_id] = 0 @@ -3462,12 +3561,12 @@ S with ==== IC Management Canister: Stopping a canister -The controllers of a canister can stop a canister. Stopping a canister goes through two steps. First, the status of the canister is set to `Stopping`; as explained above, a stopping canister rejects all incoming requests and continues processing outstanding responses. When a stopping canister has no more open call contexts that are not marked as deleted, its status is changed to `Stopped` and a response is generated. Note that when processing responses, a stopping canister can make calls to other canisters and thus create new call contexts. In addition, a canister which is stopped, or stopping will accept (and respond) to further `stop_canister` requests. +The controllers of a canister can stop a canister. Stopping a canister goes through two steps. First, the status of the canister is set to `Stopping`; as explained above, a stopping canister rejects all incoming requests and continues processing outstanding responses. When a stopping canister has no more open call contexts, its status is changed to `Stopped` and a response is generated. Note that when processing responses, a stopping canister can make calls to other canisters and thus create new call contexts. In addition, a canister which is stopped or stopping will accept (and respond) to further `stop_canister` requests. We encode this behavior via three (types of) transitions: 1. First, any `stop_canister` call sets the state of the canister to `Stopping`; we record in the status the origin (and cycles) of all `stop_canister` calls which arrive at the canister while it is stopping (or stopped). -2. Next, when the canister has no open call contexts that are not marked as deleted (so, in particular, all outstanding responses to the canister have been processed or will be discared because the call context has been marked as deleted), the status of the canister is set to `Stopped`. +2. Next, when the canister has no open call contexts (so, in particular, all outstanding responses to the canister have been processed), the status of the canister is set to `Stopped`. 3. Finally, each pending `stop_canister` call (which are encoded in the status) is responded to, to indicate that the canister is stopped. Conditions:: @@ -3507,12 +3606,12 @@ S with .... -The status of a stopping canister which has no open call contexts that are not marked as deleted is set to `Stopped`, and all pending `stop_canister` calls are replied to. +The status of a stopping canister which has no open call contexts is set to `Stopped`, and all pending `stop_canister` calls are replied to. Conditions:: .... S.canister_status[CanisterId] = Stopping Origins - ∀ Ctxt_id. S.call_contexts[Ctxt_id].deleted or S.call_contexts[Ctxt_id].canister ≠ CanisterId + ∀ Ctxt_id. S.call_contexts[Ctxt_id].canister ≠ CanisterId .... State after:: .... @@ -3704,34 +3803,51 @@ Conditions:: M.arg = candid(A) is_system_assigned CanisterId CanisterId ∉ dom(S.canisters) + A.specified_id ∉ dom(S.canisters) + if A.settings.controllers is not null: + New_controllers = A.settings.controllers + else: + New_controllers = [M.caller] .... State after:: .... S with - canisters[CanisterId] = EmptyCanister - time[CanisterId] = CurrentTime - global_timer[CanisterId] = 0 - if A.settings.controllers is not null: - controllers[CanisterId] = A.settings.controllers + if A.specified_id is not null: + canister_id = A.specified_id else: - controllers[CanisterId] = [M.caller] + canister_id = CanisterId + canisters[canister_id] = EmptyCanister + time[canister_id] = CurrentTime + global_timer[canister_id] = 0 + controllers[canister_id] = New_controllers if A.settings.freezing_threshold is not null: - freezing_threshold[CanisterId] = A.settings.freezing_threshold + freezing_threshold[canister_id] = A.settings.freezing_threshold else: - freezing_threshold[CanisterId] = 2592000 + freezing_threshold[canister_id] = 2592000 if A.amount is not null: - balances[CanisterId] = A.amount + balances[canister_id] = A.amount else: - balances[CanisterId] = DEFAULT_PROVISIONAL_CYCLES_BALANCE - certified_data[CanisterId] = "" + balances[canister_id] = DEFAULT_PROVISIONAL_CYCLES_BALANCE + certified_data[canister_id] = "" + canister_history[canister_id] = { + total_num_changes = 1 + recent_changes = { + timestamp_nanos = CurrentTime + canister_version = 0 + origin = change_origin(M.caller, A.sender_canister_version, M.origin) + details = Creation { + controllers = New_controllers + } + } + } messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin - response = Reply (candid({canister_id = CanisterId})) + response = Reply (candid({canister_id = canister_id})) refunded_cycles = M.transferred_cycles } - canister_status[CanisterId] = Running - canister_version[CanisterId] = 0 + canister_status[canister_id] = Running + canister_version[canister_id] = 0 .... ==== IC Management Canister: Top up canister @@ -3855,17 +3971,25 @@ S with ==== Canister out of cycles -Once a canister runs out of cycles, its code is uninstalled (cf. <>) and the allocations are set to zero (NB: allocations are currently not modeled in the formal model): +Once a canister runs out of cycles, its code is uninstalled (cf. <>), the canister changes in the canister history are dropped (their total number is preserved), and the allocations are set to zero (NB: allocations are currently not modeled in the formal model): Conditions:: .... S.balances[CanisterId] = 0 + S.canister_history[CanisterId] = { + total_num_changes = N; + recent_changes = H; + } .... State after:: .... S with canisters[CanisterId] = EmptyCanister certified_data[CanisterId] = "" + canister_history[CanisterId] = { + total_num_changes = N; + recent_changes = []; + } canister_version[CanisterId] = S.canister_version[CanisterId] + 1 global_timer[CanisterId] = 0 @@ -3942,6 +4066,27 @@ S with canister_version[CanisterId] = N1 .... +==== Trimming canister history + +The list of canister changes can be trimmed, but the total number of recorded canister changes cannot be altered. At least 20 changes are guaranteed to remain in the list of changes. + +Conditions:: +.... + S.canister_history[CanisterId] = { + total_num_changes = N; + recent_changes = Older_changes · Newer_changes; + } + |Newer_changes| ≥ 20 +.... +State after:: +.... +S with + canister_history[CanisterId] = { + total_num_changes = N; + recent_changes = Newer_changes; + } +.... + ==== Query call Canister query calls to `/api/v2/canister//query` can be executed directly. They can only be executed against canisters which have a status of `Running` and are also not frozen. @@ -3964,6 +4109,7 @@ Conditions:: lookup(["time"], Cert) = Found S.system_time // or “recent enough” Env = { time = S.time[Q.receiver]; + controllers = S.controllers[Q.receiver]; global_timer = S.global_timer[Q.receiver]; balance = S.balances[Q.canister_id]; freezing_limit = freezing_limit(S, Q.canister_id); @@ -4117,7 +4263,7 @@ ExecutionState = { new_certified_data : NoCertifiedData | Blob; new_global_timer : NoGlobalTimer | Nat; ingress_filter : Accept | Reject; - context : I | G | U | Q | Ry | Rt | C | F | T; + context : I | G | U | Q | Ry | Rt | C | F | T | s; } .... @@ -4126,12 +4272,6 @@ Syntactically, we express this using an implicit argument of type `ref Execution WARNING: It is nonsensical to pass to an execution function a WebAssembly store `S` that comes from a different WebAssembly module than one defining the function. -==== The concrete `CanisterModule` - -Finally we can specify the abstract `CanisterModule` that models a concrete WebAssembly module. - -* The `initial_wasm_store` mentioned below is the store of the WebAssembly module after _instantiation_ (as per WebAssembly spec) of the WasmModule contained in the <>, including executing a potential `(start)` function. - * For more convenience when creating a new `ExecutionState`, we define the following partial records: + .... @@ -4140,6 +4280,7 @@ empty_params = { caller = NoCaller; reject_code = 0; reject_message = ""; + sysenv = (undefined); cycles_refunded = 0; method_name = NoText; } @@ -4162,41 +4303,93 @@ empty_execution_state = { } .... +==== The concrete `CanisterModule` -* The `init` field of the `CanisterModule` is defined as follows: +Finally we can specify the abstract `CanisterModule` that models a concrete WebAssembly module. + +* The `initial_wasm_store` mentioned below is the store of the WebAssembly module after _instantiation_ (as per WebAssembly spec) of the WasmModule contained in the <>, before executing a potential `(start)` function. + +* We define a helper function ++ +.... +start : (CanisterId) -> Trap { cycles_used : Nat; } | Return { + new_state : WasmState; + cycles_used : Nat; + } +.... ++ +modelling execution of a potential `(start)` function. + -If the WebAssembly module does not export a function called under the name `canister_init`, then the argument blob is ignored and the `initial_wasm_store` is returned: +If the WebAssembly module does not export a function called under the name `start`, then + .... -init = λ (self_id, arg, caller, sysenv) → +start = λ (self_id) → Return { - new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" }; - new_certified_data = NoCertifiedData; - new_global_timer = NoGlobalTimer; + new_state = {store = initial_wasm_store; self_id = self_id; stable_mem = ""}; cycles_used = 0; } .... + -Otherwise, if the WebAssembly module exports a function `func` under the name `canister_init`, it is +Otherwise, if the WebAssembly module exports a function `func` under the name `start`, it is + .... -init = λ (self_id, arg, caller, sysenv) → +start = λ (self_id) → let es = ref {empty_execution_state with - wasm_state = { store = initial_wasm_store; self_id = self_id; stable_mem = "" } - params = empty_params with { arg = arg; caller = caller; sysenv } - balance = sysenv.balance - context = I - } + wasm_state = {store = initial_wasm_store; self_id = self_id; stable_mem = ""}; + context = s; + } try func() with Trap then Trap {cycles_used = es.cycles_used;} Return { new_state = es.wasm_state; - new_certified_data = es.new_certified_data; - new_global_timer = es.new_global_timer; cycles_used = es.cycles_used; } .... + -This formulation checks afterwards that the system calls `call.perform` or `msg.reply` were not invoked; an implementation can of course trap as soon as these system calls are invoked. +Note that `params` are undefined in the `(start)` function's execution state which is fine because the System API does not have access to that part of the execution state during the execution of the `(start)` function. + +* The `init` field of the `CanisterModule` is defined as follows: ++ +If the WebAssembly module does not export a function called under the name `canister_init`, then ++ +.... +init = λ (self_id, arg, caller, sysenv) → + match start(self_id) with + Trap trap → Trap trap + Return res → Return { + new_state = res.wasm_state; + new_certified_data = NoCertifiedData; + new_global_timer = NoGlobalTimer; + cycles_used = res.cycles_used; + } +.... ++ +Otherwise, if the WebAssembly module exports a function `func` under the name `canister_init`, it is ++ +.... +init = λ (self_id, arg, caller, sysenv) → + match start(self_id) with + Trap trap → Trap trap + Return res → + let es = ref {empty_execution_state with + wasm_state = res.wasm_state + params = empty_params with { + arg = arg; + caller = caller; + sysenv = sysenv with { + balance = sysenv.balance - res.cycles_used + } + } + balance = sysenv.balance - res.cycles_used + context = I + } + try func() with Trap then Trap {cycles_used = res.cycles_used + es.cycles_used;} + Return { + new_state = es.wasm_state; + new_certified_data = es.new_certified_data; + new_global_timer = es.new_global_timer; + cycles_used = res.cycles_used + es.cycles_used; + } +.... * The `pre_upgrade` field of the `CanisterModule` is defined as follows: + @@ -4293,8 +4486,6 @@ query_methods[method] = λ (arg, caller, sysenv) → λ wasm_state → } .... + -This formulation checks afterwards that the system call `ic0.call_perform` was not invoked; an implementation can of course trap already when these system calls have been invoked. -+ By construction, the (possibly modified) `es.wasm_state` is discarded. * The function `heartbeat` of the `CanisterModule` is defined if the WebAssembly program exports a function `func` named `canister_heartbeat`, and has value @@ -4532,19 +4723,6 @@ ic0.msg_cycles_refunded128(dst : i32) = let amount = es.params.cycles_refunded copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) -ic0.accept_message() = - if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} - if es.ingress_filter = Accept then Trap {cycles_used = es.cycles_used;} - es.ingress_filter = Accept - -ic0.msg_method_name_size() : i32 = - if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} - return |es.method_name| - -ic0.msg_method_name_copy(dst : i32, offset : i32, size : i32) : i32 = - if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} - copy_to_canister(dst, offset, size, es.params.method_name) - ic0.msg_cycles_accept(max_amount : i64) : i64 = if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} let amount = min(max_amount, es.cycles_available) @@ -4563,28 +4741,47 @@ ic0.msg_cycles_accept128(max_amount_high : i64, max_amount_low : i64, dst : copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) ic0.canister_self_size() : i32 = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} return |es.wasm_state.self_id| ic0.canister_self_copy(dst:i32, offset:i32, size:i32) = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.wasm_state.self_id) ic0.canister_cycle_balance() : i64 = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} if es.balance >= 2^64^ then Trap {cycles_used = es.cycles_used;} return es.balance ic0.canister_cycles_balance128(dst : i32) = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} let amount = es.balance copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) ic0.canister_status() : i32 = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} match es.params.sysenv.canister_status with Running -> return 1 Stopping -> return 2 Stopped -> return 3 ic0.canister_version() : i64 = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} return es.params.sysenv.canister_version +ic0.msg_method_name_size() : i32 = + if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} + return |es.method_name| + +ic0.msg_method_name_copy(dst : i32, offset : i32, size : i32) : i32 = + if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} + copy_to_canister(dst, offset, size, es.params.method_name) + +ic0.accept_message() = + if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter = Accept then Trap {cycles_used = es.cycles_used;} + es.ingress_filter = Accept + ic0.call_new( callee_src : i32, callee_size : i32, @@ -4623,11 +4820,6 @@ ic0.call_new( }; } -ic0.call_data_append (src : i32, size : i32) = - if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} - if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} - es.pending_call.arg := es.pending_call.arg · copy_from_canister(src, size) - ic0.call_on_cleanup (fun : i32, env : i32) = if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} if fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} @@ -4636,6 +4828,11 @@ ic0.call_on_cleanup (fun : i32, env : i32) = if es.pending_call.callback.on_cleanup ≠ NoClosure then Trap {cycles_used = es.cycles_used;} es.pending_call.callback.on_cleanup := Closure { fun = fun; env = env} +ic0.call_data_append (src : i32, size : i32) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + es.pending_call.arg := es.pending_call.arg · copy_from_canister(src, size) + ic0.call_cycles_add(amount : i64) = if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} @@ -4727,37 +4924,50 @@ ic0.stable64_read(dst : i64, offset : i64, size : i64) es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] -ic0.time() : i32 = - return es.params.sysenv.time - -ic0.global_timer_set(timestamp: i64) : i64 = - if es.context ∉ {I, U, Ry, Rt, C, T} then Trap {cycles_used = es.cycles_used;} - let prev_global_timer = es.new_global_timer - es.new_global_timer := timestamp - if prev_global_timer = NoGlobalTimer - then return es.params.sysenv.global_timer - else return prev_global_timer - ic0.certified_data_set(src: i32, size: i32) = if es.context ∉ {I, G, U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} es.new_certified_data := es.wasm_state[src..src+size] ic0.data_certificate_present() : i32 = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} if es.params.sysenv.certificate = NoCertificate then return 0 else return 1 ic0.data_certificate_size() : i32 = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} if es.params.sysenv.certificate = NoCertificate then Trap {cycles_used = es.cycles_used;} return |es.params.sysenv.certificate| ic0.data_certificate_copy(dst: i32, offset: i32, size: i32) = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} if es.params.sysenv.certificate = NoCertificate then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.sysenv.certificate) +ic0.time() : i32 = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} + return es.params.sysenv.time + +ic0.global_timer_set(timestamp: i64) : i64 = + if es.context ∉ {I, G, U, Ry, Rt, C, T} then Trap {cycles_used = es.cycles_used;} + let prev_global_timer = es.new_global_timer + es.new_global_timer := timestamp + if prev_global_timer = NoGlobalTimer + then return es.params.sysenv.global_timer + else return prev_global_timer + ic0.performance_counter(counter_type : i32) : i64 = arbitrary() +ic0.is_controller(src: i32, size: i32) : (result: i32) = + bytes = copy_from_canister(src, size) + if bytes encode a principal then + if bytes ∉ es.params.sysenv.controllers + then return 0 + else return 1 + else + Trap {cycles_used = es.cycles_used;} + ic0.debug_print(src : i32, size : i32) = return diff --git a/spec/index.md b/spec/index.md new file mode 100644 index 000000000..fc9c597b1 --- /dev/null +++ b/spec/index.md @@ -0,0 +1,5337 @@ +import Changelog from './_attachments/interface-spec-changelog.md'; + +# The Internet Computer Interface Specification + + +## Introduction + +Welcome to *the Internet Computer*! We speak of "the" Internet Computer, because although under the hood a large number of physical computers are working together in a blockchain protocol, in the end we have the appearance of a single, shared, secure and world-wide accessible computer. Developers who want to build decentralized applications (or *dapps* for short) that run on the Internet Computer blockchain and end-users who want to use those dapps need to know very little, if anything, about the underlying protocol. However, knowing some details about the interfaces that the Internet Computer exposes can allow interested developers and architects to take fuller advantages of the unique features that the Internet Computer provides. + +### Target audience + +This document describes this *external* view of the Internet Computer, i.e. the low-level interfaces it provides to dapp developers and users, and what will happen when they use these interfaces. + +:::note + +While this document describes the external interface and behavior of the Internet Computer, it is not intended as end-user or end-developer documentation. Most developers will interact with the Internet Computer through additional tooling like the SDK, Canister Development Kits and Motoko. Please see the [developer docs](../home.mdx) for suitable documentation. + +::: + +The target audience of this document are + +- those who use these low-level interfaces (e.g. implement agents, canister developments kits, emulators, other tooling). + +- those who implement these low-level interfaces (e.g. developers of the Internet Computer implementation) + +- those who want to understand the intricacies of the Internet Computer's behavior in great detail (e.g. to do a security analysis) + +:::note + +This document is a rigorous, technically dense reference. It is not an introduction to the Internet Computer, and as such most useful to those who understand the high-level concepts. Please see more high-level documentation first. + +::: + +### Scope of this document + +If you think of the Internet Computer as a distributed engine that executes WebAssembly-based dapps, then this document describes exclusively the aspect of executing those dapps. To the extent possible, this document will *not* talk about consensus protocols, nodes, subnets, orthogonal persistence or governance. + +This document tries to be implementation agnostic: It would apply just as well to a (hypothetical) compatible reimplementation of the Internet Computer. This implies that this document does not cover interfaces towards those running the Internet Computer (e.g. data center operators, protocol developers, governance users), as topics like node update, monitoring, logging are inherently tied to the actual *implementation* and its architecture. + +### Overview of the Internet Computer + +Dapps on the Internet Computer, or *IC* for short, are implemented as *canister smart contracts*, or *canisters* for short. If you want to build on the Internet Computer as a dapp developer, you first create a *canister module* that contains the WebAssembly code and configuration for your dapp, and deploy it using the [HTTPS interface](#http-interface). You can create canister modules using the Motoko language and the SDK, which is more convenient. If you want to use your own tooling, however, then this document describes [what a canister module looks like](#canister-module-format) and how the [WebAssembly code can interact with the IC](#system-api). + +Once your dapp is running on the Internet Computer, it is a canister smart contract, and users can interact with it. They can use the [HTTPS interface](#http-interface) to interact with the canister according to the [System API](#system-api). + +The user can also use the HTTPS interface to issue read-only queries, which are faster, but cannot change the state of a canister. + +```plantuml + actor Developer + actor User + participant "Internet Computer" as IC + participant "Canister 1" as Can1 + Developer -> IC : /submit create canister + create Can1 + IC -> Can1 : create + Developer <-- IC : canister-id=1 + Developer -> IC : /submit install module + IC -> Can1 : initialize + ||| + User -> IC : /submit call "hello" + IC -> Can1 : hello + return "Hello world!" + User <-- IC : "Hello World!" +``` +**A typical use of the Internet Computer. (This is a simplified view; some of the arrows represent multiple interaction steps or polling.)** + +Sections "[HTTPS Interface](#http-interface)" and "[Canister interface (System API)](#system-api)" describe these interfaces, together with a brief description of what they do. Afterwards, you will find a [more formal description](#abstract-behavior) of the Internet Computer that describes its abstract behavior with more rigor. + +### Nomenclature + +To get some consistency in this document, we try to use the following terms with precision: + +We avoid the term "client", as it could be the client of the Internet Computer or the client inside the distributed network that makes up the Internet Computer. Instead, we use the term *user* to denote the external entity interacting with the Internet Computer, even if in most cases it will be some code (sometimes called "agent") acting on behalf of a (human) user. + +The public entry points of canisters are called *methods*. Methods can be declared to be either *update methods* (state mutation is preserved) or *query methods* (state mutation is discarded, no further calls can be made). + +Methods can be *called*, from *caller* to *callee*, and will eventually incur a *response* which is either a *reply* or a *reject*. A method may have *parameters*, which are provided with concrete *arguments* in a method call. + +External calls can be update calls, which can call both kinds of methods, and query calls, which can *only* call query methods. Inter-canister calls can also call both kinds of methods. Note that calls from a canister to itself also count as "inter-canister". + +Internally, a call or a response is transmitted as a *message* from a *sender* to a *receiver*. Messages do not have a response. + +WebAssembly *functions* are exported by the WebAssembly module or provided by the System API. These are *invoked* and can either *trap* or *return*, possibly with a return value. Functions, too, have parameters and take arguments. + +External *users* interact with the Internet Computer by issuing *requests* on the HTTPS interface. Requests have responses which can either be replies or rejects. Some requests cause internal messages to be created. + +Canisters and users are identified by a *principal*, sometimes also called an *id*. + +## Pervasive concepts + +Before going into the details of the four public interfaces described in this document (namely the agent-facing [HTTPS interface](#http-interface), the canister-facing [System API](#system-api), the [virtual Management canister](#ic-management-canister) and the [System State Tree](#state-tree)), this section introduces some concepts that transcend multiple interfaces. + +### Unspecified constants and limits + +This specification may refer to certain constants and limits without specifying their concrete value (yet), i.e. they are implementation defined. Many are resource limits which are relevant only to specify the error-handling behavior of the IC (which, as mentioned above, is also not yet precisely described in this document). This list is not complete. + +- `MAX_CYCLES_PER_MESSAGE` + + Amount of cycles that a canister has to have before a message is attempted to be executed, which is deducted from the canister balance before message execution. See [Message execution](#rule-message-execution). + +- `MAX_CYCLES_PER_RESPONSE` + + Amount of cycles that the IC sets aside when a canister performs a call. This is used to pay for processing the response message, and unused cycles after the execution of the response are refunded. See [Message execution](#rule-message-execution). + +- `DEFAULT_PROVISIONAL_CYCLES_BALANCE` + + Amount of cycles allocated to a new canister by default, if not explicitly specified. See [IC method](#ic-provisional_create_canister_with_cycles). + +### Principals {#principal} + +Principals are generic identifiers for canisters, users and possibly other concepts in the future. As far as most uses of the IC are concerned they are *opaque* binary blobs with a length between 0 and 29 bytes, and there is intentionally no mechanism to tell canister ids and user ids apart. + +There is, however, some structure to them to encode specific authentication and authorization behavior. + +#### Special forms of Principals {#id-classes} + +In this section, `H` denotes SHA-224, `·` denotes blob concatenation and `|p|` denotes the length of `p` in bytes, encoded as a single byte. + +There are several classes of ids: + +1. *Opaque ids*. + + These are always generated by the IC and have no structure of interest outside of it. + +:::note + +Typically, these end with the byte `0x01`, but users of the IC should not need to care about that. + +::: + +2. *Self-authenticating ids*. + + These have the form `H(public_key) · 0x02` (29 bytes). + + An external user can use these ids as the `sender` of a request if they own the corresponding private key. The public key uses one of the encodings described in [Signatures](#signatures). + +3. *Derived ids* + + These have the form `H(|registering_principal| · registering_principal · derivation_nonce) · 0x03` (29 bytes). + + These ids are treated specially when an id needs to be registered. In such a request, whoever requests an id can provide a `derivation_nonce`. By hashing that together with the principal of the caller, every principal has a space of ids that only they can register ids from. + +:::note + +Derived IDs are currently not explicitly used in this document, but they may be used internally or in the future. + +::: + +4. *Anonymous id* + + This has the form `0x04`, and is used for the anonymous caller. It can be used in call and query requests without a signature. + +5. *Reserved ids* + + These have the form of `blob · 0x7f`, `0 ≤ |blob| < 29`. + + These ids can be useful for applications that want to re-use the [Textual representation of principals](#textual-ids) but want to indicate explicitly that the blob does not address any canisters or a user. + +When the IC creates a *fresh* id, it never creates a self-authenticating id, reserved id, an anonymous id or an id derived from what could be a canister or user. + +#### Textual representation of principals {#textual-ids} + +We specify a *canonical textual format* that is recommended whenever principals need to be printed or read in textual format, e.g. in log messages, transactions browser, command line tools, source code. + +The textual representation of a blob `b` is `Grouped(Base32(CRC32(b) · b))` where + +- `CRC32` is a four byte check sequence, calculated as defined by ISO 3309, ITU-T V.42, and [elsewhere](https://www.w3.org/TR/2003/REC-PNG-20031110/#5CRC-algorithm), and stored as big-endian, i.e., the most significant byte comes first and then the less significant bytes come in descending order of significance (MSB B2 B1 LSB). + +- `Base32` is the Base32 encoding as defined in [RFC 4648](https://tools.ietf.org/html/rfc4648#section-6), with no padding character added. + +- The middle dot denotes concatenation. + +- `Grouped` takes an ASCII string and inserts the separator `-` (dash) every 5 characters. The last group may contain less than 5 characters. A separator never appears at the beginning or end. + +The textual representation is conventionally printed with *lower case letters*, but parsed case-insensitively. + +Because the maximum size of a principal is 29 bytes, the textual representation will be no longer than 63 characters (10 times 5 plus 3 characters with 10 separators in between them). + +:::tip + +The canister with id `0xABCD01` has check sequence `0x233FF206` ([online calculator](https://crccalc.com/?crc=ABCD01&method=crc32&datatype=hex&outtype=hex)); the final id is thus `em77e-bvlzu-aq`. + +Example encoding from hex, and decoding to hex, in bash (the following can be pasted into a terminal as is): + + function textual_encode() { + ( echo "$1" | xxd -r -p | /usr/bin/crc32 /dev/stdin; echo -n "$1" ) | + xxd -r -p | base32 | tr A-Z a-z | + tr -d = | fold -w5 | paste -sd'-' - + } + + function textual_decode() { + echo -n "$1" | tr -d - | tr a-z A-Z | + fold -w 8 | xargs -n1 printf '%-8s' | tr ' ' = | + base32 -d | xxd -p | tr -d '\n' | cut -b9- | tr a-z A-Z + } + +::: + +### Canister lifecycle {#canister-lifecycle} + +Dapps on the Internet Computer are called *canisters*. Conceptually, they consist of the following pieces of state: + +- A canister id (a [principal](#principal)) + +- Their *controllers* (a possibly empty list of [principal](#principal)) + +- A cycle balance + +- The *canister status*, which is one of `running`, `stopping` or `stopped`. + +- Resource reservations + +A canister can be *empty* (e.g. directly after creation) or *non-empty*. A non-empty canister also has + +- code, in the form of a canister module + +- state (memories, globals etc.) + +- possibly further data that is specific to the implementation of the IC (e.g. queues) + +Canisters are empty after creation and uninstallation, and become non-empty through [code installation](#ic-install_code). + +If an empty canister receives a response, that response is dropped, as if the canister trapped when processing the response. The cycles set aside for its processing and the cycles carried on the responses are added to the canister's *cycles* balance. + +#### Canister cycles {#canister-cycles} + +The IC relies on *cycles*, a utility token, to manage its resources. A canister pays for the resources it uses from its *cycle balance*. The *cycle\_balance* is stored as 128-bit unsigned integers and operations on them are saturating. In particular, if *cycles* are added to a canister that would bring its total balance beyond 2128-1, then the balance will be capped at 2128-1 and any additional cycles will be lost. + +When the cycle balance of a canister falls to zero, the canister is *deallocated*. This has the same effect as + +- uninstalling the canister (as described in [IC method](#ic-uninstall_code)) + +- setting all resource reservations to zero + +Afterwards the canister is empty. It can be reinstalled after topping up its balance. + +:::note + +Once the IC frees the resources of a canister, its id, *cycles* balance, *controllers*, canister *version*, and the total number of canister changes are preserved on the IC for a minimum of 10 years. What happens to the canister after this period is currently unspecified. + +::: + +#### Canister status {#canister-status} + +The canister status can be used to control whether the canister is processing calls: + +- In status `running`, calls to the canister are processed as normal. + +- In status `stopping`, calls to the canister are rejected by the IC with reject code `CANISTER_ERROR` (5), but responses to the canister are processed as normal. + +- In status `stopped`, calls to the canister are rejected by the IC with reject code `CANISTER_ERROR` (5), and there are no outstanding responses. + +In all cases, calls to the [management canister](#ic-management-canister) are processed, regardless of the state of the managed canister. + +The controllers of the canister can initiate transitions between these states using [`stop_canister`](#ic-stop_canister) and [`start_canister`](#ic-start_canister), and query the state using [`canister_status`](#ic-canister_status) (NB: this call returns additional information, such as the cycle balance of the canister). The canister itself can also query its state using [`ic0.canister_status`](#system-api-canister-status). + +:::note + +This status is orthogonal to whether a canister is empty or not: an empty canister can be in status `running`. Calls to such a canister are still rejected by the IC with an HTTP error, but because the canister is empty. + +::: + +### Signatures {#signatures} + +Digital signature schemes are used for authenticating messages in various parts of the IC infrastructure. Signatures are domain separated, which means that every message is prefixed with a byte string that is unique to the purpose of the signature. + +The IC supports multiple signature schemes, with details given in the following subsections. For each scheme, we specify the data encoded in the public key (which is always DER-encoded, and indicates the scheme to use) as well as the form of the signatures (which are opaque blobs for the purposes of the rest of this specification). + +In all cases, the signed *payload* is the concatenation of the domain separator and the message. All uses of signatures in this specification indicate a domain separator, to uniquely identify the purpose of the signature. The domain separators are prefix-free by construction, as their first byte indicates their length. + +#### Ed25519 and ECDSA signatures {#ecdsa} + +Plain signatures are supported for the schemes + +- [**Ed25519**](https://ed25519.cr.yp.to/index.html) or + +- [**ECDSA**](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf) on curve P-256 (also known as `secp256r1`), using SHA-256 as hash function, as well as on the Koblitz curve `secp256k1`. + +- Public keys must be valid for signature schemes Ed25519 or ECDSA and are encoded as DER. + + - See [RFC 8410](https://tools.ietf.org/html/rfc8410) for DER encoding of Ed25519 public keys. + + - See [RFC 5480](https://tools.ietf.org/rfc/rfc5480) for DER encoding of ECDSA public keys; the DER encoding must not specify a hash function. For curve `secp256k1`, the OID 1.3.132.0.10 is used. The points must be specified in uncompressed form (i.e. `0x04` followed by the big-endian 32-byte encodings of `x` and `y`). + +- The signatures are encoded as the concatenation of the 32-byte big endian encodings of the two values *r* and *s*. + +#### Web Authentication {#webauthn} + +The allowed signature schemes for web authentication are + +- [**ECDSA**](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf) on curve P-256 (also known as `secp256r1`), using SHA-256 as hash function. + +- [**RSA PKCS\#1v1.5 (RSASSA-PKCS1-v1\_5)**](https://datatracker.ietf.org/doc/html/rfc8017#section-8.2), using SHA-256 as hash function. + +The signature is calculated by using the payload as the challenge in the web authentication assertion. + +The signature is checked by verifying that the `challenge` field contains the [base64url encoding](https://tools.ietf.org/html/rfc4648#section-5) of the payload, and that `signature` verifies on `authenticatorData · SHA-256(utf8(clientDataJSON))`, as specified in the [WebAuthn w3c recommendation](https://www.w3.org/TR/webauthn/#op-get-assertion). + +- The public key is encoded as a DER-wrapped COSE key. + + It uses the `SubjectPublicKeyInfo` type used for other types of public keys (see, e.g., [RFC 8410, Section 4](https://tools.ietf.org/html/rfc8410#section-4)), with OID 1.3.6.1.4.1.56387.1.1 (iso.org.dod.internet.private.enterprise.dfinity.mechanisms.der-wrapped-cose). The `BIT STRING` field `subjectPublicKey` contains the COSE encoding. See [WebAuthn w3c recommendation](https://www.w3.org/TR/webauthn/#sctn-encoded-credPubKey-examples) or [RFC 8152](https://tools.ietf.org/html/rfc8152#section-13.1) for details on the COSE encoding. + +:::tip + +A DER wrapping of a COSE key is shown below. It can be parsed via the command `sed "s/#.*//" | xxd -r -p | openssl asn1parse -inform der`. + + 30 5E # SEQUENCE of length 94 bytes + 30 0C # SEQUENCE of length 12 bytes + 06 0A 2B 06 01 04 01 83 B8 43 01 01 # OID 1.3.6.1.4.1.56387.1.1 + 03 4E 00 # BIT STRING encoding of length 78, + A501 0203 2620 0121 5820 7FFD 8363 2072 # length is at byte boundary + FD1B FEAF 3FBA A431 46E0 EF95 C3F5 5E39 # contents is a valid COSE key + 94A4 1BBF 2B51 74D7 71DA 2258 2032 497E # with ECDSA on curve P-256 + ED0A 7F6F 0009 2876 5B83 1816 2CFD 80A9 + 4E52 5A6A 368C 2363 063D 04E6 ED + +You can also view the wrapping in [an online ASN.1 JavaScript decoder](https://lapo.it/asn1js/#MF4wDAYKKwYBBAGDuEMBAQNOAKUBAgMmIAEhWCB__YNjIHL9G_6vP7qkMUbg75XD9V45lKQbvytRdNdx2iJYIDJJfu0Kf28ACSh2W4MYFiz9gKlOUlpqNowjYwY9BObt). + +::: + +- The signature is a CBOR (see [CBOR](#cbor)) value consisting of a data item with major type 6 ("Semantic tag") and tag value `55799`, followed by a map with three mandatory fields: + + - `authenticator_data` (`blob`): WebAuthn authenticator data. + + - `client_data_json` (`text`): WebAuthn client data in JSON representation. + + - `signature` (`blob`): Signature as specified in the [WebAuthn w3c recommendation](https://www.w3.org/TR/webauthn/#signature-attestation-types), which means DER encoding in the case of an ECDSA signature. + +#### Canister signatures {#canister-signatures} + +The IC also supports a scheme where a canister can sign a payload by declaring a special "certified variable". + +This section makes forward references to other concepts in this document, in particular the section [Certification](#certification). + +- The public key is a DER-wrapped structure that indicates the *signing canister*, and includes a freely choosable seed. Each choice of seed yields a distinct public key for the canister, and the canister can choose to encode information, such as a user id, in the seed. + + More concretely, it uses the `SubjectPublicKeyInfo` type used for other types of public keys (see, e.g., [RFC 8410, Section 4](https://tools.ietf.org/html/rfc8410#section-4)), with OID 1.3.6.1.4.1.56387.1.2 (iso.org.dod.internet.private.enterprise.dfinity.mechanisms.canister-signature). + + The `BIT STRING` field `subjectPublicKey` is the blob `|signing_canister_id| · signing_canister_id · seed`, where `|signing_canister_id|` is the one-byte encoding of the the length of the `signing_canister_id` and `·` denotes blob concatenation. + +- The signature is a CBOR (see [CBOR](#cbor)) value consisting of a data item with major type 6 ("Semantic tag") and tag value `55799`, followed by a map with two mandatory fields: + + - `certificate` (`blob`): A CBOR-encoded certificate as per [Encoding of certificates](#certification-encoding). + + - `tree` (`hash-tree`): A hash tree as per [Encoding of certificates](#certification-encoding). + +- Given a payload together with public key and signature in the format described above the signature can be verified by checking the following two conditions: + + - The `certificate` must be a valid certificate as described in [Certification](#certification), with + + lookup(/canister//certified_data, certificate.tree) = Found (reconstruct(tree)) + + where `signing_canister_id` is the id of the signing canister and `reconstruct` is a function that computes a root-hash for the tree. + + - If the `certificate` includes subnet delegations (possibly nested), then the `signing_canister_id` must be included in each delegation's canister id range (see [Delegation](#certification-delegation)). + + - The `tree` must be a `well_formed` tree with + + lookup(/sig//, tree) = Found "" + + where `s` is the SHA-256 hash of the `seed` used in the public key and `m` is the SHA-256 hash of the payload. + +### Supplementary Technologies {#supplementary-technologies} + +#### CBOR {#cbor} + +[Concise Binary Object Representation (CBOR)](https://www.rfc-editor.org/rfc/rfc8949) is a data format with a small code footprint, small message size and an extensible interface. CBOR is used extensively throughout the Internet Computer as the primary format for data exchange between components within the system. + +[cbor.io](https://cbor.io) and [wikipedia.org](https://en.wikipedia.org/wiki/CBOR) contain a lot of helpful background information and relevant tools. [cbor.me](https://cbor.me) in particular, is very helpful for converting between CBOR hex and diagnostic information. + +For example, the following CBOR hex: + + 82 61 61 a1 61 62 61 63 + +Can be converted into the following CBOR diagnostic format: + + ["a", {"b": "c"}] + +Particular concepts to note from the spec are: + +- [Specification of the CBOR Encoding](https://www.rfc-editor.org/rfc/rfc8949#name-specification-of-the-cbor-e) + +- [CBOR Major Types](https://www.rfc-editor.org/rfc/rfc8949#name-major-types) + +- [CBOR Self-Describe](https://www.rfc-editor.org/rfc/rfc8949#self-describe) + +#### CDDL {#cddl} + +The [Concise Data Definition Language (CDDL)](https://tools.ietf.org/html/rfc8610) is a data description language for CBOR. It is used at various points throughout this document to describe how certain data structures are encoded with CBOR. + +## The system state tree {#state-tree} + +Parts of the IC state are publicly exposed (e.g. via [Request: Read state](#http-read-state) or [Certified data](#system-api-certified-data)) in a verified way (see [Certification](#certification) for the machinery for certifying). This section describes the content of this system state abstractly. + +Conceptually, the system state is a tree with labeled children, and values in the leaves. Equivalently, the system state is a mapping from paths (sequences of labels) to values, where the domain is prefix-free. + +Labels are always blobs (but often with a human readable representation). In this document, paths are written suggestively with slashes as separators; the actual encoding is not actually using slashes as delimiters, and labels may contain the 0x2F byte (ASCII `/`) just fine. Values are either natural numbers, text values or blob values. + +This section specifies the publicly relevant paths in the tree. + +### Time {#state-tree-time} + +- `/time` (natural): + + All partial state trees include a timestamp, indicating the time at which the state is current. + +### Subnet information {#state-tree-subnet} + +The state tree contains information about the topology of the Internet Computer. + +- `/subnet//public_key` (blob) + + The public key of the subnet (a DER-encoded BLS key, see [Certification](#certification)) + +- `/subnet//canister_ranges` (blob) + + The set of canister ids assigned to this subnet, represented as a list of closed intervals of canister ids, ordered lexicographically, and encoded as CBOR (see [CBOR](#cbor)) according to this CDDL (see [CDDL](#cddl)): + + canister_ranges = tagged<[*canister_range]> + canister_range = [principal principal] + principal = bytes .size (0..29) + tagged = #6.55799(t) ; the CBOR tag + +:::note + +Because this uses the lexicographic ordering of princpials, and the byte distinguishing the various classes of ids is at the *end*, this range by construction conceptually includes principals of various classes. This specification needs to take care that the fact that principals that are not canisters may appear in these ranges does not cause confusion. + +::: + +### Request status {#state-tree-request-status} + +For each asynchronous request known to the Internet Computer, its status is in a subtree at `/request_status/`. Please see [Overview of canister calling](#http-call-overview) for more details on how asynchronous requests work. + +- `/request_status//status` (text) + + One of `received`, `processing`, `replied`, `rejected` or `done`, see [Overview of canister calling](#http-call-overview) for more details on what each status means. + +- `/request_status//reply` (blob) + + If the status is `replied`, then this path contains the reply blob, else it is not present. + +- `/request_status//reject_code` (natural) + + If the status is `rejected`, then this path contains the reject code (see [Reject codes](#reject-codes)), else it is not present. + +- `/request_status//reject_message` (text) + + If the status is `rejected`, then this path contains a textual diagnostic message, else it is not present. + +- `/request_status//error_code` (text) + + If the status is `rejected`, then this path might be present and contain an implementation-specific error code (see [Error codes](#error-codes)), else it is not present. + +:::note + +Immediately after submitting a request, the request may not show up yet as the Internet Computer is still working on accepting the request as pending. + +::: + +:::note + +Request statuses will not actually be kept around indefinitely, and eventually the Internet Computer forgets about the request. This will happen no sooner than the request's expiry time, so that replay attacks are prevented. + +::: + +### Certified data {#state-tree-certified-data} + +- `/canister//certified_data` (blob): + + The certified data of the canister with the given id, see [Certified data](#system-api-certified-data). + +### Canister information {#state-tree-canister-information} + +Users have the ability to learn about the hash of the canister's module, its current controllers, and metadata in a certified way. + +- `/canister//module_hash` (blob): + + If the canister is empty, this path does not exist. If the canister is not empty, it exists and contains the SHA256 hash of the currently installed canister module. Cf. [IC method](#ic-canister_status). + +- `/canister//controllers` (blob): + + The current controllers of the canister. The value consists of a CBOR (see [CBOR](#cbor)) data item with major type 6 ("Semantic tag") and tag value `55799`, followed by an array of principals in their binary form (CDDL `#6.55799([* bytes .size (0..29)])`, see [CDDL](#cddl)). + +- `/canister//metadata/` (blob): + + If the canister has a [custom section](https://webassembly.github.io/spec/core/binary/modules.html#custom-section) called `icp:public ` or `icp:private `, this path contains the content of the custom section. Otherwise, this path does not exist. + + It is recommended for the canister to have a custom section called "icp:public candid:service", which contains the UTF-8 encoding of [the Candid interface](https://github.com/dfinity/candid/blob/master/spec/Candid.md#core-grammar) for the canister. + +## HTTPS Interface {#http-interface} + +The concrete mechanism that users use to send requests to the Internet Computer is via an HTTPS API, which exposes three endpoints to handle interactions, plus one for diagnostics: + +- At `/api/v2/canister//call` the user can submit (asynchronous, potentially state-changing) calls. + +- At `/api/v2/canister//read_state` the user can read various information about the state of the Internet Computer. In particular, they can poll for the status of a call here. + +- At `/api/v2/canister//query` the user can perform (synchronous, non-state-changing) query calls. + +- At `/api/v2/status` the user can retrieve status information about the Internet Computer. + +In these paths, the `` is the [textual representation](#textual-ids) of the [*effective* canister id](#http-effective-canister-id). + +Requests to `/api/v2/canister//call`, `/api/v2/canister//read_state` and `/api/v2/canister//query` are POST requests with a CBOR-encoded request body, which consists of a authentication envelope (as per [Authentication](#authentication)) and request-specific content as described below. + +:::note + +This document does not yet explain how to find the location and port of the Internet Computer. + +::: + +### Overview of canister calling {#http-call-overview} + +Users interact with the Internet Computer by calling canisters. By the very nature of a blockchain protocol, they cannot be acted upon immediately, but only with a delay. Moreover, the actual node that the user talks to may not be honest or, for other reasons, may fail to get the request on the way. This implies the following high-level workflow: + +1. A user submits a call via the [HTTPS Interface](#http-interface). No useful information is returned in the immediate response (as such information cannot be trustworthy anyways). + +2. For a certain amount of time, the IC behaves as if it does not know about the call. + +3. The IC asks the targeted canister if it is willing to accept this message and be charged for the expense of processing it. This uses the [Ingress message inspection](#system-api-inspect-message) API for normal calls. For calls to the management canister, the rules in [The IC management canister](#ic-management-canister) apply. + +4. At some point, the IC may accept the call for processing and set its status to `received`. This indicates that the IC as a whole has received the call and plans on processing it (although it may still not get processed if the IC is under high load). Furthermore, the user should also be able to ask any endpoint about the status of the pending call. + +5. Once it is clear that the call will be acted upon (sufficient resources, call not yet expired), the status changes to `processing`. Now the user has the guarantee that the request will have an effect, e.g. it will reach the target canister. + +6. The IC is processing the call. For some calls this may be atomic, for others this involves multiple internal steps. + +7. Eventually, a response will be produced, and can be retrieved for a certain amount of time. The response is either a `reply`, indicating success, or a `reject`, indicating some form of error. + +8. In the case that the call has been retained for long enough, but the request has not expired yet, the IC can forget the response data and only remember the call as `done`, to prevent a replay attack. + +9. Once the expiry time is past, the IC can prune the call and its response, and completely forget about it. + +This yields the following interaction diagram: +```plantuml + (*) --> "User creates call" #DDDDDD + --> "Submitted to node\n(with 202 response)" as submit #DDDDDD + --> "received" + --> "processing" + if "" as X then + --> "replied" + --> "done" + else + --> "rejected (canister)" + --> "done" + + "X" --> "rejected (system)" + "received" --> "rejected (system)" + --> "done" + + "received" --> "pruned" #DDDDDD + "submit" --> "dropped" #DDDDDD + "done" --> "pruned" #DDDDDD + + endif +``` +State transitions may be instantaneous and not always externally visible. For example, the state of a request may move from `received` via `processing` to `replied` in one go. Similarly, the IC may not implement the `done` state at all, and keep calls in state `replied`/`rejected` until they are pruned. + +All gray states are *not* explicitly represented in the state of the IC, and are indistinguishable from "call does not exist". + +The characteristic property of the `received` state is that the call has made it past the (potentially malicious) endpoint *into the state of the IC*. It is now pointless (but harmless) to submit the (identical) call again. Before reaching that state, submitting the identical call to further nodes might be a useful safeguard against a malicious or misbehaving node. + +The characteristic property of the `processing` state is that *the initial effect of the call has happened or will happen*. This is best explained by an example: Consider a counter canister. It exports a method `inc` that increases the counter. Assume that the canister is bug free, and is not going to be forcibly removed. A user submits a call to call `inc`. If the user sees request status `processing`, the state change is guaranteed to happen. The user can stop monitoring the status and does not have to retry submitting. + +A call may be rejected by the IC or the canister. In either case, there is no guarantee about how much processing of the call has happened. + +To avoid replay attacks, the transition from `done` or `received` to `pruned` must happen no earlier than the call's `ingress_expiry` field. + +Calls must stay in `replied` or `rejected` long enough for polling users to catch the response. + +When asking the IC about the state or call of a request, the user uses the request id (see [Request ids](#request-id)) to read the request status (see [Request status](#state-tree-request-status)) from the state tree (see [Request: Read state](#http-read-state)). + +### Request: Call {#http-call} + +In order to call a canister, the user makes a POST request to `/api/v2/canister//call`. The request body consists of an authentication envelope with a `content` map with the following fields: + +- `request_type` (`text`): Always `call` + +- `sender`, `nonce`, `ingress_expiry`: See [Authentication](#authentication) + +- `canister_id` (`blob`): The principal of the canister to call. + +- `method_name` (`text`): Name of the canister method to call + +- `arg` (`blob`): Argument to pass to the canister method + +The HTTP response to this request can have the following responses: + +- 202 HTTP status with empty body. Implying the request was accepted by the IC for further processing. Users should use [`read_state`](#http-read-state) to determine the status of the call. + +- 200 HTTP status with non-empty body. Implying an execution pre-processing error occurred. The body of the response contains more information about the IC specific error encountered. The body is a CBOR map with the following fields: + + - `reject_code` (`nat`): The reject code (see [Reject codes](#reject-codes)). + + - `reject_message` (`text`): a textual diagnostic message. + + - `error_code` (`text`): an optional implementation-specific textual error code (see [Error codes](#error-codes)). + +- 4xx HTTP status for client errors (e.g. malformed request). Except for 429 HTTP status, retrying the request will likely have the same outcome. + +- 5xx HTTP status when the server has encountered an error or is otherwise incapable of performing the request. The request might succeed if retried at a later time. + +This request type can *also* be used to call a query method. A user may choose to go this way, instead of via the faster and cheaper [Request: Query call](#http-query) below, if they want to get a *certified* response. Note that the canister state will not be changed by sending a call request type for a query method. + +:::note + +The functionality exposed via the [The IC management canister](#ic-management-canister) can be used this way. + +::: + +### Request: Read state {#http-read-state} + +In order to read parts of the [The system state tree](#state-tree), the user makes a POST request to `/api/v2/canister//read_state`. The request body consists of an authentication envelope with a `content` map with the following fields: + +- `request_type` (`text`): Always `read_state` + +- `sender`, `nonce`, `ingress_expiry`: See [Authentication](#authentication) + +- `paths` (sequence of paths): A list of paths, where a path is itself a sequence of blobs. + +The HTTP response to this request consists of a CBOR (see [CBOR](#cbor)) map with the following fields: + +- `certificate` (`blob`): A certificate (see [Certification](#certification)). + + If this `certificate` includes subnet delegations (possibly nested), then the `effective_canister_id` must be included in each delegation's canister id range (see [Delegation](#certification-delegation)). + +The returned certificate reveals all values whose path is a suffix of a requested path. It also always reveals `/time`, even if not explicitly requested. + +All requested paths must have one of the following paths as prefix: + +- `/time`. Can always be requested. + +- `/subnet`. Can always be requested. + +- `/request_status/`. Can be requested if no path with such a prefix exists in the state tree or + + - the sender of the original request referenced by `` is the same as the sender of the read state request and + + - the effective canister id of the original request referenced by `` matches ``. + +- `/canisters//module_hash`. Can be requested if `` matches ``. + +- `/canisters//controllers`. Can be requested if `` matches ``. The order of controllers in the value at this path may vary depending on the implementation. + +- `/canisters//metadata/`. Can be requested if `` matches ``, `` is encoded in UTF-8, and + + - canister with canister id `` does not exist or + + - canister with canister id `` is empty or + + - canister with canister id `` does not have `` as its custom section or + + - `` is a public custom section or + + - `` is a private custom section and the sender of the read state request is a controller of the canister. + +Moreover, all paths with prefix `/request_status/` must refer to the same request ID ``. + +If a path cannot be requested, then the HTTP response to the read state request is undefined. + +Note that the paths `/canisters//certified_data` are not accessible with this method; these paths are only exposed to the canisters themselves via the System API (see [Certified data](#system-api-certified-data)). + +See [The system state tree](#state-tree) for details on the state tree. + +### Request: Query call {#http-query} + +A query call is a fast, but less secure way to call a canister. Only methods that are explicitly marked as "query methods" by the canister can be called this way. + +In order to make a query call to canister, the user makes a POST request to `/api/v2/canister//query`. The request body consists of an authentication envelope with a `content` map with the following fields: + +- `request_type` (`text`): Always `query` + +- `sender`, `nonce`, `ingress_expiry`: See [Authentication](#authentication) + +- `canister_id` (`blob`): The principal of the canister to call. + +- `method_name` (`text`): Name of the canister method to call + +- `arg` (`blob`): Argument to pass to the canister method + +If the call resulted in a reply, the response is a CBOR (see [CBOR](#cbor)) map with the following fields: + +- `status` (`text`): `replied` + +- `reply`: a CBOR map with the field `arg` (`blob`) which contains the reply data. + +If the call resulted in a reject, the response is a CBOR map with the following fields: + +- `status` (`text`): `rejected` + +- `reject_code` (`nat`): The reject code (see [Reject codes](#reject-codes)). + +- `reject_message` (`text`): a textual diagnostic message. + +- `error_code` (text): an optional implementation-specific textual error code (see [Error codes](#error-codes)). + +Canister methods that do not change the canister state can be executed more efficiently. This method provides that ability, and returns the canister's response directly within the HTTP response. + +### Effective canister id {#http-effective-canister-id} + +The `` in the URL paths of requests is the *effective* destination of the request. + +- If the request is an update call to the Management Canister (`aaaaa-aa`), then: + + - If the call is to the `provisional_create_canister_with_cycles` method, then any principal can be used as the effective canister id for this call. + + - Otherwise, if the `arg` is a Candid-encoded record with a `canister_id` field of type `principal`, then the effective canister id must be that principal. + + - Otherwise, the call is rejected by the system independently of the effective canister id. + +- If the request is an update call to a canister that is not the Management Canister (`aaaaa-aa`) or if the request is a query call, then the effective canister id must be the `canister_id` in the request. + +:::note + +The expectation is that user-side agent code shields users and developers from the notion of effective canister ID, in analogy to how the System API interface shields canister developers from worrying about routing. + +The Internet Computer blockchain mainnet rejects all requests whose effective canister id is in no subnet's canister ranges, independently of whether the remaining conditions on the effective canister id are satisfied. + +The Internet Computer blockchain mainnet does not support `provisional_create_canister_with_cycles` and thus all calls to this method are rejected independently of the effective canister id. + +In development instances of the Internet Computer Protocol (e.g. testnets), the effective canister id of a request submitted to a node must be a canister id from the canister ranges of the subnet to which the node belongs. + +::: + +### Authentication {#authentication} + +All requests coming in via the HTTPS interface need to be either *anonymous* or *authenticated* using a cryptographic signature. To that end, the following fields are present in the `content` map in all cases: + +- `nonce` (`blob`, optional): Arbitrary user-provided data, typically randomly generated. This can be used to create distinct requests with otherwise identical fields. + +- `ingress_expiry` (`nat`, required): An upper limit on the validity of the request, expressed in nanoseconds since 1970-01-01 (like [ic0.time()](#system-api-time)). This avoids replay attacks: The IC will not accept requests, or transition requests from status `received` to status `processing`, if their expiry date is in the past. The IC may refuse to accept requests with an ingress expiry date too far in the future. This applies to synchronous and asynchronous requests alike (and could have been called `request_expiry`). + +- `sender` (`Principal`, required): The user who issued the request. + +The envelope, i.e. the overall request, has the following keys: + +- `content` (`record`): the actual request content + +- `sender_pubkey` (`blob`, optional): Public key used to authenticate this request. Since a user may in the future have more than one key, this field tells the IC which key is used. + +- `sender_delegation` (`array` of maps, optional): a chain of delegations, starting with the one signed by `sender_pubkey` and ending with the one delegating to the key relating to `sender_sig`. Every public key in the chain of delegations should appear exactly once: cycles (a public key delegates to another public key that already previously appeared in the chain) or self-signed delegations (a public key delegates to itself) are not allowed and such requests will be refused by the IC. + +- `sender_sig` (`blob`, optional): Signature to authenticate this request. + +The public key must authenticate the `sender` principal: + +- A public key can authenticate a principal if the latter is a self-authenticating id derived from that public key (see [Special forms of Principals](#id-classes)). + +- The fields `sender_pubkey`, `sender_sig`, and `sender_delegation` must be omitted if the `sender` field is the anonymous principal. The fields `sender_pubkey` and `sender_sig` must be set if the `sender` field is not the anonymous principal. + +The request id (see [Request ids](#request-id)) is calculated from the content record. This allows the signature to be based on the request id, and implies that signature and public key are not semantically relevant. + +The field `sender_pubkey` contains a public key supported by one of the schemes described in [Signatures](#signatures). + +Signing transactions can be delegated from one key to another one. If delegation is used, then the `sender_delegation` field contains an array of delegations, each of which is a map with the following fields: + +- `delegation` (`map`): Map with fields: + + - `pubkey` (`blob`): Public key as described in [Signatures](#signatures). + + - `expiration` (`nat`): Expiration of the delegation, in nanoseconds since 1970-01-01, analogously to the `ingress_expiry` field above. + + - `targets` (`array` of `CanisterId`, optional): If this field is set, the delegation only applies for requests sent to the canisters in the list. The list must contain no more than 1000 elements; otherwise, the request will not be accepted by the IC. + + - `senders` (`array` of `Principal`, optional): If this field is set, the delegation only applies for requests originating from the principals in the list. + +- `signature` (`blob`): Signature on the 32-byte [representation-independent hash](#hash-of-map) of the map contained in the `delegation` field as described in [Signatures](#signatures), using the 27 bytes `\x1Aic-request-auth-delegation` as the domain separator. + + For the first delegation in the array, this signature is created with the key corresponding to the public key from the `sender_pubkey` field, all subsequent delegations are signed with the key corresponding to the public key contained in the preceding delegation. + +The `sender_sig` field is calculated by signing the concatenation of the 11 bytes `\x0Aic-request` (the domain separator) and the 32 byte [request id](#request-id) with the secret key that belongs to the key specified in the last delegation or, if no delegations are present, the public key specified in `sender_pubkey`. + +The delegation field, if present, must not contain more than 20 delegations. + +### Representation-independent hashing of structured data {#hash-of-map} + +Structured data, such as (recursive) maps, are authenticated by signing a representation-independent hash of the data. This hash is computed as follows (using SHA256 in the steps below): + +1. For each field that is present in the map (i.e. omitted optional fields are indeed omitted): + + - concatenate the hash of the field's name (in ascii-encoding, without terminal `\x00`) and the hash of the value (with the encoding specified below). + +2. Sort these concatenations from low to high + +3. Concatenate the sorted elements, and hash the result. + +The resulting hash of 256 bits (32 bytes) is the representation-independent hash. + +The following encodings of field values as blobs are used + +- Binary blobs (`canister_id`, `arg`, `nonce`, `module`) are used as-is. + +- Strings (`request_type`, `method_name`) are encoded in UTF-8, without a terminal `\x00`. + +- Natural numbers (`compute_allocation`, `memory_allocation`, `ingress_expiry`) are encoded using the shortest form [Unsigned LEB128](https://en.wikipedia.org/wiki/LEB128#Unsigned_LEB128) encoding. For example, `0` should be encoded as a single zero byte `[0x00]` and `624485` should be encoded as byte sequence `[0xE5, 0x8E, 0x26]`. + +- Integers are encoded using the shortest form [Signed LEB128](https://en.wikipedia.org/wiki/LEB128#Signed_LEB128) encoding. For example, `0` should be encoded as a single zero byte `[0x00]` and `-123456` should be encoded as byte sequence `[0xC0, 0xBB, 0x78]`. + +- Arrays (`paths`) are encoded as the concatenation of the hashes of the encodings of the array elements. + +- Maps (`sender_delegation`) are encoded by recursively computing the representation-independent hash. + +### Request ids {#request-id} + +When signing requests or querying the status of a request (see [Request status](#state-tree-request-status)) in the state tree, the user identifies the request using a *request id*, which is the [representation-independent hash](#hash-of-map) of the `content` map of the original request. A request id must have length of 32 bytes. + +:::note + +The request id is independent of the representation of the request (currently only CBOR, see [CBOR](#cbor)), and does not change if the specification adds further optional fields to a request type. + +::: + +:::note + +The recommended textual representation of a request id is a hexadecimal string with lower-case letters prefixed with '0x'. E.g., request id consisting of bytes `[00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 0A, 0B, 0C, 0D, 0E, 0F, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 1A, 1B, 1C, 1D, 1E, 1F]` should be displayed as `0x000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f`. + +::: + +:::tip + +Example calculation (where `H` denotes SHA-256 and `·` denotes blob concatenation) in which we assume that the optional nonce is not provided and thus omitted: + + hash_of_map({ request_type: "call", sender: 0x04, ingress_expiry: 1685570400000000000, canister_id: 0x00000000000004D2, method_name: "hello", arg: "DIDL\x00\xFD*"}) + = H(concat (sort + [ H("request_type") · H("call") + , H("sender") · H("0x04") + , H("ingress_expiry") · H(1685570400000000000) + , H("canister_id") · H("\x00\x00\x00\x00\x00\x00\x04\xD2") + , H("method_name") · H("hello") + , H("arg") · H("DIDL\x00\xFD*") + ])) + = H(concat (sort + [ 769e6f87bdda39c859642b74ce9763cdd37cb1cd672733e8c54efaa33ab78af9 · 7edb360f06acaef2cc80dba16cf563f199d347db4443da04da0c8173e3f9e4ed + , 0a367b92cf0b037dfd89960ee832d56f7fc151681bb41e53690e776f5786998a · e52d9c508c502347344d8c07ad91cbd6068afc75ff6292f062a09ca381c89e71 + , 26cec6b6a9248a96ab24305b61b9d27e203af14a580a5b1ff2f67575cab4a868 · db8e57abc8cda1525d45fdd2637af091bc1f28b35819a40df71517d1501f2c76 + , 0a3eb2ba16702a387e6321066dd952db7a31f9b5cc92981e0a92dd56802d3df9 · 4d8c47c3c1c837964011441882d745f7e92d10a40cef0520447c63029eafe396 + , 293536232cf9231c86002f4ee293176a0179c002daa9fc24be9bb51acdd642b6 · 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 + , b25f03dedd69be07f356a06fe35c1b0ddc0de77dcd9066c4be0c6bbde14b23ff · 6c0b2ae49718f6995c02ac5700c9c789d7b7862a0d53e6d40a73f1fcd2f70189 + ])) + = H(concat + [ 0a367b92cf0b037dfd89960ee832d56f7fc151681bb41e53690e776f5786998a · e52d9c508c502347344d8c07ad91cbd6068afc75ff6292f062a09ca381c89e71 + , 0a3eb2ba16702a387e6321066dd952db7a31f9b5cc92981e0a92dd56802d3df9 · 4d8c47c3c1c837964011441882d745f7e92d10a40cef0520447c63029eafe396 + , 26cec6b6a9248a96ab24305b61b9d27e203af14a580a5b1ff2f67575cab4a868 · db8e57abc8cda1525d45fdd2637af091bc1f28b35819a40df71517d1501f2c76 + , 293536232cf9231c86002f4ee293176a0179c002daa9fc24be9bb51acdd642b6 · 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 + , 769e6f87bdda39c859642b74ce9763cdd37cb1cd672733e8c54efaa33ab78af9 · 7edb360f06acaef2cc80dba16cf563f199d347db4443da04da0c8173e3f9e4ed + , b25f03dedd69be07f356a06fe35c1b0ddc0de77dcd9066c4be0c6bbde14b23ff · 6c0b2ae49718f6995c02ac5700c9c789d7b7862a0d53e6d40a73f1fcd2f70189 + ]) + = 1d1091364d6bb8a6c16b203ee75467d59ead468f523eb058880ae8ec80e2b101 + +::: + +### Reject codes {#reject-codes} + +An API request or inter-canister call that is pending in the IC will eventually result in either a *reply* (indicating success, and carrying data) or a *reject* (indicating an error of some sorts). A reject contains a *rejection code* that classifies the error and a hopefully helpful *reject message* string. + +Rejection codes are member of the following enumeration: + +- `SYS_FATAL` (1): Fatal system error, retry unlikely to be useful. + +- `SYS_TRANSIENT` (2): Transient system error, retry might be possible. + +- `DESTINATION_INVALID` (3): Invalid destination (e.g. canister/account does not exist) + +- `CANISTER_REJECT` (4): Explicit reject by the canister. + +- `CANISTER_ERROR` (5): Canister error (e.g., trap, no response) + +The symbolic names of this enumeration are used throughout this specification, but on all interfaces (HTTPS API, System API), they are represented as positive numbers as given in the list above. + +The error message is guaranteed to be a string, i.e. not arbitrary binary data. + +When canisters explicitly reject a message (see [Public methods](#system-api-requests)), they can specify the reject message, but *not* the reject code; it is always `CANISTER_REJECT`. In this sense, the reject code is trustworthy: If the IC responds with a `SYS_FATAL` reject, then it really was the IC issuing this reject. + +### Error codes {#error-codes} + +Implementations of the API can provide additional details for rejected messages in the form of a textual label identifying the error condition. API clients can use these labels to handle errors programmatically or suggest recovery paths to the user. The specification reserves error codes matching the regular expression `IC[0-9]+` (e.g., `IC502`) for the DFINITY implementation of the API. + +### Status endpoint {#api-status} + +Additionally, the Internet Computer provides an API endpoint to obtain various status fields at + + /api/v2/status + +For this endpoint, the user performs a GET request, and receives a CBOR (see [CBOR](#cbor)) value with the following fields. The IC may include additional implementation-specific fields. + +- `ic_api_version` (string, mandatory): Identifies the interface version supported, i.e. the version of the present document that the Internet Computer aims to support, e.g. `0.8.1`. The implementation may also return `unversioned` to indicate that it does *not* comply to a particular version, e.g. in between releases. + +- `impl_source` (string, optional): Identifies the implementation of the Internet Computer Protocol, by convention with the canonical location of the source code (e.g. `https://github.com/dfinity/ic`). + +- `impl_version` (string, optional): If the user is talking to a released version of an Internet Computer Protocol implementation, this is the version number. For non-released versions, output of `git describe` like `0.1.13-13-g2414721` would also be very suitable. + +- `impl_revision` (string, optional): The precise git revision of the Internet Computer Protocol implementation + +- `root_key` (blob, only in development instances): The public key (a DER-encoded BLS key) of the root key of this development instance of the Internet Computer Protocol. This *must* be present in short-lived development instances, to allow the agent to fetch the public key. For the Internet Computer, agents must have an independent trustworthy source for this data, and must not be tempted to fetch it from this insecure location. + +See [CBOR encoding of requests and responses](#api-cbor) for details on the precise CBOR encoding of this object. + +:::note + +Future additions may include local time, geographic location, and other useful implementation-specific information such as blockheight. This data may possibly be signed by the node. + +::: + +### CBOR encoding of requests and responses {#api-cbor} + +Requests and responses are specified here as records with named fields and using suggestive human readable syntax. The actual format in the body of the HTTP request or response, however, is CBOR (see [CBOR](#cbor)). + +Concretely, it consists of a data item with major type 6 ("Semantic tag") and tag value `55799`, followed by a record. + +Requests consist of an envelope record with keys `sender_sig` (a blob), `sender_pubkey` (a blob) and `content` (a record). The first two are metadata that are used for request authentication, while the last one is the actual content of the request. + +The following encodings are used: + +- Strings: Major type 3 ("Text string"). + +- Blobs: Major type 2 ("Byte string"). + +- Nats: Major type 0 ("Unsigned integer") if small enough to fit that type, else the [Bignum](https://www.rfc-editor.org/rfc/rfc8949#name-bignums) format is used. + +- Records: Major type 5 ("Map of pairs of data items"), followed by the fields, where keys are encoded with major type 3 ("Text string"). + +- Arrays: Major type 4 ("Array of data items"). + +As advised by [section "Creating CBOR-Based Protocols"](https://www.rfc-editor.org/rfc/rfc8949#name-creating-cbor-based-protoco) of the CBOR spec, we clarify that: + +- Floating-point numbers may not be used to encode integers. + +- Duplicate keys are prohibited in CBOR maps. + +:::tip + +A typical request would be (written in [CBOR diagnostic notation](https://www.rfc-editor.org/rfc/rfc8949#name-diagnostic-notation), which can be checked and converted on [cbor.me](http://cbor.me/)): + + 55799({ + "content": { + "request_type": "call", + "canister_id": h'ABCD01', + "method_name": "say_hello", + "arg": h'0061736d01000000' + }, + "sender_sig": h'DEADBEEF', + "sender_pubkey": h'b7a3c12dc0c8c748ab07525b701122b88bd78f600c76342d27f25e5f92444cde' + }) + +::: + +### CDDL description of requests and responses {#api-cddl} + +This section summarizes the format of the CBOR data passed to and from the entry points described above. You can also [download the file]({attachmentsdir}/requests.cddl) and see [CDDL](#cddl) for more information. + +### Ordering guarantees + +The order in which the various messages between canisters are delivered and executed is not fully specified. The guarantee provided by the IC is that function calls between two canisters are executed in order, so that a canister that requires in-order execution need not wait for the response from an earlier message to a canister before sending a later message to that same canister. + +More precisely: + +- Method calls between any *two* canisters are delivered in order, as if they were communicating over a single simple FIFO queue. + +- If a WebAssembly function, within a single invocation, makes multiple calls to the same canister, they are queued in the order of invocations to `ic0.call_perform`. + +- Responses (including replies with `ic0.msg_reply`, explicit rejects with `ic0.msg_reject` and system-generated error responses) do *not* have any ordering guarantee relative to each other or to method calls. + +- There is no particular order guarantee for ingress messages submitted via the HTTPS interface. + +### Synchronicity across nodes + +This document describes the Internet Computer as having a single global state that can be modified and queried. In reality, it consists of many nodes, which may not be perfectly in sync. + +As long as you talk to one (honest) node only, the observed behavior is nicely sequential. If you issue an update (i.e. state-mutating) call to a canister (e.g. bump a counter), and node A indicates that the call has been executed, and you then issue a query call to node A, then A's response is guaranteed to include the effect of the update call (and you will receive the updated counter value). + +If you then (quickly) issue a read request to node B, it may be that B responds to your read query based on the old state of the canister (and you might receive the old counter value). + +A related problem is that query calls are not certified, and nodes may be dishonest in their response. In that case, the user might want to get more assurance by querying multiple nodes and comparing the result. However, it is (currently) not possible to query a *specific* state. + +:::note + +Applications can work around these problems. For the first problem, the query result could be such that the user can tell if the update has been received or not. For the second problem, even if using [certified data](#system-api-certified-data) is not possible, if replies are monotonic in some sense the user can get assurance in their intersection (e.g. if the query returns a list of events that grows over time, then even if different nodes return different lists, the user can get assurance in those events that are reported by many nodes). + +::: + +## Canister module format {#canister-module-format} + +A canister module is a [WebAssembly module](https://webassembly.github.io/spec/core/index.html) that is either in binary format (typically `.wasm`) or gzip-compressed (typically `.wasm.gz`). If the module starts with byte sequence `[0x1f, 0x8b, 0x08]`, then the system decompresses the contents as a gzip stream according to [RFC-1952](https://datatracker.ietf.org/doc/html/rfc1952.html) and then parses the output as a WebAssembly binary. + +## Canister interface (System API) {#system-api} + +The System API is the interface between the running canister and the Internet Computer. It allows the WebAssembly module of a canister to expose functionality to the users (method entry points) and the IC (e.g. initialization), and exposes functionality of the IC to the canister (e.g. calling other canisters). Because WebAssembly is rather low-level, it also explains how to express higher level concepts (e.g. binary blobs). + +We want to leverage advanced WebAssembly features, such as WebAssembly host references. But as they are not yet supported by all tools involved, this section describes an initial System API that does not rely on host references. In section [Outlook: Using Host References](#host-references), we outline some of the proposed uses of WebAssembly host references. + +### WebAssembly module requirements {#system-api-module} + +In order for a WebAssembly module to be usable as the code for the canister, it needs to conform to the following requirements: + +- If it imports a memory, it must import it from `env.memory`. In the following, "the Wasm memory" refers to this memory. + +- If it imports a table, it must import it from `env.table`. In the following, "the Wasm table" refers to this table. + +- It may only import a function if it is listed in [Overview of imports](#system-api-imports). + +- It may have a `(start)` function. + +- If it exports a function called `canister_init`, the function must have type `() -> ()`. + +- If it exports a function called `canister_inspect_message`, the function must have type `() -> ()`. + +- If it exports a function called `canister_heartbeat`, the function must have type `() -> ()`. + +- If it exports a function called `canister_global_timer`, the function must have type `() -> ()`. + +- If it exports any functions called `canister_update ` or `canister_query ` for some `name`, the functions must have type `() -> ()`. + +- It may not export both `canister_update ` and `canister_query ` with the same `name`. + +- It may not export other methods the names of which start with the prefix `canister_` besides the methods allowed above. + +- It may not have both `icp:public ` and `icp:private ` with the same `name` as the custom section name. + +- It may not have other custom sections the names of which start with the prefix `icp:` besides the \`icp:public \` and \`icp:private \`. + +- The IC may reject WebAssembly modules that + + - declare more than 50,000 functions, or + + - declare more than 300 globals, or + + - declare more than 16 exported custom sections (the custom section names with prefix `icp:`), or + + - the number of all exported functions called `canister_update ` or `canister_query ` exceeds 1,000, or + + - the sum of `` lengths in all exported functions called `canister_update ` or `canister_query ` exceeds 20,000, or + + - the total size of the exported custom sections exceeds 1MiB. + +### Interpretation of numbers + +WebAssembly number types (`i32`, `i64`) do not indicate if the numbers are to be interpreted as signed or unsigned. Unless noted otherwise, whenever the System API interprets them as numbers (e.g. memory pointers, buffer offsets, array sizes), they are to be interpreted as unsigned. + +### Entry points {#entry-points} + +The canister provides entry points which are invoked by the IC under various circumstances: + +- The canister may export a function with name `canister_init` and type `() -> ()`. + +- The canister may export a function with name `canister_pre_upgrade` and type `() -> ()`. + +- The canister may export a function with name `canister_post_upgrade` and type `() -> ()`. + +- The canister may export functions with name `canister_inspect_message` with type `() -> ()`. + +- The canister may export a function with name `canister_heartbeat` with type `() -> ()`. + +- The canister may export a function with name `canister_global_timer` with type `() -> ()`. + +- The canister may export functions with name `canister_update ` and type `() -> ()`. + +- The canister may export functions with name `canister_query ` and type `() -> ()`. + +- The canister table may contain functions of type `(env : i32) -> ()` which may be used as callbacks for inter-canister calls. + +If the execution of any of these entry points traps for any reason, then all changes to the WebAssembly state, as well as the effect of any externally visible system call (like `ic0.msg_reply`, `ic0.msg_reject`, `ic0.call_perform`), are discarded. For upgrades, this transactional behavior applies to the `canister_pre_upgrade`/`canister_post_upgrade` sequence as a whole. + +#### Canister initialization {#system-api-init} + +If `canister_init` is present, then this is the first exported WebAssembly function invoked by the IC. The argument that was passed along with the canister initialization call (see [IC method](#ic-install_code)) is available to the canister via `ic0.msg_arg_data_size/copy`. + +The IC assumes the canister to be fully instantiated if the `canister_init` method entry point returns. If the `canister_init` method entry point traps, then canister installation has failed, and the canister is reverted to its previous state (i.e. empty with `install`, or whatever it was for a `reinstall`). + +#### Canister upgrades {#system-api-upgrades} + +When a canister is upgraded to a new WebAssembly module, the IC: + +1. Invokes `canister_pre_upgrade` (if present) on the old instance, to give the canister a chance to clean up (e.g. move data to [stable memory](#system-api-stable-memory)). + +2. Instantiates the new module, including the execution of `(start)`, with a fresh WebAssembly state. + +3. Invokes `canister_post_upgrade` (if present) on the new instance, passing the `arg` provided in the `install_code` call ([IC method](#ic-install_code)). + +The stable memory is preserved throughout the process; any other WebAssembly state is discarded. + +During these steps, no other entry point of the old or new canister is invoked. The `canister_init` function of the new canister is *not* invoked. + +These steps are atomic: If `canister_pre_upgrade` or `canister_post_upgrade` trap, the upgrade has failed, and the canister is reverted to the previous state. Otherwise, the upgrade has succeeded, and the old instance is discarded. + +#### Public methods {#system-api-requests} + +To define a public method of name `name`, a WebAssembly module exports a function with name `canister_update ` or `canister_query ` and type `() -> ()`. We call this the *method entry point*. The name of the exported function distinguishes update and query methods. + +:::note + +The space in `canister_update ` resp. `canister_query ` is intentional. There is exactly one space between `canister_update/canister_query` and the ``. + +::: + +The argument of the call (e.g. the content of the `arg` field in the [API request to call a canister method](#http-call)) is copied into the canister on demand using the System API functions shown below. + +Eventually, a method will want to send a response, using `ic0.reply` or `ic0.reject` + +#### Heartbeat + +For periodic or time-based execution, the WebAssembly module can export a function with name `canister_heartbeat`. The heartbeats scheduling algorithm is implementation-defined. + +`canister_heartbeat` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. Still, the function `canister_heartbeat` can initiate new calls. + +:::note + +While an implementation will likely try to keep the interval between `canister_heartbeat` invocations to within a few seconds, this is not formally part of this specification. + +::: + +#### Global timer + +For time-based execution, the WebAssembly module can export a function with name `canister_global_timer`. This function is called if the canister has set its global timer (using the System API function `ic0.global_timer_set`) and the current time (as returned by the System API function `ic0.time`) has passed the value of the global timer. + +Once the function `canister_global_timer` is scheduled, the canister's global timer is deactivated. The global timer is also deactivated upon changes to the canister's Wasm module (calling `install_code`, `uninstall_code` methods of the management canister or if the canister runs out of cycles). In particular, the function `canister_global_timer` won't be scheduled again unless the canister sets the global timer again (using the System API function `ic0.global_timer_set`). The global timer scheduling algorithm is implementation-defined. + +`canister_global_timer` is triggered by the IC, and therefore has no arguments, no caller, and cannot reply or reject. Still, the function `canister_global_timer` can initiate new calls. + +:::note + +While an implementation will likely try to keep the interval between the value of the global timer and the time-stamp of the `canister_global_timer` invocation within a few seconds, this is not formally part of this specification. + +::: + +#### Callbacks + +Callbacks are addressed by their table index (as a proxy for a Wasm `funcref`). + +In the reply callback of a [inter-canister method call](#system-api-call), the argument refers to the response to that call. In reject callbacks, no argument is available. + +### Overview of imports {#system-api-imports} + +The following sections describe various System API functions, also referred to as system calls, which we summarize here. + + ic0.msg_arg_data_size : () -> i32; // I U Q Ry F + ic0.msg_arg_data_copy : (dst : i32, offset : i32, size : i32) -> (); // I U Q Ry F + ic0.msg_caller_size : () -> i32; // I G U Q F + ic0.msg_caller_copy : (dst : i32, offset: i32, size : i32) -> (); // I G U Q F + ic0.msg_reject_code : () -> i32; // Ry Rt + ic0.msg_reject_msg_size : () -> i32; // Rt + ic0.msg_reject_msg_copy : (dst : i32, offset : i32, size : i32) -> (); // Rt + + ic0.msg_reply_data_append : (src : i32, size : i32) -> (); // U Q Ry Rt + ic0.msg_reply : () -> (); // U Q Ry Rt + ic0.msg_reject : (src : i32, size : i32) -> (); // U Q Ry Rt + + ic0.msg_cycles_available : () -> i64; // U Rt Ry + ic0.msg_cycles_available128 : (dst : i32) -> (); // U Rt Ry + ic0.msg_cycles_refunded : () -> i64; // Rt Ry + ic0.msg_cycles_refunded128 : (dst : i32) -> (); // Rt Ry + ic0.msg_cycles_accept : (max_amount : i64) -> (amount : i64); // U Rt Ry + ic0.msg_cycles_accept128 : (max_amount_high : i64, max_amount_low: i64, dst : i32) + -> (); // U Rt Ry + + ic0.canister_self_size : () -> i32; // * + ic0.canister_self_copy : (dst : i32, offset : i32, size : i32) -> (); // * + ic0.canister_cycle_balance : () -> i64; // * + ic0.canister_cycle_balance128 : (dst : i32) -> (); // * + ic0.canister_status : () -> i32; // * + ic0.canister_version : () -> i64; // * + + ic0.msg_method_name_size : () -> i32; // F + ic0.msg_method_name_copy : (dst : i32, offset : i32, size : i32) -> (); // F + ic0.accept_message : () -> (); // F + + ic0.call_new : // U Ry Rt T + ( callee_src : i32, + callee_size : i32, + name_src : i32, + name_size : i32, + reply_fun : i32, + reply_env : i32, + reject_fun : i32, + reject_env : i32 + ) -> (); + ic0.call_on_cleanup : (fun : i32, env : i32) -> (); // U Ry Rt T + ic0.call_data_append : (src : i32, size : i32) -> (); // U Ry Rt T + ic0.call_cycles_add : (amount : i64) -> (); // U Ry Rt T + ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt T + ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt T + + ic0.stable_size : () -> (page_count : i32); // * s + ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32); // * s + ic0.stable_write : (offset : i32, src : i32, size : i32) -> (); // * s + ic0.stable_read : (dst : i32, offset : i32, size : i32) -> (); // * s + ic0.stable64_size : () -> (page_count : i64); // * s + ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64); // * s + ic0.stable64_write : (offset : i64, src : i64, size : i64) -> (); // * s + ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> (); // * s + + ic0.certified_data_set : (src: i32, size: i32) -> (); // I G U Ry Rt T + ic0.data_certificate_present : () -> i32; // * + ic0.data_certificate_size : () -> i32; // * + ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> (); // * + + ic0.time : () -> (timestamp : i64); // * + ic0.global_timer_set : (timestamp : i64) -> i64; // I G U Ry Rt C T + ic0.performance_counter : (counter_type : i32) -> (counter : i64); // * s + ic0.is_controller: (src: i32, size: i32) -> ( result: i32); // * s + + ic0.debug_print : (src : i32, size : i32) -> (); // * s + ic0.trap : (src : i32, size : i32) -> (); // * s + +The comment after each function lists from where these functions may be invoked: + +- `I`: from `canister_init` or `canister_post_upgrade` + +- `G`: from `canister_pre_upgrade` + +- `U`: from `canister_update …` + +- `Q`: from `canister_query …` + +- `Ry`: from a reply callback + +- `Rt`: from a reject callback + +- `C`: from a cleanup callback + +- `s`: the `(start)` module initialization function + +- `F`: from `canister_inspect_message` + +- `T`: from *system task* (`canister_heartbeat` or `canister_global_timer`) + +- `*` = `I G U Q Ry Rt C F T` (NB: Not `(start)`) + +If the canister invokes a system call from somewhere else, it will trap. + +### Blob-typed arguments and results + +WebAssembly functions parameter and result types can only be primitive number types. To model functions that accept or return blobs or text values, the following idiom is used: + +To provide access to a string or blob `foo`, the System API provides two functions: + + ic0.foo_size : () -> i32 + ic0.foo_copy : (dst : i32, offset: i32, size : i32) -> () + +The `*_size` function indicates the size, in bytes, of `foo`. The `*_copy` function copies `size` bytes from `foo[offset..offset+size]` to `memory[dst..dst+size]`. This traps if `offset+size` is greater than the size of `foo`, or if `dst+size` exceeds the size of the Wasm memory. + +Dually, a System API function that conceptually takes a blob or string as a parameter `foo` has two parameters: + + ic0.set_foo : (src : i32, size: i32) -> … + +which copies, at the time of function invocation, the data referred to by `src`/`size` out of the canister. Unless otherwise noted, this traps if `src+size` exceeds the size of the WebAssembly memory. + +### Method arguments + +The canister can access an argument. For `canister_init`, `canister_post_upgrade` and method entry points, the argument is the argument of the call; in a reply callback, it refers to the received reply. So the lifetime of the argument data is a single WebAssembly function execution, not the whole method call tree. + +- `ic0.msg_arg_data_size : () → i32` and `ic0.msg_arg_data_copy : (dst : i32, offset : i32, size : i32) → ()` + + The message argument data. + +- `ic0.msg_caller_size : () → i32` and `ic0.msg_caller_copy : (dst : i32, offset: i32, size : i32) → ()` + + The identity of the caller, which may be a canister id or a user id. During canister installation or upgrade, this is the id of the user or canister requesting the installation or upgrade. + +- `ic0.msg_reject_code : () → i32` + + Returns the reject code, if the current function is invoked as a reject callback. + + It returns the special "no error" code `0` if the callback is *not* invoked as a reject callback; this allows canisters to use a single entry point for both the reply and reject callback, if they choose to do so. + +- `ic0.msg_reject_msg_size : () → i32` and `ic0.msg_reject_msg_copy : (dst : i32, offset : i32, size : i32) → ()` + + The reject message. Traps if there is no reject message (i.e. if `reject_code` is `0`). + +### Responding {#responding} + +Eventually, the canister will want to respond to the original call, either by replying (indicating success) or rejecting (signalling an error): + +- `ic0.msg_reply_data_append : (src : i32, size : i32) → ()` + + Appends data it to the (initially empty) data reply. + + This traps if the current call already has been or does not need to be responded to. + + Any data assembled, but not replied using `ic0.msg_reply`, gets discarded at the end of the current message execution. In particular, the reply buffer gets reset when the canister yields control without calling `ic0.msg_reply`. + +:::note + +This can be invoked multiple times within the same message execution to build up the argument with data from various places on the Wasm heap. This way, the canister does not have to first copy all the pieces from various places into one location. + +::: + +- `ic0.msg_reply : () → ()` + + Replies to the sender with the data assembled using `ic0.msg_reply_data_append`. + + This function can be called at most once (a second call will trap), and must be called exactly once to indicate success. + + See [Cycles](#system-api-cycles) for how this interacts with cycles available on this call. + +- `ic0.msg_reject : (src : i32, size : i32) → ()` + + Rejects the call. The data referred to by `src`/`size` is used for the diagnostic message. + + This system call traps if `src+size` exceeds the size of the WebAssembly memory, or if the current call already has been or does not need to be responded to, or if the data referred to by `src`/`size` is not valid UTF8. + + The other end will receive this reject with reject code `CANISTER_REJECT`, see [Reject codes](#reject-codes). + + Possible reply data assembled using `ic0.msg_reply_data_append` is discarded. + + See [Cycles](#system-api-cycles) for how this interacts with cycles available on this call. + +### Ingress message inspection {#system-api-inspect-message} + +A canister can inspect ingress messages before executing them. When the IC receives an update call from a user, the IC will use the canister method `canister_inspect_message` to determine whether the message shall be accepted. If the canister is empty (i.e. does not have a Wasm module), then the ingress message will be rejected. If the canister is not empty and does not implement `canister_inspect_message`, then the ingress message will be accepted. + +In `canister_inspect_message`, the canister can accept the message by invoking `ic0.accept_message : () → ()`. This function traps if invoked twice. If the canister traps in `canister_inspect_message` or does not call `ic0.accept_message`, then the access is denied. + +:::note + +The `canister_inspect_message` is *not* invoked for query calls, inter-canister calls or calls to the management canister. + +::: + +### Self-identification {#system-api-canister-self} + +A canister can learn about its own identity: + +- `ic0.canister_self_size : () → i32` and `ic0.canister_self_copy: (dst : i32, offset : i32, size : i32) → ()` + + These functions allow the canister to query its own canister id (as a blob). + +### Canister status {#system-api-canister-status} + +This function allows a canister to find out if it is running, stopping or stopped (see [IC method](#ic-canister_status) and [IC method](#ic-stop_canister) for context). + +- `ic0.canister_status : () → i32` + + returns the current status of the canister: + + Status `1` indicates running, `2` indicates stopping, and `3` indicates stopped. + + Status `3` (stopped) can be observed, for example, in `canister_pre_upgrade` and can be used to prevent accidentally upgrading a canister that is not fully stopped. + +### Canister version {#system-api-canister-version} + +For each canister, the system maintains a *canister version*. Upon canister creation, it is set to 0, and it is **guaranteed** to be incremented upon every change of the canister's code or settings, i.e., upon every successful management canister call of methods `update_settings`, `install_code`, and `uninstall_code` on that canister and code uninstallation due to that canister running out of cycles. The system can arbitrarily increment the canister version also if the canister's code and settings do not change. + +- `ic0.canister_version : () → i64` + + returns the current canister version. + +During the canister upgrade process, `canister_pre_upgrade` sees the old counter value, and `canister_post_upgrade` sees the new counter value. + +### Inter-canister method calls {#system-api-call} + +When handling an update call (or a callback), a canister can do further calls to another canister. Calls are assembled in a builder-like fashion, starting with `ic0.call_new`, adding more attributes using the `ic0.call_*` functions, and eventually performing the call with `ic0.call_perform`. + +- `ic0.call_new : + ( callee_src : i32, + callee_size : i32, + name_src : i32, + name_size : i32, + reply_fun : i32, + reply_env : i32, + reject_fun : i32, + reject_env : i32, + ) → ()` + +Begins assembling a call to the canister specified by `callee_src/_size` at method `name_src/_size`. + +The IC records two mandatory callback functions, represented by a table entry index `*_fun` and some additional value `*_env`. When the response comes back, the table is read at the corresponding index, expected to be a function of type `(env : i32) -> ()`, and passed the corresponding `*_env` value. + +The reply callback is executed upon successful completion of the method call, which can query the reply using `ic0.msg_arg_data_*`. + +The reject callback is executed if the method call fails asynchronously or the other canister explicitly rejects the call. The reject code and message can be queried using `ic0.msg_reject_code` and `ic0.msg_reject_msg_*`. + +This deducts `MAX_CYCLES_PER_RESPONSE` cycles from the canister balance and sets them aside for response processing. This will trap if not sufficient cycles are available. + +Subsequent calls to the following functions set further attributes of that call, until the call is concluded (with `ic0.call_perform`) or discarded (by returning without calling `ic0.call_perform` or by starting a new call with `ic0.call_new`.) + +- `ic0.call_on_cleanup : (fun : i32, env : i32) → ()` + +If a cleanup callback (of type `(env : i32) -> ()`) is specified for this call, it is executed if and only if the `reply` or the `reject` callback was executed and trapped (for any reason). + +During the execution of the `cleanup` function, only a subset of the System API is available (namely `ic0.debug_print`, `ic0.trap` and the `ic0.stable_*` functions). The cleanup function is expected to run swiftly (within a fixed, yet to be specified cycle limit) and serves to free resources associated with the callback. + +If this traps (e.g. runs out of cycles), the state changes from the `cleanup` function are discarded, as usual, and no further actions are taken related to that call. Canisters likely want to avoid this from happening. + +There must be at most one call to `ic0.call_on_cleanup` between `ic0.call_new` and `ic0.call_perform`. + +- `ic0.call_data_append : (src : i32, size : i32) -> ()` + + Appends the specified bytes to the argument of the call. Initially, the argument is empty. + + This may be called multiple times between `ic0.call_new` and `ic0.call_perform`. + +- `ic0.call_cycles_add : (amount : i64) -> ()` + + This adds cycles onto a call. See [Cycles](#system-api-cycles). + + This may be called multiple times between `ic0.call_new` and `ic0.call_perform`. + +- `ic0.call_cycles_add128 : (amount_high : i64, amount_low : i64) -> ()` + + This adds cycles onto a call. See [Cycles](#system-api-cycles). + + This may be called multiple times between `ic0.call_new` and `ic0.call_perform`. + +- `ic0.call_perform : () -> ( err_code : i32 )` + + This concludes assembling the call. It queues the call message to the given destination, but does not actually act on it until the current WebAssembly function returns without trapping. + + If the function returns `0` as the `err_code`, the IC was able to enqueue the call. In this case, the call will either be delivered, returned because the destination canister does not exist or returned because of an out of cycles condition. This also means that exactly one of the reply or reject callbacks will be executed. + + If the function returns a non-zero value, the call cannot (and will not be) performed. This can happen due to a lack of resources within the IC, but also if it would reduce the current cycle balance to a level below where the canister would be frozen. + + After `ic0.call_perform` and before the next call to `ic0.call_new`, all other `ic0.call_*` function calls trap. + +### Cycles {#system-api-cycles} + +Each canister maintains a balance of *cycles*, which are used to pay for platform usage. Cycles are represented by 128-bit values. + +:::note + +This specification currently does not go into details about which actions cost how many cycles and/or when. In general, you must assume that the canister's cycle balance can change arbitrarily between method executions, and during each System API function call, unless explicitly mentioned otherwise. + +::: + +- `ic0.canister_cycle_balance : () → i64` + + Indicates the current cycle balance of the canister. It is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add`. After execution of the message, the IC may add unused cycles from the reserve back to the balance. + +:::note + +This call traps if the current balance does not fit into a 64-bit value. Canisters that need to deal with larger cycles balances should use `ic0.canister_cycles_balance128` instead. + +::: + +- `ic0.canister_cycle_balance128 : (dst : i32) → ()` + + Indicates the current cycle balance of the canister by copying the value at the location `dst` in the canister memory. It is the canister balance before the execution of the current message, minus a reserve to pay for the execution of the current message, minus any cycles queued up to be sent via `ic0.call_cycles_add128`. After execution of the message, the IC may add unused cycles from the reserve back to the balance. + + This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +- `ic0.msg_cycles_available : () → i64` + + Returns the amount of cycles that were transferred by the caller of the current call, and is still available in this message. + + Initially, in the update method entry point, this is the amount that the caller passed to the canister. When cycles are accepted (`ic0.msg_cycles_accept`), this reports fewer cycles accordingly. When the call is responded to (reply or reject), all available cycles are refunded to the caller, and this will return 0. + +:::note + +This call traps if the amount of cycles available does not fit into a 64-bit value. Please use `ic0.msg_cycles_available128` instead. + +::: + +- `ic0.msg_cycles_available128 : (dst : i32) → ()` + + Indicates the number of cycles transferred by the caller of the current call, still available in this message. The amount of cycles is represented by a 128-bit value. This call copies this value starting at the location `dst` in the canister memory. + + Initially, in the update method entry point, this is the amount that the caller passed to the canister. When cycles are accepted (`ic0.msg_cycles_accept128`), this reports fewer cycles accordingly. When the call is responded to (reply or reject), all available cycles are refunded to the caller, and this will report 0 cycles. + + This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +- `ic0.msg_cycles_accept : (max_amount : i64) → (amount : i64)` + + This moves cycles from the call to the canister balance. It moves as many cycles as possible, up to these constraints: + + It moves no more cycles than `max_amount`. + + It moves no more cycles than available according to `ic0.msg_cycles_available`, and + + It can be called multiple times, each time possibly adding more cycles to the balance. + + The return value indicates how many cycles were actually moved. + + This system call does not trap. + +:::tip + +Example: To accept all cycles provided in a call, invoke `ic0.msg_cycles_accept(ic0.msg_cycles_available())` in the method handler or a callback handler, *before* calling reply or reject. + +::: + +- `ic0.msg_cycles_accept128 : (max_amount_high : i64, max_amount_low : i64, dst : i32) → ()` + + This moves cycles from the call to the canister balance. It moves as many cycles as possible, up to these constraints: + + It moves no more cycles than the amount obtained by combining `max_amount_high` and `max_amount_low`. Cycles are represented by 128-bit values. + + It moves no more cycles than available according to `ic0.msg_cycles_available128`, and + + It can be called multiple times, each time possibly adding more cycles to the balance. + + This call also copies the amount of cycles that were actually moved starting at the location `dst` in the canister memory. + + This does not trap. + + This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +- `ic0.call_cycles_add : (amount : i64) → ()` + + This function moves cycles from the canister balance onto the call under construction, to be transferred with that call. + + The cycles are deducted from the balance as shown by `ic0.canister_cycle_balance` immediately, and moved back if the call cannot be performed (e.g. if `ic0.call_perform` signals an error, or if the canister invokes `ic0.call_new` or returns without calling `ic0.call_perform`). + + This system call traps if trying to transfer more cycles than are in the current balance of the canister. + +- `ic0.call_cycles_add128 : (amount_high : i64, amount_low : i64) → ()` + + This function moves cycles from the canister balance onto the call under construction, to be transferred with that call. + + The amount of cycles it moves is represented by a 128-bit value which can be obtained by combining the `amount_high` and `amount_low` parameters. + + The cycles are deducted from the balance as shown by `ic0.canister_cycles_balance128` immediately, and moved back if the call cannot be performed (e.g. if `ic0.call_perform` signals an error, or if the canister invokes `ic0.call_new` or returns without calling `ic0.call_perform`). + + This traps if trying to transfer more cycles than are in the current balance of the canister. + + This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +- `ic0.msg_cycles_refunded : () → i64` + + This function can only be used in a callback handler (reply or reject), and indicates the amount of cycles that came back with the response as a refund. The refund has already been added to the canister balance automatically. + +:::note + +This call traps if the amount of cycles refunded does not fit into a 64-bit value. In general, it is recommended to use `ic0.msg_cycles_refunded128` instead. + +::: + +- `ic0.msg_cycles_refunded128 : (dst : i32) → ()` + + This function can only be used in a callback handler (reply or reject), and indicates the amount of cycles that came back with the response as a refund. The refund has already been added to the canister balance automatically. + + This system call is experimental. It may be changed or removed in the future. Canisters using it may stop working. + +### Stable memory {#system-api-stable-memory} + +Canisters have the ability to store and retrieve data from a secondary memory. The purpose of this *stable memory* is to provide space to store data beyond upgrades. The interface mirrors roughly the memory-related instructions of WebAssembly, and tries to be forward compatible with exposing this feature as an additional memory. + +The stable memory is initially empty and can be grown up to 32 GiB (provided the subnet has capacity). + +- `ic0.stable_size : () → (page_count : i32)` + + returns the current size of the stable memory in WebAssembly pages. (One WebAssembly page is 64KiB) + + This system call traps if the size of the stable memory exceeds 232 bytes. + +- `ic0.stable_grow : (new_pages : i32) → (old_page_count : i32)` + + tries to grow the memory by `new_pages` many pages containing zeroes. + + This system call traps if the *previous* size of the memory exceeds 232 bytes. + + If the *new* size of the memory exceeds 232 bytes or growing is unsuccessful, then it returns `-1`. + + Otherwise, it grows the memory and returns the *previous* size of the memory in pages. + +- `ic0.stable_write : (offset : i32, src : i32, size : i32) → ()` + + copies the data referred to by `src`/`size` out of the canister and replaces the corresponding segment starting at `offset` in the stable memory. + + This system call traps if the size of the stable memory exceeds 232 bytes. + + It also traps if `src+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. + +- `ic0.stable_read : (dst : i32, offset : i32, size : i32) → ()` + + copies the data referred to by `offset`/`size` out of the stable memory and replaces the corresponding bytes starting at `dest` in the canister memory. + + This system call traps if the size of the stable memory exceeds 232 bytes. + + It also traps if `dst+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory + +- `ic0.stable64_size : () → (page_count : i64)` + + returns the current size of the stable memory in WebAssembly pages. (One WebAssembly page is 64KiB) + +- `ic0.stable64_grow : (new_pages : i64) → (old_page_count : i64)` + + tries to grow the memory by `new_pages` many pages containing zeroes. + + If successful, returns the *previous* size of the memory (in pages). Otherwise, returns `-1`. + +- `ic0.stable64_write : (offset : i64, src : i64, size : i64) → ()` + + Copies the data from location \[src, src+size) of the canister memory to location \[offset, offset+size) in the stable memory. + + This system call traps if `src+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. + +- `ic0.stable64_read : (dst : i64, offset : i64, size : i64) → ()` + + Copies the data from location \[offset, offset+size) of the stable memory to the location \[dst, dst+size) in the canister memory. + + This system call traps if `dst+size` exceeds the size of the WebAssembly memory or `offset+size` exceeds the size of the stable memory. + +### System time {#system-api-time} + +The canister can query the IC for the current time. + +`ic0.time : () -> i64` + +The time is given as nanoseconds since 1970-01-01. The IC guarantees that + +- the time, as observed by the canister, is monotonically increasing, even across canister upgrades. + +- within an invocation of one entry point, the time is constant. + +The times observed by different canisters are unrelated, and calls from one canister to another may appear to travel "backwards in time". + +:::note + +While an implementation will likely try to keep the time returned by `ic0.time` close to the real time, this is not formally part of this specification. + +::: + +### Global timer {#global-timer} + +The canister can set a global timer to make the system schedule a call to the exported `canister_global_timer` Wasm method after the specified time. The time must be provided as nanoseconds since 1970-01-01. + +`ic0.global_timer_set : (timestamp : i64) -> i64` + +The function returns the previous value of the timer. If no timer is set before invoking the function, then the function returns zero. + +Passing zero as an argument to the function deactivates the timer and thus prevents the system from scheduling calls to the canister's `canister_global_timer` Wasm method. + +### Performance counter {#system-api-performance-counter} + +The canister can query the "performance counter", which is a deterministic monotonically increasing integer approximating the amount of work the canister has done since the beginning of the current execution. + +`ic0.performance_counter : (counter_type : i32) -> i64` + +The argument `type` decides which performance counter to return: + +- 0 : instruction counter. The number of WebAssembly instructions the system has determined that the canister has executed. + +In the future, we might expose more performance counters. + +The system resets the counter at the beginning of each [Entry points](#entry-points) invocation. + +The main purpose of this counter is to facilitate in-canister performance profiling. + +### Controller check {#system-api-controller-check} + +The canister can check whether a given principal is one of its controllers. + +`ic0.is_controller : (src : i32, size: i32) -> (result: i32)` + +Checks whether the principal identified by `src`/`size` is one of the controllers of the canister. If yes, then a value of 1 is returned, otherwise a 0 is returned. It can be called multiple times. + +This system call traps if `src+size` exceeds the size of the WebAssembly memory or the principal identified by `src`/`size` is not a valid binary encoding of a principal. + +### Certified data {#system-api-certified-data} + +For each canister, the IC keeps track of "certified data", a canister-defined blob. For fresh canisters (upon install or reinstall), this blob is the empty blob (`""`). + +- `ic0.certified_data_set : (src: i32, size : i32) -> ()` + + The canister can update the certified data with this call. The passed data must be no larger than 32 bytes. This can be used any number of times. + +When executing a query method via a query call (i.e. in non-replicated state), the canister can fetch a certificate that authenticates to third parties the value last set via `ic0.certified_data_set`. + +- `ic0.data_certificate_present : () -> i32` + + returns `1` if a certificate is present, and `0` otherwise. + + This will return `1` when called from a query method when invoked via a query call. + + This will return `0` if the query method is executed within replicated execution (e.g. when invoked via an update call or inter-canister call). + +- `ic0.data_certificate_size : () → i32` and `ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) → ()` + + Copies the certificate for the current value of the certified data to the canister. + + The certificate is a blob as described in [Certification](#certification) that contains the values at path `/canister//certified_data` and at path `/time` of [The system state tree](#state-tree). + + If this `certificate` includes subnet delegations (possibly nested), then the id of the current canister will be included in each delegation's canister id range. + + This traps if `ic0.data_certificate_present()` returns `0`. + +### Debugging aids + +In a local canister execution environment, the canister needs a way to emit textual trace messages. On the "real" network, these do not do anything. + +- `ic0.debug_print : (src : i32, size : i32) -> ()` + + When executing in an environment that supports debugging, this copies out the data specified by `src` and `size`, and logs, prints or stores it in an environment-appropriate way. The copied data may likely be a valid string in UTF8-encoding, but the environment should be prepared to handle binary data (e.g. by printing it in escaped form). The data does typically not include a terminating `\0` or `\n`. + + Semantically, this function is always a no-op, and never traps, even if the `src+size` exceeds the size of the memory, or if this function is executed from `(start)`. If the environment cannot perform the print, it just skips it. + +Similarly, the System API allows the canister to effectively trap, but give some indication about why it trapped: + +- `ic0.trap : (src : i32, size : i32) -> ()` + + This function always traps. + + The environment may copy out the data specified by `src` and `size`, and log, print or store it in an environment-appropriate way, or include it in system-generated reject messages where appropriate. The copied data may likely be a valid string in UTF8-encoding, but the environment should be prepared to handle binary data (e.g. by printing it in escaped form or substituting invalid characters). + +### Outlook: Using Host References {#host-references} + +The Internet Computer aims to make the most of the WebAssembly platform, and embraces WebAssembly features. With WebAssembly host references, we can make the platform more secure, the interfaces more abstract and more compositional. The above `ic0` System API does not yet use WebAssembly host references. Once they become available on our platform, a new version of the System API using host references will be available via the `ic` module. The changes will be, at least + +1. The introduction of a `api_nonce` reference, which models the capability to use the System API. It is passed as an argument to `canister_init`, `canister_update ` etc., and expected as an argument by almost all System API function calls. (The debugging aids remain unconstrained.) + +2. The use of references, instead of binary blobs, to address principals (user ids, canister ids), e.g. in `ic0.msg_caller` or in `ic0.call_new`. Additional functions will be provided to convert between the transparent binary representation of principals and references. + +3. Making the builder interface to create calls build calls identified by a reference, rather than having an implicit partial call in the background. + +A canister may only use the old *or* the new interface; the IC detects which interface the canister intends to use based on the names and types of its function imports and exports. + +## The IC management canister {#ic-management-canister} + +The interfaces above provide the fundamental ability for external users and canisters to contact other canisters. But the Internet Computer provides additional functionality, such as canister and user management. This functionality is exposed to external users and canisters via the *IC management canister*. + +:::note + +The *IC management canister* is just a facade; it does not actually exist as a canister (with isolated state, Wasm code, etc.). + +::: + +The IC management canister address is `aaaaa-aa` (i.e. the empty blob). + +It is possible to use the management canister via external requests (a.k.a. ingress messages). The cost of processing that request is charged to the canister that is being managed. Most methods only permit the controllers to call them. Calls to `raw_rand` and `deposit_cycles` are never accepted as ingress messages. + +### Interface overview {#ic-candid} + +The [interface description](_attachments/ic.did) below, in [Candid syntax](https://github.com/dfinity/candid/blob/master/spec/Candid.md), describes the available functionality. +``` candid name= ic-interface file file=_attachments/ic.did +``` + +The binary encoding of arguments and results are as per Candid specification. + +### IC method `create_canister` {#ic-create_canister} + +Before deploying a canister, the administrator of the canister first has to register it with the IC, to get a canister id (with an empty canister behind it), and then separately install the code. + +The optional `settings` parameter can be used to set the following settings: + +- `controllers` (`vec principal`) + + A list of principals. Must be between 0 and 10 in size. This value is assigned to the *controllers* attribute of the canister. + + Default value: A list containing only the caller of the `create_canister` call. + +- `compute_allocation` (`nat`) + + Must be a number between 0 and 100, inclusively. It indicates how much compute power should be guaranteed to this canister, expressed as a percentage of the maximum compute power that a single canister can allocate. If the IC cannot provide the requested allocation, for example because it is oversubscribed, the call will be rejected. + + Default value: 0 + +- `memory_allocation` (`nat`) + + Must be a number between 0 and 248 (i.e 256TB), inclusively. It indicates how much memory the canister is allowed to use in total. Any attempt to grow memory usage beyond this allocation will fail. If the IC cannot provide the requested allocation, for example because it is oversubscribed, the call will be rejected. If set to 0, then memory growth of the canister will be best-effort and subject to the available memory on the IC. + + Default value: 0 + +- `freezing_threshold` (`nat`) + + Must be a number between 0 and 264-1, inclusively, and indicates a length of time in seconds. + + A canister is considered frozen whenever the IC estimates that the canister would be depleted of cycles before `freezing_threshold` seconds pass, given the canister's current size and the IC's current cost for storage. + + Calls to a frozen canister will be rejected (like for a stopping canister). Additionally, a canister cannot perform calls if that would, due the cost of the call and transferred cycles, would push the balance into frozen territory; these calls fail with `ic0.call_perform` returning a non-zero error code. + + Default value: 2592000 (approximately 30 days). + +The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. + +Until code is installed, the canister is `Empty` and behaves like a canister that has no public methods. + +### IC method `update_settings` {#ic-update_settings} + +Only *controllers* of the canister can update settings. See [IC method](#ic-create_canister) for a description of settings. + +Not including a setting in the `settings` record means not changing that field. The defaults described above are only relevant during canister creation. + +The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. + +### IC method `install_code` {#ic-install_code} + +This method installs code into a canister. + +Only controllers of the canister can install code. + +- If `mode = install`, the canister must be empty before. This will instantiate the canister module and invoke its `canister_init` method (if present), as explained in Section "[Canister initialization](#system-api-init)", passing the `arg` to the canister. + +- If `mode = reinstall`, if the canister was not empty, its existing code and state is removed before proceeding as for `mode = install`. + + Note that this is different from `uninstall_code` followed by `install_code`, as that will forcibly reject all calls awaiting a response. + +- If `mode = upgrade`, this will perform an upgrade of a non-empty canister as described in [Canister upgrades](#system-api-upgrades), passing `arg` to the `canister_post_upgrade` method of the new instance. + +This is atomic: If the response to this request is a `reject`, then this call had no effect. + +:::note + +Some canisters may not be able to make sense of callbacks after upgrades; these should be stopped first, to wait for all outstanding callbacks, or be uninstalled first, to prevent outstanding callbacks from being invoked (by marking the corresponding call contexts as deleted). It is expected that the canister admin (or their tooling) does that separately. + +::: + +The `wasm_module` field specifies the canister module to be installed. The system supports multiple encodings of the `wasm_module` field, as described in [Canister module format](#canister-module-format): + +- If the `wasm_module` starts with byte sequence `[0x00, 'a', 's', 'm']`, the system parses `wasm_module` as a raw WebAssembly binary. + +- If the `wasm_module` starts with byte sequence `[0x1f, 0x8b, 0x08]`, the system parses `wasm_module` as a gzip-compressed WebAssembly binary. + +The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. + +### IC method `uninstall_code` {#ic-uninstall_code} + +This method removes a canister's code and state, making the canister *empty* again. + +Only controllers of the canister can uninstall code. + +Uninstalling a canister's code will reject all calls that the canister has not yet responded to, and drop the canister's code and state. Outstanding responses to the canister will not be processed, even if they arrive after code has been installed again. + +The canister is now [empty](#canister-lifecycle). In particular, any incoming or queued calls will be rejected. + +A canister after *uninstalling* retains its *cycles* balance, *controllers*, status, and allocations. + +The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. + +### IC method `canister_status` {#ic-canister_status} + +Indicates various information about the canister. It contains: + +- The status of the canister. It could be one of `running`, `stopping` or `stopped`. + +- A SHA256 hash of the module installed on the canister. This is `null` if the canister is empty. + +- The controllers of the canister. + +- The memory size taken by the canister. + +- The cycle balance of the canister. + +Only the controllers of the canister or the canister itself can request its status. + +### IC method `canister_info` {#ic-canister-info} + +Provides the history of the canister, its current module SHA-256 hash, and its current controllers. Every canister can call this method on every other canister (including itself). Users cannot call this method. + +The canister history consists of a list of canister changes (canister creation, code uninstallation, code deployment, or controllers change). Every canister change consists of the system timestamp at which the change was performed, the canister version after performing the change, the change's origin (a user or a canister), and its details. The change origin includes the principal (called *originator* in the following) that initiated the change and, if the originator is a canister, the originator's canister version when the originator initiated the change (if available). Code deployments are described by their mode (code install, code reinstall, code upgrade) and the SHA-256 hash of the newly deployed canister module. Canister creations and controllers changes are described by the full new set of the canister controllers after the change. + +The system can drop the oldest canister changes from the list to keep its length bounded (at least `20` changes are guaranteed to remain in the list). The system also drops all canister changes if the canister runs out of cycles. + +The following parameters should be supplied for the call: + +- `canister_id`: the canister ID of the canister to retrieve information about. + +- `num_requested_changes`: optional, specifies the number of requested canister changes. If not provided, the default value of `0` will be used. + +The returned response contains the following fields: + +- `total_num_changes`: the total number of canister changes that have been ever recorded in the history. This value does not change if the system drops the oldest canister changes from the list of changes. + +- `recent_changes`: the list containing the most recent canister changes. If `num_requested_changes` is provided, then this list contains that number of changes or, if more changes are requested than available in the history, then this list contains all changes available in the history. If `num_requested_changes` is not specified, then this list is empty. + +- `module_hash`: the SHA-256 hash of the currently installed canister module (or `null` if the canister is empty). + +- `controllers`: the current set of canister controllers. + +### IC method `stop_canister` {#ic-stop_canister} + +The controllers of a canister may stop a canister (e.g., to prepare for a canister upgrade). + +Stopping a canister is not an atomic action. The immediate effect is that the status of the canister is changed to `stopping` (unless the canister is already stopped). The IC will reject all calls to a stopping canister, indicating that the canister is stopping. Responses to a stopping canister are processed as usual. When all outstanding responses have been processed (so there are no open call contexts), the canister status is changed to `stopped` and the management canister responds to the caller of the `stop_canister` request. + +### IC method `start_canister` {#ic-start_canister} + +A canister may be started by its controllers. + +If the canister status was `stopped` or `stopping` then the canister status is simply set to `running`. In the latter case all `stop_canister` calls which are processing fail (and are rejected). + +If the canister was already `running` then the status stays unchanged. + +### IC method `delete_canister` {#ic-delete_canister} + +This method deletes a canister from the IC. + +Only controllers of the canister can delete it and the canister must already be stopped. Deleting a canister cannot be undone, any state stored on the canister is permanently deleted and its cycles are discarded. Once a canister is deleted, its ID cannot be reused. + +### IC method `deposit_cycles` {#ic-deposit_cycles} + +This method deposits the cycles included in this call into the specified canister. + +There is no restriction on who can invoke this method. + +### IC method `raw_rand` {#ic-raw_rand} + +This method takes no input and returns 32 pseudo-random bytes to the caller. The return value is unknown to any part of the IC at time of the submission of this call. A new return value is generated for each call to this method. + +### IC method `ecdsa_public_key` {#ic-ecdsa_public_key} + +:::note + +The ECDSA API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. + +::: + +This method returns a [SEC1](https://www.secg.org/sec1-v2.pdf) encoded ECDSA public key for the given canister using the given derivation path. If the `canister_id` is unspecified, it will default to the canister id of the caller. The `derivation_path` is a vector of variable length byte strings. Each byte string may be of arbitrary length, including empty. The total number of strings in `derivation_path` can be at most 255. The `key_id` is a struct specifying both a curve and a name. The availability of a particular `key_id` depends on implementation. + +For curve `secp256k1`, the public key is derived using a generalization of BIP32 (see [ia.cr/2021/1330, Appendix D](https://ia.cr/2021/1330)). To derive (non-hardened) [BIP-0032](https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki)-compatible public keys, each byte string (`blob`) in the `derivation_path` must be a 4-byte big-endian encoding of an unsigned integer less than 231. + +The return result is an extended public key consisting of an ECDSA `public_key`, encoded in [SEC1](https://www.secg.org/sec2-v2.pdf) compressed form, and a `chain_code`, which can be used to deterministically derive child keys of the `public_key`. + +This call requires that the ECDSA feature is enabled, and the `canister_id` meets the requirement of a canister id. Otherwise it will be rejected. + +### IC method `sign_with_ecdsa` {#ic-sign_with_ecdsa} + +:::note + +The ECDSA API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. + +::: + +This method returns a new [ECDSA](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf) signature of the given `message_hash` that can be separately verified against a derived ECDSA public key. This public key can be obtained by calling `ecdsa_public_key` with the caller's `canister_id`, and the same `derivation_path` and `key_id` used here. + +The signatures are encoded as the concatenation of the [SEC1](https://www.secg.org/sec2-v2.pdf) encodings of the two values r and s. For curve `secp256k1`, this corresponds to 32-byte big-endian encoding. + +This call requires that the ECDSA feature is enabled, the caller is a canister, and `message_hash` is 32 bytes long. Otherwise it will be rejected. + +### IC method `http_request` {#ic-http_request} + +:::note + +The IC http\_request API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. + +::: + +This method makes an HTTP request to a given URL and returns the HTTP response, possibly after a transformation. + +The canister should aim to issue *idempotent* requests, meaning that it must not change the state at the remote server, or the remote server has the means to identify duplicated requests. Otherwise, the risk of failure increases. + +The responses for all identical requests must match too. However, a web service could return slightly different responses for identical idempotent requests. For example, it may include some unique identification or a timestamp that would vary across responses. + +For this reason, the calling canister can supply a transformation function, which the IC uses to let the canister sanitize the responses from such unique values. The transformation function is executed separately on the corresponding response received for a request. The final response will only be available to the calling canister. + +Currently, the `GET`, `HEAD`, and `POST` methods are supported for HTTP requests. + +It is important to note the following for the usage of the `POST` method: - The calling canister must make sure that the remote server is able to handle idempotent requests sent from multiple sources. This may require, for example, to set a certain request header to uniquely identify the request. - There are no confidentiality guarantees on the request content. There is no guarantee that all sent requests are as specified by the canister. If the canister receives a response, then at least one request that was sent matched the canister's request, and the response was to that request. + +For security reasons, only HTTPS connections are allowed (URLs must start with `https://`). The IC uses industry-standard root CA lists to validate certificates of remote web servers. + +The **size** of an HTTP request from the canister or an HTTP response from the remote HTTP server is the total number of bytes representing the names and values of HTTP headers and the HTTP body. The maximal size for the request from the canister is `2MB` (`2,000,000B`). Each request can specify a maximal size for the response from the remote HTTP server. The upper limit on the maximal size for the response is `2MB` (`2,000,000B`) and this value also applies if no maximal size value is specified. An error will be returned when the request or response is larger than the maximal size. + +The following parameters should be supplied for the call: + +- `url` - the requested URL. The URL must be valid according to [RFC-3986](https://www.ietf.org/rfc/rfc3986.txt) and its length must not exceed `8192`. The URL may specify a custom port number. + +- `max_response_bytes` - optional, specifies the maximal size of the response in bytes. If provided, the value must not exceed `2MB` (`2,000,000B`). The call will be charged based on this parameter. If not provided, the maximum of `2MB` will be used. + +- `method` - currently, only GET, HEAD, and POST are supported + +- `headers` - list of HTTP request headers and their corresponding values + +- `body` - optional, the content of the request's body + +- `transform` - an optional record that includes a function that transforms raw responses to sanitized responses, and a byte-encoded context that is provided to the function upon invocation, along with the response to be sanitized. If provided, the calling canister itself must export this function. + +Cycles to pay for the call must be explicitly transferred with the call, i.e., they are not deducted from the caller's balance implicitly (e.g., as for inter-canister calls). + +The returned response (and the response provided to the `transform` function, if specified) contains the following fields: + +- `status` - the response status (e.g., 200, 404) + +- `headers` - list of HTTP response headers and their corresponding values + +- `body` - the response's body + +The `transform` function may, for example, transform the body in any way, add or remove headers, modify headers, etc. The maximal number of bytes representing the response produced by the `transform` function is `2MB` (`2,000,000B`). Note that the number of bytes representing the response produced by the `transform` function includes the serialization overhead of the encoding produced by the canister. + +When the transform function is invoked by the system due to a canister HTTP request, the caller's identity is the principal of the management canister. This information can be used by developers to implement access control mechanism for this function. + +The following additional limits apply to HTTP requests and HTTP responses from the remote sever: + +- the number of headers must not exceed `64`, + +- the number of bytes representing a header name or value must not exceed `8KiB`, and + +- the total number of bytes representing the header names and values must not exceed `48KiB`. + +:::note + +Currently, the Internet Computer mainnet only supports URLs that resolve to IPv6 destinations (i.e., the domain has a `AAAA` DNS record) in HTTP requests. + +::: + +:::warning + +If you do not specify the `max_response_bytes` parameter, the maximum of a `2MB` response will be charged for, which is expensive in terms of cycles. Always set the parameter to a reasonable upper bound of the expected network response size to not incur unnecessary cycles costs for your request. + +::: + +### IC method `provisional_create_canister_with_cycles` {#ic-provisional_create_canister_with_cycles} + +As a provisional method on development instances, the `provisional_create_canister_with_cycles` method is provided. It behaves as `create_canister`, but initializes the canister's balance with `amount` fresh cycles (using `DEFAULT_PROVISIONAL_CYCLES_BALANCE` if `amount = null`). If `specified_id` is provided, the canister is created under this id. Note that canister creation using `create_canister` or `provisional_create_canister_with_cycles` with `specified_id = null` can fail after calling `provisional_create_canister_with_cycles` with provided `specified_id`. In that case, canister creation should be retried. + +The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. + +Cycles added to this call via `ic0.call_cycles_add128` are returned to the caller. + +This method is only available in local development instances. + +### IC method `provisional_top_up_canister` {#ic-provisional_top_up_canister} + +As a provisional method on development instances, the `provisional_top_up_canister` method is provided. It adds `amount` cycles to the balance of canister identified by `amount`. + +Cycles added to this call via `ic0.call_cycles_add128` are returned to the caller. + +Any user can top-up any canister this way. + +This method is only available in local development instances. + +## The IC Bitcoin API {#ic-bitcoin-api} + +:::note + +The IC Bitcoin API is considered EXPERIMENTAL. Canister developers must be aware that the API may evolve in a non-backward-compatible way. + +::: + +The Bitcoin functionality is exposed via the management canister. Information about Bitcoin can be found in the [Bitcoin developer guides](https://developer.bitcoin.org/devguide/). Invoking the functions of the Bitcoin API will cost cycles. We refer the reader to the \[Bitcoin documentation\]() for further relevant information and the \[IC pricing page\]() for information on pricing for the Bitcoin mainnet and testnet. + +### IC method `bitcoin_get_utxos` {#ic-bitcoin_get_utxos} + +Given a `get_utxos_request`, which must specify a Bitcoin address and a Bitcoin network (`mainnet` or `testnet`), the function returns all unspent transaction outputs (UTXOs) associated with the provided address in the specified Bitcoin network based on the current view of the Bitcoin blockchain available to the Bitcoin component. The UTXOs are returned sorted by block height in descending order. + +The following address formats are supported: + +- Pay to public key hash (P2PKH) + +- Pay to script hash (P2SH) + +- Pay to witness public key hash (P2WPKH) + +- Pay to witness script hash (P2WSH) + +- Pay to taproot (P2TR) + +If the address is malformed, the call is rejected. + +The optional `filter` parameter can be used to restrict the set of returned UTXOs, either providing a minimum number of confirmations or a page reference when pagination is used for addresses with many UTXOs. In the first case, only UTXOs with at least the provided number of confirmations are returned, i.e., transactions with fewer than this number of confirmations are not considered. In other words, if the number of confirmations is `c`, an output is returned if it occurred in a transaction with at least `c` confirmations and there is no transaction that spends the same output with at least `c` confirmations. + +There is an upper bound of 144 on the minimum number of confirmations. If a larger minimum number of confirmations is specified, the call is rejected. Note that this is not a severe restriction as the minimum number of confirmations is typically set to a value around 6 in practice. + +It is important to note that the validity of transactions is not verified in the Bitcoin component. The Bitcoin component relies on the proof of work that goes into the blocks and the verification of the blocks in the Bitcoin network. For a newly discovered block, a regular Bitcoin (full) node therefore provides a higher level of security than the Bitcoin component, which implies that it is advisable to set the number of confirmations to a reasonably large value, such as 6, to gain confidence in the correctness of the returned UTXOs. + +There is an upper bound of 10,000 UTXOs that can be returned in a single request. For addresses that contain sufficiently many UTXOs, a partial set of the address's UTXOs are returned along with a page reference. + +In the second case, a page reference (a series of bytes) must be provided, which instructs the Bitcoin component to collect UTXOs starting from the corresponding "page". + +A `get_utxos_request` without the optional `filter` results in a request that considers the full blockchain, which is equivalent to setting `min_confirmations` to 0. + +The recommended workflow is to issue a request with the desired number of confirmations. If the `next_page` field in the response is not empty, there are more UTXOs than in the returned vector. In that case, the `page` field should be set to the `next_page` bytes in the subsequent request to obtain the next batch of UTXOs. + +### IC method `bitcoin_get_balance` {#ic-bitcoin_get_balance} + +Given a `get_balance_request`, which must specify a Bitcoin address and a Bitcoin network (`mainnet` or `testnet`), the function returns the current balance of this address in `Satoshi` (10^8 Satoshi = 1 Bitcoin) in the specified Bitcoin network. The same address formats as for [`bitcoin_get_utxos`](#ic-bitcoin_get_utxos) are supported. + +If the address is malformed, the call is rejected. + +The optional `min_confirmations` parameter can be used to limit the set of considered UTXOs for the calculation of the balance to those with at least the provided number of confirmations in the same manner as for the [`bitcoin_get_utxos`](#ic-bitcoin_get_utxos) call. + +Given an address and the optional `min_confirmations` parameter, `bitcoin_get_balance` iterates over all UTXOs, i.e., the same balance is returned as when calling [`bitcoin_get_utxos`](#ic-bitcoin_get_utxos) for the same address and the same number of confirmations and, if necessary, using pagination to get all UTXOs for the same tip hash. + +### IC method `bitcoin_send_transaction` {#ic-bitcoin_send_transaction} + +Given a `send_transaction_request`, which must specify a `blob` of a Bitcoin transaction and a Bitcoin network (`mainnet` or `testnet`), several checks are performed: + +- The transaction is well formed. + +- The transaction only consumes unspent outputs with respect to the current (longest) blockchain, i.e., there is no block on the (longest) chain that consumes any of these outputs. + +- There is a positive transaction fee. + +If at least one of these checks fails, the call is rejected. + +If the transaction passes these tests, the transaction is forwarded to the specified Bitcoin network. Note that the function does not provide any guarantees that the transaction will make it into the mempool or that the transaction will ever appear in a block. + +### IC method `bitcoin_get_current_fee_percentiles` {#ic-bitcoin_get_current_fee_percentiles} + +The transaction fees in the Bitcoin network change dynamically based on the number of pending transactions. It must be possible for a canister to determine an adequate fee when creating a Bitcoin transaction. + +This function returns fee percentiles, measured in millisatoshi/vbyte (1000 millisatoshi = 1 satoshi), over the last 10,000 transactions in the specified network, i.e., over the transactions in the last approximately 4-10 blocks. + +The [standard nearest-rank estimation method](https://en.wikipedia.org/wiki/Percentile#The_nearest-rank_method), inclusive, with the addition of a 0th percentile is used. Concretely, for any i from 1 to 100, the ith percentile is the fee with rank `⌈i * 100⌉`. The 0th percentile is defined as the smallest fee (excluding coinbase transactions). + +## Certification {#certification} + +Some parts of the IC state are exposed to users in a tamperproof way via certification: the IC can reveal a *partial state tree* which includes just the data of interest, together with a signature on the root hash of the state tree. This means that a user can be sure that the response is correct, even if the user happens to be communicating with a malicious node, or has received the certificate via some other untrusted way. + +To validate a value using a certificate, the user conceptually + +1. checks the validity of the partial tree using `verify_cert`, + +2. looks up the value in the certificate using `lookup` at a given path, which uses the subroutine `lookup_path` on the certificate's tree + +This mechanism is used in the `read_state` request type, and eventually also for other purposes. + +### Root of trust + +The root of trust is the *root public key*, which must be known to the user a priori. In a local canister execution environment, the key can be fetched via the [`/api/v2/status`](#api-status) endpoint. + +### Certificate + +A certificate consists of + +- a tree + +- a signature on the tree root hash valid under some *public key* + +- an optional *delegation* that links that public key to *root public key*. + +The IC will certify states by issuing certificates where the tree is a partial state tree. The state tree can be pruned by replacing subtrees with their root hashes (yielding a new and potentially smaller but still valid certificate) to only include paths pertaining to relevant data but still preserving enough information to recover the *tree root hash*. + +More formally, a certificate is described by the following data structure: + + Certificate = { + tree : HashTree + signature : Signature + delegation : NoDelegation | Delegation + } + HashTree + = Empty + | Fork HashTree HashTree + | Labeled Label HashTree + | Leaf blob + | Pruned Hash + Label = Blob + Hash = Blob + Signature = Blob + +A certificate is validated with regard to the root of trust by the following algorithm (which uses `check_delegation` defined in [Delegation](#certification-delegation)): + + verify_cert(cert) = + let root_hash = reconstruct(cert.tree) + let der_key = check_delegation(cert.delegation) // see section Delegations below + bls_key = extract_der(der_key) + verify_bls_signature(bls_key, cert.signature, domain_sep("ic-state-root") · root_hash) + + reconstruct(Empty) = H(domain_sep("ic-hashtree-empty")) + reconstruct(Fork t1 t2) = H(domain_sep("ic-hashtree-fork") · reconstruct(t1) · reconstruct(t2)) + reconstruct(Labeled l t) = H(domain_sep("ic-hashtree-labeled") · l · reconstruct(t)) + reconstruct(Leaf v) = H(domain_sep("ic-hashtree-leaf") · v) + reconstruct(Pruned h) = h + + domain_sep(s) = byte(|s|) · s + +where `H` is the SHA-256 hash function, + + verify_bls_signature : PublicKey -> Signature -> Blob -> Bool + +is the [BLS signature verification function](https://tools.ietf.org/html/draft-irtf-cfrg-bls-signature-04#section-4), ciphersuite BLS\_SIG\_BLS12381G1\_XMD:SHA-256\_SSWU\_RO\_NUL\_. See that document also for details on the encoding of BLS public keys and signatures, and + + extract_der : Blob -> Blob + +implements DER decoding of the public key, following [RFC4580](https://tools.ietf.org/html/rfc5480) using OID 1.3.6.1.4.1.44668.5.3.1.2.1 for the algorithm and 1.3.6.1.4.1.44668.5.3.2.1 for the curve. + +All state trees include the time at path `/time` (see [Time](#state-tree-time)). Users that get a certificate with a state tree can look up the timestamp to guard against working on obsolete data. + +### Lookup + +Given a (verified) tree, the user can fetch the value at a given path, which is a sequence of labels (blobs). In this document, we write paths suggestively with slashes as separators; the actual encoding is not actually using slashes as delimiters. + +The following algorithm looks up a `path` in a certificate, and returns either + +- the value + +- `Absent`, if the value is guaranteed to be absent in the original state tree, + +- `Unknown`, if this partial view does not include information about this path, or + +- `Error`, if the path does not make sense for this certificate: + +```html + +lookup(path, cert) = lookup_path(path, cert.tree) + +lookup_path([], Empty) = Absent +lookup_path([], Leaf v) = v +lookup_path([], Pruned _) = Unknown +lookup_path([], Labeled _ _) = Error +lookup_path([], Fork _ _) = Error + +lookup_path(l::ls, tree) = + match find_label(l, flatten_forks(tree)) with + | Absent -> Absent + | Unknown -> Unknown + | Error -> Error + | Found subtree -> lookup_path ls subtree + +flatten_forks(Empty) = [] +flatten_forks(Fork t1 t2) = flatten_forks(t1) · flatten_forks(t2) +flatten_forks(t) = [t] + +find_label(l, _ · Labeled l1 t · _) | l == l1 = Found t +find_label(l, _ · Labeled l1 _ · Labeled l2 _ · _) | l1 < l < l2 = Absent +find_label(l, Labeled l2 _ · _) | l < l2 = Absent +find_label(l, _ · Labeled l1 _ ) | l1 < l = Absent +find_label(l, [Leaf _]) = Absent +find_label(l, []) = Absent +find_label(l, _) = Unknown + +``` + +The IC will only produce well-formed state trees, and the above algorithm assumes well-formed trees. These have the property that labeled subtrees appear in strictly increasing order of labels, and are not mixed with leaves. More formally: + + well_formed(tree) = + (tree = Leaf _) ∨ (well_formed_forest(flatten_forks(tree))) + + well_formed_forest(trees) = + strictly_increasing([l | Label l _ ∈ trees]) ∧ + ∀ Label _ t ∈ trees. well_formed(t) ∧ + ∀ t ∈ trees. t ≠ Leaf _ + +### Delegation {#certification-delegation} + +The root key can delegate certification authority to other keys. + +A certificate by the root subnet does not have a delegation field. A certificate by other subnets include a delegation, which is itself a certificate that proves that the subnet is listed in the root subnet's state tree (see [Subnet information](#state-tree-subnet)), and reveals its public key. + +:::note + +The nested certificate *typically* does not itself again contain a delegation, although there is no reason why agents should enforce that property. + +::: + + Delegation = + Delegation { + subnet_id : Principal; + certificate : Certificate; + } + +A chain of delegations is verified using the following algorithm, which also returns the delegated key (a DER-encoded BLS key): + + check_delegation(NoDelegation) : public_bls_key = + return root_public_key + check_delegation(Delegation d) : public_bls_key = + verify_cert(d.certificate) + return lookup(["subnet",d.subnet_id,"public_key"],d.certificate) + +where `root_public_key` is the a priori known root key. + +Delegations are *scoped*, i.e., they indicate which set of canister principals the delegatee subnet may certify for. This set can be obtained from a delegation `d` using `lookup(["subnet",d.subnet_id,"canister_ranges"],d.certificate)`, which must be present, and is encoded as described in [Subnet information](#state-tree-subnet). The various applications of certificates describe if and how the subnet scope comes into play. + +### Encoding of certificates {#certification-encoding} + +The binary encoding of a certificate is a CBOR (see [CBOR](#cbor)) value according to the following CDDL (see [CDDL](#cddl)). You can also [download the file]({attachmentsdir}/certificates.cddl). + +The values in the [The system state tree](#state-tree) are encoded to blobs as follows: + +- natural numbers are leb128-encoded. + +- text values are UTF-8-encoded + +- blob values are encoded as is + +### Example + +Consider the following tree-shaped data (all single character strings denote labels, all other denote values) + + ─┬╴ "a" ─┬─ "x" ─╴"hello" + │ └╴ "y" ─╴"world" + ├╴ "b" ──╴ "good" + ├╴ "c" + └╴ "d" ──╴ "morning" + +A possible hash tree for this labeled tree might be, where `┬` denotes a fork. This is not a typical encoding (a fork with `Empty` on one side can be avoided), but it is valid. + + ─┬─┬╴"a" ─┬─┬╴"x" ─╴"hello" + │ │ │ └╴Empty + │ │ └╴ "y" ─╴"world" + │ └╴"b" ──╴"good" + └─┬╴"c" ──╴Empty + └╴"d" ──╴"morning" + +This tree has the following CBOR (see [CBOR](#cbor)) encoding + + 8301830183024161830183018302417882034568656c6c6f810083024179820345776f726c6483024162820344676f6f648301830241638100830241648203476d6f726e696e67 + +and the following root hash + + eb5c5b2195e62d996b84c9bcc8259d19a83786a2f59e0878cec84c811f669aa0 + +Pruning this tree with the following paths + + /a/y + /ax + /d + +would lead to this tree (with pruned subtree represented by their hash): + + ─┬─┬╴"a" ─┬─ 1B4FEFF9BEF8131788B0C9DC6DBAD6E81E524249C879E9F10F71CE3749F5A638 + │ │ └╴ "y" ─╴"world" + │ └╴"b" ──╴7B32AC0C6BA8CE35AC82C255FC7906F7FC130DAB2A090F80FE12F9C2CAE83BA6 + └─┬╴EC8324B8A1F1AC16BD2E806EDBA78006479C9877FED4EB464A25485465AF601D + └╴"d" ──╴"morning" + +Note that the `"b"` label is included (without content) to prove the absence of the `/ax` path. + +This tree encodes to CBOR as + + 83018301830241618301820458201b4feff9bef8131788b0c9dc6dbad6e81e524249c879e9f10f71ce3749f5a63883024179820345776f726c6483024162820458207b32ac0c6ba8ce35ac82c255fc7906f7fc130dab2a090f80fe12f9c2cae83ba6830182045820ec8324b8a1f1ac16bd2e806edba78006479c9877fed4eb464a25485465af601d830241648203476d6f726e696e67 + +and (obviously) the same root hash. + +In the pruned tree, the `lookup_path` function behaves as follows: + + lookup_path(["a", "a"], pruned_tree) = Unknown + lookup_path(["a", "y"], pruned_tree) = Found "world" + lookup_path(["aa"], pruned_tree) = Absent + lookup_path(["ax"], pruned_tree) = Absent + lookup_path(["b"], pruned_tree) = Unknown + lookup_path(["bb"], pruned_tree) = Unknown + lookup_path(["d"], pruned_tree) = Found "morning" + lookup_path(["e"], pruned_tree) = Absent + +## The HTTP Gateway protocol {#http-gateway} + +The HTTP Gateway Protocol has been moved into its own [specification](https://internetcomputer.org/docs/current/references/http-gateway-protocol-spec). + +## Abstract behavior {#abstract-behavior} + +The previous sections describe the interfaces, i.e. outer edges of the Internet Computer, but give only intuitive and vague information in prose about what these interfaces actually do. + +The present section aims to address that question with great precision, by describing the *abstract state* of the whole Internet Computer, and how this state can change in response to API function calls, or spontaneously (modeling asynchronous, distributed or non-deterministic execution). + +The design of this abstract specification (e.g. how and where pending messages are stored) are *not* to be understood to in any way prescribe a concrete implementation or software architecture. The goals here are formal precision and clarity, but not implementability, so this can lead to different ways of phrasing. + +### Notation + +We specify the behavior of the Internet Computer using ad-hoc pseudocode. + +The manipulated values are primitive values (numbers, text, binary blobs), aggregate values (lists, unordered lists a.k.a. bags, partial maps, records with fixed fields, named constructors) and functions. + +We use a concatenation operator `·` with various types: to extend sets and maps, or to concatenate lists with lists or lists with elements. + +The shape of values is described using a hand-wavy type system. We use `Foo = Nat` to define type aliases; now `Foo` can be used instead of `Nat`. Often, the right-hand side is a more complex type here, e.g. a record, or multiple possible types separated by a vertical bar (`|`). Partial maps are written as `Key ↦ Value` and the function type as `Argument → Result`. + +:::note + +All values are immutable! State change is specified by describing the new state, not by changing the existing state. + +::: + +Record fields are accessed using dot-notation (e.g. `S.request_id > 0`). To create a new record from an existing record `R` with some fields changed, the syntax `R where field = new_value` is used. This syntax can also be used to create new records with some deeply nested field changed: `R where some_map[key].field = new_value`. + +In the state transitions, upper-case variables (`S`, `C`, `Req_id`) are free variables: The state transition may be taken for any possible value of these variables. `S` always refers to the previous state. A state transition often comes with a list of *conditions*, which may restrict the values of these free variables. The *state after* is usually described using the record update syntax by starting with `S where`. + +For example, the condition `S.messages = Older_messages · M · Younger_messages` says that `M` is some message in field `messages` of the record `S`, and that `Younger_messages` and `Older_messages` are the other messages in the state. If the "state after" specifies `S with messages = Older_messages · Younger_messages`, then the message `M` is removed from the state. + +### Abstract state + +In this specification, we describe the Internet Computer as a state machine. In particular, there is a single piece of data that describes the complete state of the IC, called `S`. + +Of course, this is a huge simplification: The real Internet Computer is distributed and has a multi-component architecture, and the state is spread over many different components, some physically separated. But this simplification allows us to have a concise description of the behavior, and to easily make global decisions (such as, "is there any pending message"), without having to specify the bookkeeping that allows such global decisions. + +#### Identifiers + +Principals (canister ids and user ids) are blobs, but some of them have special form, as explained in [Special forms of Principals](#id-classes). + + type Principal = Blob + +The function + + mk_self_authenticating_id : PublicKey -> Principal + mk_self_authenticating_id pk = H(pk) · 0x02 + +calculates self-authenticating ids. + +The function + + mk_derived_id : Principal -> Blob -> Principal + mk_derived_id p nonce = H(|p| · p · nonce) · 0x03 + +calculates derived ids. With `|p|` we denote the length of the principal, in bytes, encoded as a single byte. + +The principal of the anonymous user is fixed: + + anonymous_id : Principal + anonymous_id = 0x04 + +The principal of the management canister is the empty blob (i.e. `aaaaa-aa`): + + ic_principal : Principal = "" + +These function domains and fixed values are mutually disjoint. + +Method names can be arbitrary pieces of text: + + MethodName = Text + +#### Abstract canisters {#abstract-canisters} + +The [WebAssembly System API](#system-api) is relatively low-level, and some of its details (e.g. that the argument data is queried using separate calls, and that closures are represented by a function pointer and a number, that method names need to be mangled) would clutter this section. Therefore, we abstract over the WebAssembly details as follows: + +- The state of a WebAssembly module (memory, tables, globals) is hidden behind an abstract `WasmState`. The `WasmState` contains the `StableMemory`, which can be extracted using `pre_upgrade` and passed to `post_upgrade`. + +- A canister module `CanisterModule` consists of an initial state, and a (pure) function that models function invocation. It either indicates that the canister function traps, or returns a new state together with a description of the invoked asynchronous System API calls. + + WasmState = (abstract) + StableMemory = (abstract) + Callback = (abstract) + + Arg = Blob; + CallerId = Principal; + + Timestamp = Nat; + CanisterVersion = Nat; + Env = { + time : Timestamp; + controllers : List Principal; + global_timer : Nat; + balance : Nat; + freezing_limit : Nat; + certificate : NoCertificate | Blob; + status : Running | Stopping | Stopped; + canister_version : CanisterVersion; + } + + RejectCode = Nat + Response = Reply Blob | Reject (RejectCode, Text) + MethodCall = { + callee : CanisterId; + method_name: MethodName; + arg: Blob; + transferred_cycles: Nat; + callback: Callback; + } + + UpdateFunc = WasmState -> Trap { cycles_used : Nat; } | Return { + new_state : WasmState; + new_calls : List MethodCall; + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; + response : NoResponse | Response; + cycles_accepted : Nat; + cycles_used : Nat; + } + QueryFunc = WasmState -> Trap { cycles_used : Nat; } | Return { + response : Response; + cycles_used : Nat; + } + SystemTaskFunc = WasmState -> Trap { cycles_used : Nat; } | Return { + new_state : WasmState; + new_calls : List MethodCall; + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; + cycles_used : Nat; + } + + AvailableCycles = Nat + RefundedCycles = Nat + + CanisterModule = { + init : (CanisterId, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { + new_state : WasmState; + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; + cycles_used : Nat; + } + pre_upgrade : (WasmState, Principal, Env) -> Trap { cycles_used : Nat; } | Return { + stable_memory : StableMemory; + new_certified_data : NoCertifiedData | Blob; + cycles_used : Nat; + } + post_upgrade : (CanisterId, StableMemory, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { + new_state : WasmState; + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; + cycles_used : Nat; + } + update_methods : MethodName ↦ ((Arg, CallerId, Env, AvailableCycles) -> UpdateFunc) + query_methods : MethodName ↦ ((Arg, CallerId, Env) -> QueryFunc) + heartbeat : (Env) -> SystemTaskFunc + global_timer : (Env) -> SystemTaskFunc + callbacks : (Callback, Response, RefundedCycles, Env, AvailableCycles) -> UpdateFunc + inspect_message : (MethodName, WasmState, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { + status : Accept | Reject; + cycles_used : Nat; + } + } + +This high-level interface presents a pure, mathematical model of a canister, and hides the bookkeeping required to provide the System API as seen in Section [Canister interface (System API)](#system-api). + +The `CanisterId` parameter of `init` and `post_upgrade` is merely passed through to the canister, via the `canister.self` system call. + +The `Env` parameter provides synchronous read-only access to portions of the system state and canister metadata that are always available. + +The parsing of a blob to a canister module and its public and private custom sections is modelled via the (possibly implicitly failing) functions + + parse_wasm_mod : Blob -> CanisterModule + parse_public_custom_sections : Blob -> Text ↦ Blob + parse_private_custom_sections : Blob -> Text ↦ Blob + +The concrete mapping of this abstract `CanisterModule` to actual WebAssembly concepts and the System API is described separately in section [Abstract Canisters to System API](#concrete-canisters). + +#### Call contexts + +The Internet Computer provides certain messaging guarantees: If a user or a canister calls another canister, it will eventually get a single response (a reply or a rejection), even if some canister code along the way fails. + +To ensure that only one response is generated, and also to detect when no response can be generated any more, the IC maintains a *call context*. The `needs_to_respond` field is set to `false` once the call has received a response. Further attempts to respond will now fail. + + Request = { + nonce : Blob; + ingress_expiry : Nat; + sender : UserId; + canister_id : CanisterId; + method_name : Text; + arg : Blob; + } + CallId = (abstract) + CallOrigin + = FromUser { + request : Request; + } + | FromCanister { + calling_context : CallId; + callback: Callback; + } + | FromSystemTask + CallCtxt = { + canister : CanisterId; + origin : CallOrigin; + needs_to_respond : bool; + deleted : bool; + available_cycles : Nat; + } + +#### Calls and Messages + +Calls into and within the IC are implemented as messages passed between canisters. During their lifetime, messages change shape: they begin as a call to a public method, which is resolved to a WebAssembly function that is then executed, potentially generating a response which is then delivered. + +Therefore, a message can have different shapes: + + Queue = Unordered | Queue { from : System | CanisterId; to : CanisterId } + EntryPoint + = PublicMethod MethodName Principal Blob + | Callback Callback Response RefundedCycles + | Heartbeat + | GlobalTimer + + Message + = CallMessage { + origin : CallOrigin; + caller : Principal; + callee : CanisterId; + method_name : Text; + arg : Blob; + transferred_cycles : Nat; + queue : Queue; + } + | FuncMessage { + call_context : CallId; + receiver : CanisterId; + entry_point : EntryPoint; + queue : Queue; + } + | ResponseMessage { + origin : CallOrigin; + response : Response; + refunded_cycles : Nat; + } + +The `queue` field is used to describe the message ordering behavior. Its concrete value is only used to determine when the relative order of two messages must be preserved, and is otherwise not interpreted. Response messages are not ordered, as explained above, so they have no `queue` field. + +A reference implementation would likely maintain a separate list of `messages` for each such queue to efficiently find eligible messages; this document uses a single global list for a simpler and more concise system state. + +#### API requests + +We distinguish between the *asynchronous* API requests (type `Request`) passed to `/api/v2/…/call`, which may be present in the IC state, and the *synchronous* API requests passed to `/api/v2/…/read_state` and `/api/v2/…/query`, which are only ephemeral. + +These are the synchronous read messages: + + Path = List(Blob) + APIReadRequest + = StateRead = { + nonce : Blob; + ingress_expiry : Nat; + sender : UserId; + paths : List(Path); + } + | CanisterQuery = { + nonce : Blob; + ingress_expiry : Nat; + sender : UserId; + canister_id : CanisterId; + method_name : Text; + arg : Blob; + } + +Signed delegations contain the (unsigned) delegation data in a nested record, next to the signature of that data. + + PublicKey = Blob + Signature = Blob + SignedDelegation = { + delegation : { + pubkey : PublicKey; + targets : [CanisterId] | Unrestricted; + senders : [Principal] | Unrestricted; + expiration : Timestamp + }; + signature : Signature + } + +For the signatures in a `Request`, we assume that the following function implements signature verification as described in [Authentication](#authentication). This function picks the corresponding signature scheme according to the DER-encoded metadata in the public key. + + verify_signature : PublicKey -> Signature -> Blob -> Bool + + Envelope = { + content : Request | APIReadRequest; + sender_pubkey : PublicKey | NoPublicKey; + sender_sig : Signature | NoSignature; + sender_delegation: [SignedDelegation] + } + +The evolution of a `Request` goes through these states, as explained in [Overview of canister calling](#http-call-overview): + + RequestStatus + = Received + | Processing + | Rejected (RejectCode, Text) + | Replied Blob + | Done + +A `Path` may refer to a request by way of a *request id*, as specified in [Request ids](#request-id): + + RequestId = { b ∈ Blob | |b| = 32 } + hash_of_map: Request -> RequestId + +#### The system state + +Finally, we can describe the state of the IC as a record having the following fields: + + CanState + = EmptyCanister | { + wasm_state : WasmState; + module : CanisterModule; + raw_module : Blob; + public_custom_sections: Text ↦ Blob; + private_custom_sections: Text ↦ Blob; + } + CanStatus + = Running + | Stopping (List (CallOrigin, Nat)) + | Stopped + ChangeOrigin + = FromUser { + user_id : PrincipalId; + } + | FromCanister { + canister_id : PrincipalId; + canister_version : CanisterVersion | NoCanisterVersion; + } + CodeDeploymentMode + = Install + | Reinstall + | Upgrade + ChangeDetails + = Creation { + controllers : [PrincipalId]; + } + | CodeUninstall + | CodeDeployment { + mode : CodeDeploymentMode; + module_hash : Blob; + } + | ControllersChange { + controllers : [PrincipalId]; + } + Change = { + timestamp_nanos : Timestamp; + canister_version : CanisterVersion; + origin : ChangeOrigin; + details : ChangeDetails; + } + CanisterHistory = { + total_num_changes : Nat; + recent_changes : [Change]; + } + S = { + requests : Request ↦ (RequestStatus, Principal); + canisters : CanisterId ↦ CanState; + controllers : CanisterId ↦ Set Principal; + freezing_threshold : CanisterId ↦ Nat; + canister_status: CanisterId ↦ CanStatus; + canister_version: CanisterId ↦ CanisterVersion; + time : CanisterId ↦ Timestamp; + global_timer : CanisterId ↦ Timestamp; + balances: CanisterId ↦ Nat; + certified_data: CanisterId ↦ Blob; + canister_history: CanisterId ↦ CanisterHistory; + system_time : Timestamp + call_contexts : CallId ↦ CallCtxt; + messages : List Message; // ordered! + root_key : PublicKey + } + +To convert `CanStatus` into `status : Running | Stopping | Stopped` from `Env`, we define the following conversion function: + + simple_status(Running) = Running + simple_status(Stopping _) = Stopping + simple_status(Stopped) = Stopped + +To convert `CallOrigin` into `ChangeOrigin`, we define the following conversion function: + + change_origin(principal, _, FromUser { … }) = FromUser { + user_id = principal + } + change_origin(principal, sender_canister_version, FromCanister { … }) = FromCanister { + canister_id = principal + canister_version = sender_canister_version + } + change_origin(principal, sender_canister_version, FromSystemTask) = FromCanister { + canister_id = principal + canister_version = sender_canister_version + } + +#### Initial state + +The initial state of the IC is + + { + requests = (); + canisters = (); + controllers = (); + freezing_threshold = (); + canister_status = (); + canister_version = (); + time = (); + global_timer = (); + balances = (); + certified_data = (); + system_time = T; + call_contexts = (); + messages = []; + root_key = PublicKey; + } + +for some time stamp `T`, some DER-encoded BLS public key `PublicKey`, and using `()` to denote the empty map or bag. + +### Invariants + +The following is an incomplete list of invariants that should hold for the abstract state `S`, and are not already covered by the type annotations in this section. + +- No method name is the name of an update and query method in a CanisterModule at the same time: + + ∀ _ ↦ CanState ∈ S.canisters: + dom(CanState.module.update_methods) ∩ dom(CanState.module.query_methods) = ∅ + +- Deleted call contexts were not awaiting a response: + + ∀ Ctxt_id ↦ Ctxt ∈ S.call_contexts: + if Ctxt.deleted then Ctxt.needs_to_respond = false + +- Responded call contexts have no available\_cycles left: + + ∀ Ctxt_id ↦ Ctxt ∈ S.call_contexts: + if Ctxt.needs_to_respond = false then Ctxt.available_cycles = 0 + +- A stopped canister does not have any call contexts (in particular, a stopped canister does not have any call contexts marked as deleted): + + ∀ Ctxt_id ↦ Ctxt ∈ S.call_contexts: + S.canister_status[Ctxt.canister] ≠ Stopped + +- Referenced call contexts exist: + + ∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ∈ dom(S.call_contexts) + ∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ∈ dom(S.call_contexts) + ∀ _ ↦ {needs_to_respond = true, origin = FromCanister O, …} ∈ S.call_contexts: O.calling_context ∈ dom(S.call_contexts) + ∀ _ ↦ Stopping Origins ∈ S.canister_status: ∀(FromCanister O, _) ∈ Origins. O.calling_context ∈ dom(S.call_contexts) + +### State transitions + +Based on this abstract notion of the state, we can describe the behavior of the IC. There are three classes of behaviors: + +- Asynchronous API requests that are submitted via `/api/v2/…/call`. These transitions describe checks that the request must pass to be considered received. + +- Spontaneous transitions that model the internal behavior of the IC, by describing conditions on the state that allow the transition to happen, and the state after. + +- Responses to reads (i.e. `/api/v2/…/read_state` and `/api/v2/…/query`). By definition, these do *not* change the state of the IC, and merely describe the response based on the read request (or query, respectively) and the current state. + +The state transitions are not complete with regard to error handling. For example, the behavior of sending a request to a non-existent canister is not specified here. For now, we trust implementors to make sensible decisions there. + +We model the [The IC management canister](#ic-management-canister) with one state transition per method. There, we assume a function + + candid : Value -> Blob + +that represents Candid encoding; this is implicitly taking the method types, as declared in [Interface overview](#ic-candid), into account. We model the parsing of Candid values in the "Conditions" section using `candid` as well, by treating it as a non-deterministic function. + +#### Envelope Authentication + +The following predicate describes when an envelope `E` correctly signs the enclosed request with a key belonging to a user `U`, at time `T`: It returns which canister ids this envelope may be used at (as a set of principals). + + verify_envelope({ content = C }, U, T) + = { p : p is CanisterID } if U = anonymous_id + verify_envelope({ content = C, sender_pubkey = PK, sender_sig = Sig, sender_delegation = DS}, U, T) + = TS if U = mk_self_authenticating_id E.sender_pubkey + ∧ (PK', TS) = verify_delegations(DS, PK, T, { p : p is CanisterId }, U) + ∧ verify_signature PK' Sig ("\x0Aic-request" · hash_of_map(C)) + + verify_delegations([], PK, T, TS, U) = (PK, TS) + verify_delegations([D] · DS, PK, T, TS, U) + = verify_delegations(DS, D.pubkey, T, TS ∩ delegation_targets(D), U) + if verify_signature PK D.signature ("\x1Aic-request-auth-delegation" · hash_of_map(D.delegation)) + ∧ D.delegation.expiration ≥ T + ∧ U ∈ delegated_senders(D) + + delegation_targets(D) + = if D.targets = Unrestricted + then { p : p is CanisterId } + else D.targets + + delegated_senders(D) + = if D.senders = Unrestricted + then { p : p is Principal } + else D.senders + +#### Effective canister ids + +A `Request` has an effective canister id according to the rules in [Effective canister id](#http-effective-canister-id): + + is_effective_canister_id(Request {canister_id = ic_principal, method = provisional_create_canister_with_cycles, …}, p) + is_effective_canister_id(Request {canister_id = ic_principal, arg = candid({canister_id = p, …}), …}, p) + is_effective_canister_id(Request {canister_id = p, …}, p), if p ≠ ic_principal + +#### API Request submission + +After a node accepts a request via `/api/v2/canister//call`, the request gets added to the IC state as `Received`. + +This may only happen if the signature is valid and is created with a correct key. Due to this check, the envelope is discarded after this point. + +Requests that have expired are dropped here. + +Ingress message inspection is applied, and messages that are not accepted by the canister are dropped. + +The (unspecified) function `idle_cycles_burned_rate(S, cid)` determines the idle resource consumption rate in cycles per day of the canister with id `cid`, given its current memory footprint, compute and storage cost, and memory and compute allocation. The function `freezing_limit(S, cid)` determines the freezing threshold in cycles of the canister with id `cid`, given its current memory footprint, compute and storage cost, memory and compute allocation, and current `freezing_threshold` setting. The value `freezing_limit(S, cid)` is derived from `idle_cycles_burned_rate(S, cid)` and `freezing_threshold` as follows: + + freezing_limit(S, cid) = idle_cycles_burned_rate(S, cid) * S.freezing_threshold[cid] / (24 * 60 * 60) + +Submitted request +`E : Envelope` + +Conditions + +```html + +E.content.canister_id ∈ verify_envelope(E, E.content.sender, S.system_time) +E.content ∉ dom(S.requests) +S.system_time <= E.content.ingress_expiry +is_effective_canister_id(E.content, ECID) +( E.content.canister_id = ic_principal + E.content.arg = candid({canister_id = CanisterId, …}) + E.content.sender ∈ S.controllers[CanisterId] + E.content.method_name ∈ + { "install_code", "uninstall_code", "update_settings", "start_canister", "stop_canister", + "canister_status", "delete_canister", + "provisional_create_canister_with_cycles", "provisional_top_up_canister" } +) ∨ ( + E.content.canister_id ≠ ic_principal + S.canisters[E.content.canister_id] ≠ EmptyCanister + Env = { + time = S.time[E.content.canister_id]; + controllers = S.controllers[E.content.canister_id]; + global_timer = S.global_timer[E.content.canister_id]; + balance = S.balances[E.content.canister_id] + freezing_limit = freezing_limit(S, E.content.canister_id); + certificate = NoCertificate; + status = simple_status(S.canister_status[E.content.canister_id]); + canister_version = S.canister_version[E.content.canister_id]; + } + S.canisters[E.content.canister_id].module.inspect_message + (E.content.method_name, S.canisters[E.content.canister_id].wasm_state, E.content.arg, E.content.sender, Env) = Return {status = Accept; cycles_used = Cycles_used;} + Cycles_used ≤ S.balances[E.content.canister_id] + + +``` + +State after + +```html + +S with + requests[E.content] = (Received, ECID) + if E.content.canister_id ≠ ic_principal then + balances[E.content.canister_id] = S.balances[E.content.canister_id] - Cycles_used + +``` + +:::note + +This is not instantaneous (the IC takes some time to agree it accepts the request) nor guaranteed (a node could just drop the request, or maybe it did not pass validation). But once the request has entered the IC state like this, it will be acted upon. + +::: + +#### Request rejection + +The IC may reject a received message for internal reasons (high load, low resources) or expiry. The precise conditions are not specified here, but the reject code must indicate this to be a system error. + +Conditions + +```html + +S.requests[R] = (Received, ECID) +Code = SYS_FATAL or Code = SYS_TRANSIENT + +``` + +State after + +```html + +S with + requests[R] = (Rejected (Code, Msg), ECID) + +``` + +#### Initiating canister calls + +A first step in processing a canister update call is to create a `CallMessage` in the message queue. + +The `request` field of the `FromUser` origin establishes the connection to the API message. One could use the corresponding `hash_of_map` for this purpose, but this formulation is more abstract. + +The IC does not make any guarantees about the order of incoming messages. + +Conditions + +```html + +S.requests[R] = (Received, ECID) +S.system_time <= R.ingress_expiry +C = S.canisters[R.canister_id] + +``` + +State after + +```html + +S with + requests[R] = (Processing, ECID) + messages = + CallMessage { + origin = FromUser { request = R }; + caller = R.sender; + callee = R.canister_id; + method_name = R.method_name; + arg = R.arg; + transferred_cycles = 0; + queue = Unordered; + } · S.messages + +``` + +#### Calls to stopped/stopping/frozen canisters are rejected + +A call to a canister which is stopping, stopped, or frozen is automatically rejected. + +Conditions + +```html + +S.messages = Older_messages · CallMessage CM · Younger_messages +(CM.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ CM.queue) +S.canister_status[CM.callee] = Stopped or S.canister_status[CM.callee] = Stopping _ or balances[CM.callee] < freezing_limit(S, CM.callee) + +``` + +State after + +```html + +messages = Older_messages · Younger_messages · + ResponseMessage { + origin = CM.origin; + response = Reject (CANISTER_ERROR, "canister not running"); + refunded_cycles = CM.transferred_cycles; + } + +``` + +#### Call context creation + +Before invoking a heartbeat, a global timer, or a message to a public entry point, a call context is created for bookkeeping purposes. For these invocations the canister must be running (so not stopped or stopping). Additionally, these invocations only happen for "real" canisters, not the IC management canister. + +This "bookkeeping transition" must be immediately followed by the corresponding ["Message execution" transition](#rule-message-execution). + +*Call context creation: Public entry points* + +For a message to a public entry point, the method is looked up in the list of exports. This happens for both ingress and inter-canister messages. + +The position of the message in the queue is unchanged. + +Conditions + +```html + +S.messages = Older_messages · CallMessage CM · Younger_messages +S.canisters[CM.callee] ≠ EmptyCanister +S.canister_status[CM.callee] = Running +S.balances[CM.callee] ≥ freezing_limit(S, CM.callee) + MAX_CYCLES_PER_MESSAGE +Ctxt_id ∉ dom(S.call_contexts) + +``` + +State after + +```html + +S with + messages = + Older_messages · + FuncMessage { + call_context = Ctxt_id; + receiver = CM.callee; + entry_point = PublicMethod CM.method_name CM.caller CM.arg; + queue = CM.queue; + } · + Younger_messages + call_contexts[Ctxt_id] = { + canister = CM.callee; + origin = CM.origin; + needs_to_respond = true; + deleted = false; + available_cycles = CM.transferred_cycles; + } + balances[CM.callee] = S.balances[CM.callee] - MAX_CYCLES_PER_MESSAGE + +``` + +*Call context creation: Heartbeat* + +If canister `C` exports a method with name `canister_heartbeat`, the IC will create the corresponding call context. + +Conditions + +```html + +S.canisters[C] ≠ EmptyCanister +S.canister_status[C] = Running +S.balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE +Ctxt_id ∉ dom(S.call_contexts) + +``` + +State after + +```html + +S with + messages = + FuncMessage { + call_context = Ctxt_id; + receiver = C; + entry_point = Heartbeat; + queue = Queue { from = System; to = C }; + } + · S.messages + call_contexts[Ctxt_id] = { + canister = C; + origin = FromSystemTask; + needs_to_respond = false; + deleted = false; + available_cycles = 0; + } + balances[C] = S.balances[C] - MAX_CYCLES_PER_MESSAGE + +``` + +*Call context creation: Global timer* + +If canister `C` exports a method with name `canister_global_timer`, the global timer of canister `C` is set, and the current time for canister `C` has passed the value of the global timer, the IC will create the corresponding call context and deactivate the global timer. + +Conditions + +```html + +S.canisters[C] ≠ EmptyCanister +S.canister_status[C] = Running +S.global_timer[C] ≠ 0 +S.time[C] ≥ S.global_timer[C] +S.balances[C] ≥ freezing_limit(S, C) + MAX_CYCLES_PER_MESSAGE +Ctxt_id ∉ dom(S.call_contexts) + +``` + +State after + +```html + +S with + messages = + FuncMessage { + call_context = Ctxt_id; + receiver = C; + entry_point = GlobalTimer; + queue = Queue { from = System; to = C }; + } + · S.messages + call_contexts[Ctxt_id] = { + canister = C; + origin = FromSystemTask; + needs_to_respond = false; + deleted = false; + available_cycles = 0; + } + global_timer[C] = 0 + balances[C] = S.balances[C] - MAX_CYCLES_PER_MESSAGE + +``` + +The IC can execute any message that is at the head of its queue, i.e. there is no older message with the same abstract `queue` field. The actual message execution, if successful, may enqueue further messages and --- if the function returns a response --- record this response. The new call and response messages are enqueued at the end. + +Note that new messages are executed only if the canister is Running and is not frozen. + +#### Message execution {#rule-message-execution} + +The transition models the actual execution of a message, whether it is an initial call to a public method or a response. In either case, a call context already exists (see transition "Call context creation"). + +Conditions + +```html + +S.messages = Older_messages · FuncMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +S.canisters[M.receiver] ≠ EmptyCanister +Mod = S.canisters[M.receiver].module + +Is_response = M.entry_point == Callback _ _ _ + +Env = { + time = S.time[M.receiver]; + controllers = S.controllers[M.receiver]; + global_timer = S.global_timer[M.receiver]; + balance = S.balances[M.receiver] + freezing_limit = freezing_limit(S, M.receiver); + certificate = NoCertificate; + status = simple_status(S.canister_status[M.receiver]); + canister_version = S.canister_version[M.receiver]; +} + +Available = S.call_contexts[M.call_contexts].available_cycles +( M.entry_point = PublicMethod Name Caller Arg + (F = Mod.update_methods[Name](Arg, Caller, Env, Available)) or (F = query_as_update(Mod.query_methods[Name], Arg, Caller, Env)) +) +or +( M.entry_point = Callback Callback Response RefundedCycles + F = Mod.callbacks(Callback, Response, RefundedCycles, Env, Available) +) +or +( M.entry_point = Heartbeat + F = system_task_as_update(Mod.heartbeat, Env) +) +or +( M.entry_point = GlobalTimer + F = system_task_as_update(Mod.global_timer, Env) +) + +R = F(S.canisters[M.receiver].wasm_state) + +``` + +State after + +```html + +if + R = Return res + validate_sender_canister_version(res.new_calls, S.canister_version[M.receiver]) + res.cycles_used ≤ (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE) + res.cycles_accepted ≤ Available + (res.cycles_used + ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ]) ≤ + (S.balances[M.receiver] + res.cycles_accepted + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) + New_balance = + (S.balances[M.receiver] + res.cycles_accepted + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) + - (res.cycles_used + ∑ [ MAX_CYCLES_PER_RESPONSE + call.transferred_cycles | call ∈ res.new_calls ]) + New_balance ≥ if Is_response then 0 else freezing_limit(S, M.receiver); + (res.response = NoResponse) or S.call_contexts[M.call_context].needs_to_respond +then + S with + canisters[M.receiver].wasm_state = res.new_state; + messages = + Older_messages · + Younger_messages · + [ CallMessage { + origin = FromCanister { + call_context = M.call_context; + callback = call.callback; + }; + caller = M.receiver; + callee = call.callee; + method_name = call.method_name; + arg = call.arg; + transferred_cycles = call.transferred_cycles + queue = Queue { from = M.receiver; to = call.callee }; + } + | call ∈ res.new_calls ] · + [ ResponseMessage { + origin = S.call_contexts[M.call_context].origin + response = res.response; + refunded_cycles = Available - res.cycles_accepted; + } + | res.response ≠ NoResponse ] + + if res.response = NoResponse: + call_contexts[M.call_context].available_cycles = Available - res.cycles_accepted + else + call_contexts[M.call_context].needs_to_respond = false + call_contexts[M.call_context].available_cycles = 0 + + if res.new_certified_data ≠ NoCertifiedData: + certified_data[M.receiver] = res.new_certified_data + + if res.new_global_timer ≠ NoGlobalTimer: + global_timer[M.receiver] = res.new_global_timer + + balances[M.receiver] = New_balance +else + S with + messages = Older_messages · Younger_messages + balances[M.receiver] = + (S.balances[M.receiver] + (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) + - min (R.cycles_used, (if Is_response then MAX_CYCLES_PER_RESPONSE else MAX_CYCLES_PER_MESSAGE)) + +``` + +Depending on whether this is a call message and a response messages, we have either set aside `MAX_CYCLES_PER_MESSAGE` or `MAX_CYCLES_PER_RESPONSE`, either in the call context creation rule or the Callback invocation rule. + +The cycle consumption of executing this message is modeled via the unspecified `Cycles_used` variable; the variable takes some value between 0 and `MAX_CYCLES_PER_MESSAGE`/`MAX_CYCLES_PER_RESPONSE` (for call execution and response execution, respectively). + +This transition detects certain behavior that will appear as a trap (and which an implementation may implement by trapping directly in a system call): + +- Responding if the present call context does not need to be responded to + +- Accepting more cycles than are available on the call context + +- Sending out more cycles than available to the canister + +- Consuming more cycles than allowed (and reserved) + +If message execution [*traps* (in the sense of a Wasm function)](#define-wasm-fn), the message gets dropped. No response is generated (as some other message may still fulfill this calling context). Any state mutation is discarded. If the message was a call, the associated cycles are held by its associated call context and will be refunded to the caller, see [Call context starvation](#rule-starvation). + +If message execution [*returns* (in the sense of a Wasm function)](#define-wasm-fn), the state is updated and possible outbound calls and responses are enqueued. + +Note that returning does *not* imply that the call associated with this message now *succeeds* in the sense defined in [section responding](#responding); that would require a (unique) call to `ic0.reply`. Note also that the state changes are persisted even when the IC is set to synthesize a [CANISTER\_ERROR](#CANISTER_ERROR) reject immediately afterward (which happens when this returns without calling `ic0.reply` or `ic0.reject`, the corresponding call has not been responded to and there are no outstanding callbacks, see [Call context starvation](#rule-starvation)). + +The function `validate_sender_canister_version` checks that `sender_canister_version` matches the actual canister version of the sender in all calls to the methods of the management canister that take `sender_canister_version`: + + validate_sender_canister_version(new_calls, canister_version_from_system) = + ∀ call ∈ new_calls. (call.callee = ic_principal and (call.method = 'create_canister' or call.method = 'update_settings' or call.method = 'install_code' or call.method = 'uninstall_code' or call.method = 'provisional_create_canister_with_cycles') and call.arg = candid(A) and A.sender_canister_version = n) => n = canister_version_from_system + +The functions `query_as_update` and `system_task_as_update` turns a query function resp the heartbeat or global timer into an update function; this is merely a notational trick to simplify the rule: + + query_as_update(f, arg, env) = λ wasm_state → + match f(arg, env)(wasm_state) with + Trap trap → Trap trap + Return res → Return { + new_state = wasm_state; + new_calls = []; + new_certified_data = NoCertifiedData; + new_global_timer = NoGlobalTimer; + response = res.response; + cycles_accepted = 0; + cycles_used = res.cycles_used; + } + + system_task_as_update(f, env) = λ wasm_state → + match f(env)(wasm_state) with + Trap trap → Trap trap + Return res → Return { + new_state = res.new_state; + new_calls = res.new_calls; + new_certified_data = res.new_certified_data; + new_global_timer = res.new_global_timer; + response = NoResponse; + cycles_accepted = 0; + cycles_used = res.cycles_used; + } + +Note that by construction, a query function will either trap or return with a response; it will never send calls, and it will never change the state of the canister. + +#### Call context starvation {#rule-starvation} + +If the call context is not for heartbeat or global timer and there is no call, downstream calling context or response that could possibly fulfill a calling context, then a reject is synthesized. The error message below is *not* indicative. In particular, if the IC has an idea about *why* this starved, it can put that in there (e.g. the initial message handler trapped with an out-of-memory access). + +Conditions + +```html + +S.call_contexts[Ctxt_id].needs_to_respond = true +S.call_contexts[Ctxt_id].origin ≠ FromSystemTask +∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id +∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id +∀ _ ↦ {needs_to_respond = true, origin = FromCanister O, …} ∈ S.call_contexts: O.calling_context ≠ Ctxt_id + +``` + +State after + +```html + +S with + call_contexts[Ctxt_id].needs_to_respond = false + call_contexts[Ctxt_id].available_cycles = 0 + messages = + S.messages · + ResponseMessage { + origin = S.call_contexts[Ctxt_id].origin; + response = Reject (CANISTER_ERROR, "starvation"); + refunded_cycles = S.call_contexts[Ctxt_id].available_cycles + } + +``` + +#### Call context removal + +If there is no call, downstream calling context, or response that references a call context, and the call context has been replied to or the call context corresponds to a heartbeat or global timer that had already been executed, then the call context can be removed. + +Conditions + +```html + +( + S.call_contexts[Ctxt_id].needs_to_respond = false +) or +( + S.call_contexts[Ctxt_id].origin = FromSystemTask + ∀ FuncMessage M ∈ S.messages. M.call_context ≠ Ctxt_id +) +∀ CallMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id +∀ ResponseMessage {origin = FromCanister O, …} ∈ S.messages. O.calling_context ≠ Ctxt_id +∀ _ ↦ {needs_to_respond = true, origin = FromCanister O, …} ∈ S.call_contexts: O.calling_context ≠ Ctxt_id +∀ _ ↦ Stopping Origins ∈ S.canister_status: ∀(FromCanister O, _) ∈ Origins. O.calling_context ≠ Ctxt_id + +``` + +State after + +```html + +S with + call_contexts[Ctxt_id] = (deleted) + +``` + +#### IC Management Canister: Canister creation + +The IC chooses an appropriate canister id and instantiates a new (empty) canister identified by this id. The *controllers* are set such that the sender of this request is the only controller, unless the `settings` say otherwise. All cycles on this call are now the canister's initial cycles. + +This is also when the System Time of the new canister starts ticking. + +The `compute_allocation` and `memory_allocation` settings are ignored in this abstract model of the Internet Computer, as it does not address questions of performance or scheduling. + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'create_canister' +M.arg = candid(A) +is_system_assigned CanisterId +CanisterId ∉ dom(S.canisters) +if A.settings.controllers is not null: + New_controllers = A.settings.controllers +else: + New_controllers = [M.caller] + +``` + +State after + +```html + +S with + canisters[CanisterId] = EmptyCanister + time[CanisterId] = CurrentTime + global_timer[CanisterId] = 0 + controllers[CanisterId] = New_controllers + if A.settings.freezing_threshold is not null: + freezing_threshold[CanisterId] = A.settings.freezing_threshold + else: + freezing_threshold[CanisterId] = 2592000 + balances[CanisterId] = M.transferred_cycles + certified_data[CanisterId] = "" + canister_history[CanisterId] = { + total_num_changes = 1 + recent_changes = { + timestamp_nanos = CurrentTime + canister_version = 0 + origin = change_origin(M.caller, A.sender_canister_version, M.origin) + details = Creation { + controllers = New_controllers + } + } + } + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = Reply (candid({canister_id = CanisterId})) + refunded_cycles = 0 + } + canister_status[CanisterId] = Running + canister_version[CanisterId] = 0 + +``` + +This uses the predicate + + is_system_assigned : Principal -> Bool + +which characterizes all system-assigned ids. + +To avoid clashes with potential user ids or is derived from users or canisters, we require (somewhat handwavy) that + +- `is_system_assigned (mk_self_authenticating_id pk) = false` for possible public keys `pk` and + +- `is_system_assigned (mk_derived_id p dn) = false` for any `p` that could be a user id or canister id. + +- `is_system_assigned p = false` for `|p| > 29`. + +- `is_system_assigned ic_principal = false`. + +#### IC Management Canister: Changing settings + +Only the controllers of the given canister can update the canister settings. + +The `compute_allocation` and `memory_allocation` settings are ignored in this abstract model of the Internet Computer, as it does not address questions of performance or scheduling. + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'update_settings' +M.arg = candid(A) +M.caller ∈ S.controllers[A.canister_id] +S.canister_history[A.canister_id] = { + total_num_changes = N; + recent_changes = H; +} + +``` + +State after + +```html + +S with + if A.settings.controllers is not null: + controllers[A.canister_id] = A.settings.controllers + canister_history[A.canister_id] = { + total_num_changes = N + 1; + recent_changes = H · { + timestamp_nanos = S.time[A.canister_id]; + canister_version = S.canister_version[A.canister_id] + 1; + origin = change_origin(M.caller, A.sender_canister_version, M.origin); + details = ControllersChange { + controllers = A.settings.controllers; + }; + }; + } + if A.settings.freezing_threshold is not null: + freezing_threshold[A.canister_id] = A.settings.freezing_threshold + canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = Reply (candid()) + refunded_cycles = M.transferred_cycles + } + +``` + +#### IC Management Canister: Canister status + +The controllers of a canister can obtain information about the canister. + +The `Memory_size` is the (in this specification underspecified) total size of storage in bytes. + +The `idle_cycles_burned_per_day` is the idle consumption of resources in cycles per day. + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'canister_status' +M.arg = candid(A) +M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id} + +``` + +State after + +```html + +S with + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = candid({ + status = simple_status(S.canister_status[A.canister_id]); + module_hash = + if S.canisters[A.canister_id] = EmptyCanister + then null + else opt (SHA-256(S.canisters[A.canister_id].raw_module)); + controllers = S.controllers[A.canister_id]; + memory_size = Memory_size; + cycles = S.balances[A.canister_id]; + freezing_threshold = S.freezing_threshold[A.canister_id]; + idle_cycles_burned_per_day = idle_cycles_burned_rate(S, A.canister_id); + }) + refunded_cycles = M.transferred_cycles + } + +``` + +#### IC Management Canister: Canister information + +Every canister can retrieve the canister history, current module hash, and current controllers of every other canister (including itself). + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'canister_info' +M.arg = candid(A) +if A.num_requested_changes = null then From = |S.canister_history[A.canister_id].recent_changes| +else From = max(0, |S.canister_history[A.canister_id].recent_changes| - A.num_requested_changes) +End = |S.canister_history[A.canister_id].recent_changes| - 1 + +``` + +State after + +```html + +S with + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = candid({ + total_num_changes = S.canister_history[A.canister_id].total_num_changes; + recent_changes = S.canister_history[A.canister_id].recent_changes[From..End]; + module_hash = + if S.canisters[A.canister_id] = EmptyCanister + then null + else opt (SHA-256(S.canisters[A.canister_id].raw_module)); + controllers = S.controllers[A.canister_id]; + }) + refunded_cycles = M.transferred_cycles + } + +``` + +#### IC Management Canister: Code installation + +Only the controllers of the given canister can install code. This transition installs new code over a canister. This involves invoking the `canister_init` method (see [Canister initialization](#system-api-init)), which must succeed. + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'install_code' +M.arg = candid(A) +Mod = parse_wasm_mod(A.wasm_module) +Public_custom_sections = parse_public_custom_sections(A.wasm_module); +Private_custom_sections = parse_private_custom_sections(A.wasm_module); +(A.mode = install and S.canisters[A.canister_id] = EmptyCanister) or A.mode = reinstall +M.caller ∈ S.controllers[A.canister_id] +Env = { + time = S.time[A.canister_id]; + controllers = S.controllers[A.canister_id]; + global_timer = 0; + balance = S.balances[A.canister_id]; + freezing_limit = freezing_limit(S, A.canister_id); + certificate = NoCertificate; + status = simple_status(S.canister_status[A.canister_id]); + canister_version = S.canister_version[A.canister_id] + 1; +} +Mod.init(A.canister_id, A.arg, M.caller, Env) = Return {new_state = New_state; new_certified_data = New_certified_data; new_global_timer = New_global_timer; cycles_used = Cycles_used;} +Cycles_used ≤ S.balances[A.canister_id] +dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ +S.canister_history[A.canister_id] = { + total_num_changes = N; + recent_changes = H; +} + +``` + +State after + +```html + +S with + canisters[A.canister_id] = { + wasm_state = New_state; + module = Mod; + raw_module = A.wasm_module; + public_custom_sections = Public_custom_sections; + private_custom_sections = Private_custom_sections; + } + certified_data[A.canister_id] = New_certified_data + if New_global_timer ≠ NoGlobalTimer: + global_timer[A.canister_id] = New_global_timer + else: + global_timer[A.canister_id] = 0 + canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 + balances[A.canister_id] = S.balances[A.canister_id] - Cycles_used + canister_history[A.canister_id] = { + total_num_changes = N + 1; + recent_changes = H · { + timestamp_nanos = S.time[A.canister_id]; + canister_version = S.canister_version[A.canister_id] + 1 + origin = change_origin(M.caller, A.sender_canister_version, M.origin); + details = CodeDeployment { + mode = A.mode; + module_hash = SHA-256(A.wasm_module); + }; + }; + } + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin; + response = Reply (candid()); + refunded_cycles = M.transferred_cycles; + } + +``` + +#### IC Management Canister: Code upgrade + +Only the controllers of the given canister can install new code. This changes the code of an *existing* canister, preserving the state in the stable memory. This involves invoking the `canister_pre_upgrade` method on the old and `canister_post_upgrade` method on the new canister, which must succeed and must not invoke other methods. + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'install_code' +M.arg = candid(A) +Mod = parse_wasm_mod(A.wasm_module) +Public_custom_sections = parse_public_custom_sections(A.wasm_module) +Private_custom_sections = parse_private_custom_sections(A.wasm_module) +A.mode = upgrade +M.caller ∈ S.controllers[A.canister_id] +S.canisters[A.canister_id] = { wasm_state = Old_state; module = Old_module, …} +Env = { + time = S.time[A.canister_id]; + controllers = S.controllers[A.canister_id]; + balance = S.balances[A.canister_id]; + freezing_limit = freezing_limit(S, A.canister_id); + certificate = NoCertificate; + status = simple_status(S.canister_status[A.canister_id]); +} +Env1 = Env with { + global_timer = S.global_timer[A.canister_id]; + canister_version = S.canister_version[A.canister_id]; +} +Old_module.pre_upgrade(Old_State, M.caller, Env1) = Return {stable_memory = Stable_memory; new_certified_data = New_certified_data; cycles_used = Cycles_used;} +Env2 = Env with { + global_timer = 0; + canister_version = S.canister_version[A.canister_id] + 1; +} +Mod.post_upgrade(A.canister_id, Stable_memory, A.arg, M.caller, Env2) = Return {new_state = New_state; new_certified_data = New_certified_data'; new_global_timer = New_global_timer; cycles_used = Cycles_used';} +Cycles_used + Cycles_used' ≤ S.balances[A.canister_id] +dom(Mod.update_methods) ∩ dom(Mod.query_methods) = ∅ +S.canister_history[A.canister_id] = { + total_num_changes = N; + recent_changes = H; +} + +``` + +State after + +```html + +S with + canisters[A.canister_id] = { + wasm_state = New_state; + module = Mod; + raw_module = A.wasm_module; + public_custom_sections = Public_custom_sections; + private_custom_sections = Private_custom_sections; + } + if New_certified_data' ≠ NoCertifiedData: + certified_data[A.canister_id] = New_certified_data' + else if New_certified_data ≠ NoCertifiedData: + certified_data[A.canister_id] = New_certified_data + if New_global_timer ≠ NoGlobalTimer: + global_timer[A.canister_id] = New_global_timer + else: + global_timer[A.canister_id] = 0 + canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 + balances[A.canister_id] = S.balances[A.canister_id] - (Cycles_used + Cycles_used'); + canister_history[A.canister_id] = { + total_num_changes = N + 1; + recent_changes = H · { + timestamp_nanos = S.time[A.canister_id]; + canister_version = S.canister_version[A.canister_id] + 1 + origin = change_origin(M.caller, A.sender_canister_version, M.origin); + details = CodeDeployment { + mode = Upgrade; + module_hash = SHA-256(A.wasm_module); + }; + }; + } + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin; + response = Reply (candid()); + refunded_cycles = M.transferred_cycles; + } + +``` + +#### IC Management Canister: Code uninstallation {#rule-uninstall} + +Upon uninstallation, the canister is reverted to an empty canister, and all outstanding call contexts are rejected and marked as deleted. + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'uninstall_code' +M.arg = candid(A) +M.caller ∈ S.controllers[A.canister_id] +S.canister_history[A.canister_id] = { + total_num_changes = N; + recent_changes = H; +} + +``` + +State after + +```html + +S with + canisters[A.canister_id] = EmptyCanister + certified_data[A.canister_id] = "" + canister_history[A.canister_id] = { + total_num_changes = N + 1; + recent_changes = H · { + timestamp_nanos = S.time[A.canister_id]; + canister_version = S.canister_version[A.canister_id] + 1 + origin = change_origin(M.caller, A.sender_canister_version, M.origin); + details = CodeUninstall; + }; + } + canister_version[A.canister_id] = S.canister_version[A.canister_id] + 1 + global_timer[A.canister_id] = 0 + + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = Reply (candid()) + refunded_cycles = M.transferred_cycles + } · + [ ResponseMessage { + origin = Ctxt.origin + response = Reject (CANISTER_REJECT, 'Canister has been uninstalled') + refunded_cycles = Ctxt.available_cycles + } + | Ctxt_id ↦ Ctxt ∈ S.call_contexts + , Ctxt.canister = A.canister_id + , Ctxt.needs_to_respond = true + ] + + for Ctxt_id ↦ Ctxt ∈ S.call_contexts: + if Ctxt.canister = A.canister_id: + call_contexts[Ctxt_id].deleted := true + call_contexts[Ctxt_id].needs_to_respond := false + call_contexts[Ctxt_id].available_cycles := 0 + +``` + +#### IC Management Canister: Stopping a canister + +The controllers of a canister can stop a canister. Stopping a canister goes through two steps. First, the status of the canister is set to `Stopping`; as explained above, a stopping canister rejects all incoming requests and continues processing outstanding responses. When a stopping canister has no more open call contexts, its status is changed to `Stopped` and a response is generated. Note that when processing responses, a stopping canister can make calls to other canisters and thus create new call contexts. In addition, a canister which is stopped or stopping will accept (and respond) to further `stop_canister` requests. + +We encode this behavior via three (types of) transitions: + +1. First, any `stop_canister` call sets the state of the canister to `Stopping`; we record in the status the origin (and cycles) of all `stop_canister` calls which arrive at the canister while it is stopping (or stopped). + +2. Next, when the canister has no open call contexts (so, in particular, all outstanding responses to the canister have been processed), the status of the canister is set to `Stopped`. + +3. Finally, each pending `stop_canister` call (which are encoded in the status) is responded to, to indicate that the canister is stopped. + + Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'stop_canister' +M.arg = candid(A) +S.canister_status[A.canister_id] = Running +M.caller ∈ S.controllers[A.canister_id] + +``` + +State after + +```html + +S with + messages = Older_messages · Younger_messages + canister_status[A.canister_id] = Stopping [(M.origin, M.transferred_cycles)] + +``` + +The next two transitions record any additional 'stop\_canister' requests that arrive at a stopping (or stopped) canister in its status. + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'stop_canister' +M.arg = candid(A) +S.canister_status[A.canister_id] = Stopping Origins +M.caller ∈ S.controllers[A.canister_id] + +``` + +State after + +```html + +S with + messages = Older_messages · Younger_messages + canister_status[A.canister_id] = Stopping (Origins · [(M.origin, M.transferred_cycles)]) + +``` + +The status of a stopping canister which has no open call contexts is set to `Stopped`, and all pending `stop_canister` calls are replied to. + +Conditions + +```html + +S.canister_status[CanisterId] = Stopping Origins +∀ Ctxt_id. S.call_contexts[Ctxt_id].canister ≠ CanisterId + +``` + +State after + +```html + +S with + canister_status[CanisterId] = Stopped + messages = S.Messages · + [ ResponseMessage { + origin = O + response = Reply (candid()) + refunded_cycles = C + } + | (O, C) ∈ Origins + ] + +``` + +:::note + +Sending a `stop_canister` message to an already stopped canister is acknowledged (i.e. responded with success), but is otherwise a no-op: + +::: + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'stop_canister' +M.arg = candid(A) +S.canister_status[A.canister_id] = Stopped +M.caller ∈ S.controllers[A.canister_id] + +``` + +State after + +```html + +S with + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin; + response = Reply (candid()); + refunded_cycles = M.transferred_cycles; + } + +``` + +#### IC Management Canister: Starting a canister + +The controllers of a canister can start a `stopped` canister. If the canister is already running, the command has no effect on the canister. + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'start_canister' +M.arg = candid(A) +S.canister_status[A.canister_id] = Running or S.canister_status[A.canister_id] = Stopped +M.caller ∈ S.controllers[A.canister_id] + +``` + +State after + +```html + +S with + canister_status[A.canister_id] = Running + messages = Older_messages · Younger_messages · + ResponseMessage{ + origin = M.origin + response = Reply (candid()) + refunded_cycles = M.transferred_cycles + } + +``` + +If the status of the canister was 'stopping', then the canister status is set to `running`. The pending `stop_canister` request(s) are rejected. + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'start_canister' +M.arg = candid(A) +S.canister_status[A.canister_id] = Stopping Origins +M.caller ∈ S.controllers[A.canister_id] + +``` + +State after + +```html + +S with + canister_status[A.canister_id] = Running + messages = Older_messages · Younger_messages · + ResponseMessage{ + origin = M.origin + response = Reply (candid()) + refunded_cycles = M.transferred_cycles + } · + [ ResponseMessage { + origin = O + response = Reject (CANISTER_REJECT, 'Canister has been restarted') + refunded_cycles = C + } + | (O, C) ∈ Origins + ] + +``` + +#### IC Management Canister: Canister deletion + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'delete_canister' +M.arg = candid(A) +S.canister_status[A.canister_id] = Stopped +M.caller ∈ S.controllers[A.canister_id] + +``` + +State after + +```html + +S with + canisters[A.canister_id] = (deleted) + controllers[A.canister_id] = (deleted) + freezing_threshold[A.canister_id] = (deleted) + canister_status[A.canister_id] = (deleted) + canister_version[A.canister_id] = (deleted) + time[A.canister_id] = (deleted) + global_timer[A.canister_id] = (deleted) + balances[A.canister_id] = (deleted) + certified_data[A.canister_id] = (deleted) + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = Reply (candid()) + refunded_cycles = M.transferred_cycles + } + +``` + +#### IC Management Canister: Depositing cycles + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'deposit_cycles' +M.arg = candid(A) +A.canister_id ∈ dom(S.balances) + +``` + +State after + +```html + +S with + balances[A.canister_id] = + S.balances[A.canister_id] + M.transferred_cycles + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = Reply (candid()) + refunded_cycles = 0 + } + +``` + +#### IC Management Canister: Random numbers + +The management canister can produce pseudo-random bytes. It always returns a 32-byte `blob`: + +The precise guarantees around the randomness, e.g. unpredictability, are not captured in this formal semantics. + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'raw_rand' +M.arg = candid() +|B| = 32 + +``` + +State after + +```html + +S with + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = Reply (candid(B)) + refunded_cycles = M.transferred_cycles + } + +``` + +#### IC Management Canister: Canister creation with cycles + +This is a variant of `create_canister`, which sets the initial cycle balance based on the `amount` argument. + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'provisional_create_canister_with_cycles' +M.arg = candid(A) +is_system_assigned CanisterId +CanisterId ∉ dom(S.canisters) +A.specified_id ∉ dom(S.canisters) +if A.settings.controllers is not null: + New_controllers = A.settings.controllers +else: + New_controllers = [M.caller] + +``` + +State after + +```html + +S with + if A.specified_id is not null: + canister_id = A.specified_id + else: + canister_id = CanisterId + canisters[canister_id] = EmptyCanister + time[canister_id] = CurrentTime + global_timer[canister_id] = 0 + controllers[canister_id] = New_controllers + if A.settings.freezing_threshold is not null: + freezing_threshold[canister_id] = A.settings.freezing_threshold + else: + freezing_threshold[canister_id] = 2592000 + if A.amount is not null: + balances[canister_id] = A.amount + else: + balances[canister_id] = DEFAULT_PROVISIONAL_CYCLES_BALANCE + certified_data[canister_id] = "" + canister_history[canister_id] = { + total_num_changes = 1 + recent_changes = { + timestamp_nanos = CurrentTime + canister_version = 0 + origin = change_origin(M.caller, A.sender_canister_version, M.origin) + details = Creation { + controllers = New_controllers + } + } + } + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = Reply (candid({canister_id = canister_id})) + refunded_cycles = M.transferred_cycles + } + canister_status[canister_id] = Running + canister_version[canister_id] = 0 + +``` + +#### IC Management Canister: Top up canister + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'provisional_top_up_canister' +M.arg = candid(A) +A.canister_id ∈ dom(S.canisters) + +``` + +State after + +```html + +S with + balances[A.canister_id] = S.balances[A.canister_id] + A.amount + +``` + +#### Callback invocation + +When an inter-canister call has been responded to, we can queue the call to the callback. + +This "bookkeeping transition" must be immediately followed by the corresponding ["Message execution" transition](#rule-message-execution). + +Conditions + +```html + +S.messages = Older_messages · ResponseMessage RM · Younger_messages +RM.origin = FromCanister { + call_context = Ctxt_id + callback = Callback + } +not S.call_contexts[Ctxt_id].deleted +S.call_contexts[Ctxt_id].canister ∈ dom(S.balances) + +``` + +State after + +```html + +S with + balances[S.call_contexts[Ctxt_id].canister] = + S.balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles + messages = + Older_messages · + FuncMessage { + call_context = Ctxt_id + receiver = S.call_contexts[Ctxt_id].canister + entry_point = Callback Callback RM.response RM.refunded_cycles + queue = Unordered + } · + Younger_messages + +``` + +If the responded call context does not exist anymore, because the canister has been uninstalled since, the refunded cycles are still added to the canister balance, but no function invocation is enqueued: + +Conditions + +```html + +S.messages = Older_messages · ResponseMessage RM · Younger_messages +RM.origin = FromCanister { + call_context = Ctxt_id + callback = Callback + } +S.call_contexts[Ctxt_id].deleted +S.call_contexts[Ctxt_id].canister ∈ dom(S.balances) + +``` + +State after + +```html + +S with + balances[S.call_contexts[Ctxt_id].canister] = + S.balances[S.call_contexts[Ctxt_id].canister] + RM.refunded_cycles + MAX_CYCLES_PER_RESPONSE + messages = Older_messages · Younger_messages + +``` + +#### Respond to user request + +When an ingress method call has been responded to, we can record the response in the list of queries. + +Conditions + +```html + +S.messages = Older_messages · ResponseMessage RM · Younger_messages +RM.origin = FromUser { request = M } +S.requests[M] = (Processing, ECID) + +``` + +State after + +```html + +S with + messages = Older_messages · Younger_messages + requests[M] = + | (Replied R, ECID) if M.response = Reply R + | (Rejected (c, R), ECID) if M.response = Reject (c, R) + +``` + +NB: The refunded cycles, `RM.refunded_cycles` are, by construction, empty. + +#### Request clean up + +The IC will keep the data for a completed or rejected request around for a certain, implementation defined amount of time, to allow users to poll for the data. After that time, the data of the request will be dropped: + +Conditions + +```html + +(S.requests[M] = (Replied _, ECID)) or (S.requests[M] = (Rejected _, ECID)) + +``` + +State after + +```html + +S with + requests[M] = (Done, ECID) + +``` + +At the same or some later point, the request will be removed from the state of the IC. This must happen no earlier than the ingress expiry time set in the request. + +Conditions + +```html + +(S.requests[M] = (Replied _, _)) or (S.requests[M] = (Rejected _, _)) or (S.requests[M] = (Done, _)) +M.ingress_expiry < S.system_time + +``` + +State after + +```html + +S with + requests[M] = (deleted) + +``` + +#### Canister out of cycles + +Once a canister runs out of cycles, its code is uninstalled (cf. [IC Management Canister: Code uninstallation](#rule-uninstall)), the canister changes in the canister history are dropped (their total number is preserved), and the allocations are set to zero (NB: allocations are currently not modeled in the formal model): + +Conditions + +```html + +S.balances[CanisterId] = 0 +S.canister_history[CanisterId] = { + total_num_changes = N; + recent_changes = H; +} + +``` + +State after + +```html + +S with + canisters[CanisterId] = EmptyCanister + certified_data[CanisterId] = "" + canister_history[CanisterId] = { + total_num_changes = N; + recent_changes = []; + } + canister_version[CanisterId] = S.canister_version[CanisterId] + 1 + global_timer[CanisterId] = 0 + + messages = S.messages · + [ ResponseMessage { + origin = Ctxt.origin + response = Reject (CANISTER_REJECT, 'Canister has been uninstalled') + refunded_cycles = Ctxt.available_cycles + } + | Ctxt_id ↦ Ctxt ∈ S.call_contexts + , Ctxt.canister = CanisterId + , Ctxt.needs_to_respond = true + ] + + for Ctxt_id ↦ Ctxt ∈ S.call_contexts: + if Ctxt.canister = CanisterId: + call_contexts[Ctxt_id].deleted := true + call_contexts[Ctxt_id].needs_to_respond := false + call_contexts[Ctxt_id].available_cycles := 0 + +``` + +#### Time progressing, cycle consumption, and canister version increments + +Time progresses. Abstractly, it does so independently for each canister, and in unspecified intervals. + +Conditions + +```html + +T0 = S.time[CanisterId] +T1 > T0 + +``` + +State after + +```html + +S with + time[CanisterId] = T1 + +``` + +The canister cycle balance similarly depletes at an unspecified rate, but stays non-negative: + +Conditions + +```html + +B0 = S.balances[CanisterId] +0 ≤ B1 < B0 + +``` + +State after + +```html + +S with + balances[CanisterId] = B1 + +``` + +Similarly, the system time, used to expire requests, progresses: + +Conditions + +```html + +T0 = S.system_time +T1 > T0 + +``` + +State after + +```html + +S with + system_time = T1 + +``` + +Finally, the canister version can be incremented arbitrarily: + +Conditions + +```html + +N0 = S.canister_version[CanisterId] +N1 > N0 + +``` + +State after + +```html + +S with + canister_version[CanisterId] = N1 + +``` + +#### Trimming canister history + +The list of canister changes can be trimmed, but the total number of recorded canister changes cannot be altered. At least 20 changes are guaranteed to remain in the list of changes. + +Conditions + +```html + +S.canister_history[CanisterId] = { + total_num_changes = N; + recent_changes = Older_changes · Newer_changes; + } +|Newer_changes| ≥ 20 + +``` + +State after + +```html + +S with + canister_history[CanisterId] = { + total_num_changes = N; + recent_changes = Newer_changes; + } + +``` + +#### Query call + +Canister query calls to `/api/v2/canister//query` can be executed directly. They can only be executed against canisters which have a status of `Running` and are also not frozen. + +During the execution of a query call, a certificate is provided to the canister that is valid, contains a current state tree (or "recent enough"; the specification is currently vague about how old the certificate may be) and reveals the canister's [Certified Data](#system-api-certified-data). + +Submitted request +`E` + +Conditions + +```html + +E.content = CanisterQuery Q +Q.canister_id ∈ verify_envelope(E, Q.sender, S.system_time) +is_effective_canister_id(E.content, ECID) +S.system_time <= Q.ingress_expiry +S.canisters[Q.canister_id] ≠ EmptyCanister +S.canister_status[Q.canister_id] = Running ∧ S.balances[Q.canister_id] >= freezing_limit(S, Q.canister_id) +C = S.canisters[Q.canister_id] +F = C.module.query_methods[Q.method_name] +verify_cert(Cert) +lookup(["canister",Q.canister_id,"certified_data"], Cert) = Found S.certified_data[Q.canister_id] +lookup(["time"], Cert) = Found S.system_time // or "recent enough" +Env = { + time = S.time[Q.receiver]; + controllers = S.controllers[Q.receiver]; + global_timer = S.global_timer[Q.receiver]; + balance = S.balances[Q.canister_id]; + freezing_limit = freezing_limit(S, Q.canister_id); + certificate = Cert; + status = simple_status(S.canister_status[Q.receiver]); + canister_version = S.canister_version[Q.receiver]; +} + +``` + +Read response +- If `F(Q.Arg, Q.sender, Env) = Trap trap` then + + {status: rejected; reject_code: CANISTER_ERROR, reject_message: , error_code: } + +- Else if `F(Q.Arg, Q.sender, Env) = Return {response = Reject (code, msg); …}` then + + {status: rejected; reject_code: : reject_message: , error_code: } + +- Else if `F(Q.Arg, Q.sender, Env) = Return {response = Reply R; …}` then + + {status: success; reply: { arg : } } + +#### Certified state reads + +The user can read elements of the *state tree*, using a `read_state` request to `/api/v2/canister//read_state`. + +Submitted request +`E` + +Conditions + +```html + +E.content = ReadState RS +TS = verify_envelope(E, RS.sender, S.system_time) +S.system_time <= RS.ingress_expiry +∀ path ∈ RS.paths. may_read_path(S, R.sender, path) +∀ ["request_status", Rid] · _ ∈ RS.paths. ∀ R ∈ dom(S.requests). hash_of_map(R) = Rid => R.canister_id ∈ TS + +``` + +Read response +A record with + +- `{certificate: C}` + +The predicate `may_read_path` is defined as follows, implementing the access control outlined in [Request: Read state](#http-read-state): + + may_read_path(S, _, ["time"] · _) = True + may_read_path(S, _, ["subnet"] · _) = True + may_read_path(S, _, ["request_status", Rid] · _) = + ∀ (R ↦ (_, ECID')) ∈ dom(S.requests). hash_of_map(R) = Rid => RS.sender == R.sender ∧ ECID == ECID' + may_read_path(S, _, ["canister", cid, "module_hash"] · _) = cid == ECID + may_read_path(S, _, ["canister", cid, "controllers"] · _) = cid == ECID + may_read_path(S, _, ["canister", cid, "metadata", name] · _) = cid == ECID ∧ UTF8(name) ∧ + (cid ∉ dom(S.canisters[cid]) ∨ + S.canisters[cid] = EmptyCanister ∨ + name ∉ (dom(S.canisters[cid].public_custom_sections) ∪ dom(S.canisters[cid].private_custom_sections)) ∨ + name ∈ dom(S.canisters[cid].public_custom_sections) ∨ + (name ∈ dom(S.canisters[cid].private_custom_sections) ∧ RS.sender ∈ S.controllers[cid]) + ) + may_read_path(S, _, _) = False + +where `UTF8(name)` holds if `name` is encoded in UTF-8. + +The response is a certificate `cert`, as specified in [Certification](#certification), which passes `verify_cert` (assuming `S.root_key` as the root of trust), and where for every `path` documented in [The system state tree](#state-tree) that is a suffix of a path in `RS.paths` or of `["time"]`, we have + + lookup(path, cert) = lookup_in_tree(path, state_tree(S)) + +where `state_tree` constructs a labeled tree from the IC state `S` and the (so far underspecified) set of subnets `subnets`, as per [The system state tree](#state-tree) + + state_tree(S) = { + "time": S.system_time; + "subnet": { subnet_id : { "public_key" : subnet_pk, "canister_ranges" : subnet_ranges } | (subnet_id, subnet_pk, subnet_ranges) ∈ subnets }; + "request_status": { request_id(R): request_status_tree(T) | (R ↦ (T, _)) ∈ S.requests }; + "canister": + { canister_id : + { "module_hash" : SHA256(C.raw_module) | if C ≠ EmptyCanister } ∪ + { "controllers" : CBOR(S.controllers[canister_id]) } ∪ + { "metadata": { name: blob | (name, blob) ∈ S.canisters[canister_id].public_custom_sections ∪ S.canisters[canister_id].private_custom_sections } } + | (canister_id, C) ∈ S.canisters }; + } + + request_status_tree(Received) = + { "status": "received" } + request_status_tree(Processing) = + { "status": "processing" } + request_status_tree(Rejected (code, msg)) = + { "status": "rejected"; "reject_code": code; "reject_message": msg, error_code: } + request_status_tree(Replied arg) = + { "status": "replied"; "reply": arg } + request_status_tree(Done) = + { "status": "done" } + +and where `lookup_in_tree` is a function that returns the value or `Absent` as appropriately. + +### Abstract Canisters to System API {#concrete-canisters} + +In Section [Abstract canisters](#abstract-canisters) we introduced an abstraction over the interface to a canister, to avoid cluttering the abstract specification of the Internet Computer from WebAssembly details. In this section, we will fill the gap and explain how the abstract canister interface maps to the [concrete System API](#system-api) and the WebAssembly concepts as defined in the [WebAssembly specification](https://webassembly.github.io/spec/core/index.html). + +#### The concrete `WasmState` + +The abstract `WasmState` above models the WebAssembly *store* `S`, which encompasses the functions, tables, memories and globals of the WebAssembly program, plus additional data maintained by the IC, such as the stable memory: + + WasmState = { + store : S; // a store as per WebAssembly spec + self_id : CanId; + stable_mem : Blob + } + +As explained in Section "[WebAssembly module requirements](#system-api-module)", the WebAssembly module imports at most *one* memory and at most *one* table; in the following, *the* memory (resp. table) and the fields `mem` and `table` of `S` refer to that. Any system call that accesses the memory (resp. table) will trap if the module does not import the memory (resp. table). + +We model `mem` as an array of bytes, and `table` as an array of execution functions. + +The abstract `Callback` type above models an entry point for responses: + + Closure = { + fun : i32, + env : i32, + } + + Callback = { + on_reply : Closure; + on_reject : Closure; + on_cleanup : Closure | NoClosure; + } + +#### The execution state + +We can model the execution of WebAssembly functions as stateful functions that have access to the WebAssembly store. In order to also model the behavior of the system imports, which have access to additional data structures, we extend the state as follows: + + Params = { + arg : NoArg | Blob; + caller : NoCaller | Principal; + reject_code : 0 | SYS_FATAL | SYS_TRANSIENT | …; + reject_message : Text; + sysenv : Env; + cycles_refunded : Nat; + method_name : NoText | Text; + } + ExecutionState = { + wasm_state : WasmState; + params : Params; + response : NoResponse | Response; + cycles_accepted : Nat; + cycles_available : Nat; + cycles_used : Nat; + balance : Funds; + reply_params : { arg : Blob }; + pending_call : MethodCall | NoPendingCall; + calls : List MethodCall; + new_certified_data : NoCertifiedData | Blob; + new_global_timer : NoGlobalTimer | Nat; + ingress_filter : Accept | Reject; + context : I | G | U | Q | Ry | Rt | C | F | T | s; + } + +This allows us to model WebAssembly functions, including host-provided imports, as functions with implicit mutable access to an `ExecutionState`, dubbed *execution functions*. Syntactically, we express this using an implicit argument of type `ref ExecutionState` in angle brackets (e.g. `func(x)` for the invocation of a WebAssembly function with type `(x : i32) -> ()`). The lifetime of the `ExecutionState` data structure is that of one such function invocation. + +:::warning + +It is nonsensical to pass to an execution function a WebAssembly store `S` that comes from a different WebAssembly module than one defining the function. + +::: + +- For more convenience when creating a new `ExecutionState`, we define the following partial records: + + empty_params = { + arg = NoArg; + caller = NoCaller; + reject_code = 0; + reject_message = ""; + sysenv = (undefined); + cycles_refunded = 0; + method_name = NoText; + } + + empty_execution_state = { + wasm_state = (undefined); + params = (undefined); + response = NoResponse; + cycles_accepted = 0; + cycles_available = 0; + cycles_used = 0; + balance = 0; + reply_params = { arg = "" }; + pending_call = NoPendingCall; + calls = []; + new_certified_data = NoCertifiedData; + new_global_timer = NoGlobalTimer; + ingress_filter = Reject; + context = (undefined); + } + +#### The concrete `CanisterModule` + +Finally we can specify the abstract `CanisterModule` that models a concrete WebAssembly module. + +- The `initial_wasm_store` mentioned below is the store of the WebAssembly module after *instantiation* (as per WebAssembly spec) of the WasmModule contained in the [canister module](#canister-module-format), before executing a potential `(start)` function. + +- We define a helper function + + start : (CanisterId) -> Trap { cycles_used : Nat; } | Return { + new_state : WasmState; + cycles_used : Nat; + } + + modelling execution of a potential `(start)` function. + + If the WebAssembly module does not export a function called under the name `start`, then + + start = λ (self_id) → + Return { + new_state = {store = initial_wasm_store; self_id = self_id; stable_mem = ""}; + cycles_used = 0; + } + + Otherwise, if the WebAssembly module exports a function `func` under the name `start`, it is + + start = λ (self_id) → + let es = ref {empty_execution_state with + wasm_state = {store = initial_wasm_store; self_id = self_id; stable_mem = ""}; + context = s; + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + cycles_used = es.cycles_used; + } + + Note that `params` are undefined in the `(start)` function's execution state which is fine because the System API does not have access to that part of the execution state during the execution of the `(start)` function. + +- The `init` field of the `CanisterModule` is defined as follows: + + If the WebAssembly module does not export a function called under the name `canister_init`, then + + init = λ (self_id, arg, caller, sysenv) → + match start(self_id) with + Trap trap → Trap trap + Return res → Return { + new_state = res.wasm_state; + new_certified_data = NoCertifiedData; + new_global_timer = NoGlobalTimer; + cycles_used = res.cycles_used; + } + + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_init`, it is + + init = λ (self_id, arg, caller, sysenv) → + match start(self_id) with + Trap trap → Trap trap + Return res → + let es = ref {empty_execution_state with + wasm_state = res.wasm_state + params = empty_params with { + arg = arg; + caller = caller; + sysenv = sysenv with { + balance = sysenv.balance - res.cycles_used + } + } + balance = sysenv.balance - res.cycles_used + context = I + } + try func() with Trap then Trap {cycles_used = res.cycles_used + es.cycles_used;} + Return { + new_state = es.wasm_state; + new_certified_data = es.new_certified_data; + new_global_timer = es.new_global_timer; + cycles_used = res.cycles_used + es.cycles_used; + } + +- The `pre_upgrade` field of the `CanisterModule` is defined as follows: + + If the WebAssembly module does not export a function called under the name `canister_pre_upgrade`, then it simply returns the stable memory: + + pre_upgrade = λ (old_state, caller, sysenv) → Return {stable_memory = old_state.stable_mem; new_certified_data = NoCertifiedData; cycles_used = 0;} + + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_pre_upgrade`, it is + + pre_upgrade = λ (old_state, caller, sysenv) → + let es = ref {empty_execution_state with + wasm_state = old_state + params = { empty_params with caller = caller; sysenv } + balance = sysenv.balance + context = G + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + Return { + stable_memory = es.wasm_state.stable_mem; + new_certified_data = es.new_certified_data; + cycles_used = es.cycles_used; + } + +- The `post_upgrade` field of the `CanisterModule` is defined as follows: + + If the WebAssembly module does not export a function called under the name `canister_post_upgrade`, then the argument blob is ignored and the `initial_wasm_store` is returned: + + post_upgrade = λ (self_id, stable_mem, arg, caller, sysenv) → + Return {new_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem }; new_certified_data = NoCertifiedData; new_global_timer = NoGlobalTimer; cycles_used = 0;} + + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_post_upgrade`, it is + + post_upgrade = λ (self_id, stable_mem, arg, caller, sysenv) → + let es = ref {empty_execution_state with + wasm_state = { store = initial_wasm_store; self_id = self_id; stable_mem = stable_mem } + params = { empty_params with arg = arg; caller = caller; sysenv } + balance = sysenv.balance + context = I + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + new_certified_data = es.new_certified_data; + new_global_timer = es.new_global_timer; + cycles_used = es.cycles_used; + } + +- The partial map `update_methods` of the `CanisterModule` is defined for all method names `method` for which the WebAssembly program exports a function `func` named `canister_update `, and has value + + update_methods[method] = λ (arg, caller, sysenv, available) → λ wasm_state → + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = empty_params with { arg = arg; caller = caller; sysenv } + balance = sysenv.balance + cycles_available = available; + context = U + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + new_calls = es.calls; + new_certified_data = es.new_certified_data; + new_global_timer = es.new_global_timer; + response = es.response; + cycles_accepted = es.cycles_accepted; + cycles_used = es.cycles_used; + } + +- The partial map `query_methods` of the `CanisterModule` is defined for all method names `method` for which the WebAssembly program exports a function `func` named `canister_query `, and has value + + query_methods[method] = λ (arg, caller, sysenv) → λ wasm_state → + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = empty_params with { arg = arg; caller = caller; sysenv } + balance = sysenv.balance + context = Q + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + Return { + response = es.response; + cycles_used = es.cycles_used; + } + + By construction, the (possibly modified) `es.wasm_state` is discarded. + +- The function `heartbeat` of the `CanisterModule` is defined if the WebAssembly program exports a function `func` named `canister_heartbeat`, and has value + + heartbeat = λ (sysenv) → λ wasm_state → + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = empty_params with { arg = NoArg; caller = NoCaller; sysenv } + balance = sysenv.balance + context = T + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + new_calls = es.calls; + new_certified_data = es.certified_data; + new_global_timer = es.new_global_timer; + cycles_used = es.cycles_used; + } + + otherwise it is + +```html + +heartbeat = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} + +``` + +- The function `global_timer` of the `CanisterModule` is defined if the WebAssembly program exports a function `func` named `canister_global_timer`, and has value + + global_timer = λ (sysenv) → λ wasm_state → + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = empty_params with { arg = NoArg; caller = NoCaller; sysenv } + balance = sysenv.balance + context = T + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + Return { + new_state = es.wasm_state; + new_calls = es.calls; + new_certified_data = es.certified_data; + new_global_timer = es.new_global_timer; + cycles_used = es.cycles_used; + } + + otherwise it is + +```html + +global_timer = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} + +``` + +- The function `callbacks` of the `CanisterModule` is defined as follows + + callbacks = λ(callbacks, response, refunded_cycles, sysenv, available) → λ wasm_state → + let params0 = { empty_params with + sysenv + cycles_refunded = refund_cycles; + } + let (fun, env, params, context) = match response with + Reply data -> + (callbacks.on_reply.fun, callbacks.on_reply.env, + { params0 with data}, Ry) + Reject (reject_code, reject_message)-> + (callbacks.on_reject.fun, callbacks.on_reject.env, + { params0 with reject_code; reject_message}, Rt) + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = params; + balance = sysenv.balance; + cycles_available = available; + context = context; + } + try + if fun > |es.wasm_state.store.table| then Trap + let func = es.wasm_state.store.table[fun] + if typeof(func) ≠ func (i32) -> () then Trap + + func(env) + Return { + new_state = es.wasm_state; + new_calls = es.calls; + new_certified_data = es.certified_data; + new_global_timer = es.new_global_timer; + response = es.response; + cycles_accepted = es.cycles_accepted; + cycles_used = es.cycles_used; + } + with Trap + if callbacks.on_cleanup = NoClosure then Trap {cycles_used = es.cycles_used;} + if callbacks.on_cleanup.fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} + let func = es.wasm_state.store.table[callbacks.on_cleanup.fun] + if typeof(func) ≠ func (i32) -> () then Trap {cycles_used = es.cycles_used;} + + let es' = ref { empty_execution_state with + wasm_state = wasm_state; + context = C; + } + try func(callbacks.on_cleanup.env) with Trap then Trap {cycles_used = es.cycles_used + es'.cycles_used;} + Return { + new_state = es'.wasm_state; + new_calls = []; + new_certified_data = NoCertifiedData; + new_global_timer = es'.new_global_timer; + response = NoResponse; + cycles_accepted = 0; + cycles_used = es.cycles_used + es'.cycles_used; + } + + Note that if the initial callback handler traps, the cleanup callback (if present) is executed, and the canister has the chance to update its state. + +- The `inspect_message` field of the `CanisterModule` is defined as follows. + + If the WebAssembly module does not export a function called under the name `canister_inspect_message`, then access is always granted: + + inspect_message = λ (method_name, wasm_state, arg, caller, sysenv) → + Return {status = Accept; cycles_used = 0;} + + Otherwise, if the WebAssembly module exports a function `func` under the name `canister_inspect_message`, it is + + inspect_message = λ (method_name, wasm_state, arg, caller, sysenv) → + let es = ref {empty_execution_state with + wasm_state = wasm_state; + params = empty_params with { + arg = arg; + caller = caller; + method_name = method_name; + sysenv + } + balance = sysenv.balance; + cycles_available = 0; // ingress requests have no funds + context = F; + } + try func() with Trap then Trap {cycles_used = es.cycles_used;} + Return {status = es.ingress_filter; cycles_used = es.cycles_used;}; + +#### Helper functions + +In the following section, we use the these helper functions + + copy_to_canister(dst : i32, offset : i32, size : i32, data : blob) = + if offset+size > |data| then Trap {cycles_used = es.cycles_used;} + if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + es.wasm_state.store.mem[dst..dst+size] := data[offset..offset+size] + + copy_from_canister(src : i32, size : i32) blob = + if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + return es.wasm_state.store.mem[src..src+size] + +Cycles are represented by 128-bit values so they require 16 bytes of memory. + + copy_cycles_to_canister(dst : i32, data : blob) = + let size = 16; + if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + es.wasm_state.store.mem[dst..dst+size] := data[0..size] + +#### System imports + +Upon *instantiation* of the WebAssembly module, we can provide the following functions as imports. + +The pseudo-code below does *not* explicitly enforce the restrictions of which imports are available in which contexts; for that the table in [Overview of imports](#system-api-imports) is authoritative, and is assumed to be part of the implementation. + + ic0.msg_arg_data_size() : i32 = + if es.context ∉ {I, U, Q, Ry, F} then Trap {cycles_used = es.cycles_used;} + return |es.params.arg| + + ic0.msg_arg_data_copy(dst:i32, offset:i32, size:i32) = + if es.context ∉ {I, U, Q, Ry, F} then Trap {cycles_used = es.cycles_used;} + copy_to_canister(dst, offset, size, es.params.arg) + + ic0.msg_caller_size() : i32 = + if es.context ∉ {I, G, U, Q, F} then Trap {cycles_used = es.cycles_used;} + return |es.params.caller| + + ic0.msg_caller_copy(dst:i32, offset:i32, size:i32) : i32 = + if es.context ∉ {I, G, U, Q, F} then Trap {cycles_used = es.cycles_used;} + copy_to_canister(dst, offset, size, es.params.caller) + + ic0.msg_reject_code() : i32 = + if es.context ∉ {Ry, Rt} then Trap {cycles_used = es.cycles_used;} + es.params.reject_code + + ic0.msg_reject_msg_size() : i32 = + if es.context ∉ {Rt} then Trap {cycles_used = es.cycles_used;} + return |es.params.reject_msg| + + ic0.msg_reject_msg_copy(dst:i32, offset:i32, size:i32) : i32 = + if es.context ∉ {Rt} then Trap {cycles_used = es.cycles_used;} + copy_to_canister(dst, offset, size, es.params.reject_msg) + + ic0.msg_reply_data_append(src : i32, size : i32) = + if es.context ∉ {U, Q, Ry, Rt} then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + es.reply_params.arg := es.reply_params.arg · copy_from_canister(src, size) + + ic0.msg_reply() = + if es.context ∉ {U, Q, Ry, Rt} then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + es.response := Reply (es.reply_params.arg) + es.cycles_available := 0 + + ic0.msg_reject(src : i32, size : i32) = + if es.context ∉ {U, Q, Ry, Rt} then Trap {cycles_used = es.cycles_used;} + if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} + es.response := Reject (CANISTER_REJECT, copy_from_canister(src, size)) + es.cycles_available := 0 + + ic0.msg_cycles_available() : i64 = + if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} + if es.cycles_available >= 2^64^ then Trap {cycles_used = es.cycles_used;} + return es.cycles_available + + ic0.msg_cycles_available128(dst : i32) = + if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} + let amount = es.cycles_available + copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) + + ic0.msg_cycles_refunded() : i64 = + if es.context ∉ {Rt, Ry} then Trap {cycles_used = es.cycles_used;} + if es.params.cycles_refunded >= 2^64^ then Trap {cycles_used = es.cycles_used;} + return es.params.cycles_refunded + + ic0.msg_cycles_refunded128(dst : i32) = + if es.context ∉ {Rt, Ry} then Trap {cycles_used = es.cycles_used;} + let amount = es.params.cycles_refunded + copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) + + ic0.msg_cycles_accept(max_amount : i64) : i64 = + if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} + let amount = min(max_amount, es.cycles_available) + es.cycles_available := es.cycles_available - amount + es.cycles_accepted := es.cycles_accepted + amount + es.balance := es.balance + amount + return amount + + ic0.msg_cycles_accept128(max_amount_high : i64, max_amount_low : i64, dst : i32) = + if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} + let max_amount = max_amount_high * 2^64^ + max_amount_low + let amount = min(max_amount, es.cycles_available) + es.cycles_available := es.cycles_available - amount + es.cycles_accepted := es.cycles_accepted + amount + es.balance := es.balance + amount + copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) + + ic0.canister_self_size() : i32 = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} + return |es.wasm_state.self_id| + + ic0.canister_self_copy(dst:i32, offset:i32, size:i32) = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} + copy_to_canister(dst, offset, size, es.wasm_state.self_id) + + ic0.canister_cycle_balance() : i64 = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} + if es.balance >= 2^64^ then Trap {cycles_used = es.cycles_used;} + return es.balance + + ic0.canister_cycles_balance128(dst : i32) = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} + let amount = es.balance + copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) + + ic0.canister_status() : i32 = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} + match es.params.sysenv.canister_status with + Running -> return 1 + Stopping -> return 2 + Stopped -> return 3 + + ic0.canister_version() : i64 = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} + return es.params.sysenv.canister_version + + ic0.msg_method_name_size() : i32 = + if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} + return |es.method_name| + + ic0.msg_method_name_copy(dst : i32, offset : i32, size : i32) : i32 = + if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} + copy_to_canister(dst, offset, size, es.params.method_name) + + ic0.accept_message() = + if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} + if es.ingress_filter = Accept then Trap {cycles_used = es.cycles_used;} + es.ingress_filter = Accept + + ic0.call_new( + callee_src : i32, + callee_size : i32, + name_src : i32, + name_size : i32, + reply_fun : i32, + reply_env : i32, + reject_fun : i32, + reject_env : i32, + ) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} + + discard_pending_call() + + if es.balance < MAX_CYCLES_PER_RESPONSE then Trap {cycles_used = es.cycles_used;} + es.balance := es.balance - MAX_CYCLES_PER_RESPONSE + + callee := copy_from_canister(callee_src, callee_size); + method_name := copy_from_canister(name_src, name_size); + + if reply_fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} + if typeof(es.wasm_state.store.table[reply_fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} + + if reject_fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} + if typeof(es.wasm_state.store.table[reject_fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} + + es.pending_call = MethodCall { + callee = callee; + method_name = callee; + arg = ""; + transferred_cycles = 0; + callback = Callback { + on_reply = Closure { fun = reply_fun; env = reply_env } + on_reject = Closure { fun = reject_fun; env = reject_env } + on_cleanup = NoClosure + }; + } + + ic0.call_on_cleanup (fun : i32, env : i32) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} + if fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} + if typeof(es.wasm_state.store.table[fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + if es.pending_call.callback.on_cleanup ≠ NoClosure then Trap {cycles_used = es.cycles_used;} + es.pending_call.callback.on_cleanup := Closure { fun = fun; env = env} + + ic0.call_data_append (src : i32, size : i32) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + es.pending_call.arg := es.pending_call.arg · copy_from_canister(src, size) + + ic0.call_cycles_add(amount : i64) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + if es.balance < amount then Trap {cycles_used = es.cycles_used;} + + es.balance := es.balance - amount + es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount + + ic0.call_cycles_add128(amount_high : i64, amount_low : i64) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} + let amount = amount_high * 2^64^ + amount_low + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + if es.balance < amount then Trap {cycles_used = es.cycles_used;} + + es.balance := es.balance - amount + es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount + + ic0.call_peform() : ( err_code : i32 ) = + if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} + if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} + + // are we below the threezing threshold? + // Or maybe the system has other reasons to not perform this + if es.balance < es.env.freezing_limit or system_cannot_do_this_call_now() + then + discard_pending_call() + return 1 + or + es.calls := es.calls · es.pending_call + es.pending_call := NoPendingCall + return 0 + + // helper function + discard_pending_call() = + if es.pending_call ≠ NoPendingCall then + es.balance := es.balance + MAX_CYCLES_PER_RESPONSE + es.pending_call.transferred_cycles + es.pending_call := NoPendingCall + + ic0.stable_size() : (page_count : i32) = + if |es.wasm_state.store.mem| > 2^32^ then Trap {cycles_used = es.cycles_used;} + page_count := |es.wasm_state.stable_mem| / 64k + return page_count + + ic0.stable_grow(new_pages : i32) : (old_page_count : i32) = + if |es.wasm_state.store.mem| > 2^32^ then Trap {cycles_used = es.cycles_used;} + if arbitrary() then return -1 + else + old_size := |es.wasm_state.stable_mem| / 64k + if old_size + new_pages > 2^16 then return -1 + es.wasm_state.stable_mem := + es.wasm_state.stable_mem · repeat(0x00, new_pages * 64k) + return old_size + + ic0.stable_write(offset : i32, src : i32, size : i32) + if |es.wasm_state.store.mem| > 2^32^ then Trap {cycles_used = es.cycles_used;} + if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} + + es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] + + ic0.stable_read(dst : i32, offset : i32, size : i32) + if |es.wasm_state.store.mem| > 2^32^ then Trap {cycles_used = es.cycles_used;} + if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} + if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + + es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] + + ic0.stable64_size() : (page_count : i64) = + return |es.wasm_state.stable_mem| / 64k + + ic0.stable64_grow(new_pages : i64) : (old_page_count : i64) = + if arbitrary() + then return -1 + else + old_size := |es.wasm_state.stable_mem| / 64k + es.wasm_state.stable_mem := + es.wasm_state.stable_mem · repeat(0x00, new_pages * 64k) + return old_size + + ic0.stable64_write(offset : i64, src : i64, size : i64) + if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} + + es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] + + ic0.stable64_read(dst : i64, offset : i64, size : i64) + if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} + if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} + + es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] + + ic0.certified_data_set(src: i32, size: i32) = + if es.context ∉ {I, G, U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} + es.new_certified_data := es.wasm_state[src..src+size] + + ic0.data_certificate_present() : i32 = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} + if es.params.sysenv.certificate = NoCertificate + then return 0 + else return 1 + + ic0.data_certificate_size() : i32 = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} + if es.params.sysenv.certificate = NoCertificate then Trap {cycles_used = es.cycles_used;} + return |es.params.sysenv.certificate| + + ic0.data_certificate_copy(dst: i32, offset: i32, size: i32) = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} + if es.params.sysenv.certificate = NoCertificate then Trap {cycles_used = es.cycles_used;} + copy_to_canister(dst, offset, size, es.params.sysenv.certificate) + + ic0.time() : i32 = + if es.context ∉ {I, G, U, Q, Ry, Rt, C, F, T} then Trap {cycles_used = es.cycles_used;} + return es.params.sysenv.time + + ic0.global_timer_set(timestamp: i64) : i64 = + if es.context ∉ {I, G, U, Ry, Rt, C, T} then Trap {cycles_used = es.cycles_used;} + let prev_global_timer = es.new_global_timer + es.new_global_timer := timestamp + if prev_global_timer = NoGlobalTimer + then return es.params.sysenv.global_timer + else return prev_global_timer + + ic0.performance_counter(counter_type : i32) : i64 = + arbitrary() + + ic0.is_controller(src: i32, size: i32) : (result: i32) = + bytes = copy_from_canister(src, size) + if bytes encode a principal then + if bytes ∉ es.params.sysenv.controllers + then return 0 + else return 1 + else + Trap {cycles_used = es.cycles_used;} + + ic0.debug_print(src : i32, size : i32) = + return + + ic0.trap(src : i32, size : i32) = + Trap {cycles_used = es.cycles_used;} + + From e7c09711306d6d61f887372c6a8ca16113c68690 Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Thu, 8 Jun 2023 10:57:32 +0200 Subject: [PATCH 054/102] picks deps from master --- spec/http-gateway.did | 12 +++- spec/ic.did | 56 ++++++++++++++++-- theories/IC.thy | 128 +++++++++++++++++++++++++++++++----------- 3 files changed, 156 insertions(+), 40 deletions(-) diff --git a/spec/http-gateway.did b/spec/http-gateway.did index 4ec844f65..e01456db6 100644 --- a/spec/http-gateway.did +++ b/spec/http-gateway.did @@ -5,6 +5,14 @@ type HttpRequest = record { url: text; headers: vec HeaderField; body: blob; + certificate_version: opt nat16; +}; + +type HttpUpdateRequest = record { + method: text; + url: text; + headers: vec HeaderField; + body: blob; }; type HttpResponse = record { @@ -21,7 +29,6 @@ type HttpResponse = record { type StreamingToken = /* application-specific type */ - type StreamingCallbackHttpResponse = record { body: blob; token: opt StreamingToken; @@ -36,6 +43,5 @@ type StreamingStrategy = variant { service : { http_request: (request: HttpRequest) -> (HttpResponse) query; - http_request_update: (request: HttpRequest) -> (HttpResponse); + http_request_update: (request: HttpUpdateRequest) -> (HttpResponse); } - diff --git a/spec/ic.did b/spec/ic.did index a43bcbf2f..6ac4370ff 100644 --- a/spec/ic.did +++ b/spec/ic.did @@ -1,5 +1,4 @@ type canister_id = principal; -type user_id = principal; type wasm_module = blob; type canister_settings = record { @@ -16,6 +15,37 @@ type definite_canister_settings = record { freezing_threshold : nat; }; +type change_origin = variant { + from_user : record { + user_id : principal; + }; + from_canister : record { + canister_id : principal; + canister_version : opt nat64; + }; +}; + +type change_details = variant { + creation : record { + controllers : vec principal; + }; + code_uninstall; + code_deployment : record { + mode : variant {install; reinstall; upgrade}; + module_hash : blob; + }; + controllers_change : record { + controllers : vec principal; + }; +}; + +type change = record { + timestamp_nanos : nat64; + canister_version : nat64; + origin : change_origin; + details : change_details; +}; + type http_header = record { name: text; value: text }; type http_response = record { @@ -83,19 +113,25 @@ type millisatoshi_per_byte = nat64; service ic : { create_canister : (record { - settings : opt canister_settings + settings : opt canister_settings; + sender_canister_version : opt nat64; }) -> (record {canister_id : canister_id}); update_settings : (record { canister_id : principal; - settings : canister_settings + settings : canister_settings; + sender_canister_version : opt nat64; }) -> (); install_code : (record { mode : variant {install; reinstall; upgrade}; canister_id : canister_id; wasm_module : wasm_module; arg : blob; + sender_canister_version : opt nat64; + }) -> (); + uninstall_code : (record { + canister_id : canister_id; + sender_canister_version : opt nat64; }) -> (); - uninstall_code : (record {canister_id : canister_id}) -> (); start_canister : (record {canister_id : canister_id}) -> (); stop_canister : (record {canister_id : canister_id}) -> (); canister_status : (record {canister_id : canister_id}) -> (record { @@ -106,6 +142,15 @@ service ic : { cycles: nat; idle_cycles_burned_per_day: nat; }); + canister_info : (record { + canister_id : canister_id; + num_requested_changes : opt nat64; + }) -> (record { + total_num_changes : nat64; + recent_changes : vec change; + module_hash : opt blob; + controllers : vec principal; + }); delete_canister : (record {canister_id : canister_id}) -> (); deposit_cycles : (record {canister_id : canister_id}) -> (); raw_rand : () -> (blob); @@ -142,7 +187,8 @@ service ic : { // provisional interfaces for the pre-ledger world provisional_create_canister_with_cycles : (record { amount: opt nat; - settings : opt canister_settings + settings : opt canister_settings; + specified_id: opt canister_id; }) -> (record {canister_id : canister_id}); provisional_top_up_canister : (record { canister_id: canister_id; amount: nat }) -> (); diff --git a/theories/IC.thy b/theories/IC.thy index ada3a64db..6b9f0b614 100644 --- a/theories/IC.thy +++ b/theories/IC.thy @@ -306,6 +306,9 @@ lift_definition call_ctxt_respond :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ "\ctxt. ctxt\needs_to_respond := False, available_cycles := 0\" by auto +lemma call_ctxt_respond_canister[simp]: "call_ctxt_canister (call_ctxt_respond ctxt) = call_ctxt_canister ctxt" + by transfer auto + lemma call_ctxt_respond_origin[simp]: "call_ctxt_origin (call_ctxt_respond ctxt) = call_ctxt_origin ctxt" by transfer auto @@ -319,6 +322,9 @@ lift_definition call_ctxt_deduct_cycles :: "nat \ ('p, 'uid, 'canid, "\n ctxt. ctxt\available_cycles := available_cycles ctxt - n\" by auto +lemma call_ctxt_deduct_cycles_canister[simp]: "call_ctxt_canister (call_ctxt_deduct_cycles n ctxt) = call_ctxt_canister ctxt" + by transfer auto + lemma call_ctxt_deduct_cycles_origin[simp]: "call_ctxt_origin (call_ctxt_deduct_cycles n ctxt) = call_ctxt_origin ctxt" by transfer auto @@ -332,6 +338,9 @@ lift_definition call_ctxt_delete :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_c "\ctxt. ctxt\deleted := True, needs_to_respond := False, available_cycles := 0\" by auto +lemma call_ctxt_delete_canister[simp]: "call_ctxt_canister (call_ctxt_delete ctxt) = call_ctxt_canister ctxt" + by transfer auto + lemma call_ctxt_delete_needs_to_respond[simp]: "call_ctxt_needs_to_respond (call_ctxt_delete ctxt) = False" by transfer auto @@ -470,6 +479,53 @@ lemma ic_can_status_inv_del: "ic_can_status_inv x z \ ic_can_status_inv x (z - {ctxt_id})" by (fastforce simp: ic_can_status_inv_def split: can_status.splits call_origin.splits) +definition ic_inv_call_ctxt_stopped :: "('p, 'uid, 'canid, 'b, 's, 'c, 'cid) call_ctxt set \ + ('canid, ('b, 'p, 'uid, 'canid, 's, 'c, 'cid) can_status) list_map \ bool" where + "ic_inv_call_ctxt_stopped ctxts can_status = (\ctxt \ ctxts. list_map_get can_status (call_ctxt_canister ctxt) \ Some Stopped)" + +lemma ic_inv_call_ctxt_stopped_mono1: + assumes "ic_inv_call_ctxt_stopped (list_map_range ctxts) can_status" + shows "ic_inv_call_ctxt_stopped (list_map_range (list_map_del ctxts ctxt_id)) can_status" + using assms + by (auto simp: ic_inv_call_ctxt_stopped_def list_map_range_del) + +lemma ic_inv_call_ctxt_stopped_mono2: + assumes "ic_inv_call_ctxt_stopped ctxts can_status" + shows "ic_inv_call_ctxt_stopped ctxts (list_map_del can_status can_id)" + using assms + by (auto simp: ic_inv_call_ctxt_stopped_def list_map_get_del) + +lemma ic_inv_stopped_ctxt_stopped_set_running: + assumes "ic_inv_call_ctxt_stopped ctxts can_status" + shows "ic_inv_call_ctxt_stopped ctxts (list_map_set can_status cid Running)" + using assms + by (auto simp: ic_inv_call_ctxt_stopped_def list_map_get_set) + +lemma ic_inv_call_ctxt_stopped_set_stopping: + assumes "ic_inv_call_ctxt_stopped ctxts can_status" + shows "ic_inv_call_ctxt_stopped ctxts (list_map_set can_status can_id (Stopping os))" + using assms + by (auto simp: ic_inv_call_ctxt_stopped_def list_map_get_set) + +lemma ic_inv_call_ctxt_stopped_set_stopped: + assumes "ic_inv_call_ctxt_stopped ctxts can_status" + "\ctxt. ctxt \ ctxts \ call_ctxt_canister ctxt \ cid" + shows "ic_inv_call_ctxt_stopped ctxts (list_map_set can_status cid Stopped)" + using assms + by (auto simp: ic_inv_call_ctxt_stopped_def list_map_get_set) + +lemma ic_inv_call_ctxt_stopped_set_respond: + assumes "ic_inv_call_ctxt_stopped (list_map_range ctxts) can_status" "ctxt \ list_map_range ctxts" + shows "ic_inv_call_ctxt_stopped (list_map_range (list_map_set ctxts ctxt_id (call_ctxt_respond ctxt))) can_status" + using assms + by (auto simp: ic_inv_call_ctxt_stopped_def dest!: list_map_range_setD) + +lemma ic_inv_call_ctxt_stopped_map_map: + assumes "ic_inv_call_ctxt_stopped (list_map_range ctxts) can_status" + shows "ic_inv_call_ctxt_stopped (list_map_range (list_map_map (\ctxt. if call_ctxt_canister ctxt = cid then call_ctxt_delete ctxt else ctxt) ctxts)) can_status" + using assms + by (auto simp: ic_inv_call_ctxt_stopped_def) + definition ic_inv :: "('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "ic_inv S = ((\msg \ set (messages S). case msg of Call_message (From_canister ctxt_id _) _ _ _ _ _ _ \ ctxt_id \ list_map_dom (call_contexts S) @@ -478,10 +534,11 @@ definition ic_inv :: "('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ctxt \ list_map_range (call_contexts S). call_ctxt_needs_to_respond ctxt \ (case call_ctxt_origin ctxt of From_canister ctxt_id _ \ ctxt_id \ list_map_dom (call_contexts S) | _ \ True)) \ - ic_can_status_inv (list_map_range (canister_status S)) (list_map_dom (call_contexts S)))" + ic_can_status_inv (list_map_range (canister_status S)) (list_map_dom (call_contexts S)) \ + ic_inv_call_ctxt_stopped (list_map_range (call_contexts S)) (canister_status S))" lemma ic_initial_inv: "ic_inv (initial_ic t pk)" - by (auto simp: ic_inv_def ic_can_status_inv_def initial_ic_def split: call_origin.splits) + by (auto simp: ic_inv_def ic_can_status_inv_def ic_inv_call_ctxt_stopped_def initial_ic_def split: call_origin.splits) (* Candid *) @@ -847,6 +904,9 @@ lift_definition create_call_ctxt :: "'canid \ ('b, 'p, 'uid, 'canid, "\cee orig trans_cycles. \canister = cee, origin = orig, needs_to_respond = True, deleted = False, available_cycles = trans_cycles\" by auto +lemma create_call_ctxt_canister[simp]: "call_ctxt_canister (create_call_ctxt cee orig trans_cycles) = cee" + by transfer auto + lemma create_call_ctxt_origin[simp]: "call_ctxt_origin (create_call_ctxt cee orig trans_cycles) = orig" by transfer auto @@ -883,7 +943,7 @@ lemma call_context_create_ic_inv: assumes "call_context_create_pre n ctxt_id S" "ic_inv S" shows "ic_inv (call_context_create_post n ctxt_id S)" using assms - by (auto simp: ic_inv_def call_context_create_pre_def call_context_create_post_def Let_def + by (auto simp: ic_inv_def ic_inv_call_ctxt_stopped_def call_context_create_pre_def call_context_create_post_def Let_def split: sum.splits message.splits call_origin.splits option.splits dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD intro: ic_can_status_inv_mono2) @@ -896,6 +956,9 @@ lift_definition create_call_ctxt_system_task :: "'canid \ ('p, 'uid, "\cee. \canister = cee, origin = From_system, needs_to_respond = False, deleted = False, available_cycles = 0\" by auto +lemma create_call_ctxt_system_task_canister[simp]: "call_ctxt_canister (create_call_ctxt_system_task cee) = cee" + by transfer auto + lemma create_call_ctxt_system_task_needs_to_respond[simp]: "call_ctxt_needs_to_respond (create_call_ctxt_system_task cee) = False" by transfer auto @@ -927,7 +990,7 @@ lemma call_context_heartbeat_ic_inv: assumes "call_context_heartbeat_pre cee ctxt_id S" "ic_inv S" shows "ic_inv (call_context_heartbeat_post cee ctxt_id S)" using assms - by (auto simp: ic_inv_def call_context_heartbeat_pre_def call_context_heartbeat_post_def Let_def + by (auto simp: ic_inv_def ic_inv_call_ctxt_stopped_def call_context_heartbeat_pre_def call_context_heartbeat_post_def Let_def split: sum.splits message.splits call_origin.splits option.splits dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD intro: ic_can_status_inv_mono2) @@ -963,7 +1026,7 @@ lemma call_context_global_timer_ic_inv: assumes "call_context_global_timer_pre cee ctxt_id S" "ic_inv S" shows "ic_inv (call_context_global_timer_post cee ctxt_id S)" using assms - by (auto simp: ic_inv_def call_context_global_timer_pre_def call_context_global_timer_post_def Let_def + by (auto simp: ic_inv_def ic_inv_call_ctxt_stopped_def call_context_global_timer_pre_def call_context_global_timer_post_def Let_def split: sum.splits message.splits call_origin.splits option.splits dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD intro: ic_can_status_inv_mono2) @@ -1349,7 +1412,7 @@ proof (rule message_execution_cases[OF assms(1)]) call_contexts := list_map_set (call_contexts S) ctxt_id new_ctxt, certified_data := cert_data, global_timer := glob_timer, balances := list_map_set (balances S) recv New_balance\)" using assms(2) list_map_get_range[OF ctxt] by (cases R) - (force simp: ic_inv_def ic_can_status_inv_def split: message.splits call_origin.splits can_status.splits dest!: in_set_takeD in_set_dropD list_map_range_setD)+ + (force simp: ic_inv_def ic_can_status_inv_def ic_inv_call_ctxt_stopped_def split: message.splits call_origin.splits can_status.splits dest!: in_set_takeD in_set_dropD list_map_range_setD)+ next fix recv bal Is_response cyc_used idx show "ic_inv (S\messages := take n (messages S) @ drop (Suc n) (messages S), @@ -1399,7 +1462,7 @@ lemma call_context_starvation_ic_inv: by (auto simp: ic_inv_def call_context_starvation_pre_def call_context_starvation_post_def Let_def split: sum.splits message.splits call_origin.splits option.splits dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range - intro: ic_can_status_inv_mono2) + intro: ic_can_status_inv_mono2 ic_inv_call_ctxt_stopped_set_respond) @@ -1445,7 +1508,7 @@ lemma call_context_removal_ic_inv: by (force simp: ic_inv_def call_context_removal_pre_def call_context_removal_post_def Let_def split: sum.splits message.splits call_origin.splits option.splits dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del - intro!: ic_can_status_inv_del[where ?z="list_map_dom (call_contexts S)"]) + intro!: ic_can_status_inv_del[where ?z="list_map_dom (call_contexts S)"] ic_inv_call_ctxt_stopped_mono1) @@ -1511,7 +1574,7 @@ lemma ic_canister_creation_ic_inv: by (auto simp: ic_inv_def ic_canister_creation_pre_def ic_canister_creation_post_def Let_def split: sum.splits message.splits call_origin.splits option.splits dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del - intro: ic_can_status_inv_mono1) + intro: ic_can_status_inv_mono1 ic_inv_stopped_ctxt_stopped_set_running) @@ -1639,9 +1702,9 @@ definition ic_code_installation_pre :: "nat \ ('p, 'uid, 'canid, 'b, (case parse_wasm_mod w of Some m \ parse_public_custom_sections w \ None \ parse_private_custom_sections w \ None \ - (case (list_map_get (controllers S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid, list_map_get (global_timer S) cid) of - (Some ctrls, Some t, Some bal, Some can_status, Some idx, Some timer) \ - let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\ in + (case (list_map_get (controllers S) cid, list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid) of + (Some ctrls, Some t, Some bal, Some can_status, Some idx) \ + let env = \env.time = t, env.global_timer = 0, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\ in ((mode = encode_string ''install'' \ (case list_map_get (canisters S) cid of Some None \ True | _ \ False)) \ mode = encode_string ''reinstall'') \ cer \ ctrls \ (case canister_module_init m (cid, ar, cer, env) of Inl _ \ False @@ -1656,14 +1719,14 @@ definition ic_code_installation_post :: "nat \ ('p, 'uid, 'canid, 'b (case (candid_parse_text a [encode_string ''mode''], candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of (Some mode, Some w, Some a) \ (case parse_wasm_mod w of Some m \ - (case (list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid, list_map_get (global_timer S) cid) of - (Some t, Some bal, Some can_status, Some idx, Some timer) \ - let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\; + (case (list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid) of + (Some t, Some bal, Some can_status, Some idx) \ + let env = \env.time = t, env.global_timer = 0, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\; (new_state, new_certified_data, new_global_timer, cyc_used) = (case canister_module_init m (cid, a, cer, env) of Inr ret \ (init_return.new_state ret, init_return.new_certified_data ret, init_return.new_global_timer ret, init_return.cycles_used ret)) in S\canisters := list_map_set (canisters S) cid (Some \wasm_state = new_state, module = m, raw_module = w, public_custom_sections = the (parse_public_custom_sections w), private_custom_sections = the (parse_private_custom_sections w)\), - certified_data := (case new_certified_data of None \ certified_data S | Some cd \ list_map_set (certified_data S) cid cd), + certified_data := list_map_set (certified_data S) cid (case new_certified_data of None \ empty_blob | Some cd \ cd), global_timer := list_map_set (global_timer S) cid (case new_global_timer of None \ 0 | Some new_timer \ new_timer), canister_version := list_map_set (canister_version S) cid (Suc idx), balances := list_map_set (balances S) cid (bal - cyc_used), @@ -1675,9 +1738,9 @@ definition ic_code_installation_burned_cycles :: "nat \ ('p, 'uid, ' (case (candid_parse_blob a [encode_string ''wasm_module''], candid_parse_blob a [encode_string ''arg'']) of (Some w, Some a) \ (case parse_wasm_mod w of Some m \ - (case (list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid, list_map_get (global_timer S) cid) of - (Some t, Some bal, Some can_status, Some idx, Some timer) \ - let env = \env.time = t, env.global_timer = timer, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\ in + (case (list_map_get (time S) cid, list_map_get (balances S) cid, list_map_get (canister_status S) cid, list_map_get (canister_version S) cid) of + (Some t, Some bal, Some can_status, Some idx) \ + let env = \env.time = t, env.global_timer = 0, balance = bal, freezing_limit = ic_freezing_limit S cid, certificate = None, status = simple_status can_status, canister_version = Suc idx\ in (case canister_module_init m (cid, a, cer, env) of Inr ret \ init_return.cycles_used ret))))))" lemma ic_code_installation_cycles_inv: @@ -1703,7 +1766,7 @@ lemma ic_code_installation_ic_inv: assumes "ic_code_installation_pre n S" "ic_inv S" shows "ic_inv (ic_code_installation_post n S)" proof - - obtain orig cer cee mn a trans_cycles q cid mode w ar m ctrls t bal can_status idx timer where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" + obtain orig cer cee mn a trans_cycles q cid mode w ar m ctrls t bal can_status idx where msg: "messages S ! n = Call_message orig cer cee mn a trans_cycles q" and parse: "candid_parse_cid a = Some cid" "candid_parse_text a [encode_string ''mode''] = Some mode" "candid_parse_blob a [encode_string ''wasm_module''] = Some w" @@ -1715,7 +1778,6 @@ proof - "list_map_get (balances S) cid = Some bal" "list_map_get (canister_status S) cid = Some can_status" "list_map_get (canister_version S) cid = Some idx" - "list_map_get (global_timer S) cid = Some timer" using assms by (auto simp: ic_code_installation_pre_def split: message.splits option.splits) show ?thesis @@ -1899,9 +1961,10 @@ lemma ic_code_uninstallation_ic_inv: shows "ic_inv (ic_code_uninstallation_post n S)" using assms by (auto simp: ic_inv_def ic_code_uninstallation_pre_def ic_code_uninstallation_post_def Let_def + simp del: list_map_range_map_map split: sum.splits message.splits call_origin.splits option.splits if_splits dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del - in_set_map_filter_vals) + in_set_map_filter_vals intro!: ic_inv_call_ctxt_stopped_map_map) auto @@ -1955,10 +2018,10 @@ proof - by (auto simp: ic_canister_stop_running_pre_def split: message.splits option.splits) show ?thesis using assms - by (force simp: ic_inv_def ic_canister_stop_running_pre_def ic_canister_stop_running_post_def msg Let_def + by (auto simp: ic_inv_def ic_canister_stop_running_pre_def ic_canister_stop_running_post_def msg Let_def split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del - in_set_map_filter_vals intro!: ic_can_status_inv_stopping[where ?x="list_map_range (canister_status S)" and ?os=orig]) + in_set_map_filter_vals intro!: ic_can_status_inv_stopping[where ?x="list_map_range (canister_status S)" and ?os=orig] ic_inv_call_ctxt_stopped_set_stopping) qed @@ -2017,7 +2080,7 @@ proof - by (force simp: ic_inv_def ic_canister_stop_stopping_pre_def ic_canister_stop_stopping_post_def msg Let_def split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del - in_set_map_filter_vals intro!: ic_can_status_inv_stopping_app[where ?x="list_map_range (canister_status S)" and ?os=orig]) + in_set_map_filter_vals intro!: ic_can_status_inv_stopping_app[where ?x="list_map_range (canister_status S)" and ?os=orig] ic_inv_call_ctxt_stopped_set_stopping) qed @@ -2027,7 +2090,7 @@ qed definition ic_canister_stop_done_stopping_pre :: "'canid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ bool" where "ic_canister_stop_done_stopping_pre cid S = (case list_map_get (canister_status S) cid of Some (Stopping os) \ - (\ctxt \ list_map_range (call_contexts S). call_ctxt_deleted ctxt \ call_ctxt_canister ctxt \ cid) + (\ctxt \ list_map_range (call_contexts S). call_ctxt_canister ctxt \ cid) | _ \ False)" definition ic_canister_stop_done_stopping_post :: "'canid \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic \ ('p, 'uid, 'canid, 'b, 'w, 'sm, 'c, 's, 'cid, 'pk) ic" where @@ -2058,7 +2121,7 @@ lemma ic_canister_stop_done_stopping_ic_inv: by (force simp: ic_inv_def ic_can_status_inv_def ic_canister_stop_done_stopping_pre_def ic_canister_stop_done_stopping_post_def Let_def split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del - in_set_map_filter_vals) + in_set_map_filter_vals intro!: ic_inv_call_ctxt_stopped_set_stopped) @@ -2160,7 +2223,7 @@ lemma ic_canister_start_not_stopping_ic_inv: by (auto simp: ic_inv_def ic_canister_start_not_stopping_pre_def ic_canister_start_not_stopping_post_def Let_def split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del - in_set_map_filter_vals intro: ic_can_status_inv_mono1) + in_set_map_filter_vals intro: ic_can_status_inv_mono1 ic_inv_stopped_ctxt_stopped_set_running) @@ -2219,7 +2282,7 @@ lemma ic_canister_start_stopping_ic_inv: apply (auto simp: ic_inv_def ic_canister_start_stopping_pre_def ic_canister_start_stopping_post_def Let_def split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del in_set_map_filter_vals - intro: ic_can_status_inv_mono1) + intro: ic_can_status_inv_mono1 ic_inv_stopped_ctxt_stopped_set_running) apply (fastforce simp: ic_can_status_inv_def)+ done @@ -2285,7 +2348,7 @@ lemma ic_canister_deletion_ic_inv: by (auto simp: ic_inv_def ic_canister_deletion_pre_def ic_canister_deletion_post_def Let_def split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del in_set_map_filter_vals - intro: ic_can_status_inv_mono1) + intro: ic_can_status_inv_mono1 ic_inv_call_ctxt_stopped_mono2) @@ -2451,7 +2514,7 @@ lemma ic_provisional_canister_creation_ic_inv: by (auto simp: ic_inv_def ic_provisional_canister_creation_pre_def ic_provisional_canister_creation_post_def Let_def split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits dest!: in_set_takeD in_set_dropD nth_mem[of n "messages S"] in_set_updD list_map_range_setD list_map_get_range list_map_range_del in_set_map_filter_vals - intro: ic_can_status_inv_mono1) + intro: ic_can_status_inv_mono1 ic_inv_stopped_ctxt_stopped_set_running) @@ -2750,9 +2813,10 @@ lemma canister_out_of_cycles_ic_inv: shows "ic_inv (canister_out_of_cycles_post cid S)" using assms by (auto simp: ic_inv_def canister_out_of_cycles_pre_def canister_out_of_cycles_post_def Let_def + simp del: list_map_range_map_map split: sum.splits message.splits call_origin.splits option.splits if_splits can_status.splits dest!: in_set_takeD in_set_dropD in_set_updD list_map_range_setD list_map_get_range list_map_range_del - in_set_map_filter_vals) + in_set_map_filter_vals intro!: ic_inv_call_ctxt_stopped_map_map) auto From ebbed3c44be3c4c7ac2274264045b8a7010b4d9a Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Thu, 8 Jun 2023 10:58:21 +0200 Subject: [PATCH 055/102] pick deps from master --- spec/ic0.txt | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/spec/ic0.txt b/spec/ic0.txt index 454ff815f..2a17f9d3e 100644 --- a/spec/ic0.txt +++ b/spec/ic0.txt @@ -45,14 +45,14 @@ ic0.call_cycles_add : (amount : i64) -> (); // U ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt T ic0.call_perform : () -> ( err_code : i32 ); // U Ry Rt T -ic0.stable_size : () -> (page_count : i32); // * -ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32); // * -ic0.stable_write : (offset : i32, src : i32, size : i32) -> (); // * -ic0.stable_read : (dst : i32, offset : i32, size : i32) -> (); // * -ic0.stable64_size : () -> (page_count : i64); // * -ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64); // * -ic0.stable64_write : (offset : i64, src : i64, size : i64) -> (); // * -ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> (); // * +ic0.stable_size : () -> (page_count : i32); // * s +ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32); // * s +ic0.stable_write : (offset : i32, src : i32, size : i32) -> (); // * s +ic0.stable_read : (dst : i32, offset : i32, size : i32) -> (); // * s +ic0.stable64_size : () -> (page_count : i64); // * s +ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64); // * s +ic0.stable64_write : (offset : i64, src : i64, size : i64) -> (); // * s +ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> (); // * s ic0.certified_data_set : (src: i32, size: i32) -> (); // I G U Ry Rt T ic0.data_certificate_present : () -> i32; // * @@ -60,8 +60,9 @@ ic0.data_certificate_size : () -> i32; // * ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> (); // * ic0.time : () -> (timestamp : i64); // * -ic0.global_timer_set : (timestamp : i64) -> i64; // I U Ry Rt C T +ic0.global_timer_set : (timestamp : i64) -> i64; // I G U Ry Rt C T ic0.performance_counter : (counter_type : i32) -> (counter : i64); // * s +ic0.is_controller: (src: i32, size: i32) -> ( result: i32); // * s ic0.debug_print : (src : i32, size : i32) -> (); // * s ic0.trap : (src : i32, size : i32) -> (); // * s From 4176010c95dcad12e805bcec93a8bfe5fff9de77 Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Thu, 8 Jun 2023 11:00:27 +0200 Subject: [PATCH 056/102] pick deps from master --- spec/requests.cddl | 97 ---------------------------------------------- 1 file changed, 97 deletions(-) diff --git a/spec/requests.cddl b/spec/requests.cddl index f964bafad..e69de29bb 100644 --- a/spec/requests.cddl +++ b/spec/requests.cddl @@ -1,97 +0,0 @@ -; The "root type" of the CDDL file is a hack: We want to define multiple -; types in one file (to shared common definitions), but CDDL requires a single -; root type. So we just list the types defined here. This is a common CDDL idiom. -start = - call-request / - read-state-request / - query-request / - read-state-response / - query-response - -; common wrappers - -tagged = #6.55799(t) ; the CBOR tag - -envelope = tagged<{ - content: t - ? sender_pubkey: pubkey - ? sender_delegation: [*4 signed-delegation] - ? sender_sig: signature -}> - - -; A request as submitted to /api/v2/.../call -call-request = envelope -call-request-content = { - request_type: "call" - ? nonce : bytes - ingress_expiry : timestamp - sender : principal - canister_id : principal - method_name : text - arg : bytes -} - -; A request as submitted to /api/v2/.../read_state -read-state-request = envelope -read-state-content = { - request_type: "read_state" - ? nonce : bytes - ingress_expiry : timestamp - sender : principal - paths: [* state-path] -} - -state-path = [* path-label] -path-label = bytes - -; The response, as returned from /api/v2/.../read_state -read-state-response = tagged<{ - certificate: bytes -}> - -; A request as submitted to /api/v2/.../query -query-request = envelope -query-content = { - request_type: "query" - ? nonce : bytes - ingress_expiry : timestamp - sender : principal - canister_id : principal - method_name : text - arg : bytes -} - -; The response, as returned from /api/v2/.../query -query-response = tagged<{ - status: "replied" - reply: call-reply - // - status: "rejected" - reject_code: unsigned - reject_message: text - ? error_code: text -}> - -call-reply = { - arg : bytes -} - -; subnet delegations - -signed-delegation = { - delegation: { - pubkey: bytes - expiration: timestamp - ? targets: [* principal] - } - signature: bytes -} - -; some common data types - -principal = bytes .size (0..29) - -pubkey = bytes -signature = bytes -timestamp = unsigned From ab76ef3e59a7cef46e63e6491766645897396071 Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Thu, 8 Jun 2023 11:01:08 +0200 Subject: [PATCH 057/102] pick deps from master --- spec/requests.cddl | 98 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 98 insertions(+) diff --git a/spec/requests.cddl b/spec/requests.cddl index e69de29bb..74a509656 100644 --- a/spec/requests.cddl +++ b/spec/requests.cddl @@ -0,0 +1,98 @@ +; The "root type" of the CDDL file is a hack: We want to define multiple +; types in one file (to shared common definitions), but CDDL requires a single +; root type. So we just list the types defined here. This is a common CDDL idiom. +start = + call-request / + read-state-request / + query-request / + read-state-response / + query-response + +; common wrappers + +tagged = #6.55799(t) ; the CBOR tag + +envelope = tagged<{ + content: t + ? sender_pubkey: pubkey + ? sender_delegation: [*4 signed-delegation] + ? sender_sig: signature +}> + + +; A request as submitted to /api/v2/.../call +call-request = envelope +call-request-content = { + request_type: "call" + ? nonce : bytes + ingress_expiry : timestamp + sender : principal + canister_id : principal + method_name : text + arg : bytes +} + +; A request as submitted to /api/v2/.../read_state +read-state-request = envelope +read-state-content = { + request_type: "read_state" + ? nonce : bytes + ingress_expiry : timestamp + sender : principal + paths: [* state-path] +} + +state-path = [* path-label] +path-label = bytes + +; The response, as returned from /api/v2/.../read_state +read-state-response = tagged<{ + certificate: bytes +}> + +; A request as submitted to /api/v2/.../query +query-request = envelope +query-content = { + request_type: "query" + ? nonce : bytes + ingress_expiry : timestamp + sender : principal + canister_id : principal + method_name : text + arg : bytes +} + +; The response, as returned from /api/v2/.../query +query-response = tagged<{ + status: "replied" + reply: call-reply + // + status: "rejected" + reject_code: unsigned + reject_message: text + ? error_code: text +}> + +call-reply = { + arg : bytes +} + +; user delegations + +signed-delegation = { + delegation: { + pubkey: bytes + expiration: timestamp + ? targets: [* principal] + ? senders: [* principal] + } + signature: bytes +} + +; some common data types + +principal = bytes .size (0..29) + +pubkey = bytes +signature = bytes +timestamp = unsigned From cd42e78fa1ccb1ec1b0994f88744f568ffe8ef83 Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Thu, 8 Jun 2023 11:06:21 +0200 Subject: [PATCH 058/102] pick up some unmerged changes --- spec/index.adoc | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/spec/index.adoc b/spec/index.adoc index 2f1294e2c..ae7e7d928 100644 --- a/spec/index.adoc +++ b/spec/index.adoc @@ -1673,7 +1673,7 @@ The controllers of a canister may stop a canister (e.g., to prepare for a canist Stopping a canister is not an atomic action. The immediate effect is that the status of the canister is changed to `stopping` (unless the canister is already stopped). The IC will reject all calls to a stopping canister, indicating that the canister is stopping. Responses to a stopping canister are processed as usual. -When all outstanding responses to call contexts that are not marked as deleted have been processed (so there are no open call contexts that are not marked as deleted), the canister status is changed to `stopped` and the management canister responds to the caller of the `stop_canister` request. +When all outstanding responses have been processed (so there are no open call contexts), the canister status is changed to `stopped` and the management canister responds to the caller of the `stop_canister` request. [#ic-start_canister] === IC method `start_canister` @@ -2843,7 +2843,6 @@ For a message to a public entry point, the method is looked up in the list of ex The position of the message in the queue is unchanged. Conditions:: -+ .... S.messages = Older_messages · CallMessage CM · Younger_messages S.canisters[CM.callee] ≠ EmptyCanister @@ -2853,7 +2852,6 @@ Conditions:: .... State after:: -+ .... S with messages = @@ -3608,7 +3606,7 @@ S with .... -The status of a stopping canister which has no open call contexts that are not marked as deleted is set to `Stopped`, and all pending `stop_canister` calls are replied to. +The status of a stopping canister which has no open call contexts is set to `Stopped`, and all pending `stop_canister` calls are replied to. Conditions:: .... From 1c57fbb228f616d98b6a09eba83ad51cd060cf79 Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Thu, 8 Jun 2023 11:17:13 +0200 Subject: [PATCH 059/102] switch to ubuntu-latest --- .github/workflows/antora-branch.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/antora-branch.yml b/.github/workflows/antora-branch.yml index 017a9852d..a9d04187b 100644 --- a/.github/workflows/antora-branch.yml +++ b/.github/workflows/antora-branch.yml @@ -5,7 +5,7 @@ on: push: jobs: antora: - runs-on: ubuntu-18.04 + runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 with: From cfdec1fbbe05b7a75d63d575144abdc3e5f22841 Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Fri, 9 Jun 2023 09:04:35 +0200 Subject: [PATCH 060/102] fix broken links --- spec/{ => _attachments}/certificates.cddl | 0 spec/{ => _attachments}/http-gateway.did | 0 spec/{ => _attachments}/ic.did | 0 spec/{ => _attachments}/requests.cddl | 0 spec/http-gateway-protocol-spec.md | 524 ++++++++++++++++++++++ spec/index.md | 4 +- 6 files changed, 526 insertions(+), 2 deletions(-) rename spec/{ => _attachments}/certificates.cddl (100%) rename spec/{ => _attachments}/http-gateway.did (100%) rename spec/{ => _attachments}/ic.did (100%) rename spec/{ => _attachments}/requests.cddl (100%) create mode 100644 spec/http-gateway-protocol-spec.md diff --git a/spec/certificates.cddl b/spec/_attachments/certificates.cddl similarity index 100% rename from spec/certificates.cddl rename to spec/_attachments/certificates.cddl diff --git a/spec/http-gateway.did b/spec/_attachments/http-gateway.did similarity index 100% rename from spec/http-gateway.did rename to spec/_attachments/http-gateway.did diff --git a/spec/ic.did b/spec/_attachments/ic.did similarity index 100% rename from spec/ic.did rename to spec/_attachments/ic.did diff --git a/spec/requests.cddl b/spec/_attachments/requests.cddl similarity index 100% rename from spec/requests.cddl rename to spec/_attachments/requests.cddl diff --git a/spec/http-gateway-protocol-spec.md b/spec/http-gateway-protocol-spec.md new file mode 100644 index 000000000..1af78bb18 --- /dev/null +++ b/spec/http-gateway-protocol-spec.md @@ -0,0 +1,524 @@ +# The HTTP Gateway Protocol Specification + +## Introduction + +The HTTP Gateway Protocol is an extension of the Internet Computer Protocol that allows conventional HTTP clients to interact with the Internet Computer network. This is important for software such as web browsers to be able to fetch and render client-side canister code, including HTML, CSS, and JavaScript as well as other static assets such as images or videos. The HTTP Gateway does this by translating between standard HTTP requests and [API canister calls](https://internetcomputer.org/docs/current/references/ic-interface-spec/#http-interface) that the Internet Computer Protocol will understand. + +Such an HTTP Gateway could be a stand-alone proxy, it could be implemented in web browsers (natively, via a plugin or a service worker) or in other ways. This document describes the interface and semantics of this protocol independent of a concrete HTTP Gateway so that all HTTP Gateway Protocol implementations can be compatible. + +## Overview + +An HTTP request by an HTTP client is handled by these steps: + +1. An HTTP client makes a request. +2. The HTTP Gateway intercepts the request. +3. The HTTP Gateway resolves the canister ID that the request is intended for. +4. The HTTP Gateway Candid encodes the HTTP request. +5. The HTTP Gateway invokes the canister via a query call to the `http_request` interface. +6. The canister handles the request and returns an HTTP response, encoded in Candid, together with additional metadata. +7. If requested by the canister, the HTTP Gateway sends the request again via an update call to `http_request_update`. +8. If applicable, the HTTP Gateway fetches further response body data via streaming query calls. +9. If applicable, the HTTP Gateway validates the certificate of the response. +10. The HTTP Gateway Candid decodes the response and returns it to the HTTP client. + +## Canister ID Resolution + +The HTTP Gateway needs to know the canister ID of the canister to talk to, and obtains that information from the hostname as follows: + +1. If the hostname is in the following table, use the given canister ids: + + | Hostname | Canister id | + | -------------------- | ----------------------------- | + | `identity.ic0.app` | `rdmx6-jaaaa-aaaaa-aaadq-cai` | + | `nns.ic0.app` | `qoctq-giaaa-aaaaa-aaaea-cai` | + | `dscvr.one` | `h5aet-waaaa-aaaab-qaamq-cai` | + | `dscvr.ic0.app` | `h5aet-waaaa-aaaab-qaamq-cai` | + | `personhood.ic0.app` | `g3wsl-eqaaa-aaaan-aaaaa-cai` | + +2. Check whether the hostname is _raw_ (e.g., `.raw.ic0.app`). If it is the case, fail and handle the request as a Web2 request, otherwise, continue. + +3. Check whether the canister ID is embedded in the hostname by splitting the hostname and finding the first occurrence of a valid canister ID from the right. If there is a canister ID embedded in the hostname, use it. + +4. Check whether the canister is hosted on the IC using a custom domain. There are two options: + + - Check whether there is a TXT record containing a canister ID at the `_canister-id`-subdomain (e.g., to see whether `foo.com` is hosted on the IC, make a DNS lookup for the TXT record of `_canister-id.foo.com`) and use the specified canister ID; + + - Make a `HEAD` request to the hostname. If the response contains an `x-ic-canister-id` header, use the value of this header as the canister ID. + +5. Else fail and handle the request as a Web2 request. + +If the hostname was of the form `.ic0.app`, it is a _safe_ hostname; if it was of the form `.raw.ic0.app`, it is a _raw_ hostname. Note that other domains may also be used to access canisters, such as `icp0.io`. The same logic concerning \_raw\_\_ domains can also be applied to these alternative domains. + +## API Gateway Resolution + +An API Gateway forwards Candid encoded HTTP requests to the relevant replica node. Any requests to the Internet Computer made by an HTTP Gateway are forwarded through these API gateways. The hostname of the API gateways is always `icp-api.io`. + +## HTTP Request Encoding + +An HTTP request is encoded using the following [Candid](https://github.com/dfinity/candid/blob/master/spec/Candid.md) interface: + +``` +type HeaderField = record { text; text; }; + +type HttpRequest = record { + method: text; + url: text; + headers: vec HeaderField; + body: blob; + certificate_version: opt nat16; +}; +``` + +The full [Candid](https://github.com/dfinity/candid/blob/master/spec/Candid.md) interface is described in [Canister HTTP Interface](#canister-http-interface). + +- The `method` field contains the HTTP method in all upper case letters, e.g. `"GET"`. +- The `url` field contains the URL from the HTTP request line, i.e. without protocol or hostname, and includes query parameters. +- The `headers` field contains the headers of the HTTP request. +- The `body` field contains the body of the HTTP request (without any content encodings processed by the HTTP Gateway). +- The `certificate_version` field indicates the maximum supported version of [response verification](#response-verification). + - A value of `2` will request the current standard of [response verification](#response-verification), while a missing version or a value of `1` will request the [legacy standard](#legacy-response-verification). + - Current HTTP Gateway implementations will always request version 2, but older HTTP Gateways may still request version 1. + +## Query Calls + +The encoded HTTP request is sent as a query call according to the [HTTPS Interface](https://internetcomputer.org/docs/current/references/ic-interface-spec#http-query) via the API Gateway resolved according to [API Gateway Resolution](#api-gateway-resolution). + +## HTTP Response Decoding + +An HTTP response is decoded from the result of the [query call](#query-calls) using the following [Candid](https://github.com/dfinity/candid/blob/master/spec/Candid.md) interface: + +``` +type HeaderField = record { text; text; }; + +type HttpResponse = record { + status_code: nat16; + headers: vec HeaderField; + body: blob; + upgrade : opt bool; + streaming_strategy: opt StreamingStrategy; +}; +``` + +The full [Candid](https://github.com/dfinity/candid/blob/master/spec/Candid.md) interface is described in [Canister HTTP Interface](#canister-http-interface). + +- The HTTP response status code is taken from the `status_code` field. +- The HTTP response headers are taken from the `headers` field. +- The HTTP response body is initialized with the value of the `body` field and further assembled as per the [response body streaming protocol](#response-body-streaming). + +Notes: + +- Not all HTTP Gateway implementations may be able to pass on all forms of headers. In particular, Service Workers are unable to pass on [forbidden headers](https://fetch.spec.whatwg.org/#forbidden-header-name). +- HTTP Gateways may add additional headers. In particular, the following headers may be set: + - `access-control-allow-origin: \*` + - `access-control-allow-methods: GET, POST, HEAD, OPTIONS` + - `access-control-allow-headers: DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Cookie` + - `access-control-expose-headers: Content-Length,Content-Range` + - `x-cache-status: MISS` + +## Response Verification + +The HTTP Gateway will primarily be used to load static assets needed to run frontend canister code, so both low latency and security are essential for providing a good experience to end users. [Query calls](https://internetcomputer.org/docs/current/references/ic-interface-spec/#http-query) are more performant but less secure than [Update calls](https://internetcomputer.org/docs/current/references/ic-interface-spec/#http-call). + +Response verification fills the security gap left by query calls. It is a versioned subprotocol that allows for an HTTP Gateway to verify a certified response received as a result of performing a query call to the Internet Computer. Two versions are currently supported, the current version of response verification is covered in this section and the legacy version is covered in [another section](#legacy-response-verification). The legacy version only includes a mapping of the request URL to the response body so it is quite limiting in what it can verify. The current version builds on the legacy version by optionally including the following extra parameters in the certification process: + +- Request URL query params +- Request method +- Request headers +- Response status code +- Response headers + +### Response Verification Outline + +1. Case-insensitive search for the `IC-Certificate` response header. + - If no such header is found, verification fails. + - If the header value is not structured as per [the certificate header](#the-certificate-header), verification fails. +2. Parse the `certificate` and `tree` fields from the `IC-Certificate` header value as per [the certificate header](#the-certificate-header). +3. Perform [certificate validation](#certificate-validation). +4. Parse the `version` field from the `IC-Certificate` header value as per [the certificate header](#the-certificate-header). + - If the `version` field is missing or equal to `1` then proceed with [legacy response verification](#legacy-response-verification). + - If the `version` field is equal to `2` then continue. + - Otherwise, verification fails. +5. Parse the `expr_path` field from the `IC-Certificate` header value as per [the certificate header](#the-certificate-header). +6. The parsed `expr_path` is valid as per [Expression Path](#expression-path) otherwise, verification fails. +7. Case-insensitive search for the `IC-CertificationExpression` header. + - If no such header is found, verification fails. + - If the header value is not structured as per [the certificate expression header](#the-certificate-expression-header), verification fails. +8. Let `expr_hash` be the label of the node in the tree at path `expr_path`. + - If no such label exists, verification fails. + - If `expr_hash` does not match the sha256 hash of the `IC-CertificateExpression` header value, verification fails. + - If `no_certification` is set, verification succeeds. + - Let `response_hash` be the response hash calculated according to [Response Hash Calculation](#response-hash-calculation) + - If `no_request_certification` is set: + - If the `expr_hash` label node has an empty leaf node at the subpath `["", response_hash]`, verification succeeds. + - Otherwise, verification fails. + - Let `request_hash` be the request hash calculated according to [Request Hash Calculation](#request-hash-calculation). + - If there is not an empty leaf node at the subpath `[request_hash, response_hash]`, verification fails. + +### The Certificate Header + +The `IC-Certificate` header is a structured header according to [RFC 8941](https://www.rfc-editor.org/rfc/rfc8941.html) with the following mandatory fields: + +- `certificate`: [Base64 encoded](https://www.rfc-editor.org/rfc/rfc4648#section-4) string of self-describing, [CBOR-encoded](https://www.rfc-editor.org/rfc/rfc8949.html) bytes that decode into a valid [certificate](https://internetcomputer.org/docs/current/references/ic-interface-spec/#certification). +- `tree`: [Base64 encoded](https://www.rfc-editor.org/rfc/rfc4648#section-4) string of self-describing, [CBOR-encoded](https://www.rfc-editor.org/rfc/rfc8949.html) bytes that decode into a valid hash tree as per [certificate encoding](https://internetcomputer.org/docs/current/references/ic-interface-spec/#certification-encoding). + +The following additional fields are mandatory for response verification version 2 and upwards: + +- `version`: String representation of an integer that represents the version of response verification that was used to build the `tree`. +- `expr_path`: [Base64 encoded](https://www.rfc-editor.org/rfc/rfc4648#section-4) string of self-describing, [CBOR-encoded](https://www.rfc-editor.org/rfc/rfc8949.html) bytes that decode into an array of strings. + +### Expression Path + +The decoded `expr_path` field of [The Certificate Header](#the-certificate-header) is an array of strings that corresponds to a path in the `tree` field of the same header: + +- The first segment is always `http_expr`. +- The last segment is always `<$>` or `<*>`. +- No segment, aside from the last segment, will be `<$>` or `<*>`. +- Each segment between `http_expr` and `<$>` or `<*>` will contain a [percent-encoded](https://www.rfc-editor.org/rfc/rfc3986#section-2) segment of the current request URL. +- The path must be the most specific path for the current request URL in the tree, i.e. a lookup of more specific paths must return `Absent` as per [lookup](https://internetcomputer.org/docs/current/references/ic-interface-spec/#lookup). +- An `expr_path` that ends in `<$>` is an exact match for the current request URL. +- `<*>` is treated as a wildcard, so an `expr_path` that ends in `<*>` is a partial match for the current request URL. + +### Certificate Validation + +Certificate validation is performed as part of [response verification](#response-verification) as per [Canister Signatures](https://internetcomputer.org/docs/current/references/ic-interface-spec/#canister-signatures) and [Certification](https://internetcomputer.org/docs/current/references/ic-interface-spec/#certificate). It is expanded on here concerning [response verification](#response-verification) for completeness: + +1. Case-insensitive search for a response header called `IC-Certificate`. +2. The value of the header corresponds to the format described in [the certificate header](#the-certificate-header) section. +3. The decoded `certificate` must pass the following validations: + - The certificate is signed by the root key of the NNS subnet or by a subnet delegation signed by that same root key. + - If the certificate contains a subnet delegation, the delegation must be valid for the given canister. + - The timestamp at the `/time` path must be recent, e.g. 5 minutes. + - The subnet state tree in the certificate must reveal the canister's [certified data](https://internetcomputer.org/docs/current/references/ic-interface-spec/#system-api-certified-data). +4. The root hash of the decoded `tree` must match the canister's [certified data](https://internetcomputer.org/docs/current/references/ic-interface-spec/#system-api-certified-data). + +### The Certificate Expression Header + +The `IC-CertificateExpression` header carries additional information instructing the HTTP Gateway how to reconstruct the certification, it can instruct the HTTP Gateway to: + +- Exclude the complete request/response pair or the request only. +- Include specific request headers. +- Include specific request URL query parameters. +- Include or exclude specific response headers. + +The format of the `IC-CertificateExpression` header is as follows: + +``` +IC-CertificateExpression: default_certification(ValidationArgs{}) +``` + +The value of this header must have valid [CEL syntax](https://github.com/google/cel-spec), such that `default_certification` could be implemented as a function provided by the HTTP Gateway to validate the certification. + +The properties supplied to this function are as follows: + +- `certified_request_headers` - a list of request header names to include. This list can be empty. + - Mutually exclusive with the `no_request_certification` property. +- `certified_query_parameters` - a list of request URL query parameter names to include. This list can be empty. + - Mutually exclusive with the `no_request_certification` property. +- `certified_response_headers` - a list of response header names to include. + - Must not include `IC-Certificate` or `IC-CertificateExpression`. + - Mutually exclusive with the `response_header_exclusions` property. +- `response_header_exclusions` - a list of response header names to exclude. All other headers are included. + - Must not include `IC-Certificate` or `IC-CertificateExpression`. + - Mutually exclusive with the `certified_response_headers` property. +- `no_request_certification` - disables certification of the request for this HTTP response. + - Mutually exclusive with the `certified_request_headers` and `certified_query_parameters` properties. + - This feature has security implications. If it is used on a path that serves dynamic content using the [upgrade to update call](#upgrade-to-update-calls) feature, malicious replica nodes can always return the certified response, instead of setting the upgrade flag on the response. +- `no_certification` - disables certification for this HTTP request/response pair. + - This feature has security implications. Clients will not be able to verify the authenticity of the response content if it is used. Dynamic content can be returned securely by making use of the [upgrade to update](#upgrade-to-update-calls) feature. Only use `no_certification` if the content is dynamic, the latency of an update call is too high and the impact of a malicious response on that path is benign. + +The `ValidationArgs` object has the following [Protocol Buffer 3](https://protobuf.dev/reference/protobuf/proto3-spec/) definition: + +```protobuf +message ResponseHeaderList { + repeated string headers = 1; +} + +message RequestCertification { + repeated string certified_request_headers = 1; + repeated string certified_query_parameters = 2; +} + +message ResponseCertification { + oneof response_headers { + ResponseHeaderList certified_response_headers = 1; + ResponseHeaderList response_header_exclusions = 2; + } +} + +message Certification { + oneof request { + RequestCertification request_certification = 1; + Empty no_request_certification = 2; + } + ResponseCertification response_certification = 3; +} + +message ValidationArgs { + oneof certification { + Certification certification = 1; + Empty no_certification = 2; + } +} +``` + +The syntax of the header is defined by the following [EBNF](https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form): + +``` +CHAR = /[^\0\n"]/ +STRING = '"', { CHAR }, '"' +STRING-LIST = '[', { STRING }, ']' + +RESPONSE-HEADER-LIST = 'ResponseHeaderList{headers:', STRING-LIST, '}' + +REQUEST-CERTIFICATION = 'RequestCertification{certified_request_headers:', STRING-LIST, ',certified_query_parameters:', STRING-LIST, '}' + +RESPONSE-CERTIFICATION = 'ResponseCertification{', ('response_header_exclusions:' | 'certified_response_headers:'), RESPONSE-HEADER-LIST, '}' + +CERTIFICATION = 'Certification{', ('no_request_certification: Empty{}' | 'request_certification:', REQUEST-CERTIFICATION), ',response_certification:', RESPONSE-CERTIFICATION, '}' + +VALIDATION-ARGS = 'ValidationArgs{', ('no_certification: Empty{}' | 'certification:', CERTIFICATION), '}' + +HEADER-VALUE = 'default_certification(', VALIDATION-ARGS, ')' + +HEADER = 'IC-CertificateExpression: ', HEADER-VALUE +``` + +### Request Hash Calculation + +The request hash is calculated as follows: + +1. Let `request_headers_hash` be the [representation-independent hash](https://internetcomputer.org/docs/current/references/ic-interface-spec#hash-of-map) of the request headers: + - Only include headers listed in the `certified_request_headers` field of [the certificate expression header](#the-certificate-expression-header). + - If the field is empty or no value was supplied, no headers are included. + - Headers can be repeated and each repetition should be included. + - Include an additional `:ic-cert-method` header that contains the HTTP method of the request. + - Include an additional `:ic-cert-query` header that contains a value according to the following steps: + - Parse the query string and build a list of tuples `(, )` while maintaining the order. + - Exclude all tuples where `` does not exactly match a value listed in the `certified_query_parameters` field of [the certificate expression header](#the-certificate-expression-header). If `certified_query_parameters` is empty then the resulting list of tuples should also be empty. + - Concatenate each `` with the corresponding `` and then concatenate all of these concatenations using the original separators and order. + - Calculate the sha256 hash of the UTF-8 representation of the resulting string. +2. Let `request_body_hash` be the sha256 of the request body. +3. Concatenate `request_headers_hash` and `request_body_hash` and calculate the sha256 of that concatenation. + +### Response Hash Calculation + +The response hash is calculated as follows: + +1. Let `response_headers_hash` be the [representation-independent hash](https://internetcomputer.org/docs/current/references/ic-interface-spec#hash-of-map) of the response headers: + - The `IC-Certificate` header is always excluded. + - The `IC-CertificateExpression` header is always included. + - If the `no_certification` field of [the certificate expression header](#the-certificate-expression-header) is present: + - This request/response pair is exempt from certification and the response hash calculation can be skipped altogether + - If the `certified_response_headers` field of [the certificate expression header](#the-certificate-expression-header) is present: + - All headers listed by certified_response_headers are included (except for the `IC-Certificate` header) + - All others are excluded (except for the `IC-CertificateExpression` header) + - If the `response_header_exclusions` field of [the certificate expression header](#the-certificate-expression-header) is present: + - All headers listed (except for the `IC-CertificateExpression` header) are excluded from the certification + - All other headers (except for the IC-Certificate header) are included in the certification + - Headers can be repeated and each repetition should be included. +2. Let `response_body_hash` be the sha256 of the response body. +3. Concatenate `response_headers_hash` and `response_body_hash` and calculate the sha256 of that concatenation. + +### Multiple CEL Expression Hashes Per Expression Path + +Adding one CEL expression hash per expression path should be the default and most common case as it is the most secure approach. It is, however, possible to add multiple CEL expression hashes per expression path, if the flexibility is needed by a canister. This feature is quite dangerous and must be used with extreme caution. By adding a 2nd CEL expression hash, a canister is giving a malicious replica node freedom to choose a different CEL expression hash for a request than what is intended by the canister. This could be used to expose a potential vulnerability that does not exist if the intended CEL expression hash is used. This should only be used in cases where the difference in CEL expression hashes is benign and will not pose a security threat to the canister, or there is not sufficient overlap between the CEL expressions to allow the replica node to freely choose between them. + +### Multiple Response Hashes Per Request Hash + +Similar to [Multiple CEL Expression Hashes Per Expression Path](#multiple-cel-expression-hashes-per-expression-path), this is a feature that is intended to allow for flexibility when it is needed. It is also dangerous and must be used with great care. By adding multiple response hashes for a single request hash, a malicious replica can freely choose between any of those response hashes for that request hash. This should only be used in cases where the difference between the responses is benign and will not pose a security threat to the canister. + +## Response Body Streaming + +The HTTP Gateway protocol has provisions to transfer further chunks of the body data from the canister to the HTTP Gateway, to overcome the message limit of the Internet Computer. This streaming protocol is independent of any possible streaming of data between the HTTP Gateway and the HTTP client. The HTTP Gateway may assemble the response as a whole before passing it on, or pass the chunks on directly, on the TCP or HTTP level, as it sees fit. When the HTTP Gateway is certifying the response, it must not pass on uncertified chunks. + +If the `streaming_strategy` field of the `HttpResponse` is set, the HTTP Gateway then uses further query calls to obtain further chunks to append to the body: + +If the function reference in the callback field of the `streaming_strategy` is not a method of the given canister, the HTTP Gateway fails the request. + +Else, it makes a query call to the given method, passing the token value given in the `streaming_strategy` as the argument. + +That query method returns a `StreamingCallbackHttpResponse`. The body therein is appended to the body of the HTTP response. This is repeated as long as the method returns some token in the token field until that field is null. + +The type of the token value is chosen by the canister; the HTTP Gateway obtains the Candid type of the encoded message from the canister and uses it when passing the token back to the canister. This generic use of Candid is not covered by the Candid specification, and may not be possible in some cases (e.g. when using "future types"). Canister authors may have to use "simple" types. + +## Upgrade to Update Calls + +If the canister sets `upgrade = opt true` in the `HttpResponse` reply from the `http_request` call, then the HTTP Gateway ignores all other fields of the response. The HTTP Gateway performs an [update](https://internetcomputer.org/docs/current/references/ic-interface-spec#http-call) call to `http_request_update`, passing an `HttpUpdateRequest` record as the argument, and uses the resulting response from `http_request_update` instead. The `HttpUpdateRequest` record is identical to the original `HttpRequest`, with the `certificate_version` field excluded. + +The value of the `upgrade` field returned from `http_request_update` is ignored. + +## Legacy Response Verification + +Version 1 response verification only supports verifying a request path and response body pair with only one response per request path. This is quite restrictive in the number of scenarios it can support. For example, redirection or client-side caching is not safe since the status code and headers required to verify responses of that nature are not included in the certification. Upon a query call to a canister’s `http_request` method, a single malicious node or boundary node can modify these parts of the HTTP response, leading to the following issues: + +- dApps cannot load the service worker when embedded within iFrames. +- The use of redirects and cookies is unsafe as they can be manipulated by malicious nodes. +- This is unexpected for developers and will lead to vulnerabilities in dApps sooner or later. +- The effectiveness of security headers (such as Content Security Policy) is diminished as they can be omitted or modified by malicious nodes. + +[Response Verification version 2](#response-verification) overcomes these issues. + +The steps for response verification are as follows: + +- See the [response verification outline](#response-verification-outline) for the full subprotocol description. +- Assert that the canister returning the response does not have support for response verification v2 via [Response Verification Version Assertion](#response-verification-version-assertion). + - If the canister reports that it has support for response verification v2, verification fails. + - Otherwise, continue. +- The path `["http_assets", ]` exists in the `tree` and is a leaf with a value, where `` is the utf8-encoded URL from the `HttpRequest`. +- Otherwise, the path `["http_assets", "/index.html"]` must exist in the `tree` and be a leaf. +- That leaf must contain the SHA-256 hash of the decoded body. + - If the `streaming_strategy` field of the `HttpResponse` is set, all chunks are streamed and concatenated according to [response body streaming](#response-body-streaming) before decoding. + - The body is decoded according to the `Content-Encoding` header if present. Supported values for the `Content-Encoding` header include `gzip` and `deflate`. + +## Response Verification Version Assertion + +Canisters can report the versions of response verification that they support using public metadata in the [system state tree](https://internetcomputer.org/docs/current/references/ic-interface-spec/#state-tree-canister-information). This metadata will be read by the HTTP Gateway using a [read_state request](https://internetcomputer.org/docs/current/references/ic-interface-spec/#http-read-state). This metadata is a comma-delimited string of versions under the key "supported_certificate_versions”, for example: "1,2". This is treated as an optional, additional layer of security for canisters supporting multiple versions. If the metadata has not been added, then the HTTP Gateway will allow for whatever version the canister has responded with. + +The request for the metadata will only be made by the HTTP Gateway if there is a downgrade. If the HTTP Gateway requests v2 and the canister responds with v2, then a request will not be made. If the HTTP Gateway requests v2 and the canister responds with v1, a request will be made. If a request is made, the HTTP Gateway will not accept any response from the canister that is below the max version supported by both the HTTP Gateway and the canister. This will guarantee that a canister supporting both v1 and v2 will always have v2 security when accessed by an HTTP Gateway that supports v2. + +## Canister HTTP Interface + +The full [Candid](https://github.com/dfinity/candid/blob/master/spec/Candid.md) interface that a canister is expected to implement is as follows: + +``` +type HeaderField = record { text; text; }; + +type HttpRequest = record { + method: text; + url: text; + headers: vec HeaderField; + body: blob; + certificate_version: opt nat16; +}; + +type HttpUpdateRequest = record { + method: text; + url: text; + headers: vec HeaderField; + body: blob; +}; + +type HttpResponse = record { + status_code: nat16; + headers: vec HeaderField; + body: blob; + upgrade : opt bool; + streaming_strategy: opt StreamingStrategy; +}; + +// Each canister that uses the streaming feature gets to choose their concrete +// type; the HTTP Gateway will treat it as an opaque value that is only fed to +// the callback method + +type StreamingToken = /* application-specific type */ + +type StreamingCallbackHttpResponse = record { + body: blob; + token: opt StreamingToken; +}; + +type StreamingStrategy = variant { + Callback: record { + callback: func (StreamingToken) -> (opt StreamingCallbackHttpResponse) query; + token: StreamingToken; + }; +}; + +service : { + http_request: (request: HttpRequest) -> (HttpResponse) query; + http_request_update: (request: HttpUpdateRequest) -> (HttpResponse); +} +``` + +You can also [download the file](./_attachments/http-gateway.did). + +Not all of this interface is required. The following sections detail what can be optionally omitted depending on the requirements of the canister in question. + +### Response Verification Interface + +The `certificate_version` field of the `HttpRequest` interface is optional depending on the version of [response verification](#response-verification) that the canister is implementing. It is omitted in older canisters that do not implement response verification version 2 or later. + +``` +type HttpRequest = record { + // ... + certificate_version: opt nat16; +}; +``` + +### Upgrade to Update Calls Interface + +The `http_request_update` method of the `service` interface along with the `upgrade` field of the `HttpResponse` interface is optional depending on whether the canister needs to use the [upgrade to update calls](#upgrade-to-update-calls) feature. Not that the `HttpUpdateRequest` type is the same as the `HttpRequest` type, but excludes the `certificate_version` field since this should not affect the response to an [update](https://internetcomputer.org/docs/current/references/ic-interface-spec#http-call) call from a canister. + +``` +type HttpUpdateRequest = record { + method: text; + url: text; + headers: vec HeaderField; + body: blob; +}; + +type HttpResponse = record { + // ... + upgrade : opt bool; + // ... +}; + +service : { + // ... + http_request_update: (request: HttpUpdateRequest) -> (HttpResponse); +} +``` + +### Response Body Streaming Interface + +The `StreamingToken`, `StreamingCallbackHttpResponse`, and `StreamingStrategy` interfaces along with the `streaming_strategy` field of the `HttpResponse` interface are optional depending on whether the canister needs to use the [response body streaming](#response-body-streaming) feature. + +``` +type HttpResponse = record { + // ... + streaming_strategy: opt StreamingStrategy; +}; + +// Each canister that uses the streaming feature gets to choose their concrete +// type; the HTTP Gateway will treat it as an opaque value that is only fed to +// the callback method + +type StreamingToken = /* application-specific type */ + +type StreamingCallbackHttpResponse = record { + body: blob; + token: opt StreamingToken; +}; + +type StreamingStrategy = variant { + Callback: record { + callback: func (StreamingToken) -> (opt StreamingCallbackHttpResponse) query; + token: StreamingToken; + }; +}; +``` + +### Minimum Canister Interface + +If all of the above optional features are not needed by a canister, the minimum [Candid](https://github.com/dfinity/candid/blob/master/spec/Candid.md) interface that it needs to implement is as follows: + +``` +type HeaderField = record { text; text; }; + +type HttpRequest = record { + method: text; + url: text; + headers: vec HeaderField; + body: blob; +}; + +type HttpResponse = record { + status_code: nat16; + headers: vec HeaderField; + body: blob; +}; + +service : { + http_request: (request: HttpRequest) -> (HttpResponse) query; +} +``` diff --git a/spec/index.md b/spec/index.md index fc9c597b1..cdbbaa5bd 100644 --- a/spec/index.md +++ b/spec/index.md @@ -946,7 +946,7 @@ A typical request would be (written in [CBOR diagnostic notation](https://www.rf ### CDDL description of requests and responses {#api-cddl} -This section summarizes the format of the CBOR data passed to and from the entry points described above. You can also [download the file]({attachmentsdir}/requests.cddl) and see [CDDL](#cddl) for more information. +This section summarizes the format of the CBOR data passed to and from the entry points described above. You can also [download the file](_attachments/requests.cddl) and see [CDDL](#cddl) for more information. ### Ordering guarantees @@ -2241,7 +2241,7 @@ Delegations are *scoped*, i.e., they indicate which set of canister principals t ### Encoding of certificates {#certification-encoding} -The binary encoding of a certificate is a CBOR (see [CBOR](#cbor)) value according to the following CDDL (see [CDDL](#cddl)). You can also [download the file]({attachmentsdir}/certificates.cddl). +The binary encoding of a certificate is a CBOR (see [CBOR](#cbor)) value according to the following CDDL (see [CDDL](#cddl)). You can also [download the file](_attachments/certificates.cddl). The values in the [The system state tree](#state-tree) are encoded to blobs as follows: From 2cb598806c013b906f7ad9d38f04b6dc1731d4f6 Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Fri, 9 Jun 2023 09:17:09 +0200 Subject: [PATCH 061/102] fix http-gateway-protocol-spec link --- spec/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/index.md b/spec/index.md index cdbbaa5bd..38d5c9b96 100644 --- a/spec/index.md +++ b/spec/index.md @@ -2313,7 +2313,7 @@ In the pruned tree, the `lookup_path` function behaves as follows: ## The HTTP Gateway protocol {#http-gateway} -The HTTP Gateway Protocol has been moved into its own [specification](https://internetcomputer.org/docs/current/references/http-gateway-protocol-spec). +The HTTP Gateway Protocol has been moved into its own [specification](./http-gateway-protocol-spec.md). ## Abstract behavior {#abstract-behavior} From b32db09b6d8333f307a4d5f36c622b8c46ba0930 Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Tue, 11 Jul 2023 18:18:58 +0200 Subject: [PATCH 062/102] release 0.20.0 --- spec/_attachments/interface-spec-changelog.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/_attachments/interface-spec-changelog.md b/spec/_attachments/interface-spec-changelog.md index d6d0eeb46..a1b365bf1 100644 --- a/spec/_attachments/interface-spec-changelog.md +++ b/spec/_attachments/interface-spec-changelog.md @@ -1,6 +1,6 @@ ## Changelog {#changelog} -### ∞ (unreleased) +### 0.20.0 (2023-07-11) {#0_20_0} * IC Bitcoin API, ECDSA API, canister HTTPS outcalls API, and 128-bit cycles System API are considered stable. * Add conditions on requested paths in read state requests. * Add composite queries. From aadc1c74179c7f1d5613de6b7304b7fc5e435e0a Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Thu, 10 Aug 2023 11:33:14 +0200 Subject: [PATCH 063/102] updated management canister .did --- spec/_attachments/ic.did | 9 +++++++++ spec/index.md | 6 ++++++ 2 files changed, 15 insertions(+) diff --git a/spec/_attachments/ic.did b/spec/_attachments/ic.did index 44b44e722..f176fca1d 100644 --- a/spec/_attachments/ic.did +++ b/spec/_attachments/ic.did @@ -46,6 +46,8 @@ type change = record { details : change_details; }; +type chunk_hash = blob; + type http_header = record { name: text; value: text }; type http_response = record { @@ -121,6 +123,13 @@ service ic : { settings : canister_settings; sender_canister_version : opt nat64; }) -> (); + upload_chunk : (record { + canister_id : principal; + chunk : blob; + }) -> chunk_hash; + stored_chunks: (record {canister_id : canister_id}) -> vec chunk_hash; + delete_chunks: (record {hash_list : vec chunk_hash}) -> (); + clear_chunk_store: (record {canister_id: canister_id}) -> (); install_code : (record { mode : variant {install; reinstall; upgrade}; canister_id : canister_id; diff --git a/spec/index.md b/spec/index.md index 9482029c2..f93be6259 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1806,6 +1806,12 @@ Not including a setting in the `settings` record means not changing that field. The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. + +### IC method `upload_chunk` {#ic-upload_chunk} +Canisters have associated some storage space where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister to upload such chunks. + + + ### IC method `install_code` {#ic-install_code} This method installs code into a canister. From 43d4d9284c2512c637bc907a0fd3a8eef6fb0879 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Mon, 14 Aug 2023 11:42:29 +0200 Subject: [PATCH 064/102] high level descriptions for IC methods --- spec/_attachments/ic.did | 2 +- spec/index.md | 7 ++++++- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/spec/_attachments/ic.did b/spec/_attachments/ic.did index f176fca1d..fa0d1df21 100644 --- a/spec/_attachments/ic.did +++ b/spec/_attachments/ic.did @@ -128,7 +128,7 @@ service ic : { chunk : blob; }) -> chunk_hash; stored_chunks: (record {canister_id : canister_id}) -> vec chunk_hash; - delete_chunks: (record {hash_list : vec chunk_hash}) -> (); + delete_chunks: (record {canister_id : canister_id, hash_list : vec chunk_hash}) -> (); clear_chunk_store: (record {canister_id: canister_id}) -> (); install_code : (record { mode : variant {install; reinstall; upgrade}; diff --git a/spec/index.md b/spec/index.md index f93be6259..5487922bf 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1808,8 +1808,13 @@ The optional `sender_canister_version` parameter can contain the caller's canist ### IC method `upload_chunk` {#ic-upload_chunk} -Canisters have associated some storage space where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister to upload such chunks. +Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister to upload such chunks. The method returns the hash of the chunk that was stored. +### IC method `delete_chunks` {#ic-delete_chunks} +Canister controllers can delete chunks stored in the canister's chunk storage. The caller provides as input a list of hashes; the result is that the IC deletes the chunks corresponding to those hashes from the canister's chunk storage. + +### IC method `clear_store` {#ic-clear_store} +Canister controllers can clear the entire chunk storage of a canister. ### IC method `install_code` {#ic-install_code} From f596bc1ef7856b1a50f9e1eb94092ec3bae0fca3 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Mon, 14 Aug 2023 17:28:45 +0200 Subject: [PATCH 065/102] chunked-wasm cddl --- spec/_attachments/chunked-wasms.cddl | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 spec/_attachments/chunked-wasms.cddl diff --git a/spec/_attachments/chunked-wasms.cddl b/spec/_attachments/chunked-wasms.cddl new file mode 100644 index 000000000..646b38efb --- /dev/null +++ b/spec/_attachments/chunked-wasms.cddl @@ -0,0 +1,15 @@ + +; The specification of a large Wasm; consists of a location (canister principal) and a list of hashes. + + + +chunked-wasm = tagged<{ + ? location : principal + list_of_hashes : [2* hash] +}> + +hash = bytes .size (0..32) + +principal = bytes .size (0..29) + +tagged = #6.55799(t) ; the CBOR tag \ No newline at end of file From 8a6c0aeb0c7f774efc97f4bf3d6443430c6a4e73 Mon Sep 17 00:00:00 2001 From: mraszyk <31483726+mraszyk@users.noreply.github.com> Date: Tue, 15 Aug 2023 10:28:03 +0200 Subject: [PATCH 066/102] Update spec/_attachments/chunked-wasms.cddl --- spec/_attachments/chunked-wasms.cddl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/_attachments/chunked-wasms.cddl b/spec/_attachments/chunked-wasms.cddl index 646b38efb..48675638b 100644 --- a/spec/_attachments/chunked-wasms.cddl +++ b/spec/_attachments/chunked-wasms.cddl @@ -5,7 +5,7 @@ chunked-wasm = tagged<{ ? location : principal - list_of_hashes : [2* hash] + list_of_hashes : [* hash] }> hash = bytes .size (0..32) From acd844efeaaedbe93da5127ac06a10b4c684555e Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Tue, 15 Aug 2023 10:45:44 +0200 Subject: [PATCH 067/102] hashes are 32 bytes --- spec/_attachments/chunked-wasms.cddl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/_attachments/chunked-wasms.cddl b/spec/_attachments/chunked-wasms.cddl index 646b38efb..9cf5c5ed4 100644 --- a/spec/_attachments/chunked-wasms.cddl +++ b/spec/_attachments/chunked-wasms.cddl @@ -8,7 +8,7 @@ chunked-wasm = tagged<{ list_of_hashes : [2* hash] }> -hash = bytes .size (0..32) +hash = bytes .size 32 principal = bytes .size (0..29) From 446e7dd09630b8dbaf92137ad58b69dcaf90203a Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Tue, 15 Aug 2023 11:12:59 +0200 Subject: [PATCH 068/102] fix did file --- spec/_attachments/ic.did | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/spec/_attachments/ic.did b/spec/_attachments/ic.did index fa0d1df21..ac14a6e30 100644 --- a/spec/_attachments/ic.did +++ b/spec/_attachments/ic.did @@ -126,9 +126,9 @@ service ic : { upload_chunk : (record { canister_id : principal; chunk : blob; - }) -> chunk_hash; - stored_chunks: (record {canister_id : canister_id}) -> vec chunk_hash; - delete_chunks: (record {canister_id : canister_id, hash_list : vec chunk_hash}) -> (); + }) -> (chunk_hash); + stored_chunks: (record {canister_id : canister_id}) -> (vec chunk_hash); + delete_chunks: (record {canister_id : canister_id; hash_list : vec chunk_hash}) -> (); clear_chunk_store: (record {canister_id: canister_id}) -> (); install_code : (record { mode : variant {install; reinstall; upgrade}; From dcffe0ec0f9bd24b2be13e5fab0f1006aea3611f Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Mon, 21 Aug 2023 09:53:44 +0200 Subject: [PATCH 069/102] informal descriptions for storage management & installing large wasms --- spec/_attachments/chunked-wasms.cddl | 4 ++-- spec/index.md | 8 +++++--- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/spec/_attachments/chunked-wasms.cddl b/spec/_attachments/chunked-wasms.cddl index 9cf5c5ed4..479941979 100644 --- a/spec/_attachments/chunked-wasms.cddl +++ b/spec/_attachments/chunked-wasms.cddl @@ -5,11 +5,11 @@ chunked-wasm = tagged<{ ? location : principal - list_of_hashes : [2* hash] + list_of_hashes : [* hash] }> hash = bytes .size 32 principal = bytes .size (0..29) -tagged = #6.55799(t) ; the CBOR tag \ No newline at end of file +tagged = #6.55799(t) ; the CBOR tag diff --git a/spec/index.md b/spec/index.md index 5487922bf..1f2fdbcc8 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1808,15 +1808,14 @@ The optional `sender_canister_version` parameter can contain the caller's canist ### IC method `upload_chunk` {#ic-upload_chunk} -Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister to upload such chunks. The method returns the hash of the chunk that was stored. +Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister to upload such chunks. The method returns the hash of the chunk that was stored. The size of each chunk should be up to 1MB. The size of the chunk store is bounded: currently it can hold up to 40chunks. ### IC method `delete_chunks` {#ic-delete_chunks} -Canister controllers can delete chunks stored in the canister's chunk storage. The caller provides as input a list of hashes; the result is that the IC deletes the chunks corresponding to those hashes from the canister's chunk storage. +Canister controllers can delete chunks stored in the canister's chunk storage. The caller provides as input a list of hashes; the effect of the call is that the IC deletes from the canister's chunk storage the chunks that correspond to hashes in the provided list. ### IC method `clear_store` {#ic-clear_store} Canister controllers can clear the entire chunk storage of a canister. - ### IC method `install_code` {#ic-install_code} This method installs code into a canister. @@ -1845,6 +1844,9 @@ The `wasm_module` field specifies the canister module to be installed. The syste - If the `wasm_module` starts with byte sequence `[0x1f, 0x8b, 0x08]`, the system parses `wasm_module` as a gzip-compressed WebAssembly binary. +- If the `wasm_module` starts with the byte sequence `0xd9 0xd9 0xf7 `, the system parses `wasm_module` as a self describing CBOR encoding of a map that specifies: an optional canister identifier `storage_canister` and a list of hash values, `[h0,h1,...,hk]`. In this case, the system looks up in the chunk store of `storage_canister` (or that of the target canister if this parameter is not provided) blobs corresponding to `h1,...,hk`, concatenates them to obtain a blob of bytes and checks that `h0` is the hash of the resulting blob. The caller must be the controller of the `storage_canister`. If the lookup succeeds, then the system interprets the blob as a vanilla wasm module (or a gzipped one) per the rules above. + + The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. ### IC method `uninstall_code` {#ic-uninstall_code} From 94f5bee609a1ed8e4c75ca0fa44a662695299856 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Mon, 21 Aug 2023 17:22:35 +0200 Subject: [PATCH 070/102] upload_chunk abstract spec --- spec/index.md | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/spec/index.md b/spec/index.md index 759a90f73..872993169 100644 --- a/spec/index.md +++ b/spec/index.md @@ -106,6 +106,10 @@ This specification may refer to certain constants and limits without specifying Maximum amount of cycles that can be used in total (across all calls to query and composite query methods and their callbacks) during evaluation of a query call. +- `MAX_CHUNK_STORE_SIZE` + + Maximum number of chunks that can be stored within the chunk store of a canister. + - `DEFAULT_PROVISIONAL_CYCLES_BALANCE` Amount of cycles allocated to a new canister by default, if not explicitly specified. See [IC method](#ic-provisional_create_canister_with_cycles). @@ -2443,6 +2447,8 @@ The [WebAssembly System API](#system-api) is relatively low-level, and some of i WasmState = (abstract) StableMemory = (abstract) Callback = (abstract) + ChunkStore = Hash -> Blob + Arg = Blob; CallerId = Principal; @@ -2684,6 +2690,7 @@ Finally, we can describe the state of the IC as a record having the following fi CanState = EmptyCanister | { wasm_state : WasmState; + chunk_store: ChunkStore; module : CanisterModule; raw_module : Blob; public_custom_sections: Text ↦ Blob; @@ -3433,6 +3440,7 @@ S with time[CanisterId] = CurrentTime global_timer[CanisterId] = 0 controllers[CanisterId] = New_controllers + chunk_store[CanisterId] = () if A.settings.freezing_threshold is not null: freezing_threshold[CanisterId] = A.settings.freezing_threshold else: @@ -3618,6 +3626,41 @@ S with ``` + +#### IC Management Canister: Upload Chunk + +The controller of a canister, or the canister itself can upload chunks to the chunk store of that canister. + +Conditions + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.method_name = 'upload_chunk' +M.arg = candid(A) +chunk_store_size = |chunk_store[M.canister_id]| +chunk_store_size < MAX_CHUNK_STORE_SIZE and (M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id}) +hash = SHA-256(A.chunk) + +``` + +State after + +```html + +S with + chunk_store[M.canister_id](hash') = chunk_store[M.canister_id] if hash'≠ hash + chunk_store[M.canister_id](hash) = A.chunk + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = candid(hash) + } + +``` + + #### IC Management Canister: Code installation Only the controllers of the given canister can install code. This transition installs new code over a canister. This involves invoking the `canister_init` method (see [Canister initialization](#system-api-init)), which must succeed. From 43ac367a87f8139b85744d357be312c4bf45d6f0 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Mon, 21 Aug 2023 17:26:39 +0200 Subject: [PATCH 071/102] clarified size of chunk stores --- spec/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/index.md b/spec/index.md index 872993169..05c05764f 100644 --- a/spec/index.md +++ b/spec/index.md @@ -3639,7 +3639,7 @@ S.messages = Older_messages · CallMessage M · Younger_messages (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) M.method_name = 'upload_chunk' M.arg = candid(A) -chunk_store_size = |chunk_store[M.canister_id]| +chunk_store_size = |{x | chunk_store[M.canister_id][x] not null}| chunk_store_size < MAX_CHUNK_STORE_SIZE and (M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id}) hash = SHA-256(A.chunk) From 92f683bb96fd4d146b048c5f80cdf34549bda828 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Mon, 21 Aug 2023 17:31:06 +0200 Subject: [PATCH 072/102] canister management canister: clear store --- spec/index.md | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-) diff --git a/spec/index.md b/spec/index.md index 05c05764f..dff5e0653 100644 --- a/spec/index.md +++ b/spec/index.md @@ -3639,7 +3639,7 @@ S.messages = Older_messages · CallMessage M · Younger_messages (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) M.method_name = 'upload_chunk' M.arg = candid(A) -chunk_store_size = |{x | chunk_store[M.canister_id][x] not null}| +chunk_store_size = |{x | chunk_store[A.canister_id][x] not null}| chunk_store_size < MAX_CHUNK_STORE_SIZE and (M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id}) hash = SHA-256(A.chunk) @@ -3650,8 +3650,8 @@ State after ```html S with - chunk_store[M.canister_id](hash') = chunk_store[M.canister_id] if hash'≠ hash - chunk_store[M.canister_id](hash) = A.chunk + chunk_store[A.canister_id](hash') = chunk_store[A.canister_id] if hash'≠ hash + chunk_store[A.canister_id](hash) = A.chunk messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin @@ -3660,6 +3660,33 @@ S with ``` +#### IC Management Canister: Clear Chunk Store + +The controller of a canister, or the canister itself can clear the chunk store of that canister. + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.method_name = 'clear_store' +M.arg = candid(A) +``` + +State after + +```html + +S with + chunk_store[A.canister_id] = () + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = candid() + } + +``` + + #### IC Management Canister: Code installation From 43db1c5f55e37b5d45bd8feaf94a7dec63a0c5c7 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Mon, 21 Aug 2023 21:46:38 +0200 Subject: [PATCH 073/102] delete chunks --- spec/index.md | 224 ++++++++++++++++++++++++++++---------------------- 1 file changed, 128 insertions(+), 96 deletions(-) diff --git a/spec/index.md b/spec/index.md index dff5e0653..a3405405e 100644 --- a/spec/index.md +++ b/spec/index.md @@ -203,7 +203,7 @@ Example encoding from hex, and decoding to hex, in bash (the following can be pa xxd -r -p | base32 | tr A-Z a-z | tr -d = | fold -w5 | paste -sd'-' - } - + function textual_decode() { echo -n "$1" | tr -d - | tr a-z A-Z | fold -w 8 | xargs -n1 printf '%-8s' | tr ' ' = | @@ -1187,11 +1187,11 @@ The following sections describe various System API functions, also referred to a ic0.msg_reject_code : () -> i32; // Ry Rt CRy CRt ic0.msg_reject_msg_size : () -> i32; // Rt CRt ic0.msg_reject_msg_copy : (dst : i32, offset : i32, size : i32) -> (); // Rt CRt - + ic0.msg_reply_data_append : (src : i32, size : i32) -> (); // U Q CQ Ry Rt CRy CRt ic0.msg_reply : () -> (); // U Q CQ Ry Rt CRy CRt ic0.msg_reject : (src : i32, size : i32) -> (); // U Q CQ Ry Rt CRy CRt - + ic0.msg_cycles_available : () -> i64; // U Rt Ry ic0.msg_cycles_available128 : (dst : i32) -> (); // U Rt Ry ic0.msg_cycles_refunded : () -> i64; // Rt Ry @@ -1199,18 +1199,18 @@ The following sections describe various System API functions, also referred to a ic0.msg_cycles_accept : (max_amount : i64) -> (amount : i64); // U Rt Ry ic0.msg_cycles_accept128 : (max_amount_high : i64, max_amount_low: i64, dst : i32) -> (); // U Rt Ry - + ic0.canister_self_size : () -> i32; // * ic0.canister_self_copy : (dst : i32, offset : i32, size : i32) -> (); // * ic0.canister_cycle_balance : () -> i64; // * ic0.canister_cycle_balance128 : (dst : i32) -> (); // * ic0.canister_status : () -> i32; // * ic0.canister_version : () -> i64; // * - + ic0.msg_method_name_size : () -> i32; // F ic0.msg_method_name_copy : (dst : i32, offset : i32, size : i32) -> (); // F ic0.accept_message : () -> (); // F - + ic0.call_new : // U CQ Ry Rt CRy CRt T ( callee_src : i32, callee_size : i32, @@ -1226,7 +1226,7 @@ The following sections describe various System API functions, also referred to a ic0.call_cycles_add : (amount : i64) -> (); // U Ry Rt T ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt T ic0.call_perform : () -> ( err_code : i32 ); // U CQ Ry Rt CRy CRt T - + ic0.stable_size : () -> (page_count : i32); // * s ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32); // * s ic0.stable_write : (offset : i32, src : i32, size : i32) -> (); // * s @@ -1235,17 +1235,17 @@ The following sections describe various System API functions, also referred to a ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64); // * s ic0.stable64_write : (offset : i64, src : i64, size : i64) -> (); // * s ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> (); // * s - + ic0.certified_data_set : (src: i32, size: i32) -> (); // I G U Ry Rt T ic0.data_certificate_present : () -> i32; // * ic0.data_certificate_size : () -> i32; // * ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> (); // * - + ic0.time : () -> (timestamp : i64); // * ic0.global_timer_set : (timestamp : i64) -> i64; // I G U Ry Rt C T ic0.performance_counter : (counter_type : i32) -> (counter : i64); // * s ic0.is_controller: (src: i32, size: i32) -> ( result: i32); // * s - + ic0.debug_print : (src : i32, size : i32) -> (); // * s ic0.trap : (src : i32, size : i32) -> (); // * s @@ -1871,7 +1871,7 @@ The `wasm_module` field specifies the canister module to be installed. The syste - If the `wasm_module` starts with byte sequence `[0x1f, 0x8b, 0x08]`, the system parses `wasm_module` as a gzip-compressed WebAssembly binary. - If the `wasm_module` starts with the byte sequence `0xd9 0xd9 0xf7 `, the system parses `wasm_module` as a self describing CBOR encoding of a map that specifies: an optional canister identifier `storage_canister` and a list of hash values, `[h0,h1,...,hk]`. In this case, the system looks up in the chunk store of `storage_canister` (or that of the target canister if this parameter is not provided) blobs corresponding to `h1,...,hk`, concatenates them to obtain a blob of bytes and checks that `h0` is the hash of the resulting blob. The caller must be the controller of the `storage_canister`. If the lookup succeeds, then the system interprets the blob as a vanilla wasm module (or a gzipped one) per the rules above. - + The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. @@ -2183,13 +2183,13 @@ A certificate is validated with regard to the root of trust by the following alg let der_key = check_delegation(cert.delegation) // see section Delegations below bls_key = extract_der(der_key) verify_bls_signature(bls_key, cert.signature, domain_sep("ic-state-root") · root_hash) - + reconstruct(Empty) = H(domain_sep("ic-hashtree-empty")) reconstruct(Fork t1 t2) = H(domain_sep("ic-hashtree-fork") · reconstruct(t1) · reconstruct(t2)) reconstruct(Labeled l t) = H(domain_sep("ic-hashtree-labeled") · l · reconstruct(t)) reconstruct(Leaf v) = H(domain_sep("ic-hashtree-leaf") · v) reconstruct(Pruned h) = h - + domain_sep(s) = byte(|s|) · s where `H` is the SHA-256 hash function, @@ -2253,7 +2253,7 @@ The IC will only produce well-formed state trees, and the above algorithm assume well_formed(tree) = (tree = Leaf _) ∨ (well_formed_forest(flatten_forks(tree))) - + well_formed_forest(trees) = strictly_increasing([l | Label l _ ∈ trees]) ∧ ∀ Label _ t ∈ trees. well_formed(t) ∧ @@ -2452,7 +2452,7 @@ The [WebAssembly System API](#system-api) is relatively low-level, and some of i Arg = Blob; CallerId = Principal; - + Timestamp = Nat; CanisterVersion = Nat; Env = { @@ -2465,7 +2465,7 @@ The [WebAssembly System API](#system-api) is relatively low-level, and some of i status : Running | Stopping | Stopped; canister_version : CanisterVersion; } - + RejectCode = Nat Response = Reply Blob | Reject (RejectCode, Text) MethodCall = { @@ -2475,7 +2475,7 @@ The [WebAssembly System API](#system-api) is relatively low-level, and some of i transferred_cycles: Nat; callback: Callback; } - + UpdateFunc = WasmState -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; new_calls : List MethodCall; @@ -2502,10 +2502,10 @@ The [WebAssembly System API](#system-api) is relatively low-level, and some of i new_global_timer : NoGlobalTimer | Nat; cycles_used : Nat; } - + AvailableCycles = Nat RefundedCycles = Nat - + CanisterModule = { init : (CanisterId, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; @@ -2594,7 +2594,7 @@ Therefore, a message can have different shapes: | Callback Callback Response RefundedCycles | Heartbeat | GlobalTimer - + Message = CallMessage { origin : CallOrigin; @@ -2661,7 +2661,7 @@ Signed delegations contain the (unsigned) delegation data in a nested record, ne For the signatures in a `Request`, we assume that the following function implements signature verification as described in [Authentication](#authentication). This function picks the corresponding signature scheme according to the DER-encoded metadata in the public key. verify_signature : PublicKey -> Signature -> Blob -> Bool - + Envelope = { content : Request | APIReadRequest; sender_pubkey : PublicKey | NoPublicKey; @@ -2859,19 +2859,19 @@ The following predicate describes when an envelope `E` correctly signs the enclo = TS if U = mk_self_authenticating_id E.sender_pubkey ∧ (PK', TS) = verify_delegations(DS, PK, T, { p : p is CanisterId }, U) ∧ verify_signature PK' Sig ("\x0Aic-request" · hash_of_map(C)) - + verify_delegations([], PK, T, TS, U) = (PK, TS) verify_delegations([D] · DS, PK, T, TS, U) = verify_delegations(DS, D.pubkey, T, TS ∩ delegation_targets(D), U) if verify_signature PK D.signature ("\x1Aic-request-auth-delegation" · hash_of_map(D.delegation)) ∧ D.delegation.expiration ≥ T ∧ U ∈ delegated_senders(D) - + delegation_targets(D) = if D.targets = Unrestricted then { p : p is CanisterId } else D.targets - + delegated_senders(D) = if D.senders = Unrestricted then { p : p is Principal } @@ -3330,7 +3330,7 @@ The functions `query_as_update` and `system_task_as_update` turns a query functi cycles_accepted = 0; cycles_used = res.cycles_used; } - + system_task_as_update(f, env) = λ wasm_state → match f(env)(wasm_state) with Trap trap → Trap trap @@ -3660,7 +3660,7 @@ S with ``` -#### IC Management Canister: Clear Chunk Store +#### IC Management Canister: Clear chunk store The controller of a canister, or the canister itself can clear the chunk store of that canister. @@ -3687,6 +3687,38 @@ S with ``` +#### IC Management Canister: Delete chunks + +The controller of a canister, or the canister itself can delete chunks from the canister chunk store. + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.method_name = 'delete_chunks' +M.arg = candid(A) +chunk_store'(hash) = chunk_store[M.canister_id](hash) if hash ∉ A.hash_list +chunk_store'(hash) = null if hash ∈ A.hash_list + +``` + +State after + +```html + +S with + chunk_store[A.canister_id] = chunk_store' + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = candid() + } + +``` + + + + #### IC Management Canister: Code installation @@ -4792,7 +4824,7 @@ where `state_tree` constructs a labeled tree from the IC state `S` and the (so f { "metadata": { name: blob | (name, blob) ∈ S.canisters[canister_id].public_custom_sections ∪ S.canisters[canister_id].private_custom_sections } } | (canister_id, C) ∈ S.canisters }; } - + request_status_tree(Received) = { "status": "received" } request_status_tree(Processing) = @@ -4830,7 +4862,7 @@ The abstract `Callback` type above models an entry point for responses: fun : i32, env : i32, } - + Callback = { on_reply : Closure; on_reject : Closure; @@ -4886,7 +4918,7 @@ It is nonsensical to pass to an execution function a WebAssembly store `S` that cycles_refunded = 0; method_name = NoText; } - + empty_execution_state = { wasm_state = (undefined); params = (undefined); @@ -5160,7 +5192,7 @@ global_timer = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} if fun > |es.wasm_state.store.table| then Trap let func = es.wasm_state.store.table[fun] if typeof(func) ≠ func (i32) -> () then Trap - + func(env) Return { new_state = es.wasm_state; @@ -5176,7 +5208,7 @@ global_timer = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} if callbacks.on_cleanup.fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} let func = es.wasm_state.store.table[callbacks.on_cleanup.fun] if typeof(func) ≠ func (i32) -> () then Trap {cycles_used = es.cycles_used;} - + let es' = ref { empty_execution_state with wasm_state = wasm_state; context = C; @@ -5217,7 +5249,7 @@ global_timer = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} if fun > |es.wasm_state.store.table| then Trap let func = es.wasm_state.store.table[fun] if typeof(func) ≠ func (i32) -> () then Trap - + func(env) Return { new_state = es.wasm_state; @@ -5230,7 +5262,7 @@ global_timer = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} if callbacks.on_cleanup.fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} let func = es.wasm_state.store.table[callbacks.on_cleanup.fun] if typeof(func) ≠ func (i32) -> () then Trap {cycles_used = es.cycles_used;} - + let es' = ref { empty_execution_state with wasm_state = wasm_state; context = CC; @@ -5278,7 +5310,7 @@ In the following section, we use the these helper functions if offset+size > |data| then Trap {cycles_used = es.cycles_used;} if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} es.wasm_state.store.mem[dst..dst+size] := data[offset..offset+size] - + copy_from_canister(src : i32, size : i32) blob = if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} return es.wasm_state.store.mem[src..src+size] @@ -5299,68 +5331,68 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im ic0.msg_arg_data_size() : i32 = if es.context ∉ {I, U, Q, CQ, Ry, CRy, F} then Trap {cycles_used = es.cycles_used;} return |es.params.arg| - + ic0.msg_arg_data_copy(dst:i32, offset:i32, size:i32) = if es.context ∉ {I, U, Q, CQ, Ry, CRy, F} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.arg) - + ic0.msg_caller_size() : i32 = if es.context = s then Trap {cycles_used = es.cycles_used;} return |es.params.caller| - + ic0.msg_caller_copy(dst:i32, offset:i32, size:i32) : i32 = if es.context = s then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.caller) - + ic0.msg_reject_code() : i32 = if es.context ∉ {Ry, Rt, CRy, CRt} then Trap {cycles_used = es.cycles_used;} es.params.reject_code - + ic0.msg_reject_msg_size() : i32 = if es.context ∉ {Rt, CRt} then Trap {cycles_used = es.cycles_used;} return |es.params.reject_msg| - + ic0.msg_reject_msg_copy(dst:i32, offset:i32, size:i32) : i32 = if es.context ∉ {Rt, CRt} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.reject_msg) - + ic0.msg_reply_data_append(src : i32, size : i32) = if es.context ∉ {U, Q, CQ, Ry, Rt, CRy, CRt} then Trap {cycles_used = es.cycles_used;} if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.reply_params.arg := es.reply_params.arg · copy_from_canister(src, size) - + ic0.msg_reply() = if es.context ∉ {U, Q, CQ, Ry, Rt, CRy, CRt} then Trap {cycles_used = es.cycles_used;} if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.response := Reply (es.reply_params.arg) es.cycles_available := 0 - + ic0.msg_reject(src : i32, size : i32) = if es.context ∉ {U, Q, CQ, Ry, Rt, CRy, CRt} then Trap {cycles_used = es.cycles_used;} if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.response := Reject (CANISTER_REJECT, copy_from_canister(src, size)) es.cycles_available := 0 - + ic0.msg_cycles_available() : i64 = if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} if es.cycles_available >= 2^64 then Trap {cycles_used = es.cycles_used;} return es.cycles_available - + ic0.msg_cycles_available128(dst : i32) = if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} let amount = es.cycles_available copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) - + ic0.msg_cycles_refunded() : i64 = if es.context ∉ {Rt, Ry} then Trap {cycles_used = es.cycles_used;} if es.params.cycles_refunded >= 2^64 then Trap {cycles_used = es.cycles_used;} return es.params.cycles_refunded - + ic0.msg_cycles_refunded128(dst : i32) = if es.context ∉ {Rt, Ry} then Trap {cycles_used = es.cycles_used;} let amount = es.params.cycles_refunded copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) - + ic0.msg_cycles_accept(max_amount : i64) : i64 = if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} let amount = min(max_amount, es.cycles_available) @@ -5368,7 +5400,7 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im es.cycles_accepted := es.cycles_accepted + amount es.balance := es.balance + amount return amount - + ic0.msg_cycles_accept128(max_amount_high : i64, max_amount_low : i64, dst : i32) = if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} let max_amount = max_amount_high * 2^64 + max_amount_low @@ -5377,49 +5409,49 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im es.cycles_accepted := es.cycles_accepted + amount es.balance := es.balance + amount copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) - + ic0.canister_self_size() : i32 = if es.context = s then Trap {cycles_used = es.cycles_used;} return |es.wasm_state.self_id| - + ic0.canister_self_copy(dst:i32, offset:i32, size:i32) = if es.context = s then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.wasm_state.self_id) - + ic0.canister_cycle_balance() : i64 = if es.context = s then Trap {cycles_used = es.cycles_used;} if es.balance >= 2^64 then Trap {cycles_used = es.cycles_used;} return es.balance - + ic0.canister_cycles_balance128(dst : i32) = if es.context = s then Trap {cycles_used = es.cycles_used;} let amount = es.balance copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) - + ic0.canister_status() : i32 = if es.context = s then Trap {cycles_used = es.cycles_used;} match es.params.sysenv.canister_status with Running -> return 1 Stopping -> return 2 Stopped -> return 3 - + ic0.canister_version() : i64 = if es.context = s then Trap {cycles_used = es.cycles_used;} return es.params.sysenv.canister_version - + ic0.msg_method_name_size() : i32 = if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} return |es.method_name| - + ic0.msg_method_name_copy(dst : i32, offset : i32, size : i32) : i32 = if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.method_name) - + ic0.accept_message() = if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} if es.ingress_filter = Accept then Trap {cycles_used = es.cycles_used;} es.ingress_filter = Accept - + ic0.call_new( callee_src : i32, callee_size : i32, @@ -5431,21 +5463,21 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im reject_env : i32, ) = if es.context ∉ {U, CQ, Ry, Rt, CRy, CRt, T} then Trap {cycles_used = es.cycles_used;} - + discard_pending_call() - + if es.balance < MAX_CYCLES_PER_RESPONSE then Trap {cycles_used = es.cycles_used;} es.balance := es.balance - MAX_CYCLES_PER_RESPONSE - + callee := copy_from_canister(callee_src, callee_size); method_name := copy_from_canister(name_src, name_size); - + if reply_fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} if typeof(es.wasm_state.store.table[reply_fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} - + if reject_fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} if typeof(es.wasm_state.store.table[reject_fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} - + es.pending_call = MethodCall { callee = callee; method_name = callee; @@ -5457,7 +5489,7 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im on_cleanup = NoClosure }; } - + ic0.call_on_cleanup (fun : i32, env : i32) = if es.context ∉ {U, CQ, Ry, Rt, CRy, CRt, T} then Trap {cycles_used = es.cycles_used;} if fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} @@ -5465,35 +5497,35 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} if es.pending_call.callback.on_cleanup ≠ NoClosure then Trap {cycles_used = es.cycles_used;} es.pending_call.callback.on_cleanup := Closure { fun = fun; env = env} - + ic0.call_data_append (src : i32, size : i32) = if es.context ∉ {U, CQ, Ry, Rt, CRy, CRt, T} then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} es.pending_call.arg := es.pending_call.arg · copy_from_canister(src, size) - + ic0.call_cycles_add(amount : i64) = if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} if es.balance < amount then Trap {cycles_used = es.cycles_used;} if es.balance - amount < es.params.sysenv.freezing_limit then Trap {cycles_used = es.cycles_used;} - + es.balance := es.balance - amount es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount - + ic0.call_cycles_add128(amount_high : i64, amount_low : i64) = if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} let amount = amount_high * 2^64 + amount_low if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} if es.balance < amount then Trap {cycles_used = es.cycles_used;} if es.balance - amount < es.params.sysenv.freezing_limit then Trap {cycles_used = es.cycles_used;} - + es.balance := es.balance - amount es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount - + ic0.call_peform() : ( err_code : i32 ) = if es.context ∉ {U, CQ, Ry, Rt, CRy, CRt, T} then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} - + // are we below the threezing threshold? // Or maybe the system has other reasons to not perform this if es.balance < es.params.sysenv.freezing_limit or system_cannot_do_this_call_now() @@ -5504,18 +5536,18 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im es.calls := es.calls · es.pending_call es.pending_call := NoPendingCall return 0 - + // helper function discard_pending_call() = if es.pending_call ≠ NoPendingCall then es.balance := es.balance + MAX_CYCLES_PER_RESPONSE + es.pending_call.transferred_cycles es.pending_call := NoPendingCall - + ic0.stable_size() : (page_count : i32) = if |es.wasm_state.store.mem| > 2^32 then Trap {cycles_used = es.cycles_used;} page_count := |es.wasm_state.stable_mem| / 64k return page_count - + ic0.stable_grow(new_pages : i32) : (old_page_count : i32) = if |es.wasm_state.store.mem| > 2^32 then Trap {cycles_used = es.cycles_used;} if arbitrary() then return -1 @@ -5525,24 +5557,24 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im es.wasm_state.stable_mem := es.wasm_state.stable_mem · repeat(0x00, new_pages * 64k) return old_size - + ic0.stable_write(offset : i32, src : i32, size : i32) if |es.wasm_state.store.mem| > 2^32 then Trap {cycles_used = es.cycles_used;} if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} - + es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] - + ic0.stable_read(dst : i32, offset : i32, size : i32) if |es.wasm_state.store.mem| > 2^32 then Trap {cycles_used = es.cycles_used;} if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} - + es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] - + ic0.stable64_size() : (page_count : i64) = return |es.wasm_state.stable_mem| / 64k - + ic0.stable64_grow(new_pages : i64) : (old_page_count : i64) = if arbitrary() then return -1 @@ -5551,43 +5583,43 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im es.wasm_state.stable_mem := es.wasm_state.stable_mem · repeat(0x00, new_pages * 64k) return old_size - + ic0.stable64_write(offset : i64, src : i64, size : i64) if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} - + es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] - + ic0.stable64_read(dst : i64, offset : i64, size : i64) if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} - + es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] - + ic0.certified_data_set(src: i32, size: i32) = if es.context ∉ {I, G, U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} es.new_certified_data := es.wasm_state[src..src+size] - + ic0.data_certificate_present() : i32 = if es.context = s then Trap {cycles_used = es.cycles_used;} if es.params.sysenv.certificate = NoCertificate then return 0 else return 1 - + ic0.data_certificate_size() : i32 = if es.context = s then Trap {cycles_used = es.cycles_used;} if es.params.sysenv.certificate = NoCertificate then Trap {cycles_used = es.cycles_used;} return |es.params.sysenv.certificate| - + ic0.data_certificate_copy(dst: i32, offset: i32, size: i32) = if es.context = s then Trap {cycles_used = es.cycles_used;} if es.params.sysenv.certificate = NoCertificate then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.sysenv.certificate) - + ic0.time() : i32 = if es.context = s then Trap {cycles_used = es.cycles_used;} return es.params.sysenv.time - + ic0.global_timer_set(timestamp: i64) : i64 = if es.context ∉ {I, G, U, Ry, Rt, C, T} then Trap {cycles_used = es.cycles_used;} let prev_global_timer = es.new_global_timer @@ -5595,10 +5627,10 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im if prev_global_timer = NoGlobalTimer then return es.params.sysenv.global_timer else return prev_global_timer - + ic0.performance_counter(counter_type : i32) : i64 = arbitrary() - + ic0.is_controller(src: i32, size: i32) : (result: i32) = bytes = copy_from_canister(src, size) if bytes encode a principal then @@ -5607,10 +5639,10 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im else return 1 else Trap {cycles_used = es.cycles_used;} - + ic0.debug_print(src : i32, size : i32) = return - + ic0.trap(src : i32, size : i32) = Trap {cycles_used = es.cycles_used;} From 835c1d949eb92f6f5e0c2057647597fecbe182d3 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Mon, 21 Aug 2023 21:53:57 +0200 Subject: [PATCH 074/102] list stored chunks --- spec/index.md | 36 ++++++++++++++++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-) diff --git a/spec/index.md b/spec/index.md index a3405405e..4ab7d1b8b 100644 --- a/spec/index.md +++ b/spec/index.md @@ -3642,6 +3642,8 @@ M.arg = candid(A) chunk_store_size = |{x | chunk_store[A.canister_id][x] not null}| chunk_store_size < MAX_CHUNK_STORE_SIZE and (M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id}) hash = SHA-256(A.chunk) +M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id} + ``` @@ -3670,6 +3672,7 @@ S.messages = Older_messages · CallMessage M · Younger_messages (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) M.method_name = 'clear_store' M.arg = candid(A) +M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id} ``` State after @@ -3697,8 +3700,9 @@ S.messages = Older_messages · CallMessage M · Younger_messages (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) M.method_name = 'delete_chunks' M.arg = candid(A) -chunk_store'(hash) = chunk_store[M.canister_id](hash) if hash ∉ A.hash_list -chunk_store'(hash) = null if hash ∈ A.hash_list +chunk_store'[hash] = chunk_store[A.canister_id][hash] if hash ∉ A.hash_list +chunk_store'[hash] = null if hash ∈ A.hash_list +M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id} ``` @@ -3716,6 +3720,34 @@ S with ``` +#### IC Management Canister: List stored chunks + +The controller of a canister, or the canister itself can list the hashes of the chunks stored in the chunk store. + +```html + +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.method_name = 'stored_chunks' +M.arg = candid(A) +stored_chunks = {hash | chunk_store[A.canister_id][hash] ≠ null} +M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id} + +``` + +State after + +```html + +S with + chunk_store[A.canister_id] = chunk_store' + messages = Older_messages · Younger_messages · + ResponseMessage { + origin = M.origin + response = candid(stored_chunks) + } + +``` From 9ed5c63e53fbbe2748771ac0bf87963c4077abcd Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Tue, 22 Aug 2023 10:49:32 +0200 Subject: [PATCH 075/102] Apply suggestions from code review Thanks Martin! Co-authored-by: mraszyk <31483726+mraszyk@users.noreply.github.com> --- spec/index.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/spec/index.md b/spec/index.md index 4ab7d1b8b..70b820eb2 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1834,7 +1834,7 @@ The optional `sender_canister_version` parameter can contain the caller's canist ### IC method `upload_chunk` {#ic-upload_chunk} -Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister to upload such chunks. The method returns the hash of the chunk that was stored. The size of each chunk should be up to 1MB. The size of the chunk store is bounded: currently it can hold up to 40chunks. +Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister to upload such chunks. The method returns the hash of the chunk that was stored. The size of each chunk should be up to 1MB. The size of the chunk store is bounded: currently it can hold up to 40 chunks. ### IC method `delete_chunks` {#ic-delete_chunks} Canister controllers can delete chunks stored in the canister's chunk storage. The caller provides as input a list of hashes; the effect of the call is that the IC deletes from the canister's chunk storage the chunks that correspond to hashes in the provided list. @@ -1870,7 +1870,7 @@ The `wasm_module` field specifies the canister module to be installed. The syste - If the `wasm_module` starts with byte sequence `[0x1f, 0x8b, 0x08]`, the system parses `wasm_module` as a gzip-compressed WebAssembly binary. -- If the `wasm_module` starts with the byte sequence `0xd9 0xd9 0xf7 `, the system parses `wasm_module` as a self describing CBOR encoding of a map that specifies: an optional canister identifier `storage_canister` and a list of hash values, `[h0,h1,...,hk]`. In this case, the system looks up in the chunk store of `storage_canister` (or that of the target canister if this parameter is not provided) blobs corresponding to `h1,...,hk`, concatenates them to obtain a blob of bytes and checks that `h0` is the hash of the resulting blob. The caller must be the controller of the `storage_canister`. If the lookup succeeds, then the system interprets the blob as a vanilla wasm module (or a gzipped one) per the rules above. +- If the `wasm_module` starts with the byte sequence `[0xd9, 0xd9, 0xf7]`, the system parses `wasm_module` as a self describing CBOR encoding of a map that specifies: an optional canister identifier `storage_canister` and a list of hash values, `[h0,h1,...,hk]`. In this case, the system looks up in the chunk store of `storage_canister` (or that of the target canister if this parameter is not provided) blobs corresponding to `h1,...,hk`, concatenates them to obtain a blob of bytes and checks that `h0` is the hash of the resulting blob. The caller must be a controller of the `storage_canister`. If the lookup succeeds, then the system interprets the blob as a vanilla wasm module (or a gzipped one) per the rules above. The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. @@ -3629,7 +3629,7 @@ S with #### IC Management Canister: Upload Chunk -The controller of a canister, or the canister itself can upload chunks to the chunk store of that canister. +A controller of a canister, or the canister itself can upload chunks to the chunk store of that canister. Conditions From d7abc8647220f0931d346658a9dadd2f967cca7d Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Wed, 23 Aug 2023 10:20:00 +0200 Subject: [PATCH 076/102] added parameteres for no of chunks in large wasms --- spec/index.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/spec/index.md b/spec/index.md index 70b820eb2..6b6608fca 100644 --- a/spec/index.md +++ b/spec/index.md @@ -106,10 +106,14 @@ This specification may refer to certain constants and limits without specifying Maximum amount of cycles that can be used in total (across all calls to query and composite query methods and their callbacks) during evaluation of a query call. -- `MAX_CHUNK_STORE_SIZE` +- `CHUNK_STORE_SIZE` Maximum number of chunks that can be stored within the chunk store of a canister. +- `MAX_CHUNKS_IN_LARGE_WASM` + + Maximum number of chunks that can comprise a large Wasm module. + - `DEFAULT_PROVISIONAL_CYCLES_BALANCE` Amount of cycles allocated to a new canister by default, if not explicitly specified. See [IC method](#ic-provisional_create_canister_with_cycles). @@ -1026,7 +1030,7 @@ Applications can work around these problems. For the first problem, the query re ## Canister module format {#canister-module-format} -A canister module is a [WebAssembly module](https://webassembly.github.io/spec/core/index.html) that is either in binary format (typically `.wasm`) or gzip-compressed (typically `.wasm.gz`). If the module starts with byte sequence `[0x1f, 0x8b, 0x08]`, then the system decompresses the contents as a gzip stream according to [RFC-1952](https://datatracker.ietf.org/doc/html/rfc1952.html) and then parses the output as a WebAssembly binary. +A canister module is a [WebAssembly module](https://webassembly.github.io/spec/core/index.html) that is either in binary format (typically `.wasm`), gzip-compressed (typically `.wasm.gz`), or specified through a list of hashes. If the module starts with byte sequence `[0x1f, 0x8b, 0x08]`, then the system decompresses the contents as a gzip stream according to [RFC-1952](https://datatracker.ietf.org/doc/html/rfc1952.html) and then parses the output as a WebAssembly binary. If the module starts with `[0xd9, 0xd9, 0xf7]` then the system interprets the content as a list of hashes. In this case, the actual Wasm module is obtained by concatenating the individual bitstrings that correspond to these hashes. ## Canister interface (System API) {#system-api} @@ -1834,7 +1838,7 @@ The optional `sender_canister_version` parameter can contain the caller's canist ### IC method `upload_chunk` {#ic-upload_chunk} -Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister to upload such chunks. The method returns the hash of the chunk that was stored. The size of each chunk should be up to 1MB. The size of the chunk store is bounded: currently it can hold up to 40 chunks. +Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister to upload such chunks. The method returns the hash of the chunk that was stored. The size of each chunk should be up to 1MB. The size of the chunk store is bounded: currently it can hold up to `CHUNK_STORE_SIZE` chunks. ### IC method `delete_chunks` {#ic-delete_chunks} Canister controllers can delete chunks stored in the canister's chunk storage. The caller provides as input a list of hashes; the effect of the call is that the IC deletes from the canister's chunk storage the chunks that correspond to hashes in the provided list. @@ -1870,7 +1874,7 @@ The `wasm_module` field specifies the canister module to be installed. The syste - If the `wasm_module` starts with byte sequence `[0x1f, 0x8b, 0x08]`, the system parses `wasm_module` as a gzip-compressed WebAssembly binary. -- If the `wasm_module` starts with the byte sequence `[0xd9, 0xd9, 0xf7]`, the system parses `wasm_module` as a self describing CBOR encoding of a map that specifies: an optional canister identifier `storage_canister` and a list of hash values, `[h0,h1,...,hk]`. In this case, the system looks up in the chunk store of `storage_canister` (or that of the target canister if this parameter is not provided) blobs corresponding to `h1,...,hk`, concatenates them to obtain a blob of bytes and checks that `h0` is the hash of the resulting blob. The caller must be a controller of the `storage_canister`. If the lookup succeeds, then the system interprets the blob as a vanilla wasm module (or a gzipped one) per the rules above. +- If the `wasm_module` starts with the byte sequence `[0xd9, 0xd9, 0xf7]`, the system parses `wasm_module` as a self describing CBOR encoding of a map that specifies: an optional canister identifier `storage_canister` and a list of hash values, `[h0,h1,...,hk]` with `k <= MAX_CHUNKS_IN_LARGE_WASM`. In this case, the system looks up in the chunk store of `storage_canister` (or that of the target canister if this parameter is not provided) blobs corresponding to `h1,...,hk`, concatenates them to obtain a blob of bytes and checks that `h0` is the hash of the resulting blob. The caller must be a controller of the `storage_canister`. If the lookup succeeds, then the system interprets the blob as a vanilla wasm module (or a gzipped one) per the rules above. The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. @@ -3640,7 +3644,7 @@ S.messages = Older_messages · CallMessage M · Younger_messages M.method_name = 'upload_chunk' M.arg = candid(A) chunk_store_size = |{x | chunk_store[A.canister_id][x] not null}| -chunk_store_size < MAX_CHUNK_STORE_SIZE and (M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id}) +chunk_store_size < CHUNK_STORE_SIZE and (M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id}) hash = SHA-256(A.chunk) M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id} From 6376d78694fd525d1ca327cf144d87f261b58c14 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Thu, 24 Aug 2023 10:18:16 +0200 Subject: [PATCH 077/102] Update spec/index.md Co-authored-by: mraszyk <31483726+mraszyk@users.noreply.github.com> --- spec/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/index.md b/spec/index.md index 6b6608fca..d586b221f 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1841,7 +1841,7 @@ The optional `sender_canister_version` parameter can contain the caller's canist Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister to upload such chunks. The method returns the hash of the chunk that was stored. The size of each chunk should be up to 1MB. The size of the chunk store is bounded: currently it can hold up to `CHUNK_STORE_SIZE` chunks. ### IC method `delete_chunks` {#ic-delete_chunks} -Canister controllers can delete chunks stored in the canister's chunk storage. The caller provides as input a list of hashes; the effect of the call is that the IC deletes from the canister's chunk storage the chunks that correspond to hashes in the provided list. +Canister controllers can delete chunks stored in the canister's chunk storage. The caller provides as input a list of hashes; the effect of the call is that the IC deletes from the canister's chunk storage the chunks whose hashes are contained in the provided list. ### IC method `clear_store` {#ic-clear_store} Canister controllers can clear the entire chunk storage of a canister. From f24431930fe6cfa1265f2036153f8bc9e2d15f9d Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Thu, 24 Aug 2023 10:44:41 +0200 Subject: [PATCH 078/102] Update spec/index.md Co-authored-by: mraszyk <31483726+mraszyk@users.noreply.github.com> --- spec/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/index.md b/spec/index.md index d586b221f..3ea49d5ae 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1838,7 +1838,7 @@ The optional `sender_canister_version` parameter can contain the caller's canist ### IC method `upload_chunk` {#ic-upload_chunk} -Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister to upload such chunks. The method returns the hash of the chunk that was stored. The size of each chunk should be up to 1MB. The size of the chunk store is bounded: currently it can hold up to `CHUNK_STORE_SIZE` chunks. +Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister to upload such chunks. The method returns the hash of the chunk that was stored. The size of each chunk should be up to 1MiB. The size of the chunk store is bounded: currently it can hold up to `CHUNK_STORE_SIZE` chunks. ### IC method `delete_chunks` {#ic-delete_chunks} Canister controllers can delete chunks stored in the canister's chunk storage. The caller provides as input a list of hashes; the effect of the call is that the IC deletes from the canister's chunk storage the chunks whose hashes are contained in the provided list. From 764f52ff40cc5fc0ddaa37a641c8e4b0ac0563af Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Thu, 24 Aug 2023 10:45:47 +0200 Subject: [PATCH 079/102] Update spec/index.md Co-authored-by: mraszyk <31483726+mraszyk@users.noreply.github.com> --- spec/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/index.md b/spec/index.md index 3ea49d5ae..bd8910385 100644 --- a/spec/index.md +++ b/spec/index.md @@ -3643,7 +3643,7 @@ S.messages = Older_messages · CallMessage M · Younger_messages (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) M.method_name = 'upload_chunk' M.arg = candid(A) -chunk_store_size = |{x | chunk_store[A.canister_id][x] not null}| +chunk_store_size = |dom(S.chunk_store[A.canister_id])| chunk_store_size < CHUNK_STORE_SIZE and (M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id}) hash = SHA-256(A.chunk) M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id} From b48e22e1d5a23ae389b4d84bc80191d4d4b43111 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Thu, 24 Aug 2023 10:47:16 +0200 Subject: [PATCH 080/102] Update spec/index.md Co-authored-by: mraszyk <31483726+mraszyk@users.noreply.github.com> --- spec/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/index.md b/spec/index.md index bd8910385..bc9f9b06b 100644 --- a/spec/index.md +++ b/spec/index.md @@ -3715,7 +3715,7 @@ State after ```html S with - chunk_store[A.canister_id] = chunk_store' + chunk_store[A.canister_id](hash) = (deleted) for all hash ∈ A.hash_list messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin From c9f08f26f23989191dc7ded65e8aee80128bd01e Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Thu, 24 Aug 2023 10:48:58 +0200 Subject: [PATCH 081/102] Update spec/index.md Co-authored-by: mraszyk <31483726+mraszyk@users.noreply.github.com> --- spec/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/index.md b/spec/index.md index bc9f9b06b..1c1cb2bd0 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1030,7 +1030,7 @@ Applications can work around these problems. For the first problem, the query re ## Canister module format {#canister-module-format} -A canister module is a [WebAssembly module](https://webassembly.github.io/spec/core/index.html) that is either in binary format (typically `.wasm`), gzip-compressed (typically `.wasm.gz`), or specified through a list of hashes. If the module starts with byte sequence `[0x1f, 0x8b, 0x08]`, then the system decompresses the contents as a gzip stream according to [RFC-1952](https://datatracker.ietf.org/doc/html/rfc1952.html) and then parses the output as a WebAssembly binary. If the module starts with `[0xd9, 0xd9, 0xf7]` then the system interprets the content as a list of hashes. In this case, the actual Wasm module is obtained by concatenating the individual bitstrings that correspond to these hashes. +A canister module is a [WebAssembly module](https://webassembly.github.io/spec/core/index.html) that is either in binary format (typically `.wasm`), gzip-compressed (typically `.wasm.gz`), or specified through a list of hashes. If the module starts with byte sequence `[0x1f, 0x8b, 0x08]`, then the system decompresses the contents as a gzip stream according to [RFC-1952](https://datatracker.ietf.org/doc/html/rfc1952.html) and then parses the output as a WebAssembly binary. If the module starts with the byte sequence `[0xd9, 0xd9, 0xf7]` (self-describe CBOR tag), then the system interprets the content as a CBOR (see [CBOR](#cbor)) value according to the following [CDDL](_attachments/chunked-wasms.cddl) (see [CDDL](#cddl)). In this case, the actual Wasm module is obtained by concatenating the individual chunks (from the chunk store of the installed canister or from the chunk store of a specified storage canister) that correspond to these hashes (skipping the first hash which is the hash of the overall concatenated Wasm module). ## Canister interface (System API) {#system-api} From 2d540a34a86b3cbe44469c622aa37b512920fd85 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Thu, 24 Aug 2023 10:49:34 +0200 Subject: [PATCH 082/102] Update spec/index.md Co-authored-by: mraszyk <31483726+mraszyk@users.noreply.github.com> --- spec/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/index.md b/spec/index.md index 1c1cb2bd0..afa146493 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1874,7 +1874,7 @@ The `wasm_module` field specifies the canister module to be installed. The syste - If the `wasm_module` starts with byte sequence `[0x1f, 0x8b, 0x08]`, the system parses `wasm_module` as a gzip-compressed WebAssembly binary. -- If the `wasm_module` starts with the byte sequence `[0xd9, 0xd9, 0xf7]`, the system parses `wasm_module` as a self describing CBOR encoding of a map that specifies: an optional canister identifier `storage_canister` and a list of hash values, `[h0,h1,...,hk]` with `k <= MAX_CHUNKS_IN_LARGE_WASM`. In this case, the system looks up in the chunk store of `storage_canister` (or that of the target canister if this parameter is not provided) blobs corresponding to `h1,...,hk`, concatenates them to obtain a blob of bytes and checks that `h0` is the hash of the resulting blob. The caller must be a controller of the `storage_canister`. If the lookup succeeds, then the system interprets the blob as a vanilla wasm module (or a gzipped one) per the rules above. +- If the `wasm_module` starts with the byte sequence `[0xd9, 0xd9, 0xf7]`, the system parses `wasm_module` as a self describing CBOR encoding of a map that specifies: an optional canister identifier `storage_canister` and a list of hash values, `[h0,h1,...,hk]` with `k <= MAX_CHUNKS_IN_LARGE_WASM`. In this case, the system looks up in the chunk store of `storage_canister` (or that of the target canister if this parameter is not provided) blobs corresponding to `h1,...,hk`, concatenates them to obtain a blob of bytes and checks that `h0` is the hash of the resulting blob. The caller must be a controller of the `storage_canister` or the caller must be the `storage_canister`. If the lookup succeeds, then the system interprets the blob as a vanilla wasm module (or a gzipped one) per the rules above. The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. From 0fedf820adcb8635cbab2b8c128be88d1a750efd Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Thu, 24 Aug 2023 13:41:59 +0200 Subject: [PATCH 083/102] implement Martin's suggestions from PR review --- spec/index.md | 19 ++++++------------- 1 file changed, 6 insertions(+), 13 deletions(-) diff --git a/spec/index.md b/spec/index.md index afa146493..fefa5c441 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1838,13 +1838,13 @@ The optional `sender_canister_version` parameter can contain the caller's canist ### IC method `upload_chunk` {#ic-upload_chunk} -Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister to upload such chunks. The method returns the hash of the chunk that was stored. The size of each chunk should be up to 1MiB. The size of the chunk store is bounded: currently it can hold up to `CHUNK_STORE_SIZE` chunks. +Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister (and the canister itself) to upload such chunks. The method returns the hash of the chunk that was stored. The size of each chunk should be up to 1MiB. The size of the chunk store is bounded: currently it can hold up to `CHUNK_STORE_SIZE` chunks. ### IC method `delete_chunks` {#ic-delete_chunks} -Canister controllers can delete chunks stored in the canister's chunk storage. The caller provides as input a list of hashes; the effect of the call is that the IC deletes from the canister's chunk storage the chunks whose hashes are contained in the provided list. +Canister controllers (and the canister itself) can delete chunks stored in the canister's chunk storage. The caller provides as input a list of hashes; the effect of the call is that the IC deletes from the canister's chunk storage the chunks whose hashes are contained in the provided list. ### IC method `clear_store` {#ic-clear_store} -Canister controllers can clear the entire chunk storage of a canister. +Canister controllers (and the canister itself) can clear the entire chunk storage of a canister. ### IC method `install_code` {#ic-install_code} @@ -3643,9 +3643,7 @@ S.messages = Older_messages · CallMessage M · Younger_messages (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) M.method_name = 'upload_chunk' M.arg = candid(A) -chunk_store_size = |dom(S.chunk_store[A.canister_id])| -chunk_store_size < CHUNK_STORE_SIZE and (M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id}) -hash = SHA-256(A.chunk) +|dom(S.chunk_store[A.canister_id]) ∪ {SHA-256(A.chunk}| <= CHUNK_STORE_SI M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id} @@ -3656,8 +3654,7 @@ State after ```html S with - chunk_store[A.canister_id](hash') = chunk_store[A.canister_id] if hash'≠ hash - chunk_store[A.canister_id](hash) = A.chunk + chunk_store[A.canister_id](SHA-256(A.chunk)) = A.chunk messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin @@ -3704,8 +3701,6 @@ S.messages = Older_messages · CallMessage M · Younger_messages (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) M.method_name = 'delete_chunks' M.arg = candid(A) -chunk_store'[hash] = chunk_store[A.canister_id][hash] if hash ∉ A.hash_list -chunk_store'[hash] = null if hash ∈ A.hash_list M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id} ``` @@ -3734,7 +3729,6 @@ S.messages = Older_messages · CallMessage M · Younger_messages (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) M.method_name = 'stored_chunks' M.arg = candid(A) -stored_chunks = {hash | chunk_store[A.canister_id][hash] ≠ null} M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id} ``` @@ -3744,11 +3738,10 @@ State after ```html S with - chunk_store[A.canister_id] = chunk_store' messages = Older_messages · Younger_messages · ResponseMessage { origin = M.origin - response = candid(stored_chunks) + response = candid(dom(S.chunk_store[A.canister_id])) } ``` From b86f46e569d2687c9818b61145a2c36a4bb2f21b Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Thu, 24 Aug 2023 16:42:18 +0200 Subject: [PATCH 084/102] fix to .cddl --- spec/_attachments/chunked-wasms.cddl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/_attachments/chunked-wasms.cddl b/spec/_attachments/chunked-wasms.cddl index 479941979..f75c9bd8f 100644 --- a/spec/_attachments/chunked-wasms.cddl +++ b/spec/_attachments/chunked-wasms.cddl @@ -4,7 +4,7 @@ chunked-wasm = tagged<{ - ? location : principal + ? storage_canister : principal list_of_hashes : [* hash] }> From b78c75e08b97b6e960278b5c7daf10d4e43181d5 Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Fri, 25 Aug 2023 11:44:43 +0200 Subject: [PATCH 085/102] drop whitespaces on empty lines --- spec/index.md | 188 +++++++++++++++++++++++++------------------------- 1 file changed, 94 insertions(+), 94 deletions(-) diff --git a/spec/index.md b/spec/index.md index 41be98565..1f85ecd2e 100644 --- a/spec/index.md +++ b/spec/index.md @@ -207,7 +207,7 @@ Example encoding from hex, and decoding to hex, in bash (the following can be pa xxd -r -p | base32 | tr A-Z a-z | tr -d = | fold -w5 | paste -sd'-' - } - + function textual_decode() { echo -n "$1" | tr -d - | tr a-z A-Z | fold -w 8 | xargs -n1 printf '%-8s' | tr ' ' = | @@ -1191,11 +1191,11 @@ The following sections describe various System API functions, also referred to a ic0.msg_reject_code : () -> i32; // Ry Rt CRy CRt ic0.msg_reject_msg_size : () -> i32; // Rt CRt ic0.msg_reject_msg_copy : (dst : i32, offset : i32, size : i32) -> (); // Rt CRt - + ic0.msg_reply_data_append : (src : i32, size : i32) -> (); // U Q CQ Ry Rt CRy CRt ic0.msg_reply : () -> (); // U Q CQ Ry Rt CRy CRt ic0.msg_reject : (src : i32, size : i32) -> (); // U Q CQ Ry Rt CRy CRt - + ic0.msg_cycles_available : () -> i64; // U Rt Ry ic0.msg_cycles_available128 : (dst : i32) -> (); // U Rt Ry ic0.msg_cycles_refunded : () -> i64; // Rt Ry @@ -1203,18 +1203,18 @@ The following sections describe various System API functions, also referred to a ic0.msg_cycles_accept : (max_amount : i64) -> (amount : i64); // U Rt Ry ic0.msg_cycles_accept128 : (max_amount_high : i64, max_amount_low: i64, dst : i32) -> (); // U Rt Ry - + ic0.canister_self_size : () -> i32; // * ic0.canister_self_copy : (dst : i32, offset : i32, size : i32) -> (); // * ic0.canister_cycle_balance : () -> i64; // * ic0.canister_cycle_balance128 : (dst : i32) -> (); // * ic0.canister_status : () -> i32; // * ic0.canister_version : () -> i64; // * - + ic0.msg_method_name_size : () -> i32; // F ic0.msg_method_name_copy : (dst : i32, offset : i32, size : i32) -> (); // F ic0.accept_message : () -> (); // F - + ic0.call_new : // U CQ Ry Rt CRy CRt T ( callee_src : i32, callee_size : i32, @@ -1230,7 +1230,7 @@ The following sections describe various System API functions, also referred to a ic0.call_cycles_add : (amount : i64) -> (); // U Ry Rt T ic0.call_cycles_add128 : (amount_high : i64, amount_low: i64) -> (); // U Ry Rt T ic0.call_perform : () -> ( err_code : i32 ); // U CQ Ry Rt CRy CRt T - + ic0.stable_size : () -> (page_count : i32); // * s ic0.stable_grow : (new_pages : i32) -> (old_page_count : i32); // * s ic0.stable_write : (offset : i32, src : i32, size : i32) -> (); // * s @@ -1239,17 +1239,17 @@ The following sections describe various System API functions, also referred to a ic0.stable64_grow : (new_pages : i64) -> (old_page_count : i64); // * s ic0.stable64_write : (offset : i64, src : i64, size : i64) -> (); // * s ic0.stable64_read : (dst : i64, offset : i64, size : i64) -> (); // * s - + ic0.certified_data_set : (src: i32, size: i32) -> (); // I G U Ry Rt T ic0.data_certificate_present : () -> i32; // * ic0.data_certificate_size : () -> i32; // * ic0.data_certificate_copy : (dst: i32, offset: i32, size: i32) -> (); // * - + ic0.time : () -> (timestamp : i64); // * ic0.global_timer_set : (timestamp : i64) -> i64; // I G U Ry Rt C T ic0.performance_counter : (counter_type : i32) -> (counter : i64); // * s ic0.is_controller: (src: i32, size: i32) -> ( result: i32); // * s - + ic0.debug_print : (src : i32, size : i32) -> (); // * s ic0.trap : (src : i32, size : i32) -> (); // * s @@ -2187,13 +2187,13 @@ A certificate is validated with regard to the root of trust by the following alg let der_key = check_delegation(cert.delegation) // see section Delegations below bls_key = extract_der(der_key) verify_bls_signature(bls_key, cert.signature, domain_sep("ic-state-root") · root_hash) - + reconstruct(Empty) = H(domain_sep("ic-hashtree-empty")) reconstruct(Fork t1 t2) = H(domain_sep("ic-hashtree-fork") · reconstruct(t1) · reconstruct(t2)) reconstruct(Labeled l t) = H(domain_sep("ic-hashtree-labeled") · l · reconstruct(t)) reconstruct(Leaf v) = H(domain_sep("ic-hashtree-leaf") · v) reconstruct(Pruned h) = h - + domain_sep(s) = byte(|s|) · s where `H` is the SHA-256 hash function, @@ -2257,7 +2257,7 @@ The IC will only produce well-formed state trees, and the above algorithm assume well_formed(tree) = (tree = Leaf _) ∨ (well_formed_forest(flatten_forks(tree))) - + well_formed_forest(trees) = strictly_increasing([l | Label l _ ∈ trees]) ∧ ∀ Label _ t ∈ trees. well_formed(t) ∧ @@ -2456,7 +2456,7 @@ The [WebAssembly System API](#system-api) is relatively low-level, and some of i Arg = Blob; CallerId = Principal; - + Timestamp = Nat; CanisterVersion = Nat; Env = { @@ -2469,7 +2469,7 @@ The [WebAssembly System API](#system-api) is relatively low-level, and some of i status : Running | Stopping | Stopped; canister_version : CanisterVersion; } - + RejectCode = Nat Response = Reply Blob | Reject (RejectCode, Text) MethodCall = { @@ -2479,7 +2479,7 @@ The [WebAssembly System API](#system-api) is relatively low-level, and some of i transferred_cycles: Nat; callback: Callback; } - + UpdateFunc = WasmState -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; new_calls : List MethodCall; @@ -2506,10 +2506,10 @@ The [WebAssembly System API](#system-api) is relatively low-level, and some of i new_global_timer : NoGlobalTimer | Nat; cycles_used : Nat; } - + AvailableCycles = Nat RefundedCycles = Nat - + CanisterModule = { init : (CanisterId, Arg, CallerId, Env) -> Trap { cycles_used : Nat; } | Return { new_state : WasmState; @@ -2598,7 +2598,7 @@ Therefore, a message can have different shapes: | Callback Callback Response RefundedCycles | Heartbeat | GlobalTimer - + Message = CallMessage { origin : CallOrigin; @@ -2665,7 +2665,7 @@ Signed delegations contain the (unsigned) delegation data in a nested record, ne For the signatures in a `Request`, we assume that the following function implements signature verification as described in [Authentication](#authentication). This function picks the corresponding signature scheme according to the DER-encoded metadata in the public key. verify_signature : PublicKey -> Signature -> Blob -> Bool - + Envelope = { content : Request | APIReadRequest; sender_pubkey : PublicKey | NoPublicKey; @@ -2863,19 +2863,19 @@ The following predicate describes when an envelope `E` correctly signs the enclo = TS if U = mk_self_authenticating_id E.sender_pubkey ∧ (PK', TS) = verify_delegations(DS, PK, T, { p : p is CanisterId }, U) ∧ verify_signature PK' Sig ("\x0Aic-request" · hash_of_map(C)) - + verify_delegations([], PK, T, TS, U) = (PK, TS) verify_delegations([D] · DS, PK, T, TS, U) = verify_delegations(DS, D.pubkey, T, TS ∩ delegation_targets(D), U) if verify_signature PK D.signature ("\x1Aic-request-auth-delegation" · hash_of_map(D.delegation)) ∧ D.delegation.expiration ≥ T ∧ U ∈ delegated_senders(D) - + delegation_targets(D) = if D.targets = Unrestricted then { p : p is CanisterId } else D.targets - + delegated_senders(D) = if D.senders = Unrestricted then { p : p is Principal } @@ -3334,7 +3334,7 @@ The functions `query_as_update` and `system_task_as_update` turns a query functi cycles_accepted = 0; cycles_used = res.cycles_used; } - + system_task_as_update(f, env) = λ wasm_state → match f(env)(wasm_state) with Trap trap → Trap trap @@ -4853,7 +4853,7 @@ where `state_tree` constructs a labeled tree from the IC state `S` and the (so f { "metadata": { name: blob | (name, blob) ∈ S.canisters[canister_id].public_custom_sections ∪ S.canisters[canister_id].private_custom_sections } } | (canister_id, C) ∈ S.canisters }; } - + request_status_tree(Received) = { "status": "received" } request_status_tree(Processing) = @@ -4891,7 +4891,7 @@ The abstract `Callback` type above models an entry point for responses: fun : i32, env : i32, } - + Callback = { on_reply : Closure; on_reject : Closure; @@ -4947,7 +4947,7 @@ It is nonsensical to pass to an execution function a WebAssembly store `S` that cycles_refunded = 0; method_name = NoText; } - + empty_execution_state = { wasm_state = (undefined); params = (undefined); @@ -5221,7 +5221,7 @@ global_timer = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} if fun > |es.wasm_state.store.table| then Trap let func = es.wasm_state.store.table[fun] if typeof(func) ≠ func (i32) -> () then Trap - + func(env) Return { new_state = es.wasm_state; @@ -5237,7 +5237,7 @@ global_timer = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} if callbacks.on_cleanup.fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} let func = es.wasm_state.store.table[callbacks.on_cleanup.fun] if typeof(func) ≠ func (i32) -> () then Trap {cycles_used = es.cycles_used;} - + let es' = ref { empty_execution_state with wasm_state = wasm_state; context = C; @@ -5278,7 +5278,7 @@ global_timer = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} if fun > |es.wasm_state.store.table| then Trap let func = es.wasm_state.store.table[fun] if typeof(func) ≠ func (i32) -> () then Trap - + func(env) Return { new_state = es.wasm_state; @@ -5291,7 +5291,7 @@ global_timer = λ (sysenv) → λ wasm_state → Trap {cycles_used = 0;} if callbacks.on_cleanup.fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} let func = es.wasm_state.store.table[callbacks.on_cleanup.fun] if typeof(func) ≠ func (i32) -> () then Trap {cycles_used = es.cycles_used;} - + let es' = ref { empty_execution_state with wasm_state = wasm_state; context = CC; @@ -5339,7 +5339,7 @@ In the following section, we use the these helper functions if offset+size > |data| then Trap {cycles_used = es.cycles_used;} if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} es.wasm_state.store.mem[dst..dst+size] := data[offset..offset+size] - + copy_from_canister(src : i32, size : i32) blob = if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} return es.wasm_state.store.mem[src..src+size] @@ -5360,68 +5360,68 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im ic0.msg_arg_data_size() : i32 = if es.context ∉ {I, U, Q, CQ, Ry, CRy, F} then Trap {cycles_used = es.cycles_used;} return |es.params.arg| - + ic0.msg_arg_data_copy(dst:i32, offset:i32, size:i32) = if es.context ∉ {I, U, Q, CQ, Ry, CRy, F} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.arg) - + ic0.msg_caller_size() : i32 = if es.context = s then Trap {cycles_used = es.cycles_used;} return |es.params.caller| - + ic0.msg_caller_copy(dst:i32, offset:i32, size:i32) : i32 = if es.context = s then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.caller) - + ic0.msg_reject_code() : i32 = if es.context ∉ {Ry, Rt, CRy, CRt} then Trap {cycles_used = es.cycles_used;} es.params.reject_code - + ic0.msg_reject_msg_size() : i32 = if es.context ∉ {Rt, CRt} then Trap {cycles_used = es.cycles_used;} return |es.params.reject_msg| - + ic0.msg_reject_msg_copy(dst:i32, offset:i32, size:i32) : i32 = if es.context ∉ {Rt, CRt} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.reject_msg) - + ic0.msg_reply_data_append(src : i32, size : i32) = if es.context ∉ {U, Q, CQ, Ry, Rt, CRy, CRt} then Trap {cycles_used = es.cycles_used;} if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.reply_params.arg := es.reply_params.arg · copy_from_canister(src, size) - + ic0.msg_reply() = if es.context ∉ {U, Q, CQ, Ry, Rt, CRy, CRt} then Trap {cycles_used = es.cycles_used;} if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.response := Reply (es.reply_params.arg) es.cycles_available := 0 - + ic0.msg_reject(src : i32, size : i32) = if es.context ∉ {U, Q, CQ, Ry, Rt, CRy, CRt} then Trap {cycles_used = es.cycles_used;} if es.response ≠ NoResponse then Trap {cycles_used = es.cycles_used;} es.response := Reject (CANISTER_REJECT, copy_from_canister(src, size)) es.cycles_available := 0 - + ic0.msg_cycles_available() : i64 = if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} if es.cycles_available >= 2^64 then Trap {cycles_used = es.cycles_used;} return es.cycles_available - + ic0.msg_cycles_available128(dst : i32) = if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} let amount = es.cycles_available copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) - + ic0.msg_cycles_refunded() : i64 = if es.context ∉ {Rt, Ry} then Trap {cycles_used = es.cycles_used;} if es.params.cycles_refunded >= 2^64 then Trap {cycles_used = es.cycles_used;} return es.params.cycles_refunded - + ic0.msg_cycles_refunded128(dst : i32) = if es.context ∉ {Rt, Ry} then Trap {cycles_used = es.cycles_used;} let amount = es.params.cycles_refunded copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) - + ic0.msg_cycles_accept(max_amount : i64) : i64 = if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} let amount = min(max_amount, es.cycles_available) @@ -5429,7 +5429,7 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im es.cycles_accepted := es.cycles_accepted + amount es.balance := es.balance + amount return amount - + ic0.msg_cycles_accept128(max_amount_high : i64, max_amount_low : i64, dst : i32) = if es.context ∉ {U, Rt, Ry} then Trap {cycles_used = es.cycles_used;} let max_amount = max_amount_high * 2^64 + max_amount_low @@ -5438,49 +5438,49 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im es.cycles_accepted := es.cycles_accepted + amount es.balance := es.balance + amount copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) - + ic0.canister_self_size() : i32 = if es.context = s then Trap {cycles_used = es.cycles_used;} return |es.wasm_state.self_id| - + ic0.canister_self_copy(dst:i32, offset:i32, size:i32) = if es.context = s then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.wasm_state.self_id) - + ic0.canister_cycle_balance() : i64 = if es.context = s then Trap {cycles_used = es.cycles_used;} if es.balance >= 2^64 then Trap {cycles_used = es.cycles_used;} return es.balance - + ic0.canister_cycles_balance128(dst : i32) = if es.context = s then Trap {cycles_used = es.cycles_used;} let amount = es.balance copy_cycles_to_canister(dst, amount.to_little_endian_bytes()) - + ic0.canister_status() : i32 = if es.context = s then Trap {cycles_used = es.cycles_used;} match es.params.sysenv.canister_status with Running -> return 1 Stopping -> return 2 Stopped -> return 3 - + ic0.canister_version() : i64 = if es.context = s then Trap {cycles_used = es.cycles_used;} return es.params.sysenv.canister_version - + ic0.msg_method_name_size() : i32 = if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} return |es.method_name| - + ic0.msg_method_name_copy(dst : i32, offset : i32, size : i32) : i32 = if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.method_name) - + ic0.accept_message() = if es.context ∉ {F} then Trap {cycles_used = es.cycles_used;} if es.ingress_filter = Accept then Trap {cycles_used = es.cycles_used;} es.ingress_filter = Accept - + ic0.call_new( callee_src : i32, callee_size : i32, @@ -5492,21 +5492,21 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im reject_env : i32, ) = if es.context ∉ {U, CQ, Ry, Rt, CRy, CRt, T} then Trap {cycles_used = es.cycles_used;} - + discard_pending_call() - + if es.balance < MAX_CYCLES_PER_RESPONSE then Trap {cycles_used = es.cycles_used;} es.balance := es.balance - MAX_CYCLES_PER_RESPONSE - + callee := copy_from_canister(callee_src, callee_size); method_name := copy_from_canister(name_src, name_size); - + if reply_fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} if typeof(es.wasm_state.store.table[reply_fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} - + if reject_fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} if typeof(es.wasm_state.store.table[reject_fun]) ≠ func (anyref, i32) -> () then Trap {cycles_used = es.cycles_used;} - + es.pending_call = MethodCall { callee = callee; method_name = callee; @@ -5518,7 +5518,7 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im on_cleanup = NoClosure }; } - + ic0.call_on_cleanup (fun : i32, env : i32) = if es.context ∉ {U, CQ, Ry, Rt, CRy, CRt, T} then Trap {cycles_used = es.cycles_used;} if fun > |es.wasm_state.store.table| then Trap {cycles_used = es.cycles_used;} @@ -5526,35 +5526,35 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} if es.pending_call.callback.on_cleanup ≠ NoClosure then Trap {cycles_used = es.cycles_used;} es.pending_call.callback.on_cleanup := Closure { fun = fun; env = env} - + ic0.call_data_append (src : i32, size : i32) = if es.context ∉ {U, CQ, Ry, Rt, CRy, CRt, T} then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} es.pending_call.arg := es.pending_call.arg · copy_from_canister(src, size) - + ic0.call_cycles_add(amount : i64) = if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} if es.balance < amount then Trap {cycles_used = es.cycles_used;} if es.balance - amount < es.params.sysenv.freezing_limit then Trap {cycles_used = es.cycles_used;} - + es.balance := es.balance - amount es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount - + ic0.call_cycles_add128(amount_high : i64, amount_low : i64) = if es.context ∉ {U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} let amount = amount_high * 2^64 + amount_low if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} if es.balance < amount then Trap {cycles_used = es.cycles_used;} if es.balance - amount < es.params.sysenv.freezing_limit then Trap {cycles_used = es.cycles_used;} - + es.balance := es.balance - amount es.pending_call.transferred_cycles := es.pending_call.transferred_cycles + amount - + ic0.call_peform() : ( err_code : i32 ) = if es.context ∉ {U, CQ, Ry, Rt, CRy, CRt, T} then Trap {cycles_used = es.cycles_used;} if es.pending_call = NoPendingCall then Trap {cycles_used = es.cycles_used;} - + // are we below the threezing threshold? // Or maybe the system has other reasons to not perform this if es.balance < es.params.sysenv.freezing_limit or system_cannot_do_this_call_now() @@ -5565,18 +5565,18 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im es.calls := es.calls · es.pending_call es.pending_call := NoPendingCall return 0 - + // helper function discard_pending_call() = if es.pending_call ≠ NoPendingCall then es.balance := es.balance + MAX_CYCLES_PER_RESPONSE + es.pending_call.transferred_cycles es.pending_call := NoPendingCall - + ic0.stable_size() : (page_count : i32) = if |es.wasm_state.store.mem| > 2^32 then Trap {cycles_used = es.cycles_used;} page_count := |es.wasm_state.stable_mem| / 64k return page_count - + ic0.stable_grow(new_pages : i32) : (old_page_count : i32) = if |es.wasm_state.store.mem| > 2^32 then Trap {cycles_used = es.cycles_used;} if arbitrary() then return -1 @@ -5586,24 +5586,24 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im es.wasm_state.stable_mem := es.wasm_state.stable_mem · repeat(0x00, new_pages * 64k) return old_size - + ic0.stable_write(offset : i32, src : i32, size : i32) if |es.wasm_state.store.mem| > 2^32 then Trap {cycles_used = es.cycles_used;} if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} - + es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] - + ic0.stable_read(dst : i32, offset : i32, size : i32) if |es.wasm_state.store.mem| > 2^32 then Trap {cycles_used = es.cycles_used;} if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} - + es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] - + ic0.stable64_size() : (page_count : i64) = return |es.wasm_state.stable_mem| / 64k - + ic0.stable64_grow(new_pages : i64) : (old_page_count : i64) = if arbitrary() then return -1 @@ -5612,43 +5612,43 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im es.wasm_state.stable_mem := es.wasm_state.stable_mem · repeat(0x00, new_pages * 64k) return old_size - + ic0.stable64_write(offset : i64, src : i64, size : i64) if src+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} - + es.wasm_state.stable_mem[offset..offset+size] := es.wasm_state.store.mem[src..src+size] - + ic0.stable64_read(dst : i64, offset : i64, size : i64) if offset+size > |es.wasm_state.stable_mem| then Trap {cycles_used = es.cycles_used;} if dst+size > |es.wasm_state.store.mem| then Trap {cycles_used = es.cycles_used;} - + es.wasm_state.store.mem[offset..offset+size] := es.wasm_state.stable.mem[src..src+size] - + ic0.certified_data_set(src: i32, size: i32) = if es.context ∉ {I, G, U, Ry, Rt, T} then Trap {cycles_used = es.cycles_used;} es.new_certified_data := es.wasm_state[src..src+size] - + ic0.data_certificate_present() : i32 = if es.context = s then Trap {cycles_used = es.cycles_used;} if es.params.sysenv.certificate = NoCertificate then return 0 else return 1 - + ic0.data_certificate_size() : i32 = if es.context = s then Trap {cycles_used = es.cycles_used;} if es.params.sysenv.certificate = NoCertificate then Trap {cycles_used = es.cycles_used;} return |es.params.sysenv.certificate| - + ic0.data_certificate_copy(dst: i32, offset: i32, size: i32) = if es.context = s then Trap {cycles_used = es.cycles_used;} if es.params.sysenv.certificate = NoCertificate then Trap {cycles_used = es.cycles_used;} copy_to_canister(dst, offset, size, es.params.sysenv.certificate) - + ic0.time() : i32 = if es.context = s then Trap {cycles_used = es.cycles_used;} return es.params.sysenv.time - + ic0.global_timer_set(timestamp: i64) : i64 = if es.context ∉ {I, G, U, Ry, Rt, C, T} then Trap {cycles_used = es.cycles_used;} let prev_global_timer = es.new_global_timer @@ -5656,10 +5656,10 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im if prev_global_timer = NoGlobalTimer then return es.params.sysenv.global_timer else return prev_global_timer - + ic0.performance_counter(counter_type : i32) : i64 = arbitrary() - + ic0.is_controller(src: i32, size: i32) : (result: i32) = bytes = copy_from_canister(src, size) if bytes encode a principal then @@ -5668,10 +5668,10 @@ The pseudo-code below does *not* explicitly enforce the restrictions of which im else return 1 else Trap {cycles_used = es.cycles_used;} - + ic0.debug_print(src : i32, size : i32) = return - + ic0.trap(src : i32, size : i32) = Trap {cycles_used = es.cycles_used;} From 313fdec9646e979d12f8f9172bc1a4732387d11b Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Mon, 28 Aug 2023 10:01:18 +0200 Subject: [PATCH 086/102] fix docusaurus build --- docusaurus/docs/_attachments/chunked-wasms.cddl | 1 + 1 file changed, 1 insertion(+) create mode 120000 docusaurus/docs/_attachments/chunked-wasms.cddl diff --git a/docusaurus/docs/_attachments/chunked-wasms.cddl b/docusaurus/docs/_attachments/chunked-wasms.cddl new file mode 120000 index 000000000..a0aa1d15d --- /dev/null +++ b/docusaurus/docs/_attachments/chunked-wasms.cddl @@ -0,0 +1 @@ +../../../spec/_attachments/chunked-wasms.cddl \ No newline at end of file From 593974cba527c77c510cf9c610eb8e3085fa2e3d Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Mon, 28 Aug 2023 10:01:48 +0200 Subject: [PATCH 087/102] add check for chunked-wasms.cddl into CI --- .github/workflows/check-cddl-candid.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/check-cddl-candid.yml b/.github/workflows/check-cddl-candid.yml index 1dcb6a517..3ccf38b96 100644 --- a/.github/workflows/check-cddl-candid.yml +++ b/.github/workflows/check-cddl-candid.yml @@ -10,6 +10,7 @@ jobs: - name: Check cddl files run: | docker run --rm -v $PWD/spec/_attachments:/workdir ghcr.io/anweiss/cddl-cli:0.9.1 compile-cddl --cddl /workdir/certificates.cddl + docker run --rm -v $PWD/spec/_attachments:/workdir ghcr.io/anweiss/cddl-cli:0.9.1 compile-cddl --cddl /workdir/chunked-wasms.cddl docker run --rm -v $PWD/spec/_attachments:/workdir ghcr.io/anweiss/cddl-cli:0.9.1 compile-cddl --cddl /workdir/requests.cddl - name: Check candid files run: | From baa74530f4f7e0f66001840bde258debde76976e Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Wed, 30 Aug 2023 11:07:38 +0200 Subject: [PATCH 088/102] Update spec/index.md Co-authored-by: mraszyk <31483726+mraszyk@users.noreply.github.com> --- spec/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/index.md b/spec/index.md index 1f85ecd2e..06d7d6dd5 100644 --- a/spec/index.md +++ b/spec/index.md @@ -3643,7 +3643,7 @@ S.messages = Older_messages · CallMessage M · Younger_messages (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) M.method_name = 'upload_chunk' M.arg = candid(A) -|dom(S.chunk_store[A.canister_id]) ∪ {SHA-256(A.chunk}| <= CHUNK_STORE_SI +|dom(S.chunk_store[A.canister_id]) ∪ {SHA-256(A.chunk)}| <= CHUNK_STORE_SIZE M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id} From cb0cbd26f1708e92f6251c784db2d10a21f5700d Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Wed, 13 Sep 2023 13:17:22 +0200 Subject: [PATCH 089/102] details on same subnet & chunk storage cost --- spec/index.md | 35 +++-------------------------------- 1 file changed, 3 insertions(+), 32 deletions(-) diff --git a/spec/index.md b/spec/index.md index af2c5d7a3..0f9263b91 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1838,10 +1838,8 @@ The optional `sender_canister_version` parameter can contain the caller's canist ### IC method `upload_chunk` {#ic-upload_chunk} -Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister (and the canister itself) to upload such chunks. The method returns the hash of the chunk that was stored. The size of each chunk should be up to 1MiB. The size of the chunk store is bounded: currently it can hold up to `CHUNK_STORE_SIZE` chunks. - -### IC method `delete_chunks` {#ic-delete_chunks} -Canister controllers (and the canister itself) can delete chunks stored in the canister's chunk storage. The caller provides as input a list of hashes; the effect of the call is that the IC deletes from the canister's chunk storage the chunks whose hashes are contained in the provided list. +Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister (and the canister itself) to upload such chunks. The method returns the hash of the chunk that was stored. The size of each chunk should be up to 1MiB. The size of the chunk store is bounded: currently it can hold up to `CHUNK_STORE_SIZE` chunks. The storage cost of each chunk is fixed and corresponds to storing 1MiB of data. + ### IC method `clear_store` {#ic-clear_store} Canister controllers (and the canister itself) can clear the entire chunk storage of a canister. @@ -1874,7 +1872,7 @@ The `wasm_module` field specifies the canister module to be installed. The syste - If the `wasm_module` starts with byte sequence `[0x1f, 0x8b, 0x08]`, the system parses `wasm_module` as a gzip-compressed WebAssembly binary. -- If the `wasm_module` starts with the byte sequence `[0xd9, 0xd9, 0xf7]`, the system parses `wasm_module` as a self describing CBOR encoding of a map that specifies: an optional canister identifier `storage_canister` and a list of hash values, `[h0,h1,...,hk]` with `k <= MAX_CHUNKS_IN_LARGE_WASM`. In this case, the system looks up in the chunk store of `storage_canister` (or that of the target canister if this parameter is not provided) blobs corresponding to `h1,...,hk`, concatenates them to obtain a blob of bytes and checks that `h0` is the hash of the resulting blob. The caller must be a controller of the `storage_canister` or the caller must be the `storage_canister`. If the lookup succeeds, then the system interprets the blob as a vanilla wasm module (or a gzipped one) per the rules above. +- If the `wasm_module` starts with the byte sequence `[0xd9, 0xd9, 0xf7]`, the system parses `wasm_module` as a self describing CBOR encoding of a map that specifies: an optional canister identifier `storage_canister` and a list of hash values, `[h0,h1,...,hk]` with `k <= MAX_CHUNKS_IN_LARGE_WASM`. In this case, the system looks up in the chunk store of `storage_canister` (or that of the target canister if this parameter is not provided) blobs corresponding to `h1,...,hk`, concatenates them to obtain a blob of bytes and checks that `h0` is the hash of the resulting blob. The caller must be a controller of the `storage_canister` or the caller must be the `storage_canister` and `storage_canister` must be on the same subnet as the target canister. If the lookup succeeds, then the system interprets the blob as a vanilla wasm module (or a gzipped one) per the rules above. The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. @@ -3696,33 +3694,6 @@ S with ``` -#### IC Management Canister: Delete chunks - -The controller of a canister, or the canister itself can delete chunks from the canister chunk store. - -```html - -S.messages = Older_messages · CallMessage M · Younger_messages -(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) -M.method_name = 'delete_chunks' -M.arg = candid(A) -M.caller ∈ S.controllers[A.canister_id] ∪ {A.canister_id} - -``` - -State after - -```html - -S with - chunk_store[A.canister_id](hash) = (deleted) for all hash ∈ A.hash_list - messages = Older_messages · Younger_messages · - ResponseMessage { - origin = M.origin - response = candid() - } - -``` #### IC Management Canister: List stored chunks From 59670b9e171ade25b8c19412444bc83158460102 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Wed, 20 Sep 2023 15:27:50 +0200 Subject: [PATCH 090/102] removed delete_chunks from ic.did --- spec/_attachments/ic.did | 1 - 1 file changed, 1 deletion(-) diff --git a/spec/_attachments/ic.did b/spec/_attachments/ic.did index ac14a6e30..0ee30f30d 100644 --- a/spec/_attachments/ic.did +++ b/spec/_attachments/ic.did @@ -128,7 +128,6 @@ service ic : { chunk : blob; }) -> (chunk_hash); stored_chunks: (record {canister_id : canister_id}) -> (vec chunk_hash); - delete_chunks: (record {canister_id : canister_id; hash_list : vec chunk_hash}) -> (); clear_chunk_store: (record {canister_id: canister_id}) -> (); install_code : (record { mode : variant {install; reinstall; upgrade}; From 553789850fe67bc994a9865d422e9d5c51208912 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Tue, 26 Sep 2023 19:02:43 +0200 Subject: [PATCH 091/102] added method install_chunked_code to the management --- spec/_attachments/ic.did | 15 +++++++++++++++ spec/index.md | 14 +++++++++++--- 2 files changed, 26 insertions(+), 3 deletions(-) diff --git a/spec/_attachments/ic.did b/spec/_attachments/ic.did index 0d8c1798f..ab9d575ff 100644 --- a/spec/_attachments/ic.did +++ b/spec/_attachments/ic.did @@ -142,6 +142,21 @@ service ic : { arg : blob; sender_canister_version : opt nat64; }) -> (); + install_chunked_code: (record { + mode : variant { + install; + reinstall; + upgrade : opt record { + skip_pre_upgrade: opt bool; + }; + }; + target_canister: canister_id; + storage_canister: opt canister_id; + wasm_module_hash: blob; + chunk_hashes_list: vec chunk_hash; + arg : blob; + sender_canister_version : opt nat64; + }) uninstall_code : (record { canister_id : canister_id; sender_canister_version : opt nat64; diff --git a/spec/index.md b/spec/index.md index 64dfac831..d51b15908 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1941,13 +1941,21 @@ The `wasm_module` field specifies the canister module to be installed. The syste - If the `wasm_module` starts with byte sequence `[0x1f, 0x8b, 0x08]`, the system parses `wasm_module` as a gzip-compressed WebAssembly binary. -- If the `wasm_module` starts with the byte sequence `[0xd9, 0xd9, 0xf7]`, the system parses `wasm_module` as a self describing CBOR encoding of a map that specifies: an optional canister identifier `storage_canister` and a list of hash values, `[h0,h1,...,hk]` with `k <= MAX_CHUNKS_IN_LARGE_WASM`. In this case, the system looks up in the chunk store of `storage_canister` (or that of the target canister if this parameter is not provided) blobs corresponding to `h1,...,hk`, concatenates them to obtain a blob of bytes and checks that `h0` is the hash of the resulting blob. The caller must be a controller of the `storage_canister` or the caller must be the `storage_canister` and `storage_canister` must be on the same subnet as the target canister. If the lookup succeeds, then the system interprets the blob as a vanilla wasm module (or a gzipped one) per the rules above. - - The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. This method traps if the canister's cycle balance decreases below the canister's freezing limit after executing the method. +### IC method `install_chunked_code` {#ic-install_chunked_code} +This method installs code that had been previously uploaded in chunks. + +The `mode, arg, sender_canister_version` parameters are as above. + The `target_canister` specifies the canister where the code should be uploaded to. +The optional `storage_canister` parameters specifies the canister where the chunks are stored. +The caller must be a controller of the `storage_canister` or the caller must be the `storage_canister` and `storage_canister` must be on the same subnet as the target canister. + +The `chunk_hashse_list` specifies a list of hash values `[h0,h1,...,hk]` with `k <= MAX_CHUNKS_IN_LARGE_WASM`. The system looks up in the chunk store of `storage_canister` (or that of the target canister if this parameter is not provided) blobs corresponding to `h1,...,hk`, concatenates them to obtain a blob of bytes `wasm_module` and checks that `h0` is the hash of the resulting blob. +It then calls `install_code` with parameters (`mode,target_canister,wasm_module,arg,sender_canister_version`). + ### IC method `uninstall_code` {#ic-uninstall_code} This method removes a canister's code and state, making the canister *empty* again. From d80ee16c94bf8420f77a8619e23794639b6ce0a7 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Wed, 27 Sep 2023 19:40:16 +0200 Subject: [PATCH 092/102] install_chunked_wasm management canister method added --- spec/_attachments/ic.did | 2 +- spec/index.md | 48 ++++++++++++++++++++++++++++++++++++---- 2 files changed, 45 insertions(+), 5 deletions(-) diff --git a/spec/_attachments/ic.did b/spec/_attachments/ic.did index ab9d575ff..e9cc76bc5 100644 --- a/spec/_attachments/ic.did +++ b/spec/_attachments/ic.did @@ -152,8 +152,8 @@ service ic : { }; target_canister: canister_id; storage_canister: opt canister_id; - wasm_module_hash: blob; chunk_hashes_list: vec chunk_hash; + wasm_module_hash: blob; arg : blob; sender_canister_version : opt nat64; }) diff --git a/spec/index.md b/spec/index.md index d51b15908..b5c41c84d 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1953,8 +1953,7 @@ The `mode, arg, sender_canister_version` parameters are as above. The optional `storage_canister` parameters specifies the canister where the chunks are stored. The caller must be a controller of the `storage_canister` or the caller must be the `storage_canister` and `storage_canister` must be on the same subnet as the target canister. -The `chunk_hashse_list` specifies a list of hash values `[h0,h1,...,hk]` with `k <= MAX_CHUNKS_IN_LARGE_WASM`. The system looks up in the chunk store of `storage_canister` (or that of the target canister if this parameter is not provided) blobs corresponding to `h1,...,hk`, concatenates them to obtain a blob of bytes `wasm_module` and checks that `h0` is the hash of the resulting blob. -It then calls `install_code` with parameters (`mode,target_canister,wasm_module,arg,sender_canister_version`). +The `chunk_hashse_list` specifies a list of hash values `[h1,...,hk]` with `k <= MAX_CHUNKS_IN_LARGE_WASM`. The system looks up in the chunk store of `storage_canister` (or that of the target canister if this parameter is not provided) blobs corresponding to `h1,...,hk`, concatenates them to obtain a blob of bytes `wasm_module`. It then checks that the SHA256 hash of `wasm_module` is equal to the `wasm_module_hash` parameter of the call, and calls `install_code` with parameters (`mode,target_canister,wasm_module,arg,sender_canister_version`). ### IC method `uninstall_code` {#ic-uninstall_code} @@ -3045,7 +3044,7 @@ is_effective_canister_id(E.content, ECID) E.content.arg = candid({canister_id = CanisterId, …}) E.content.sender ∈ S.controllers[CanisterId] E.content.method_name ∈ - { "install_code", "uninstall_code", "update_settings", "start_canister", "stop_canister", + { "install_code", "install_chunked_code", "uninstall_code", "update_settings", "start_canister", "stop_canister", "canister_status", "delete_canister", "provisional_create_canister_with_cycles", "provisional_top_up_canister" } ) ∨ ( @@ -3509,7 +3508,7 @@ Note that returning does *not* imply that the call associated with this message The function `validate_sender_canister_version` checks that `sender_canister_version` matches the actual canister version of the sender in all calls to the methods of the management canister that take `sender_canister_version`: validate_sender_canister_version(new_calls, canister_version_from_system) = - ∀ call ∈ new_calls. (call.callee = ic_principal and (call.method = 'create_canister' or call.method = 'update_settings' or call.method = 'install_code' or call.method = 'uninstall_code' or call.method = 'provisional_create_canister_with_cycles') and call.arg = candid(A) and A.sender_canister_version = n) => n = canister_version_from_system + ∀ call ∈ new_calls. (call.callee = ic_principal and (call.method = 'create_canister' or call.method = 'update_settings' or call.method = 'install_code' or call.method = `install_chunked_code` or call.method = 'uninstall_code' or call.method = 'provisional_create_canister_with_cycles') and call.arg = candid(A) and A.sender_canister_version = n) => n = canister_version_from_system The functions `query_as_update` and `system_task_as_update` turns a query function (note that composite query methods cannot be called when executing a message during this transition) resp the heartbeat or global timer into an update function; this is merely a notational trick to simplify the rule: @@ -4091,6 +4090,7 @@ S with ``` + #### IC Management Canister: Code upgrade Only the controllers of the given canister can install new code. This changes the code of an *existing* canister, preserving the state in the stable memory. This involves invoking the `canister_pre_upgrade` method, if the `skip_pre_upgrade` flag is not set to `opt true`, on the old and `canister_post_upgrade` method on the new canister, which must succeed and must not invoke other methods. @@ -4225,6 +4225,46 @@ S with ``` + +#### IC Management Canister: Install chunked code + + + + +Conditions + +```html +S.messages = Older_messages · CallMessage M · Younger_messages +(M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) +M.callee = ic_principal +M.method_name = 'install_chunked_code' +if A.storage_canister = None then + let storage_canister = A.target_canister + else let storage_canister A. storage_canister +M.caller ∈ S.controllers[storage_canister] ∩ S.controllers[target_canister] +∀ h ∈ A.chunk_hashes_list = [h1,h2,...,hk]: h ∈ dom(S.chunk_store[storage_canister]) +module = S.chunk_store[storage_canister][h1] || ... || S.chunk_store[storage_canister][hk] +A.wasm_module_hash = SHA256(module) + + +``` + +State after +```html +S.messages = Older_messages · + CallMessage { + caller = M.caller + mode = A.mode + canister_id = A.target_canister + wasm_module = module + args = A.args + sender_canister_version = M.sender_canister_version + } · Younger_messages + +``` + + + #### IC Management Canister: Code uninstallation {#rule-uninstall} Upon uninstallation, the canister is reverted to an empty canister, and all outstanding call contexts are rejected and marked as deleted. From cbfc95ec1dd3f49da34491233c5399c3ef123445 Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Fri, 29 Sep 2023 13:36:55 +0200 Subject: [PATCH 093/102] break a long sentence into two --- spec/index.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/spec/index.md b/spec/index.md index b5c41c84d..813a6d98c 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1951,7 +1951,8 @@ This method installs code that had been previously uploaded in chunks. The `mode, arg, sender_canister_version` parameters are as above. The `target_canister` specifies the canister where the code should be uploaded to. The optional `storage_canister` parameters specifies the canister where the chunks are stored. -The caller must be a controller of the `storage_canister` or the caller must be the `storage_canister` and `storage_canister` must be on the same subnet as the target canister. +The caller must be a controller of the `storage_canister` or the caller must be the `storage_canister`. +The `storage_canister` must be on the same subnet as the target canister. The `chunk_hashse_list` specifies a list of hash values `[h1,...,hk]` with `k <= MAX_CHUNKS_IN_LARGE_WASM`. The system looks up in the chunk store of `storage_canister` (or that of the target canister if this parameter is not provided) blobs corresponding to `h1,...,hk`, concatenates them to obtain a blob of bytes `wasm_module`. It then checks that the SHA256 hash of `wasm_module` is equal to the `wasm_module_hash` parameter of the call, and calls `install_code` with parameters (`mode,target_canister,wasm_module,arg,sender_canister_version`). From 5d41627b9321303ae532d9f919a010f9be25f1b7 Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Fri, 29 Sep 2023 13:39:03 +0200 Subject: [PATCH 094/102] fix ic.did --- spec/_attachments/ic.did | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/spec/_attachments/ic.did b/spec/_attachments/ic.did index e9cc76bc5..7fb3c2d57 100644 --- a/spec/_attachments/ic.did +++ b/spec/_attachments/ic.did @@ -147,7 +147,7 @@ service ic : { install; reinstall; upgrade : opt record { - skip_pre_upgrade: opt bool; + skip_pre_upgrade: opt bool; }; }; target_canister: canister_id; @@ -156,7 +156,7 @@ service ic : { wasm_module_hash: blob; arg : blob; sender_canister_version : opt nat64; - }) + }) -> (); uninstall_code : (record { canister_id : canister_id; sender_canister_version : opt nat64; From d9fce244fb59001af13c88c4ddab35b590bbdfde Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Fri, 29 Sep 2023 14:01:47 +0200 Subject: [PATCH 095/102] formulation --- spec/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/index.md b/spec/index.md index 813a6d98c..f56f5a053 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1905,7 +1905,7 @@ The optional `sender_canister_version` parameter can contain the caller's canist ### IC method `upload_chunk` {#ic-upload_chunk} -Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister (and the canister itself) to upload such chunks. The method returns the hash of the chunk that was stored. The size of each chunk should be up to 1MiB. The size of the chunk store is bounded: currently it can hold up to `CHUNK_STORE_SIZE` chunks. The storage cost of each chunk is fixed and corresponds to storing 1MiB of data. +Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister (and the canister itself) to upload such chunks. The method returns the hash of the chunk that was stored. The size of each chunk must be at most 1MiB. The maximum number of chunks in the chunk store is `CHUNK_STORE_SIZE` chunks. The storage cost of each chunk is fixed and corresponds to storing 1MiB of data. ### IC method `clear_store` {#ic-clear_store} From 902841169617b30ad5b8cf69aebe4693d64737c4 Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Fri, 29 Sep 2023 14:06:25 +0200 Subject: [PATCH 096/102] remove chunk cddl --- .github/workflows/check-cddl-candid.yml | 1 - docusaurus/docs/_attachments/chunked-wasms.cddl | 1 - spec/_attachments/chunked-wasms.cddl | 15 --------------- spec/index.md | 2 +- 4 files changed, 1 insertion(+), 18 deletions(-) delete mode 120000 docusaurus/docs/_attachments/chunked-wasms.cddl delete mode 100644 spec/_attachments/chunked-wasms.cddl diff --git a/.github/workflows/check-cddl-candid.yml b/.github/workflows/check-cddl-candid.yml index fb4033864..ae94c2ba3 100644 --- a/.github/workflows/check-cddl-candid.yml +++ b/.github/workflows/check-cddl-candid.yml @@ -10,7 +10,6 @@ jobs: - name: Check cddl files run: | docker run --rm -v $PWD/spec/_attachments:/workdir ghcr.io/anweiss/cddl-cli:0.9.1 compile-cddl --cddl /workdir/certificates.cddl - docker run --rm -v $PWD/spec/_attachments:/workdir ghcr.io/anweiss/cddl-cli:0.9.1 compile-cddl --cddl /workdir/chunked-wasms.cddl docker run --rm -v $PWD/spec/_attachments:/workdir ghcr.io/anweiss/cddl-cli:0.9.1 compile-cddl --cddl /workdir/requests.cddl - name: Check candid files run: | diff --git a/docusaurus/docs/_attachments/chunked-wasms.cddl b/docusaurus/docs/_attachments/chunked-wasms.cddl deleted file mode 120000 index a0aa1d15d..000000000 --- a/docusaurus/docs/_attachments/chunked-wasms.cddl +++ /dev/null @@ -1 +0,0 @@ -../../../spec/_attachments/chunked-wasms.cddl \ No newline at end of file diff --git a/spec/_attachments/chunked-wasms.cddl b/spec/_attachments/chunked-wasms.cddl deleted file mode 100644 index f75c9bd8f..000000000 --- a/spec/_attachments/chunked-wasms.cddl +++ /dev/null @@ -1,15 +0,0 @@ - -; The specification of a large Wasm; consists of a location (canister principal) and a list of hashes. - - - -chunked-wasm = tagged<{ - ? storage_canister : principal - list_of_hashes : [* hash] -}> - -hash = bytes .size 32 - -principal = bytes .size (0..29) - -tagged = #6.55799(t) ; the CBOR tag diff --git a/spec/index.md b/spec/index.md index f56f5a053..74dccaa11 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1089,7 +1089,7 @@ Applications can work around these problems. For the first problem, the query re ## Canister module format {#canister-module-format} -A canister module is a [WebAssembly module](https://webassembly.github.io/spec/core/index.html) that is either in binary format (typically `.wasm`), gzip-compressed (typically `.wasm.gz`), or specified through a list of hashes. If the module starts with byte sequence `[0x1f, 0x8b, 0x08]`, then the system decompresses the contents as a gzip stream according to [RFC-1952](https://datatracker.ietf.org/doc/html/rfc1952.html) and then parses the output as a WebAssembly binary. If the module starts with the byte sequence `[0xd9, 0xd9, 0xf7]` (self-describe CBOR tag), then the system interprets the content as a CBOR (see [CBOR](#cbor)) value according to the following [CDDL](_attachments/chunked-wasms.cddl) (see [CDDL](#cddl)). In this case, the actual Wasm module is obtained by concatenating the individual chunks (from the chunk store of the installed canister or from the chunk store of a specified storage canister) that correspond to these hashes (skipping the first hash which is the hash of the overall concatenated Wasm module). +A canister module is a [WebAssembly module](https://webassembly.github.io/spec/core/index.html) that is either in binary format (typically `.wasm`) or gzip-compressed (typically `.wasm.gz`). If the module starts with byte sequence `[0x1f, 0x8b, 0x08]`, then the system decompresses the contents as a gzip stream according to [RFC-1952](https://datatracker.ietf.org/doc/html/rfc1952.html) and then parses the output as a WebAssembly binary. ## Canister interface (System API) {#system-api} From ecc00eb96096f5f0fb5b5022f0362cdbbabb3dad Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Fri, 29 Sep 2023 14:32:33 +0200 Subject: [PATCH 097/102] clean up --- spec/_attachments/ic.did | 2 +- spec/index.md | 65 +++++++++++++++++++--------------------- 2 files changed, 31 insertions(+), 36 deletions(-) diff --git a/spec/_attachments/ic.did b/spec/_attachments/ic.did index 7fb3c2d57..c0d684461 100644 --- a/spec/_attachments/ic.did +++ b/spec/_attachments/ic.did @@ -127,8 +127,8 @@ service ic : { canister_id : principal; chunk : blob; }) -> (chunk_hash); + clear_chunk_store: (record {canister_id : canister_id}) -> (); stored_chunks: (record {canister_id : canister_id}) -> (vec chunk_hash); - clear_chunk_store: (record {canister_id: canister_id}) -> (); install_code : (record { mode : variant { install; diff --git a/spec/index.md b/spec/index.md index 74dccaa11..5cb30be77 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1903,14 +1903,18 @@ Not including a setting in the `settings` record means not changing that field. The optional `sender_canister_version` parameter can contain the caller's canister version. If provided, its value must be equal to `ic0.canister_version`. - ### IC method `upload_chunk` {#ic-upload_chunk} + Canisters have associated some storage space (hence forth chunk storage) where they can hold chunks of Wasm modules that are too lage to fit in a single message. This method allows the controllers of a canister (and the canister itself) to upload such chunks. The method returns the hash of the chunk that was stored. The size of each chunk must be at most 1MiB. The maximum number of chunks in the chunk store is `CHUNK_STORE_SIZE` chunks. The storage cost of each chunk is fixed and corresponds to storing 1MiB of data. - ### IC method `clear_store` {#ic-clear_store} + Canister controllers (and the canister itself) can clear the entire chunk storage of a canister. +### IC method `stored_chunks` {#ic-stored_chunks} + +Canister controllers (and the canister itself) can list the hashes of chunks in the chunk storage of a canister. + ### IC method `install_code` {#ic-install_code} This method installs code into a canister. @@ -1946,15 +1950,16 @@ The optional `sender_canister_version` parameter can contain the caller's canist This method traps if the canister's cycle balance decreases below the canister's freezing limit after executing the method. ### IC method `install_chunked_code` {#ic-install_chunked_code} -This method installs code that had been previously uploaded in chunks. -The `mode, arg, sender_canister_version` parameters are as above. - The `target_canister` specifies the canister where the code should be uploaded to. -The optional `storage_canister` parameters specifies the canister where the chunks are stored. +This method installs code that had previously been uploaded in chunks. + +The `mode`, `arg`, and `sender_canister_version` parameters are as for `install_code`. +The `target_canister` specifies the canister to which the code should be installed. +The optional `storage_canister` specifies the canister in whose chunk storage the chunks are stored (defaults to `target_canister` if not specified). The caller must be a controller of the `storage_canister` or the caller must be the `storage_canister`. The `storage_canister` must be on the same subnet as the target canister. -The `chunk_hashse_list` specifies a list of hash values `[h1,...,hk]` with `k <= MAX_CHUNKS_IN_LARGE_WASM`. The system looks up in the chunk store of `storage_canister` (or that of the target canister if this parameter is not provided) blobs corresponding to `h1,...,hk`, concatenates them to obtain a blob of bytes `wasm_module`. It then checks that the SHA256 hash of `wasm_module` is equal to the `wasm_module_hash` parameter of the call, and calls `install_code` with parameters (`mode,target_canister,wasm_module,arg,sender_canister_version`). +The `chunk_hashes_list` specifies a list of hash values `[h1,...,hk]` with `k <= MAX_CHUNKS_IN_LARGE_WASM`. The system looks up in the chunk store of `storage_canister` (or that of the target canister if `storage_canister` is not specified) blobs corresponding to `h1,...,hk` and concatenates them to obtain a blob of bytes referred to as `wasm_module` in `install_code`. It then checks that the SHA-256 hash of `wasm_module` is equal to the `wasm_module_hash` parameter and calls `install_code` with parameters `(record {mode; target_canister; wasm_module; arg; sender_canister_version})`. ### IC method `uninstall_code` {#ic-uninstall_code} @@ -2565,7 +2570,6 @@ The [WebAssembly System API](#system-api) is relatively low-level, and some of i Callback = (abstract) ChunkStore = Hash -> Blob - Arg = Blob; CallerId = Principal; @@ -3887,7 +3891,6 @@ S with ``` - #### IC Management Canister: Upload Chunk A controller of a canister, or the canister itself can upload chunks to the chunk store of that canister. @@ -3947,8 +3950,6 @@ S with ``` - - #### IC Management Canister: List stored chunks The controller of a canister, or the canister itself can list the hashes of the chunks stored in the chunk store. @@ -4091,7 +4092,6 @@ S with ``` - #### IC Management Canister: Code upgrade Only the controllers of the given canister can install new code. This changes the code of an *existing* canister, preserving the state in the stable memory. This involves invoking the `canister_pre_upgrade` method, if the `skip_pre_upgrade` flag is not set to `opt true`, on the old and `canister_post_upgrade` method on the new canister, which must succeed and must not invoke other methods. @@ -4226,45 +4226,40 @@ S with ``` - #### IC Management Canister: Install chunked code - - - Conditions ```html + S.messages = Older_messages · CallMessage M · Younger_messages (M.queue = Unordered) or (∀ msg ∈ Older_messages. msg.queue ≠ M.queue) M.callee = ic_principal M.method_name = 'install_chunked_code' -if A.storage_canister = None then - let storage_canister = A.target_canister - else let storage_canister A. storage_canister -M.caller ∈ S.controllers[storage_canister] ∩ S.controllers[target_canister] -∀ h ∈ A.chunk_hashes_list = [h1,h2,...,hk]: h ∈ dom(S.chunk_store[storage_canister]) -module = S.chunk_store[storage_canister][h1] || ... || S.chunk_store[storage_canister][hk] -A.wasm_module_hash = SHA256(module) - +if A.storage_canister = null then + storage_canister = A.target_canister +else + storage_canister = A.storage_canister +M.caller ∈ S.controllers[A.target_canister] +M.caller ∈ S.controllers[storage_canister] ∪ {storage_canister} +∀ h ∈ A.chunk_hashes_list. h ∈ dom(S.chunk_store[storage_canister]) +A.chunk_hashes_list = [h1,h2,...,hk] +wasm_module = S.chunk_store[storage_canister][h1] || ... || S.chunk_store[storage_canister][hk] +A.wasm_module_hash = SHA-256(wasm_module) +M' = M with + method_name = 'install_code' + arg = candid(record {A.mode; A.target_canister; wasm_module; A.arg; A.sender_canister_version}) ``` State after -```html -S.messages = Older_messages · - CallMessage { - caller = M.caller - mode = A.mode - canister_id = A.target_canister - wasm_module = module - args = A.args - sender_canister_version = M.sender_canister_version - } · Younger_messages -``` +```html +S with + messages = Older_messages · CallMessage M' · Younger_messages +``` #### IC Management Canister: Code uninstallation {#rule-uninstall} From 2b88f6760fc411fcde6dbdf04e4799ef3f4a4eb0 Mon Sep 17 00:00:00 2001 From: Martin Raszyk Date: Fri, 29 Sep 2023 14:37:05 +0200 Subject: [PATCH 098/102] mention install_chunk_code in canister global timer and version specs --- spec/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/spec/index.md b/spec/index.md index 5cb30be77..be8983473 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1231,7 +1231,7 @@ While an implementation will likely try to keep the interval between `canister_h For time-based execution, the WebAssembly module can export a function with name `canister_global_timer`. This function is called if the canister has set its global timer (using the System API function `ic0.global_timer_set`) and the current time (as returned by the System API function `ic0.time`) has passed the value of the global timer. -Once the function `canister_global_timer` is scheduled, the canister's global timer is deactivated. The global timer is also deactivated upon changes to the canister's Wasm module (calling `install_code`, `uninstall_code` methods of the management canister or if the canister runs out of cycles). In particular, the function `canister_global_timer` won't be scheduled again unless the canister sets the global timer again (using the System API function `ic0.global_timer_set`). The global timer scheduling algorithm is implementation-defined. +Once the function `canister_global_timer` is scheduled, the canister's global timer is deactivated. The global timer is also deactivated upon changes to the canister's Wasm module (calling `install_code`, `install_chunked_code`, `uninstall_code` methods of the management canister or if the canister runs out of cycles). In particular, the function `canister_global_timer` won't be scheduled again unless the canister sets the global timer again (using the System API function `ic0.global_timer_set`). The global timer scheduling algorithm is implementation-defined. `canister_global_timer` is triggered by the IC, and therefore has no arguments and cannot reply or reject. Still, the function `canister_global_timer` can initiate new calls. @@ -1474,7 +1474,7 @@ This function allows a canister to find out if it is running, stopping or stoppe ### Canister version {#system-api-canister-version} -For each canister, the system maintains a *canister version*. Upon canister creation, it is set to 0, and it is **guaranteed** to be incremented upon every change of the canister's code or settings and successful message execution except for successful message execution of a query method, i.e., upon every successful management canister call of methods `update_settings`, `install_code`, and `uninstall_code` on that canister, code uninstallation due to that canister running out of cycles, and successful execution of update methods, response callbacks, heartbeats, and global timers. The system can arbitrarily increment the canister version also if the canister's code and settings do not change. +For each canister, the system maintains a *canister version*. Upon canister creation, it is set to 0, and it is **guaranteed** to be incremented upon every change of the canister's code or settings and successful message execution except for successful message execution of a query method, i.e., upon every successful management canister call of methods `update_settings`, `install_code`, `install_chunked_code`, and `uninstall_code` on that canister, code uninstallation due to that canister running out of cycles, and successful execution of update methods, response callbacks, heartbeats, and global timers. The system can arbitrarily increment the canister version also if the canister's code and settings do not change. - `ic0.canister_version : () → i64` From 721e205eea626810f6d4dc1f29c1efcb057e2fa8 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Mon, 2 Oct 2023 15:45:55 +0200 Subject: [PATCH 099/102] target & storage must be on the same subnet --- spec/index.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/spec/index.md b/spec/index.md index be8983473..90787df66 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1954,10 +1954,10 @@ This method traps if the canister's cycle balance decreases below the canister's This method installs code that had previously been uploaded in chunks. The `mode`, `arg`, and `sender_canister_version` parameters are as for `install_code`. -The `target_canister` specifies the canister to which the code should be installed. -The optional `storage_canister` specifies the canister in whose chunk storage the chunks are stored (defaults to `target_canister` if not specified). -The caller must be a controller of the `storage_canister` or the caller must be the `storage_canister`. -The `storage_canister` must be on the same subnet as the target canister. +The `target_canister` specifies the canister where the code should be installed. +The optional `storage_canister` specifies the canister in whose chunk storage the chunks are stored (this parameter defaults to `target_canister` if not specified). +For the call to succeed, the caller must be a controller of the `storage_canister` or the caller must be the `storage_canister` and +the `storage_canister` must be on the same subnet as the target canister. The `chunk_hashes_list` specifies a list of hash values `[h1,...,hk]` with `k <= MAX_CHUNKS_IN_LARGE_WASM`. The system looks up in the chunk store of `storage_canister` (or that of the target canister if `storage_canister` is not specified) blobs corresponding to `h1,...,hk` and concatenates them to obtain a blob of bytes referred to as `wasm_module` in `install_code`. It then checks that the SHA-256 hash of `wasm_module` is equal to the `wasm_module_hash` parameter and calls `install_code` with parameters `(record {mode; target_canister; wasm_module; arg; sender_canister_version})`. @@ -4242,6 +4242,7 @@ else storage_canister = A.storage_canister M.caller ∈ S.controllers[A.target_canister] M.caller ∈ S.controllers[storage_canister] ∪ {storage_canister} +S.canister_subnet[A.target_canister] = S.canister_subnet[strorage_canister] ∀ h ∈ A.chunk_hashes_list. h ∈ dom(S.chunk_store[storage_canister]) A.chunk_hashes_list = [h1,h2,...,hk] wasm_module = S.chunk_store[storage_canister][h1] || ... || S.chunk_store[storage_canister][hk] From 9fcca02d072f155e1c15791158a316b82b9d313b Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Mon, 2 Oct 2023 15:48:32 +0200 Subject: [PATCH 100/102] chunk store is emptied if a canister is uninstalled --- spec/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/spec/index.md b/spec/index.md index 90787df66..bca08f0da 100644 --- a/spec/index.md +++ b/spec/index.md @@ -4290,6 +4290,7 @@ State after S with canisters[A.canister_id] = EmptyCanister certified_data[A.canister_id] = "" + chunk_store = () canister_history[A.canister_id] = { total_num_changes = N + 1; recent_changes = H · { From 6c690d43172515eb90e17d96bc2a378d0a27e1c9 Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Thu, 5 Oct 2023 18:04:10 +0200 Subject: [PATCH 101/102] fixed the and/or binding --- spec/index.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/spec/index.md b/spec/index.md index bca08f0da..2d29890cb 100644 --- a/spec/index.md +++ b/spec/index.md @@ -1956,8 +1956,7 @@ This method installs code that had previously been uploaded in chunks. The `mode`, `arg`, and `sender_canister_version` parameters are as for `install_code`. The `target_canister` specifies the canister where the code should be installed. The optional `storage_canister` specifies the canister in whose chunk storage the chunks are stored (this parameter defaults to `target_canister` if not specified). -For the call to succeed, the caller must be a controller of the `storage_canister` or the caller must be the `storage_canister` and -the `storage_canister` must be on the same subnet as the target canister. +For the call to succeed, the caller must be a controller of the `storage_canister` or the caller must be the `storage_canister`. The `storage_canister` must be on the same subnet as the target canister. The `chunk_hashes_list` specifies a list of hash values `[h1,...,hk]` with `k <= MAX_CHUNKS_IN_LARGE_WASM`. The system looks up in the chunk store of `storage_canister` (or that of the target canister if `storage_canister` is not specified) blobs corresponding to `h1,...,hk` and concatenates them to obtain a blob of bytes referred to as `wasm_module` in `install_code`. It then checks that the SHA-256 hash of `wasm_module` is equal to the `wasm_module_hash` parameter and calls `install_code` with parameters `(record {mode; target_canister; wasm_module; arg; sender_canister_version})`. @@ -3028,7 +3027,7 @@ Requests that have expired are dropped here. Ingress message inspection is applied, and messages that are not accepted by the canister are dropped. -The (unspecified) function `idle_cycles_burned_rate(compute_allocation, memory_allocation, memory_usage, subnet_size)` determines the idle resource consumption rate in cycles per day of a canister given its current compute and memory allocation, memory usage, and subnet size. The function `freezing_limit(compute_allocation, memory_allocation, freezing_threshold, memory_usage, subnet_size)` determines the freezing limit in cycles of a canister given its current compute and memory allocation, freezing threshold in seconds, memory usage, and subnet size. The value `freezing_limit(compute_allocation, memory_allocation, freezing_threshold, memory_usage, subnet_size)` is derived from `idle_cycles_burned_rate(compute_allocation, memory_allocation, memory_usage, subnet_size)` and `freezing_threshold` as follows: +The (unspecified) function `idle_cycles_burned_rate(compute_allocation, memory_allocation, memory_usage, subnet_size)` determines the idle resource consumption rate in cycles per day of a canister given its current compute and memory allocation, memory usage, and subnet size. The function `freezing_limit(compute_allocation, memory_allocation, freezing_threshold, memory_usage, subnet_size)` determines the freezing limit in cycles of a canister given its current compute and memory allocation, freezing threshold in seconds, memory usage, chunk store usage & and subnet size. The value `freezing_limit(compute_allocation, memory_allocation, freezing_threshold, memory_usage, subnet_size)` is derived from `idle_cycles_burned_rate(compute_allocation, memory_allocation, memory_usage, subnet_size)` and `freezing_threshold` as follows: freezing_limit(compute_allocation, memory_allocation, freezing_threshold, memory_usage, subnet_size) = idle_cycles_burned_rate(compute_allocation, memory_allocation, memory_usage, subnet_size) * freezing_threshold / (24 * 60 * 60) From 8f4ba87ae662122344ce24cb7ab84dc37bcf9c3d Mon Sep 17 00:00:00 2001 From: bogwar <51327868+bogwar@users.noreply.github.com> Date: Fri, 6 Oct 2023 16:18:25 +0200 Subject: [PATCH 102/102] fix --- spec/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/index.md b/spec/index.md index 2d29890cb..71309d009 100644 --- a/spec/index.md +++ b/spec/index.md @@ -3027,7 +3027,7 @@ Requests that have expired are dropped here. Ingress message inspection is applied, and messages that are not accepted by the canister are dropped. -The (unspecified) function `idle_cycles_burned_rate(compute_allocation, memory_allocation, memory_usage, subnet_size)` determines the idle resource consumption rate in cycles per day of a canister given its current compute and memory allocation, memory usage, and subnet size. The function `freezing_limit(compute_allocation, memory_allocation, freezing_threshold, memory_usage, subnet_size)` determines the freezing limit in cycles of a canister given its current compute and memory allocation, freezing threshold in seconds, memory usage, chunk store usage & and subnet size. The value `freezing_limit(compute_allocation, memory_allocation, freezing_threshold, memory_usage, subnet_size)` is derived from `idle_cycles_burned_rate(compute_allocation, memory_allocation, memory_usage, subnet_size)` and `freezing_threshold` as follows: +The (unspecified) function `idle_cycles_burned_rate(compute_allocation, memory_allocation, memory_usage, subnet_size)` determines the idle resource consumption rate in cycles per day of a canister given its current compute and memory allocation, memory usage, and subnet size. The function `freezing_limit(compute_allocation, memory_allocation, freezing_threshold, memory_usage, subnet_size)` determines the freezing limit in cycles of a canister given its current compute and memory allocation, freezing threshold in seconds, memory usage & and subnet size. The value `freezing_limit(compute_allocation, memory_allocation, freezing_threshold, memory_usage, subnet_size)` is derived from `idle_cycles_burned_rate(compute_allocation, memory_allocation, memory_usage, subnet_size)` and `freezing_threshold` as follows: freezing_limit(compute_allocation, memory_allocation, freezing_threshold, memory_usage, subnet_size) = idle_cycles_burned_rate(compute_allocation, memory_allocation, memory_usage, subnet_size) * freezing_threshold / (24 * 60 * 60)