diff --git a/docs/general/faq.md b/docs/general/faq.md index fed5f1f617f3..43f3750867ef 100644 --- a/docs/general/faq.md +++ b/docs/general/faq.md @@ -45,6 +45,37 @@ DOT. For more information on the Polkadot roadmap please visit the [official Polkadot website](https://polkadot.network/technology/#roadmap). +## Consensus + +### Why do we need Consensus? + +Consensus is a method for coming to agreement over a shared state. In order for the state of the +blockchain to continue to build and move forward, all nodes in the network must agree and come to +consensus. It is the way that the nodes in a decentralized network are able to stay synced with each +other. Without consensus for the decentralized network of nodes in a blockchain, there is no way to +ensure that the state one node believes is true will be shared by the other nodes. Consensus aims to +provide the _objective_ view of the state amid participants who each have their own _subjective_ +views of the network. It is the process by which these nodes communicate and come to agreement, and +are able to build new blocks. + +### What are PoW and PoS? + +Proof of Work (PoW) and Proof of Stake (PoS) have been inaccurately used as short hand to refer to +consensus mechanisms of blockchains, but that does not capture the full picture. PoW is the method +for agreeing on a block author and part of the fuller +[Nakamoto consensus](../learn/learn-consensus.md#nakamoto-consensus) that also encompasses a chain +selection algorithm (longest chain rule in Bitcoin). Similarly, PoS is a set of rules for selecting +the validator set and does not specify a chain selection rule or how a chain might reach finality. +PoS algorithms have traditionally been paired with an algorithm for coming to Byzantine agreement +between nodes. For example, [Tendermint](../learn/learn-comparisons-cosmos.md) is a practical +Byzantine fault tolerant algorithm that uses PoS as its validator set selection method. + +### Why not Proof of Work? + +Although simple and effective in coming to a decentralized consensus on the next block producer, +proof of work with Nakamoto consensus consumes an incredible amount of energy, has no economic or +provable finality, and has no effective strategy in resisting cartels. + ## Validators ### How do I apply to be a validator? diff --git a/docs/learn/learn-consensus.md b/docs/learn/learn-consensus.md index 2ead7c0ce564..9807981fc8f2 100644 --- a/docs/learn/learn-consensus.md +++ b/docs/learn/learn-consensus.md @@ -1,103 +1,48 @@ --- id: learn-consensus -title: Polkadot Consensus +title: Polkadot's Consensus Protocols sidebar_label: Consensus -description: The Consensus Mechanism of Polkadot. +description: The Consensus Mechanisms of Polkadot. keywords: [consensus, proof of stake, nominated proof of stake, hybrid consensus, finality] slug: ../learn-consensus --- -## Why do we need Consensus? - -Consensus is a method for coming to agreement over a shared state. In order for the state of the -blockchain to continue to build and move forward, all nodes in the network must agree and come to -consensus. It is the way that the nodes in a decentralized network are able to stay synced with each -other. Without consensus for the decentralized network of nodes in a blockchain, there is no way to -ensure that the state one node believes is true will be shared by the other nodes. Consensus aims to -provide the _objective_ view of the state amid participants who each have their own _subjective_ -views of the network. It is the process by which these nodes communicate and come to agreement, and -are able to build new blocks. - -## What are PoW and PoS? - -Proof of Work (PoW) and Proof of Stake (PoS) have been inaccurately used as short hand to refer to -consensus mechanisms of blockchains, but that does not capture the full picture. PoW is the method -for agreeing on a block author and part of the fuller [Nakamoto consensus](#nakamoto-consensus) that -also encompasses a chain selection algorithm (longest chain rule in Bitcoin). Similarly, PoS is a -set of rules for selecting the validator set and does not specify a chain selection rule or how a -chain might reach finality. PoS algorithms have traditionally been paired with an algorithm for -coming to Byzantine agreement between nodes. For example, [Tendermint](learn-comparisons-cosmos.md) -is a practical Byzantine fault tolerant algorithm that uses PoS as its validator set selection -method. - -## Why not Proof of Work? - -Although simple and effective in coming to a decentralized consensus on the next block producer, -proof of work with Nakamoto consensus consumes an incredible amount of energy, has no economic or -provable finality, and has no effective strategy in resisting cartels. - -## Nominated Proof of Stake - In traditional PoS systems, block production participation is dependent on token holdings as opposed to computational power. While PoS developers usually have a proponent for equitable participation in -a decentralized manner, most projects end up proposing some level of centralized operation, where -the number of validators with full participation rights is limited. These validators are often seen -to be the most wealthy, and, as a result, influence the PoS network as they are the most staked. -Usually, the number of candidates to maintain the network with the necessary knowledge (and -equipment) is limited; this can directly increase operational costs as well. Systems with a large -number of validators tend to form pools to decrease the variance of their revenue and profit from -economies of scale. These pools are often off-chain. +a decentralized manner, most projects propose some level of centralized operation, where the number +of validators with full participation rights is limited. These validators are often seen to be the +most wealthy and, as a result, influence the PoS network as they are the most staked. Usually, the +number of candidates to maintain the network with the necessary knowledge (and equipment) is +limited; this can also increase operational costs. Systems with a large number of validators tend to +form pools to decrease the variance of their revenue and profit from economies of scale. These pools +are often off-chain. A way to alleviate this is to implement pool formation on-chain and allow token holders to vote with their stake for validators to represent them. -{{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} uses NPoS (Nominated Proof-of-Stake) -as its mechanism for selecting the validator set. It is designed with the roles of **validators** -and **nominators**, to maximize chain security. Actors who are interested in maintaining the network -can run a validator node. - -Validators assume the role of producing new blocks in [BABE](#block-production-babe), validating -parachain blocks, and guaranteeing finality. Nominators can choose to back select validators with -their stake. Nominators can approve candidates that they trust and back them with their tokens. - -## Probabilistic vs. Provable Finality - -A pure Nakamoto consensus blockchain that runs PoW is only able to achieve the notion of -_probabilistic finality_ and reach _eventual consensus_. Probabilistic finality means that under -some assumptions about the network and participants, if we see a few blocks building on a given -block, we can estimate the probability that it is final. Eventual consensus means that at some point -in the future, all nodes will agree on the truthfulness of one set of data. This eventual consensus -may take a long time and will not be able to be determined how long it will take ahead of time. -However, finality gadgets such as GRANDPA (GHOST-based Recursive ANcestor Deriving Prefix Agreement) -or Ethereum's Casper FFG (the Friendly Finality Gadget) are designed to give stronger and quicker -guarantees on the finality of blocks - specifically, that they can never be reverted after some -process of Byzantine agreements has taken place. The notion of irreversible consensus is known as -_provable finality._ - -In the [GRANDPA paper](https://github.com/w3f/consensus/blob/master/pdf/grandpa.pdf), it is phrased -in this way: - -:::note +## Nominated Proof of Stake -We say an oracle A in a protocol is _eventually consistent_ if it returns the same value to all -participants after some unspecified time. +{{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} uses NPoS (Nominated Proof-of-Stake) +as its mechanism for selecting the validator set. It is designed with the roles of +[**validators**](./learn-validator.md) and [**nominators**](./learn-nominator.md), to maximize chain +security. Actors who are interested in maintaining the network can run a validator node. -::: +Validators assume the role of producing new blocks, validating parachain blocks, and guaranteeing +finality. Nominators can choose to backselect validators with their stake. Nominators can approve +candidates that they trust and back them with their tokens. ## Hybrid Consensus -There are two protocols we use when we talk about the consensus protocol of -{{ polkadot: Polkadot, :polkadot }}{{ kusama: Kusama, :kusama }} GRANDPA and BABE (Blind Assignment -for Blockchain Extension). We talk about both of these because {{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} uses what is known as _hybrid -consensus_. Hybrid consensus splits up the finality gadget from the block production mechanism. - -This is a way of getting the benefits of probabilistic finality (the ability to always produce new -blocks) and provable finality (having a universal agreement on the canonical chain with no chance -for reversion) in {{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }}. It also avoids the -corresponding drawbacks of each mechanism (the chance of unknowingly following the wrong fork in -probabilistic finality, and a chance for "stalling" - not being able to produce new blocks - in -provable finality). By combining these two mechanisms, +consensus_. Hybrid consensus splits up the finality gadget ([GRANDPA](#finality-gadget-grandpa)) +from the block production mechanism ([BABE](#block-production-babe)). + +This is a way of getting the benefits of **probabilistic finality** (the ability always to produce +new blocks) and **provable finality** (having a universal agreement on the canonical chain with no +chance for reversion) in {{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }}. It also +avoids the corresponding drawbacks of each mechanism (the chance of unknowingly following the wrong +fork in probabilistic finality, and a chance for "stalling" - not being able to produce new blocks - +in provable finality). By combining these two mechanisms, {{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} allows for blocks to be rapidly produced, and the slower finality mechanism to run in a separate process to finalize blocks without risking slower transaction processing or stalling. @@ -113,7 +58,7 @@ the validator nodes and determines the authors of new blocks. BABE is comparable [Ouroboros Praos](https://eprint.iacr.org/2017/573.pdf), with some key differences in chain selection rule and slot time adjustments. BABE assigns block production slots to validators according to stake and using the {{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} -[randomness cycle](./learn-cryptography.md#randomness). The chains runtime is required to provide +[randomness cycle](./learn-cryptography.md#randomness). The chain’s runtime is required to provide the BABE authority list and randomness to the host via a consensus message in the header of the first block of each epoch. @@ -134,17 +79,17 @@ resulting in inconsistent block time. When multiple validators are block producer candidates in a given slot, all will produce a block and broadcast it to the network. At that point, it's a race. The validator whose block reaches most of the network first wins. Depending on network topology and latency, both chains will continue to -build in some capacity, until finalization kicks in and amputates a fork. See Fork Choice below for -how that works. +build in some capacity until finalization kicks in and amputates a fork. See +[Fork Choice](#fork-choice) below for how that works. ### No Validators in Slot When no validators have rolled low enough in the randomness lottery to qualify for block production, a slot can remain seemingly blockless. We avoid this by running a secondary, round-robin style validator selection algorithm in the background. The validators selected to produce blocks through -this algorithm always produce blocks, but these _secondary_ blocks are ignored if the same slot also -produces a primary block from a [VRF-selected](./learn-cryptography.md#randomness) validator. Thus, -a slot can have either a _primary_ or a _secondary_ block, and no slots are ever skipped. +this algorithm always produce blocks. Still, these _secondary_ blocks are ignored if the same slot +also produces a primary block from a [VRF-selected](./learn-cryptography.md#randomness) validator. +Thus, a slot can have either a _primary_ or a _secondary_ block, and no slots are ever skipped. For more details on BABE, please see the [BABE paper](https://research.web3.foundation/Polkadot/protocols/block-production/Babe). @@ -156,8 +101,8 @@ implemented for the {{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama The {{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} Host uses the GRANDPA Finality protocol to finalize blocks. Finality is obtained by consecutive rounds of voting by the validator -nodes. Validators execute GRANDPA finality process in parallel to Block Production as an independent -service. +nodes. Validators execute the GRANDPA finality process in parallel to Block Production as an +independent service. It works in a partially synchronous network model as long as 2/3 of nodes are honest and can cope with 1/5 Byzantine nodes in an asynchronous setting. @@ -166,45 +111,47 @@ A notable distinction is that GRANDPA reaches agreements on chains rather than b speeding up the finalization process, even after long-term network partitioning or other networking failures. -In other words, as soon as more than 2/3 of validators attest to a chain containing a certain block, -all blocks leading up to that one are finalized at once. +In other words, as soon as more than 2/3 of validators attest to a chain containing a particular +block, all blocks leading up to that one are finalized at once. -### Protocol +:::info GRANDPA description and implementation Please refer to [the GRANDPA paper](https://github.com/w3f/consensus/blob/master/pdf/grandpa.pdf) -for a full description of the protocol. +for a full description of the protocol. GRANDPA is implemented as a +[module of the Substrate Frame System](https://github.com/paritytech/polkadot-sdk/blob/master/substrate/frame/grandpa/src/lib.rs). -### Implementation +::: -The -[Substrate implementation of GRANDPA](https://github.com/paritytech/polkadot-sdk/blob/master/substrate/frame/grandpa/src/lib.rs) -is part of Substrate Frame. +### Probabilistic vs. Provable Finality -## Bridging: BEEFY +A pure Nakamoto consensus blockchain that runs PoW is only able to achieve the notion of +_probabilistic finality_ and reach _eventual consensus_. Probabilistic finality means that under +some assumptions about the network and participants, if we see a few blocks building on a given +block, we can estimate the probability that it is final. Eventual consensus means that at some point +in the future, all nodes will agree on the truthfulness of one set of data. This eventual consensus +may take a long time, and will not be able to determine how long it will take ahead of time. +However, finality gadgets such as GRANDPA (GHOST-based Recursive ANcestor Deriving Prefix Agreement) +or Ethereum's Casper FFG (the Friendly Finality Gadget) are designed to give stronger and quicker +guarantees on the finality of blocks - specifically, that they can never be reverted after some +process of Byzantine agreements has taken place. The notion of irreversible consensus is known as +_provable finality._ -The BEEFY (Bridge Efficiency Enabling Finality Yielder) is a secondary protocol to GRANDPA to -support efficient bridging between the -{{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} network (relay chain) and remote, -segregated blockchains, such as Ethereum, which were not built with the -{{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} interchain operability in mind. The -protocol allows participants of the remote network to verify finality proofs created by the -{{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} relay chain validators. In other -words: clients in the Ethereum network should able to verify that the -{{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} network is at a specific state. +In the [GRANDPA paper](https://github.com/w3f/consensus/blob/master/pdf/grandpa.pdf), it is phrased +in this way: -Storing all the information necessary to verify the state of the remote chain, such as the block -headers, is too expensive. BEEFY stores the information in a space-efficient way and clients can -request additional information over the protocol. +:::note -For additional implementation details, check out -[this](https://spec.polkadot.network/#sect-grandpa-beefy) section of the Polkadot Spec. +We say an Oracle A in a protocol is _eventually consistent_ if it returns the same value to all +participants after some unspecified time. + +::: ## Fork Choice Bringing BABE and GRANDPA together, the fork choice of {{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} becomes clear. BABE must always build -on the chain that has been finalized by GRANDPA. When there are forks after the finalized head, BABE -provides probabilistic finality by building on the chain with the most primary blocks. +on the chain that GRANDPA has finalized. BABE provides probabilistic finality when there are forks +after the finalized head by building on the chain with the most primary blocks. ![Best chain choice](../assets/best_chain.png) @@ -213,9 +160,9 @@ with a "1" are primary blocks; those marked with a "2" are secondary blocks. Eve chain is the longest chain on the latest finalized block, it does not qualify because it has fewer primaries at the time of evaluation than the one below it. -# Comparisons +## Comparisons -## Nakamoto consensus +### Nakamoto consensus Nakamoto consensus consists of the longest chain rule using proof of work as its Sybil resistance mechanism and leader election. @@ -224,18 +171,18 @@ Nakamoto consensus only gives us probabilistic finality. Probabilistic finality in the past is only as safe as the number of confirmations it has, or the number of blocks that have been built on top of it. As more blocks are built on top of a specific block in a Proof of Work chain, more computational work has been expended behind this particular chain. However, it does not -guarantee that the chain containing the block will always remain the agreed-upon chain, since an +guarantee that the chain containing the block will always remain the agreed-upon chain since an actor with unlimited resources could potentially build a competing chain and expend enough computational resources to create a chain that did not contain a specific block. In such a situation, the longest chain rule employed in Bitcoin and other proof of work chains would move to this new chain as the canonical one. -## PBFT / Tendermint +### PBFT / Tendermint Please see the [relevant section](learn-comparisons-cosmos.md#consensus) in the Cosmos comparison article. -## Casper FFG +### Casper FFG The two main differences between GRANDPA and Casper FFG are: @@ -243,7 +190,37 @@ The two main differences between GRANDPA and Casper FFG are: - GRANDPA only depends on finalized blocks to affect the fork-choice rule of the underlying block production mechanism -# Resources +## Bridging: BEEFY + +The BEEFY (Bridge Efficiency Enabling Finality Yielder) is a secondary protocol to GRANDPA to +support efficient bridging between the +{{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} network (relay chain) and remote, +segregated blockchains, such as Ethereum, which were not built with the +{{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} interchain operability in mind. The +protocol allows participants of the remote network to verify finality proofs created by the +{{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} relay chain validators. In other +words: clients in the Ethereum network should be able to verify that the +{{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} network is at a specific state. + +Storing all the information necessary to verify the state of the remote chain, such as the block +headers, is too expensive. BEEFY addresses the limitations of GRANDPA finality for certain use +cases, such as bridges to chains like Ethereum, by providing a more lightweight and efficient +finality solution. + +BEEFY operates on top of GRANDPA, utilizing a consensus extension and a light client protocol. This +allows for smaller consensus justifications and efficient communication between nodes. It utilizes +Merkle Mountain Ranges (MMR) as an efficient data structure for storing and transmitting block +headers and signatures to light clients. Payload, signed commitment, and witness data are essential +for verifying finality proofs. + +Overall, BEEFY aims to enhance the efficiency and reliability of cross-chain communication within +the {{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} ecosystem by providing a +lightweight finality solution compatible with a variety of target chains. + +For additional implementation details, see +[the Polkadot Specification](https://spec.polkadot.network/#sect-grandpa-beefy). + +## Resources - [BABE paper](https://research.web3.foundation/Polkadot/protocols/block-production/Babe) - The academic description of the BABE protocol. @@ -253,8 +230,8 @@ The two main differences between GRANDPA and Casper FFG are: implementation and the accompanying [Substrate pallet](https://github.com/paritytech/polkadot-sdk/blob/master/substrate/frame/grandpa/src/lib.rs). - [Block Production and Finalization in Polkadot](https://www.crowdcast.io/e/polkadot-block-production) - - An explanation of how BABE and GRANDPA work together to produce and finalize blocks on Kusama, - with Bill Laboon. + An explanation of how BABE and GRANDPA work together to produce and finalize blocks on Kusama with + Bill Laboon. - [Block Production and Finalization in Polkadot: Understanding the BABE and GRANDPA Protocols](https://www.youtube.com/watch?v=1CuTSluL7v4&t=4s) - An academic talk by Bill Laboon, given at MIT Cryptoeconomic Systems 2020, describing Polkadot's hybrid consensus model in-depth. diff --git a/docs/learn/learn-cryptography.md b/docs/learn/learn-cryptography.md index 9934a86323c0..ea8058286644 100644 --- a/docs/learn/learn-cryptography.md +++ b/docs/learn/learn-cryptography.md @@ -98,8 +98,9 @@ broadcast this certificate via an extrinsic. {{ polkadot: Polkadot :polkadot }}{{ kusama: Kusama :kusama }} uses six session keys: - Authority Discovery: sr25519 -- GRANDPA: ed25519 - BABE: sr25519 +- BEEFY: ecdsa +- GRANDPA: ed25519 - I'm Online: sr25519 - Parachain Assignment: sr25519 - Parachain Validator: ed25519 diff --git a/docs/learn/learn-staking.md b/docs/learn/learn-staking.md index 99da5133e73d..69f5f3a38fc9 100644 --- a/docs/learn/learn-staking.md +++ b/docs/learn/learn-staking.md @@ -45,7 +45,7 @@ Here you will learn about what staking is, why it is important and how it works ## Proof-of-Stake (PoS) -Blockchain networks use [consensus](learn-consensus.md/#why-do-we-need-consensus) mechanisms to +Blockchain networks use [consensus](../general/faq.md#why-do-we-need-consensus) mechanisms to finalize blocks on the chain. Consensus is the process of agreeing on something, in this case, the progression of the blockchain or how blocks are added to the chain. Consensus consists of two actions: diff --git a/kusama-guide/sidebars.js b/kusama-guide/sidebars.js index f8dd6d6ae30a..cf4685e20c5b 100644 --- a/kusama-guide/sidebars.js +++ b/kusama-guide/sidebars.js @@ -304,7 +304,6 @@ module.exports = { "learn/learn-polkadot-host", 'learn/learn-wasm', "learn/learn-runtime-upgrades", - "learn/learn-consensus", ], }, ], @@ -415,6 +414,7 @@ module.exports = { id: 'learn/learn-architecture', }, items: [ + "learn/learn-consensus", { type: "category", label: "Parachains", diff --git a/polkadot-wiki/sidebars.js b/polkadot-wiki/sidebars.js index 15f140e6244e..af56e1e97a72 100644 --- a/polkadot-wiki/sidebars.js +++ b/polkadot-wiki/sidebars.js @@ -314,7 +314,6 @@ module.exports = { "learn/learn-polkadot-host", 'learn/learn-wasm', "learn/learn-runtime-upgrades", - "learn/learn-consensus", ], }, ], @@ -425,6 +424,7 @@ module.exports = { id: 'learn/learn-architecture', }, items: [ + "learn/learn-consensus", { type: "category", label: "Parachains",