Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate to CometBFT 0.37 #2263

Closed
4 tasks done
conorsch opened this issue Mar 27, 2023 · 6 comments
Closed
4 tasks done

Migrate to CometBFT 0.37 #2263

conorsch opened this issue Mar 27, 2023 · 6 comments
Assignees
@conorsch
Milestone
Testnet 62: Iapetus

Comments

@conorsch
Copy link
Contributor

conorsch commented Mar 27, 2023 •
edited
Loading

The Tendermint project has been forked as CometBFT. We should plan to migrate our use Tendermint accordingly. At first, let's aim

Required changes

  • Spike on compatibility by swapping cometbft for tendermint binary, joining the testnet-preview deployment. For now, let's keep the version the on the 0.34.x series.
  • Update testnet generate logic to use new config dir paths.
  • Update container references to use cometbft, rather than tendermint, container images
  • (Later) Update cometbft to latest release series, e.g. 0.37.x. Check whether LTS channel exists.

Relevant docs
@conorsch conorsch self-assigned this Mar 27, 2023
@hdevalence
Copy link
Member

There's currently no LTS channel for CometBFT releases. We can't easily switch to 0.37.x because it changes the ABCI version, so we'll need to update tendermint-rs and tower-abci and also change our Consensus and Mempool implementations to match the new ABCI semantics.

@zbuc zbuc added this to Testnets Mar 31, 2023
@zbuc zbuc moved this to Future in Testnets Mar 31, 2023
@zbuc zbuc moved this from Future to Next in Testnets Mar 31, 2023
@conorsch
Copy link
Contributor Author

@erwanor has some work in flight that's related to this penumbra-zone/tower-abci#25. To be clear, we're not blocked here: we can still proceed with testing compat on the current version. Before we reach for upgrading versions, let's make sure to check compat in e.g. tower-abci.

@plaidfinch plaidfinch moved this from Next to Future in Testnets May 12, 2023
@conorsch conorsch mentioned this issue Jul 28, 2023
Closed
@hdevalence hdevalence changed the title Migrate from Tendermint to CometBFT Migrate to CometBFT 0.37 Sep 11, 2023
@hdevalence
Copy link
Member

Updated issue title to reference a goal of switching to 0.37; the ABCI support should have landed in tower-abci by now.

@conorsch conorsch mentioned this issue Sep 14, 2023
Closed
17 tasks
@conorsch
Copy link
Contributor Author

As of #3033, we are running cometbft v0.34.27 in the preview environment. So far, so good. Updated the checklist. Final remaining task is to increment minor versions, which we'll need to do carefully, making changes to pd and/or tower-abci where necessary.

@erwanor
Copy link
Member

erwanor commented Sep 16, 2023 •
edited by conorsch
Loading

The only changes that we will need to do are to pd, specifically:

  • use the v037 module of tower-abci and tower-trace
  • handle v037 new messages like PrepareProposal, ProcessProposal

We can start with naive implementations for the new messages and refine later, see here

@conorsch conorsch mentioned this issue Sep 19, 2023
Merged
conorsch added a commit that referenced this issue Sep 21, 2023
@conorsch
ci: tendermint -> cometbft renaming
ecadf3b 
Follow-up to #3064. Refs #2263.
conorsch added a commit that referenced this issue Sep 25, 2023
@conorsch
docs: fix cometbft renaming in guide
1154b35 
Refs #2263. Follow-up to #3064.
conorsch added a commit that referenced this issue Sep 27, 2023
@conorsch
docs: update systemd guide for cometbft
19ea69d 
Refs #2263.
@conorsch conorsch mentioned this issue Sep 27, 2023
Merged
conorsch added a commit that referenced this issue Sep 28, 2023
@conorsch
docs: update systemd guide for cometbft
4db2ef3 
Refs #2263.
@conorsch conorsch moved this from Future to In Progress (Already claimed) in Testnets Sep 28, 2023
conorsch added a commit that referenced this issue Sep 28, 2023
@conorsch
feat(pd)!: migrate to cometbft v0.37
9e8f2f7 
Bumps the rust imports for tendermint-rs and tower-abci to use the v037,
rather than v034, modules. The substantive change is the addition of two
new ABCI event types [0]:

  * PrepareProposal
  * ProcessProposal

This code as written compiles. Still to come:

* verify the logic actually conforms to spec
* refactor new consensus logic into self methods
* update docs/deployment to use new version

Refs #2263.

[0] https://github.com/tendermint/tendermint/blob/v0.37.x/spec/abci/abci%2B%2B_tmint_expected_behavior.md#adapting-existing-applications-that-use-abci
@conorsch conorsch mentioned this issue Sep 28, 2023
Merged
conorsch added a commit that referenced this issue Sep 29, 2023
@conorsch
feat(pd)!: migrate to cometbft v0.37
15adbf1 
Bumps the rust imports for tendermint-rs and tower-abci to use the v037,
rather than v034, modules. The substantive change is the addition of two
new ABCI event types [0]:

  * PrepareProposal
  * ProcessProposal

This code as written compiles. Still to come:

* verify the logic actually conforms to spec
* refactor new consensus logic into self methods
* update docs/deployment to use new version

Refs #2263.

[0] https://github.com/tendermint/tendermint/blob/v0.37.x/spec/abci/abci%2B%2B_tmint_expected_behavior.md#adapting-existing-applications-that-use-abci
conorsch added a commit that referenced this issue Sep 29, 2023
@conorsch
feat(pd)!: migrate to cometbft v0.37
71fa39a 
Bumps the rust imports for tendermint-rs and tower-abci to use the v037,
rather than v034, modules. The substantive change is the addition of two
new ABCI event types [0]:

  * PrepareProposal
  * ProcessProposal

This code as written compiles. Still to come:

* verify the logic actually conforms to spec
* refactor new consensus logic into self methods
* update docs/deployment to use new version

Refs #2263.

[0] https://github.com/tendermint/tendermint/blob/v0.37.x/spec/abci/abci%2B%2B_tmint_expected_behavior.md#adapting-existing-applications-that-use-abci
conorsch added a commit that referenced this issue Sep 29, 2023
@conorsch
feat(pd)!: migrate to cometbft v0.37
16f4fc7 
Bumps the rust imports for tendermint-rs and tower-abci to use the v037,
rather than v034, modules. The substantive change is the addition of two
new ABCI event types:

  * PrepareProposal
  * ProcessProposal

The consensus logic has been updated according to spec [0].

Refs #2263.

[0] https://github.com/cometbft/cometbft/blob/v0.37.2/spec/abci/abci%2B%2B_comet_expected_behavior.md#adapting-existing-applications-that-use-abci
conorsch added a commit that referenced this issue Sep 29, 2023
@conorsch
feat(pd)!: migrate to cometbft v0.37
72f9f19 
Bumps the rust imports for tendermint-rs and tower-abci to use the v037,
rather than v034, modules. The substantive change is the addition of two
new ABCI event types:

  * PrepareProposal
  * ProcessProposal

The consensus logic has been updated according to spec [0].

Refs #2263.

[0] https://github.com/cometbft/cometbft/blob/v0.37.2/spec/abci/abci%2B%2B_comet_expected_behavior.md#adapting-existing-applications-that-use-abci
@conorsch
Copy link
Contributor Author

This is done. Docs are updated and preview is now running with v0.37. We'll make sure to call out the version change in the release notes for 62 (#3123).

@github-project-automation github-project-automation bot moved this from In Progress (Already claimed) to Testnet 61: Dione in Testnets Sep 29, 2023
@conorsch conorsch moved this from Testnet 61: Dione to Testnet 62: Iapetus in Testnets Sep 29, 2023
conorsch added a commit that referenced this issue Oct 2, 2023
@conorsch
docs: revert to cometbft v034 for testnet docs
0dcd100 
We've migrated to cometbft v0.37.x on preview, and updated the docs at
the same time, but it's too early for docs changes: those trying to join
the currently active Testnet 61 must still use v0.34.27. When we release
62, we'll update the docs then to match.

Refs #2263.
@conorsch conorsch mentioned this issue Oct 2, 2023
Merged
conorsch added a commit that referenced this issue Oct 2, 2023
@conorsch
docs: revert to cometbft v034 for testnet docs
be6e8aa 
We've migrated to cometbft v0.37.x on preview, and updated the docs at
the same time, but it's too early for docs changes: those trying to join
the currently active Testnet 61 must still use v0.34.27. When we release
62, we'll update the docs then to match.

Refs #2263.
conorsch added a commit that referenced this issue Oct 3, 2023
@conorsch
build(pd): upgrade to tower-abci v0.10.0
54944f5 
While working on #2263, noticed that we aren't using the most recent
version of tower-abci. This change bumps to latest, ahead of upcoming
more substantive changes in v0.11.0 [0]. Opted to preserve the TCP-only
ABCI support for pd out of simplicity.

Also includes mention of `jmt` in the cargo lockfile as a follow-up to
f482b48.

[0] penumbra-zone/tower-abci#34
@conorsch conorsch mentioned this issue Oct 3, 2023
Merged
conorsch added a commit that referenced this issue Oct 3, 2023
@conorsch
build(pd): upgrade to tower-abci v0.10.0
c219e97 
While working on #2263, noticed that we aren't using the most recent
version of tower-abci. This change bumps to latest, ahead of upcoming
more substantive changes in v0.11.0 [0]. Opted to preserve the TCP-only
ABCI support for pd out of simplicity.

Also includes mention of `jmt` in the cargo lockfile as a follow-up to
f482b48.

[0] penumbra-zone/tower-abci#34
conorsch added a commit that referenced this issue Oct 3, 2023
@conorsch
build(pd): upgrade to tower-abci v0.10.0
8d16446 
While working on #2263, noticed that we aren't using the most recent
version of tower-abci. This change bumps to latest, ahead of upcoming
more substantive changes in v0.11.0 [0]. Opted to preserve the TCP-only
ABCI support for pd out of simplicity.

Also includes mention of `jmt` in the cargo lockfile as a follow-up to
f482b48.

[0] penumbra-zone/tower-abci#34
conorsch added a commit that referenced this issue Oct 4, 2023
@conorsch
chore: update cometbft config template
7e51ba7 
As of CometBFT v0.37.x, fastsync has been renamed to blocksync, and soon
it'll be mandatory. To update our config, I ran `cometbft init` and
manually reviewed the diff, to pull in updates, otherwise preserving
our custom values.

Closes #3144. Refs #2263.
@conorsch conorsch mentioned this issue Oct 4, 2023
Merged
@aubrika aubrika added this to the Testnet 62: Iapetus milestone Oct 4, 2023
conorsch added a commit that referenced this issue Oct 4, 2023
@conorsch
chore: update cometbft config template
0ff4bf9 
As of CometBFT v0.37.x, fastsync has been deprecated in favor of blocksync,
but v0.38.x will disregard both options. We can't yet update the value since
we're on v0.37.x still, but I'm still pulling in the config file changes
to comments and explanations created by diffing a newly generated
CometBFT config against our TOML template.

Refs #3144. Refs #2263.
conorsch added a commit that referenced this issue Oct 4, 2023
@conorsch
chore: update cometbft config template
6e80e4b 
As of CometBFT v0.37.x, fastsync has been deprecated in favor of blocksync,
but v0.38.x will disregard both options. We can't yet update the value since
we're on v0.37.x still, but I'm still pulling in the config file changes
to comments and explanations created by diffing a newly generated
CometBFT config against our TOML template.

Refs #3144. Refs #2263.
conorsch added a commit that referenced this issue Oct 5, 2023
@conorsch
chore: update cometbft config template
bb2089f 
As of CometBFT v0.37.x, fastsync has been deprecated in favor of blocksync,
but v0.38.x will disregard both options. We can't yet update the value since
we're on v0.37.x still, but I'm still pulling in the config file changes
to comments and explanations created by diffing a newly generated
CometBFT config against our TOML template.

Refs #3144. Refs #2263.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@conorsch conorsch

Labels
None yet
Projects
No open projects
Testnets
Status: Testnet 62: Iapetus
Milestone
Testnet 62: Iapetus
Development

No branches or pull requests
4 participants
@hdevalence @conorsch @erwanor @aubrika