Skip to content

Commit

Permalink
Merge branch 'master' into iomekam-port-allocation
Browse files Browse the repository at this point in the history
  • Loading branch information
iomekam committed Apr 22, 2024
2 parents fbab6d3 + 84de950 commit f4c84ec
Show file tree
Hide file tree
Showing 87 changed files with 3,852 additions and 493 deletions.
20 changes: 11 additions & 9 deletions .github/actions/restore-node/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -131,21 +131,23 @@ runs:
sudo apt-get update
sudo apt-get install libbsd-dev
fi
# Preserve the original package.json to restore it after Endo replacements.
cp -a package.json package.json.orig
# Replace the Endo packages with the ones built from the checked-out branch.
if test -e ~/endo; then
scripts/get-packed-versions.sh ~/endo | scripts/resolve-versions.sh
fi
yarn install
if ! cmp -s <(git cat-file blob HEAD:package.json) package.json; then
# In the event that the package.json has been modified by Endo
# replacements, we need to have a yarn-installed.sum that matches
# the unmodified package.json. As long as we don't explicitly `yarn
# install` anywhere other than in this action and in the bin/agd
# script, we should be able to reuse even Endo-overridden built caches
# successfully.
git checkout HEAD -- package.json
fi
# In the event that the package.json has been modified by Endo
# replacements, we need to have a yarn-installed timestamp that
# corresponds to the unmodified package.json. As long as we don't
# explicitly `yarn install` anywhere other than in this action and in
# the bin/agd script, we should be able to reuse even Endo-overridden
# built caches successfully.
mv package.json.orig package.json
if test -e ~/endo; then
# Stage the redirected `yarn install` consequences.
Expand Down
87 changes: 73 additions & 14 deletions a3p-integration/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Overview

This folder contains an end-to-end integration test executed against a synthetic agoric-3 chain. The test performs a chain software upgrade to the software contained in the enclosing `agoric-sdk` repository, then executes a series of functional tests verifying the upgrade accomplished its goal.
This directory contains an end-to-end integration test executed against a synthetic agoric-3 chain. The test performs a chain software upgrade to the software contained in the enclosing `agoric-sdk` repository, then executes a series of functional tests verifying the upgrade accomplished its goal.

# How to run

Expand Down Expand Up @@ -38,9 +38,17 @@ yarn test --debug -m "proposal-name"

## Package layering

Proposals are packages under the `/a3p-integration/proposals` folder, and are separate from each other, from the `a3p-integration` test environment, and from the enclosing `agoric-sdk` repository. They run inside the docker image, and cannot access SDK code. The are lower case, and executed in lexical order.
Proposals are packages under the `/a3p-integration/proposals` folder, and are
separate from each other, from the `a3p-integration` test environment, and from
the enclosing `agoric-sdk` repository. They run inside the docker image, and
cannot access SDK code. Their names must be lower case.

In the release branches, the end-to-end `a3p-integration` test usually only has a single proposal package named `a:upgrade-NN`, which performs a chain software upgrade proposal to the corresponding upgrade plan. In the `master` branch, there may also be core-eval proposal packages, either before or after the chain software upgrade proposal.
In the release branches, the end-to-end `a3p-integration` test usually only has
a single proposal package named `a:upgrade-NN`, which performs a chain software
upgrade proposal to the corresponding upgrade plan. In the `master` branch, the
next release's changes are staged in `a:upgrade-next`. There may also be
core-eval proposal packages, either before or after the chain software upgrade
proposal.

The details of a proposal package are configured through the `agoricProposal` field of its `package.json`.

Expand All @@ -51,16 +59,36 @@ More details about synthetic-chain proposals can be found in the [`agoric-3-prop
For a chain software upgrade proposal, the `type` is `"Software Upgrade Proposal"`, and the relevant fields are `sdkImageTag`, `planName` and `upgradeInfo`.

- `sdkImageTag` is the docker image tag to use that contains the upgraded chain software. It has a value of `unreleased`, which is the tag for the image that is built from the enclosing `agoric-sdk` repository.

- `planName` is the "upgrade name" included in the proposal which must match the value in the upgraded chain software. In the `master` branch its value is `UNRELEASED_UPGRADE`. In the release branches, it's `agoric-upgrade-NN`.
- `upgradeInfo` contains other details passed to the governance proposal. In particular, the info can have a `coreProposal` field which instruct the chain software to run other core proposals besides the one already configured in the chain software's upgrade handler (see `CoreProposalSteps` in `/golang/cosmos/app/app.go`). This field is likely not relevant for release branches.

For an example, see `a:upgrade-next` in master.
- `upgradeInfo` contains other details passed to the governance proposal. In
particular, it can have a `coreProposals` field which instructs the chain
software to run other core proposals in addition to the one configured in the
chain software's upgrade handler (see `CoreProposalSteps` in
`/golang/cosmos/app/app.go`).
- See **Generating core-eval submissions** below for details.

For an (evolving) example, see `a:upgrade-next` in master.

### Core-eval proposal

The `type` of a core-eval proposal is `"/agoric.swingset.CoreEvalProposal"`, and content is submitted from a `submission` subfolder.
The `type` of a core-eval proposal is `"/agoric.swingset.CoreEvalProposal"`. By
default, the submission in the subdir `submission` is evaluated. The proposal
package can override this by providing an `eval.sh` that is executed instead.
See [run_eval.sh](https://github.com/Agoric/agoric-3-proposals/blob/main/packages/synthetic-chain/public/upgrade-test-scripts/run_eval.sh)

If the proposal is planned to be executed after the chain software upgrade, and
the source of the proposal is in `agoric-sdk`, don't commit the built proposal
because CI willl test that instead of the most recent code.


If the proposal is planned to be executed after the chain software upgrade, and the source of the proposal is in `agoric-sdk`, it's recommended to not check-in the `submission` content in source control and instead generate it automatically when testing. Since proposals cannot access SDK code, a script can be used to generate the `submission` content. Until there is [native support for build scripts in the `synthetic-chain` tool](https://github.com/Agoric/agoric-3-proposals/issues/87), `a3p-integration`'s `build:submissions` step invokes `/script/generate-a3p-submission.sh` in `agoric-sdk` before starting the upgrade test.
Instead, generate it automatically in each test run. Since proposals cannot
access SDK code, a script can be used to generate the `submission` content.
Until there is [native support for build scripts in the `synthetic-chain`
tool](https://github.com/Agoric/agoric-3-proposals/issues/87),
`a3p-integration`'s `build:submissions` step invokes
`generate-a3p-submissions.sh` in `agoric-sdk` before starting the upgrade test.

For core eval proposals executing before the chain software upgrade, the `submission` should be checked in, since bundles built from newer software may not be compatible with older chains.

Expand Down Expand Up @@ -97,13 +125,44 @@ make -C ../packages/deployment docker-build-sdk

## Generating core-eval submissions

Some core-eval proposals `submission` content are generated from the `agoric-sdk`
code, and must be rebuilt every time there is a change. The
`scripts/generate-a3p-submission.sh` script contains commands to generate the
core-eval content and move it to the expected proposal package's submission
directory. It is executed as part of `a3p-integration`'s `build:submissions` step.
Each proposal that requires such a build step should add an `sdk-generate` property
in its `agoricProposal` config.
In a3p-integration, many core-eval proposals' `submission` content has to be
generated from the local `agoric-sdk`, and must be rebuilt every time there is a
change. The `scripts/generate-a3p-submissions.sh` script contains commands to
generate the core-eval content and move it to the expected proposal package's
submission directory. The generation is executed as part of `a3p-integration`'s
`build:submissions` step. Each proposal that requires such a build step should
add an entry to the `sdk-generate` array in the `agoricProposal` section of
`package.json`.

Submissions that don't need to pass in options or generate references to source
bundles can be written directly in a `foo-submission` subdirectory of the
proposal. These would include a .js script, accompanied by a similarly-named
-permit.json file (which can contain just `true` or a complete permission
manifest.) The submission should return the script, which can take
BootstrapPowers as a parameter.

If the submission does require bundle references or other options to be
provided, it should be written as two parts: a core eval (in
`.../vats/proposals`) and a builder for it (in `.../builders/scripts/vats`).

The `generate-a3p-submissions.sh` script reads instructions from
`agoricProposal.sdk-generate` in `package.json`. That field contains a list of
strings, each of which describes a single submission. If there is only one
submission, it can use the default directory name `submission` by specifying
only the name of the script file in `builders/scripts/vars`:
```json
"sdk-generate": ["test-localchain"],
```
If there are multiple submissions, each entry has to specify the script name and
a distinct directory name.
```json
"sdk-generate": [
"probe-zcf-bundle probe-submission",
"updatePriceFeeds priceFeed-submission",
"add-auction newAuction-submission"
],
```


## Building synthetic-chain images

Expand Down
6 changes: 3 additions & 3 deletions a3p-integration/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"private": true,
"agoricSyntheticChain": {
"fromTag": "use-upgrade-13"
"fromTag": "latest"
},
"scripts": {
"build": "yarn run build:sdk && yarn run build:submissions && yarn run build:synthetic-chain",
Expand All @@ -12,8 +12,8 @@
"doctor": "yarn synthetic-chain doctor"
},
"dependencies": {
"@agoric/synthetic-chain": "^0.0.7"
"@agoric/synthetic-chain": "^0.0.10"
},
"packageManager": "[email protected].0",
"packageManager": "[email protected].1",
"license": "Apache-2.0"
}
2 changes: 2 additions & 0 deletions a3p-integration/proposals/a:upgrade-next/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,5 @@ in `agoric-sdk/golang/cosmos/app/app.go`.
This test also includes a core proposal in its `upgradeInfo`. This is executed
after the core proposals defined in the software's upgrade handler. See `agoricProposal`
in `package.json`.

The "binaries" property of `upgradeInfo` is now required since Cosmos SDK 0.46, however it cannot be computed for an unreleased upgrade. To disable the check, `releaseNotes` is set to `false`.
74 changes: 0 additions & 74 deletions a3p-integration/proposals/a:upgrade-next/ante-fees.test.js

This file was deleted.

36 changes: 36 additions & 0 deletions a3p-integration/proposals/a:upgrade-next/exit-reclaim.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
import test from 'ava';
import { $ } from 'execa';
import { execFileSync } from 'node:child_process';
import { makeAgd, waitForBlock } from './synthetic-chain-excerpt.js';

const offerId = 'bad-invitation-15'; // cf. prepare.sh
const from = 'gov1';

test('exitOffer tool reclaims stuck payment', async t => {
const showAndExec = (file, args, opts) => {
console.log('$', file, ...args);
return execFileSync(file, args, opts);
};
const agd = makeAgd({ execFileSync: showAndExec }).withOpts({
keyringBackend: 'test',
});

const addr = await agd.lookup(from);
t.log(from, 'addr', addr);

const getBalance = async target => {
const { balances } = await agd.query(['bank', 'balances', addr]);
const { amount } = balances.find(({ denom }) => denom === target);
return Number(amount);
};

const before = await getBalance('uist');
t.log('uist balance before:', before);

await $`node ./exitOffer.js --id ${offerId} --from ${from}`;

await waitForBlock(2);
const after = await getBalance('uist');
t.log('uist balance after:', after);
t.true(after > before);
});
97 changes: 97 additions & 0 deletions a3p-integration/proposals/a:upgrade-next/exitOffer.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
// Note: limit imports to node modules for portability
import { parseArgs, promisify } from 'node:util';
import { execFile } from 'node:child_process';
import { writeFile, mkdtemp, rm } from 'node:fs/promises';
import { join } from 'node:path';

const options = /** @type {const} */ ({
id: { type: 'string' },
from: { type: 'string' },
bin: { type: 'string', default: '/usr/src/agoric-sdk/node_modules/.bin' },
});

const Usage = `
Try to exit an offer, reclaiming any associated payments.
node exitOffer.js --id ID --from FROM [--bin PATH]
Options:
--id <offer id>
--from <address or key name>
--bin <path to agoric and agd> default: ${options.bin.default}
`;

const badUsage = () => {
const reason = new Error(Usage);
reason.name = 'USAGE';
throw reason;
};

const { stringify: q } = JSON;
// limited to JSON data: no remotables/promises; no undefined.
const toCapData = data => ({ body: `#${q(data)}`, slots: [] });

const { entries } = Object;
/**
* @param {Record<string, string>} obj - e.g. { color: 'blue' }
* @returns {string[]} - e.g. ['--color', 'blue']
*/
const flags = obj =>
entries(obj)
.map(([k, v]) => [`--${k}`, v])
.flat();

const execP = promisify(execFile);

const showAndRun = (file, args) => {
console.log('$', file, ...args);
return execP(file, args);
};

const withTempFile = async (tail, fn) => {
const tmpDir = await mkdtemp('offers-');
const tmpFile = join(tmpDir, tail);
try {
const result = await fn(tmpFile);
return result;
} finally {
await rm(tmpDir, { recursive: true, force: true }).catch(err =>
console.error(err),
);
}
};

const doAction = async (action, from) => {
await withTempFile('offer.json', async tmpOffer => {
await writeFile(tmpOffer, q(toCapData(action)));

const out = await showAndRun('agoric', [
'wallet',
...flags({ 'keyring-backend': 'test' }),
'send',
...flags({ offer: tmpOffer, from }),
]);
return out.stdout;
});
};

const main = async (argv, env) => {
const { values } = parseArgs({ args: argv.slice(2), options });
const { id: offerId, from, bin } = values;
(offerId && from) || badUsage();

env.PATH = `${bin}:${env.PATH}`;
const action = { method: 'tryExitOffer', offerId };
const out = await doAction(action, from);
console.log(out);
};

main(process.argv, process.env).catch(e => {
if (e.name === 'USAGE' || e.code === 'ERR_PARSE_ARGS_UNKNOWN_OPTION') {
console.error(e.message);
} else {
console.error(e);
}
process.exit(1);
});
Loading

0 comments on commit f4c84ec

Please sign in to comment.