Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move port allocation in network vat to PortAllocator #9228

Merged
merged 18 commits into from
Apr 30, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions packages/boot/test/bootstrapTests/ibcClientMock.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,14 @@ import { V as E } from '@agoric/vat-data/vow.js';
/**
* @param {ZCF} zcf
* @param {{
* address: string,
* networkVat: ERef<ReturnType<typeof import('@agoric/vats/src/vat-network').buildRootObject>>;
* portAllocator: ERef<PortAllocator>;
* }} privateArgs
* @param {import("@agoric/vat-data").Baggage} _baggage
*/
export const start = async (zcf, privateArgs, _baggage) => {
const { address, networkVat } = privateArgs;
const myPort = await E(networkVat).bindPort(address);
const { portAllocator } = privateArgs;

const myPort = await E(portAllocator).allocateIBCPort();

const { log } = console;
let connP;
Expand Down
6 changes: 3 additions & 3 deletions packages/boot/test/bootstrapTests/ibcServerMock.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@ const { log } = console;
* @param {ZCF} zcf
* @param {{
* address: string,
* networkVat: ERef<ReturnType<typeof import('@agoric/vats/src/vat-network').buildRootObject>>;
* portAllocator: ERef<PortAllocator>;
* }} privateArgs
* @param {import("@agoric/vat-data").Baggage} _baggage
*/
export const start = async (zcf, privateArgs, _baggage) => {
const { address, networkVat } = privateArgs;
const { portAllocator } = privateArgs;

const boundPort = await E(networkVat).bindPort(address);
const boundPort = await E(portAllocator).allocateIBCPort();

/** @type {Array<[label: string, resolve: (value: any) => void, reject: (reason: any) => void]>} */
const queue = [];
Expand Down
6 changes: 3 additions & 3 deletions packages/boot/test/bootstrapTests/test-net-ibc-upgrade.ts
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ const upgradeVats = async (t, EV, vatsToUpgrade) => {
test.serial('upgrade at many points in network API flow', async t => {
const { installation } = t.context;
const { EV } = t.context.runUtils;
const networkVat = await EV.vat('bootstrap').consumeItem('networkVat');
const portAllocator = await EV.vat('bootstrap').consumeItem('portAllocator');
const zoe: ZoeService = await EV.vat('bootstrap').consumeItem('zoe');

const flow = entries({
Expand All @@ -122,7 +122,7 @@ test.serial('upgrade at many points in network API flow', async t => {
installation.ibcServerMock,
{},
{},
{ address: '/ibc-port/', networkVat },
{ portAllocator },
);
t.truthy(started.creatorFacet, `${label} ibcServerMock`);
return [label, { server: started.creatorFacet }];
Expand All @@ -140,7 +140,7 @@ test.serial('upgrade at many points in network API flow', async t => {
installation.ibcClientMock,
{},
{},
{ address: '/ibc-port/', networkVat },
{ portAllocator },
);
t.truthy(started.creatorFacet, `${label} ibcClientMock`);
return [label, { ...opts, client: started.creatorFacet }];
Expand Down
33 changes: 0 additions & 33 deletions packages/boot/test/bootstrapTests/test-vats-restart.ts
Original file line number Diff line number Diff line change
Expand Up @@ -105,20 +105,6 @@ test.serial('run network vat proposal', async t => {
t.pass(); // reached here without throws
});

test.serial('register network protocol before upgrade', async t => {
const { EV } = t.context.runUtils;
const net = await EV.vat('bootstrap').consumeItem('networkVat');
const h1 = await EV(net).makeLoopbackProtocolHandler();

t.log('register P1');
await EV(net).registerProtocolHandler(['P1'], h1);

t.log('register P1 again? No.');
await t.throwsAsync(EV(net).registerProtocolHandler(['P1'], h1), {
message: /key "P1" already registered/,
});
});

test.serial('make IBC callbacks before upgrade', async t => {
const { EV } = t.context.runUtils;
const vatStore = await EV.vat('bootstrap').consumeItem('vatStore');
Expand Down Expand Up @@ -155,25 +141,6 @@ test.serial('use IBC callbacks after upgrade', async t => {
t.truthy(h.bridgeHandler, 'bridgeHandler');
});

test.serial('networkVat registrations are durable', async t => {
const { EV } = t.context.runUtils;
const net = await EV.vat('bootstrap').consumeItem('networkVat');

const h2 = await EV(net).makeLoopbackProtocolHandler();
t.log('register P1 again? No.');
await t.throwsAsync(EV(net).registerProtocolHandler(['P1'], h2), {
message: /key "P1" already registered/,
});

t.log('IBC protocol handler already registered?');
await t.throwsAsync(
EV(net).registerProtocolHandler(['/ibc-port', '/ibc-hop'], h2),
{
message: /key "\/ibc-port" already registered in collection "prefix"/,
},
);
});

test.serial('read metrics', async t => {
const { EV } = t.context.runUtils;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,6 @@ export const makeMock = log =>

network: Far('network', {
registerProtocolHandler: noop,
bindPort: () => harden({ addListener: noop }),
}),
},
});
Expand Down
11 changes: 4 additions & 7 deletions packages/network/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,15 +57,12 @@ E(home.ibcport[0]).connect(remoteEndpoint, connectionHandler)

The other side of `connect()` is a "listening port". These ports are waiting for inbound connections to be established.

To get a listening port, you need a `NetworkInterface` object (such as the one on your `ag-solo` under `home.network`) and ask it to `bindPort()` to an endpoint. You can either provide a specific port name, or allow the API to allocate a random one for you. The endpoint specifies the type of connection that this port will be able to accept (IBC, TCP, etc), and some properties of that connection. `bindPort()` uses a "multiaddress" to encode this information.
To get a listening port, you need a `NetworkInterface` object (such as the one on your `ag-solo` under `home.network`) and ask it for a port, via the `PortAllocator`.

```js
// ask for a random allocation - ends with a slash
E(home.network).bindPort('/ibc-port/')
.then(port => usePort(port));

// or ask for a specific port name
E(home.network).bindPort('/ibc-port/my-cool-port-name')
E(home.network).getPortAllocator()
.then(portAllocator => E(portAllocator).allocateIBCPort())
.then(port => usePort(port));
```

Expand Down Expand Up @@ -147,7 +144,7 @@ Note that if you want to listen on this port again, you can just call `port.addL

### Closing the Port Entirely

Removing a listener doesn't release the port address to make it available for other `bindPort()` requests. You can call:
Removing a listener doesn't release the port address to make it available for other `PortAllocator` requests. You can call:

```js
port.revoke();
Expand Down
43 changes: 43 additions & 0 deletions packages/network/src/network.js
Original file line number Diff line number Diff line change
Expand Up @@ -1426,3 +1426,46 @@ export function prepareLoopbackProtocolHandler(zone, { watch, allVows }) {

return makeLoopbackProtocolHandler;
}

/**
*
* @param {import('@agoric/base-zone').Zone} zone
* @param {ReturnType<import('@agoric/vow').prepareVowTools>} powers
*/
export const preparePortAllocator = (zone, { watch }) =>
zone.exoClass(
'PortAllocator',
M.interface('PortAllocator', {
allocateIBCPort: M.callWhen().returns(Shape.Vow$(Shape.Port)),
allocateICAControllerPort: M.callWhen().returns(Shape.Vow$(Shape.Port)),
allocateIBCPegasusPort: M.callWhen().returns(Shape.Vow$(Shape.Port)),
allocateLocalPort: M.callWhen().returns(Shape.Vow$(Shape.Port)),
}),
({ protocol }) => ({ protocol, lastICAPortNum: 0n }),
{
allocateIBCPort() {
const { state } = this;
// Allocate an IBC port with a unique generated name.
return watch(E(state.protocol).bindPort(`/ibc-port/`));
},
allocateICAControllerPort() {
const { state } = this;
state.lastICAPortNum += 1n;
return watch(
E(state.protocol).bindPort(
`/ibc-port/icacontroller-${state.lastICAPortNum}`,
),
);
},
allocateIBCPegasusPort() {
const { state } = this;
// Allocate a Pegasus IBC port with a unique generated name.
iomekam marked this conversation as resolved.
Show resolved Hide resolved
return watch(E(state.protocol).bindPort(`/ibc-port/pegasus`));
},
allocateLocalPort() {
const { state } = this;
// Allocate a local port with a unique generated name.
return watch(E(state.protocol).bindPort(`/local/`));
},
},
);
7 changes: 7 additions & 0 deletions packages/network/src/types.js
Original file line number Diff line number Diff line change
Expand Up @@ -188,3 +188,10 @@
* ) => PromiseVow<Connection>} outbound
* Create an outbound connection
*/

/**
* @typedef {object} PortAllocator
* @property {() => PromiseVow<Port>} allocateIBCPort
* @property {() => PromiseVow<Port>} allocateICAControllerPort
* @property {() => PromiseVow<Port>} allocateIBCPegasusPort
*/
iomekam marked this conversation as resolved.
Show resolved Hide resolved
33 changes: 33 additions & 0 deletions packages/network/test/test-network-misc.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import {
prepareRouter,
prepareLoopbackProtocolHandler,
prepareNetworkProtocol,
preparePortAllocator,
} from '../src/index.js';

import '../src/types.js';
Expand Down Expand Up @@ -167,6 +168,38 @@ test('handled protocol', async t => {
await when(port.revoke());
});

test('verify port allocation', async t => {
const zone = makeDurableZone(
provideBaggage('network-verify-port-allocation'),
);
const powers = prepareVowTools(zone);
const { when } = powers;
const makeNetworkProtocol = prepareNetworkProtocol(zone, powers);
const makeEchoConnectionHandler = prepareEchoConnectionKit(zone);
const makeProtocolHandler = prepareProtocolHandler(
zone,
t,
makeEchoConnectionHandler,
powers,
);
const protocol = makeNetworkProtocol(makeProtocolHandler());
const makePortAllocator = preparePortAllocator(zone, powers);
const portAllocator = makePortAllocator({ protocol });

const ibcPort = await when(portAllocator.allocateIBCPort());
t.is(ibcPort.getLocalAddress(), '/ibc-port/1');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look like the correct result (the IBC standard states that port identifiers need to be at least 2 characters in length). The local address should be /ibc-port/port-1. Maybe a port- prefix got dropped somewhere in the PRs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We actually mock the generatePortID method, and in that implementation, we drop the port prefix. I've updated it to be inline with what we do in production


const icaControllerPort1 = await when(
portAllocator.allocateICAControllerPort(),
);
t.is(icaControllerPort1.getLocalAddress(), '/ibc-port/icacontroller-1');

const icaControllerPort2 = await when(
portAllocator.allocateICAControllerPort(),
);
t.is(icaControllerPort2.getLocalAddress(), '/ibc-port/icacontroller-2');
});

test('protocol connection listen', async t => {
const zone = makeDurableZone(provideBaggage('network-protocol-connection'));
const powers = prepareVowTools(zone);
Expand Down
14 changes: 5 additions & 9 deletions packages/orchestration/src/orchestration.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import { makeTxPacket, parsePacketAck } from './utils/tx.js';
import '@agoric/network/exported.js';

/**
* @import { AttenuatedNetwork } from './types.js';
* @import { AttenuatedPortAllocator } from './types.js';
* @import { IBCConnectionID } from '@agoric/vats';
* @import { Zone } from '@agoric/base-zone';
* @import { TxBody } from '@agoric/cosmic-proto/cosmos/tx/v1beta1/tx.js';
Expand All @@ -25,7 +25,7 @@ const trace = makeTracer('Orchestration');

/**
* @typedef {object} OrchestrationPowers
* @property {ERef<AttenuatedNetwork>} network
* @property {ERef<AttenuatedPortAllocator>} portAllocator
*/

/**
Expand Down Expand Up @@ -203,17 +203,13 @@ const prepareOrchestration = (zone, createChainAccount) =>
powers.init(/** @type {keyof OrchestrationPowers} */ (name), power);
}
}
return { powers, icaControllerNonce: 0 };
return { powers };
},
{
self: {
async bindPort() {
const network = getPower(this.state.powers, 'network');
const port = await E(network).bindPort(
`/ibc-port/icacontroller-${this.state.icaControllerNonce}`,
);
this.state.icaControllerNonce += 1;
return port;
const portAllocator = getPower(this.state.powers, 'portAllocator');
return E(portAllocator).allocateICAControllerPort();
},
},
public: {
Expand Down
22 changes: 11 additions & 11 deletions packages/orchestration/src/proposals/orchestration-proposal.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@
import { V as E } from '@agoric/vat-data/vow.js';
import { Far } from '@endo/far';

/** @import { AttenuatedNetwork, Orchestration, OrchestrationVat } from '../types' */
/** @import { AttenuatedPortAllocator, Orchestration, OrchestrationVat } from '../types' */

/**
* @param {BootstrapPowers & {
* consume: {
* loadCriticalVat: VatLoader<any>;
* networkVat: NetworkVat;
* portAllocator: AttenuatedPortAllocator;
* };
* produce: {
* orchestration: Producer<any>;
Expand All @@ -25,7 +25,7 @@ import { Far } from '@endo/far';
*/
export const setupOrchestrationVat = async (
{
consume: { loadCriticalVat, networkVat },
consume: { loadCriticalVat, portAllocator },
produce: {
orchestrationVat,
orchestration,
Expand All @@ -45,17 +45,17 @@ export const setupOrchestrationVat = async (
orchestrationVat.reset();
orchestrationVat.resolve(vats.orchestration);

await networkVat;
/** @type {AttenuatedNetwork} */
const network = Far('Attenuated Network', {
/** @param {string} localAddr */
async bindPort(localAddr) {
return E(networkVat).bindPort(localAddr);
await portAllocator;

/** @type {AttenuatedPortAllocator} */
const allocator = Far('PortAllocator', {
async allocateICAControllerPort() {
return E(portAllocator).allocateICAControllerPort();
},
});

const newOrchestrationKit = await E(vats.orchestration).makeOrchestration({
network,
portAllocator: allocator,
});

orchestration.reset();
Expand Down Expand Up @@ -84,7 +84,7 @@ export const getManifestForOrchestration = (_powers, { orchestrationRef }) => ({
[setupOrchestrationVat.name]: {
consume: {
loadCriticalVat: true,
networkVat: true,
portAllocator: 'portAllocator',
},
produce: {
orchestration: 'orchestration',
Expand Down
7 changes: 5 additions & 2 deletions packages/orchestration/src/types.d.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
import type { RouterProtocol } from '@agoric/network/src/router';
import type { PortAllocator } from '@agoric/network/src/network';

export type AttenuatedNetwork = Pick<RouterProtocol, 'bindPort'>;
export type AttenuatedPortAllocator = Pick<
PortAllocator,
'allocateICAControllerPort'
>;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can get rid of this typedef (and Far('PortAllocator', ...) in orchestration-proposal.js) and just use PortAllocator directly. Feel free to leave as is and we can make this change in the course of #9198 where we'll need to add allocateICQControllerPort.

export type * from './orchestration.js';
export type * from './vat-orchestration.js';
Expand Down
6 changes: 3 additions & 3 deletions packages/pegasus/src/proposals/core-proposal.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ export const getManifestForPegasus = ({ restoreRef }, { pegasusRef }) => ({
},
},
listenPegasus: {
consume: { networkVat: t, pegasusConnectionsAdmin: t, zoe: t },
consume: { portAllocator: t, pegasusConnectionsAdmin: t, zoe: t },
produce: { pegasusConnections: t, pegasusConnectionsAdmin: t },
instance: {
consume: { [CONTRACT_NAME]: t },
Expand Down Expand Up @@ -93,7 +93,7 @@ export const addPegasusTransferPort = async (
harden(addPegasusTransferPort);

export const listenPegasus = async ({
consume: { networkVat, pegasusConnectionsAdmin: pegasusNameAdmin, zoe },
consume: { portAllocator, pegasusConnectionsAdmin: pegasusNameAdmin, zoe },
produce: { pegasusConnections, pegasusConnectionsAdmin },
instance: {
consume: { [CONTRACT_NAME]: pegasusInstance },
Expand All @@ -104,7 +104,7 @@ export const listenPegasus = async ({
pegasusConnectionsAdmin.resolve(nameAdmin);

const pegasus = await E(zoe).getPublicFacet(pegasusInstance);
const port = await E(networkVat).bindPort('/ibc-port/pegasus');
const port = await E(portAllocator).allocateIBCPegasusPort();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes me somewhat uncomfortable. It seems to illustrate that allocateIBCPegasusPort() is too specific and doesn't really solve the need for custom port names. Is there a solution that exists between the power of bindPort() and the rigidity of allocateIBCPegasusPort()?

return addPegasusTransferPort(port, pegasus, pegasusNameAdmin);
};
harden(listenPegasus);
Loading
Loading