Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cancellation to RPC handlers #846

Draft
wants to merge 2 commits into
base: staging
Choose a base branch
from

Conversation

aryanjassal
Copy link
Contributor

Description

Without proper abortion or cancellation for a RPC, it will keep running and conclude first. This ends up adding a significant amount of time when closing the agent, as all outstanding RPC calls must resolve beforehand.

Issues Fixed

Tasks

  • 1. Add cancellation to RPC
    • AgentLockAll
    • AgentStatus
    • AgentStop
    • AgentUnlock
    • AuditEventsGet
    • AuditMetricGet
    • GestaltsActionsGetByIdentity
    • GestaltsActionsGetByNode
    • GestaltsActionsSetByIdentity
    • GestaltsActionsSetByNode
    • GestaltsActionsUnsetByIdentity
    • GestaltsActionsUnsetByNode
    • GestaltsDiscoveryByIdentity
    • GestaltsDiscoveryByNode
    • GestaltsDiscoveryQueue
    • GestaltsGestaltsGetByIdentity
    • GestaltsGestaltsGetByNode
    • GestaltsGestaltsList
    • GestaltsGestaltsTrustByIdentity
    • GestaltsGestaltsTrustByNode
    • IdentitiesAuthenticate
    • IdentitiesAuthenticatedGet
    • IdentitiesClaim
    • IdentitiesInfoConnectedGet
    • IdentitiesInfoGet
    • IdentitiesInvite
    • IdentitiesProvidersList
    • IdentitiesTokenDelete
    • IdentitiesTokenGet
    • IdentitiesTokenPut
    • KeysCertsChainGet
    • KeysCertsGet
    • KeysDecrypt
    • KeysEncrypt
    • KeysKeyPair
    • KeysKeyPairRenew
    • KeysKeyPairReset
    • KeysPasswordChange
    • KeysPublicKey
    • KeysSign
    • KeysVerify
    • NodesAdd
    • NodesClaim
    • NodesFind
    • NodesGetAll
    • NodesListConnections
    • NodesPing
    • NotificationsInboxClear
    • NotificationsInboxRead
    • NotificationsInboxRemove
    • NotificationsOutboxClear
    • NotificationsOutboxClear
    • NotificationsOutboxRead
    • NotificationsOutboxRemove
    • NotificationsSend
    • VaultsClone
    • VaultsCreate
    • VaultsDelete
    • VaultsList
    • VaultsLog
    • VaultsPermissionGet
    • VaultsPermissionSet
    • VaultsPermissionUnset
    • VaultsPull
    • VaultsRename
    • VaultsScan
    • VaultsSecretsCat
    • VaultsSecretsEnv
    • VaultsSecretsGet
    • VautlsSecretsList
    • VaultsSecretsMkdir
    • VaultsSecretsNew
    • VaultsSecretsNewDir
    • VaultsSecretsRemove
    • VaultsSecretsRename
    • VaultsSecretsStat
    • VaultsSecretsWriteFile
    • VaultsVersion

Final checklist

  • Domain specific tests
  • Full tests
  • Updated inline-comment documentation
  • Lint fixed
  • Squash and rebased
  • Sanity check the final build

@aryanjassal aryanjassal self-assigned this Nov 19, 2024
Copy link

linear bot commented Nov 19, 2024

@aryanjassal
Copy link
Contributor Author

For each RPC handler, we need to provide a cancel, meta, and ctx parameter. The type of cancel is (reason?: any) => void, meta is Record<string, JSONValue> | undefined, and ctx is ContextTimed. The type for ctx is easy to import and use, but it is long and error-prone to have to manually enter the type for cancel and meta.

Should I make a commit into js-rpc staging which exports the types properly, or should I just define it inside Polykey itself?

@tegefaulkes
Copy link
Contributor

Don't bother defining the type for it unless you're actually using it. So you can just name the import parameter as _import it indicate it's not being used. I think I had the types of these parameters being inferred properly at some point. There's some oddity with abstract classes that make the types not work properly here.

@aryanjassal
Copy link
Contributor Author

Don't bother defining the type for it unless you're actually using it. So you can just name the import parameter as _import it indicate it's not being used. I think I had the types of these parameters being inferred properly at some point. There's some oddity with abstract classes that make the types not work properly here.

I just did this to mute the warnings for type hints, as TS assumes it is any by default, and warns me about it. For unused methods, I do indeed prefix it with an underscore to avoid the usage warnings.

const commits = new Set<string>();
const trees = new Set<string>();
const blobs = new Set<string>();
const tags = new Set<string>();
const havesSet: Set<string> = new Set(haves);

async function walk(objectId: ObjectId, type: ObjectType): Promise<void> {
// ctx.signal.throwIfAborted();
// If object was listed as a have then we don't need to walk over it
if (havesSet.has(objectId)) return;
switch (type) {
Copy link
Contributor Author

@aryanjassal aryanjassal Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is used in VaultInternal in garbageCollectGitObjects, which is called when initialising VaultInternal. As such, we don't have access to a context. Should I manually create a context and pass it in, or make context an optional parameter here?

@tegefaulkes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll need to pass the ctx to whatever needs it. For public methods in a class you should use the @timedCancellable decorator. Look at other cancellable methods for reference. For internal methods you can just pass along the ctx.

@aryanjassal aryanjassal marked this pull request as draft November 26, 2024 04:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Review and refactor RPC handlers to handle cancellation properly
2 participants