Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

graphql-ws: optimise the "resolve" routine #548

Merged

Conversation

oliver-sanders
Copy link
Member

@oliver-sanders oliver-sanders commented Jan 10, 2024

  • Partially address
  • The resolve routine is implmented recursively in the graphql-ws library.
  • Because the function is async this results in a large number of async tasks being created when the library is used with large, deeply nested schema.
  • Async tasks have an overhead, above that of regular function calls.
  • For the example in efficiency: investigate bottleneck #547 this resulted in over 10 seconds of overheads.

Results 96% saving!:

  • Before (base_async.resolve):
    • cumtime: 11.24s
    • calls: 268'053
  • After: (resolve.resolve):
    • cumtime - 0.3965s
    • calls: 24

Suggest adding this routine to the cylc-uiserver library for now as the graphql-ws library is not currently maintained (see #333).

The original implementation can be found here: https://github.com/graphql-python/graphql-ws/blob/7ef25ecfaf5390bf1a0cb4023272d8cb074368c2/graphql_ws/base_async.py#L37-L62

TODO:

  • Reviewers, please replicate profiling results.
  • Changelog entry.
  • License legal stuff.

Check List

  • I have read CONTRIBUTING.md and added my name as a Code Contributor.
  • Contains logically grouped changes (else tidy your branch by rebase).
  • Does not contain off-topic changes (use other PRs for other changes).
  • Applied any dependency changes to both setup.cfg (and conda-environment.yml if present).
  • Tests are included (or explain why tests are not needed).
  • CHANGES.md entry included if this is a change that can affect users
  • Cylc-Doc pull request opened if required at cylc/cylc-doc/pull/XXXX.
  • If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

@oliver-sanders oliver-sanders added this to the 1.5.0 milestone Jan 10, 2024
@oliver-sanders oliver-sanders self-assigned this Jan 10, 2024
@oliver-sanders oliver-sanders marked this pull request as draft January 10, 2024 13:31
* Partially addresses cylc#547
* The resolve routine is implmented recursively in the graphql-ws
  library.
* Because the function is async this results in a large number of async
  tasks being created when the library is used with large, deeply nested
  schema.
* Async tasks have an overhead, above that of regular function calls.
* For the example in cylc#547
  this resulted in over 10 seconds of overheads.
@wxtim
Copy link
Member

wxtim commented Jan 10, 2024

IMO Adding it and modifying it : Would be nice if these were separate commits to keep that info in the repo history?

@oliver-sanders oliver-sanders force-pushed the graphql-ws-resolve-optimisation branch from 65b274f to a5d49e2 Compare January 10, 2024 15:54
* Due to inheritance, we were calling
  `resolve(execution_result.data)` twice.
@oliver-sanders oliver-sanders force-pushed the graphql-ws-resolve-optimisation branch from a5d49e2 to 1311fbb Compare January 10, 2024 15:59
@wxtim
Copy link
Member

wxtim commented Jan 10, 2024

That's somewhat satisfying @oliver-sanders !

foo2

Copy link
Member

@wxtim wxtim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Read code. I have views on readability, but I'm happy that the change replicates the functioning of the original. Might be nice to unit test it?
  • Tried it out. Looks like there is a considerable speed gain for larger workflows.

Not pressed the approve button because it's still draft, presumably indicating futher work to come?

Might we unit test this function?

@oliver-sanders oliver-sanders marked this pull request as ready for review January 16, 2024 14:44
@oliver-sanders oliver-sanders requested a review from wxtim January 29, 2024 13:58
@wxtim wxtim self-requested a review January 30, 2024 14:28
Copy link
Member

@wxtim wxtim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Took the liberty of removing the trailing whitespace causeing the style check to fail.
  • Reviewed the changed code.
  • Dave has given it a 👍🏼

Do you want to appoint a second review?

@oliver-sanders
Copy link
Member Author

oliver-sanders commented Jan 30, 2024

The actual diff is trivial and we're choked up ATM so might just slide this through on 1.

@oliver-sanders oliver-sanders merged commit c821ecc into cylc:master Jan 30, 2024
5 of 6 checks passed
@oliver-sanders oliver-sanders deleted the graphql-ws-resolve-optimisation branch January 30, 2024 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants