Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: get strategies data from DB directly #718

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

wa0x6e
Copy link
Contributor

@wa0x6e wa0x6e commented Oct 18, 2023

🧿 Current issues / What's wrong ?

The strategies query on the hub is using data from space's cache.
This create refresh lag, and dependency on the space cache object.

💊 Fixes / Solution

Use a SQL query to get strategies data from the database

🚧 Changes

  • Get strategies data from the database
  • Remove the strategiesObj global object. The data can already be queried using a simple find on the strategies global variable.

🛠️ Tests

  • Query some strategies and strategy data from graphql => it should return same results as before

@wa0x6e wa0x6e requested a review from ChaituVR October 18, 2023 06:51
const results = new Map(
(
await db.queryAsync(`
SELECT
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be an heavy request isn't? How long does it take to resolve? How often is it called?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

~40ms to resolve, on my dataset of 13k spaces.

It's still not 100% live query, as it's only run along the fetch strategies query from score url, so once every 1min

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want more refactoring, and use 100% live counters (and not cached for 1min) ?

This will bring more consistency, as no other counters are cached and lagging behind.

By enforcing only known strategies when submitting via sequencer, we could also get rid of the condition, checking if the strategy is a valid one from score-api, and have true live data from the strategies operation. (can also get rid of the /helpers/startegies.ts file)

Copy link
Member

@bonustrack bonustrack Oct 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer not using such query as live query, also not sure how you would get ride of strategies helpers, this is loading strategies metadata that are not available from the hub

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

true, forgot about the other strategies data. We get less than 1 request/min on the hub for the strategies object, should have very minimal impact on performance

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Takes around 2.8s on prod, will increase when we get more space 🤔

@wa0x6e wa0x6e added this to the Remove space cache milestone Oct 19, 2023
@wa0x6e wa0x6e marked this pull request as draft November 6, 2023 08:05
@wa0x6e
Copy link
Contributor Author

wa0x6e commented Nov 6, 2023

Return to WIP, will add some cache and update strategy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants