Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Second step for refactor distributed query #10

Closed
wants to merge 32 commits into from

Conversation

zhuoyuan-liu
Copy link
Owner

@zhuoyuan-liu zhuoyuan-liu commented Dec 11, 2024

In this step, we would like to refactor the way of completing a distributed query and minimize the write request to the database.

Changelog:

  • Removed TrackExcution function and relevant table
  • Replaced VerifyComplete with CleanupCompletedQueries. In this case, we don't need to check the complete status whenever we receive new data from Osquery. Instead, we check it periodically with the expired function. Since we changed how to fetch the list of target queries, keeping a distributed query in an active state does not affect performance.
  • Update how to get lists of target nodes in API and osctrl-admin. We use the intersection to get a list of targeted nodes. We can easily extend this part to support more tags and unions or other operations.

TODO:

  • Remove everything related to CreateTarget function and relevant database table

@zhuoyuan-liu
Copy link
Owner Author

@javuto Could you please take a look this PR, I have no idea why the pipeline is failing even though I reverted the last commit.

dependabot bot and others added 11 commits December 11, 2024 22:06
Bumps the go_modules group with 1 update in the /users directory: [golang.org/x/crypto](https://github.com/golang/crypto).


Updates `golang.org/x/crypto` from 0.28.0 to 0.31.0
- [Commits](golang/crypto@v0.28.0...v0.31.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  dependency-group: go_modules
...

Signed-off-by: dependabot[bot] <[email protected]>
…/go_modules-5a9c29dde4

Bump golang.org/x/crypto from 0.28.0 to 0.31.0 in /users in the go_modules group across 1 directory
Dependencies bump to prevent build errors
Bumps the go_modules group with 1 update in the /metrics directory: [golang.org/x/crypto](https://github.com/golang/crypto).


Updates `golang.org/x/crypto` from 0.21.0 to 0.31.0
- [Commits](golang/crypto@v0.21.0...v0.31.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
  dependency-group: go_modules
...

Signed-off-by: dependabot[bot] <[email protected]>
…cs/go_modules-5a9c29dde4

Bump golang.org/x/crypto from 0.21.0 to 0.31.0 in /metrics in the go_modules group across 1 directory
Add log level and format configuration options for services
@zhuoyuan-liu zhuoyuan-liu deleted the complete-query branch February 25, 2025 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants