Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add metrics api #1162

Merged
merged 29 commits into from
Sep 5, 2024
Merged

Add metrics api #1162

merged 29 commits into from
Sep 5, 2024

Conversation

malteish
Copy link
Collaborator

@malteish malteish commented Sep 2, 2024

First step of enabling metrics collection.

Changes:

  • introduce 2 new values to delivery service properties: metrics collection interval duration and metrics retention duration
  • count messages received by ds, including size.
  • count notifications sent by ds
  • make counted values available at the /metrics endpoint: https://testing.dm3.network/ds/metrics
  • censor the current collection interval's values to improve privacy

because of concurrency issues, we increment the
metrics using redis atomic incrby. This means we
have to store them in individual keys. Once we do
this, we can set an expiration on each key to
automatically delete old metrics.
@malteish malteish marked this pull request as ready for review September 5, 2024 12:07
@malteish
Copy link
Collaborator Author

malteish commented Sep 5, 2024

I looked into the overhead of the function:

db.countMessage took 4.27 milliseconds, messageProcessor took 390.64 milliseconds. Total time: 394.9124559999909 milliseconds
db.countMessage took 2.14 milliseconds, messageProcessor took 337.57 milliseconds. Total time: 339.70516799998586 milliseconds
db.countMessage took 2.40 milliseconds, messageProcessor took 195.36 milliseconds. Total time: 197.76026800001273 milliseconds
db.countMessage took 1.96 milliseconds, messageProcessor took 456.14 milliseconds. Total time: 458.1027860000031 milliseconds
db.countMessage took 1.88 milliseconds, messageProcessor took 326.22 milliseconds. Total time: 328.10131399999955 milliseconds
db.countMessage took 1.44 milliseconds, messageProcessor took 324.39 milliseconds. Total time: 325.8282449999824 milliseconds
db.countMessage took 1.46 milliseconds, messageProcessor took 397.84 milliseconds. Total time: 399.30192700002226 milliseconds
db.countMessage took 1.68 milliseconds, messageProcessor took 345.11 milliseconds. Total time: 346.79461799998535 milliseconds
db.countMessage took 2.28 milliseconds, messageProcessor took 379.97 milliseconds. Total time: 382.24883900000714 milliseconds
db.countMessage took 1.54 milliseconds, messageProcessor took 357.71 milliseconds. Total time: 359.24879699997837 milliseconds
db.countMessage took 1.66 milliseconds, messageProcessor took 304.92 milliseconds. Total time: 306.58208699998795 milliseconds
db.countMessage took 1.26 milliseconds, messageProcessor took 325.23 milliseconds. Total time: 326.48554900000454 milliseconds
db.countMessage took 1.71 milliseconds, messageProcessor took 184.28 milliseconds. Total time: 185.994085000013 milliseconds
db.countMessage took 1.90 milliseconds, messageProcessor took 316.07 milliseconds. Total time: 317.9682070000272 milliseconds
db.countMessage took 2.21 milliseconds, messageProcessor took 357.79 milliseconds. Total time: 360.0047370000102 milliseconds
db.countMessage took 2.50 milliseconds, messageProcessor took 195.12 milliseconds. Total time: 197.61943799999426 milliseconds
db.countMessage took 3.47 milliseconds, messageProcessor took 342.11 milliseconds. Total time: 345.57568299997365 milliseconds
db.countMessage took 3.74 milliseconds, messageProcessor took 223.62 milliseconds. Total time: 227.3596560000151 milliseconds

happy with that, will keep it as is

Copy link
Collaborator

@AlexNi245 AlexNi245 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@malteish malteish merged commit 0d5cd69 into develop Sep 5, 2024
18 checks passed
@malteish malteish deleted the addMetricsApi branch September 5, 2024 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants