Skip to content

use statsd for indexer monitoring #3279

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Oct 15, 2020

Conversation

vladak
Copy link
Member

@vladak vladak commented Oct 14, 2020

This change introduces remote monitoring of indexer via statsd registry in Micrometer. I wanted to use statsd for indexer because it does not require scraping and therefore the need for a web server. This would either conflict with already running web server on the same machine (assuming the indexer is usually run on the same machine as the web application) or would involve poking holes into firewall to allow the scraping to pass through to distinct TCP port.

Example read-only configuration:

<?xml version="1.0" encoding="UTF-8"?>
<java version="11.0.4" class="java.beans.XMLDecoder">
 <object class="org.opengrok.indexer.configuration.Configuration" id="Configuration0">

  <void property="statsdConfig">
     <void property="port">
       <int>8125</int>
     </void>
     <void property="host">
       <string>localhost</string>
     </void>
     <void property="flavor">
       <string>etsy</string>
     </void>
  </void>

 </object>
</java>

This was used to generate the graphs in #3248 (comment) (I used statsd-exporter for Prometheus and Grafana)

The metrics are not tagged unless the indexer is run per project.

During testing I noticed that some of the metrics recorded in the final stages of indexing are not sent out even with buffering set to false and big delay at the end of the Indexer#main. This is not such a big deal given that statsd traffic is usually not reliable anyway (assuming UDP by default). I will definitely investigate.

Once this is merged in I will update https://github.com/oracle/opengrok/wiki/Monitoring with example rig configuration.

Vladimir Kotal and others added 5 commits October 14, 2020 16:02
Co-authored-by: Adam Hornáček <[email protected]>
Co-authored-by: Adam Hornáček <[email protected]>
Co-authored-by: Adam Hornáček <[email protected]>
@coveralls
Copy link

coveralls commented Oct 15, 2020

Pull Request Test Coverage Report for Build 5493

  • 75 of 97 (77.32%) changed or added relevant lines in 12 files are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage decreased (-0.005%) to 75.7%

Changes Missing Coverage Covered Lines Changed/Added Lines %
opengrok-indexer/src/main/java/org/opengrok/indexer/configuration/Configuration.java 5 6 83.33%
opengrok-web/src/main/java/org/opengrok/web/servlet/MetricsServlet.java 0 1 0.0%
opengrok-indexer/src/main/java/org/opengrok/indexer/configuration/RuntimeEnvironment.java 6 8 75.0%
opengrok-web/src/main/java/org/opengrok/web/StatisticsFilter.java 0 4 0.0%
opengrok-indexer/src/main/java/org/opengrok/indexer/Metrics.java 21 35 60.0%
Files with Coverage Reduction New Missed Lines %
opengrok-indexer/src/main/java/org/opengrok/indexer/history/BazaarRepository.java 2 69.0%
Totals Coverage Status
Change from base Build 5478: -0.005%
Covered Lines: 41401
Relevant Lines: 54691

💛 - Coveralls

@vladak vladak merged commit 0d7ace5 into oracle:master Oct 15, 2020
@vladak vladak deleted the monitoring_statsd_indexer_PR branch October 15, 2020 10:20
@vladak
Copy link
Member Author

vladak commented Oct 15, 2020

Thanks for the review !

vladak pushed a commit to vladak/OpenGrok that referenced this pull request Oct 19, 2020
introduce StatsdRegistry

Co-authored-by: Adam Hornáček <[email protected]>

fixes oracle#3245
vladak pushed a commit that referenced this pull request Apr 9, 2021
introduce StatsdRegistry

Co-authored-by: Adam Hornáček <[email protected]>

fixes #3245
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants