Skip to content

Commit

Permalink
update metric Otel conversion
Browse files Browse the repository at this point in the history
  • Loading branch information
narcis96 authored and narcis.gemene committed Sep 10, 2024
2 parents 12de166 + 71ce97e commit 4208d83
Show file tree
Hide file tree
Showing 53 changed files with 911 additions and 715 deletions.
6 changes: 0 additions & 6 deletions receiver/githubreceiver/doc.go

This file was deleted.

140 changes: 0 additions & 140 deletions receiver/githubreceiver/go.mod

This file was deleted.

File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,27 +1,23 @@
# GitHub Receiver
# Git Provider Receiver

<!-- status autogenerated section -->
| Status | |
| ------------- |-----------|
| Stability | [alpha]: metrics |
| Distributions | [contrib] |
| Issues | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aopen%20label%3Areceiver%2Fgithub%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Areceiver%2Fgithub) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aclosed%20label%3Areceiver%2Fgithub%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Areceiver%2Fgithub) |
| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | [@adrielp](https://www.github.com/adrielp), [@andrzej-stencel](https://www.github.com/andrzej-stencel), [@crobert-1](https://www.github.com/crobert-1), [@TylerHelmuth](https://www.github.com/TylerHelmuth) |
| Stability | [development]: metrics |
| Distributions | [] |
| Issues | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aopen%20label%3Areceiver%2Fgitprovider%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Areceiver%2Fgitprovider) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aclosed%20label%3Areceiver%2Fgitprovider%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Areceiver%2Fgitprovider) |
| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | [@adrielp](https://www.github.com/adrielp), [@andrzej-stencel](https://www.github.com/andrzej-stencel), [@crobert-1](https://www.github.com/crobert-1) |

[alpha]: https://github.com/open-telemetry/opentelemetry-collector#alpha
[contrib]: https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib
[development]: https://github.com/open-telemetry/opentelemetry-collector#development
<!-- end autogenerated section -->

The GitHub receiver receives data from [GitHub](https://github.com). As a
starting point it scrapes metrics from repositories but will be extended to
include traces and logs.
The Git Provider receiver scrapes data from Version Control Systems (VCS) that use Git as their underlying technology.

The current default set of metrics can be found in
[documentation.md](./documentation.md).
As a starting point, this receiver can infer many of the same core VCS metrics across vendors, while being able to receive additional data specific to vendors.

These metrics can be used as leading indicators ([capabilities][doracap])
to the [DORA][dorafour] metrics; helping provide insight into modern-day
engineering practices.
The current default set of metrics common across all vendors can be found in [documentation.md](./documentation.md).

These VCS metrics can be used as leading indicators ([capabilities][doracap]) to the [DORA][dorafour] metrics; helping provide insight into modern-day engineering practices.

[doracap]: https://dora.dev/capabilities/
[dorafour]: https://dora.dev/guides/dora-metrics-four-keys/
Expand All @@ -35,7 +31,7 @@ The collection interval is common to all scrapers and is set to 30 seconds by de
> and organizations. You may run into significantly more rate limiting.
```yaml
github:
gitprovider:
collection_interval: <duration> #default = 30s recommended 300s
scrapers:
<scraper1>:
Expand All @@ -51,7 +47,7 @@ extensions:
token: ${env:GH_PAT}

receivers:
github:
gitprovider:
initial_delay: 1s
collection_interval: 60s
scrapers:
Expand All @@ -68,16 +64,23 @@ service:
extensions: [bearertokenauth/github]
pipelines:
metrics:
receivers: [..., github]
receivers: [..., gitprovider]
processors: []
exporters: [...]
```
A Grafana Dashboard exists on the marketplace for metrics from this receiver
and can be found
[here](https://grafana.com/grafana/dashboards/20976-engineering-effectiveness-metrics/).
This receiver is developed upstream in the [liatrio-otel-collector distribution](https://github.com/liatrio/liatrio-otel-collector)
where a quick start exists with an [example config](https://github.com/liatrio/liatrio-otel-collector/blob/main/config/config.yaml)
A Grafana Dashboard exists on the marketplace for this receiver and can be
found [here](https://grafana.com/grafana/dashboards/20976-engineering-effectiveness-metrics/).
## Scraping
The available scrapers are:
| Scraper | Description |
|----------|-------------------------|
| [github] | VCS Metrics from [GitHub](https://github.com/) |
## GitHub Scraper
> Important:
> * The GitHub scraper does not emit metrics for branches that have not had
Expand All @@ -86,9 +89,7 @@ and can be found
> change when rebases occur, recreating the commits with new timestamps.
<!-- TODO: Combine this documentation once the scraper code is restructured due scope change -->
For additional context on GitHub scraper limitations and inner workings please
see the [Scraping README][ghsread].
see the [GitHub Scraper README][ghsread].
[ghsread]: internal/scraper/githubscraper/README.md#github-limitations
[ghsread]: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/gitproviderreceiver/internal/scraper/githubscraper/README.md#github-limitations
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package githubreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/githubreceiver"
package gitproviderreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/gitproviderreceiver"

import (
"errors"
Expand All @@ -11,8 +11,8 @@ import (
"go.opentelemetry.io/collector/confmap"
"go.opentelemetry.io/collector/receiver/scraperhelper"

"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/githubreceiver/internal"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/githubreceiver/internal/metadata"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/gitproviderreceiver/internal"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/gitproviderreceiver/internal/metadata"
)

const (
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package githubreceiver
package gitproviderreceiver

import (
"path/filepath"
Expand All @@ -15,9 +15,9 @@ import (
"go.opentelemetry.io/collector/otelcol/otelcoltest"
"go.opentelemetry.io/collector/receiver/scraperhelper"

"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/githubreceiver/internal"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/githubreceiver/internal/metadata"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/githubreceiver/internal/scraper/githubscraper"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/gitproviderreceiver/internal"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/gitproviderreceiver/internal/metadata"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/gitproviderreceiver/internal/scraper/githubscraper"
)

func TestLoadConfig(t *testing.T) {
Expand All @@ -33,12 +33,12 @@ func TestLoadConfig(t *testing.T) {
require.NoError(t, err)
require.NotNil(t, cfg)

assert.Len(t, cfg.Receivers, 2)
assert.Equal(t, len(cfg.Receivers), 2)

r0 := cfg.Receivers[component.NewID(metadata.Type)]
defaultConfigGitHubScraper := factory.CreateDefaultConfig()
defaultConfigGitHubScraper.(*Config).Scrapers = map[string]internal.Config{
metadata.Type.String(): (&githubscraper.Factory{}).CreateDefaultConfig(),
githubscraper.TypeStr: (&githubscraper.Factory{}).CreateDefaultConfig(),
}

assert.Equal(t, defaultConfigGitHubScraper, r0)
Expand All @@ -50,7 +50,7 @@ func TestLoadConfig(t *testing.T) {
InitialDelay: 1 * time.Second,
},
Scrapers: map[string]internal.Config{
metadata.Type.String(): (&githubscraper.Factory{}).CreateDefaultConfig(),
githubscraper.TypeStr: (&githubscraper.Factory{}).CreateDefaultConfig(),
},
}

Expand Down Expand Up @@ -80,7 +80,7 @@ func TestLoadInvalidConfig_InvalidScraperKey(t *testing.T) {
// nolint:staticcheck
_, err = otelcoltest.LoadConfigAndValidate(filepath.Join("testdata", "config-invalidscraperkey.yaml"), factories)

require.Contains(t, err.Error(), "error reading configuration for \"github\": invalid scraper key: \"invalidscraperkey\"")
require.Contains(t, err.Error(), "error reading configuration for \"gitprovider\": invalid scraper key: \"invalidscraperkey\"")
}

func TestConfig_Unmarshal(t *testing.T) {
Expand Down
6 changes: 6 additions & 0 deletions receiver/gitproviderreceiver/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

//go:generate mdatagen metadata.yaml

package gitproviderreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/gitproviderreceiver"
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[comment]: <> (Code generated by mdatagen. DO NOT EDIT.)

# github
# gitprovider

## Default Metrics

Expand Down
Loading

0 comments on commit 4208d83

Please sign in to comment.