Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP]refactor: Log component refactoring of Cloudwego #5

Closed
wants to merge 64 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
b196d00
combine logging for kitex and hertz
smx-Morgan Jul 23, 2024
878aba7
combine logging for kitex and hertz V2
smx-Morgan Jul 23, 2024
e1e40e6
combine logging for kitex and hertz V2
smx-Morgan Jul 23, 2024
ae5509f
combine metrics
smx-Morgan Jul 23, 2024
a0fc887
otel V2
smx-Morgan Jul 27, 2024
446d328
logging down
smx-Morgan Jul 28, 2024
4685ebb
logging V2
smx-Morgan Jul 30, 2024
9e71cff
Merge pull request #1 from smx-Morgan/mainV3
smx-Morgan Jul 30, 2024
b3729e2
propagtor V1
smx-Morgan Jul 31, 2024
6597632
propagtor V2
smx-Morgan Aug 1, 2024
89c2ad3
fix
smx-Morgan Aug 1, 2024
f5b2114
可观测组件整合 V2
smx-Morgan Aug 2, 2024
c55b0a3
抽象otel和promth共性逻辑
smx-Morgan Aug 4, 2024
327a8f9
抽象otel和promth共性逻辑 update
smx-Morgan Aug 4, 2024
af34e39
Merge pull request #2 from smx-Morgan/mainV3
smx-Morgan Aug 4, 2024
55b9056
fix bug
smx-Morgan Aug 4, 2024
830ef57
fix Reference Path
smx-Morgan Aug 4, 2024
a0801d1
add promProvider
smx-Morgan Aug 8, 2024
bd21d3f
add labelControl
smx-Morgan Aug 12, 2024
8b63a53
fix bugs
smx-Morgan Aug 12, 2024
0e8ecb7
fix format
smx-Morgan Aug 12, 2024
458b34a
fix CI
smx-Morgan Aug 15, 2024
446e393
fix labelControl name, fix promProvider,promTracer
smx-Morgan Aug 16, 2024
d5dbc22
fix labelControl name, fix promProvider,promTracer
smx-Morgan Aug 16, 2024
9cfcc20
move Measure build to promProvider
smx-Morgan Aug 17, 2024
7b61315
Unified Label Standard
smx-Morgan Aug 19, 2024
7430b81
fix CI
smx-Morgan Aug 19, 2024
2961603
fix CI bugs
smx-Morgan Aug 19, 2024
29621e2
fix CI bugs
smx-Morgan Aug 19, 2024
611aefc
fix CI bug
smx-Morgan Aug 20, 2024
afa535b
fix remove registry from tracer
smx-Morgan Aug 20, 2024
a406cd2
fix CI gofumpt
smx-Morgan Aug 20, 2024
7770544
fix change name to telemetry
smx-Morgan Aug 20, 2024
bbedf5e
fix remove promtracer
smx-Morgan Aug 21, 2024
991e025
fix tracer
smx-Morgan Aug 26, 2024
f1ad26b
fix file_name
smx-Morgan Aug 26, 2024
a9c2581
remove labelcontrol
smx-Morgan Aug 27, 2024
c3889ae
Standardized code
smx-Morgan Aug 30, 2024
e022803
add recorder for retry
smx-Morgan Aug 30, 2024
5a96cf2
update test
smx-Morgan Sep 2, 2024
57a8f2d
fix Data Race
smx-Morgan Sep 2, 2024
0e50950
fix Data Race
smx-Morgan Sep 2, 2024
95901c1
remove promProvider test add example
smx-Morgan Sep 2, 2024
be0e2ea
fix format
smx-Morgan Sep 2, 2024
2358778
move go.mod to telemetry
smx-Morgan Sep 3, 2024
d6cfb97
update makefile
smx-Morgan Sep 3, 2024
0a83f00
update makefile, update file name
smx-Morgan Sep 3, 2024
3de654b
update makefile,update labels
smx-Morgan Sep 4, 2024
3c59ee5
update measure structure
smx-Morgan Sep 7, 2024
971f968
update measure structure
smx-Morgan Sep 8, 2024
f662adb
update formate
smx-Morgan Sep 9, 2024
ced4246
update TelemetryProvider
smx-Morgan Sep 16, 2024
634b01d
update Code Formatting
smx-Morgan Sep 16, 2024
91f7ee3
update build name func
smx-Morgan Sep 16, 2024
7f07ace
Optimize logical structure
smx-Morgan Sep 17, 2024
54b55b6
Support simultaneous monitoring of HTTP and RPC
smx-Morgan Sep 19, 2024
8d527a0
remove measure form promProvider
smx-Morgan Sep 26, 2024
a3d2728
update labels
smx-Morgan Sep 29, 2024
3ccb78c
otel log component compatible with Kitex
smx-Morgan Sep 30, 2024
bc37466
fix path
smx-Morgan Oct 3, 2024
5dde55d
Refactor log
smx-Morgan Oct 4, 2024
6fdcfb6
remove FullLogger
smx-Morgan Oct 5, 2024
0f3992a
fix name
smx-Morgan Oct 5, 2024
ada1b1c
fix ci
smx-Morgan Oct 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
propagtor V1
smx-Morgan committed Jul 31, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit b3729e2000c9be02450a8cc572e974c132aedd65
1 change: 1 addition & 0 deletions cwmetrics/option.go
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
package cwmetrics
File renamed without changes.
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -50,7 +50,7 @@ require (
github.com/fsnotify/fsnotify v1.5.4 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/pprof v0.0.0-20230509042627-b1315fad0c5a // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.18.0 // indirect
2 changes: 1 addition & 1 deletion logging/zerolog/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/cloudwego-contrib/obs-opentelemetry/logging/zerolog

go 1.19
go 1.21

require (
github.com/cloudwego/hertz v0.4.0
11 changes: 6 additions & 5 deletions obs-logging/slog/logger.go
Original file line number Diff line number Diff line change
@@ -28,10 +28,15 @@ func NewLogger(opts ...Option) *Logger {
Logger: *cfg.logger,
config: cfg,
}
logger.SetTraceLogger()
logger.setTraceLogger()
return logger
}

func (l *Logger) setTraceLogger() {
log := slog.New(NewTraceHandler(l.GetOutput(), l.config.logger.GetHandler(), l.config.traceConfig))
l.Logger.SetLogger(log)
}

func (l *Logger) SetOutput(writer io.Writer) {
log := slog.New(NewTraceHandler(writer, l.config.logger.GetHandler(), l.config.traceConfig))
l.config.logger.SetOutput(writer)
@@ -40,7 +45,3 @@ func (l *Logger) SetOutput(writer io.Writer) {
func (l *Logger) SetLevel(level logging.Level) {
l.Logger.SetLevel(level)
}
func (l *Logger) SetTraceLogger() {
log := slog.New(NewTraceHandler(l.GetOutput(), l.config.logger.GetHandler(), l.config.traceConfig))
l.Logger.SetLogger(log)
}
2 changes: 1 addition & 1 deletion tracing/hertz/middleware.go
Original file line number Diff line number Diff line change
@@ -47,7 +47,7 @@ func (sh *StringHeader) Visit(f func(k, v string)) {
}

func ClientMiddleware(opts ...Option) client.Middleware {
cfg := newConfig(opts)
cfg := NewConfig(opts)
histogramRecorder := make(map[string]metric.Float64Histogram)
counters := make(map[string]metric.Int64Counter)

5 changes: 4 additions & 1 deletion tracing/hertz/options.go
Original file line number Diff line number Diff line change
@@ -61,7 +61,7 @@ type Config struct {
shouldIgnore ConditionFunc
}

func newConfig(opts []Option) *Config {
func NewConfig(opts []Option) *Config {
cfg := defaultConfig()

for _, opt := range opts {
@@ -119,6 +119,9 @@ func defaultConfig() *Config {
},
}
}
func (c Config) GetTextMapPropagator() propagation.TextMapPropagator {
return c.textMapPropagator
}

// WithRecordSourceOperation configures record source operation dimension
func WithRecordSourceOperation(recordSourceOperation bool) Option {
4 changes: 2 additions & 2 deletions tracing/hertz/propagator_test.go
Original file line number Diff line number Diff line change
@@ -16,10 +16,10 @@ package hertz

import (
"context"
"github.com/bytedance/gopkg/cloud/metainfo"
"reflect"
"testing"

"github.com/bytedance/gopkg/cloud/metainfo"
"github.com/cloudwego/hertz/pkg/protocol"
"github.com/stretchr/testify/assert"
"go.opentelemetry.io/contrib/propagators/b3"
@@ -74,7 +74,7 @@ func TestExtract(t *testing.T) {
}

func TestInject(t *testing.T) {
cfg := newConfig([]Option{WithTextMapPropagator(propagation.NewCompositeTextMapPropagator(
cfg := NewConfig([]Option{WithTextMapPropagator(propagation.NewCompositeTextMapPropagator(
b3.New(),
ot.OT{},
propagation.Baggage{},
2 changes: 1 addition & 1 deletion tracing/hertz/tracer_server.go
Original file line number Diff line number Diff line change
@@ -45,7 +45,7 @@ type serverTracer struct {
}

func NewServerTracer(opts ...Option) (serverconfig.Option, *Config) {
cfg := newConfig(opts)
cfg := NewConfig(opts)
st := &serverTracer{
config: cfg,
counters: make(map[string]metric.Int64Counter),
4 changes: 2 additions & 2 deletions tracing/kitex/middleware.go
Original file line number Diff line number Diff line change
@@ -28,7 +28,7 @@ import (
)

// ClientMiddleware inject span context into req meta
func ClientMiddleware(cfg *config) endpoint.Middleware {
func ClientMiddleware(cfg *Config) endpoint.Middleware {
return func(next endpoint.Endpoint) endpoint.Endpoint {
return func(ctx context.Context, req, resp interface{}) (err error) {
span := oteltrace.SpanFromContext(ctx)
@@ -53,7 +53,7 @@ func ClientMiddleware(cfg *config) endpoint.Middleware {
}

// ServerMiddleware extract req meta into span context
func ServerMiddleware(cfg *config) endpoint.Middleware {
func ServerMiddleware(cfg *Config) endpoint.Middleware {
return func(next endpoint.Endpoint) endpoint.Endpoint {
return func(ctx context.Context, req, resp interface{}) (err error) {
tc := internal.TraceCarrierFromContext(ctx)
21 changes: 12 additions & 9 deletions tracing/kitex/options.go
Original file line number Diff line number Diff line change
@@ -27,16 +27,16 @@ const (

// Option opts for opentelemetry tracer provider
type Option interface {
apply(cfg *config)
apply(cfg *Config)
}

type option func(cfg *config)
type option func(cfg *Config)

func (fn option) apply(cfg *config) {
func (fn option) apply(cfg *Config) {
fn(cfg)
}

type config struct {
type Config struct {
tracer trace.Tracer
meter metric.Meter

@@ -47,7 +47,7 @@ type config struct {
recordSourceOperation bool
}

func newConfig(opts []Option) *config {
func newConfig(opts []Option) *Config {
cfg := defaultConfig()

for _, opt := range opts {
@@ -67,24 +67,27 @@ func newConfig(opts []Option) *config {
return cfg
}

func defaultConfig() *config {
return &config{
func defaultConfig() *Config {
return &Config{
tracerProvider: otel.GetTracerProvider(),
meterProvider: otel.GetMeterProvider(),
textMapPropagator: otel.GetTextMapPropagator(),
}
}
func (c Config) GetTextMapPropagator() propagation.TextMapPropagator {
return c.textMapPropagator
}

// WithRecordSourceOperation configures record source operation dimension
func WithRecordSourceOperation(recordSourceOperation bool) Option {
return option(func(cfg *config) {
return option(func(cfg *Config) {
cfg.recordSourceOperation = recordSourceOperation
})
}

// WithTextMapPropagator configures propagation
func WithTextMapPropagator(p propagation.TextMapPropagator) Option {
return option(func(cfg *config) {
return option(func(cfg *Config) {
cfg.textMapPropagator = p
})
}
4 changes: 2 additions & 2 deletions tracing/kitex/propagator.go
Original file line number Diff line number Diff line change
@@ -53,12 +53,12 @@ func (m *metadataProvider) Keys() []string {
}

// Inject injects span context into the kitex metadata info
func Inject(ctx context.Context, c *config, metadata map[string]string) {
func Inject(ctx context.Context, c *Config, metadata map[string]string) {
c.textMapPropagator.Inject(ctx, &metadataProvider{metadata: metadata})
}

// Extract returns the baggage and span context
func Extract(ctx context.Context, c *config, metadata map[string]string) (baggage.Baggage, trace.SpanContext) {
func Extract(ctx context.Context, c *Config, metadata map[string]string) (baggage.Baggage, trace.SpanContext) {
ctx = c.textMapPropagator.Extract(ctx, &metadataProvider{metadata: CGIVariableToHTTPHeaderMetadata(metadata)})
return baggage.FromContext(ctx), trace.SpanContextFromContext(ctx)
}
4 changes: 2 additions & 2 deletions tracing/kitex/propagator_test.go
Original file line number Diff line number Diff line change
@@ -37,7 +37,7 @@ func TestExtract(t *testing.T) {

type args struct {
ctx context.Context
c *config
c *Config
metadata map[string]string
}
tests := []struct {
@@ -95,7 +95,7 @@ func TestInject(t *testing.T) {

type args struct {
ctx context.Context
c *config
c *Config
metadata map[string]string
}
tests := []struct {
4 changes: 2 additions & 2 deletions tracing/kitex/tracer_client.go
Original file line number Diff line number Diff line change
@@ -31,11 +31,11 @@ import (
var _ stats.Tracer = (*clientTracer)(nil)

type clientTracer struct {
config *config
config *Config
histogramRecorder map[string]metric.Float64Histogram
}

func newClientOption(opts ...Option) (client.Option, *config) {
func newClientOption(opts ...Option) (client.Option, *Config) {
cfg := newConfig(opts)
ct := &clientTracer{config: cfg}

4 changes: 2 additions & 2 deletions tracing/kitex/tracer_server.go
Original file line number Diff line number Diff line change
@@ -33,11 +33,11 @@ import (
var _ stats.Tracer = (*serverTracer)(nil)

type serverTracer struct {
config *config
config *Config
histogramRecorder map[string]metric.Float64Histogram
}

func newServerOption(opts ...Option) (server.Option, *config) {
func newServerOption(opts ...Option) (server.Option, *Config) {
cfg := newConfig(opts)
st := &serverTracer{
config: cfg,
8 changes: 0 additions & 8 deletions tracing/meter/meter.go

This file was deleted.

117 changes: 117 additions & 0 deletions tracing/propagation/metadataProvider.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
package propagation

import (
"context"
"github.com/bytedance/gopkg/cloud/metainfo"
"github.com/cloudwego-contrib/obs-opentelemetry/tracing/hertz"
"github.com/cloudwego-contrib/obs-opentelemetry/tracing/kitex"
"github.com/cloudwego/hertz/pkg/protocol"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/baggage"
"go.opentelemetry.io/otel/propagation"
"go.opentelemetry.io/otel/trace"
)

var _ propagation.TextMapCarrier = &metadataProviderHttp{}

type metadataProviderHttp struct {
metadata map[string]string
headers *protocol.RequestHeader
}

// Get a value from metadata by key
func (m *metadataProviderHttp) Get(key string) string {
return m.headers.Get(key)
}

// Set a value to metadata by k/v
func (m *metadataProviderHttp) Set(key, value string) {
m.headers.Set(key, value)
}

// Keys Iteratively get all keys of metadata
func (m *metadataProviderHttp) Keys() []string {
out := make([]string, 0, len(m.metadata))

m.headers.VisitAll(func(key, value []byte) {
out = append(out, string(key))
})

return out
}
func getMetadataProviderHTTP(headers *protocol.RequestHeader) *metadataProviderHttp {
return &metadataProviderHttp{headers: headers}
}

// Inject injects span context into the hertz metadata info
func InjectHTTP(ctx context.Context, c *hertz.Config, headers *protocol.RequestHeader) {
c.GetTextMapPropagator().Inject(ctx, getMetadataProviderHTTP(headers))
}

// Extract returns the baggage and span context
func ExtractHTTP(ctx context.Context, c *hertz.Config, headers *protocol.RequestHeader) (baggage.Baggage, trace.SpanContext) {
ctx = c.GetTextMapPropagator().Extract(ctx, getMetadataProviderHTTP(headers))
return baggage.FromContext(ctx), trace.SpanContextFromContext(ctx)
}

var _ propagation.TextMapCarrier = &metadataProviderRPC{}

type metadataProviderRPC struct {
metadata map[string]string
}

// Get a value from metadata by key
func (m *metadataProviderRPC) Get(key string) string {
if v, ok := m.metadata[key]; ok {
return v
}
return ""
}

// Set a value to metadata by k/v
func (m *metadataProviderRPC) Set(key, value string) {
m.metadata[key] = value
}

// Keys Iteratively get all keys of metadata
func (m *metadataProviderRPC) Keys() []string {
out := make([]string, 0, len(m.metadata))
for k := range m.metadata {
out = append(out, k)
}
return out
}
func getMetadataProviderRPC(metadata map[string]string) *metadataProviderRPC {
return &metadataProviderRPC{metadata: metadata}
}

// Inject injects span context into the kitex metadata info
func InjectRPC(ctx context.Context, c *kitex.Config, metadata map[string]string) {
c.GetTextMapPropagator().Inject(ctx, getMetadataProviderRPC(metadata))
}

// Extract returns the baggage and span context
func ExtractRPC(ctx context.Context, c *kitex.Config, metadata map[string]string) (baggage.Baggage, trace.SpanContext) {
ctx = c.GetTextMapPropagator().Extract(ctx, getMetadataProviderRPC(CGIVariableToHTTPHeaderMetadata(metadata)))
return baggage.FromContext(ctx), trace.SpanContextFromContext(ctx)
}

// CGIVariableToHTTPHeaderMetadata converts all CGI variable into HTTP header key.
// For example, `ABC_DEF` will be converted to `abc-def`.
func CGIVariableToHTTPHeaderMetadata(metadata map[string]string) map[string]string {
res := make(map[string]string, len(metadata))
for k, v := range metadata {
res[metainfo.CGIVariableToHTTPHeader(k)] = v
}
return res
}

// ExtractFromPropagator get metadata from propagator
func ExtractFromPropagator(ctx context.Context) map[string]string {
metadata := metainfo.GetAllValues(ctx)
if metadata == nil {
metadata = make(map[string]string)
}
otel.GetTextMapPropagator().Inject(ctx, getMetadataProviderRPC(metadata))
return metadata
}
Loading