Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a dynamic logger (Fixes #1181) #2245

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Create a dynamic logger (Fixes #1181) #2245

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3e1dcf2
Add debug-related files to .gitignore
derrix060 Oct 30, 2024
55eb191
Add runtime log level control via HTTP (Fix #1181)
derrix060 Oct 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,12 @@ config/
# Default path for Juno DB files. It will get created if you follow the
# README and/or run `./build/juno` command.
/juno/
p2p-dbs
p2p-dbs

# Files created when debugging Juno. It will get created if you use the vscode debugger
*.log
*.sst
MANIFEST-*
OPTIONS-*
CURRENT
LOCK
18 changes: 15 additions & 3 deletions cmd/juno/juno.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@
corsEnableF = "rpc-cors-enable"
versionedConstantsFileF = "versioned-constants-file"
pluginPathF = "plugin-path"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add the logF flag to keep it consistent with the rest of the configuration? Like metricsF, grpcF, or httpF?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdym? I've added this logHostF and logPortF

Do you mean a boolean flag to enable/disable the log host?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeep, but indeed it might be a little bit redundant, but the rest of the configuration includes it. Perhaps it’s better to simplify the others instead

logHostF = "log-host"
logPortF = "log-port"

defaultConfig = ""
defaulHost = "localhost"
Expand Down Expand Up @@ -123,6 +125,7 @@
defaultCorsEnable = false
defaultVersionedConstantsFile = ""
defaultPluginPath = ""
defaultLogPort = 0
wojciechos marked this conversation as resolved.
Show resolved Hide resolved

configFlagUsage = "The YAML configuration file."
logLevelFlagUsage = "Options: trace, debug, info, warn, error."
Expand Down Expand Up @@ -176,6 +179,8 @@
corsEnableUsage = "Enable CORS on RPC endpoints"
versionedConstantsFileUsage = "Use custom versioned constants from provided file"
pluginPathUsage = "Path to the plugin .so file"
logHostUsage = "The interface on which the log level HTTP server will listen for requests."
logPortUsage = "The port on which the log level HTTP server will listen for requests."
)

var Version string
Expand All @@ -202,7 +207,13 @@
return err
}

n, err := node.New(config, Version)
logLevel := utils.NewLogLevel(utils.INFO)
err = logLevel.Set(config.LogLevel)
if err != nil {
return err
}

Check warning on line 214 in cmd/juno/juno.go

View check run for this annotation

Codecov / codecov/patch

cmd/juno/juno.go#L210-L214 

Added lines #L210 - L214 were not covered by tests

n, err := node.New(config, Version, logLevel)

Check warning on line 216 in cmd/juno/juno.go

View check run for this annotation

Codecov / codecov/patch

cmd/juno/juno.go#L216 

Added line #L216 was not covered by tests
if err != nil {
return err
}
Expand Down Expand Up @@ -307,13 +318,12 @@

// For testing purposes, these variables cannot be declared outside the function because Cobra
// may mutate their values.
defaultLogLevel := utils.INFO
defaultNetwork := utils.Mainnet
defaultMaxVMs := 3 * runtime.GOMAXPROCS(0)
defaultCNUnverifiableRange := []int{} // Uint64Slice is not supported in Flags()

junoCmd.Flags().StringVar(&cfgFile, configF, defaultConfig, configFlagUsage)
junoCmd.Flags().Var(&defaultLogLevel, logLevelF, logLevelFlagUsage)
junoCmd.Flags().String(logLevelF, utils.INFO.String(), logLevelFlagUsage)
junoCmd.Flags().Bool(httpF, defaultHTTP, httpUsage)
junoCmd.Flags().String(httpHostF, defaulHost, httpHostUsage)
junoCmd.Flags().Uint16(httpPortF, defaultHTTPPort, httpPortUsage)
Expand Down Expand Up @@ -364,6 +374,8 @@
junoCmd.Flags().String(versionedConstantsFileF, defaultVersionedConstantsFile, versionedConstantsFileUsage)
junoCmd.MarkFlagsMutuallyExclusive(p2pFeederNodeF, p2pPeersF)
junoCmd.Flags().String(pluginPathF, defaultPluginPath, pluginPathUsage)
junoCmd.Flags().String(logHostF, defaulHost, logHostUsage)
junoCmd.Flags().Uint16(logPortF, defaultLogPort, logPortUsage)

junoCmd.AddCommand(GenP2PKeyPair(), DBCmd(defaultDBPath))

Expand Down
48 changes: 39 additions & 9 deletions cmd/juno/juno_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ func TestConfigPrecedence(t *testing.T) {
// tested for sanity. These tests are not intended to perform semantics
// checks on the config, those will be checked by the node implementation.
defaultHost := "localhost"
defaultLogLevel := utils.INFO
defaultLogLevel := "info"
defaultHTTP := false
defaultHTTPPort := uint16(6060)
defaultWS := false
Expand Down Expand Up @@ -83,7 +83,7 @@ func TestConfigPrecedence(t *testing.T) {
"--cn-core-contract-address", "0xc662c410C0ECf747543f5bA90660f6ABeBD9C8c4",
},
expectedConfig: &node.Config{
LogLevel: utils.DEBUG,
LogLevel: "debug",
HTTP: defaultHTTP,
HTTPHost: "0.0.0.0",
HTTPPort: 4576,
Expand All @@ -110,6 +110,8 @@ func TestConfigPrecedence(t *testing.T) {
DBMaxHandles: defaultMaxHandles,
RPCCallMaxSteps: defaultCallMaxSteps,
GatewayTimeout: defaultGwTimeout,
LogHost: defaultHost,
LogPort: 0,
},
},
"custom network config file": {
Expand All @@ -128,7 +130,7 @@ cn-core-contract-address: 0xc662c410C0ECf747543f5bA90660f6ABeBD9C8c4
cn-unverifiable-range: [0,10]
`,
expectedConfig: &node.Config{
LogLevel: utils.DEBUG,
LogLevel: "debug",
HTTP: defaultHTTP,
HTTPHost: "0.0.0.0",
HTTPPort: 4576,
Expand All @@ -155,6 +157,8 @@ cn-unverifiable-range: [0,10]
DBMaxHandles: defaultMaxHandles,
RPCCallMaxSteps: defaultCallMaxSteps,
GatewayTimeout: defaultGwTimeout,
LogHost: defaultHost,
LogPort: 0,
},
},
"default config with no flags": {
Expand Down Expand Up @@ -187,6 +191,8 @@ cn-unverifiable-range: [0,10]
DBMaxHandles: defaultMaxHandles,
RPCCallMaxSteps: defaultCallMaxSteps,
GatewayTimeout: defaultGwTimeout,
LogHost: defaultHost,
LogPort: 0,
},
},
"config file path is empty string": {
Expand Down Expand Up @@ -219,6 +225,8 @@ cn-unverifiable-range: [0,10]
DBMaxHandles: defaultMaxHandles,
RPCCallMaxSteps: defaultCallMaxSteps,
GatewayTimeout: defaultGwTimeout,
LogHost: defaultHost,
LogPort: 0,
},
},
"config file doesn't exist": {
Expand Down Expand Up @@ -256,6 +264,8 @@ cn-unverifiable-range: [0,10]
DBMaxHandles: defaultMaxHandles,
RPCCallMaxSteps: defaultCallMaxSteps,
GatewayTimeout: defaultGwTimeout,
LogHost: defaultHost,
LogPort: 0,
},
},
"config file with all settings but without any other flags": {
Expand All @@ -268,7 +278,7 @@ network: sepolia
pprof: true
`,
expectedConfig: &node.Config{
LogLevel: utils.DEBUG,
LogLevel: "debug",
HTTP: defaultHTTP,
HTTPHost: "0.0.0.0",
HTTPPort: 4576,
Expand All @@ -295,6 +305,8 @@ pprof: true
DBMaxHandles: defaultMaxHandles,
RPCCallMaxSteps: defaultCallMaxSteps,
GatewayTimeout: defaultGwTimeout,
LogHost: defaultHost,
LogPort: 0,
},
},
"config file with some settings but without any other flags": {
Expand All @@ -304,7 +316,7 @@ http-host: 0.0.0.0
http-port: 4576
`,
expectedConfig: &node.Config{
LogLevel: utils.DEBUG,
LogLevel: "debug",
HTTP: defaultHTTP,
HTTPHost: "0.0.0.0",
HTTPPort: 4576,
Expand All @@ -331,6 +343,8 @@ http-port: 4576
DBMaxHandles: defaultMaxHandles,
RPCCallMaxSteps: defaultCallMaxSteps,
GatewayTimeout: defaultGwTimeout,
LogHost: defaultHost,
LogPort: 0,
},
},
"all flags without config file": {
Expand All @@ -339,7 +353,7 @@ http-port: 4576
"--db-path", "/home/.juno", "--network", "sepolia-integration", "--pprof", "--db-cache-size", "1024",
},
expectedConfig: &node.Config{
LogLevel: utils.DEBUG,
LogLevel: "debug",
HTTP: defaultHTTP,
HTTPHost: "0.0.0.0",
HTTPPort: 4576,
Expand All @@ -366,6 +380,8 @@ http-port: 4576
RPCCallMaxSteps: defaultCallMaxSteps,
GatewayTimeout: defaultGwTimeout,
PendingPollInterval: defaultPendingPollInterval,
LogHost: defaultHost,
LogPort: 0,
},
},
"some flags without config file": {
Expand All @@ -374,7 +390,7 @@ http-port: 4576
"--network", "sepolia",
},
expectedConfig: &node.Config{
LogLevel: utils.DEBUG,
LogLevel: "debug",
HTTP: defaultHTTP,
HTTPHost: "0.0.0.0",
HTTPPort: 4576,
Expand All @@ -401,6 +417,8 @@ http-port: 4576
DBMaxHandles: defaultMaxHandles,
RPCCallMaxSteps: defaultCallMaxSteps,
GatewayTimeout: defaultGwTimeout,
LogHost: defaultHost,
LogPort: 0,
},
},
"all setting set in both config file and flags": {
Expand Down Expand Up @@ -433,7 +451,7 @@ db-cache-size: 1024
"--db-cache-size", "9",
},
expectedConfig: &node.Config{
LogLevel: utils.ERROR,
LogLevel: "error",
HTTP: true,
HTTPHost: "127.0.0.1",
HTTPPort: 4577,
Expand All @@ -460,6 +478,8 @@ db-cache-size: 1024
DBMaxHandles: defaultMaxHandles,
RPCCallMaxSteps: defaultCallMaxSteps,
GatewayTimeout: defaultGwTimeout,
LogHost: defaultHost,
LogPort: 0,
},
},
"some setting set in both config file and flags": {
Expand All @@ -471,7 +491,7 @@ network: sepolia
`,
inputArgs: []string{"--db-path", "/home/flag/.juno"},
expectedConfig: &node.Config{
LogLevel: utils.WARN,
LogLevel: "warn",
HTTP: defaultHTTP,
HTTPHost: "0.0.0.0",
HTTPPort: 4576,
Expand All @@ -498,6 +518,8 @@ network: sepolia
DBMaxHandles: defaultMaxHandles,
RPCCallMaxSteps: defaultCallMaxSteps,
GatewayTimeout: defaultGwTimeout,
LogHost: defaultHost,
LogPort: 0,
},
},
"some setting set in default, config file and flags": {
Expand Down Expand Up @@ -532,6 +554,8 @@ network: sepolia
DBMaxHandles: defaultMaxHandles,
RPCCallMaxSteps: defaultCallMaxSteps,
GatewayTimeout: defaultGwTimeout,
LogHost: defaultHost,
LogPort: 0,
},
},
"only set env variables": {
Expand Down Expand Up @@ -564,6 +588,8 @@ network: sepolia
DBMaxHandles: defaultMaxHandles,
RPCCallMaxSteps: defaultCallMaxSteps,
GatewayTimeout: defaultGwTimeout,
LogHost: defaultHost,
LogPort: 0,
},
},
"some setting set in both env variables and flags": {
Expand Down Expand Up @@ -597,6 +623,8 @@ network: sepolia
DBMaxHandles: defaultMaxHandles,
RPCCallMaxSteps: defaultCallMaxSteps,
GatewayTimeout: defaultGwTimeout,
LogHost: defaultHost,
LogPort: 0,
},
},
"some setting set in both env variables and config file": {
Expand Down Expand Up @@ -631,6 +659,8 @@ network: sepolia
DBMaxHandles: defaultMaxHandles,
RPCCallMaxSteps: defaultCallMaxSteps,
GatewayTimeout: defaultGwTimeout,
LogHost: defaultHost,
LogPort: 0,
},
},
}
Expand Down
4 changes: 2 additions & 2 deletions db/pebble/db.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@ func New(path string) (db.DB, error) {
func NewWithOptions(path string, cacheSizeMB uint, maxOpenFiles int, colouredLogger bool) (db.DB, error) {
// Ensure that the specified cache size meets a minimum threshold.
cacheSizeMB = max(cacheSizeMB, minCacheSizeMB)

dbLog, err := utils.NewZapLogger(utils.ERROR, colouredLogger)
log := utils.NewLogLevel(utils.ERROR)
dbLog, err := utils.NewZapLogger(log, colouredLogger)
if err != nil {
return nil, fmt.Errorf("create DB logger: %w", err)
}
Expand Down
28 changes: 28 additions & 0 deletions docs/docs/monitoring.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,3 +85,31 @@ To have Juno write logs to a file, use the command:
![Grafana dashboard](/img/grafana-1.png)

![Grafana dashboard](/img/grafana-2.png)

## Change log level in runtime

In case you want to change the log level in runtime without the need to restart the juno process, you can do it via HTTP calls.

To enable this feature, use the following configuration options:

- `log-host`: The interface to listen for requests. Defaults to `localhost`.
- `log-port`: The port to listen for requests. REQUIRED

Examples:

```console
# Start juno specifying the log port
juno --log-port=6789 --log-level=error ...

# Get current level
curl -X GET 'localhost:6789/log/level'
error

# Set level
curl -X PUT 'localhost:6789/log/level?level=trace'
Replaced log level with 'trace' successfully

# Get new level
curl -X GET 'localhost:6789/log/level'
trace
```
10 changes: 10 additions & 0 deletions node/http.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,16 @@
promhttp.HandlerFor(prometheus.DefaultGatherer, promhttp.HandlerOpts{Registry: prometheus.DefaultRegisterer}))
}

// Create a new service that updates the log level setting.
func makeLogService(host string, port uint16, logLevel *utils.LogLevel) *httpService {
mux := http.NewServeMux()
mux.HandleFunc("/log/level", func(w http.ResponseWriter, r *http.Request) {
utils.HTTPLogSettings(w, r, logLevel)
})
var handler http.Handler = mux
return makeHTTPService(host, port, handler)

Check warning on line 142 in node/http.go

View check run for this annotation

Codecov / codecov/patch

node/http.go#L136-L142 

Added lines #L136 - L142 were not covered by tests
}

type grpcService struct {
srv *grpc.Server
host string
Expand Down
Loading