Merge pull request #3 from 40ants/configurable-logging

Work nicely with LOG4SLY and changing levels for packages from Emacs.

docs/changelog.lisp

@@ -11,6 +11,14 @@
                               "LOG4SLY"
                               "LOG4SLYNK"
                               "HTTP"))
+  (0.3.0 2025-01-12
+         "
+## Changed
+
+* Now 40ANTS-LOGGING:SETUP-FOR-BACKEND function and 40ANTS-LOGGING:SETUP-FOR-CLI function set the given level not only for appender but also for a root logger, preventing REPL pollution with all debug logs.
+* Also, 40ANTS-LOGGING:SETUP-FOR-REPL function now uses :DEBUG as default for :LEVEL argument. This way you only need to set needed log level for selected packages using LOG4SLY and Emacs.
+"
+         )
   (0.2.0 2024-11-20
          "
 ## Changed

docs/index.lisp

@@ -47,11 +47,17 @@
                                    "ASDF"
                                    "JSON"
                                    "LOG4CL"
+                                   "LOG4SLY"
                                    "SLY"
+                                   "ERROR"
+                                   "WARN"
+                                   "INFO"
+                                   "DEBUG"
+                                   "IDE"
                                    "STDOUT"
                                    "LOG:CONFIG"
                                    "CLI"
-                                   "40A")
+                                   "40Ants")
                     :external-docs ("https://40ants.com/log4cl-extras/"
                                     "https://40ants.com/slynk/"))
   (40ants-logging system)
@@ -82,9 +88,29 @@ You can install this library from Quicklisp, but you want to receive updates qui
 
 (defsection @usage (:title "Usage")
   "
-This small library encapsulates a logging approach for all 40Ants projects. It provides
+This small library encapsulates a logging approach for all `40Ants` projects. It provides
 a few functions to setup structured logging for two kinds of applications: backend and command-line utility.
 
+# The main idea
+
+The idea of our approach to logging is that an application work in two modes:
+
+- regular;
+- IDE connected.
+
+In regular mode application should log to the standard output or to the file usually using JSON format. These logs should be collected and stored in some kind of log store like ElasticSearch. Usually you want to limit log to store only WARN and ERROR levels.
+
+In the second mode, a developer has connected to the app and wants to be able to see some log outputs in the REPL.
+
+We define two log appenders for these two modes:
+
+- main log appender writes logs in regular mode.
+- repl log appender can be added when REPL is enabled. This is done automatically if you start Slynk using 40ANTS-SLYNK system.
+
+Note, a developer don't need to see all INFO and DEBUG logs but only these logs from some package. So, we keep root logger's level the same as was specified for the main log appender. For example, imagine the main appender was configured to log WARN and INFO, but REPL appender configured to show DEBUG. When you'll connect to the REPL, it will not be cluttered with DEBUG messages from the all packages, instead only WARN and ERROR will be logged to the REPL the same as they will be logged to the main appender. But if you want to debug some package, you can set DEBUG level for this package only using LOG4SLY.
+
+# Details
+
 For a backend you need to call 40ANTS-LOGGING:SETUP-FOR-BACKEND function. It configures LOG4CL to output all logs to STDOUT in JSON format. We are doing this because these days most backends are running in the Docker or Kubernetes where easiest way to collect logs is to capture daemon's STDOUT.
 
 For a command line utilities we are configuring LOG4CL to use plaintext format. Call 40ANTS-LOGGING:SETUP-FOR-CLI to make the job. Why LOG:CONFIG is not enought? LOG:CONFIG uses LOG4CL appenders which are not aware of fields added by structured logging macro LOG4CL-EXTRAS/CONTEXT:WITH-FIELDS.

src/core.lisp

@@ -45,20 +45,32 @@
                     :layout :json
                     :filter level)))
 
-  (let ((children (log4cl::%logger-child-hash log4cl:*root-logger*))
-        (orig-root-logger log4cl:*root-logger*))
-    (log:info "Changing" orig-root-logger)
-
-
+  (let ((children (log4cl::%logger-child-hash log4cl:*root-logger*)))
+
     ;; We need to setup
     (setf (log4cl::%logger-child-hash log4cl:*root-logger*)
           (make-hash-table :test 'equal))
 
     (log4cl-extras/config:setup
-     (list :level :debug
+     ;; Here we set for the root logger
+     ;; the same level as for the main appender.
+     ;; Usually this level will be higher than level
+     ;; for the REPL appender. We are doing this
+     ;; to not show INFO and DEBUG messages in the REPL
+     ;; by default if they are not logged by the main appender.
+     ;; But you can use LOG4SLY to setup more verbose log
+     ;; levels for subcategories. For example, main appender
+     ;; can be configured to log WARNs and REPL appender
+     ;; configured to show DEBUG. But when you'll connect
+     ;; to the REPL, it will not be cluttered with DEBUG
+     ;; messages from the all packages, instead only WARNs and ERRORs
+     ;; will be logged to the REPL the same as they will be logged
+     ;; to the main appender. But if you want to debug some package,
+     ;; you can set DEBUG level for it using LOG4SLY.
+     (list :level level
            :appenders (append *repl-appenders*
                               *core-appenders*)))
-    ;; Here we need to restore children because the might be
+    ;; Here we need to restore children because they might be
     ;; changed interactively using LOG4SLIME or LOG4SLY and we don't
     ;; want to loose these settings:
     (setf (log4cl::%logger-child-hash log4cl:*root-logger*)
@@ -93,7 +105,7 @@
   (values))
 
 
-(defun setup-for-repl (&key (level *level*)
+(defun setup-for-repl (&key (level :debug)
                        (stream *debug-io*))
   "Configures LOG4CL for logging in REPL when you connect to the running lisp image already configured as a backend or CLI application.
 
@@ -108,7 +120,8 @@
 
   (let ((children (log4cl::%logger-child-hash log4cl:*root-logger*)))
     (log4cl-extras/config:setup
-     (list :level :debug
+     (list :level (or *level*
+                      level)
            :appenders (append *repl-appenders*
                               *core-appenders*)))
     ;; Here we need to restore children because the might be