Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Debugger filterx #356

Open
wants to merge 52 commits into
base: main
Choose a base branch
from
Open

Debugger filterx #356

wants to merge 52 commits into from

Commits on Oct 27, 2024

  1. scratch-buffers: do not use ivykis in non-ivykis threads

    scratch_buffers_lazy_update_stats() uses the ivykis time state to check
    if it is time to update the stats about scratch buffers. Do not
    do that if ivykis is not initialized. This may happen in control threads and
    the debugger threads that do use scratch buffers but don't use ivykis.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    1eb3781 View commit details
    Browse the repository at this point in the history
  2. control: make it possible to query if a worker relates to a connection

    A better solution would be to have a connection specific worker list,
    and a list of connections. But for now this suffices for my purposes
    of being able to cancel connection specific workers.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    d14cb21 View commit details
    Browse the repository at this point in the history
  3. control: cancel connection related workers

    Up to now, control worker threads were only cancelled at exit. Truth be
    told we never really detected if the peer has disconnected either.
    
    This patch implements thread cancellation whenever a connection closes,
    to detect the closure of the connection comes in a separate patch.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    273eb9b View commit details
    Browse the repository at this point in the history
  4. control: call app_thread_start/stop from command threads

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    b4020dd View commit details
    Browse the repository at this point in the history
  5. control: add support for passing 3 fds through the control socket

    This will be used to pass over the stdio file descriptors from syslog-ng-ctl
    so we can attach to the syslog-ng process after it was started in the
    background.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    fe854f7 View commit details
    Browse the repository at this point in the history
  6. console: new module to track the interactive connection to a terminal

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    43f28e7 View commit details
    Browse the repository at this point in the history
  7. gprocess: integrate startup with the new console management module

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    aad0567 View commit details
    Browse the repository at this point in the history
  8. mainloop-control: add support for ATTACH STDIO

    This new command allows to reconnect the stdio streams even if syslog-ng
    runs in the background.  syslog-ng-ctl will be able to pass 3 fds to the
    syslog-ng process, which will be duplicated into the standard fds and with
    that syslog-ng will happily start displaying its stdout to that terminal.
    
    The ATTACH command itself is threaded and the control socket is only used to
    indicate that the peers are still alive.  syslog-ng will start sending
    "ALIVE" messages to this stream every second or so.  If sending this packet
    is not successful, the connection is closed and the thread is cancelled.
    Upon cancellation, the stdio fds are restored to point to /dev/null.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    11fcfc8 View commit details
    Browse the repository at this point in the history
  9. syslog-ng-ctl: add support for "attach" command

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    6242056 View commit details
    Browse the repository at this point in the history
  10. control: remove and bump verbosity for some control socket related me…

    …ssages
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    66e575e View commit details
    Browse the repository at this point in the history
  11. gprocess: get rid off the stderr_present variable

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    6dcf9ab View commit details
    Browse the repository at this point in the history
  12. logpipe: uninline log_pipe_queue()

    This is never inlined anyway, but duplicated into all modules using
    log_pipe_queue(), it is then better to just have it in one location.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    1f68f2b View commit details
    Browse the repository at this point in the history
  13. logpipe: eliminate s->queue == NULL trick

    log_pipe_forward_msg() is not inlined anyway, so it will just add an extra branch.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    44a8275 View commit details
    Browse the repository at this point in the history
  14. logpipe: whitespace fix

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    78cf282 View commit details
    Browse the repository at this point in the history
  15. logpipe: move PIF_CONFIG_RELATED check to the call site

    Checking the global variable pipe_single_step_hook is quite complicated as
    it is addressed $rip relative, so let's filter out the config related
    pipes first and remove the same check from the debugger.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    695b3c0 View commit details
    Browse the repository at this point in the history
  16. debugger: add cancellation support to Tracer

    The tracer is responsible with the communication between the debugger
    thread and any worker threads. Up to this point this was not cancellable,
    e.g. once a breakpoint fired, the worker stopped with no way of
    cancelling this.
    
    With the addition to tracer_cancel() pending breakpoints are let go and
    at the same time tracer_wait_for_breakpoint() returns as well with
    a gboolean to indicate whether it was cancelled.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    0031f3e View commit details
    Browse the repository at this point in the history
  17. debugger: add debugger_exit()

    The new debugger_exit() can be called from any thread, triggers the
    cancellation of the tracer and then waits for the debugger thread
    to finish.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    79d2f3c View commit details
    Browse the repository at this point in the history
  18. debugger: rename interactive_thread to debugger_thread

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    f43d0fb View commit details
    Browse the repository at this point in the history
  19. debugger: attach and deattach the single step hook in a synchronized …

    …manner
    
    The debugger installed the pipe_single_step_hook without locking, but that
    only works if we do that at startup. Since I want to be able to
    start the debugger on demand, this needs to sync with any workers that
    might be executing.
    
    On x86 it might be safe to just store the pointer, but other less
    forgiving architectures (e.g. ARM) may not like that. Let's do this
    properly.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    abe1d39 View commit details
    Browse the repository at this point in the history
  20. debugger: add debugger_stop() call alongside debugger_start()

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    4f15ff1 View commit details
    Browse the repository at this point in the history
  21. debugger: make sure that debugger_start_console and debugger_exit() r…

    …un from the main thread
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    d24ee74 View commit details
    Browse the repository at this point in the history
  22. tracer: move the Tracer struct to the implementation file

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    b3ecc34 View commit details
    Browse the repository at this point in the history
  23. debugger: support multi-threaded mode

    The key ingredient to support multi-threaded debugger session is to allow
    the breakpoint sites to submit debugger_stop_on_breakpoint() calls
    in parallel so that these requests are queued and no variables are
    used for the different invocations.
    
    The solution is to introduce the BreakpointSite struct, this is where
    we store state that relates to each distinct breakpoint events.
    
    This struct contains both the state the debugger may want to inspect (e.g.
    msg, pipe) and also  the state related to inter-thread communication
    (e.g. resume_requested).
    
    Each such site instances are queued in an internal queue which is then
    processed by the interactive debugger.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    3a402c3 View commit details
    Browse the repository at this point in the history
  24. syslog-ng-ctl: add support for "attach debugger"

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    ca8b036 View commit details
    Browse the repository at this point in the history
  25. mainloop: do not register signal handlers as exclusive

    As long as we have a single signal registrations, these will be executed
    anyway. IV_SIGNAL_FLAG_EXCLUSIVE are for registrations that want
    to suppress previous registrations.
    
    I intend to use SIGINT in the debugger to break into the debugger to
    allow executing commands even outside of the pipeline (e.g. setting
    breakpoints for the future).
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    de5c76e View commit details
    Browse the repository at this point in the history
  26. tracer: implement interrupt support

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    45672ea View commit details
    Browse the repository at this point in the history
  27. debugger: get into the debugger on SIGINT

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    677d350 View commit details
    Browse the repository at this point in the history
  28. logreader: retry on main_loop_io_worker_job_submit() failure

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    5ba6d4d View commit details
    Browse the repository at this point in the history
  29. console: extract console_printf from gprocess

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    3228a54 View commit details
    Browse the repository at this point in the history
  30. gprocess: add fatal signal logging with a backtrace

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    a136792 View commit details
    Browse the repository at this point in the history
  31. logreader: add exit-on-eof flag()

    With the exit-on-eof flag, you can request syslog-ng to exit whenever the
    first EOF is hit on the specific source, similarly to how the stdin() source
    behaves.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    8949ea5 View commit details
    Browse the repository at this point in the history
  32. cfg-source: add num_lines argument to _print_underlined_source_block()

    Instead of using "NULL" as the indicator for the end of the lines to print,
    just pass the number of entries in the lines array.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    5ce77a2 View commit details
    Browse the repository at this point in the history
  33. cfg-source: refactor source file printing to use line numbers

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    00996cb View commit details
    Browse the repository at this point in the history
  34. cfg-source: add cfg_source_print_source_text() function

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    e02b15a View commit details
    Browse the repository at this point in the history
  35. debugger: add source display using the cfg-source module

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    f263108 View commit details
    Browse the repository at this point in the history
  36. debugger: extract _set_command()

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    34e2e1f View commit details
    Browse the repository at this point in the history
  37. logpipe: add PIF_BREAKPOINT flag

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    f67c4bf View commit details
    Browse the repository at this point in the history
  38. debugger: add debugger mode

    The mode field in debugger determines how we respond to trace and
    breakpoints and is used by the hook to trigger various debugging
    scenarios.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    709de3b View commit details
    Browse the repository at this point in the history
  39. debugger: filter out uninteresting stop events

    Some breakpoints are not interesting once the mode changes, even
    though they were submitted by the pipe hook and are waiting to
    be resumed.
    
    Let's quickly acknowledge them and don't bother the user.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    11144b7 View commit details
    Browse the repository at this point in the history
  40. debugger: clean up trace command

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    bd2cdb6 View commit details
    Browse the repository at this point in the history
  41. debugger: add welcome blurb on startup

    Also, stop at the prompt immediately, instead of wait for a message.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    3415137 View commit details
    Browse the repository at this point in the history
  42. debugger: add step and follow commands

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    d1d8188 View commit details
    Browse the repository at this point in the history
  43. filterx: make FilterXExpr ref_cnt atomic

    This is not a fastpath, so we can use atomics and the debugger needs to
    ref/unref expressions in the debugger thread, which causes failed
    assertions.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    51439b3 View commit details
    Browse the repository at this point in the history
  44. filterx: return list of exprs from the grammar

    Instead of returning a fully formed compound expression, just return
    the list of expressions, so the brace is processed by the parent grammar.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    2f7d3a4 View commit details
    Browse the repository at this point in the history
  45. filterx: don't require filterx expr parsing to run in the main thread

    Exclude other threads simply by only allowing these calls to be
    made while parsing is in progress, which is inherently single-threaded.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    bb6053e View commit details
    Browse the repository at this point in the history
  46. filterx: rename error formatting functions for consistency

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    ca77ae9 View commit details
    Browse the repository at this point in the history
  47. filterx: add filterx_error_format() function

    Which returns a borrowed string instead of an EVTTAG.
    
    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    e7d3cb2 View commit details
    Browse the repository at this point in the history
  48. cfg-source: fix off-by-one in report_buffer_location()

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    92a161a View commit details
    Browse the repository at this point in the history
  49. debugger: add path_options argument to debugger stop/trace entry points

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    8f6124c View commit details
    Browse the repository at this point in the history
  50. debugger: move commands into their separate implementation file

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    1c5f08a View commit details
    Browse the repository at this point in the history
  51. debugger: make the help output a bit more consistent

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    96d0b94 View commit details
    Browse the repository at this point in the history
  52. debugger: implement printx command to evaluate filterx expressions

    Signed-off-by: Balazs Scheidler <[email protected]>
    bazsi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    225043f View commit details
    Browse the repository at this point in the history