Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-37973: Display Czar monitoring parameters on the Qserv Web Dashboard #809

Merged
merged 15 commits into from
Sep 21, 2023

Commits on Sep 14, 2023

  1. Configuration menu
    Copy the full SHA
    3b97352 View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2023

  1. Relaxed syntax requirements for the connections strings

    The new implementation allows not to provide database names
    in the connection strings. The connection string parser was
    extended with an additional parameter allowing to privide
    the default database name if the one is found missing in
    the input connection string.
    iagaponenko committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    2575532 View commit details
    Browse the repository at this point in the history
  2. Added configuration parameter (url) for the MySQL proxy

    The change was made to the Configuraton servive of the Replication/Ingest
    system.
    iagaponenko committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    b898453 View commit details
    Browse the repository at this point in the history
  3. Move class replica::AsyncTimer into the utility module

    Refactored dependencies accordingly. The move allows re-using the
    class by other Qserv modules w/o adding a direct depedency onto
    the module 'replica' which was the original location of the class.
    
    Fixed a bug in the implementation of the timer to allow subsequent
    restarts. Unblocking implementation of the callback calling mechanism.
    Extended a model of the callbacks to allow ordering automatic restarts
    of the timer via the return value (boolean type) of the callbacks.
    Minor refactpring of the implementation.
    iagaponenko committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    af7a427 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    6522e9a View commit details
    Browse the repository at this point in the history
  5. Improved and extended implementation of the Czar monitoring

    Extend the Histogram class to report the histogram label (id).
    Added a histogram to capture the performance of the result file
    reads at Czar. Added counters for the number of on-going
    result file reads (file-based result delivery protocol only)
    and merges. Fixed database merge histogram load.
    Added integral counters for metrics reset at the start up time
    of Czar. Added the timestamp (milliseconds) attribute to the JSON
    object that returns a state of the monitoring parameters. The timestamps
    are going to be used for the performance monitoring of Qserv.
    iagaponenko committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    ff22018 View commit details
    Browse the repository at this point in the history
  6. Instrumented Qserv Czar to capture query progess history

    The history is recorded in the transient monitoring store
    (class qserv::CzarStats) from where it could be sampled
    at run time via the Qserv mysql-proxy service.
    iagaponenko committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    102aecf View commit details
    Browse the repository at this point in the history
  7. Processing Czar monitoring requests via MySQL proxy

    The requests are made by calling a special stored procedure
    that is recognized by Qserv. The procedure allows a single parameter
    (a command) that is interpreted by Czar and processed accordingly.
    iagaponenko committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    d3a821f View commit details
    Browse the repository at this point in the history
  8. Temporary fix to the connection management of the query manager

    The fix was introduced after discovering that the shared connection
    usage approach in the original design of this and related classes
    is highly unreliable. A connection shared by the queries seems
    to be sensitive to the service startup ordering and occasional
    disconnects. As a result of that the shared connection could just
    stop working or it could not be alive from the very start of Czar.
    It's also possible that the shared connection object is not properly
    synchronized to allow using it in the multi-threaded environment
    (which would be a bug). The last theory will be further investigated later.
    iagaponenko committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    e49651d View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    54ce44d View commit details
    Browse the repository at this point in the history
  10. Split the service reporting info on both ongoing and past queries

    The service was replaced with two separate services. The first one
    reports info on the ongoing queries. The second service is used for
    searching and displaying info on the past queries.
    iagaponenko committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    6e0d7fc View commit details
    Browse the repository at this point in the history
  11. Web Dashboard: minor refactoring in the page update mechanism

    The improvement aimed at reducing code duplication as well as the total
    amounbt of code in the application.
    iagaponenko committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    1cb1eee View commit details
    Browse the repository at this point in the history
  12. Web Dashboard: split the page displaying info on user queries

    Separated the query into into two pages - the active queries and
    the past queries. Also rearranges sub-tabs under the 'Status' tab
    for more intuitive navigation.
    iagaponenko committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    25e8a97 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    37198d9 View commit details
    Browse the repository at this point in the history
  14. Web Dashboard: added a page for displaying query progress at Czar

    Added a link (an additional column on the active queries table) to
    the newely added Query progression plot.
    Fixed minor layout bugs on the active and past query tables.
    Loading the Highcharts.js library on demand.
    iagaponenko committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    dc3b3b3 View commit details
    Browse the repository at this point in the history