Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-45908: Fix client-side HTTP timeouts when communicating with Butler server #1065

Merged
merged 5 commits into from
Aug 30, 2024

Commits on Aug 29, 2024

  1. Increase timeout for RemoteButler HTTP requests

    The default HTTPX timeout of 5 seconds is much too low when communicating with Butler server -- many requests can exceed that while waiting for the database to respond.
    dhirving committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    595115b View commit details
    Browse the repository at this point in the history
  2. Add keep-alive messages to long-running server queries

    Fetching database results for Butler queries can take an unpredictable amount of time, in some cases several minutes.  To ensure that clients do not time out while waiting, send a keep-alive message to the stream every fifteen seconds.
    dhirving committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    311fece View commit details
    Browse the repository at this point in the history
  3. Remove GZIP middleware from Butler server

    Remove the GzipMiddleware for two reasons:
    1. It was preventing "keep-alive" messages from working in streamed responses, because it was batching them until it had a full gzip chunk.
    2.  It was slowing down queries by about 10% at the RSP -- it turns out that compressing/decompressing is expensive, and there is lots of bandwidth available within Google.
    dhirving committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    670ba4d View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    3d29584 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. Configuration menu
    Copy the full SHA
    7efb63b View commit details
    Browse the repository at this point in the history