Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Override Code:0 with the StatusCode value #66

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ekartsev
Copy link

@ekartsev ekartsev commented Aug 12, 2024

Summary

The Amplitude client is set up like the following:

amplitude.Client
    -> destination.amplitudePlugin
        -> storages.inMemoryEventStorage
        -> internal.AmplitudeHTTPClient
        -> internal.AmplitudeResponseProcessor

The flow works like the following:

  1. Clients call amplitude.Client.Track() to send an event.
  2. The event is added to storages.inMemoryEventStorage (by destination.amplitudePlugin)
  3. Then, destination.amplitudePlugin pulls the events from the store asynchronously and sends them to the Amplitude (server) via internal.AmplitudeHTTPClient.Send()
  4. After, destination.amplitudePlugin waits for the http response and calls internal.AmplitudeResponseProcessor.Process() to transform the response from AmplitudeResponse to AmplitudeProcessorResult
  5. And finally, AmplitudeProcessorResult is used to create types.ExecuteResult and call the client's callback

AmplitudeResponse has 2 status codes and 2 errors:

type AmplitudeResponse struct {
    Status int `json:"-"`            // An HTTP Response Code
    Err error `json:"-"`              // An HTTP Response Err

    Code int `json:"code"`        // Code from the Response Body json
    Error string `json:"error"`.  // Error from the Response Body json
    ...
}

Code field is populated from the json response body.
For example:

{
  "code": 200,
  "events_ingested": 50,
  "payload_size_bytes": 50,
  "server_upload_time": 1396381378123
}

See: https://amplitude.com/docs/apis/analytics/batch-event-upload#successsummary


Q: What happens if an http request fails, there's no response body or the body is not json?

The Code field will be 0.

This leads to many issues:

One.
#64 - a fix, which only takes care of 413

Two.
Customers get Code=0 via the callback.
When this happens the customers don't know how to handle it.
Does it mean we should retry or drop the message?
This leads to a potential data loss.

A few examples of errors:

  • Event reached max retry times 1: code=0, events=...
  • HTTP request failed: Post "https://api2.amplitude.com/batch": unexpected EOF: code=0, events=...

In this change we make internal.AmplitudeResponseProcessor to assign Code from StatusCode (effectively http response status code) if Code=0 (if request failed or json parsing failed).

This will tell the customer the real reason of a failure and the customer will be able to decide what to do with the message.

Testing

  • new unit tests

Checklist

  • Does your PR title have the correct title format?
  • Does your PR have a breaking change?: No

@ekartsev ekartsev marked this pull request as ready for review August 12, 2024 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant