MonitoringFactory does not trigger callbacks for PutObject and GetObject

[VasylHryhorzhevskyi]

Describe the bug

Used C++ SDK version 1.11.407
During the application working there was calls fro requests:
HeadBucket
CreateSession
CreateBucket
ListObjectsV2
DeleteObjects

But for previous version of SDK there was triggered also:
PutObject
GetObject

Also we have moved from cUrl and WinHttp to S3Crt and CrtHttp.

Do you need something else?

Expected Behavior

MonitoringFactory triggers:
PutObject
GetObject

Current Behavior

MonitoringFactory does not trigger:
PutObject
GetObject

Reproduction Steps

1. use MonitoringFactory for the application
2. do PutObject or GetObject
3. check requestName in the MonitoringFactory::OnRequestFailed or MonitoringFactory::OnRequestSucceeded functions calls.

Currently step 3 does not happens.

Possible Solution

No response

Additional Information/Context

Application build for Mac Win and Al2 operations systems.

AWS CPP SDK version used

1.11.407

Compiler and Version used

clang-1500.3.9.4,

Operating System and version

Windows_Server-2022-English-Full-Base-2024.06.13, amzn-ec2-macos-14.6.1-20240814-230108, Amazon Linux 2 AMI (HVM) - Kernel 5.10

[sbera87]

On checking the code, PutObject and GetObject for crt client doesn't invoke the monitoring callback methods. While we explore how to add support for this, please also consider using telemetry provider already available in client configuration.

https://github.com/aws/aws-sdk-cpp/blob/main/src/aws-cpp-sdk-core/include/aws/core/client/ClientConfiguration.h#L417-L421

[sbera87]

On checking the code, PutObject and GetObject for crt client doesn't invoke the monitoring callback methods. While we explore how to add support for this, please also consider using telemetry provider already available in client configuration.

https://github.com/aws/aws-sdk-cpp/blob/main/src/aws-cpp-sdk-core/include/aws/core/client/ClientConfiguration.h#L417-L421

Monitor support for s3 crt client get/put object is underway.

[VasylHryhorzhevskyi]

We use AWS monitoring to get the following information:

1. Status of individual request (re)try (success/failure)
2. Thread ID (or similar information about context to distinguish interleaved log messages from requests that complete in parallel)
3. Request name (PutObject, GetObject, HeadBucket, etc)
4. Number of attempts after which request succeeded or failed
5. HTTP response code
6. Remote IP
7. Whether request is retry-able (see AWSError::ShouldRetry())
8. AWSError::GetErrorType()
9. AWSError::GetExceptionName()
10. AWSError::GetMessage()
11. AWS request ID (see AWSError::GetRequestId())
12. Duration of individual request (re)try and total duration of request (all retries added up)
13. HTTP client metrics (connect latency, ssl latency, etc)
14. Request headers
15. Response headers
16. Response XML payload
17. Request URI (for PutObject and GetObject it includes object name)

@sbera87 could you provide a sample code for telemetry provider that demonstrates how to obtain this information?

[sbera87]

So, telemetry provides tracer (for logging data points) and a meter (for statistics on data points).
You need to use this with the the utils:

aws-sdk-cpp/src/aws-cpp-sdk-core/include/smithy/tracing/TracingUtils.h

Line 21 in c5eb78b

class SMITHY_API TracingUtils {

Span: src/aws-cpp-sdk-core/include/smithy/tracing/TraceSpan.h

Meter: src/aws-cpp-sdk-core/include/smithy/tracing/Meter.h

Example usage in sdk:
Constructor for telemetry provider:

aws-sdk-cpp/src/aws-cpp-sdk-core/source/client/AWSClient.cpp

Line 124 in c5eb78b

m_telemetryProvider(configuration.telemetryProvider ? configuration.telemetryProvider : configuration.configFactories.telemetryProviderCreateFn()),

Usage of meter:

aws-sdk-cpp/generated/src/aws-cpp-sdk-dynamodb/source/DynamoDBClient.cpp

Line 237 in c5eb78b

auto meter = m_clientConfiguration.telemetryProvider->getMeter(this->GetServiceClientName(), {});

How sdk measures http metrics:

aws-sdk-cpp/src/aws-cpp-sdk-core/source/client/AWSClient.cpp

Line 308 in c5eb78b

TracingUtils::EmitCoreHttpMetrics(httpRequest->GetRequestMetrics(),

It does mean that with telemetry provider you need to implement those hooks that are called in the sdk

[sbera87]

Monitoring has been added for the concerned S3 APIs. Will close this for now . If any issue faced, please reopen this or create a new issue.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.