Keep running across dns resolution failure

[MrPresent-Han]

version: 1.9.234
client: S3CrtClient
server: Local minio
scheme: HTTP
problem: cannot download files due to dns resoution failure

My code can list-bucket, list objects, put object, but it cannot GetObject, and the error are as above, due to dns resolustion failure

I set up my S3CrtClient with such code;
`static const char* ALLOCATION_TAG = "BucketAndObjectOperationTest";
Aws::S3Crt::ClientConfiguration s3ClientConfig;
s3ClientConfig.region = "us-east-1";
s3ClientConfig.scheme = Aws::Http::Scheme::HTTP;
s3ClientConfig.executor = Aws::MakeSharedAws::Utils::Threading::PooledThreadExecutor(ALLOCATION_TAG, 4);
s3ClientConfig.throughputTargetGbps = 2.0;
s3ClientConfig.partSize = 5 * 1024 * 1024;
s3ClientConfig.endpointOverride = ConvertToAwsString("minio.local:9000");

Aws::Auth::AWSCredentials credentials(ConvertToAwsString("minioadmin"),
                                      ConvertToAwsString("minioadmin"));

auto client = Aws::MakeShared<Aws::S3Crt::S3CrtClient>(ALLOCATION_TAG,
                                                       credentials,
                                              s3ClientConfig,
                                      Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Always /*signPayloads*/,
                                      false);`

It's rather wired:

1. I've set false for 'useVirtualAddressing'
2. why list objects can work, but getObject need to resolve dns?

any advices will be appreciated

[sbiscigl]

minio.local

why should this name be resolvable? how are you creating a entry for that name? minio by default runs on localhost, what is your setup where you expect that to be available.

[MrPresent-Han]

Hi, @sbiscigl

1. just ignore minio.local which is just a hostname I configured in the local /etc/hosts.
2. Yet even with s3ClientConfig.endpointOverride = ConvertToAwsString("localhost:9000"), my aws-sdk-cpp cannot get the file wanted with the error: no response body

[MrPresent-Han]

when setting s3ClientConfig.endpointOverride = ConvertToAwsString("localhost:9000"), I got error as follow:

20250414 18:16:09.868304 1457349 MinioChunkManager.cpp:117] [SERVER][ProcessFormattedStatement][milvus][][AWS LOG] [DEBUG] 2025-04-14 10:16:09.868 dns [132413028042304] static: resolving host localhost:9000
I20250414 18:16:09.868389 1457349 MinioChunkManager.cpp:117] [SERVER][ProcessFormattedStatement][milvus][][AWS LOG] [ERROR] 2025-04-14 10:16:09.868 dns [132413028042304] static: getaddrinfo failed with error_code -2: Name or service not known
I20250414 18:16:09.868407 1457349 MinioChunkManager.cpp:117] [SERVER][ProcessFormattedStatement][milvus][][AWS LOG] [WARN] 2025-04-14 10:16:09.868 dns [132413028042304] static, resolving host localhost:9000 failed, ec 1059 (aws-c-io: AWS_IO_DNS_INVALID_NAME, Host name was invalid for dns resolution.)
I20250414 18:16:09.868420 1457349 MinioChunkManager.cpp:117] [SERVER][ProcessFormattedStatement][milvus][][AWS LOG] [DEBUG] 2025-04-14 10:16:09.868 dns [132413028042304] static, invoking resolution callback for host localhost:9000 with failure
I20250414 18:16:09.868443 1457349 MinioChunkManager.cpp:117] [SERVER][ProcessFormattedStatement][milvus][][AWS LOG] [DEBUG] 2025-04-14 10:16:09.868 dns [132413028042304] static, invoking resolution callback for host localhost:9000 with failure
I20250414 18:16:09.868453 1457349 MinioChunkManager.cpp:117] [SERVER][ProcessFormattedStatement][milvus][][AWS LOG] [ERROR] 2025-04-14 10:16:09.868 channel-bootstrap [132413028042304] id=0x786e0d900360: dns resolution failed, or all socket connections to the endpoint failed.
I20250414 18:16:09.868463 1457349 MinioChunkManager.cpp:117] [SERVER][ProcessFormattedStatement][milvus][][AWS LOG] [ERROR] 2025-04-14 10:16:09.868 http-connection [132413028042304] static: Client connection failed with error 1059 (AWS_IO_DNS_INVALID_NAME).

it seems that aws-sdk try to parse the whole 'localhost:9000' as an ip and failed?
is it sth to do with force-path-style? But I can't find a way to configure force-path-style for s3CrtClient

[sbiscigl]

static: getaddrinfo failed with error_code -2: Name or service not known

we dont do anything special with the host override name you give us, we just pass it to the http client, if the http client cant resolve it, then its something wrong at the system configuration problem. so the question here is why cant getaddrinfo find that name? i.e.

#include <assert.h>
#include <netdb.h>

int main(){
  struct addrinfo hints, *res;
  int error;
  error = getaddrinfo("localhost", "http", nullptr, &res);
  freeaddrinfo(res);
  assert(error == 0);
}

additionally when i run a minio server using their docker instructions

mkdir -p ${HOME}/minio/data

docker run \
   -p 9000:9000 \
   -p 9001:9001 \
   --user $(id -u):$(id -g) \
   --name minio1 \
   -e "MINIO_ROOT_USER=DONTCOPYPASTETHIS" \
   -e "MINIO_ROOT_PASSWORD=PLEASEDONTUSETHIS" \
   -v ${HOME}/minio/data:/data \
   quay.io/minio/minio server /data --console-address ":9001"

then i have a example that uses this

#include <aws/core/Aws.h>
#include <aws/s3-crt/S3CrtClient.h>

using namespace Aws;
using namespace Aws::Utils;
using namespace Aws::S3Crt;
using namespace Aws::S3Crt::Model;

auto main() -> int {
  SDKOptions options;
  options.loggingOptions.logLevel = Logging::LogLevel::Debug;
  InitAPI(options);
  {
    S3CrtClientConfiguration client_configuration{};
    client_configuration.endpointOverride = "http://localhost:9000";
    S3CrtClient client{client_configuration};
    auto list_buckets_request = ListBucketsRequest();
    const auto list_buckets_outcome = client.ListBuckets();
    assert(list_buckets_outcome.IsSuccess());
  }
  ShutdownAPI(options);
  return 0;
}

i can see it working given the env variables for creds are set i.e.

export AWS_ACCESS_KEY_ID=DONTCOPYPASTETHIS
export AWS_SECRET_ACCESS_KEY=PLEASEDONTUSETHIS

[SergeyRyabinin]

Hello,
I can see you are using a pretty old version.

version: 1.9.234

Starting 1.10, the endpoint resolution has been completely overwritten and I also remember doing some fixes for endpoint override for the S3 client.
Could you please try with the latest 1.11.x released version?

Best regards.

[MrPresent-Han]

Thankds, SergeyRyabinin
after upgrading to 1.11.352, it worked with same code