region set in profile in $HOME/.aws/config is ignored

[olfek]

Describe the bug

region set in profile in $HOME/.aws/config IS NOT loaded in.

Expected Behavior

region set in profile in $HOME/.aws/config IS loaded in.

Current Behavior

region set in profile, in $HOME/.aws/config
loaded into Amazon.Runtime.CredentialManagement.CredentialProfile.Region
IS NOT CONSIDERED in Amazon.Runtime.ClientConfig.get_RegionEndpoint.

Reproduction Steps

N/A

Possible Solution

region set in profile, in $HOME/.aws/config
loaded into Amazon.Runtime.CredentialManagement.CredentialProfile.Region
SHOULD BE CONSIDERED in Amazon.Runtime.ClientConfig.get_RegionEndpoint.

Just like how:

endpoint_url set in profile, in $HOME/.aws/config
loaded into Amazon.Runtime.CredentialManagement.CredentialProfile.EndpointUrl
IS CONSIDERED in Amazon.Runtime.ClientConfig.get_ServiceURL.

Additional Information/Context

N/A

AWS .NET SDK and/or Package version used

3.7.30X.XX

Targeted .NET Platform

.NET 7

Operating System and version

Windows 11

[olfek]

https://github.com/aws/aws-sdk-net/blob/7302897f02be1298041a882321179624bf56191d/sdk/src/Core/Amazon.Runtime/CredentialManagement/CredentialProfile.cs

https://github.com/aws/aws-sdk-net/blob/cd07e59f0a3317f7fad9f14f4eefc0aff8596aaa/sdk/src/Core/Amazon.Runtime/ClientConfig.cs

[bhoradc]

Hi @olfek,

Thank you for submitting the issue. Using latest .NET SDK and Package version, I unable to reproduce the scenario you mentioned.

Can you please review below steps and let me know if I am missing anything? Or kindly assist in providing steps to reproduce the issue.

• ~/.aws/config file has test profile

[test]
region=us-west-2

• Executed below program

using Amazon.Runtime.CredentialManagement;
using Amazon.S3;

            var credentialsFile = new SharedCredentialsFile(@"C:\Users\**\.aws\config");

            CredentialProfile profile;

            if (credentialsFile.TryGetProfile("test", out profile))
            {
                Console.WriteLine("Profile Region: " + profile.Region.SystemName);
                var s3Client = new AmazonS3Client(profile.Region);
                Console.WriteLine("Region Endpoint: " + s3Client.Config.RegionEndpoint);
            }

• Execution result

Profile Region: us-west-2
Region Endpoint: US West (Oregon) (us-west-2)

Regards,
Chaitanya

[olfek]

@bhoradc ...

I believe your understanding is different to what this issue is about.

1. You explicitly read the config file.
   I am expecting the config file to be read implicitly by the ClientConfig code like this - https://github.com/aws/aws-sdk-net/blob/cd07e59f0a3317f7fad9f14f4eefc0aff8596aaa/sdk/src/Core/Amazon.Runtime/ClientConfig.cs#L310C1-L317C26
2. You explicitly pass the region to the client constructor.
   I am expecting the region to be set implicitly using the implicitly read config file, like this -

   aws-sdk-net/sdk/src/Core/Amazon.Runtime/ClientConfig.cs

   Line 334 in cd07e59

   this.ServiceURL = profile.EndpointUrl;

Keep in mind using the RegionEndpoint or ServiceURL setters will nullify each other. In the case of profiles in the external config file, I think the SDK code should trust it to be valid and load it in as is without any additional validation/cleanup logic. So perhaps this line (

aws-sdk-net/sdk/src/Core/Amazon.Runtime/ClientConfig.cs

Line 334 in cd07e59

this.ServiceURL = profile.EndpointUrl;

) should be this.(s)erviceURL = profile.EndpointUrl and similarly this.(r)egionEndpoint = profile.Region.

[normj]

@olfek How are you configuring the SDK to use a specific profile? There a few cases with the .NET SDK doesn't know what profile is being used because it is working with the low level primitives like credentials. In those cases it falls back to looking for configuration with the default profile or the profile the AWS_PROFILE environment variable is set to.

[olfek]

@normj --- I've set the AWS_PROFILE environment variable, this is my $HOME/.aws/config file:

[profile localstack]
region = eu-west-2
endpoint_url = http://localhost:4566

endpoint_url is loaded in, but region is not.

I'm using https://github.com/localstack/localstack by the way.

[normj]

@olfek The SDK either has the region or the endpoint_url set with the last one set overruling the previous. In this case the endpoint_url from config is being set on the client config last causing the region to be cleared out. What are you expecting with having both the region and endpoint_url set?

[olfek]

@normj ---

What are you expecting with having both the region and endpoint_url set?

I'm expecting the ability to use a specific region with LocalStack, configured outside of code, to facilitate multi-region services.

[ashishdhingra]

@olfek As @normj mentioned in #3305 (comment), even if region was parsed from ~/.aws/config file correctly, either one of region or endpoint_url would be overwritten as clarified in documentation as well. If you wish to set a different authentication region, you may use AuthenticationRegion as mentioned in ClientConfig documentation.

[olfek]

@ashishdhingra ...

either one of region or endpoint_url would be overwritten as clarified in documentation as well

Which is why I proposed this

#3305 (comment)

[normj]

I'm still a bit confused on the intent. Is the goal to have the region set for signing purposes but we don't want to have to change code to set the AuthenticationRegion property. Or is the intent to have an environment where some service clients go to AWS and some service clients go to LocalStack. Sorry I have little knowledge of running LocalStack.

[olfek]

@normj ...

Is the goal to have the region set for signing purposes

No (at least not from my POV)

... but we don't want to have to change code to set the AuthenticationRegion property

& No (at least not from my POV)

As a basic user of the SDK, I neither care about "signing" nor the "authentication region" at this point in time.

To put it simply, from a basic user POV, I expect the SDK to adhere to ALL config specified in the config file.

[ashishdhingra]

@olfek We do not support SDK with non-AWS solutions. While it works in most scenarios, but it is not guaranteed to work. Our API documentation for ClientConfig explicitly calls out that RegionEndpoint and ServiceURL are mutually exclusive properties. Whichever property is set last will cause the other to automatically be reset to null..

@normj Please review if we could add something here.

Thanks,
Ashish

[github-actions]

This issue has not received a response in 5 days. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.

[olfek]

go away bot

[dscpinheiro]

@olfek Apologies, but we're not planning to change how the SDK handles both ServiceUrl and RegionEndpoint being set; as @ashishdhingra mentioned, we try to support 3rd party solutions as much as we can, but updating the current behavior would be a runtime breaking change.

I also understand LocalStack mentions multi-region support, but that's not how the vast majority of AWS services work (this is even called out in their documentation: AWS provides individual API endpoints for each region, and typically, resources can only be accessed within their respective regions); the SDKs must know the region when signing requests, this is usually transparent to customers but needs to happen for the request to succeed (which is why Norm recommended the AuthenticationRegion property).

---

I think if we were designing how ServiceUrl and RegionEndpoint interact with each other from scratch, we'd probably take a different approach on them being mutually exclusive (but obviously that's easy to say in hindsight).

[github-actions]

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[olfek]

@dscpinheiro ...

updating the current behavior would be a runtime breaking change

You could put up a breaking change notice.

But would it really be a breaking change? People using this SDK may have done/thought this:

1. I'd like region to be set outside the code, oh I know, I'll set it in $HOME/.aws/config.
2. Wait a minute, this isn't working, must be a bug or oversight.
3. Ok, people on the internet are saying to set the AuthenticationRegion property in code instead.

In the case above, the change would be non-breaking, since the AuthenticationRegion property set in code would have a higher priority.

If the developer decides to remove the AuthenticationRegion property in code, they will already have an idea of what becomes the effective region next.

Can you think of any cases where an actual breaking change occurs?

[dscpinheiro]

It could be a breaking change if someone has a) a region set in their config file and b) an explicit ServiceUrl when creating the service client (the SDK could end up sending requests to a different than expected endpoint).

Again, I understand this is not the best customer experience when using LocalStack, but we will not change this behavior. AuthenticationRegion is the recommended workaround for this scenario; see previous discussions: #1781 (comment), #1999 (comment), and https://stackoverflow.com/a/71906694

[olfek]

@dscpinheiro ...

In that case, if the ServiceUrl includes a region - prioritise it and use it, if the ServiceUrl does not include a region, use the one in the config file.

but we will not change this behavior

Why won't you?