Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SRA] Rules-based authentication resolvers #3540

Merged
merged 2 commits into from
Nov 6, 2024
Merged

[SRA] Rules-based authentication resolvers #3540

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c6b2962
refactor: Update authentication resolvers to handle services with sch…
dscpinheiro Nov 1, 2024
0bdf9dd
refactor: Address PR feedback
dscpinheiro Nov 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 3 additions & 13 deletions generator/ServiceClientGeneratorLib/GeneratorDriver.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,18 +112,6 @@ public class GeneratorDriver

private static ConcurrentBag<string> codeGeneratedServiceNames = new ConcurrentBag<string>();

/// <summary>
/// Collection of services for which modeled auth resolvers (i.e. using information from their model files) should be generated.
/// </summary>
/// <remarks>
/// This is an allow-list for now (to prevent creating >400 files), but will be updated to a deny-list in the future (only a handful
/// of services such as S3 and EventBridge won't use modeled auth resolvers).
/// </remarks>
private static readonly HashSet<string> _allowListModeledAuthResolvers = new HashSet<string>
{
"AutoScaling",
};

public GeneratorDriver(ServiceConfiguration config, GenerationManifest generationManifest, GeneratorOptions options)
{
FilesWrittenToGeneratorFolder = new HashSet<string>();
Expand Down Expand Up @@ -209,7 +197,9 @@ public void Execute()
ExecuteTestGenerator(new EndpointProviderTests(), Configuration.ClassName + "EndpointProviderTests.cs", "Endpoints");
}

if (_allowListModeledAuthResolvers.Contains(Configuration.ClassName))
// TODO: We'll eventually generate auth resolvers for all services, but only including a couple for now to keep
// the number of generated changes low.
if (Configuration.ClassName == "AutoScaling" || Configuration.ClassName == "S3")
{
ExecuteGenerator(new ModeledResolver(), "Amazon" + Configuration.ClassName + "AuthResolver.cs", "Internal");
}
Expand Down
115 changes: 87 additions & 28 deletions generator/ServiceClientGeneratorLib/Generators/AuthResolvers/ModeledResolver.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,37 +79,96 @@ public override string TransformText()

#line default
#line hidden
this.Write("AuthSchemeHandler : BaseAuthResolverHandler\r\n {\r\n /// <summary>\r\n " +
" /// Auth scheme resolver for AutoScaling.\r\n /// </summary>\r\n pub" +
"lic Amazon");
this.Write("AuthSchemeHandler : BaseAuthResolverHandler\r\n {\r\n");

#line 38 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
if (IsServiceAllowListedForRulesBasedResolver()) {

#line default
#line hidden
this.Write(" private readonly Amazon");

#line 39 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
this.Write(this.ToStringHelper.ToStringWithCulture(this.Config.ClassName));

#line default
#line hidden
this.Write("EndpointResolver _endpointResolver = new();\r\n\r\n");

#line 41 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
}

#line default
#line hidden
this.Write(" /// <summary>\r\n /// Modeled auth scheme resolver for ");

#line 43 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
this.Write(this.ToStringHelper.ToStringWithCulture(this.Config.ClassName));

#line default
#line hidden
this.Write(".\r\n /// </summary>\r\n public Amazon");

#line 45 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
this.Write(this.ToStringHelper.ToStringWithCulture(this.Config.ClassName));

#line default
#line hidden
this.Write("AuthSchemeResolver AuthSchemeResolver { get; } = new();\r\n\r\n /// <inheritdo" +
"c/>\r\n protected override List<IAuthSchemeOption> ResolveAuthOptions(IRequ" +
"estContext requestContext)\r\n {\r\n var mappedParameters = new Am" +
"azon");
"c/>\r\n protected override List<IAuthSchemeOption> ResolveAuthOptions(IExec" +
"utionContext executionContext)\r\n {\r\n");

#line 50 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
if (IsServiceAllowListedForRulesBasedResolver()) {

#line 46 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line default
#line hidden
this.Write(" // Since ");

#line 51 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
this.Write(this.ToStringHelper.ToStringWithCulture(this.Config.ClassName));

#line default
#line hidden
this.Write(@" includes auth schemes in its endpoint rules, we'll attempt to delegate resolution to the endpoint
// resolver first (falling back to the modeled resolver if no options are returned).
var endpoint = _endpointResolver.GetEndpoint(executionContext);

// This means the endpoints resolver is executed twice intentionally (at this point and then later in the pipeline
// to determine which endpoint the SDK should use for the request).
var endpointAuthSchemes = RetrieveSchemesFromEndpoint(endpoint);
if (endpointAuthSchemes != null)
{
return endpointAuthSchemes;
}

");

#line 63 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
}

#line default
#line hidden
this.Write(" var requestContext = executionContext.RequestContext;\r\n va" +
"r mappedParameters = new Amazon");

#line 65 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
this.Write(this.ToStringHelper.ToStringWithCulture(this.Config.ClassName));

#line default
#line hidden
this.Write("AuthSchemeParameters\r\n {\r\n Operation = requestContext.R" +
"equest.RequestName,\r\n");

#line 49 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 68 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
if (IsSigV4Supported()) {

#line default
#line hidden
this.Write(" Region = requestContext.ClientConfig.RegionEndpoint?.SystemName,\r" +
"\n");

#line 51 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 70 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
}

#line default
Expand All @@ -118,136 +177,136 @@ public override string TransformText()
"arameters);\r\n }\r\n }\r\n\r\n /// <inheritdoc cref=\"IAuthSchemeResolver{T" +
"}\" />\r\n public class Amazon");

#line 59 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 78 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
this.Write(this.ToStringHelper.ToStringWithCulture(this.Config.ClassName));

#line default
#line hidden
this.Write("AuthSchemeResolver : IAuthSchemeResolver<Amazon");

#line 59 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 78 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
this.Write(this.ToStringHelper.ToStringWithCulture(this.Config.ClassName));

#line default
#line hidden
this.Write("AuthSchemeParameters>\r\n {\r\n /// <inheritdoc />\r\n public List<IAu" +
"thSchemeOption> ResolveAuthScheme(Amazon");

#line 62 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 81 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
this.Write(this.ToStringHelper.ToStringWithCulture(this.Config.ClassName));

#line default
#line hidden
this.Write("AuthSchemeParameters authParameters)\r\n {\r\n switch (authParamete" +
"rs.Operation)\r\n {\r\n");

#line 66 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 85 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
foreach (var operation in GetOperationsWithAuthSchemes()) {

#line default
#line hidden
this.Write("\t\t\t\tcase \"");

#line 67 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 86 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
this.Write(this.ToStringHelper.ToStringWithCulture(operation.Name));

#line default
#line hidden
this.Write("Request\":\r\n");

#line 68 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 87 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
if (IsKnownSchemeList(operation.AuthSchemes, out string operationSchemesContent)) {

#line default
#line hidden
this.Write(" return ");

#line 69 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 88 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
this.Write(this.ToStringHelper.ToStringWithCulture(operationSchemesContent));

#line default
#line hidden
this.Write(";\r\n");

#line 70 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 89 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
} else {

#line default
#line hidden
this.Write(" return new List<IAuthSchemeOption> \r\n {\r\n");

#line 73 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 92 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
foreach (var operationScheme in operation.AuthSchemes) {

#line default
#line hidden
this.Write(" new AuthSchemeOption { SchemeId = \"");

#line 74 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 93 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
this.Write(this.ToStringHelper.ToStringWithCulture(operationScheme));

#line default
#line hidden
this.Write("\" },\r\n");

#line 75 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 94 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
}

#line default
#line hidden
this.Write(" };\r\n");

#line 77 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 96 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
}}

#line default
#line hidden
this.Write(" default:\r\n // Default for the service, applies" +
" to all remaining operations.\r\n");

#line 80 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 99 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
if (IsKnownSchemeList(GetServiceAuthSchemes(), out string serviceSchemesContent)) {

#line default
#line hidden
this.Write(" return ");

#line 81 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 100 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
this.Write(this.ToStringHelper.ToStringWithCulture(serviceSchemesContent));

#line default
#line hidden
this.Write(";\r\n");

#line 82 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 101 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
} else {

#line default
#line hidden
this.Write(" return new List<IAuthSchemeOption> \r\n {\r\n");

#line 85 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 104 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
foreach (var serviceScheme in GetServiceAuthSchemes()) {

#line default
#line hidden
this.Write(" \t\t new AuthSchemeOption { SchemeId = \"");

#line 86 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 105 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
this.Write(this.ToStringHelper.ToStringWithCulture(serviceScheme));

#line default
#line hidden
this.Write("\" },\r\n");

#line 87 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 106 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
}

#line default
#line hidden
this.Write(" };\r\n");

#line 89 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
#line 108 "C:\Projects\aws-sdk-net-v4\generator\ServiceClientGeneratorLib\Generators\AuthResolvers\ModeledResolver.tt"
}

#line default
Expand Down
14 changes: 14 additions & 0 deletions generator/ServiceClientGeneratorLib/Generators/AuthResolvers/ModeledResolver.partial.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,20 @@ namespace ServiceClientGenerator.Generators.AuthResolvers
{
public partial class ModeledResolver
{
private readonly HashSet<string> _allowListedServices = new HashSet<string>
{
"S3",
"EventBridge",
"SimpleEmailServiceV2",
"CloudFrontKeyValueStore",
};

/// <summary>
/// Returns whether the generated resolver should delegate auth scheme resolution to the endpoints resolver.
/// </summary>
private bool IsServiceAllowListedForRulesBasedResolver() =>
_allowListedServices.Contains(Config.ClassName);

/// <summary>
/// Returns whether this service supports SigV4 (and therefore should include region in its auth scheme parameters).
/// </summary>
Expand Down
23 changes: 21 additions & 2 deletions generator/ServiceClientGeneratorLib/Generators/AuthResolvers/ModeledResolver.tt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,33 @@ namespace <#=this.Config.Namespace#>.Internal
/// </summary>
public class Amazon<#=this.Config.ClassName#>AuthSchemeHandler : BaseAuthResolverHandler
{
<# if (IsServiceAllowListedForRulesBasedResolver()) { #>
private readonly Amazon<#=this.Config.ClassName#>EndpointResolver _endpointResolver = new();

<# } #>
/// <summary>
/// Auth scheme resolver for AutoScaling.
/// Modeled auth scheme resolver for <#=this.Config.ClassName#>.
/// </summary>
public Amazon<#=this.Config.ClassName#>AuthSchemeResolver AuthSchemeResolver { get; } = new();

/// <inheritdoc/>
protected override List<IAuthSchemeOption> ResolveAuthOptions(IRequestContext requestContext)
protected override List<IAuthSchemeOption> ResolveAuthOptions(IExecutionContext executionContext)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to get feedback on this approach I took: The SRA docs mention we should code generate the mapping from auth scheme parameters to endpoint resolver parameters, however that was a lot of duplication for S3. By calling the generated resolver instead, we'll use all endpoint parameters anyways (including new ones we might add in the future):

aws-sdk-net/sdk/src/Services/S3/Generated/Internal/AmazonS3EndpointResolver.cs

Lines 70 to 82 in 36af043

protected override EndpointParameters MapEndpointsParameters(IRequestContext requestContext)
{
var config = (AmazonS3Config)requestContext.ClientConfig;
var result = new S3EndpointParameters();
result.Region = config.RegionEndpoint?.SystemName;
result.UseFIPS = config.UseFIPSEndpoint;
result.UseDualStack = config.UseDualstackEndpoint;
result.Endpoint = config.ServiceURL;
result.ForcePathStyle = config.ForcePathStyle;
result.Accelerate = config.UseAccelerateEndpoint;
result.UseGlobalEndpoint = config.USEast1RegionalEndpointValue == S3UsEast1RegionalEndpointValue.Legacy;
result.DisableMultiRegionAccessPoints = config.DisableMultiregionAccessPoints;
result.UseArnRegion = config.UseArnRegion;

One benefit is that it allowed me to keep the implementation to retrieve auth schemes from an endpoint (in sdk/src/Core/Amazon.Runtime/Pipeline/Handlers/BaseAuthResolverHandler.cs) simpler.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would rerunning the endpoint resolver twice work when the SDK supports account id based endpoint rules? Because we would need the identity to get the account id for the endpoint rules I believe.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do need to run endpoint rules twice and it will use the exact same parameters we could stash the results in executionContext.RequestContext.ContextAttributes and then have the endpoint resolver check for the existence and if so reuse those results.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the internal design doc (that recommended running the EP2.0 resolver twice), there was a comment on not allowing S3 to use the AWS::Auth::AccountId parameter in rules that have authentication schemes.

I'll follow up to see if that was ever built or if it's something we should do in our generator.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the other comment, I thought about populating the existing EndpointAttributes property:

aws-sdk-net/sdk/src/Core/Amazon.Runtime/Internal/IRequest.cs

Line 388 in ad13fd4

IPropertyBag EndpointAttributes { get; set; }

But it looks like when the endpoints resolver calls GetEndpoint it calculates them again anyway (this constructor is called by the generated code):

aws-sdk-net/sdk/src/Core/Amazon.Runtime/Endpoints/Endpoint.cs

Lines 45 to 49 in ad13fd4

if (!string.IsNullOrEmpty(attributesJson))
{
var attributes = JsonMapper.ToObject(attributesJson);
Attributes = PropertyBag.FromJsonData(attributes);
}

Or did you mean I should cache the entire endpoint? (I don't think that's the case since we'd need to change whenever we add support for account-ID based endpoints).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding was running the endpoint rules twice would only work if we the service didn't use account id as part of the rule. In that case would we need to change this when the SDK supports account id based rules? So yes I was suggesting caching the results within the scope of the current request in stead of running the endpoint rules twice. But maybe I'm misunderstanding something and the second time could cause a different results in some scenarios.

{
<# if (IsServiceAllowListedForRulesBasedResolver()) { #>
// Since <#=this.Config.ClassName#> includes auth schemes in its endpoint rules, we'll attempt to delegate resolution to the endpoint
// resolver first (falling back to the modeled resolver if no options are returned).
var endpoint = _endpointResolver.GetEndpoint(executionContext);

// This means the endpoints resolver is executed twice intentionally (at this point and then later in the pipeline
// to determine which endpoint the SDK should use for the request).
var endpointAuthSchemes = RetrieveSchemesFromEndpoint(endpoint);
if (endpointAuthSchemes != null)
{
return endpointAuthSchemes;
}

<# } #>
var requestContext = executionContext.RequestContext;
var mappedParameters = new Amazon<#=this.Config.ClassName#>AuthSchemeParameters
{
Operation = requestContext.Request.RequestName,
Expand Down
2 changes: 1 addition & 1 deletion sdk/src/Core/Amazon.Runtime/Credentials/Internal/AnonymousAuthScheme.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ namespace Amazon.Runtime.Credentials.Internal
public class AnonymousAuthScheme : IAuthScheme<AnonymousAWSCredentials>
{
/// <inheritdoc/>
public string SchemeId => "smithy.api#noAuth";
public string SchemeId => AuthSchemeOption.NoAuth;

/// <inheritdoc/>
public IIdentityResolver GetIdentityResolver(IIdentityResolverConfiguration configuration)
Expand Down
Loading