User creation with AdditionalData set fails with ODataError "The following extension properties are not available"

[Eagle3386]

Describe the bug

I'm trying to create users in an Azure B2C tenant via Graph which fails when using the code inside a gRPC web-service, but succeeds in a console POC-style app - using the exact same code snippet?!

Fun fact: setting the 2nd extension property without casting it from uint to int results in the API only complaining about that property, not the other 2.

Expected behavior

Code succeeds, no matter which way its run.

How to reproduce

Code snippet of the actual Graph code:

var user = await graphServiceClient.Users
                                   .PostAsync(new()
                                   {
                                     AccountEnabled = true,
                                     AdditionalData = new Dictionary<string, object>
                                     {
                                       { "extension_{appId}_CustomerIds", "1,2,3" },
                                       { "extension_{appId}_TenantId", (int)1u },
                                       { "extension_{appId}_TenantIds", "4,5,6" }
                                     },
                                     CompanyName = "Test Co.",
                                     DisplayName = "Test Pilot",
                                     GivenName   = "Test",
                                     Identities  =
                                     [
                                       new()
                                       {
                                         Issuer           = "{ourB2Csubdomain}.onmicrosoft.com",
                                         IssuerAssignedId = "[email protected]",
                                         SignInType       = "emailAddress",
                                       }
                                     ],
                                     Mail             = "[email protected]",
                                     PasswordPolicies = "DisablePasswordExpiration",
                                     PasswordProfile  = new()
                                     {
                                       ForceChangePasswordNextSignIn = false,
                                       Password                      = "Test12345!"
                                     },
                                     PreferredDataLocation = "EUR",
                                     PreferredLanguage     = "de-DE",
                                     Surname               = "Pilot",
                                     UsageLocation         = "DE",
                                   })
                                   .ConfigureAwait(false);

Code snippet for the actual setup in Program.cs of the web-service:

// … code left out for brevity…
var configuration   = builder.Configuration;
var isNonProduction = !builder.Environment.IsProduction();
// … code left out for brevity…
.AddAuthentication()
.AddMicrosoftIdentityWebApi(options =>
  {
    LogCompleteSecurityArtifact = ShowPII = isNonProduction;
    builder.Configuration.Bind(Constants.AzureAdB2C, options);
    options.TokenValidationParameters = new()
    {
      // … code left out for brevity…
    };
  },
  options => builder.Configuration.Bind(Constants.AzureAdB2C, options),
  subscribeToJwtBearerMiddlewareDiagnosticsEvents: isNonProduction)
.EnableTokenAcquisitionToCallDownstreamApi(options =>
{
  // … code left out for brevity…
  options.EnablePiiLogging = isNonProduction;
  options.LogLevel         = isNonProduction ? LogLevel.Always : LogLevel.Info;
})
.AddMicrosoftGraphAppOnly(_ =>
  new(
    new Azure.Identity.ClientSecretCredential(
      configuration[$"{Constants.AzureAdB2C}:{nameof(MicrosoftIdentityOptions.TenantId)}"],
      clientId,
      clientSecret,
      new()
      {
        Diagnostics =
        {
          // … code left out for brevity…
        },
        IsUnsafeSupportLoggingEnabled = isNonProduction
      })))
.AddInMemoryTokenCaches(options =>
  options.AbsoluteExpirationRelativeToNow = TimeSpan.FromDays(isNonProduction ? 14 : 90))
// … code left out for brevity…

SDK Version

5.58

Latest version known to work for scenario above?

AFAICT: none

Known Workarounds

None, because neither using a POC-style console app nor adding those extension properties via PATCH HTTP request is suitable - I expect the Graph API to handle user creation successfully even when extension properties are part of the request's payload.

Debug output

Click to expand log

Microsoft.Graph.Models.ODataErrors.ODataError:
  The following extension properties are not available:
     extension_{appId}_CustomerIds,extension_{appId}_TenantId,extension_{appId}_TenantIds.
   at Microsoft.Kiota.Http.HttpClientLibrary.HttpClientRequestAdapter.ThrowIfFailedResponse(
     HttpResponseMessage response,
     Dictionary`2 errorMapping,
     Activity activityForAttributes,
     CancellationToken cancellationToken)
   at Microsoft.Kiota.Http.HttpClientLibrary.HttpClientRequestAdapter.SendAsync[ModelType](
     RequestInformation requestInfo,
     ParsableFactory`1 factory,
     Dictionary`2 errorMapping,
     CancellationToken cancellationToken)
   at Microsoft.Kiota.Http.HttpClientLibrary.HttpClientRequestAdapter.SendAsync[ModelType](
     RequestInformation requestInfo,
     ParsableFactory`1 factory,
     Dictionary`2 errorMapping,
     CancellationToken cancellationToken)
   at Microsoft.Graph.Users.UsersRequestBuilder.PostAsync(
     User body,
     Action`1 requestConfiguration,
     CancellationToken cancellationToken)
   at My.Services.Migration.GraphService.CreateUser(CreateUserRequest request, ServerCallContext context)
   in D:\Code\Services\My.Services.Migration\GraphService.cs:line 24

Configuration

• OS: Windows 11, latest patches & Ubuntu Ubuntu 22.04.4 LTS, latest patches
• architecture: x64 (Windows) & x86_64 (Ubuntu)

Other information

I find the Graph SDK docs regarding Azure B2C user properties rather confusing, because the summary of AdditionalData states:

Stores additional data not described in the OpenAPI description found when deserializing. Can be used for serialization as well.

while the one for Extensions states (emphasis by me):

The collection of open extensions defined for the user. Read-only. Supports $expand. Nullable.

Because that suggests 2 things: a) use AdditionalData for reading and writing, but Extensions only for reading - why 2 properties for basically the same amount of information?

[andrueastman]

Thanks for raising this @Eagle3386

From the error log,

The following extension properties are not available:
     extension_{appId}_CustomerIds,extension_{appId}_TenantId,extension_{appId}_TenantIds.

It looks like the extension properties being sent to the API may not be setup. Did you intend to use string interpolation to insert an appId into the property names?

I believe if you use the API documentaion here, you should be able to list the correct names to use for the extension properties and whether they are available in the tenant you are trying to call.

https://learn.microsoft.com/en-us/graph/api/directoryobject-getavailableextensionproperties?view=graph-rest-1.0&tabs=http

[Eagle3386]

@andrueastman Nope, that's just me, redacting the app ID - in the actual exception, it's extension_abc[…]423_CustomerIds, etc.

Furthermore, rest assured, the extension properties are 100% correct - no doubt about it.
That's even confirmed by what I wrote initially:

(…) but succeeds in a console POC-style app - using the exact same code snippet (…)

Here's even a screenshot from our Azure B2C tenant's User attributes blade to confirm proper naming:

And using the Graph API you've linked to, I get this:

For convenience, the relevant columns as text:

DataType	IsMultiValued	Name	TargetObjects	DeletedDateTime	AdditionalData	BackingStore
String	False	extension_abc[…]423_TenantIds	[User]	""	[]	InMemoryBackingStore
Integer	False	extension_abc[…]423_TenantId	[User]	""	[]	InMemoryBackingStore
String	False	extension_abc[…]423_CustomerIds	[User]	""	[]	InMemoryBackingStore

So, I already confirmed that everything is set up as required, yet the API complains where it shouldn't even try to start & simply do what it's ordered to: create a user with additional data / extension properties.

[andrueastman]

Thanks for coming back to this @Eagle3386

When performing the request using the console app vs the service, do they use the same appId and permissions?
Are you also by any chance able to make the request successfully on Graph Explorer?

[Eagle3386]

@andrueastman

Yes, of course they do!

Besides, how could I possibly get The following extension properties are not available upon user creation, yet querying for all extension properties does return a non-empty collection, if the permissions aren't sufficient?

Here's the POC app's relevant code snippet as kinda proof:

  private static GraphServiceClient AppClient { get; set; }

  private static ClientSecretCredential Credential { get; set; }

  private static Settings Settings { get; set; }

  public static async Task<string> CreateUserAsync()
  {
    var user = await AppClient.Users
                              .PostAsync(new()
                              {
                                AccountEnabled = true,
                                AdditionalData = new Dictionary<string, object>
                                {
                                  { "extension_abc[…]423_CustomerIds", "1,2,3" },
                                  { "extension_abc[…]423_TenantId", (int)1u },
                                  { "extension_abc[…]423_TenantIds", "4,5,6" }
                                },
                                CompanyName = "Test Co.",
                                DisplayName = "Test Pilot",
                                GivenName   = "Test",
                                Identities  =
                                [
                                  new()
                                  {
                                    Issuer           = "{ourB2Csubdomain}.onmicrosoft.com",
                                    IssuerAssignedId = "[email protected]",
                                    SignInType       = "emailAddress",
                                  }
                                ],
                                Mail             = "[email protected]",
                                PasswordPolicies = "DisablePasswordExpiration",
                                PasswordProfile  = new()
                                {
                                  ForceChangePasswordNextSignIn = false,
                                  Password                      = "Test12345!"
                                },
                                PreferredDataLocation = "EUR",
                                PreferredLanguage     = "de-DE",
                                Surname               = "Pilot",
                                UsageLocation         = "DE",
                              })
                              .ConfigureAwait(false);
    user = await AppClient.Users[user.Id]
                          .GetAsync(configuration => configuration.QueryParameters.Select =
                          [
                              "extension_abc[…]423_CustomerIds",
                              "extension_abc[…]423_TenantId",
                              "extension_abc[…]423_TenantIds",
                              "GivenName",
                              "Id",
                              "Identities",
                              "Surname"
                          ])
                          .ConfigureAwait(false);
    return $"Name:         {user.GivenName} {user.Surname
      }\nMail:         {user.Identities![0].IssuerAssignedId
      }\nID:           {user.Id
      }\nCustomer IDs: {string.Join(", ", user.AdditionalData.First(data => data.Key.EndsWith("CustomerIds")).Value)
      }\nTenant ID:    {user.AdditionalData.First(data => data.Key.EndsWith("TenantId")).Value
      }\nTenant IDs:   {string.Join(", ", user.AdditionalData.First(data => data.Key.EndsWith("TenantIds")).Value)}";
  }

  public static async Task<string> GetAppOnlyAccessTokenAsync() =>
    (await (Credential ?? throw new NullReferenceException("Graph uninitialized for app-only auth."))
        .GetTokenAsync(new TokenRequestContext([ "https://graph.microsoft.com/.default" ]))).Token;

  public static void InitializeGraphForAppOnlyAuth(Settings settings)
  {
    Settings   =   settings ?? throw new NullReferenceException($"{nameof(Settings)} cannot be null.");
    Credential ??= new(Settings.TenantId, Settings.ClientId, Settings.ClientSecret);
    AppClient  ??= new(Credential, [ "https://graph.microsoft.com/.default" ]);
  }

… which returns:

User created:
Name:         Test Pilot
Mail:         [email protected]
ID:           d0a9aeb0-2a4a-47eb-83fa-c79d89fae676
Customer IDs: 1,2,3
Tenant ID:    1
Tenant IDs:   4,5,6

Using that snippet's code for retrieval inside the actual web-service, the user's AdditionalData property is populated with exactly one entry:

Key: @odata.context
Value: https://graph.microsoft.com/v1.0/$metadata#users(extension_abc[…]423_CustomerIds,extension_abc[…]423_TenantId,extension_abc[…]423_TenantIds,givenName,id,identities,surname)/$entity

… without any exception, so querying works, just as the extension properties are there.

So, can we finally switch from "Where's the error in my code?" to "Where's the bug in Graph?"
Because up to now, it seems we're trying pretty much everything to enforce the impression of the former instead of locating the latter.

For example, you could tell me how I can enable Graph Explorer to connect to our B2C tenant, because it pretty much only connects to our workforce one.

Lastly, rest assured that I already tried the user creation via Graph Explorer & it perfectly created the user, including the extension properties - though, as already stated, it creates them in the workforce tenant which is obviously not what's desired.

[andrueastman]

Are you able to capture the request-id and client request id from the error when it fails? Using this, we can file an issue with the API team to understand what could be wrong in this scenario.

You should be able to use the guidance here and retrieve the properties from innerError

https://github.com/microsoftgraph/msgraph-sdk-dotnet/blob/main/docs/errors.md#handling-errors-in-the-microsoft-graph-net-client-library

[Eagle3386]

Yes, of course! Thanks for asking, @andrueastman! 👍🏻

There you go:

• Error.InnerError.RequestId: fae3c201-8215-45b3-89aa-f81b5139ea5a
• Error.InnerError.ClientRequestId: 16077f71-69b0-42ed-984f-56054a577dc2

[andrueastman]

Thanks for this. I've raised an issue with the workload and will give feedback based on what they find.

https://portal.microsofticm.com/imp/v5/incidents/details/549539260/summary

[Eagle3386]

Thanks so much, @andrueastman! 👏🏻
Awaiting their/your feedback now.

[andrueastman]

@Eagle3386 Feedback from the API team is that the string interpolated here is incorrect. extension_{appId}_CustomerIds.

It looks like you may be interpolating the tenant_id instead instead of the app_id in the failing scenario. From their perspective, the extension attributes exist but the request has the incorrect id that look like the tenant id.

Any chance you can double check to confirm?

[Eagle3386]

@andrueastman please report precisely back to them:

You're wrong, in multiple ways & right from the start:

1. There's no string interpolation happening - at all!
   This is even backed as my latest log - which should've been provided to you instead of the older one to begin with - clearly mentions the shortened extension_abc[…]423_ instead of any _{appId}_.
2. When ignoring this fact, there's no interpolation of our Azure tenant's ID whatsoever, but instead simply one (and only one!) extension property whose name ends with tenantId.
3. Even ignoring this fact, too, there's still one ultimate fact: not only is the actual GraphServiceClient's code for creating the user identical in both executables, but is any used ID (Azure app/client ID, Azure tenant ID, etc.), too!

So, once again: I did double checke - Again! - yet, the error remains the same - just like the code remains to comply with both, the API & its docs, too.

Now please, with all due respect as I don't feel you're taking this seriously: rather do debug the given ClientRequestId / RequestId in your backend at least once than trying to blame my code.

[andrueastman]

@Eagle3386

Apologies for any frustrations here. I may have not shared enough details before. The team has indeed debugged the request id.

The request id shared above here has the following details from the back-end logs. The API team has also confirmed the extension properties do not exist in the given tenant based of the information coming in from the request.

App ID : 682xxxxxxxxxxxx032
Tenant ID : f677xxxxxxxxxxxx0668f

The received request on the backend logs shows the error as where the id matches the tenant id and not the app id
The following extension properties are not available: extension_f677xxxxxxxxx668f_CustomerIds.

Any chance you can confirm if this info is incorrect from your end? If not I can feedback any inconsistencies back to them...

[Eagle3386]

@andrueastman

First of all, regarding this:

Apologies for any frustrations here. I may have not shared enough details before. The team has indeed debugged the request id.

Apologies accepted, no hard feelings & thanks for confirming actual debugging!

Now, let's get back to work:

The request id shared above here has the following details from the back-end logs. The API team has also confirmed the extension properties do not exist in the given tenant based of the information coming in from the request.

App ID : 682xxxxxxxxxxxx032 Tenant ID : f677xxxxxxxxxxxx0668f

I can confirm, both are correct - see the app registration's overview as proof:

The received request on the backend logs shows the error as where the id matches the tenant id and not the app id The following extension properties are not available: extension_f677xxxxxxxxx668f_CustomerIds.

Any chance you can confirm if this info is incorrect from your end? If not I can feedback any inconsistencies back to them...

I can partly confirm this, but by all means, I truly apologize for messing both ID values up first!
Though, after fixing them, i.e., using app ID instead of tenant ID (which I seem to have messed up due to some docs asking to use it as AzureAdB2C:Domain's value, too), I'm still receiving the very same error:

The following extension properties are not available:
extension_682[…]032_CustomerIds,extension_682[…]032_TenantId,extension_682[…]032_TenantIds.

Error.InnerError.RequestId: 2e785e06-f1db-43d9-ad24-095a9077c1f6
Error.InnerError.ClientRequestId: cd4e34ba-6f89-4b01-9818-e022eca11184

Please, can you check what's still causing the error?

Additionally, I really want to know how this can actually work in the POC program, but only the actual service complains about the wrong app ID?! 😳😅

[andrueastman]

Thanks for the extra info here. I've fed this new request id back to the API team issue request and will give an update on any new findings from their point of view.

[Eagle3386]

Again, thanks so much, @andrueastman! 👏🏻
Eagerly awaiting their/your feedback.. 😉