Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Subpartitioning: Fixes bug for queries on subpartitioned containers #3934

Merged
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
e7e7427
initial fix, needs testing on prod
NaluTripician Jun 22, 2023
630ffba
test fix
NaluTripician Jun 22, 2023
3ee6878
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Jun 22, 2023
cb9f7f4
clean up pr
NaluTripician Jun 23, 2023
ccb41a5
Merge branch 'users/nalutripician/subpartitionQueryFix' of https://gi…
NaluTripician Jun 23, 2023
8125bbd
query rework
NaluTripician Jun 26, 2023
c1c140f
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Jun 26, 2023
876b78d
refactors previous changes
NaluTripician Jun 26, 2023
a4b94ac
requested changes and bug fixes
NaluTripician Jun 27, 2023
a87664e
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Jun 27, 2023
0f20287
nits
NaluTripician Jun 27, 2023
aaf7f9f
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Jun 27, 2023
77f1eee
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Jul 11, 2023
daa09da
requested changes
NaluTripician Jul 12, 2023
1d5aade
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Jul 12, 2023
d138213
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Jul 14, 2023
7e00909
bug fixes
NaluTripician Jul 14, 2023
cfc5e44
Merge branch 'users/nalutripician/subpartitionQueryFix' of https://gi…
NaluTripician Jul 14, 2023
62d676d
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Jul 18, 2023
83c3209
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Jul 25, 2023
2a5f728
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Aug 4, 2023
e2901fe
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Aug 9, 2023
e24811f
start of test
NaluTripician Aug 9, 2023
b2df07c
added test
NaluTripician Aug 9, 2023
95d055a
nit: changed name of EffectivePartitionKeyRanges to EffectiveRangesFo…
NaluTripician Aug 9, 2023
abceb74
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Aug 10, 2023
ca1b51e
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Aug 14, 2023
c4c555f
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Aug 28, 2023
c55f693
Address code comments
SrinikhilReddy Aug 30, 2023
9ef07f3
Address code comments
SrinikhilReddy Aug 30, 2023
95fd321
saving work
SrinikhilReddy Aug 30, 2023
dea977a
addresses code comments
NaluTripician Aug 30, 2023
0ff8b5a
nit, spacing
NaluTripician Aug 30, 2023
f226fee
PartitionKeyHash fixes
NaluTripician Aug 30, 2023
4b7b118
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Aug 30, 2023
f8bda5c
Fixes bugs in tests
NaluTripician Aug 31, 2023
e70e91f
Merge branch 'users/nalutripician/subpartitionQueryFix' of https://gi…
NaluTripician Aug 31, 2023
dfc1a30
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Aug 31, 2023
2060c76
Removed bad method, added additional test coverage
NaluTripician Aug 31, 2023
c0f4553
Removed EffectivePartitionKeyString use
NaluTripician Sep 1, 2023
ca98d2f
test fix
NaluTripician Sep 1, 2023
9333cdd
requested changes
NaluTripician Sep 5, 2023
8daf050
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Sep 5, 2023
48557c9
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Sep 8, 2023
4d3b03c
Requested changes
NaluTripician Sep 11, 2023
149c5f7
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Sep 11, 2023
964acb7
fixed test
NaluTripician Sep 11, 2023
1ada08e
Merge branch 'users/nalutripician/subpartitionQueryFix' of https://gi…
NaluTripician Sep 11, 2023
eafe083
Test fix
NaluTripician Sep 11, 2023
87a34a3
Merge branch 'master' into users/nalutripician/subpartitionQueryFix
NaluTripician Sep 12, 2023
5559163
Added comment
NaluTripician Sep 12, 2023
46eba06
Merge branch 'users/nalutripician/subpartitionQueryFix' of https://gi…
NaluTripician Sep 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -211,10 +211,10 @@ private static async Task<TryCatch<IQueryPipelineStage>> TryCreateCoreContextAsy

// Only thing that matters is that we target the correct range.
Documents.PartitionKeyDefinition partitionKeyDefinition = GetPartitionKeyDefinition(inputParameters, containerQueryProperties);
List<Documents.PartitionKeyRange> targetRanges = await cosmosQueryContext.QueryClient.GetTargetPartitionKeyRangesByEpkStringAsync(
NaluTripician marked this conversation as resolved.
Show resolved Hide resolved
List<Documents.PartitionKeyRange> targetRanges = await cosmosQueryContext.QueryClient.GetTargetPartitionKeyRangesAsync(
cosmosQueryContext.ResourceLink,
containerQueryProperties.ResourceId,
containerQueryProperties.EffectivePartitionKeyString,
containerQueryProperties.EffectivePartitionKeyRanges,
forceRefresh: false,
createQueryPipelineTrace);

Expand Down Expand Up @@ -635,12 +635,12 @@ private static async Task<PartitionedQueryExecutionInfo> GetPartitionedQueryExec
ITrace trace)
{
List<Documents.PartitionKeyRange> targetRanges;
if (containerQueryProperties.EffectivePartitionKeyString != null)
if (containerQueryProperties.EffectivePartitionKeyRanges != null)
{
targetRanges = await queryClient.GetTargetPartitionKeyRangesByEpkStringAsync(
targetRanges = await queryClient.GetTargetPartitionKeyRangesAsync(
resourceLink,
containerQueryProperties.ResourceId,
containerQueryProperties.EffectivePartitionKeyString,
containerQueryProperties.EffectivePartitionKeyRanges,
forceRefresh: false,
trace);
}
Expand Down Expand Up @@ -771,14 +771,13 @@ private static Documents.PartitionKeyDefinition GetPartitionKeyDefinition(InputP
else
{
Documents.PartitionKeyDefinition partitionKeyDefinition = GetPartitionKeyDefinition(inputParameters, containerQueryProperties);
if (inputParameters.PartitionKey != null)
if (inputParameters.PartitionKey.HasValue)
{
Debug.Assert(partitionKeyDefinition != null, "CosmosQueryExecutionContextFactory Assert!", "PartitionKeyDefinition cannot be null if partitionKey is defined");

targetRanges = await cosmosQueryContext.QueryClient.GetTargetPartitionKeyRangesByEpkStringAsync(
targetRanges = await cosmosQueryContext.QueryClient.GetTargetPartitionKeyRangesAsync(
cosmosQueryContext.ResourceLink,
containerQueryProperties.ResourceId,
containerQueryProperties.EffectivePartitionKeyString,
containerQueryProperties.EffectivePartitionKeyRanges,
forceRefresh: false,
trace);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,24 +4,26 @@

namespace Microsoft.Azure.Cosmos.Query.Core.QueryClient
{
using System.Collections.Generic;
using Microsoft.Azure.Documents;
using Microsoft.Azure.Documents.Routing;

internal readonly struct ContainerQueryProperties
{
public ContainerQueryProperties(
string resourceId,
string effectivePartitionKeyString,
List<Range<string>> effectivePartitionKeyRanges,
NaluTripician marked this conversation as resolved.
Show resolved Hide resolved
PartitionKeyDefinition partitionKeyDefinition,
Cosmos.GeospatialType geospatialType)
{
this.ResourceId = resourceId;
this.EffectivePartitionKeyString = effectivePartitionKeyString;
this.EffectivePartitionKeyRanges = effectivePartitionKeyRanges;
this.PartitionKeyDefinition = partitionKeyDefinition;
this.GeospatialType = geospatialType;
}

public string ResourceId { get; }
public string EffectivePartitionKeyString { get; }
public List<Range<string>> EffectivePartitionKeyRanges { get; }
ealsur marked this conversation as resolved.
Show resolved Hide resolved
NaluTripician marked this conversation as resolved.
Show resolved Hide resolved
public PartitionKeyDefinition PartitionKeyDefinition { get; }
public Cosmos.GeospatialType GeospatialType { get; }
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,17 +63,26 @@ public override async Task<ContainerQueryProperties> GetCachedContainerQueryProp
trace,
cancellationToken);

string effectivePartitionKeyString = null;
List<Range<string>> effectivePartitionKeyRange = null;
if (partitionKey != null)
{
// Dis-ambiguate the NonePK if used
PartitionKeyInternal partitionKeyInternal = partitionKey.Value.IsNone ? containerProperties.GetNoneValue() : partitionKey.Value.InternalKey;
effectivePartitionKeyString = partitionKeyInternal.GetEffectivePartitionKeyString(containerProperties.PartitionKey);
effectivePartitionKeyRange = new List<Range<string>>
{
PartitionKeyInternal.GetEffectivePartitionKeyRange(
containerProperties.PartitionKey,
new Range<PartitionKeyInternal>(
min: partitionKeyInternal,
max: partitionKeyInternal,
isMinInclusive: true,
isMaxInclusive: true))
NaluTripician marked this conversation as resolved.
Show resolved Hide resolved
};
}

return new ContainerQueryProperties(
containerProperties.ResourceId,
effectivePartitionKeyString,
effectivePartitionKeyRange,
containerProperties.PartitionKey,
containerProperties.GeospatialConfig.GeospatialType);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ public async Task TryGetOverlappingRangesAsyncTest()

Assert.IsNotNull(containerProperties);
Assert.IsNotNull(containerProperties.ResourceId);
Assert.IsNotNull(containerProperties.EffectivePartitionKeyString);
Assert.IsNotNull(containerProperties.EffectivePartitionKeyRanges);

IReadOnlyList<Documents.PartitionKeyRange> pkRange = await this.queryClientCore.TryGetOverlappingRangesAsync(
collectionResourceId: containerProperties.ResourceId,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -959,9 +959,16 @@ public override Task ForceRefreshCollectionCacheAsync(string collectionLink, Can

public override Task<ContainerQueryProperties> GetCachedContainerQueryPropertiesAsync(string containerLink, Cosmos.PartitionKey? partitionKey, ITrace trace, CancellationToken cancellationToken)
{
return Task.FromResult(new ContainerQueryProperties(
return Task.FromResult(new ContainerQueryProperties(
"test",
WFConstants.BackendHeaders.EffectivePartitionKeyString,
new List<Range<string>>
{
new Range<string>(
PartitionKeyInternal.MinimumInclusiveEffectivePartitionKey,
PartitionKeyInternal.MaximumExclusiveEffectivePartitionKey,
true,
true)
},
new PartitionKeyDefinition(),
Cosmos.GeospatialType.Geometry));
}
Expand Down