BatchWrite doesn't respect IgnoreNullValues

[glenserraview]

Describe the bug

Prerequisites:

• Create a DynamoDb table and set a value for a field

Following these steps:

• Create a DynamoDBContext
• Create a batch using CreateBatchWrite
• Constuct a DynamoDBOperationConfig with IgnoreNullValues set to true
• Save an object with the field above property set to null

Expected Behavior

I would expect the value of the field in the DynamoDb table to retain it's value, since I set IgnoreNullValues and have set the value to null on the saved object.

Current Behavior

The value is cleared in the table

Reproduction Steps

using Amazon;
using Amazon.DynamoDBv2;
using Amazon.DynamoDBv2.DataModel;

var client = new AmazonDynamoDBClient(RegionEndpoint.APSoutheast2);
var context = new DynamoDBContext(client);


var config = new DynamoDBOperationConfig { IgnoreNullValues = true };

// Save an item with the SubTest field set
var testItem = new Test { Id = "id", Field1 = "Field 1 value", SubTest = new SubTest { Field1 = "Subtest Field 1 value" } };
var batch = context.CreateBatchWrite<Test>(config);
batch.AddPutItem(testItem);
await batch.ExecuteAsync(CancellationToken.None);

// At this point, the database has an entry with Field1 and Subtest having values

// Do another save, but with the SubTest object set to null - given we have IgnoreNullValues set, 
//  we expect that the SubTest field will be retained in the database
testItem.SubTest = null;
var batch2 = context.CreateBatchWrite<Test>(config);
batch2.AddPutItem(testItem);
await batch2.ExecuteAsync(CancellationToken.None);

[DynamoDBTable("test")]
public class Test
{
    [DynamoDBHashKey("Id")] public string Id { get; set; }
    [DynamoDBProperty] public string Field1 { get; set; }
    [DynamoDBProperty] public SubTest SubTest { get; set; }
}

public class SubTest
{
    [DynamoDBProperty] public string Field1 { get; set; }
}

Possible Solution

No response

Additional Information/Context

No response

AWS .NET SDK and/or Package version used

AWSSDK.DynamodDBv2.3.7.201.7

Targeted .NET Platform

.NET 6

Operating System and version

Windows 11

[ashishdhingra]

Hi @glenserraview,

Good afternoon.

Thanks for reporting the issue. I do not think IgnoreNullValues works for BatchWrite. The documentation for IgnoreNullValues at DynamoDBOperationConfig mentions that that this property directs DynamoDBContext to ignore null values on attributes during a Save operation. It only specifies the Save operation, not the BatchWrite operation.

For testing,

• I was able to reproduce the issue using the code you shared.
• Thereafter, for the 2nd part, I modified the code to below:

  Test item = await context.LoadAsync<Test>("id", CancellationToken.None);
  item.SubTest = null;
  item.Field1 = item.Field1 + " Modified";
  await context.SaveAsync(item, config);

  The SubTest property was ignored since IgnoreNullValues was set to true, wheras the Field1 property was modified.

Thanks,
Ashish

[glenserraview]

Thanks @ashishdhingra. So it sounds like this isn't a bug. Is it something that can be raised as a feature request? We would like to optimise our code to save records as a batch, but this is blocking us, as we need to use IgnoreNullValues.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[ashishdhingra]

Needs review with the team.

[ashishdhingra]

Discussed with the team. This is probably a bug.

[muhammad-othman]

After further investigation, it seems that this is a limitation of DynamoDB itself and not something that we can fix in the SDK as BatchWriteItem does replace the whole item and doesn't allow specific attributes updates.
Based on the API reference of BatchWriteItem

BatchWriteItem cannot update items. If you perform a BatchWriteItem operation on an existing item, that item's values will be overwritten by the operation and it will appear like it was updated. To update items, we recommend you use the UpdateItem action.

https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchWriteItem.html

UpdateItem on the other hand (this is what DynamoDBContext.Save use internally) does allow AttributeUpdates which the SDK uses to update only the attributes that were changed when IgnoreNullValues is set to true.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.