chore(go): add plain text migration examples #1966

rishav-karanjit · 2025-08-01T23:45:19Z

Issue #, if available:

Description of changes:

Add examples to migrate from plain text table to DB-ESDK encrypted table.
These tests is tested as a unit test but not by round tripping in main because we need to run various combination of stages of migration.
Add utility functions required by examples in common utils file so that all examples can re-use same utils.
Add README.md file
Some nits in existing Java example.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…ase-encryption-sdk-dynamodb into plaintextmigration

imabhichow · 2025-08-04T23:13:01Z

Examples/runtimes/go/migration/PlaintextToAWSDBE/awsdbe/step2_test.go

+	MigrationStep1(kmsKeyID, tableName, partitionKey, sortKeys[1])
+	// When: Execute Step 2 with sortReadValue=1, Then: Success (i.e. can read encrypted values)
+	MigrationStep2(kmsKeyID, tableName, partitionKey, sortKeys[1])
+


Can we also add testcase to write with step 3 and read with step 2? also for step 1_test.go

see similar test in Python.

aws-database-encryption-sdk-dynamodb/Examples/runtimes/python/Migration/plaintext_to_awsdbe/test/awsdbe/client/test_migration_step_2.py

Lines 48 to 56 in de4588f

# Given: Step 3 has succeeded

migration_step_3.migration_step_3_with_client(

kms_key_id=TEST_KMS_KEY_ID, ddb_table_name=TEST_DDB_TABLE_NAME, sort_read_value=3

)

# When: Execute Step 2 with sort_read_value=3

# Then: Success (i.e. can read values in encrypted format)

migration_step_2.migration_step_2_with_client(

kms_key_id=TEST_KMS_KEY_ID, ddb_table_name=TEST_DDB_TABLE_NAME, sort_read_value=3

)

Thanks. It was a very good finding.

I did it in this commit: 4eb1d8a. I also figured that a change in local to handle error was not committed. So, this Commit include that too.

ShubhamChaturvedi7 · 2025-08-05T20:01:48Z

.github/workflows/ci_test_go.yml

@@ -140,8 +140,9 @@ jobs:
        run: |
          make test_go

-      - name: Test Examples
+      - name: Run and Test Examples


Why run and test both? Since it's an example, I don't think we need to assert all cases via unit tests, but if it's already in then no worries.

For migrations, I needed to run a lot of combination of steps that made sense of run as a unit test. For all other test, just running a round trip was enough but migration is different so it was not enough

ShubhamChaturvedi7 · 2025-08-05T20:23:49Z

Examples/runtimes/go/migration/PlaintextToAWSDBE/awsdbe/step1.go

+		Item:      item,
+	}
+
+	_, err = ddb.PutItem(context.TODO(), &putInput)


Should we use a plain vanilla client to demonstrate that we actually aren't doing any encryption and can read unencrypted items using our client?

We do this in Test.

This is one combination of the test: In step 1 test, we write with vanilla DDB client and read with intercepted DDB client. In step 0 test, we write with intercepted DDB client and read with the vanilla one.

ShubhamChaturvedi7 · 2025-08-05T20:24:42Z

Examples/runtimes/go/migration/PlaintextToAWSDBE/awsdbe/common.go

+		Generator: &kmsKeyID,
+	}
+	kmsKeyring, err := matProv.CreateAwsKmsMrkMultiKeyring(context.Background(), keyringInput)
+	utils.HandleError(err)


Nice touch.

ShubhamChaturvedi7 · 2025-08-05T20:25:16Z

Examples/runtimes/go/migration/PlaintextToAWSDBE/awsdbe/step1.go

+
+	// We return this error because we run test against the error.
+	// When used in production code, you can decide how you want to handle errors.
+	if err != nil {


Why not use utils here as well?

Before writing migration example, I thought panic on error everywhere is good enough but with all the test combinations in migration test, it was not good enough. I think many in the team will still have a thought that we panic on every example. So, to make it visually distinct immediately visible in the code, I did not use utils.

ShubhamChaturvedi7 · 2025-08-05T20:26:10Z

Examples/runtimes/go/migration/PlaintextToAWSDBE/awsdbe/step1.go

+	if partitionKeyValue != result.Item["partition_key"].(*types.AttributeValueMemberS).Value {
+		panic("Decrypted item does not match original item")
+	}
+	if encryptedAndSignedValue != result.Item["attribute1"].(*types.AttributeValueMemberS).Value {


What about other attributes? Maybe add comment if we don't want full assertion.

Will this change look good to you? https://github.com/aws/aws-database-encryption-sdk-dynamodb/pull/1966/files#r2255583436

I will apply this change to other places as well.

ShubhamChaturvedi7 · 2025-08-05T20:27:28Z

Examples/runtimes/go/migration/PlaintextToAWSDBE/awsdbe/step1_test.go

+	sortKeys := []string{"0", "1", "2", "3"}
+
+	// Successfully executes Step 1
+	err := MigrationStep1(kmsKeyID, tableName, partitionKey, sortKeys[1])


Since we aren't mocking anything here, what's the difference between this and run?

We run this only once with the test.

ShubhamChaturvedi7 · 2025-08-05T20:32:35Z

Examples/runtimes/go/utils/exampleUtils.go

@@ -124,6 +133,33 @@ func HandleError(err error) {
 	}
 }

+func AssertServiceError(err error, service string, operation string, errorMessage string) {
+	if err == nil {
+		panic("Expected error but got no error")


Don't we want to have this function as no-op in case of no errors? In that way, we can always call this instead of figuring out when to call this.

Not really. Every time we call this we need to have an Error. If we don't get an error then that means something is wrong.

ShubhamChaturvedi7 · 2025-08-05T20:33:48Z

Examples/runtimes/go/utils/exampleUtils.go

+	HandleError(err)
+
+	// Create DynamoDB client
+	client := dynamodb.NewFromConfig(cfg)


Or you could just use the already created client :)

This functions is used where we don't have access of the ddb client. So, I am re-creating it. We also re-create ddb client in every example so that examples are easier to follow.

rishav-karanjit · 2025-08-06T00:21:54Z

Examples/runtimes/go/migration/PlaintextToAWSDBE/awsdbe/step1.go

+		return err
+	}
+
+	// Verify we got the expected item back


Suggested change

// Verify we got the expected item back

// Verify we got the expected item back.

// Since we are mostly concerned with Plaintext Override,

// our focus is to assert the encrypted and signed value.

…ase-encryption-sdk-dynamodb into plaintextmigration

rishav-karanjit and others added 30 commits July 31, 2025 12:13

auto commit

1049f69

auto commit

1c3a811

auto commit

546e4df

auto commit

fc93915

auto commit

db00601

auto commit

5760865

auto commit

a369dcb

auto commit

38befdd

auto commit

681dcb0

auto commit

11cc980

auto commit

28394e8

auto commit

7d507eb

auto commit

2670157

auto commit

654dceb

auto commit

d5e36b8

auto commit

46c116b

auto commit

e1e4664

auto commit

14fdbc4

auto commit

02a895f

auto commit

6320c71

auto commit

158077a

Merge branch 'main' into plaintextmigration

910e2c2

auto commit

2f82fff

Merge branch 'plaintextmigration' of https://github.com/aws/aws-datab…

21a1432

…ase-encryption-sdk-dynamodb into plaintextmigration

auto commit

6f5bc22

auto commit

8939baa

auto commit

1cac0e9

auto commit

21573ff

auto commit

845d51f

auto commit

be195f2

rishav-karanjit added 18 commits August 4, 2025 11:10

auto commit

ced6cbf

auto commit

6c3971b

auto commit

83f4bb3

auto commit

1d943e1

auto commit

041e19f

auto commit

6098aa9

auto commit

475d77e

auto commit

08fc555

auto commit

6f5f836

auto commit

e0a2c7e

auto commit

13cf62c

auto commit

cde3989

auto commit

e952d00

auto commit

debaf5a

auto commit

b478e98

auto commit

9b51cb2

auto commit

8412c88

auto commit

14a3eba

rishav-karanjit marked this pull request as ready for review August 4, 2025 22:38

rishav-karanjit requested a review from a team as a code owner August 4, 2025 22:38

rishav-karanjit changed the title ~~chore(go): add plain text migration~~ chore(go): add plain text migration examples Aug 4, 2025

imabhichow reviewed Aug 4, 2025

View reviewed changes

rishav-karanjit and others added 2 commits August 4, 2025 16:56

add step 3 -> step 1/2

4eb1d8a

Merge branch 'main' into plaintextmigration

295a4bd

ShubhamChaturvedi7 reviewed Aug 5, 2025

View reviewed changes

rishav-karanjit commented Aug 6, 2025

View reviewed changes

rishav-karanjit added 3 commits August 6, 2025 15:58

sortkeywritevalue

ea9b480

Merge branch 'plaintextmigration' of https://github.com/aws/aws-datab…

c50b7d1

…ase-encryption-sdk-dynamodb into plaintextmigration

sortkeywritevalue

e2e45eb

	# Given: Step 3 has succeeded
	migration_step_3.migration_step_3_with_client(
	kms_key_id=TEST_KMS_KEY_ID, ddb_table_name=TEST_DDB_TABLE_NAME, sort_read_value=3
	)
	# When: Execute Step 2 with sort_read_value=3
	# Then: Success (i.e. can read values in encrypted format)
	migration_step_2.migration_step_2_with_client(
	kms_key_id=TEST_KMS_KEY_ID, ddb_table_name=TEST_DDB_TABLE_NAME, sort_read_value=3
	)

-	// Verify we got the expected item back
+	// Verify we got the expected item back.
+	// Since we are mostly concerned with Plaintext Override,
+	// our focus is to assert the encrypted and signed value.

chore(go): add plain text migration examples #1966

Are you sure you want to change the base?

chore(go): add plain text migration examples #1966

Conversation

rishav-karanjit commented Aug 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rishav-karanjit Aug 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rishav-karanjit Aug 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rishav-karanjit Aug 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rishav-karanjit commented Aug 1, 2025 •

edited

Loading

rishav-karanjit Aug 5, 2025 •

edited

Loading

rishav-karanjit Aug 5, 2025 •

edited

Loading

rishav-karanjit Aug 5, 2025 •

edited

Loading