WriteCapacity limits on restore

[mlimaloureiro]

Hi guys,

would like to ask what's the expected behavior on restores. Shouldn't it change write capacity limits when using -c flag ?

Thanks

[sdesalas]

Hi Miguel,

Yep, the -c (--concurrency) flag sets both the write capacity during restore and the number of http requestes that write to dynamodb concurrently from your machine.

To set the write capacity after restore you can use the -rc and -wc flags (--readcapacity and --writecapacity). These two flags should be used together.

See following examples from the Examples section of the README.md:

# Restore over existing table, 1000 concurrent requests. Stop if any batch fails 1000 times.
$ ./bin/dynamo-restore-from-s3 -t acme-customers -c 1000 -s s3://my-backups/acme-customers.json --overwrite -sf

# Auto-generate table. 2000 write units during restore. When finished set 50 write units and 100 write units (both needed).
$ ./bin/dynamo-restore-from-s3 -t acme-orders -c 2000 -s s3://my-backups/acme-orders.json --readcapacity 100 --writecapacity 50

[mlimaloureiro]

Hi steven, thanks for you quick reply. I was asking because I did try the following command ./bin/dynamo-restore-from-s3 -t staging-properties -c 2000 -s s3://my-bucket/staging-properties.json --aws-key key --aws-secret secret --aws-region eu-west-1 --overwrite and it isn't changing the write throughput on my dynamodb table. I'm checking the code and it seems it only changes the throughput if we need to create a new table https://github.com/markitx/dynamo-backup-to-s3/blob/master/lib/dynamo-restore.js#L117 which with the overwrite I assume it will never reach this code https://github.com/markitx/dynamo-backup-to-s3/blob/master/lib/dynamo-restore.js#L113 am I correct ? Thanks for your time, ML 2017-04-23 0:30 GMT+01:00 Steven de Salas <[email protected]>:

…

Hi Miguel, Yep, the -c flag sets both the write capacity *during restore* and the number of http requestes that write to the database concurrently. To set the write capacity *after restore* you can use the -rc and -wc flags. These two flags should be used together. See following examples from the Examples section of the README.md <https://github.com/markitx/dynamo-backup-to-s3#examples>: # Restore over existing table, 1000 concurrent requests. Stop if any batch fails 1000 times. $ ./bin/dynamo-restore-from-s3 -t acme-customers -c 1000 -s s3://my-backups/acme-customers.json --overwrite -sf # Auto-generate table. 2000 write units during restore. When finished set 50 write units and 100 write units (both needed). $ ./bin/dynamo-restore-from-s3 -t acme-orders -c 2000 -s s3://my-backups/acme-orders.json --readcapacity 100 --writecapacity 50 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#34 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB8ICVFcDsLXgSGMYBmAkURuYZ32GW0iks5ryo2cgaJpZM4NEJka> .

[sdesalas]

Ah, well spotted. You're absolutely right, its not updating the provisioned write capacity if the table already exists. Relatively simple change but I need to test it first, thanks for letting me/us know!

Steven

[mlimaloureiro]

Thanks, opened a pr for this issue #35