Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create code examples for basic operations that use checksums and create examples for multipart uploads #5373

Merged
merged 5 commits into from
Sep 21, 2023
Merged

Create code examples for basic operations that use checksums and create examples for multipart uploads #5373

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
bf35446
Create code examples for basic operations that use checksums and crea…
tkhill-AWS Sep 6, 2023
a297d2a
Create code examples for basic operations that use checksums and crea…
tkhill-AWS Sep 6, 2023
c56d240
Changes made from the review of code example team
tkhill-AWS Sep 21, 2023
7231db1
Merge remote-tracking branch 'myfork/s3_checksum_basic' into s3_check…
tkhill-AWS Sep 21, 2023
d7f8d91
Remove outer try/catch block
tkhill-AWS Sep 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 78 additions & 0 deletions .doc_gen/metadata/s3_metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2749,5 +2749,83 @@ s3_Scenario_UploadStream:
snippet_tags:
- s3.tm.java2.upload_stream.import
- s3.tm.java2.upload_stream.main

services:
s3: {}
s3_Scenario_MultipartUpload:
title: Perform a multipart upload to an &S3; object using an &AWS; SDK
title_abbrev: Perform a multipart upload
synopsis: perform a multipart upload to an &S3; object.
category: Scenarios
languages:
Java:
versions:
- sdk_version: 2
github: java/example_code/s3
sdkguide:
excerpts:
- description: The code examples use the following imports.
snippet_tags:
- s3.java2.performMultiPartUpload.import
- description: >-
Use the <ulink url="sdk-for-java/latest/developer-guide/transfer-manager.html"
type="documentation">S3 Transfer Manager</ulink> on top of the <ulink
url="sdk-for-java/latest/developer-guide/crt-based-s3-client.html" type="documentation">&AWS;
CRT-based S3 client</ulink> to transparently perform a multipart upload when the size of the
content exceeds a threshold. The default threshold size is 8 MB.
snippet_tags:
- s3.java2.performMultiPartUpload.transferManager
- description: >-
Use the <ulink
url="https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Client.html">S3Client API</ulink>
or (S3AsyncClient API) to perform a multipart upload.
snippet_tags:
- s3.java2.performMultiPartUpload.s3Client
services:
s3: {CreateMultipartUpload, UploadPart, CompleteMultipartUpload}
s3_Scenario_UseChecksums:
title: Use checksums to work with an &S3; object using an &AWS; SDK
title_abbrev: Use checksums
synopsis: use checksums to work with an &S3; object.
category: Scenarios
languages:
Java:
versions:
- sdk_version: 2
github: java/example_code/s3
sdkguide:
excerpts:
- description: The code examples use a subset of the following imports.
snippet_tags:
- s3.java2.basicOpsWithChecksums.import
- description: >-
Specify a checksum algorithm for the <code>putObject</code> method when you <ulink url="https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/PutObjectRequest.Builder.html">build the
<code>PutObjectRequest</code></ulink>.
snippet_tags:
- s3.java2.basicOpsWithChecksums.putObject
- description: >-
Verify the checksum for the <code>getObject</code> method when you <ulink url="https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/GetObjectRequest.Builder.html">build the GetObjectRequest</ulink>.
snippet_tags:
- s3.java2.basicOpsWithChecksums.getObject
- description: >-
Pre-calculate a checksum for the <code>putObject</code> method when you <ulink url="https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/PutObjectRequest.Builder.html">build the <code>PutObjectRequest</code></ulink>.
snippet_tags:
- s3.java2.basicOpsWithChecksums.putObjectPreCalc
- description: >-
Use the <ulink url="sdk-for-java/latest/developer-guide/transfer-manager.html" type="documentation">S3 Transfer Manager</ulink>
on top of the <ulink url="sdk-for-java/latest/developer-guide/crt-based-s3-client.html" type="documentation" >&AWS; CRT-based S3 client</ulink>
to transparently perform a multipart upload when the size of the content exceeds a threshold. The default threshold size is 8 MB.</para>
<para>You can specify a checksum algorithm for the SDK to use. By default, the SDK uses the CRC32 algorithm.
snippet_tags:
- s3.java2.basicOpsWithChecksums.multiPartTm
- description: >-
Use the <ulink url="https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Client.html">S3Client API</ulink>
or (S3AsyncClient API) to perform a multipart upload. If you specify
an additional checksum, you must specify the algorithm to use on the initiation of the upload.
You must also specify the algorithm for each part request and provide the checksum calculated
for each part after it is
uploaded.
snippet_tags:
- s3.java2.basicOpsWithChecksums.multiPartS3Client
services:
s3: {CreateMultipartUpload, UploadPart, CompleteMultipartUpload}
93 changes: 75 additions & 18 deletions javav2/example_code/s3/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<artifactId>S3J2Project</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<java.version>1.8</java.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<build>
<plugins>
Expand All @@ -22,16 +22,46 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<version>3.11.0</version>
<configuration>
<source>${java.version}</source>
<target>${java.version}</target>
<release>11</release>
</configuration>
<!-- The following execution section process the plugin annotations for apache logging for the custom memory appender, MemoryLog4jAppender, used in the test for ParseUri.java -->
tkhill-AWS marked this conversation as resolved.
Show resolved Hide resolved
<executions>
<execution>
<id>log4j-plugin-processor</id>
<goals>
<goal>compile</goal>
</goals>
<phase>process-classes</phase>
<configuration>
<proc>only</proc>
<annotationProcessors>
<annotationProcessor>org.apache.logging.log4j.core.config.plugins.processor.PluginProcessor</annotationProcessor>
</annotationProcessors>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
<artifactId>maven-checkstyle-plugin</artifactId>
<version>3.2.2</version>
<configuration>
<configLocation>checkstyle.xml</configLocation>
<consoleOutput>true</consoleOutput>
<failsOnError>true</failsOnError>
<linkXRef>false</linkXRef>
</configuration>
<executions>
<execution>
<id>validate</id>
<phase>validate</phase>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
Expand All @@ -44,6 +74,13 @@
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-bom</artifactId>
<version>2.20.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
Expand All @@ -53,15 +90,6 @@
<version>5.9.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>secretsmanager</artifactId>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.10.1</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
Expand All @@ -78,6 +106,11 @@
<artifactId>junit-platform-commons</artifactId>
<version>1.9.2</version>
</dependency>
<dependency>
<groupId>software.amazon.awssdk.crt</groupId>
<artifactId>aws-crt</artifactId>
<version>0.25.1</version>
</dependency>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-launcher</artifactId>
Expand All @@ -88,23 +121,47 @@
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>sso</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>ssooidc</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3-transfer-manager</artifactId>
<version>2.20.26</version>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>kms</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>secretsmanager</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3control</artifactId>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.20.0</version>
<artifactId>log4j-core</artifactId>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j2-impl</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.12.0</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.10.1</version>
</dependency>
</dependencies>
</project>
Loading
Loading