Skip to content

Latest commit

 

History

History
264 lines (202 loc) · 7.9 KB

aws-cli.md

File metadata and controls

264 lines (202 loc) · 7.9 KB
copyright lastupdated keywords subcollection
years
2017, 2020
2020-10-26
cli, command line interface, object storage, s3
cloud-object-storage

{{site.data.keyword.attribute-definition-list}}

Using the AWS CLI

{: #aws-cli}

The official command-line interface for AWS is compatible with the {{site.data.keyword.cos_full}} S3 API. {: shortdesc}

Written in Python, it can be installed from the Python Package Index via pip install awscli. By default, access keys are sourced from ~/.aws/credentials, but can also be set as environment variables.

These examples were generated by using version 1.14.2 of the CLI. To check the version installed, run aws --version.

Configure the CLI to connect to {{site.data.keyword.cos_short}}

{: #aws-cli-config}

To configure the AWS CLI, type aws configure. Provide your HMAC credentials and a default region name. The "region name" used by AWS S3 corresponds to the code (LocationConstraint) that {{site.data.keyword.cos_short}} uses to define a storage class.

A list of valid codes for LocationConstraint can be referenced in the Storage Classes guide.

aws configure
AWS Access Key ID [None]: {Access Key ID}
AWS Secret Access Key [None]: {Secret Access Key}
Default region name [None]: {Provisioning Code}
Default output format [None]: json

This creates two files:

~/.aws/credentials:

[default]
aws_access_key_id = {Access Key ID}
aws_secret_access_key = {Secret Access Key}

{:codeblock}

~/.aws/config:

[default]
region = {Provisioning Code}
output = json

{:codeblock}

You can also use environment variables to set HMAC credentials:

export AWS_ACCESS_KEY_ID="{Access Key ID}"
export AWS_SECRET_ACCESS_KEY="{Secret Access Key}"

{:codeblock}

The IBM COS endpoint must be sourced by using the --endpoint-url option, and can't be set in the credentials file.

High-level syntax commands

{: #aws-cli-high-level}

Simple use cases can be accomplished by using aws --endpoint-url {endpoint} s3 <command>. For more information about endpoints, see Endpoints and storage locations. Objects are managed by using familiar shell commands, such as ls, mv, cp, and rm. Buckets can be created by using mb and deleted by using rb.

List all buckets within a service instance

{: #aws-cli-high-level-list-buckets}

aws --endpoint-url {endpoint} s3 ls
2016-09-09 12:48  s3://bucket-1
2016-09-16 21:29  s3://bucket-2

List objects within a bucket

{: #aws-cli-high-level-list-objects}

aws --endpoint-url {endpoint} s3 ls s3://bucket-1
2016-09-28 15:36       837   s3://bucket-1/c1ca2-filename-00001
2016-09-09 12:49       533   s3://bucket-1/c9872-filename-00002
2016-09-28 15:36     14476   s3://bucket-1/98837-filename-00003
2016-09-29 16:24     20950   s3://bucket-1/abfc4-filename-00004

Make a new bucket

{: #aws-cli-high-level-new-bucket}

Note: Personally Identifiable Information (PII): When naming buckets or objects, do not use any information that can identify any user (natural person) by name, location, or any other means. {:tip}

If the default region in the ~/.aws/config file corresponds the same location as the chosen endpoint, then bucket creation is straightforward.

aws --endpoint-url {endpoint} s3 mb s3://bucket-1
make_bucket: s3://bucket-1/

Add an object to a bucket

{: #aws-cli-high-level-upload}

aws --endpoint-url {endpoint} s3 cp large-dataset.tar.gz s3://bucket-1
upload: ./large-dataset.tar.gz to s3://bucket-1/large-dataset.tar.gz

You can also set a new object key that is different from the file name:

aws --endpoint-url {endpoint} s3 cp large-dataset.tar.gz s3://bucket-1/large-dataset-for-project-x
upload: ./large-dataset.tar.gz to s3://bucket-1/large-dataset-for-project-x

Copying an object from one bucket to another within the same region:

{: #aws-cli-high-level-copy}

$ aws --endpoint-url {endpoint} s3 cp s3://bucket-1/new-file s3://bucket-2/
copy: s3://bucket-1/new-file to s3://bucket-2/new-file

Delete an object from a bucket

{: #aws-cli-high-level-delete-object}

aws --endpoint-url {endpoint} s3 rm s3://mybucket/argparse-1.2.1.tar.gz
delete: s3://mybucket/argparse-1.2.1.tar.gz

Remove a bucket

{: #aws-cli-high-level-delete-bucket}

aws --endpoint-url {endpoint} s3 rb s3://bucket-1
remove_bucket: s3://bucket-1/

Create presigned URLs

{: #aws-cli-high-level-presign}

The CLI can create pre-signed URLs. These URLs allow for temporary public access to objects without changing any existing access controls.

$ aws --endpoint-url {endpoint} s3 presign s3://bucket-1/new-file

It's also possible to set a time-to-live for the URL in seconds (default is 3600):

$ aws --endpoint-url {endpoint} s3 presign s3://bucket-1/new-file --expires-in 600

Low-level syntax commands

{: #aws-cli-low-level}

The AWS CLI also allows direct API calls that provide the same responses as direct HTTP requests by using the s3api command.

Listing buckets:

{: #aws-cli-low-level-list-buckets}

$ aws --endpoint-url {endpoint} s3api list-buckets
{
    "Owner": {
        "DisplayName": "{storage-account-uuid}",
        "ID": "{storage-account-uuid}"
    },
    "Buckets": [
        {
            "CreationDate": "2016-09-09T12:48:52.442Z",
            "Name": "bucket-1"
        },
        {
            "CreationDate": "2016-09-16T21:29:00.912Z",
            "Name": "bucket-2"
        }
    ]
}

Listing objects within a bucket

{: #aws-cli-low-level-list-objects}

$ aws --endpoint-url {endpoint} s3api list-objects --bucket bucket-1
{
    "Contents": [
        {
            "LastModified": "2016-09-28T15:36:56.807Z",
            "ETag": "\"13d567d518c650414c50a81805fff7f2\"",
            "StorageClass": "STANDARD",
            "Key": "c1ca2-filename-00001",
            "Owner": {
                "DisplayName": "{storage-account-uuid}",
                "ID": "{storage-account-uuid}"
            },
            "Size": 837
        },
        {
            "LastModified": "2016-09-09T12:49:58.018Z",
            "ETag": "\"3ca744fa96cb95e92081708887f63de5\"",
            "StorageClass": "STANDARD",
            "Key": "c9872-filename-00002",
            "Owner": {
                "DisplayName": "{storage-account-uuid}",
                "ID": "{storage-account-uuid}"
            },
            "Size": 533
        },
        {
            "LastModified": "2016-09-28T15:36:17.573Z",
            "ETag": "\"a54ed08bcb07c28f89f4b14ff54ce5b7\"",
            "StorageClass": "STANDARD",
            "Key": "98837-filename-00003",
            "Owner": {
                "DisplayName": "{storage-account-uuid}",
                "ID": "{storage-account-uuid}"
            },
            "Size": 14476
        },
        {
            "LastModified": "2016-10-06T14:46:26.923Z",
            "ETag": "\"2bcc8ee6bc1e4b8cd2f9a1d61d817ed2\"",
            "StorageClass": "STANDARD",
            "Key": "abfc4-filename-00004",
            "Owner": {
                "DisplayName": "{storage-account-uuid}",
                "ID": "{storage-account-uuid}"
            },
            "Size": 20950
        }
    ]
}

Configure a Static Website

{: #aws-cli-configure-static-web}

aws --endpoint-url=https://<endpoint> s3 website s3://<bucketname>/ --index-document index.html --error-document error.html

{: pre}

Next Steps

{: #aws-cli-next-steps}

The detailed description of the RESTful API for {{site.data.keyword.cos_full_notm}} can be found in the S3 Compatibility API Documentation{: external} or the Configuration API Documentation{: external}.