AWS CLI: 10 useful S3 commands

Chetan Kapoor

8 years ago

Here are 10 useful s3 commands

Install Virtual | 10 useful s3 commands. The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts.

The AWS CLI introduces a new set of simple file commands for efficient file transfers to and from Amazon S3.

How to install AWS CLI on Linux

The first things how to install the aws cli on linux, just execute the following command on your terminal before you need Python 2 version 2.6.5+ or Python 3 version 3.3+

pip install --upgrade --user awscli

How to install AWS CLI on Mac

We will use brew to install aws cli

Install Brew on mac

brew install awscli

Create S3 bucket using AWS CLI

We will start with creating the bucket using aws cli, we will use mb command to create a bucket

aws s3 mb s3://installvirtual make_bucket: installvirtual

List of S3 buckets

The next thing is to list all the buckets available

aws s3 ls

Then you will get list of all your buckets available

Copy files into S3 bucket

Now the next step is to copy some files into our s3 bucket

aws s3 cp sample.txt s3://installvirtual

Then you will see output

upload: ./sample.txt to s3://installvirtual/sample.txt

List files inside s3 bucket

It’s time to list all the files we have inside our bucket, just execute following command and change the name of bucket with your bucket name

aws s3 ls s3://installvirtual

It will show the following output

2017-06-14 11:40:27 0 sample.txt

If you have more file it will show all the file here.

Copy file from S3 to our machine

Now there would be a situation where you want to download files from s3 to local system so you can do this by following

aws s3 cp s3://installvirtual/sample.txt .

Now this command will copy the sample.txt from s3 bucket to our system in current directory

download: s3://installvirtual/sample.txt to ./sample.txt

You will get the above output. You can change the directory also

aws s3 cp s3://installvirtual/sample.txt /tmp download: s3://installvirtual/sample.txt to ../../tmp/sample.txt

It will download files in /tmp directory.

Sync S3 bucket

There must be a situation that you need to sync s3 bucket with another or you need to sync local directory with s3 bucket so s3 sync is a command to solve our problem let’s see how we can so this

Sync local directory with S3 bucket

aws s3 sync app s3://installvirtual

The above command will sync local app directory with s3 bucket but it will only upload the files but doesn’t remove the existing or extra files which is already available in s3 bucket bu not in local directory, if you want to sync and delete the extra files use –delete flag as shown below

aws s3 sync s3://installvirtual . --delete

delete: ./sample.txt download: s3://installvirtual/abc8.txt to ./abc8.txt download: s3://installvirtual/abc10.txt to ./abc10.txt download: s3://installvirtual/abc2.txt to ./abc2.txt download: s3://installvirtual/abc5.txt to ./abc5.txt download: s3://installvirtual/abc6.txt to ./abc6.txt download: s3://installvirtual/abc3.txt to ./abc3.txt download: s3://installvirtual/abc4.txt to ./abc4.txt download: s3://installvirtual/abc1.txt to ./abc1.txt download: s3://installvirtual/abc9.txt to ./abc9.txt download: s3://installvirtual/abc7.txt to ./abc7.txt

Sync s3 bucket to another s3 bucket

aws s3 sync s3://installvirtual s3://installvirtual1 copy: s3://installvirtual/abc3.txt to s3://installvirtual1/abc3.txt copy: s3://installvirtual/abc6.txt to s3://installvirtual1/abc6.txt copy: s3://installvirtual/abc5.txt to s3://installvirtual1/abc5.txt copy: s3://installvirtual/abc8.txt to s3://installvirtual1/abc8.txt copy: s3://installvirtual/abc7.txt to s3://installvirtual1/abc7.txt copy: s3://installvirtual/abc10.txt to s3://installvirtual1/abc10.txt copy: s3://installvirtual/abc2.txt to s3://installvirtual1/abc2.txt copy: s3://installvirtual/abc1.txt to s3://installvirtual1/abc1.txt copy: s3://installvirtual/abc4.txt to s3://installvirtual1/abc4.txt copy: s3://installvirtual/abc9.txt to s3://installvirtual1/abc9.txt

Remove files from s3 bucket

Now we will remove file from the bucket using rm command let’s see

aws s3 rm s3://installvirtual/abc1.txt delete: s3://installvirtual/abc1.txt

This above will delete the specified file.

Remove all files from s3 bucket

Now you need to empty the bucket or remove all the files from your S3 bucket so just use the following commands to do this

aws s3 rm s3://installvirtual/ --recursive delete: s3://installvirtual/abc3.txt delete: s3://installvirtual/abc10.txt delete: s3://installvirtual/abc2.txt delete: s3://installvirtual/abc5.txt delete: s3://installvirtual/abc4.txt delete: s3://installvirtual/abc8.txt delete: s3://installvirtual/abc9.txt delete: s3://installvirtual/abc7.txt delete: s3://installvirtual/abc6.txt

Here we used –recursive option to remove all file from the bucket

Removing S3 bucket

Now there will be a time you need to remove s3 bucket so let’s remove it, you need to emplty the bucket before deleting otherwise you will get error like

remove_bucket failed: s3://installvirtual An error occurred (BucketNotEmpty) when calling the DeleteBucket operation: The bucket you tried to delete is not empty

So i have removed all the files from the bucket then execute

aws s3 rb s3://installvirtual remove_bucket: installvirtual

This command will remove your bucket.

I think these commands would be very useful while working with aws cli.

If you like it just share it with your friends or colleagues or comment below.

If you got some issues using this please comment below.