How to Download Multiple Files From S3 Using AWS CLI

The AWS CLI is a “one” solution for almost all AWS Services and their operations. The AWS CLI can easily be used to configure any service and perform any action, even administrative actions like adding and removing users. The AWS CLI can also be used to download files from the AWS S3 to your local pc, which is exactly what will be demonstrated in this guide.

The prerequisites of this guide include:

• An S3 Bucket
• Configured AWS CLI

Downloading Files From S3 With AWS CLI

To download files from the aws s3, the “cp” command is used, and the syntax of the command is as

However, to download multiple files from this command, you also need to include the “recursive” flag which would mean that this copy command will be executed for every item in the specified “bucket\folder”.

To demonstrate the working of this command, this is the bucket that has been created in the S3:

Download all Files From S3 Bucket

Open up a command prompt and then type the following command access and list the files inside of the S3 bucket:

In the above command, make sure to replace the name of the bucket with the name of the bucket that you have created on your AWS account. Upon the execution of this command, the following result is displayed on the Command Prompt:

Now, let’s download all of these files using the following command:

This command will initiate the download process, and the files will be downloaded in the current working directory of the command prompt:

To confirm that the files have been downloaded, head inside the current directory of the command prompt, which in the above examples is “\Desktop\Download\”:

This confirms that all of the files have been downloaded using the above command.

Downloading Multiple Files Using AWS CLI

In case you do not want to copy all the files from inside a bucket but rather want to download specific files at the same time, then include the “–exclude “*” and then include the name of the files using the flag “–include “nameOfFile”.

To do this, use the following command:

In the above command:

• The exclude “*” tell the copy command to exclude all the files
• Then the “–include” flag is used to specify which files are to include one by one
• Overall, 2 files are being downloaded, 1.gif and demo.txt

Executing the above command produces the following outcome on the Command Prompt:

Heading inside the folder “Desktop\Download” shows us the following files:

Both of the files have been downloaded to the local PC.

Conclusion

To download multiple files using the AWS CLI, using the “cp” command but with the recursive flag “–recursive”. However, this will cause the copy command to be executed for each item in the bucket, and thus, as a result, it will download all files from the S3 bucket. In case, you want to specify which files to download, then simply use the exclude flag to first exclude all the files from the recursion and then manually include each file to be downloaded using the “include” flag.