How to Configure Event Notifications on S3 Bucket

In AWS, S3 is a simple storage service that allows you to store your files and other data as S3 objects on a cloud without managing the underlying hardware. AWS S3 service is serverless, meaning that you do not need to provide any storage size for an S3 bucket, so whenever you place an object in a bucket, the bucket capacity itself increases to accommodate the newly added object. The maximum size of the S3 bucket is theoretically unlimited with a single file size upto 5 TB, and you can upload a single file or object upto 5 GB, so you need to split any file greater than 5 GB in size.

We usually perform multiple operations over an S3 bucket, so at a certain point, we may need to trigger some other function, operation or service whenever there is some operation performed on our bucket. This can be achieved using S3 event notifications. These event notifications can be set to upload the files, update the versions of files, delete the older objects, S3 lifecycle transactions, and many other operations. The S3 bucket will notify the target about that particular action with event notifications enabled. The notification delivery time is short and in the range of seconds and minutes.

At present, three AWS services can be triggered with AWS S3.

• Simple Queue Service (SQS)
• Simple Notification Service (SNS)
• Lambda Functions

Now let’s take a deeper dive and see how it is practically done.

Configure S3 Notifications for AWS SQS

First, we will create an S3 bucket for which we will configure event notifications. For this, simply login to your AWS account and search for S3 service using the search bar available at the top of your management console.

The S3 console will appear in front of you; simply click on create bucket.

In this section, you need to provide a bucket name that must be universally unique.

Next, you need to specify the AWS region where you want your S3 bucket to be placed. We know that S3 is a global service, and it is accessible from any region, but still, you need to define in which region you want to store your data.

Next, there are some other settings you can manage like versioning, encryption and public access, but you can simply leave them as default as we have nothing to do with them at present. So just simply drag down and click on the create bucket in the bottom right corner to finish the bucket creation process.

So finally, we have created a new S3 bucket in our AWS account.

Now we need to create a target for our event notifications, and we will choose the SQS service. In the top search bar of the AWS console, search for amazon’s simple queue service.

Click on Create Queue to create a new SQS topic.

The first option is to choose here is the type of our SQS that we will leave to default, i.e., the standard one.

Next, we need to provide the SQS topic name.

Now to enable our S3 bucket to write to this SQS service, we need to provide the right policy to SQS. For this, select advanced in the access policy section.

The policy can be edited directly if you have good skills with JSON and other stuff, but for simplicity, we will use the AWS-provided Policy generator. Simply click on the link below to open the policy generator.

In the policy generator, first select the policy type, which will be SQS Queue Policy.

Next, we need to choose the effect which is ALLOW here. The principle will be our S3 bucket for which we are configuring the event notifications, so we need to put the bucket ARN here. The resource is the service on which we need this policy to be applied, so this must be our SQS topic ARN.

The Action we need to select from the drop-down list is Send Messages, as we want our S3 bucket to send the message to our SQS queue.

Now click on add statement, and then select the create policy to get the policy in json format.

Simply copy the policy contents from there and place them in the SQS policy.

Now scroll down in the SQS create section and click on create a queue. Leave all other settings as it is for now.

Finally, now get back to our S3 bucket and select the properties tab.

Drag down to Event Notifications and click on create event notifications.

First, provide the name of your event.

You need to choose the event type for which you want to enable the notifications. There is a long list you can choose from; for now, let’s choose all object creation events.

We need to select the destination for our event notifications. For this, select the SQS service and then select the name of your SQS topic.

At last, simply click on save changes to complete the process.

So finally, we are good to go with the S3 event notifications. Now upload an object in the S3 bucket and check if it triggers the SQS.

We have uploaded an object to our S3 bucket.

If you go to our SQS and poll for messages, you will get notifications for those events we have enabled.

So we have successfully configured S3 event notifications for an SQS topic.

Configure S3 Notifications for AWS SNS

We will see another example for S3 event notifications for SNS, but this time instead of using the management console, we will be using the AWS command-line interface (CLI). So first step is to simply create the buckets in our AWS account, for which we simply need to use the following command.

To view the available S3 buckets in your account, you can use this command

Now we need to create an SNS topic. For that, use the following command on CLI

We need a configuration file to define the event type and SNS topic ARN. The file will be in JSON format.

Now finally, we can enable the event notification using the following command.

So we have successfully enabled S3 event notifications for our SNS topic.

Conclusion:

S3 event notifications allow us to notify other services about the operation carried out on our S3 bucket. These notifications can be used to trigger Lambda functions, using which we can invoke rest APIs to run another automated task. The notifications can also be sent to SQS or SNS, which are very useful for decoupling our applications. Using SQS and SNS, this scheme can further be linked with RDS so we can develop our database for particular S3 events.