DynamoDB allows you to delete or put terabytes of data using a single BatchWriteItem operation. The code or command requires a single BatchWriteItem call to make up to 25 put or delete item operations. While this is equivalent to up to 16MB of stored content, the data can be more than that since some items can be above 400 KB during transmission.
This tutorial focuses on how to Batch Delete in DynamoDB. It will focus on the BatchWriteItem operation and provide various methods of how to Batch Delete items using the BatchWriteItem API call. Of course, you will find viable examples.
Methods of How to Batch Delete in DynamoDB
Methods of How to Batch Delete in DynamoDB
Using the BatchWriteItem Operation
In Amazon DynamoDB, you can use the batchWriteItem operation to delete multiple items from one or more tables in a single request. The utility takes a request object that contains a RequestItems map, where the keys are table names and the values are lists of DeleteRequest objects. Each DeleteRequest object must have the primary key of the item you want to delete.
Below is an example of how you can use the batchWriteItem operation in the AWS SDK for Python (Boto3) to delete multiple items from a table:
# Create a client for DynamoDB
dynamodb = boto3.client('dynamodb')
# Define the table name and primary key names
table_name = 'MyTableName'
key_name = 'Id'
# Create a list of items to delete
items_to_delete = [
{key_name: {'S': 'item1'}},
{key_name: {'S': 'item2'}},
{key_name: {'S': 'item3'}},
]
# Prepare the request object for the batchWriteItem operation
request = {
'RequestItems': {
table_name: [
{
'DeleteRequest': {
'Key': item
}
} for item in items_to_delete
]
}
}
# Execute the batchWriteItem operation
response = dynamodb.batch_write_item(**request)
print(response)
You can use the BatchWriteItem operation with your preferred AWS SDK.
Batch Delete Items in DynamoDB Using the WriteBatchItem Operation
You should consider using the batchWriteItem operation in combination with a Scan operation to retrieve items that match a specified filter. Then, delete them in multiple batches if you need very many items deleted at once.
Your code will be as follows:
# Create a client for DynamoDB
dynamodb = boto3.client('dynamodb')
# Define the table name and primary key name
table_name = 'MyTableName'
key_name = 'Id'
# Define the filter expression and initial values for the scan
filter_expression = 'contains(#attr, :val)'
expression_attribute_names = {'#attr': 'some_attribute'}
expression_attribute_values = {':val': {'S': 'some_value'}}
# Initialize an empty list to store the items to delete
items_to_delete = []
# Use the scan operation to retrieve items that match the filter
response = dynamodb.scan(
TableName=table_name,
FilterExpression=filter_expression,
ExpressionAttributeNames=expression_attribute_names,
ExpressionAttributeValues=expression_attribute_values
)
# Extract the items from the response and append them to the list
items_to_delete.extend(response['Items'])
# Keep iterating while there are more items to retrieve
while 'LastEvaluatedKey' in response:
response = dynamodb.scan(
TableName=table_name,
FilterExpression=filter_expression,
ExpressionAttributeNames=expression_attribute_names,
ExpressionAttributeValues=expression_attribute_values,
ExclusiveStartKey=response['LastEvaluatedKey']
)
items_to_delete.extend(response['Items'])
# Define the maximum number of items that can be deleted in a single batch
batch_size = 25
# Initialize a variable to keep track of the starting index for the next batch
start_index = 0
# Loop until all items have been deleted
while start_index < len(items_to_delete):
# Determine the ending index for the current batch
end_index = min(start_index + batch_size, len(items_to_delete))
# Prepare the request object for the batchWriteItem operation
request = {
'RequestItems': {
table_name: [
{
'DeleteRequest': {
'Key': {
key_name: item[key_name]
}
}
} for item in items_to_delete[start_index:end_index]
]
}
}
# Execute the batchWriteItem operation
dynamodb.batch_write_item(**request)
# Update the starting index for the next batch
start_index = end_index
print(‘Deleted {} items from table {}’.format(len(items_to_delete), table_name))
In the example above, the scan operation retrieves items from the table that match a specified filter (some_attribute contains some_value). The items will make up a list before being processed in batches of 25 (or fewer) items simultaneously using the batchWriteItem operation. The script loops through all the batches of items and deletes them one batch after another.
Conclusion
The Batch Delete feature in DynamoDB can significantly boost your database’s performance, particularly when dealing with a lot of data. Notably, the BatchWriteItem operation does not behave like the DeleteItem command. For example, it does not return the details of the deleted items in your response.
