This article focuses on filter expressions. Therefore, we will define the filter expressions, explain why and when they are applicable, and provide a step-by-step guide on how to use them through relevant examples.
What Are Filter Expressions?
Filter expressions are a popular technique to filter the data in DynamoDB during Query and Scan operations. In DynamoDB, proper data modeling and organization rely on filtering. While most applications always have tons of saved data, you may urgently need an item from the vast mess.
Your ability to retrieve the correct data whenever you need it depends on the filtering capabilities of your database, and this is where the filter expressions help. They specify the results within the query item that you want to be returned to you as they discard the rest of the items.
You can apply the filter expressions on the server-side filters on item attributes after a Query operation finishes but before the server brings back the results of your Query call. This implies that your Query still consumes the same amount of read capacity, irrespective of whether you use a filter expression.
Besides, like ordinary Query operations, your 1 MB data limit for Query operations happens before the evaluation of your filter expression operation. You can use this operation to reduce the payload, search for specific items, and improve the simplicity and readability during application development.
Filter Expression Syntax and Examples
Notably, both filter expressions and key expressions use the same syntax. Besides, filter expressions and condition expressions can also use the same functions, comparators, and logical operators.
The other operators that filter the expressions can use also include the CONTAINS operator, the OR operator, the not-equals () Operator, the IN Operator, the BETWEEN operator, the BEGINS_WITH operator, the SIZE operator, and the EXISTS operator.
Example 1: Querying Using AWS CLI and DynamoDB Primary Keys
This example queries the Music table for a particular genre (partition key) and a specific artist (sort key). The utility only brings back a result for the items that match the particular partition key and sort key for the songs with the most views.
You can specify the number of views (#v) in the command. For example, we tag our minimum limit to 1,000 views to imply that only the results for the songs with over 1000 views will come back.
--table-name Music \
--key-condition-expression "Genre = :fn and Artist = :sub" \
--filter-expression "#v >= :num(1000)" \
--expression-attribute-names '{"#v": "Views"}' \
--expression-attribute-values file://values.json
Example 2: Using AWS CLI with Condition Expression
We can restructure the same Query as in the previous example but now with condition keys alongside our filters. It doesn’t include the sort key. Instead, it retrieves all the records for the specified artist with more than 1,000 views. It can also be reconstructed to provide orders above a given number for a particular username (customer_id).
--table-name Music \
--key-condition-expression "Username = :username" \
--filter-expression "Amount > :amount" \
--expression-attribute-values '{
":username": { "S": "artist" },
":amount": { "N": "1000" }
}' \
$LOCAL
An example of the result looks like this:
The given illustration shows that out of the 56 song titles for the same artist, only seven songs have more than 1,000 views. We have, however, truncated the figure for brevity purposes and included just the first and the last results in the list.
Example 3: Using Filter Expressions with the No-Equal () Operator
In the following Java utility, we want to query our table (Movie Collection) for all movies that do not equal to “Movie X”. Make sure that you use a filter expression with an attribute (#name) alongside an expression attribute value (:name) as illustrated in the following:
AWS.config.update({ region: "eu-west-1" });
const dynamodb = new AWS.DynamoDB.DocumentClient();
var params = {
TableName: 'movie collection',
KeyConditionExpression: '#PK = :PK',
FilterExpression : "#name :name", (filter expression)
ExpressionAttributeNames: { "#PK": "PK","#name": "name" }, (condition expression)
ExpressionAttributeValues: {
':PK': 'OgejhHrdRS453HGD4Ht44',
':name':"Movie X"
}
};
dynamodb.query(parameters, function (err, data) {
if (err) console.log(err);
else console.log(data);
});
Example 4: Using Filter Expressions with the Scan Operator
While the previous command uses <> to fetch only those items that aren’t equal to the movie name called Movie X, make sure that you use the key condition expressions here together with the filter expression. This is because it is impossible to filter the data in the Query operator without using a key condition expression.
TableName: 'movie collection',
FilterExpression : "PK = :PK and #name :name",
ExpressionAttributeNames: { "#name": "name" },
ExpressionAttributeValues: {
':PK': 'OgejhHrdRS453HGD4Ht44',
':name':"Movie X"
}
};
dynamodb.scan(parameters, function (err, data) {
if (err) console.log(err);
else console.log(data);
});
Conclusion
That is the end of our DynamoDB tutorial on filter expressions. You can use the filter expressions to retrieve a set of preferred data, filter the retrieved data after a scan or query, or return a set of data to a client. While it is applicable with an array of tools, there are instances when using the filter expressions is not viable. For example, you can only use them if you have a proper data model, when using a primary key, and when extracting large portions of data.
