Unlocking the Power of DynamoDB Condition Expressions

TL;DR;

This article explores DynamoDB condition expressions, highlighting their potential to enhance database operations. They help prevent accidental data overwriting and facilitate conditional updates. The article also details how to handle errors efficiently when condition checks fail, thus reducing the need for additional requests. By leveraging these expressions, developers can create more efficient and robust applications.

Introduction

Are you tired of your PutItem operations overwriting existing items?

Or your UpdateItem inserting a new object if the provided keys don't exist?

Are you still reading an object before updating it to ensure its current state?

If you replied to any of the above questions with a yes, the title triggered your inner refactoring freak or you're just keen to learn new approaches, then this article is for you. In this article we will discover and learn how to take the most advantage of (what I think) is a somewhat unknown feature: DynamoDB Condition Expressions.

DynamoDB Expressions

When working with DynamoDB developers use expressions to state what actions or conditions they want DynamoDB to process.

There are different types of expressions that developers can use:

• Projection Expressions - Similar to the SELECT statement in SQL, this type of expression is used to select what attributes you want DynamoDB to return when reading objects. By default, DynamoDB returns all attributes (same as using SELECT *).

• Update Expressions - Similar to the UPDATE statement in SQL, this expression is used in the UpdateItem requests to specify what changes you want to apply to the desired object. Developers can ADD, SET, REMOVE or DELETE attributes.

• Condition Expressions - This article's star of the show; could be compared to the WHERE clause of a SQL statement. This expression type will allow developers, as the name implies, to ensure a condition is met before the data is manipulated.

Using Condition Expressions

Developers can use condition expressions in any DynamoDB operation that manipulates data, such as PutItem or UpdateItem, and using such expressions opens the door to a whole new set of business logic and fail-safes that can be implemented without adding any additional requests to DynamoDB.

Avoid overriding existing objects

DynamoDB will, by default, override any existing data if an PutItem operation is triggered with existing keys.

There are scenarios where one would like to store a new object but only if it doesn't exist, for example, if we want to do some kind of idempotency check or if we want to avoid overriding existing data.

To do that, one could approach the issue by doing a Get/Query and only triggering the Put if the item doesn't exist, but using condition expressions allows us to do everything in a single request:

  const command = new PutCommand({
    ...,
    ConditionExpression: "attribute_not_exists(#id)",
    ExpressionAttributeNames: {
      "#id": "PK",
    },
  });
  await client.send(command);

In the above example snippet we're doing a PutItem request and adding the condition expression of attribute_not_exists(#id), this will indicate DynamoDB to only store the provided object if the current primary key (PK) is not in use.

By doing it, DynamoDB won't override any existing data and will also throw an error if the condition is not met (an object already exists).

Full example here.

Update only existing items

Similar to the insert, DynamoDBs behaviour on UpdateItem can also be a tad counter-intuitive, as it will execute and UPSERT operations and not an UPDATE.

For some scenarios it won't be an issue to always execute UPSERT operations by default, but in general one will want the UPDATE operation to fail if the record doesn't exist to avoid creating new records with only partial data that will probably trigger errors downstream.

Implementing that change is very easy, as one just needs to add a simple condition expression as follows.

    const command = new UpdateCommand({
      ...,
      ConditionExpression: "attribute_exists(#id)",
      ExpressionAttributeNames: {
        "#id": "PK",
      },
    });
    await docClient.send(command);

In the above example snippet we're doing a UpdateItem request and adding the condition expression of attribute_exists(#id), this will indicate DynamoDB to only update the desired object if the current primary key (PK) exists.

With this condition, if DynamoDB can't find an update with the provided keys, the current primary key (PK) will not exist and the condition would fail, meaning that the UpdateItem request will throw an error.

Full example here.

Conditional Updates

But the magic of Condition Expressions doesn't stop on avoiding unexpected behaviours from DynamoDB, they open a whole new range of possibilities.

For example, let's imagine a scenario where, during the login flow, you want to check if the account is frozen or not (user should not be able to log in) and if the account is active, update a field storing the timestamp and device information of the current login.

For this scenario, one could first query the data to check the account status and if the check is successfull send another request to update the object with the desired login information. This approach is technically correct and would probably work for most requirements, but it opens the following concerns:

• Number of Requests - The most common scenario will be that the account is not frozen and the login should succeed, meaning that for almost all logins two requests will be sent to DynamoDB.

• Data Consistency - This approach would be unable to handle or control any change on the data between both requests, meaning that, if the account is being deactivated at the same time, the login could succeed when it should have failed.

With the use of the Condition Expression we can move that business logic to be execute on DynamoDBs side.

🤯
Did you know that you can edit nested attributes directly? You just need to use the .(dot) or [x] notation when referencing it! Check the docs

    const command = new UpdateCommand({
      ...,
      ConditionExpression: "attribute_exists(#id) AND #obj.#key = :expected",
      ExpressionAttributeNames: {
        "#id": "PK",
        "#obj": "accountInformation",
        "#key": "isFrozen",
      },
      ExpressionAttributeValues: {
        ":expected": false,
      },
    });
    await docClient.send(command);

In this example snippet we're adding the condition expression of attribute_exists(#id) AND #obj.#key = :expected, this will indicate DynamoDB to only update the desired object if the current primary key (PK) exists and if the nested attribute accountInformation.isFrozen is set to false.

With this condition, we move the business logic to check the account status to be executed by DynamoDB directly, meaning that the object will only be updated if the condition is met. This way we only need to consider that the login should only fail if the UpdateItem request has failed.

Full example here.

Handling Conditional Check Failures

Now we know how to implement the condition expressions but we are missing what to do when the condition fails.

What if the expression contains multiple conditions and we want to have a custom error message depending on wich condition failed, should we trigger a new request to query the object?

The answer is no, and this is probably one of the hidden features that will help you the most. At this point, you are probably aware of the ReturnValues option to configure what information you would like DynamoDB to return, but did you know there is also ReturnValuesOnConditionCheckFailure?

It works similarly as the ReturnValues option, but with only two available options:

• NONE - the default, the thrown error won't have any information regarding the current object attributes and values.

• ALL_OLD - setting this option will request DynamoDB to throw an error that contains the old (current) object information in the response.
  

  try {
    const command = new UpdateCommand({
      ...,
      ReturnValuesOnConditionCheckFailure: "ALL_OLD",
    });

    const response = await docClient.send(command);
    return response;
  } catch (error) {
    if (!(error instanceof ConditionalCheckFailedException)) throw error;
    const oldRecord = unmarshall(error.Item);
    // Apply any logic to check why the update failed
  }

The above snippet showcases how one could approach the error handling for conditional expressions with three simple steps:

1. Set ReturnValuesOnConditionCheckFailure: "ALL_OLD" to ensure you receive the object information if the condition fails.

2. Ensure you only catch the expected exception types by re-throwing any error that is not an instance of ConditionalCheckFailedException.

3. Retrieve and unmarshall the record data from the returned error with unmarshall(error.Item). The Item value will be the marshalled object that we tried to update, it's important to unmarshall it or ensure we access it's properties respecting the marshalling.

After those steps have been implemented, one can add any additional desired business logic to check why the operation failed and handle the error appropriately based on which of the conditions failed.

Full example here.

Conclusions

In conclusion, DynamoDB condition expressions provide a powerful tool that can greatly enhance the efficiency and reliability of your database operations. They allow developers to implement complex business logic directly within their database operations, reducing the number of requests and mitigating risks of data inconsistency.

This feature can help prevent unintended data overwriting, ensure updates only apply to existing items, and enable conditional updates based on specific criteria. With the ability to return current object data when condition checks fail, developers can handle errors more effectively without making additional requests.

By fully utilizing this feature, you can unlock the true potential of DynamoDB and create more robust and efficient applications.