Clean DynamoDB data automaticlly

During one of my last project I had to remove user data that was older than two weeks.

I was looking for setting up an AWS Lambda function that is running on a recurring schedule. The idea was that this function scan all the date fields in the dynamoDB table.

It is expensive to scan all items and filter out the items which are out of date. Then you need to set up some functionality to delete these items via a batch in Dynamodb. It is possible to set up it, but there is a more easier way to do it.

TTL in DynamoDB

Make sure you have a date field in your Dynamodb table. It needs to have a Unix epoch time format. You can give this field any name.

The next step is to select the TTL field on the table. Unfortunately this is not supported from the Cloudformation templates in AWS Amplify. This is a manual process in each new environment that you set up.

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/TTL.html

Javascript

This is how you can achieve this Unix epoch time format in JavaScript.

const numWeeks = 2; // after how many weeks the data needs to be removed
const now = date;
now.setDate(now.getDate() + numWeeks * 7);
const ttl = Math.round(now.getTime() / 1000);

 const setTTL = {
      ttlFieldName: ttl
  }; 

AWS Amplify

In your API schema you can create it like this.

type modelName
  @model{
  id: ID!
  ttl: Int!
}

Update

It is also possible to accomplish this via code. You can give this package a try: https://github.com/flogy/graphql-ttl-transformer

Conclusion

Your data will be removed automatically now (in this case in two weeks) according a recurring schedule.

About me

I am a huge fan of AWS Amplify and blog about certain topics that come along with it. If you have any questions related to the framework, React of React Native then you can always reach out to me.

Twitter

Do you want to be updated about new blogs?
Follow me on twitter: https://twitter.com/ramonpostulart