DEV Community

Cover image for NEW: DynamoDB Streams Filtering in Serverless Framework

NEW: DynamoDB Streams Filtering in Serverless Framework

Pawel Zubkiewicz on December 06, 2021

From this article, you will learn how to utilize recently released functionality of Streams Filtering with DynamoDB and Lambda. We will move deep...
Collapse
 
droizman profile image
kolektiv

One more catch, you can't use filtering with global tables, your filter will not be evaluated and your function will not be called. Confirmed with aws support.

Collapse
 
nosqlknowhow profile image
Kirk Kirkconnell

Are you saying the two cannot be used at the same time or that you cannot use the event filter to filter Global Tables traffic from source to destination regions? These are two VERY different things.

Collapse
 
droizman profile image
kolektiv

You cannot use stream filtering on a Global Table. Your Global Table will continue replicate/sync but your stream filter will not be evaluated and your trigger will not fire.

Thread Thread
 
simiobs profile image
simi-obs

I am sorry but this answer IMO is very misleading. You did not actually answer Kirk's question properly. He is correct when he says you cannot use the event filter to filter Global Tables traffic from source to destination regions

But you can actually use this feature (of event filtering for lambdas). I confirmed with AWS. Here is the link: repost.aws/questions/QUgOGCJJhAStm...

Thread Thread
 
droizman profile image
kolektiv

this is the reply from AWS support:

-->
That said, DynamoDB streams capture any modification to a DynamoDB table for example an insert,update or delete. We can attach a trigger to the stream, specifically a lambda function. This lambda function will be invoked every time a modification is made to the table. There is no option to filter this action on only certain items, the reason for this is that the streams are required to keep the replica table in the different region in sync with the base table.

We can however add logic to our trigger function to discard any items that do not contain the required/desired tag/value. However the function will still be triggered if the item updated/inserted or deleted does not contain the value/tag you want to filter on.
<--

So you can see according to AWS, on a global table your Lambda still gets called and ignores the filter.

Thread Thread
 
leeroy_hannigan profile image
Leeroy Hannigan • Edited

This is not correct information. The filtering is on the Event Source Mapping on the Lambda side which is completely decoupled from DynamoDB Stream. Event filtering works regardless, as Global Table replication system is completely separate from your Lambda trigger.

On a side-note, try this filter @koletiv

{
  "filters": [
    {
      "pattern": "{\"dynamodb\":{\"NewImage\":{\"region\":{\"S\":[\"us-west-2\"]}}}}"
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode
Thread Thread
 
pzubkiewicz profile image
Pawel Zubkiewicz

@droizman have you tested that?

Collapse
 
vince_hirefunnel_co profile image
Vince Fulco (It / It's)

Great article. Thank you.

One typo, "Therefor, let me take you through the refactoring process."-->"Therefore, let me take you through the refactoring process."

Collapse
 
dimsum_chen_78104a4aa3c31 profile image
Dimsum Chen

Can filtering be used to compare the NewImage value of an attribute with the OldImage value of an attribute?

Collapse
 
suhasdeshpande profile image
Suhas Deshpande

Did you find any reasonable way out of this?

Collapse
 
steveswinbank profile image
Steve

Apparently the answer is 'no': repost.aws/questions/QUgOG4PLWlSt2...