DEV Community

Discussion on: NEW: DynamoDB Streams Filtering in Serverless Framework

Collapse
 
droizman profile image
kolektiv

One more catch, you can't use filtering with global tables, your filter will not be evaluated and your function will not be called. Confirmed with aws support.

Collapse
 
nosqlknowhow profile image
Kirk Kirkconnell

Are you saying the two cannot be used at the same time or that you cannot use the event filter to filter Global Tables traffic from source to destination regions? These are two VERY different things.

Collapse
 
droizman profile image
kolektiv

You cannot use stream filtering on a Global Table. Your Global Table will continue replicate/sync but your stream filter will not be evaluated and your trigger will not fire.

Thread Thread
 
simiobs profile image
simi-obs

I am sorry but this answer IMO is very misleading. You did not actually answer Kirk's question properly. He is correct when he says you cannot use the event filter to filter Global Tables traffic from source to destination regions

But you can actually use this feature (of event filtering for lambdas). I confirmed with AWS. Here is the link: repost.aws/questions/QUgOGCJJhAStm...

Thread Thread
 
droizman profile image
kolektiv

this is the reply from AWS support:

-->
That said, DynamoDB streams capture any modification to a DynamoDB table for example an insert,update or delete. We can attach a trigger to the stream, specifically a lambda function. This lambda function will be invoked every time a modification is made to the table. There is no option to filter this action on only certain items, the reason for this is that the streams are required to keep the replica table in the different region in sync with the base table.

We can however add logic to our trigger function to discard any items that do not contain the required/desired tag/value. However the function will still be triggered if the item updated/inserted or deleted does not contain the value/tag you want to filter on.
<--

So you can see according to AWS, on a global table your Lambda still gets called and ignores the filter.

Thread Thread
 
leeroy_hannigan profile image
Leeroy Hannigan • Edited

This is not correct information. The filtering is on the Event Source Mapping on the Lambda side which is completely decoupled from DynamoDB Stream. Event filtering works regardless, as Global Table replication system is completely separate from your Lambda trigger.

On a side-note, try this filter @koletiv

{
  "filters": [
    {
      "pattern": "{\"dynamodb\":{\"NewImage\":{\"region\":{\"S\":[\"us-west-2\"]}}}}"
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode
Thread Thread
 
pzubkiewicz profile image
Pawel Zubkiewicz

@droizman have you tested that?