DynamoDb stream lambda trigger

Assignment

Let's take a look on a typical scenario, once it is required to subscribe to DynamoDb stream events and trigger AWS lambda function. In theory and in tutorials the task looks quite simple:

• setup a lambda
• add event source with the table attached. Sample:

private subscribeToDynamoDbStream(
    lambdaRole: Role,
    layer: LayerVersion,
    applicationName: string,
    targetTable: ITable) {
    const lambda = new Function(this, `${applicationName}-lambda`, 
    {
          code: Code.fromAsset('functions/custom-trigger/src'),
          functionName: `${applicationName}-lambda`,
          handler: handler,
          layers: [layer],
          role: lambdaRole,
          runtime: Runtime.NODEJS_12_X,
     });

     lambda.addEventSource(new DynamoEventSource(targetTable, {
     startingPosition: StartingPosition.TRIM_HORIZON, }));
}

This scenario works well in case of the single repository, once lambda, dynamodb and dynamodb stream are provisioned in the same stack.

Issue

In more complex layout the dynamodb itself is created and corresponding dynamodb stream is provisioned in a different stack. The dynamodb arn is saved to the SSM parameter store, or the name is known.
So, first import table:

private importTargetTable(environment: string, tableArn: string) {
    const table = Table.fromTableArn(this, 'ImportedTable', tableArn);
    return table;
  }

Then call subscribeToDynamoDbStream function:

const importedTable = importTargetTable(this.environment, this.tableArn);
this.subscribeToDynamoDbStream(lambdaRole, layer, applicationName, importedTable);

The result is unexpected: DynamoDB Streams must be enabled on the table [tablename]. The AWS console clearly shows that the Dynamodb stream is enabled. Quick googling recommends the most obvious thing: enable already enabled feature. CDK source code inspection reveals that during fromTableArn invocation the stream arn most probably has not been loaded.

Solution

First of all there should be DynamoDb stream arn exported to SSM during DynamoDB provisioning:

const targetTable = new Table(this, `targeTableName`, {
   partitionKey: { ... },
   sortKey: { ... },
   ...
   stream: StreamViewType.NEW_AND_OLD_IMAGES,
   tableName: `targetTableName`
    });
   // Export target table stream arn to SSM
   new StringParameter(this.ctx, `targetTable-stream-arn`, {
      description: "Exported stream arn",
      parameterName: `/targeTable-stream-arn`,
      stringValue: targetTable.tableStreamArn!
    });  

In a stack where a lambda is created and subscription to the dynamodb stream is setup, tableStreamArn is read from SSM.

Option 1

Change the way how ITable is constructed:

private importTargetTable(environment: string, tableArn: string, tableStreamArn: string) {
    const table = Table.fromTableAttributes(this, 'ImportedTable', {
        tableArn: tableArn,
        tableStreamArn: tableStreamArn
    return table;
  }

Option 2

Technically speaking AWS lambda with DynamoDb stream trigger does not have to be aware of DynamoDb table (and it's arn). Option 2 could be considered as an improvement of the code above: 'Invent and Simplify'.
Once the lambda is created, EventSourceMapping object could be used to enable DynamoDb stream trigger for the lambda usin tableStreamArn only:

const lambda = new Function(this, ...);
new EventSourceMapping(this, `lambda-trigger-mapping`, {
    eventSourceArn: tableStreamArn,
    startingPosition: StartingPosition.TRIM_HORIZON,
    // any necessary options of a choice, e.g. batchSize, retryAttempt, etc.
    target: lambda
});

I hope that this post would help you in your CDK learning journey.
Best wishes..

Comments

[troylateral]

Option 1 code snippet seems incomplete.
Also i dont have a TableStreamArn so how can i call this function to get one? Do i manually create a tableStreamArn and if so how?