How to Observe EventBridge Events with AppSync Subscriptions

I recently came across David Boyne's blog post: How to Observe EventBridge Events with Postman and WebSockets. What a great idea! But, then I thought:

I can do the same with AppSync Subscriptions!

I had to try! Here is what I achieved:

Building the basic AppSync API

The idea was simple. I needed the following components:

• An AppSync API
• A Mutation that receives events from EventBridge
• A Subscription that is attached to the aforementioned Mutation
• An EventBridge rule that sends events to the AppSync Mutation (target)

I also wanted to be able to filter events I was interested in. Here, I thought about two options:

1. Filter the events in the EventBridge rule.
2. Send all events to AppSync and use AppSync to filter them, thanks to subscription arguments

I went with the second approach. It would give me more flexibility to filter the events at query time instead of having to re-deploy each time I wanted a new filter.

Here is the GraphQL Schema I created:

type Mutation {
  sendEvent(event: EventBridgeMessageInput!): EventBridgeMessage
}

type Subscription {
  subscribe(
    source: String
    detailType: String
    account: String
    resources: [String!]
  ): EventBridgeMessage
    @aws_subscribe(mutations: ["sendEvent"])
}

type EventBridgeMessage {
  id: ID!
  version: String!
  detailType: String!
  source: String!
  account: String!
  time: AWSDateTime!
  region: String!
  resources: [String!]
  detail: AWSJSON!
}

input EventBridgeMessageInput {
  id: ID!
  version: String!
  detailType: String!
  source: String!
  account: String!
  time: AWSDateTime!
  region: String!
  resources: [String!]
  detail: AWSJSON!
}

I also needed to setup the Mutation. I used a NONE data source for that and a simple mapping template that just returns the received payload.

All done! Now, by executing the sendEvent Mutation, it gets delivered to the subscription! 🙌

All that was left to do was to configure EventBrige and set the Mutation as a target.

First attempt: API Destinations

My first attempt was to use API Destinations. I followed this awesome tutorial and defined my Input Path and Input Transformer rules which looked like this:

InputPathsMap:
  version: $.version
  id: $.id
  detailType: $.detail-type
  source: $.source
  account: $.account
  time: $.time
  region: $.region
  resources: $.resources
  detail: $.detail
InputTemplate: |
  {
    "query": "mutation SendEvent($event: EventInput!) { sendEvent(event: $event) { version id detailType source account time region resources detail } }",
    "operationName": "SendEvent",
    "variables": {
      "event": {
        "version": "<version>",
        "id": "<id>",
        "detailType": "<detailType>",
        "source": "<source>",
        "account": "<account>",
        "time": "<time>",
        "region": "<region>",
        "resources": "<resources>",
        "detail": <detail>
      }
    }

Unfortunately, that didn't work! 😞

The problem is that in EventBridge, the detail attribute is an arbitrary JSON object which could have any shape. This is the reason I used an AWSJSON type in my GraphQL schema (I wanted to receive any event). The problem is that AppSync expects the JSON to be stringified!

After some investigation, I could not find any way for EventBridge to stringify JSONs. So, that was a dead end.

AWS Lambda to the rescue!

If EventBridge cannot do it, Lambda surely can! So, I wrote a simple lambda that receives the event, reformats it and calls the AppSync endpoint. I then just configured the Lambda as an EventBridge target. (See the code here).

✍️ Note: I also added an IAM authentication method to the AppSync API that Lambda can use to call the Mutation (in addition to the API key used by the subscription).

All set! Now, running the following subscription:

subscription MySubscription {
  subscribe {
    resources
    region
    source
    version
    detailType
    detail
  }
}

And sending an event into Event Bridge

aws events put-events --entries '[{"DetailType": "my.detail.type", "Source": "my.source", "Detail": "{\"foo\": \"bar\"}"}]'

Response:

{
  "data": {
    "subscribe": {
      "resources": [],
      "region": "us-east-1",
      "source": "my.source",
      "version": "0",
      "detailType": "my.detail.type",
      "detail": "{\"foo\":\"bar\"}"
    }
  }
}

It works! 🎉

The power of AppSync subscriptions

One of the great features of AppSync subscriptions is that you can specify which changes you are interested in at query time. You can do that by adding arguments to the subscription endpoint. Whatever value you pass in the input, you will only receive changes that match the Mutation's response fields values.

So, I can now do queries such as

## Will match events with detail-type = "my.detail" only
subscription {
  subscribe(detailType: "my.detail") {
    id
    detailType
    detail
  }
}

## Will match events with source = "my.source" only
subscription {
  subscribe(source: "my.source") {
    id
    detailType
    detail
  }
}

## Will match events with detail-type = "my.detail" AND source = "my.source"
subscription {
  subscribe(detailType: "my.detail", source: "my.source") {
    id
    detailType
    detail
  }
}

Isn't that great? I can now listen to exactly the events I am interested in 🔥

Limitations & gotchas

Unfortunately, this technique has some limitations. It cannot filter events based on the content of the detail field. This is because the data comes stringified.

Also, filters only work when the values exactly match. You cannot use advanced filters such as prefix, anything-but, etc. These are filters supported by eventBridge, not by AppSync.

Note that any advanced filter can still be achieved through filters at the EventBridge rule level, of course!

Conclusion

In this post, I showed you how we can observe EventBridge events through AppSync subscriptions and how we can even filter them at query time. Although its usage is somewhat limited, it can probably still be very helpful when you only need to filter on the detailType or source values, for example. You can easily use it to debug/test your application.

Find the full code of this implementation on Github

A big thanks to David Boyne for the inspiration!

Comments

[Jones Zachariah Noel]

Benoît, this is a wonderful implementation. But I believe EventBridge API destination with AppSync endpoint passing the parameters with mutation body. No proof of concept just that I believe it works.

[Benoît Bouré]

Thanks @zachjonesnoel
Sorry, I am not sure what you meant by that.

As I explained in the article, an issue with that is that, because the content of the event can be arbitrary, it cannot match any GraphQL Type for it.
Using AWSJSON requires stringifying the payload, which is not supported by EventBridge transformations.

If you have an idea, I'd be happy to hear about it 🙌

[Jones Zachariah Noel]

Ah! Understood that EventBridge doesn't allow for stringifying so that we can use AWSJSON but what

I was intending to say is like if we have to perform a query/mutation from Lambda we can invoke it as API with request or axios (with NodeJS run-time) similarly as an API destination we could do a AppSync mutation invocation also. Let me see if I can successfully try it what I'm thinking! 😅 And will get back to you about it.