DEV Community: Convoy

Why Microservices Need Webhook Gateways

Lotanna Nwose — Wed, 21 Jun 2023 14:16:35 +0000

A Webhook Gateway serves as a forward and reverses proxy to send and receive webhooks to client endpoints and from third-party providers. They are designed to enforce security, resilience and high availability on all webhook events for an organisation. We wrote a more detailed description here.

The image above describes the flow of webhook events from the microservices to client endpoints as well as from third parties to the microservices. In this article, we will discuss why webhooks gateways are relevant to the microservices architecture, and what common features are found in the webhooks gateway. Finally, we will discuss some of the caveats to bear in mind while adopting a webhooks gateway.

Why Do We Need Webhook Gateways?

The various advantages of a microservice architecture subsequently present unique challenges that Webhook Gateways are purpose-built to address.

Centralised Infrastructure to Decentralised Microservices

Microservice architectures split up several functions of an application into separate services, that are owned by different teams. This separation is typically based on a common domain e.g. Payments service, Notifications service etc. This design pattern engenders each team to build, deploy and maintain their services independently. Unlike a monolith application, teams are able to increase velocity and deliver features faster. This approach means increased complexity for webhook management: How are we able to route webhook events from third parties to the right microservice(s)? How are we able to route webhook events from backend services to client endpoints?

This is where a webhooks gateway comes in. They’re able to provide a central infrastructure capable of ingesting, and routing webhooks both internally to backend services and externally to client endpoints. This central platform also reduces the learning curve for new teams to add webhooks to their services.

Microservice Language and Protocol Agnostic

One of the major benefits of the microservice approach is the ability for teams to utilise the best tools for the job. For example, one team can decide to build their API using Rails, GraphQL and MongoDB, while another team can decide to build their API using Golang, gRPC and PostgreSQL.

Regardless of the technology stack chosen by each team. Webhook gateways rely on a consistent payload structure to send to client endpoints and an endpoint to receive from a third-party. As long as the payload format and structure remain consistent, developers can implement the business logic with any technology stack they prefer. While leaving the gateway to cross-cutting concerns like retries, rate limiting and signing the payload.

Services can be rewritten from Python to Rust and redeployed with the Gateway. To the client endpoints, the payload remains the same, and it can continue to rely on the payload format and structure to remain the same.

Routing to Microservices based on payload structure

When webhooks are ingested into your systems from third-party providers, in a monolith all events are routed to the same application all the time. However, in a microservice architecture, one or more microservice(s) can rely on the event from the provider to carry out separate actions.

Webhook Gateways can be configured to receive events and route them to one or more destinations. They can match both headers and payload to determine webhook's destination.

Traffic Control to prevent overloading of resources

Internet-facing services are common targets for malicious attacks and exploitation. They need to handle a sudden spike in requests from valid users and bad actors. This requires features like request throttling and blacklist capabilities to keep systems reliable and secure.

Webhook Gateways are an effective barrier against Distributed Denial of Service (DDoS) attacks—throttling the number of requests made to affected services, so the service isn’t overwhelmed and protecting it against becoming unresponsive.

Common Features & Benefits

A Webhook Gateway is located at the outer edge of your microservices and acts as a proxy to manage all ingress and egress webhook traffic. The Webhook Gateway handles several features:

Security

Webhook Events sent to client endpoints need to be securely delivered to client endpoints. Clients need to validate that the message has not been tampered with in transit, and they need to know that events are not being re-transmitted by malicious actors, in some sensitive environments they need to authenticate the webhook provider and authorise their IP address through their network firewall.

When receiving events from third-party providers (e.g. Stripe, Twilio ), consumers need to verify the webhooks before routing them to backend services for processing

Webhooks Gateway is able to centralise cross-cutting concerns like webhooks security and securely receive events from backend services, sign the payload, timestamp the payload, and deliver them reliably with a static IP address. Since they exist at the outer edge of your microservices they are able to prevent exposing any of your backend services directly to the internet and reduce the attack surface area for malicious actors.

Resilience

Managing HTTP Endpoints is one of the core responsibilities of a Webhook Gateway. Standard webhooks flow is a post request to an endpoint, and expect a 2xx status code in return, indicating success. However, these endpoints fail from time to time for several reasons: flaky networks, spikes in traffic, code or infrastructure changes that resulted in downtime, and degraded performance in the client’s application dependencies.

Due to the various types of issues that can prevent reliable webhooks delivery, webhook Gateways add important features like timeouts; to prevent waiting endlessly for responses, and retries; to automatically re-transmit webhook events, rate limiting; to deliver events to client endpoints at a steady pace the client can process, and circuit breaking; to prevent wasting system resources on zombie endpoints (i.e. endpoints that have failed consecutively over time)

Customer-Facing Dashboard

Debugging webhooks delivery is an important aspect of the responsibilities of webhooks providers to clients. Best practice enables engineers to debug webhook delivery without human interaction with the engineering team of the provider.

Webhook Gateways provide such a utility to generate rich customer-facing webhooks dashboard where users can directly (zero human intervention) debug webhook delivery issues and fix failing endpoints with a few clicks.

Multiple Data Source Options

Webhook Gateways need to be flexible and versatile. Microservices emit events through various channels. For example, building on Google Cloud, developers will often use Google PubSub to emit happening in one or more microservices, similar to AWS for Amazon SQS, and other tools like Kafka and RabbitMQ.

Plugging into all these options makes a webhook gateway such a versatile tool for streaming webhook events from multiple event-generating microservice to send the events to the client endpoints.

Caveats of using Webhooks Gateways

While there are several benefits to deploying a Webhooks gateway, there are some caveats to keep in mind when adopting a Webhooks gateway for your organisation, such as:

The webhooks gateway is a new software component in your environment. It would require learning and building expertise in it. Platform Engineers and SREs will need to integrate, deploy and maintain this new software component and build production excellence running the webhooks gateway.

By design, a webhooks gateway is responsible for webhooks traffic (inbound and outbound) of your organisation, This, therefore, becomes a single point of failure for the entire organisation. Webhooks gateway then needs to be designed for high availability to ensure no data loss for any webhook traffic.

In the end, webhook gateways aren’t one-size fits all. You can build a webhook gateway in-house or acquire an off-the-shelf solution like Convoy while analysing your system's unique requirements.

Final Words

A Webhook Gateway can enhance your microservice-based application with capabilities like secure webhook delivery with signatures, static IPs, resilience with retries, rate limiting, and customer-facing webhook dashboards. However, before adding a webhooks gateway to your application architecture, you need to understand what problems you’re trying to address, what events you’ll be sending and if the product you are choosing offers those features.

Convoy is the fastest open-source webhooks gateway available today. With Convoy, you’ll be able to configure advanced event routing, incoming and outgoing webhooks, and customer-facing webhooks dashboard, and securely deliver events with payload signing and Static IPs.

If you're exploring solutions for your webhooks, we welcome you to try Convoy Cloud for free today.

A little favour

If you feel like this article helped you understand Webhooks better! I would be super happy if you could give us a star! And let me also know in the comments ❤️
https://github.com/frain-dev/convoy

Thanks for reading!

5 Reasons Engineers do not use Webhook Gateways Yet

Lotanna Nwose — Fri, 16 Jun 2023 11:14:49 +0000

Speaking with platform engineers at events, in various online communities and even within the Convoy community validated the notion that managing webhooks in production is not always easy and can require a lot of effort. Platform engineers believe strongly in frictionless self-service developer experience, and solving for efficiency to quickly provide business value. Let’s take a look at why some haven't adopted a webhooks gateway such as Convoy to manage webhook events and integrations yet and what we are doing to change that.

Security Concerns

Webhooks can be vulnerable to attacks such as man-in-the-middle and replay attacks, which can compromise the security of the entire system. Some engineers worry that using a webhooks gateway could introduce additional security risks, particularly if the gateway is hosted by a third-party provider.

Convoy provides advanced webhook security with endpoint authentication, payload signing, rolling webhook secrets, replay attack prevention, and forward-compatible scheme upgrades. For data security and compliance concerns, you can self-host Convoy using the Community Edition or Convoy Enterprise on-prem.

Cost Considerations

We are currently in a bear market and so engineering teams are working towards capital efficiency so cost is another factor that is preventing engineers from adopting a webhooks gateway. Engineers worry that these costs will add up over time, particularly if they have multiple webhook integrations to manage.

We have made that decision even easier at Convoy, making the product free forever for developers and then having the best cost-effective pricing in the market today. Our commitment is to builders and we would continue to show that with our pricing strategies.

Integration Complexities

Readiness to integrate is another interesting issue, some of the startup to mid-level teams we have spoken to are sometimes spent for time. Some engineers are not ready to integrate with a new tool just yet, others might be in the middle of a project or not have the bandwidth to take on something new.

On the other hand, there is a general assumption that integrating new tools can take days and lots of effort. With Convoy, you can get up and running in a few minutes without even speaking to anyone. If you do need to, there is a growing community of platform engineers including the Convoy engineers available to answer questions at any time.

Lack of Awareness

Due to how relatively new the webhooks management space is, lots of platform engineers find Convoy from the content we put out, recommendations from their network and a small percentage from keyword searches. Also, just like any new space, lots of engineers are not aware of benefits and features that webhook gateways have out-of-the-box to make their workflows more efficient.

We are creating more valuable content around webhooks and breaking down its complexities, the more we do, the more you can find Convoy when you do a quick Google Search for anything webhooks related.

Resistance to Change

Many engineers are used to managing webhook integrations manually, and are hesitant to switch to a new platform or service. This is true, especially in large organisations where it takes months to go from one developer buy-in to team-wide adoption. This is the classic build or buy analogy, we do know that adopting a webhooks gateway can provide significant benefits, and significantly reduced debugging time exponentially as you scale.

With Convoy you can manage both incoming and outgoing webhooks, API providers can start their change management process by updating their Docs and telling their customers to receive their webhooks with Convoy. Time and resources saved by customer success teams using this approach is usually a good motivation to buy later on.

Wrapping Up

Do you have any unmentioned reason why you have not tried out the Convoy webhooks gateway? Convoy is a reliable webhook gateway for sending and receiving millions of webhook events with robust support for Retries, Rate Limiting, Static IPs, Circuit Breaking, and Rolling Secrets. You can secure your payloads, scale horizontally and get endpoint failure alerts to debug faster with Convoy. You should try it out here.

Help me out!

Thanks for reading!

10 Common Ways Engineers Use Webhook Gateways

Lotanna Nwose — Wed, 14 Jun 2023 16:05:31 +0000

Introduction

A webhook gateway is a webhook management tool that sits between a webhook provider and the webhooks consumer. It acts as both a reverse and forward proxy for webhooks. It can be deployed by both the API provider that sends webhooks and the consumer that receives webhooks to handle webhook events multiplexing and de-multiplexing respectively.

This article will consider ten of the most common use cases of a webhook gateway, with emphasis on the features offered by Convoy. Convoy is a high-performance open-source webhooks gateway to which platform teams can deploy to manage their webhooks end to end.

The use cases to be discussed are

Reliable Webhook Delivery
Routing in Microservices
Routing with Message Brokers
Static IPs
Security
Scale
Monitoring and Alerts
Easy Debugging
Geolocation Based Routing
Versioning

Reliable Webhook Delivery

Webhooks–which are simply HTTP push– fail frequently due to reasons that include but are not limited to downtimes, spikes in network requests, flaky internet, and expired SSL certificates.

A webhook gateway solves this problem by implementing features such as the following:

Retries: to automatically resend webhook events a specified number of times.
Circuit breaking: to stop sending requests to endpoints that have failed consecutively, thereby not wasting resources on a dead endpoint.
Rate limiting: to prevent overloading client endpoints by delivering events at a rate that the client can process.
Timeouts; to prevent waiting endlessly for response.

A webhook gateway like Convoy exists to ensure the deliverability of your webhook events by implementing these features, while the rest of your application handles other business logic.

Routing in Microservices

In a microservice architecture, many of the services send out webhook events, they may also receive events from third-party providers. How do you manage these ingress and egress webhooks traffic? A webhook gateway can be deployed and configured to route incoming webhooks to the services where they are to be consumed, in the same manner, it would receive webhook events from your services and forward them to client endpoints. This flow of events is illustrated in the image below.

Source: Why do microservices need a Webhooks Gateway?

Webhook gateways are able to determine the destination of webhook events by checking the header and payload of each request.

Routing with Message Brokers

Webhook events are asynchronous by nature and Message Brokers are designed to facilitate asynchronous communication, hence, your webhook deployment that integrates with message brokers allow you to completely bypass the shortcomings of REST API
Connections when sending webhook events.

Convoy recently announced the release of its Message Broker integrations to ingest webhooks from backend services to Convoy. For API providers, this means that they will be able to ingest webhook events from their backend services into Convoy using Message brokers. Convoy currently supports Google PubSub and Amazon SQS.

The diagram above illustrates Convoy pulling webhook events from a message queue and then sending them to a client endpoint. Convoy allows you to send events through both message brokers and a REST API. Read more about it in this blog post.

Static IPs

If you are an API provider that sends webhook to consumers who are in sectors where security is a top priority, they may have strict security policies that include whitelisting IPs for all inbound webhooks on their firewalls. To serve these consumers, you need to provide them with a Static IP address from which your events would be coming for them to whitelist. However, most modern ways of deploying applications make it such that IP addresses are ephemeral by default instead of Static.

One way to fix this is to deploy a webhook gateway that would serve as a forward proxy for your webhook-sending application. Convoy allows you to configure your outbound webhook events with a forward proxy which will have a Static IP.

The simplified diagram shows how the IP of your outbound requests changes as it is transmitted to a client's endpoints.

Security

On the consumer end, the webhook gateway is still the ideal point to implement several security mechanisms. Both the webhook publisher and the webhook consumer play complementary roles to secure requests against malicious attackers. Some security measures that API providers and consumers can apply at the webhook gateway level include:

Signed Payload: An API publisher can prevent a replay attack when they sign their payload and include a generated timestamp to their signature on each request. On the consumer side, the signed payload would be matched against each expected variable, to verify that the signature has not been altered.
Static IP: A consumer app would be able to verify that the Static IP address of all incoming webhook events is whitelisted.
Mutual TLS: Consumer apps equally use mutual TLS to ensure that connection requests from third-party providers are truly from the said provider.

Scale

We live in a world of APIs. Since APIs are the glue that holds the digital world together, it is not surprising that billions of webhooks are delivered every day. Below is an example of a Shopify BFCM Webhooks stats:

With a webhook gateway handling all of your webhook traffic, you can independently scale up and down during and after peak periods. This also means that your webhooks implementations stay decoupled from the rest of your backend services. If your webhook gateway is a SaaS like Convoy, the developers in your product team do not have to get deeply involved in the webhook implementation nor get caught up in its complexity, permitting them to focus on your core business product.

Monitoring and Alerts

Webhooks fail all the time, one reason is that App owners are constantly pushing changes to their apps. It is good behaviour for a webhook delivery system to be able to inform the app owner when their app is down. Convoy does well in this area as it allows you to implement different solutions for monitoring. For example, you can monitor uptime or monitor the average request per minute on your webhooks route, and flag it when they're below a certain threshold. Also, after an endpoint consistently fails, Convoy disables the endpoint and sends an email to the developers to triage.

Easy Debugging

In addition to enabling monitoring, a webhook gateway can make it easier to debug failed webhook requests. For example, Convoy includes a web interface where an operator can filter through event logs, search through the webhook payloads, see the response body from each request, discover the point where a webhook is failing, and even manually resend these webhooks. The dashboard is available to you whether you are an API provider or a consumer.

Geolocation-Based Routing

Another possible way to use your webhook gateway to scale your product is to deploy instances of it at various locations closer to your consumers. Routing webhook requests based on clients' location works similarly to the traditional API Geolocation Based Routing. There are several factors to put into consideration when deciding whether you need a geo-location-based routing solution. But the end result is the same when you do decide to implement it: you primarily mitigate issues caused by latency.

To illustrate how this would work in a real application; your backend services are deployed in a server in the UK, but you have consumer apps running in India. When you deploy Convoy as your webhook gateway closer to India and write webhook events to a message broker, Convoy automatically receives them and then routes them to your clients in India. Routing these events to clients in India would involve lesser network overhead because of the location of your webhook gateway.

This practice is common with API gateways, we think that in the case of a webhook gateway, this might only be relevant for a large company with several consumers in distant parts of the world.

Versioning

Webhook requests signatures evolve over time either in response to bug reports or the need for a new feature, but as they evolve, versioning these signatures become important as you gradually face out old features. Versioning also allows API consumers to adopt new changes to the signature at their own pace. The Convoy OSS and Cloud software also supports this
feature.

Conclusion

So far you have seen the most common ways that one can utilize a webhook gateway.
Convoy democratizes the best features of webhook infrastructures used at top tech companies such as Stripe and PagerDuty into a single binary. When setting up Convoy for your product, you can pick any of these features that suit your needs right out of the box.

Help me out!

What are Reverse-API Gateways?

Lotanna Nwose — Fri, 09 Jun 2023 08:51:05 +0000

Introduction

Webhooks are known as reverse APIs; this is because they reverse the usual flow of communication. So, instead of the client requesting data from the server(APIs), the server sends data to the client when a specific event occurs(Webhooks).
Now that you know this, let us dive into today's agenda!

What are Webhook Gateways?

A webhook gateway is a webhook management tool that sits between a webhook provider and webhook consumer. It acts as a reverse and forward proxy for webhooks. It can be deployed by both the API provider that sends webhooks and the consumer that receives webhooks to handle webhook events multiplexing and de-multiplexing respectively.

Why use a Webhook Gateway?

Speed is a competitive advantage in today's ever-changing user requirements. Development teams are required to move fast and deliver user value. More teams are adopting a serverless, micro-service, and service-oriented architecture to move fast. Teams are leveraging technologies like an API Gateway to consolidate duplicate tasks like authentication, rate limiting, circuit breaking etc. while product teams focus on their core business. With webhooks, teams and tech leads would need to answer the following questions:

How can we provide a consistent infrastructure to receive events from third-party providers and route these events to one or more micro-services to handle them?
How can we provide a consistent infrastructure to collect events from multiple backend services and send them to client endpoints?
How can we consolidate duplicate webhook responsibilities like endpoint authentication and security, rate limiting, and endpoint failure notifications?
Similar to API Gateways, how can we increase developer efficiency across the organisation, where webhooks become plug & play?
How can we provide an independent infrastructure to scale to handle webhooks traffic in peak periods? See Shopify BFCM Webhooks Stats.

Your challenge is offering developers in your organisation a simple and dependable experience in the face of any webhook complexity. A webhook gateway is a way to decouple all your microservices from your webhook implementations. When a microservice needs to send a webhook event, it should write to the broker, and the webhook gateway receives it and routes them to the right endpoint(s), whilst keeping track of everything.

Architecture of Webhook Gateways

The diagram above provides a high-level view of how a webhook gateway operates. We used AWS-specific elements but this can be any cloud environment - Azure, GCP, Digital Ocean, etc or even On-Prem. The arrows from the left show the flow of traffic from your backend services to client endpoints on the right, and the arrows from the right show the flow of traffic from third-party providers like (stripe, Twilio, etc.) to your backend services for processing.

Generally, to avoid vendor lock-in, a webhook gateway will (should) support multiple broker systems like Amazon SQS, Google PubSub, Kafka etc. to allow you to use the best broker for your scenario. Visit this page to learn about Convoy's internal architecture.

API Gateways vs. Webhook Gateways

Webhook gateways share similarities with API gateways such as Tyk and Kong

	API Gateway	Webhooks Gateway
Entrypoint	Entry into the API.	Exit from the API.
Key Metric	Throughput & Latency	Throughput
API Type	Synchronous API	Asynchronous API
Protocols	Multiple Protocols - HTTP, Websockets, gRPC	HTTP
Message Format	JSON, XML & Protocol Buffers.	Mostly JSON.
State	Stateless	Stateful

Conclusion

Convoy is the first ever open-source webhooks gateway to manage millions of webhooks end-to-end. If you’re exploring solutions for your webhooks, we welcome you to try out Convoy webhooks Gateway for free today.

Help me out!

If you feel like this article helped you understand Webhooks better! I would be super happy if you could give us a star! And let me also know in the comments ❤️
https://bit.ly/convoywebhooks

Thanks for reading!

Publish Webhooks From Your FastAPI API With Convoy

Abdulazeez Abdulazeez — Mon, 21 Nov 2022 16:07:18 +0000

Webhooks are messages ( or payload ) sent from an application on the execution of an operation. They are also used to communicate between a chain of services; for example, a payment provider emits webhook events to an e-commerce application’s endpoint after an operation.

Convoy facilitates publishing webhook events from your application to your clients by serving as a reliable egress.

💡 Unlike traditional webhook servers, Convoy allows you to retry and replay your webhook events.

In this article, you will learn how to use Convoy’s Python SDK to publish webhooks events to multiple endpoints in your FastAPI application. You’ll start by setting up your Convoy instance on Convoy cloud ( or Convoy OSS ), create a todo API, and lastly, integrate the Convoy SDK to publish webhook events to endpoints.

Setup Convoy

To set up your Convoy instance, sign in to your dashboard and create a new project:

Create a new outgoing project from your dashboard. An outgoing project is a project for sending out webhook events.

A project API key will be generated for the newly created outgoing project. Store the key in a safe place as you’ll use it in the API to authenticate your Convoy instance to publish webhooks:

Configure your outgoing project. For this article, you’ll be configuring your outgoing project manually. Select the ***Setup Without SDK*** option to manually configure your outgoing project:

💡All actions performed manually in this article can be performed via our SDKs.

Create an application: application refers to your backend app with valid endpoints where webhooks are delivered to. An application can contain as many endpoints as necessary:

ℹ️ Applications are created per user to distinguish their webhook events.

Create an endpoint: an endpoint is a specific destination that can receive webhook events. The endpoint URL used in this article is generated from webhooks.site:

Create a subscription for the application: Subscriptions on Convoy connect events to their respective application endpoint.

You’re all set to publish events and monitor them from your dashboard:

With your Convoy instance up and running, you’ll build the API to publish your webhook events in the next section.

Project setup

The first step to publishing webhooks from your FastAPI-powered API is to build the API. Start by creating a new folder and creating a virtual environment:

$ mkdir fastapi-todo-convoy && cd fastapi-todo-convoy
$ python3 -m venv venv

ℹ️ You may use a different virtual environment manager like Pipenv or poetry.

API structure

The API folder structure is outlined:

fastapi-todo-convoy
 +- .env # store Convoy API key and application ID.
 +- api/
   +- __init__.py # module file
   |
   +- config.py # interact with the environment file
   | 
   +- api.py # todo API source file
   | 
   +- events.py # store for various event types for your webhook payload
 +- main.py # entrypoint to run the application
 +- requirements.txt # application requirements file

In the project folder, add the following files:

$ touch .env api/{config,api,events}.py main.py

Install dependencies

As a next step, activate the virtual environment and install the following dependencies:

$ source venv/bin/activate
$ pip install fastapi uvicorn pydantic[dotenv]
$ pip freeze > requirements.txt

Lastly, install the convoy-python package:

$ pip install git+ssh://git@github.com/frain-dev/convoy-python

Set environment variables

In the .env file, add the variables:

CONVOY_API_KEY=<your-api-key>
CONVOY_APP_ID=<your-app-id>

Replace <your-api-key> in the environment file above with the API key generated earlier on and retrieve your Convoy application ID from the dashboard:

To allow your API to retrieve the values stored in the environment file, define a BaseSettings child class in config.py:

from pydantic import BaseSettings
from typing import Optional

class Settings(BaseSettings):
    # convoy credentials
    CONVOY_API_KEY: Optional[str] = None
    CONVOY_APP_ID: Optional[str] = None

    class Config:
        env_file = ".env"

settings = Settings()

The Settings class defined above reads the variables defined in the environment file. The API can directly access the variables from the instance variable of the Settings class settings.

Define event types

The next step is to define the various event payload to be included in the webhook to be published from the APIs. In [events.py](http://events.py), add the code:

events = {
    "ping": {
        "event": "ping",
        "description": "Webhook test from application."
    },
    "created": {
        "event": "todo.created",
        "description": "Todo created successfully"
    },
    "retrieved": {
        "event": "todo.retrieved",
        "description": "Todo retrieved successfully"
    },
    "updated": {
        "event": "todo.updated",
        "description": "Todo updated successfully"
    },
    "deleted": {
        "event": "todo.deleted",
        "description": "Todo deleted successfully"
    },
    "failed": {
        "event": "todo.failure",
        "description": "Todo not found."
    }
}

In the code block above, you have six event types all prefixed with todo. accompanied by a description of the event.

Build the API

The API performs a Create, Read, Update and Delete ( CRUD ) operation on todos stored in an in-app database, an array variable todos.

In api.py, import the dependencies and create an instance of FastAPI:

from convoy import Convoy
from fastapi import FastAPI

from .events import events
from .config import settings

app = FastAPI()

Next, create the todos variable and an instance of Convoy:

todos = []

convoy = Convoy({"api_key": settings.CONVOY_API_KEY})
app_id = settings.CONVOY_APP_ID

In the code block above, you connect to your Convoy instance using the API key stored in the environment file using the convoy-python SDK installed. The app_id variable is also set to the value stored in the environment file.

Next, you’ll define a function send_webhook_event that takes an argument event_type and uses the convoy.event.create() method to publish events to your application and by extension, your application endpoints.

def send_webhook_event(event_type: str):
    event = {
        "app_id": app_id,
        "event_type": event_type,
        "data": events[event_type]
    }

    (res, err) = convoy.event.create({}, event)
    return res

The convoy.event.create() method takes an optional query and an event payload body. The event payload body is an object containing the application ID, the event type and payload. In your API, the event type and event payload have been defined in events.py.

With the webhook publisher function in place, add the code below to complete your API:

@app.get("/")
async def ping():
    send_webhook_event("ping")
    return {"message": "Wilkomen!"}

@app.get("/todo", tags=["todos"])
async def get_todos() -> dict:
    send_webhook_event("retrieved")
    return { "data": todos }

@app.post("/todo", tags=["todos"])
async def add_todo(todo: dict) -> dict:
    todos.append(todo)
    send_webhook_event("created")
    return {
        "data": { "Todo added." }
    }

@app.put("/todo/{id}", tags=["todos"])
async def update_todo(id: int, body: dict) -> dict:
    for todo in todos:
        if int(todo["id"]) == id:
            todo["item"] = body["item"]
            send_webhook_event("updated")
            return {
                "data": f"Todo with id {id} has been updated."
            }
    send_webhook_event("failed")
    return {
        "data": f"Todo with id {id} not found."
    }

@app.delete("/todo/{id}", tags=["todos"])
async def delete_todo(id: int) -> dict:
    for todo in todos:
        if int(todo["id"]) == id:
            todos.remove(todo)
            send_webhook_event("deleted")
            return {
                "data": f"Todo with id {id} has been removed."
            }

    send_webhook_event("failed")
    return {
        "data": f"Todo with id {id} not found."
    }

You have defined the routes to perform various CRUD operations in the code block above. Each successful operation emits a webhook to your Convoy application, and a failed operation emits a failed webhook event.

Lastly, in the API’s entry point file main.py, add the following:

import uvicorn

if __name__ == "__main__":
    uvicorn.run('api.api:app', host="0.0.0.0", port=8080, reload=True)

The code above starts the API on port 8080 on the localhost. Start your application with the command:

$ python main.py

Publishing webhooks

It’s time to publish your first webhook! To publish your first webhook event, send a cURL request to the API:

$ curl http://0.0.0.0:8080/

The cURL request returns a response:

`{"message":"Wilkomen!"}`

In your Convoy dashboard, the event is delivered and logged:

In the Event Deliveries tab, you can verify the status of the delivery as well as view the number of attempts:

If you click on the event, a detailed delivery page is displayed:

Let’s create a todo using a cURL request and verify if your API will publish a webhook event:

$ curl -X 'POST' \
  'http://0.0.0.0:8080/todo' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "id": 1,
  "item": "Create a Convoy outgoing project today"
}'

The API returns a successful response:

{
  "data": [
    "Todo added."
  ]
}

Was the webhook for a create action emitted? Verify from your dashboard:

You can verify that the API webhook successfully emitted the event type created from the dashboard. A detailed view of the event delivery can is seen from the event delivery tab:

A visit to the endpoint URL strengthens the webhook delivery status from the Convoy dashboard:

Go on, execute more operations on the todo API and publish webhooks today on Convoy!

Conclusion

Convoy provides the ability to publish webhooks to various endpoints in one request. In this article, you learned how to create and configure a Convoy outgoing project and publish webhooks from an API you built to your configured endpoint.

Convoy provides you with reliability and replayability out of the box. If this sounds suitable for your architecture,try it outand give us feedback on ourslackcommunity!

Let's implement Stripe webhook signatures in Golang

Abdulazeez Abdulazeez — Fri, 04 Nov 2022 10:26:00 +0000

NOTE: This article originally appeared on the GetConvoy Blog

Building a webhook publishing infrastructure requires providing a way to validate the message’s integrity to enable consumers to validate the webhook event origin. Generating webhook signatures would require us to implement certain important features which include:

Prevent Replay Attacks
Forward Compatibility.
Zero Downtime Key Rotation.

More on each item later on. But these properties exist in Stripe’s webhook signature implementation, see below:

Stripe-Signature:
t=1492774577,
v1=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd,
v1=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd,
v0=6ffbb59b2300aae63f272406069a9788598b792a944a07aba816edb039989a39

In this article, we would replicate this implementation in Golang, with one additional requirement; We want our implementation to be backward compatible with the common implementation, like the below:

Stripe-Signature: 
5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd

This backward compatibility allows new API consumers to opt-in to this new system at their choosing. For the purpose of this article, we define the latter signature specification as simple signatures and the former as advanced signatures. Implementing simple signatures is pretty trivial and commonplace, but advanced signatures aren’t common.

We start by breaking down the requirements to discuss advanced signatures even further.

Prevent Replay Attacks

Stripe-Signature:
t=1492774577,
v1=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd,
v1=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd,
v0=6ffbb59b2300aae63f272406069a9788598b792a944a07aba816edb039989a39

A replay attack occurs when an attacker intercepts a valid payload, and its signature, then re-transmits them. The aim is to exploit unsuspecting webhook consumers to perform an action multiple times. Idempotent keys aren’t sufficient against attacks like this because, webhooks are transient data, assuming consumers purge their webhook log after a certain period, it means re-transmitting purged webhook events all of sudden becomes valid.

To mitigate against this, we generate a timestamp and include it in the signed payload, so it is verified alongside the signature, so the attacker cannot change the timestamp without invalidating the signature. This ensures that events after a given threshold are regarded invalid.

When retrying events each delivery attempt should re-generate the timestamp, this ensures the timestamp is fresh.

Forward Compatibility

Stripe-Signature:
t=1492774577,
v1=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd,
v1=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd,
v0=6ffbb59b2300aae63f272406069a9788598b792a944a07aba816edb039989a39

Webhooks implementation evolves over time, providers can determine whether to switch from hex to base64 for encoding or change the hash function or the template of the payload being signed. To enable smooth upgrades for consumers, we version signatures in the example above. This allows consumers to verify against at least just one signature and migrate to the newest version at their convenience.

Zero Downtime Key Rotation

Stripe-Signature:
t=1492774577,
v1=xdz+2j9aMVQUUjSy0KUz/CsjD4jaD6wHJGGf1c3eZzrWxHTf1cAjZ3aL07O9NZXMhg5gajfi+TYuBU1aoU18xA==,
v1=5257a869e7ecebeda32affa62cdca3fa51cad7e77a0e56ff536d0ce8e108d8bd,
v0=cvt+CsjD4jaD6wHJGGf1/2j9aMVQUUjSy0KUzc3eZzrWxHTf1cTYuBU1aoU18xAAjZ3aL07O9+NZXMhg5gajfi==
v0=df51a848684dac3901d2b8bd17e5c8d2d971b15c544fa923493232df1fe0fbad

Webhooks rely on a shared secret that needs to be rotated periodically to be kept safe. Building effective key rotation mechanisms is a major implementation of great webhooks implementations. In the above sample, you can see v1 and v0 appear twice this means, we used two secrets to generate two different schemes.

Core Implementation

The core implementation goes thus:

type Scheme struct {
    // Secret represents a list of active secrets used for
    // a scheme. It is used to implement rolled secrets.
    // Its order is irrelevant.
    Secret []string

    Hash     string
    Encoding string
}

type Signature struct {
    Payload json.RawMessage

    // The order of these Schemes is a core part of this API.
    // We use the index as the version number. That is:
    // Index 1 = v1, Index 2 = v2
    Schemes []Scheme

    // This flag allows for backward-compatible implementation
    // of this type. You're either generating a simple header
    // or a complex header.
    Advanced bool

    // This function is used to generate a timestamp for signing
    // your payload. It was only added to aid testing.
    generateTimestampFn func() string
}

func (s *Signature) ComputeHeaderValue() (string, error) {
    // Encode Payload
    tBuf, err := s.encodePayload()
    if err != nil {
        return "", err
    }

    // Generate Simple Signatures
    if !s.Advanced {
        sch := s.Schemes[len(s.Schemes)-1]
        sec := sch.Secret[len(sch.Secret)-1]

        sig, err := s.generateSignature(sch, sec, tBuf)
        if err != nil {
            return "", err
        }

        return sig, nil
    }

    // Generate Advanced Signatures
    var signedPayload strings.Builder
    var hStr strings.Builder
    var ts string

    // Add timestamp.
    if s.generateTimestampFn != nil {
        ts = s.generateTimestampFn()
    } else {
        ts = fmt.Sprintf("%d", time.Now().Unix())
    }

    // Generate Payload
    signedPayload.WriteString(ts)
    signedPayload.WriteString(",")
    signedPayload.WriteString(string(tBuf))

    // Generate Header
    tPrefix := fmt.Sprintf("t=%s", ts)
    hStr.WriteString(tPrefix)

    for k, sch := range s.Schemes {
        v := fmt.Sprintf(",v%d=", k+1)

        var hSig string
        for _, sec := range sch.Secret {
            sig, err := s.generateSignature(sch, sec, []byte(signedPayload.String()))
            if err != nil {
                return "", err
            }

            hSig = fmt.Sprintf("%s%s", v, sig)
            hStr.WriteString(hSig)
        }
    }

    return hStr.String(), nil
}

Let’s break down the above code listing:

We use the Advanced flag to determine what type of signature to generate.
We use the Scheme type to encapsulate all versions, and the order in which they’re passed in determines their version. We map to index 0 to v1 etc similar to how we don’t define API versioning as /api/v0.
ComputeHeaderValue will either generate a simple signature string or an advanced signature string based on the Advanced flag.

Other aspects of this library were removed for brevity, you can find the full code here.

SDK

To enable easy migration, we’ve added webhook verification logic to our Ruby, Python & Golang SDKs to parse and validate this format. This verification will automatically identify either simple or advanced signatures and validate them respectively.

Separate API Keys from Webhook Secrets

Another useful benefit of Advanced Signatures is we can stop using our API Keys as webhooks secrets with zero downtime. This is good because compromised webhook secrets do not equal compromised API Keys and vice-versa. This is also known as the principle of least privilege. To achieve this do the following:

Create endpoints, and set the webhook secret as the API Key.
Update your apps to verify Advanced Signatures.
Roll over the current webhook secret with an expiry time.
Set the new webhook secret in your apps. 🎉

Conclusion

In this article, we showed how to implement Stripe-Like webhooks and build them in a backward-compatible fashion with your current implementation. We shipped this feature to Convoy OSS and Cloud. You can sign up here to get started!

Publish An Event to Multiple Endpoints using Convoy

Daniel Oluojomu — Fri, 11 Mar 2022 13:05:50 +0000

One common scenario in publishing webhook events is enabling a user to provide multiple endpoints to receive events. One easy example of this is publishing an event that the user needs to process at more than one location. This location could be a no-code platform like zapier, a newly minted microservice or serverless function, or a good old slack notification. In this article, I’d like to explain how you can achieve this using Convoy.

Without Convoy, your users have to build in the fan-out mechanism themselves which is a lot more stressful.

Steps

Start Convoy Instance

To follow through with this article you’d need to run an instance of Convoy:

$ docker run \
    -p 5005:5005 \
    --name convoy-server \
    -v `pwd`/convoy.json:/convoy.json \
    ghcr.io/frain-dev/convoy:v0.4.18

Create an Application

Next, we have to create an application under this group.

Sample Payload

{
  "name": "test-app",
  "support_email": "test@gmail.com",
  "secret": "eyJhbGciOiJIUzI1NiJ9"
}

Bash

$ curl \
    --request POST \
    --data @app.json \
    -H "Content-Type: application/json" \
    http://localhost:5005/api/v1/applications

Response

{
    "status": true,
    "message": "App created successfully",
    "data": {
        "uid": "2b1e9973-ed03-403c-a8b0-341edd51fb14",
        "group_id": "f0a187f4-edaa-4f8e-adec-75b9a36b3c68",
        "name": "test-app",
        "support_email": "test@gmail.com",
        "endpoints": [],
        "created_at": "2022-03-09T14:17:51.111+01:00",
        "updated_at": "2022-03-09T14:17:51.111+01:00",
        "events": 0
    }
}

Create Two Endpoints

Now we can create multiple endpoints under this app. The first endpoint will take the * event type. Essentially this event type means all incoming events to that app will be published to that endpoint. The second endpoint will take the event type payment.created. Only incoming with the exact type payment.created will be published to that endpoint. Convoy tries to match the event type to all available endpoints under that app, the event is then published to all the matched endpoints.

For the first endpoint:

Sample Payload

{
  "description": "test-endpoint-1",
  "events": [
    "*"
  ],
  "secret": "12345",
  "url": "<your-endpoint>"
}

Bash

$ curl \
    --request POST \
    --data @endpoint-1.json \
    -H "Content-Type: application/json" \
    http://localhost:5005/api/v1/applications/{appID}/endpoints

Response

{
    "status": true,
    "message": "App endpoint created successfully",
    "data": {
        "uid": "2901bbc9-092e-4685-868d-a17298fe86ba",
        "target_url": "<your-endpoint>",
        "description": "test-endpoint-1",
        "status": "active",
        "secret": "12345",
        "events": [
            "*"
        ],
        "created_at": "2022-03-09T14:18:14.493+01:00",
        "updated_at": "2022-03-09T14:18:14.493+01:00"
    }
}

For the second endpoint:

Sample Payload

{
  "description": "test-endpoint-2",
  "events": [
    "payment.created"
  ],
  "secret": "12345",
  "url": "<your-endpoint>"
}

Bash

$ curl \
    --request POST \
    --data @endpoint-2.json \
    -H "Content-Type: application/json" \
    http://localhost:5005/api/v1/applications/{appID}/endpoints

Response

{
    "status": true,
    "message": "App endpoint created successfully",
    "data": {
        "uid": "2901bbc9-092e-4685-868d-a17298fe86ba",
        "target_url": "<your-endpoint>",
        "description": "test-endpoint-2",
        "status": "active",
        "secret": "12345",
        "events": [
            "payment.created"
        ],
        "created_at": "2022-03-09T14:18:14.493+01:00",
        "updated_at": "2022-03-09T14:18:14.493+01:00"
    }
}

Publish Event

Now let us publish an event with the type payment.created to this app. The payment.created will match both endpoints, since * will match all event types, and the payment.created type of the second endpoint matches exactly.

Sample Payload

{
  "app_id": "2b1e9973-ed03-403c-a8b0-341edd51fb14",
  "data": {
        "blog":"medium.com"
    },
  "event_type": "payment.created"
}

Bash

$ curl \
    --request POST \
    --data @event.json \
    -H "Content-Type: application/json" \
    http://localhost:5005/api/v1/events

Response

{
    "status": true,
    "message": "App event created successfully",
    "data": {
        "uid": "ddcd3928-1d7e-4527-901f-47673fd569ce",
        "event_type": "payment.created",
        "matched_endpoints": 2,
        "provider_id": "",
        "data": {
            "blog": "medium.com"
        },
        "app_metadata": {
            "uid": "2b1e9973-ed03-403c-a8b0-341edd51fb14",
            "title": "test-app",
            "group_id": "e6bbde4b-4c43-45a6-8d4c-c8eed1c2bb41",
            "support_email": "test@gmail.com"
        },
        "created_at": "2022-03-09T15:14:27.569+01:00",
        "updated_at": "2022-03-09T15:14:27.569+01:00"
    }
}

Show Endpoint Response

Illustration of the events coming through

Footnote

This article was originally published on our main blog. Check it out for more articles about convoy!

Introducing Convoy

Subomi Oluwalana — Fri, 18 Feb 2022 18:52:42 +0000

Hi dev.to, I’d like to show you Convoy.

Convoy is an open-source webhooks service. You can think of it as a swiss army knife for publishing webhook events. Few months ago, I was building a Fintech API and I was looking around for a tool to push webhooks events. I wanted something simply, ease-to-use, language-agnostic, and cloud-native. Ideally, I wanted a container I could deploy, publish events to it, and it reliably publishes events to the endpoints. I couldn’t find a such a tool, so we built it. Convoy exposes a REST API to register endpoints, and publish events to respective endpoints.

Webhook Issues

On the surface, webhooks isn't simply HTTP Push. There exist several naunces and some use-cases needed for some type of integrations. I'd like to talk about a few:

Bad Endpoints

Publishing webhooks requires you to deal with several poorly designed endpoints, endpoints that respond with large payload sizes, endpoints that hang unending and eventually timeout, some other endpoints have expired certificates. A good webhook system will provide visibility into these issues for both publishers and consumers.

Delivery Attempt Logs

Building against webhooks can be non-trival for developers at times. Developers often times, use ngrok to pipe events to their local machine to debug several things; signing payload, parsing payload structure, responding with a 200 status code fast, monitoring successful retries, testing url per events. Building a great webhooks delivery system requires building a delivery attempt log showing what was sent vs. what was returned.

Delayed Job Implementation.

Because webhooks is inherently an asynchronous activity, It needs an underlying resilient job/task queuing system to account for scalability, retries for transient & non-transient failures of events. A standard webhooks delivery system is an implementation over a job queuing system (e.g. redis, kafka, rabbit, nats etc) to provide such features.

Security

Publishing webhook events with HTTP requires consumers to secure their endpoints from malicious users. Consumers need to know who is posting and if they're authorized to post events. Several consumers have several layers of security: Signing payload with a shared secret (common), Static IPs (common in fintech), Mutual TLS (found in PagerDuty). Essentially, a good webhook delivery system needs to support all forms of security depending on what respective consumers require.

URL per Events.

In this age of no-code tools, and serverless functions. It is common for developers to receive webhook events from a provider (e.g. Stripe) and fan-out to several apps behind the scene to perform any desired action. Stripe, as an example provides first-class support for this where users can subscribe an endpoint to specific events, but isn't found in many other applications. A good webhook delivery system should provide first-class support for URL per events.

It is all this fragmented implementations of webhooks, we are trying to solve with Convoy.

Getting Started

To get started you can head over to https://getconvoy.io/docs/guide and follow the step by step guide to running Convoy on your local machine.

Conclusion

We believe we're still very early in abstracting the fragmented implementations of webhooks into Convoy, but we're happy to share our progress so far and invite to you be a part of our community

You can join our slack community here

We can’t wait to hear your feedback!