Understanding GitHub Webhooks: Leveraging Reverse Proxy with Ngrok for Local Development

Webhooks are a powerful and flexible way for applications to communicate with each other in real-time. They are a crucial feature for developers who want to automate workflows and integrate with other services. In this blog post, we will explore what GitHub webhooks are, why we use them, and how to leverage a reverse proxy like Ngrok to trigger events on a local development environment.

What are GitHub Webhooks?

Webhooks allow you to subscribe to specific events occurring within a software system and automatically receive detailed data delivered to your server in real-time whenever those events take place.

When you create a webhook, you specify a URL and subscribe to specific events that occur within your GitHub repository. These events can range from code pushes and pull requests to issues and comments. Whenever an event that your webhook is subscribed to occurs, GitHub sends an HTTP POST request with detailed data about the event to the URL you specified.

The webhook's payload URL is the endpoint that will receive these HTTP requests. Your server must be set up to listen for webhook deliveries at this URL. When it receives a request, it can parse the event data and take appropriate actions. This allows for a wide variety of automated workflows, such as triggering CI/CD pipelines, sending notifications, or updating external systems with the latest repository information.

Detailed Workflow of GitHub Webhooks

1. Event Subscription: When configuring a webhook, you choose which events your webhook should listen for. These could include events like push, pull_request, issues, release, and more.

2. Event Trigger: When one of the subscribed events occurs in the repository, GitHub prepares an HTTP POST request with a payload containing details about the event. For example, a push event payload includes information about the commits, the branch involved, and the repository.

3. HTTP Request to Payload URL: GitHub sends the HTTP POST request to the specified payload URL. This request includes the payload data as JSON, providing all the necessary details about the event.

4. Server Processing: Your server, which is set up to handle requests at the payload URL, receives the webhook request. It then processes the payload data to determine what action to take. For instance, it might trigger a Jenkins build, post a message to a Slack channel, or update a database.

5. Response to GitHub: After processing the event, your server responds to GitHub with an HTTP status code. A 2xx status code indicates that the request was successfully received and processed.

Using Ngrok as a Reverse Proxy

In a local development environment, your server might not be accessible from the internet, making it difficult to test webhooks that require a publicly accessible endpoint. This is where Ngrok comes in handy. Ngrok is a tool that creates a secure tunnel to your local server, providing a public URL that GitHub can send webhooks to.

How does Ngrok Works?

Ngrok establishes a secure connection between your local machine and a publicly accessible URL, effectively exposing your local server to the internet. Here’s a detailed breakdown of the process:

1. Create an account in ngrok.com:

1. Follow given instructions to install ngrok and add authtoken:

Example: Setting Up Ngrok to Trigger a Simple CI/CD Pipeline in Jenkins

Let's walk through an example of setting up Ngrok to expose a local Jenkins server and configure a GitHub webhook to trigger a simple CI/CD pipeline when code is pushed to a repository.

Step 1: Ensure Jenkins is Running

For this example, let's assume that your Jenkins server is already installed and running on port 8080. If Jenkins is running correctly, you should be able to access it in your browser at http://localhost:8080.

Step 2: Install Github plugin

To allow Jenkins to be triggered by GitHub webhooks, you need to install the GitHub plugin:

1. Go to "Manage Jenkins” -> “Plugins”
2. Install the GitHub Plugin and restart Jenkins if required.

Step 3: Create a Jenkins Job

1. Click on “New Item,” enter a name for your job, select “Freestyle project,” and click “OK.”

2. Add git repository and specify branch(s)

3. Select the “GitHub hook trigger for GITScm polling” option.

4. Configure your job. For simplicity, you might just echo a message in the Build section.

Step 4: Start Ngrok

Open a terminal and start Ngrok to create a secure tunnel to your local Jenkins server:

ngrok http 8080

Ngrok will provide a public URL, such as:

Step 5: Configure GitHub Webhook

In your GitHub repository, configure a webhook to send events to your Jenkins server through the Ngrok URL:

1. Navigate to your GitHub repository and go to “Settings” -> “Webhooks”
2. Click “Add webhook”
3. In the “Payload URL” field, enter the Ngrok URL followed by /github-webhook/.
4. Click “Add webhook” to save.

Note: If your Jenkins server denies anonymous user access, you need to include authentication credentials in the webhook URL to avoid a 401 Unauthorized error. The URL format should be https://user:password@PUBLIC_URL/github-webhook/.

Step 6: Test the Setup
To test the webhook setup, push a change to your GitHub repository. This should trigger the webhook and send a request to your Jenkins server via the Ngrok tunnel. Jenkins will then trigger the configured job.

You can verify this by checking the Jenkins job console output. You should see the echo message "Build triggered by GitHub webhook," indicating that the webhook successfully triggered the CI/CD pipeline.

Conclusion

GitHub webhooks enable real-time automation by notifying external services about repository events. Testing these integrations locally can be challenging due to accessibility constraints. Ngrok simplifies this by creating a secure tunnel to your local server, allowing you to test webhook configurations as if they were exposed on the internet.

In our example, we demonstrated how to use Ngrok to trigger a Jenkins CI/CD pipeline from GitHub events. This setup allows for seamless testing of webhooks in a local environment, ensuring your integrations work as expected before deployment. Overall, leveraging GitHub webhooks with Ngrok enhances development efficiency and reliability.