DEV Community: Tarek Touati

Getting Started with Synthetic Monitoring on GCP and Datadog

Tarek Touati — Mon, 01 Jul 2024 21:17:50 +0000

When you are talking about Monitoring, we often think about getting information on cpu usage, memory usage and other classic ops metrics. A 2024 trend in the monitoring landscape is also to gather end to end metrics and that’s why we see more and more “Synthetic Monitoring” panels in modern observability tools.
Like in the pyramid testing model, testing end to end metrics is a complementary approach to the metrics that we are used to check.

With new SEO constraints linked to the performance of web applications, many companies are now managing end to end metrics to get the full picture of what is working and how fast their websites are.

“Everything fails, all the time” is a famous quote from Amazon's Chief Technology Officer Werner Vogels. This means that software and distributed systems may eventually fail because something can always go wrong.

Definition & Concepts

In a few words : It’s a way to monitor your application, by simulating user actions and business critical scenarios. This aims to warn you of production related issues, so you can fix them before impacting most of your users.
If you are familiar with End-to-end testing, you are almost good to go. The idea, here, is to execute scripts that mimic your users and ”run” most business critical scenarios.
The key difference is that you are running this on your production environments, and focus will be given on performance of your application.
Once your scenario is executed, you’ll get plenty of metrics around your application, whether it’s related to web performance, or on one of your internal components.
Based on these metrics, you’ll be able to know where the effort needs to be made on your assets, to make your application more robust and fault-tolerant.
In general, there are different type of monitoring that refer to Synthetic monitoring :

1/ Availability Monitoring :

Will verify that your service is available at any time. It is a bit more sophisticated than a simple health check control, instead this monitor ensures that the service is running well and responding as it intends to be.

2/ Transaction Monitoring :

It is a step ahead of Availability Monitoring, now we are going to add scripts that will simulate users interactions to make sure business critical scenarios are working as expected.

3/ Web performance monitoring :

As its name suggests, it will focus on application performance throughout Core Web Vitals, this helps identify improvements/ degradation for your end-users.

Use-case

In this article, we will focus on Transaction & Web performance monitoring :

Our critic scenario is :

A user navigates to https://training.zenika.com
Type in search bar : “CKA”
Should be redirected to a result page containing results.
Choose the first training.
Should be redirected to the training detail page.

Synthetic monitoring setup on Google Cloud

Demo: https://github.com/Tarektouati/GCP-synthetic-monitoring
A synthetic monitor is composed of 2 Google Cloud components :
A cloud function
A monitor attached to the cloud function.

We’ll create a Node.JS cloud function with a puppeteer inside to which iterate on our user’s scenario.

As it’s puppeteer, you can write it by yourself, and rely on testing-library best parcticies and install pptr-testing-library.

if you are lazy, that’s why you can use Chrome recorder, to generate your puppeteer user journey.

Once, your are good to go, deploy cloud function in the your desired region, by running :

gcloud functions deploy <YOUR_CLOUD_FUNC_NAME> — gen2 — runtime=nodejs18 — region=<REGION> — source=. — entry-point=<YOUR_CLOUD_FUNC_ENTRYPOINT> — memory=2G — timeout=60 — trigger-http

You should see you cloud function available on GCP console

Next, attach a monitor to that cloud function :

gcloud monitoring uptime create <YOUR_MONITOR_NAME> — synthetic-target=projects/<PROJECT_ID>/locations/<REGION>/functions/<YOUR_CLOUD_FUNC_NAME> — period=5

You should also see the monitor available and configured on the GCP console

Navigate to the monitor detail page, you can see whether it status is passing or not.

Go ahead and create an Alerting policy, this step is crucial as it will notify you when your monitoring goes wrong.

Define a duration before an incident is declared, select a channel (email, slack, pager-duty, …). And add instruction to help the duty-girl/boy understand the incident.

For now, we have seen Transaction Monitoring with GCP, but what about Web performance monitoring ?

This doesn’t come out of the box, but still possible on GCP by combining puppeteer with lighthouse (checkout puppeteer documentation for this https://github.com/GoogleChrome/lighthouse/blob/main/docs/puppeteer.md).

Set-up on synthetic monitoring on Datadog

A synthetic monitor on Datadog is composed of different components :

Select UX Monitoring > Synthetic tests > New Test then create a “Browser test”

Configure your test by filling:

Target URL
Name,
Browser targets (Chrome, Firefox, …)
Locations: Chose one or multiple locations based on your business requirements

Next, same as the GCP part, we need to define the test period and alerting conditions.

To create our test scenario it’s quite easy and fast, Datadog allow you to record a journey directly from your browser (This requires a browser extension to be installed)

Go ahead and create your own journey, and once you are satisfied, create your monitor.

By default, a Browser performance dashboard is already available.

This one showcases the metrics like :

success rate per browser (chosen in the test configuration in the steps above)
Core Web Vitals
Long running tasks (which can be painful for your users)
3-party integration
…

Conclusion

Leveraging Google Cloud for Availability and Transaction Monitoring is a robust and efficient choice, especially for those already integrated into the Google Cloud ecosystem.

The seamless integration and comprehensive tools available within Google Cloud ensure thorough and effective transaction monitoring.

However, when it comes to web performance monitoring, while it’s still possible on Google Cloud, exploring Datadog can provide additional benefits.

Datadog’s Real User Monitoring (RUM) is particularly noteworthy, offering advanced capabilities and insights that might better serve your web performance monitoring needs.

More information :

Link to Datadog doc
Link to Google Cloud doc

How to embed a web server in your React-Native app in Swift

Tarek Touati — Wed, 27 Nov 2019 22:31:38 +0000

Introduction

React Native makes it easy to build native iOS and Android apps, there's a lot of modules that allow us to use native APIs made by an awesome community. But sometimes it can be frustrating to realize that your needed module does not exist and the only solution you have is to create it on your own. The thing is native modules using React Native should be developed in Swift and Objective-C for iOS and Java/Kotlin for Android.

In this article, we will be creating a module for React Native that interacts with a Swift SDK. The purpose of this module is to turn your iOS device into an HTTP server to serve a static HTML file.

I know what are you thinking about, why would I turn my device into an HTTP server?

So there's a lot of use cases some of them:

File Manager/Text editor app and the ability to retrieve/share your files from any other device on your network
As a Gateway for IoT
As an Ad-hoc server

And maybe the most valuable case is: Just for the fun of it.

Setup new React Native project

You can skip this step if you already have a React Native project.

The first thing to do is to create a new project :

react-native init WebServerApp
cd WebServerApp

Install GCDWebServer pod

GCDWebServer is a library that allows us to create a lightweight HTTP 1.1 server.
To install this library make sure you have CocoaPods installed if it isn't I suggest you Cocoapods Getting Started guide.

let's install our dependency :

Dive into WebServerApp/ios folder.

Open Podfile file in your editor and add :

pod "GCDWebServer", "~> 3.5.3"

Then run the Cocoapod install command

pod install

Bridging Objective-C with Swift :

React Native was made to communicate with Objective-C, that's why we will need to create a bridging header.

In your ios folder, open the code project called WebServerApp.xcworkspace in Xcode.

In Xcode :

File -> New -> File or (Cmd + N)
Select Swift file
Name it WebServerManager

After creating our class, Xcode will suggest you create an Objective-C bridging header (usually this file is called <MY_PROJECT_NAME>-Bridging-header.h):

Press Create Bridging Header button, and you should have a WebServerApp-Bridging-header.h file created

Add to WebServerApp-Bridging-header.h file :

// React Native Bridge
#import "React/RCTBridgeModule.h"

// GCDWebServer headers
#import <GCDWebServer/GCDWebServer.h>
#import <GCDWebServer/GCDWebServerDataResponse.h>

Create WebServerManager Swift class

open WebServerManager.swift file and declare WebServerManager class.

Our class inherited from NSObject so we can expose it to Objective-C

requiresMainQueueSetup method let React Native know if your module needs to be initialized on the main thread

import Foundation

@objc(WebServerManager)
class WebServerManager: NSObject {

     override init(){
        super.init()
    }

    @objc static func requiresMainQueueSetup() -> Bool {
        return true
    }
}

Exposed WebServerManager methods

Our module will expose only 2 methods which are :

startServer
stopServer

`startServer` method :

This method will initialize the server, retrieve the HTML content and return Promise with server URL or threw an error.

  /**
  Start `webserver` on the Main Thread
 - Returns:`Promise` to JS side, resolve the server URL and reject thrown errors
  */
  @objc public func startServer(_ resolve: RCTPromiseResolveBlock,
                          rejecter reject: RCTPromiseRejectBlock) -> Void
  {
    if (serverRunning == ServerState.Stopped){
      DispatchQueue.main.sync{
        do{
          try self.initWebServer()
          serverRunning = ServerState.Running
          webServer.start(withPort: 8080, bonjourName: "RN Web Server")
          resolve(webServer.serverURL?.absoluteString )
        } catch {

          reject("0", "Server init failed : \(error.localizedDescription)", error)
        }
      }
    } else {
      let errorMessage : String = "Server start failed"
      reject("0", errorMessage, createError(message:errorMessage))
    }
  }

We are using DispatchQueue.main.sync method because it needs to be executed on the Main thread.

Add private Variables and Enumerations

ServerState enumerations are differents server state
Errors enumerations are error cases
webServer variable is an instance GCDWebServer
serverRunning variable is the webserver state

  private enum ServerState {
    case Stopped
    case Running
  }
  private enum Errors: Error {
    case fileNotFound
    case fileNotReadable
  }
  private let webServer: GCDWebServer = GCDWebServer()
  private var serverRunning : ServerState =  ServerState.Stopped

Add HTML file :

Create an HTML file with the content you want to serve.

Example :

<html>
  <body>
    <div>
      <img
        src="https://media1.tenor.com/images/3d124f67efd8e08b6fd3f0e748255a95/tenor.gif"
      />
      <p>This web page is served from your React-Native App</p>
    </div>
  </body>
  <style>
    body {
      background-color: #282c34;
    }
    div {
      display: flex;
      flex-direction: column;
      justify-content: center;
      align-items: center;
      text-align: center;
      min-height: 100vh;
    }
    p {
      color: #fff;
      font-size: xx-large;
      font-weight: 900;
      font-family: sans-serif;
    }
  </style>
</html>

Retrieve the HTML content :

getfileContent method will try to find index.html file and return its content

If the file does not exist or if it can't read the file, it will throw an error


  /**
  Read `index.html` file and return its content

 - Throws: `Errors.fileNotReadable`
        if the content of `filePath` is unreadable
        `Errors.fileNotFound`
        if file in `filePath` is not found
 - Returns: File content
  */

  private func getfileContent() throws -> String{
    if let filePath = Bundle.main.path(forResource: "index", ofType: "html") {
        do {
            let contents = try String(contentsOfFile: filePath)
            return contents
        } catch {
           throw Errors.fileNotReadable
        }
    } else {
      throw Errors.fileNotFound
    }
  }

Generic error method :

createError method takes an error message and returns an NSError

  /**
  Creates an NSError with a given message.

 - Parameter message: The error message.

 - Returns: An error including a domain, error code, and error      message.
  */
  private func createError(message: String)-> NSError{
    let error = NSError(domain: "app.domain", code: 0,userInfo: [NSLocalizedDescriptionKey: message])
    return error
  }

Initialize the server :

  /**
  Initialization  of the `webserver`
   - Throws: `Errors.fileNotReadable`
              if the content of `filePath` is unreadable
             `Errors.fileNotFound`
              if  file in `filePath` is not found
  */
  public func initWebServer()throws{
   do{
      let content = try getfileContent()
     webServer.addDefaultHandler(forMethod: "GET", request: GCDWebServerRequest.self, processBlock: {request in
        return GCDWebServerDataResponse(html:content)
      })
   } catch Errors.fileNotFound {
      throw createError(message:"File not found")
  } catch Errors.fileNotReadable {
     throw createError(message:"File not readable")
    }
  }

`stopServer` method :

This method is executed when the server is running. It simply stops the server

  /**
  Stop `webserver` and update serverRunning variable to Stopped case
  */
  @objc public func stopServer() -> Void{
    if(serverRunning == ServerState.Running){
      webServer.stop()
      serverRunning = ServerState.Stopped
    }
  }

Expose `WebServerManager` methods to React Native Bridge

As I said previously, RN was made to talk with Objective-C. So we need to create bridging headers.

File -> New -> File or (Cmd + N)
Select Objective-C file
Name it WebServerManager

And add :

#import "React/RCTBridgeModule.h"

@interface RCT_EXTERN_MODULE(WebServerManager, NSObject)
RCT_EXTERN_METHOD(initWebServer)
RCT_EXTERN_METHOD(startServer: (RCTPromiseResolveBlock) resolve
                  rejecter: (RCTPromiseRejectBlock)reject)
RCT_EXTERN_METHOD(stopServer)
@end

We import RCTBridgeModule to use React Native Macros.
Then RCT_EXTERN_MODULE to expose our WebServerManager class to JS side.

Bridging Objective-C and JavaScript

Import WebServerManager module on JS side.

import { NativeModules } from "react-native";

// our native module
const { WebServerManager } = NativeModules;

Complete `App.js` file

First install react-native-elements dependecy.

yarn add react-native-elements react-native-vector-icons

react-native link react-native-vector-icons

import React, { useState } from "react";
import {
  SafeAreaView,
  StyleSheet,
  View,
  Text,
  StatusBar,
  NativeModules,
  TouchableOpacity
} from "react-native";

import { Icon } from "react-native-elements";
// we import our native module
const { WebServerManager } = NativeModules;

const App: () => React$Node = () => {
  const [endpoint, setEndpint] = useState("");
  const [isServerRunning, setServerState] = useState(false);

  const startServer = () => {
    WebServerManager.startServer()
      .then(url => setEndpint(url))
      .then(() => setServerState(true))
      .catch(err => console.error(err));
  };

  const stopServer = () => {
    WebServerManager.stopServer();
    setEndpint("");
    setServerState(false);
  };

  return (
    <>
      <StatusBar barStyle="light-content" />
      <SafeAreaView style={styles.safeView}>
        <View style={styles.infoBlock}>
          <Text style={styles.text}>
            Press button to turn {isServerRunning ? "Off" : "On"} server
          </Text>
        </View>
        <View style={styles.container}>
          <TouchableOpacity>
            <Icon
              raised
              name="power-off"
              type="font-awesome"
              color={isServerRunning ? "#01b907" : "#f44336"}
              onPress={() => (isServerRunning ? stopServer() : startServer())}
            />
          </TouchableOpacity>
        </View>
        {isServerRunning ? (
          <View style={styles.container}>
            <Text style={{ ...styles.text, ...styles.urlEndpoint }}>
              Server is available at this Url : {endpoint}
            </Text>
          </View>
        ) : (
          <View style={styles.container} />
        )}
      </SafeAreaView>
    </>
  );
};

const styles = StyleSheet.create({
  safeView: {
    backgroundColor: "#282c34",
    height: "100%"
  },
  urlEndpoint: {
    paddingTop: 20
  },
  text: {
    color: "#FFF",
    fontWeight: "900",
    fontSize: 20,
    textAlign: "center"
  },
  infoBlock: {
    flex: 1,
    alignItems: "center",
    justifyContent: "center"
  },
  container: {
    flex: 1,
    justifyContent: "center",
    alignItems: "center"
  }
});

export default App;

Then run the app in the simulator.

react-native run-ios

Your app should look like this :

Press button and enter given URL in your browser and you should see this :

You can find the complete project on Github