DEV Community: Rahul Ranjan

Elastic RUM (Real User Monitoring) with Open Telemetry (OTel)

Rahul Ranjan — Sun, 18 Aug 2024 06:53:33 +0000

This article continues the OpenTelemetry with Elastic Observability article, covering “How to set up the OpenTelemetry demo with Elastic Observability” using Docker Compose or Kubernetes.

Elastic real user monitoring, or RUM, captures user interactions with the web browser and provides a detailed view of the “real user experience” of your web applications from a performance perspective.

Elastic’s RUM Agent is a JavaScript Agent, which means it supports any JavaScript-based application. RUM can provide valuable insight into your applications.

Benefits of RUM (Real User Monitoring)
RUM performance data can help you identify bottlenecks and discover how site performance issues affect your visitors’ experience
User-agent information captured by RUM enables you to identify the browsers, devices, and platforms most used by your customers so that you can make informed optimizations to your application
Along with location information, individual user performance data from RUM helps you understand the regional performance of your website worldwide
RUM provides insight and measurement for your application’s service level agreements (SLA)
RUM gathers information on customer visits and clicks behavior over time that can be useful for development teams to identify the impact of new features.

We will cover step-by-step mechanisms for enabling RUM(Real User Monitoring) in the application and then monitor it in Elasticsearch Observability.

Download and Install the application from Github Repo either on Docker or Kubernetes and configure it as described in OpenTelemetry with Elastic Observability.

Once the application is up and running, Go to Your deployment → Integration→APM →RumJS→copy configuration as seen in below image.
Copy the below configuration settings for setting up the Agent as a dependency.

You can install the Agent as a dependency on your application with

npm install @elastic/apm-rum --save

The Agent can then be initialized and configured in your application like below. Refer to the integration documentation for more details.

Alternatively, copy the below configuration settings for setting up the Agent with Script Tags.

Now we will configure the frontend application with these settings to RUM.

Navigate to your application service folder(opentelemetry-demo/src/frontend/Dockerfile) and update the docker file with

RUN npm install @elastic/apm-rum --save

Navigate to the application(frontend) service folder (opentelemetry-demo/src/frontend/pages/_app.tsx) and update the configuration for RUM.
You can customize the service name and environment parameters.

//CONFIGURE RUM AGENT
import { init as initApm } from '@elastic/apm-rum';
if (typeof window !== 'undefined'){
   initApm({
       serviceName: 'otel-frontend',    
       serverUrl: '<url>',
       serviceVersion: '',
       environment: 'demo'
   });
}

Service name: It represents your application in the APM UI.

Service version: This is the version of your application. This version number is also used by the APM server to find the right source map.

Server URL: This is the APM server URL. Note that the APM server URL is normally accessible from the public internet because your RUM agent reports data to it from end-user browsers on the internet.

environment: This is helpful in a shared environment where multiple users are accessing for better filtering.
Login to Kibana and then navigate to the User Experience section of Observability and analyze the RUM data and play around with the available metrics.

Reference:

https://www.linkedin.com/pulse/opentelemetry-elastic-observability-rahul-ranjan-3l6bc/

https://opentelemetry.io/docs/demo/

https://www.elastic.co/guide/en/observability/8.14/apm-configuration-rum.html

OpenTelemetry with Elastic Observability

Rahul Ranjan — Fri, 16 Aug 2024 15:21:49 +0000

OpenTelemetry is an open-source framework for observability that, when combined with Elastic Observability, provides powerful insights into distributed systems. It enables organizations to efficiently monitor, troubleshoot, and optimize their applications. In this article, we will provide you with a detailed guide on how to set up an OpenTelemetry demo with Elastic Observability. We will cover essential steps, configurations, and best practices that will help you leverage the full potential of observability in your environment.

Understanding OpenTelemetry and Elastic Observability:

OpenTelemetry:
OpenTelemetry is a project under the Cloud Native Computing Foundation (CNCF) that aims to provide a cohesive approach to instrument, generate, collect, and export telemetry data, which includes metrics, traces, and logs, from software applications. It offers libraries for instrumenting code in various programming languages and provides standardized APIs to capture telemetry data from different components of distributed systems.

Elastic Observability:
Elastic Observability is a complete solution for observability provided by Elastic. It provides integrated tools for monitoring, logging, and tracing distributed applications. The solution includes Elastic APM (Application Performance Monitoring), Elastic Logs, and Elastic Metrics, all of which are seamlessly integrated within the Elastic Stack.

Setup Details:
This doc will cover both How to set up the OpenTelemetry demo with Elastic Observability using Docker compose or Kubernetes.

Download the source code of the application from GitHub Repo.
Download your application on a Kubernetes cluster in your cloud service of choice or local Kubernetes platform. First, clone the directory locally. Make sure you have kubectl and helm also installed locally:

git clone https://github.com/elastic/opentelemetry-demo.git

OTEL_EXPORTER_OTLP_ENDPOINT is Elastic's APM Server
OTEL_EXPORTER_OTLP_HEADERS Elastic Authorization

Under Integrations->APM in your Elastic cloud, find these values in OpenTelemetry setup instructions.

Docker compose
Start a free trial on Elastic Cloud and copy the endpoint and secretToken from the Elastic APM setup instructions in your Kibana.

Open the file src/otelcollector/otelcol-elastic-config-extras.yml in an editor and replace the following two placeholders:

YOUR_APM_ENDPOINT_WITHOUT_HTTPS_PREFIX: Your Elastic APM endpoint (without https:// prefix) that must also include the port (for example: 987654.xyz.com:443).
YOUR_APM_SECRET_TOKEN: your Elastic APM secret token.
The updated file should look like the below(Make sure to note the actual format for the secret token including the Bearer keyword)

exporters:
  otlp/elastic:
    # !!! Elastic APM https endpoint WITHOUT the "https://" prefix
    endpoint: "11111111111.apm.xyz.xyz.cloud.es.io:443"
    compression: none
    headers:
      Authorization: "Bearer aaaaaaaaaaaaaaa"

exporters:
  otlp/elastic:
    # !!! Elastic APM https endpoint WITHOUT the "https://" prefix
    endpoint: "11111111111.apm.xyz.xyz.cloud.es.io:443"
    compression: none
    headers:
      Authorization: "Bearer aaaaaaaaaaaaaaa"

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [spanmetrics, otlp/elastic]
    metrics:
      receivers: [otlp, spanmetrics]
      processors: [batch]
      exporters: [otlp/elastic]
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/elastic]

The updated file should look something like the above.

Optional: If you are using a shared Elastic Cluster, set the environment name so you can distinguish your data. Go to the (project root)/.env file , and locate the _OTEL_RESOURCE_ATTRIBUTES _variable, and add your environment name like below.

OTEL_RESOURCE_ATTRIBUTES="service.namespace=opentelemetry-demo,deployment.environment=<Any Name>"

Start the demo with the below command from the repository’s root directory:

docker-compose up -d

Verify that the application is running correctly by checking the links below and testing their functionality.

Web store: http://localhost:8080/
Grafana: http://localhost:8080/grafana/
Load Generator UI: http://localhost:8080/loadgen/
Jaeger UI: http://localhost:8080/jaeger/ui/

To stop the application, run the following from the project root directory:

docker-compose down

Kubernetes

Create a Kubernetes cluster. Set up Kubectl and Helm.
Set up Elastic Observability on Elastic Cloud.

Create a secret in Kubernetes with the following command.

kubectl create secret generic elastic-secret \
  --from-literal=elastic_apm_endpoint='YOUR_APM_ENDPOINT_WITHOUT_HTTPS_PREFIX' \
  --from-literal=elastic_apm_secret_token='YOUR_APM_SECRET_TOKEN'

Execute the following commands to deploy the OpenTelemetry demo to your Kubernetes cluster.

# switch to the kubernetes/elastic-helm directory
cd kubernetes/elastic-helm

# Add the open-telemetry Helm repostiroy
helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts

#Update Repo
helm repo update open-telemetry

# deploy the demo through helm install
helm install -f values.yaml my-otel-demo open-telemetry/opentelemetry-demo

Once your application is up on Kubernetes, validate that all the pods are running in the default namespace.

kubectl get pods -n default

Verify that the application is running correctly by checking the links below and testing their functionality.

Web store: http://localhost:8080/
Grafana: http://localhost:8080/grafana/
Load Generator UI: http://localhost:8080/loadgen/
Jaeger UI: http://localhost:8080/jaeger/ui/

Kubernetes Monitoring:

This demo includes cluster-level metrics and Kubernetes events collection. To enable Node-level metrics collection and autodiscovery for Redis Pods, run an additional Otel collector Daemonset.

helm install daemonset open-telemetry/opentelemetry-collector --values daemonset.yaml

Explore and analyze the data With Elastic

View your OTel instrumented services in Kibana's APM Service Map. To access, go to APM in Elastic Observability UI and select servicemap.

If you are seeing this, it means that data is being sent to the Elastic cluster by the OpenTelemetry Collector. You can now explore the data and experiment with it.

To get a comprehensive understanding of all the services and transaction flows between them, you can refer to the APM service map (as demonstrated in the previous step). Additionally, you have the option to examine individual services and the collected transactions.

As you can see, the loadgenerator details are listed:

Average service latency
Throughput
Main transactions
Failed traction rate
Errors
Dependencies

Now Click on Transactions → GET(or any request) and we can see the full trace with all the spans. You can further explore and analyze data, examining it with minute detail.

Elastic utilizes machine learning to identify potential latency issues across services by analyzing the trace. Users can easily access the Latency Correlations tab and run the correlation.

Troubleshoot OTEL data ingest issue:

If you are unable to see any data in your Environment view, start by checking the logs of the OTEL collector with the following command:

docker-compose logs -f otelcol

If you come across any errors related to the connection, please double-check the URL and token configuration.
If you are unable to see any services running, try running a full restart of the docker compose and see if that helps.

Analyze
Analyze your data with Elastic machine learning (ML)
After integrating OpenTelemetry metrics with Elastic, you can begin to analyze your data using Elastic's machine-learning capabilities.

References:

https://www.linkedin.com/pulse/opentelemetry-elastic-observability-rahul-ranjan-3l6bc/

https://medium.com/@rahul.fiem/opentelemetry-demo-with-elastic-observability-45d938a65ef8

https://rranjan.hashnode.dev/opentelemetry-with-elastic-observability

Repo: https://github.com/elastic/opentelemetry-demo

Real-Time News Aggregator with Elastic: Leveraging APM, RUM, and Elasticsearch for Optimized Performance

Rahul Ranjan — Thu, 15 Aug 2024 09:48:57 +0000

This guide will explain how to set up a Dockerized news aggregator. This aggregator will retrieve data from an external API, store it in Elasticsearch, and display it using a Flask UI. Additionally, we will integrate our application with Elastic APM for performance monitoring and gather RUM (Real User Monitoring) data from the front-end application.

Prerequisites
Docker and Docker Compose are installed on your machine.
Elastic Cloud account (or a local Elasticsearch instance).
API Key for the news data source (e.g., NewsAPI).
Basic knowledge of Python and Flask

Project Overview
This project revolves around a dual-service architecture, powered by Docker containers:
News Aggregation Service:
Flask UI Service:

Real User Monitoring (RUM) Integration
To capture real-time data on how users interact with the UI, I integrated Elastic's RUM agent into the index.html file of the Flask service. The RUM agent collects critical data such as page load times, user interactions, and any client-side errors, providing a comprehensive view of the end-user experience.
This integration allows me to optimize the UI continuously, ensuring that any issues affecting the user experience are identified and resolved quickly.

Project Structure

Step-by-Step Guide to Building the Project

Step 1: Set Up Elasticsearch on Elastic Cloud

Sign Up: Begin by signing up for an account on Elastic Cloud.
Create Deployment: Create a new Elasticsearch deployment. This will provide you with a cluster where all news articles will be indexed and stored.

Get Credentials:

Note down the cloud ID, username, and password for your deployment. These credentials will be needed later to connect your services.

Step 2: Develop the News Aggregation Service

In the backend, we implement the logic to fetch news from an external API, process the data, and index it into Elasticsearch. We use a specific library to fetch data and interact with Elasticsearch. Additionally, the backend is instrumented with Elastic APM to capture performance metrics.

Step 3: Develop the Flask UI Service

The Flask UI is a straightforward web application that retrieves news articles from Elasticsearch and presents them on a web page. We utilize Flask's templating engine to generate the HTML and CSS for the frontend. Additionally, Elastic APM is integrated into the Flask app to monitor frontend performance.

Step 4: Integrate Real User Monitoring (RUM)
RUM Agent Setup: Added the RUM agent script to the index.html file to capture real-time user interactions. Update the actual value for serviceName, serverUrl and secretToken.

<!-- Elastic APM RUM Agent Configuration -->
    <script>
        function handleScriptError() {
            console.error('Failed to load Elastic APM RUM agent script.');
        }

        function initializeApm() {
            if (window.elasticApm) {
                window.elasticApm.init({
                    serviceName: '',
                    serverUrl: '',
                    secretToken: '',
                    environment: 'news'
                });
                console.log('Elastic APM RUM initialized.');
            } else {
                console.error('Elastic APM RUM agent not loaded.');
            }
        }
    </script>
    <script src="https://cdn.jsdelivr.net/npm/@elastic/apm-rum@5.9.0/dist/bundles/elastic-apm-rum.umd.min.js"
        onload="initializeApm()" onerror="handleScriptError()"></script>

Step 5: Containerize the Services with Docker

Dockerize the News Aggregation Service: Write a Dockerfile for the news aggregator.

# Use a lightweight Python image
FROM python:3.9-slim

# Set the working directory in the container
WORKDIR /app

# Copy the requirements file into the container
COPY requirements.txt .

# Install the required Python packages including Elastic APM
RUN pip install --no-cache-dir -r requirements.txt elastic-apm

# Copy the rest of the application code into the container
COPY . .

# Set environment variables (Update with your actual values)
ENV NEWS_API_KEY=''
ENV ELASTICSEARCH_HOST=''
ENV ELASTICSEARCH_USERNAME=''
ENV ELASTICSEARCH_PASSWORD=''
ENV ELASTIC_APM_SERVER_URL=''
ENV ELASTIC_APM_SECRET_TOKEN=''
ENV ELASTIC_APM_SERVICE_NAME='r'
ENV ELASTIC_APM_ENVIRONMENT='php'
# Run the application
CMD ["python", "stream_news.py"]

Dockerize the Flask UI Service: Write a Dockerfile for the Flask UI service.

# Use the official Python image
FROM python:3.9-slim

# Set the working directory in the container
WORKDIR /app

# Copy the requirements file into the container
COPY requirements.txt .

# Install the required Python packages including Elastic APM
RUN pip install --no-cache-dir -r requirements.txt elastic-apm

# Copy the rest of the application code into the container
COPY . .

# Set environment variables for Elasticsearch (Update with your actual values)
ENV ELASTICSEARCH_HOST=''
ENV ELASTICSEARCH_USERNAME=''
ENV ELASTICSEARCH_PASSWORD=''
ENV ELASTIC_APM_SERVER_URL=''
ENV ELASTIC_APM_SECRET_TOKEN=''
ENV ELASTIC_APM_SERVICE_NAME=''
ENV ELASTIC_APM_ENVIRONMENT=''

# Expose the port Flask runs on
EXPOSE 5001

# Run the Flask application
CMD ["flask", "run", "--host=0.0.0.0", "--port=5001"]

Docker Compose: Create a docker-compose.yml to orchestrate the services.

We define two services in our docker-compose.yml file: news_aggregator and flask_ui. Each service has its own Dockerfile and environment variables, including the Elastic APM configuration.

services:
  news_aggregator:
    build:
      context: ./news_aggregator
      dockerfile: Dockerfile.news_aggregator
    container_name: news_aggregator
    environment:
      NEWS_API_KEY: ''
      ELASTICSEARCH_HOST: ''
      ELASTICSEARCH_USERNAME: ''
      ELASTICSEARCH_PASSWORD: ''
      ELASTIC_APM_SERVER_URL: ''
      ELASTIC_APM_SECRET_TOKEN: ''
      ELASTIC_APM_SERVICE_NAME: ''
      ELASTIC_APM_ENVIRONMENT: ''
    restart: always

  flask_ui:
    build:
      context: ./flask_ui
      dockerfile: Dockerfile.ui
    container_name: flask_ui
    ports:
      - "5001:5001"
    environment:
      ELASTICSEARCH_HOST: ''
      ELASTICSEARCH_USERNAME: ''
      ELASTICSEARCH_PASSWORD: ''
      ELASTIC_APM_SERVER_URL: ''
      ELASTIC_APM_SECRET_TOKEN: ''
      ELASTIC_APM_SERVICE_NAME: 'i'
      ELASTIC_APM_ENVIRONMENT: ''
    depends_on:
      - news_aggregator
    restart: always

Step 6: Deploy and Monitor

Run the Services: Use Docker Compose to start the services.
docker-compose up --build -d

Monitor with Kibana
Head over to your Elastic Cloud deployment and use Kibana to monitor the performance of your services through the APM and RUM dashboards.

Real User Monitoring(RUM) Dashboard

Launch the Flask UI application to stream the news by logging in to http://127.0.0.1:5001/

Flask UI Page

The project code is available at my GitHub repo for download and ready to use by replacing the Elasticsearch, APM credentials.

Python #Flask #Elasticsearch #ElasticAPM #Docker #WebDevelopment #RealTimeApplications #Monitoring #DevOps #search

Ref https://www.linkedin.com/pulse/building-real-time-news-aggregator-elastic-stack-apm-rum-rahul-ranjan-28jpe/