DEV Community: Das

Real-Time Test Results in Robot Framework

Das — Sat, 21 Mar 2026 20:44:19 +0000

No more waiting for the suite to finish to find out what failed.

Introduction

In a previous organisation, a proof of concept was built to stream Robot Framework test results into a live monitoring dashboard using the ELK stack (Elasticsearch, Logstash, and Kibana). The idea was simple: instead of waiting for the full run to complete and then reading the output report, results would appear in a dashboard the moment each test finished.

The problem with ELK is that it is no longer fully open source. Elastic changed its licensing in 2021. OpenSearch is the community fork that stayed open. This project is a rebuild of that original setup using OpenSearch, with the same goal: live test visibility while the suite is still running.

The approach works with any test framework that exposes hooks into the test lifecycle. Robot Framework is used here because that is where the original work was done, but the same pattern applies to pytest, Selenium, and others.

The Problem

When a test suite runs, results are only written to output.xml when the entire run completes. For short suites this is fine. For longer ones it creates a gap.

During the run, the only signal is whatever the terminal prints. A failure shows up as FAIL with a short message, but by the time the run finishes and the report is available, the context around that failure is gone. Was it a transient error? A timeout that only happens under load? Something that affected multiple tests in the same suite? There is no way to know without digging through logs after the fact.

What would actually help is seeing each result as it happens, with the full message, the test name, the suite, and the duration, in a searchable dashboard that stays available after the run.

Tech Stack

Robot Framework - test framework with a listener API for hooking into test execution
OpenSearch - open-source search and analytics engine for storing results
OpenSearch Dashboards - visualisation layer for building live dashboards
Docker - runs OpenSearch and the dashboard in containers
Python - the listener that ships results to OpenSearch

Architecture

+-------------------------------+
|  Robot Framework (local)      |
|  listener fires after each    |
|  test and ships the result    |
+---------------+---------------+
                |
                v
+---------------+---------------+        +---------------------------+
|  OpenSearch (Docker)          | -----> |  OpenSearch Dashboards    |
|  port 9200                    |        |  port 5601                |
+-------------------------------+        +---------------------------+

Robot Framework and tests run locally as usual. Docker handles OpenSearch and the dashboard without any manual installation needed.

Step by Step

Prerequisites

1. How the Listener Works

Robot Framework exposes a listener API, a Python class it calls at specific points during execution. The hook used here is end_test, which fires immediately after each test completes, before the next one starts.

class OpenSearchListener:
    ROBOT_LISTENER_API_VERSION = 2

    def end_test(self, name, attrs):
        doc = {
            "run_id": self.run_id,
            "suite": self.suite_name,
            "test": name,
            "status": attrs["status"],
            "message": attrs["message"],
            "tags": list(attrs["tags"]),
            "start_time": ...,
            "elapsed_seconds": attrs["elapsedtime"] / 1000,
            "indexed_at": datetime.utcnow().isoformat() + "Z",
        }
        self.client.index(index=self.index, body=doc)

Each test result is indexed into OpenSearch straight away. The document is in the index before the next test has started.

Each document looks like this:

{
  "run_id": "a3f2c1...",
  "suite": "Tests.Login",
  "test": "Valid credentials should log in",
  "status": "FAIL",
  "message": "Element not found: #submit-btn",
  "tags": ["smoke", "auth"],
  "start_time": "2026-03-21T12:20:35.701Z",
  "elapsed_seconds": 3.001
}

The run_id field is generated once per test run, which makes it straightforward to filter the dashboard to a specific run or compare runs over time.

2. Setup

Clone the repo and install dependencies:

git clone https://github.com/007bsd/results-execution-monitoring
cd results-execution-monitoring
pip install -r requirements.txt

Start OpenSearch and the dashboard:

docker compose up -d

Note: The first run downloads around 1 GB of images. Once containers are up, confirm OpenSearch is ready with curl http://localhost:9200. A JSON response means it is running.

Run the setup script once to create the index pattern and a starter dashboard:

python setup_dashboard.py

This is meant to get things running quickly. The dashboard it creates is a starting point, not the final word. See the section below on building dashboards manually.

3. Running Tests with the Listener

python -m robot --listener opensearch_listener.OpenSearchListener tests/

The listener confirms it has started and prints the run_id and index name:

As the run progresses, each result is confirmed in the terminal:

Open http://localhost:5601 before or during the run to watch results appear in real time.

4. Verifying Data in OpenSearch

To confirm results are being indexed correctly, the Dev Tools console at http://localhost:5601/app/dev_tools#/console can be used to query the index directly:

GET robot-results/_mapping

GET robot-results/_search
{
  "size": 1
}

5. Exploring Results in Discover

OpenSearch Dashboards has a Discover view that lets you search and filter all indexed documents. It is useful for digging into specific failures, filtering by tag, run, or suite, and understanding patterns across multiple runs.

6. Building Dashboards Manually

The setup script creates a starter dashboard to get things going, but most people will want to build their own visualisations on top of the data. OpenSearch Dashboards has a full visual editor for this, no configuration files or scripts required.

To create a visualisation:

Go to http://localhost:5601 and open Visualize from the menu
Click Create visualization and choose a chart type
Select robot-results as the index
Configure the metric (e.g. Count) and bucket (e.g. Terms on status) to get a pass/fail breakdown
Save the visualisation with a name

To build a dashboard from those visualisations:

Open Dashboard from the menu
Click Create new dashboard
Click Add from library and select the saved visualisations
Arrange the panels, set a title, and save

Some useful combinations to start with:

Pie or donut chart of status for an overall pass/fail ratio
Data table of recent failures showing test, suite, and message
Bar chart of elapsed_seconds by test name to spot slow tests
A metric panel showing total fail count for the current run
Filter by run_id to isolate and compare specific runs

Result

Once the stack is running, every test result appears in the dashboard the moment it is indexed. Failures are immediately visible with the full message, test name, suite, and duration, without waiting for the suite to complete.

Conclusion

The standard approach to test reporting is batch: everything is available when the run finishes, and not before. For short suites this is acceptable. For longer ones it is a genuine visibility problem.

Streaming results to OpenSearch as each test completes inverts that. Results are available immediately, failures are visible in context, and the history of every run is retained and searchable without any extra work.

The listener pattern used here is Robot Framework-specific, but the underlying idea applies to any test framework with a comparable hook system.

Resources

GitHub - github.com/007bsd/results-execution-monitoring
Robot Framework Listener API - robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#listener-interface
OpenSearch Documentation - opensearch.org/docs/latest
OpenSearch Dashboards - opensearch.org/docs/latest/dashboards
Docker - docs.docker.com

If you try setting this up and encounter any issues, please leave a comment. The complete code for this project is available on GitHub

Real-Time Data Monitoring Using InfluxDB and Grafana

Das — Sat, 14 Mar 2026 11:51:04 +0000

A practical guide to building a live monitoring dashboard for any data source.

Introduction

In a previous organisation, I built and maintained several internal monitoring dashboards. The idea came from patterns I had seen in engineering blogs about observability systems, and I experimented internally with a similar setup to bring different metrics into a single real-time view. The architecture behind those dashboards was straightforward and reusable.

This project is a recreation of that setup using publicly available APIs, tracking weather across two cities, commodity prices, and exchange rates EUR->INR as a working example. It continuously collects data, stores it in a time-series database, and visualises it in a live Grafana dashboard. It is a starting point for anyone who wants to experiment with real-time data or build similar dashboards for their own projects.

Organisations and individuals often have access to data from multiple sources that updates continuously. The challenge is not finding the data. Most of it is freely available. The challenge is aggregating it into a single, live view that is easy to read and act on.

Without a proper pipeline, data either gets checked manually, sits in spreadsheets, or simply goes unmonitored. A real-time dashboard solves this by automating the collection, storage, and visualisation in one place.

The same problem exists at different scales. Whether it is infrastructure monitoring inside a company, tracking external data feeds, or keeping an eye on any system that produces regular measurements, the approach is the same.

Tech Stack

InfluxDB — time-series database that stores all collected data
Grafana — visualisation layer that renders live dashboards
Docker — runs the entire stack in containers
Python — polling script that fetches data and writes to InfluxDB
Telegraf — alternative data collection agent by InfluxData

Note: This project uses a Python script for data collection, but Telegraf or any other tool that can write to InfluxDB works just as well.

Architecture

There are two separate flows in the project.

Live data flow

Data Sources (APIs)
        │
        ▼
   poller.py
   (runs every 15m)
        │
        ▼
     InfluxDB
     (storage)
        │
        ▼
     Grafana
        │
        ▼
     Browser

Dashboard setup flow

dashboard.yml
      │
      ▼
build_dashboard.py
      │
      ▼
dashboard.json
      │
      ▼
Grafana loads it automatically

The live data flow runs continuously in the background. The dashboard setup flow runs once locally, and again whenever the dashboard configuration is updated.

Note: The dashboard can also be created manually through the Grafana UI without using any configuration files or scripts. The code-based approach is used here to keep the setup version controlled and easy to modify.

Step by Step

Prerequisites

Before getting started, make sure the following are installed:

Docker Desktop — required to run the stack
Python — required to generate the dashboard configuration

Note: Docker is used here to demonstrate the full setup locally. InfluxDB and Grafana can also be installed directly on any machine or server without Docker if preferred.

1. Data Sources

Three publicly available APIs are used in this project, covering weather, commodity prices, and exchange rates. No API keys or signups are required.

Any API or data source that returns regularly updating values can be used here. The structure of the pipeline does not change regardless of what data is being collected.

2. Data Collection

The poller is a Python script that queries the APIs every fifteen minutes and writes the results directly to InfluxDB.

One concept worth understanding in InfluxDB before writing data:

Tags describe the data and are indexed for filtering. Examples: city name, source identifier, category.
Fields contain the actual numeric values that get stored and plotted.

Getting this distinction right makes querying and building dashboards significantly simpler.

3. Storage with InfluxDB

InfluxDB is a time-series database built for handling measurements that arrive continuously. Data is written as timestamped points, which makes it well suited for infrastructure metrics, sensor readings, financial feeds, or any other regularly updating values.

Each record written contains a measurement name, tags, numeric fields, and a timestamp.

To verify the connection is working, the Grafana Explore view can be used to query the bucket directly. A simple query returning the last hour of data confirms that the poller is writing successfully and Grafana can read it.

4. Visualisation with Grafana

Grafana reads from InfluxDB and renders the data as a live dashboard. It supports charts, stat panels, colour thresholds, gauges, and auto-refresh at configurable intervals.

It also connects to many other data sources including Prometheus, PostgreSQL, and others, making it flexible beyond just this stack.

One of the more useful features is the time range selector. The same dashboard can show data from the last 5 minutes all the way through to months of history, and the panels update instantly as the range changes.

Grafana also supports alerting. Alerts can be configured on any panel to trigger notifications when a value crosses a threshold — useful for staying on top of data without having to watch the dashboard continuously. Notifications can be sent to email, Slack, PagerDuty, and many other channels.

5. Dashboard as Code

Grafana dashboards are often created manually through the UI. That works for quick experiments, but the exported JSON becomes large and difficult to maintain over time.

In this project the dashboard is defined in a dashboard.yml configuration file. A script converts it into the JSON format Grafana expects, and Grafana loads it automatically on startup.

This keeps the dashboard fully version controlled. Adding a new panel is a matter of updating one line in the config file.

6. Running the Stack

The entire system starts with one command:

docker compose up -d

This launches three containers: InfluxDB, Grafana, and the poller. The dashboard is immediately available at localhost:3000. Pre-configured, pre-seeded, already collecting.

Result

Once the stack is running, the dashboard provides a continuously updating view of all collected data. It refreshes automatically, is colour-coded by thresholds, and is queryable over any time range.

The same setup can be pointed at any data source without changing the underlying infrastructure. The only part that changes is the polling script.

Conclusion

The pattern covered in this project: collect, store, visualize. It is a solid foundation for building active monitoring dashboards around any kind of live data.

The data collection piece is intentionally flexible. Telegraf works well for infrastructure and system metrics with minimal configuration. A custom script makes sense when working with external APIs that need specific handling. A cloud function or a scheduled job fits just as well depending on the environment.

What stays consistent is the rest of the stack. InfluxDB handles the time-series storage reliably regardless of where the data comes from. Grafana turns it into something usable, with the added ability to set up alerts and query across any time range without much overhead.

The same approach applies whether the goal is monitoring application performance, tracking external data feeds, or building visibility into any system that produces regular measurements.

Resources

GitHub — github.com/007bsd/live-monitor
InfluxDB Documentation — docs.influxdata.com
Grafana Documentation — grafana.com/docs
Grafana Alerting — grafana.com/docs/grafana/latest/alerting
Telegraf — influxdata.com/telegraf
Docker — docs.docker.com

If you try setting this up and encounter any issues, please leave a comment. The complete code for this project is available on GitHub.

Building a Fully Local LaTeX Thesis Workflow with VS Code and MiKTeX

Das — Sat, 14 Mar 2026 11:11:52 +0000

Why I Didn't Use Word

I tried writing in Word at first. It works fine for small documents, but once the thesis started growing, it became harder to manage. Formatting, figures, and updating the table of contents sometimes caused other sections to shift unexpectedly. Fixing small layout issues was taking more time than actual writing.

For a long academic document with references, numbered figures, and strict formatting requirements, I wanted something more structured.

Why I Didn't Use Overleaf Either

Overleaf is the go-to recommendation for LaTeX beginners, it runs in the browser, there's nothing to install, and it has a live PDF preview. For a quick document, it's fine.

But for a full thesis, it has real limitations:

For security reasons, you may not want your research or unpublished work stored on a public platform
The free tier has limits
Your data lives on their servers
You can't work offline

The alternative? Run everything locally. Your machine, your files, your Git repository. Free forever. Works offline. Private by default.

The Stack: LaTeX + MiKTeX + VS Code

LaTeX — the document preparation system. You write in plain text with markup, and LaTeX produces a perfectly formatted PDF
MiKTeX — a LaTeX distribution for Windows that manages all the packages you need, installing them automatically when required
VS Code — the code editor, used here as a LaTeX editor with the LaTeX Workshop extension
XeLaTeX — the compiler. Better than pdfLaTeX for modern fonts and Unicode
Biber — bibliography manager, works with BibTeX .bib files
Makeglossaries — handles abbreviation lists and acronyms
Python + Pygments — required for the minted package for syntax-highlighted code blocks

Step-by-Step Setup

Before you start: This guide covers Windows. The same tools exist for macOS and Linux with slightly different installation steps.

Step 1: Install MiKTeX

MiKTeX is the LaTeX engine. It comes with XeLaTeX, Biber, and Makeglossaries, and it will automatically download any missing LaTeX packages the first time you compile.

Go to miktex.org/download
Download the Windows installer and run it
Choose Install for all users if you have admin rights, otherwise install for yourself
After installation, open a terminal and verify:

xelatex --version
biber --version
makeglossaries --version

If anything is missing, MiKTeX's package manager can install them.

Step 2: Install Python and Pygments

The template uses the minted package for code syntax highlighting. Check if Python is already installed:

python --version
pygmentize -V

If not, download from python.org/downloads. Then install Pygments:

pip install pygments

Step 3: Install VS Code and Extensions

Download from code.visualstudio.com
Install and open VS Code
Search and install the LaTeX Workshop extension from the Extensions panel

Step 4: Clone the Template and Configure the Compiler

git clone https://github.com/007bsd/metropolia-thesis-latex.git
cd metropolia-thesis-latex
code .

Or download the ZIP from GitHub directly and open the folder in VS Code.

The project includes a .vscode/settings.json that already configures LaTeX Workshop to:

Use XeLaTeX as the compiler
Set main.tex as the root document
Auto-clean auxiliary files after building

Step 5: Compile Your First PDF

Open main.tex. Click the green ▶ Build LaTeX project button in the top right, or press Ctrl+Alt+B.

LaTeX Workshop will run the full compilation sequence:

xelatex -shell-escape -8bit main
biber main
makeglossaries main
xelatex -shell-escape -8bit main
xelatex -shell-escape -8bit main

The first compile may take a minute as MiKTeX downloads any missing packages. When it succeeds, main.pdf appears in the project root and opens automatically in the VS Code PDF preview panel.

Step 6: Start Writing

Open any file in the chapters/ folder and start editing. Every time you save (Ctrl+S), LaTeX Workshop auto-recompiles and the PDF updates in real time.

Project structure:

my-thesis/
├── main.tex          ← Master file
├── chapters/         ← One .tex file per chapter
├── biblio.bib        ← Bibliography
├── illustration/     ← Images and figures
├── code/             ← Code files for minted snippets
└── style/            ← Formatting rules

Tips From Actually Using This

Manage references with Zotero. Collect and organize references in Zotero, export directly as a .bib file into biblio.bib. Works seamlessly with Biber and saves a lot of manual formatting.

Keep your Git commits small. Commit chapter by chapter, not just at the end. You will want to roll back if you rewrite a section and change your mind.

Don't edit your style files. These contain the formatting rules. If you change them and something breaks, debugging is painful.

The Result

After compilation succeeds you have a properly formatted PDF — table of contents, numbered figures, bibliography, and acronym list all generated automatically. No style settings touched manually. No Word. No cloud.

Why This Matters Beyond the Thesis

For students in technical fields, getting used to LaTeX means getting comfortable with plain text and structured writing. That mindset helps beyond just thesis work.

The setup took me around 30–45 minutes. After that, it was just writing.

Resources

Template repository: github.com/007bsd/metropolia-thesis-latex
MiKTeX: miktex.org
VS Code: code.visualstudio.com
LaTeX Workshop extension: Search "LaTeX Workshop" in VS Code Extensions
Pygments: pip install pygments
Zotero: zotero.org

If you run into any issues setting this up, feel free to leave a comment — happy to help based on my experience.