DEV Community: Volodymyr Obrizan

Testinel: How I Improved the Automated Testing Workflow with pytest, Selenium, and Playwright

Volodymyr Obrizan — Sat, 18 Jul 2026 17:40:24 +0000

In this post, I want to share my experience improving the automated testing process for web applications using Python, pytest, Selenium, and Playwright, and explain how I developed Testinel — a test sentinel, a guardian watching over your tests. My motivation for creating this tool was to reduce the time engineers spend fixing failures in automated tests and to lower the barrier to entry for test automation engineers.

Introduction

One of my favorite XKCD comics is “Programmers’ #1 Excuse for Legitimately Slacking Off” by Randall Munroe:

Manager:

— Hey! Get back to work!

Programmers:

— The code is compiling!

— Oh, okay then.

Almost 20 years have passed since it was published, but the comic is still relevant not only to large projects written in compiled languages such as C++, Rust, or Swift, but also to the process of automating web application testing.

— Get back to work!

— The tests are running!

— Oh, okay then.

What should a test automation engineer do while automated tests are running? When the entire test suite finishes within a few minutes, you can make yourself a cup of coffee or lean back in your chair and take a short break. But what if the tests take tens of minutes or even hours? There are several options:

do nothing and relax;
do nothing and stare at the console or the CI server page;
work on something else: write new tests, communicate with the team, or perform manual testing.

If you have other options, tell me in the comments what you do when your tests take an entire hour to complete.

Two Problems with Monitoring Automated Test Execution

This is where I noticed the first problem in the testing process: while a long automated test suite is running, the engineer uses their working time inefficiently. They either do nothing — the XKCD option — or switch to other tasks, which reduces productivity because they lose context and stop actively monitoring the tests. Task switching itself also has a negative effect.

What do I mean by monitoring automated test execution? If all tests pass, which happens fairly often, that is good news and the engineer will not have much work to do. They only need to report to the manager and the team that the automated tests did not detect any problems.

If some tests fail, however, the difficult work of processing the report begins:

identify which tests failed;
determine the reason for each failure: a web application defect, invalid test data, an error in the automated test code, or a failure in the testing infrastructure;
fix errors in the tests or failures in the testing infrastructure and rerun the tests;
create or reopen a bug tracker ticket for defects in the web application.

Why does the engineer not monitor test execution from the very beginning? Because the pytest test runner is designed to generate a detailed test report only after all tests have completed. See the video below.

The engineer can see that a test has failed — FAILED — but pytest provides the reason for the failure, including the exception, message, and location in the code, only after every test has finished running.

This problem is even more noticeable on a CI server because various testing artifacts — logs, screenshots, videos, and so on — also become available only after the automated tests have completed.

Conclusion No. 1: during automated test execution, the engineer does not have all the information needed to determine the cause of a failure. Without this information, the engineer can neither create a high-quality bug tracker ticket nor properly fix the automated test scripts.

The second problem that makes engineers work inefficiently is that diagnostic information about test failures is poorly structured. Anyone who works with automated testing knows how painful it can be to investigate the failure of even a single test:

the traceback can be very long;
the exception message can be very long;
the place where the exception occurs is not necessarily the place you should inspect, because you usually need to look at the automated test code rather than the library code;
all of this is presented as one long wall of text.

Those who do not work with automated tests can watch the end of the video above or look at the screenshot below. And this is only part of the output, because a typical traceback does not fit on a single monitor screen.

Conclusion No. 2: engineers spend a significant amount of time reading large amounts of text to understand why a test failed and how to fix it.

How I Solved These Two Problems

Let me recap the problems I am trying to solve:

pytest does not generate a detailed report until testing is complete.
Diagnostic information about test failures is poorly structured.

I solved the problem of delayed reporting by developing the pytest-testinel plugin. It streams test events and diagnostic information to a cloud server while the tests are running. At the same time, it uploads screenshots, videos, and Playwright traces to cloud storage so that everything becomes available as soon as an individual test finishes, rather than only after the entire test suite has completed.

There is no need to rewrite, refactor, or annotate your tests in any special way. It is a drop-in plugin, which means you only need to install and configure it:

Sign up, create a project, and obtain a TESTINEL_DSN.
Run pip install pytest-testinel.
Run export TESTINEL_DSN=https://testinel.dev/ingest/your_secret_dsn/.

That is all!

This video demonstrates test events being streamed from Bitbucket Pipelines to the cloud server. Notice that test results appear progressively as the tests run on the CI server. You can begin investigating failures immediately instead of waiting for the entire test suite to finish.

I solved the problem of poorly structured diagnostic information by presenting it in a clear and organized way:

I extracted the key information from messages and hid unnecessary noise;
I separated each traceback frame from the others;
I classified frames as test, page object, Selenium, and so on, and hid the code of library frames because engineers need to inspect them far less often than the test code;
I added screenshots, videos, and Playwright traces directly to the report, so engineers no longer need to search for them on the CI server or in a reports directory;
I added a little color to the code to make it easier on the eyes.

As a result, I got one big beautiful™ test report:

As a bonus, I also got a history of automated test runs. All the structured information was already stored in the database, so why not display it?

Anyone running tests with Python, pytest, Selenium, and Playwright can use this system for free: testinel.dev

Conclusions

I have been testing this solution with our test automation team for six months. On many occasions, I noticed that commits fixing errors in the automated tests appeared in Git before the test suite had even finished running.

This means that the value hypothesis behind real-time event streaming was confirmed not only by positive feedback from engineers, but also in practice.

Anyone interested in trying it with their own tests can do so for free: testinel.dev

“Volodymyr, you are several years too late. Humans do not read these stack traces anymore. An agent does not care what color the code is in the traceback!”

You’re absolutely right!™ I am late!

I came up with the idea for this kind of system long before the AI revolution, and I decided to bring it to production to finally close that gestalt.

Besides, agents do not solve the problems of real-time event streaming or cloud storage for diagnostic information. To make Testinel useful in this new environment, I am planning to develop a skill and a CLI utility that will make all of this diagnostic information available to agents.

I therefore believe that the tool will remain useful, provided that such an integration is implemented.

Vova, always close your gestalts.

What is Functional Programming?

Volodymyr Obrizan — Tue, 15 Apr 2025 09:14:35 +0000

Functional programming (FP) is a programming paradigm where computations are treated as the evaluation of mathematical functions rather than as a sequence of instructions that change state.

At the heart of FP are pure functions, immutability of data, and the absence of side effects. This approach helps create code that is clear, predictable, and easy to test.

Key Features of Functional Programming

Pure Functions — A function always returns the same result for the same inputs and does not alter any external state.
Immutability — Instead of modifying existing variables, new values are created. 3.First-Class Functions — Functions are treated as first-class citizens — they can be passed as arguments, returned from other functions, and assigned to variables.
No Side Effects — Functions do not affect the external environment: no file writing, no global state changes, no console output.
Recursion Instead of Loops — Recursion is often used instead of iterative constructs like for and while loops.
Expression-Oriented Programming — Everything is an expression that returns a value, rather than just a sequence of instructions.
Function Composition — Complex behavior is built by combining simple functions together.

Where Is Functional Programming Used?

Big Data Processing — Due to its suitability for parallel computation, FP is popular in big data tools like Apache Spark (Scala).
Financial and Scientific Applications — Where precision, predictability, and mathematical rigor are essential.
Web Services and APIs — Used in languages like JavaScript or Elixir to build scalable reactive systems.
Software Testing and Verification FP simplifies unit testing due to the predictability of pure functions.
Frontend Development — Libraries like React apply many functional programming concepts (e.g., pure components, immutable state).

For my Ukrainian-speaking followers

For anyone who wants to dive deeper into functional programming, I’ve translated the tutorial “Functional Programming in Python” (Функціональне програмування на Python) into Ukrainian, made the examples interactive, and added knowledge quizzes. It’s online and free.

Why Bitbucket Pipelines is Slow and How to Optimize

Volodymyr Obrizan — Tue, 30 Jul 2024 12:26:42 +0000

Bitbucket Pipelines can be slow due to poor CI infrastructure performance and the time-consuming process of installing dependencies for each build. Here's how to address these issues:

Performance Issues:

Infrastructure Delays: Delays in provisioning and build setup can occur, with performance inconsistencies noted in monitoring projects.
Dependency Installation: Installing libraries, tools, packages, and third-party dependencies each time consumes significant build time.

Optimization Approach:

Pre-configure Docker Image:

Dockerfile Creation: Move dependency installation commands to a Dockerfile.
Build and Push Image: Use Docker Buildx to create an image and push it to Docker Hub.
Update Pipeline: Modify bitbucket-pipelines.yml to use the new Docker image, eliminating repetitive setup steps.

Example Dockerfile:

FROM python:3.11.7-slim-bullseye
RUN apt-get update
RUN apt-get install -y build-essential postgresql postgresql-client curl
RUN curl https://dl.min.io/client/mc/release/linux-amd64/mc --create-dirs -o ~/minio-binaries/mc
RUN chmod +x ~/minio-binaries/mc
COPY requirements.txt .
RUN pip install --ignore-installed -r requirements.txt

Example Updated bitbucket-pipelines.yml:

image: dntl/1l2c-backend-for-tests:latest
pipelines:
  default:
    - step:
        script:
          - export PATH=$PATH:~/minio-binaries/
          - mc alias set testing-minio http://localhost:9000 root root-pass
          - mc admin user svcacct add --access-key "test_access_key" --secret-key "test_secret_key" testing-minio root
          - mc mb testing-minio/dean-testing
          - PGPASSWORD=root psql -h localhost -p 5432 -U root postgres -c 'create database dean_unit_tests_db;'
          - python -m pytest . --test_application_name=dean --junitxml=./test-reports/dean_out_report.xml
        services:
          - postgres
          - elasticsearch
          - minio

Results:

Setup Time Reduction: Dependencies are pre-configured, reducing setup time from several minutes to less than 10 seconds.

Overall Efficiency: Test execution time decreased significantly, saving valuable build minutes.

Conclusion:

By pre-configuring a Docker image, you can optimize Bitbucket Pipelines, ensuring faster and more reliable builds. This approach minimizes dependency on CI infrastructure performance and reduces the total build time.

For more detailed information, visit the full article here.

Why Programmers Have a Tough Job

Volodymyr Obrizan — Fri, 12 Jan 2024 08:44:42 +0000

In one of the projects we're developing, 85% of the code volume is from third-party libraries. That is, these are the libraries that need to be installed in the project so that the other 15% of the code can work with something. There are a total of 93 such libraries.

Since the software market is very large and attractive (hundreds of billions of dollars), there are many players, and the competition is frankly fierce.
Players try to innovate, create new features for products, make improvements, in order to stand out favorably from their competitors. In most cases, all these improvements are essentially complete nonsensical nonsense.

If you stop releasing new versions, you can quickly start losing market share. After all, a competitor will continue to vigorously imitate innovation every month, release versions, press releases, create news hooks, and speak at conferences.

Programmers have to adapt. With the release of each new version, they need to see how it differs from the previous one and update their current project. As a result, a good part of a programmer's time is spent not on solving the task at hand, but on upgrading libraries, tools, services, and fixing all the inconsistencies in integration that arise from this.

And this gets very tiresome quickly.

Can you stop and not update libraries? I do not advise it. Over time, old libraries stop being supported by their manufacturers. First, it becomes impossible to compile the product. And then even the compiled ones stop working. The second aspect: the market demands specialists who can work with modern technologies, not with Algol 68 or Delphi (remember how hyped that technology was?).

Yes, programmers are well paid. But if you consider motivation besides money, employees are also motivated by success: successfully completed projects, and in the short term - successfully closed tasks.

And there is also a big problem with this: if you need to constantly integrate new libraries with which there is no experience, then both the estimation of labor costs and the reliability of the solutions obtained suffer greatly. A programmer is forced to constantly study the features of new libraries, adapt to them. And the fact that it is impossible to predict the outcome of a task in terms of labor costs and reliability is very demotivating. A modern programmer rarely leaves work happy: "I completed my tasks and I did the job well".

This is one of the reasons why programmers burn out very quickly. It seems they are paid well, but the job is complete crap.