DEV Community: Mickey Streicher

Vedro Hooks

Mickey Streicher — Thu, 28 Nov 2024 17:48:32 +0000

Vedro offers powerful extensibility through its plugin system, allowing you to create robust, reusable solutions that can be shared across different projects and teams. But what if you're just experimenting with your codebase, prototyping a concept, or adding a small tweak? Writing a full plugin might feel like overkill. That’s where vedro-hooks comes in.

vedro-hooks is a lightweight library that lets you attach custom hooks to various Vedro events. Whether you're starting a mock server before tests run, launching a browser for end-to-end testing or setting up custom logging, vedro-hooks enables you to inject functionality with minimal boilerplate.

A Practical Example

Suppose you want to identify slow tests in your suite — let's define "slow" as any test that takes longer than 1 second to run. Traditionally, you’d need to create a custom plugin for this. Here’s how that might look:

from vedro.core import Dispatcher, Plugin, PluginConfig
from vedro.events import ScenarioFailedEvent, ScenarioPassedEvent

class SlowTestPlugin(Plugin):
    def subscribe(self, dispatcher: Dispatcher):
        dispatcher.listen(ScenarioPassedEvent, self.on_scenario_end)
        dispatcher.listen(ScenarioFailedEvent, self.on_scenario_end)

    def on_scenario_end(self, event: ScenarioPassedEvent | ScenarioFailedEvent):
        elapsed = event.scenario_result.elapsed
        if elapsed > 1.0:
            event.scenario_result.add_extra_details("⚠️ Slow test!")

class SlowTestPluginConfig(PluginConfig):
    plugin = SlowTestPlugin

This approach works, but creating a full-fledged plugin involves more setup and additional boilerplate. It’s great for reusable solutions but can feel cumbersome for quick experimentation.

Simplifying with Hooks

With vedro-hooks, you can achieve the same functionality with just a few lines of code:

from vedro_hooks import on_scenario_passed, on_scenario_failed

@on_scenario_passed
@on_scenario_failed
def highlight_slow_tests(event):
    elapsed = event.scenario_result.elapsed
    if elapsed > 1.0:
        event.scenario_result.add_extra_details("⚠️ Slow test!")

This code uses decorators to register a function that will be called when a scenario passes or fails. It checks the elapsed time and adds extra details if the scenario took longer than 1 second.

Scenarios
*
 ✔ retrieve user info (0.52s)
 ✔ retrieve user repos (1.02s)
   |> ⚠️ Slow test!

# 2 scenarios, 2 passed, 0 failed, 0 skipped (1.54s)

Managing Hooks: Downsides and Solutions

One downside of using hooks in this way is that they can be registered from anywhere in your project, which might make them harder to track down later. In contrast, plugins in Vedro are registered in the vedro.cfg.py file, providing a centralized location for all your plugin configurations.

To help mitigate the downside of hooks being registered throughout your codebase, vedro-hooks provides the --hooks-show command-line argument. When enabled, after the testing process completes, a summary of all registered hooks along with their source locations will be displayed. This is useful for debugging and verifying which hooks are active.

Scenarios
*
 ✔ retrieve user repos

# [vedro-hooks] Hooks:
#  - 'highlight_slow_tests' (ScenarioFailedEvent) vedro.cfg.py:26
#  - 'highlight_slow_tests' (ScenarioPassedEvent) vedro.cfg.py:26
# 1 scenario, 1 passed, 0 failed, 0 skipped (0.55s)

While --hooks-show is helpful, you need to remember to use it during debugging. It's still best practice to register your hooks in a central location like vedro.cfg.py to maintain clarity and consistency with plugins configurations.

Conclusion

vedro-hooks is a fantastic tool for enhancing your Vedro tests without the overhead of creating a custom plugin. It shines when you need a quick, focused solution for extending functionality. By using it wisely and keeping your configuration organized, you can enjoy the best of both worlds: simplicity and maintainability.

Enhancing Merge Requests with Vedro and GitLab Test Reports

Mickey Streicher — Sat, 06 Jan 2024 09:59:51 +0000

As more development teams adopt Continuous Integration (CI) practices, the ability to visualize and manage test results within merge requests becomes increasingly important. GitLab CI provides developers with a user-friendly platform to view test reports directly in the context of their changes, streamlining the workflow and speeding up the review process.

When configured correctly, your GitLab merge request will display a new section illustrating the test report, showcasing which tests passed or failed. This section makes it incredibly easy for reviewers to assess the impact of code changes on the existing codebase, all without leaving the merge request view. The integration appears as follows:

Full Report:

Setting Up GitLab CI Test Reports

Step 1: Installing vedro-xunit-reporter

Before generating test reports, you need to install the vedro-xunit-reporter plugin. This plugin will produce test reports in the xUnit XML format, which GitLab CI can interpret and display within merge requests.

To install the plugin, run the following command:

$ vedro plugin install vedro-xunit-reporter

After installing the plugin, remember to add it to your requirements.txt file, ensuring its installation during CI pipeline executions.

Step 2: Configuring .gitlab-ci.yml

The next step involves updating your .gitlab-ci.yml file to define how artifacts and test reports are handled within your GitLab CI pipeline. You will need to specify that the artifact produced by the test stage is a report in the xUnit XML format, and indicate the path where this report can be found.

Here is an example configuration for your .gitlab-ci.yml file:

test:
  image: python:3.11
  stage: test

  before_script:
    - pip install -r requirements.txt

  script:
    - vedro run -r rich xunit --xunit-report-path xunit_report.xml

  artifacts:
    when: always
    reports:
      junit: xunit_report.xml

In the example above, the CI pipeline includes a test stage, running Vedro with the rich and xunit reporters. The --xunit-report-path option specifies the output filename for the test report, which, in this case, is xunit_report.xml.

Additionally, the artifacts:reports:junit section instructs GitLab CI to treat the generated xunit_report.xml file as a JUnit format test report. when: always ensures that the report is saved as an artifact regardless of whether the test passes or fails.

By following these two steps, you can initiate Vedro tests within your GitLab CI pipeline and visually represent the results in merge requests. This integration enhances the efficiency of code reviews by providing immediate and clear feedback on the success of tests related to the proposed code changes. It also helps to catch issues early, prior to merging your code, thus maintaining the stability and reliability of your code base.

Vedro: How To Define Environment Variables

Mickey Streicher — Sat, 25 Nov 2023 12:43:32 +0000

Environment variables are key-value pairs used in software development to store configuration settings and control application behavior without hardcoding them into the source code. They facilitate managing different configurations across various environments, like local development or staging. This approach is essential for maintaining clean, adaptable code and securely handling sensitive information, such as API keys.

In automated testing, environment variables facilitate seamless transitions between various settings or data sources, simplifying the testing process across multiple deployment environments.

Scenario: Retrieving a List of Asteroids

To illustrate, consider a Vedro test scenario that requires loading an API key from an environment variable to access an endpoint providing a list of asteroids. For this example, the focus will be on the NeoWs (Near Earth Object Web Service) provided by NASA, which offers comprehensive data about near-Earth asteroids.

Here's the basic structure of a Vedro test scenario:

import httpx
import vedro

class Scenario(vedro.Scenario):
    subject = "retrieve list of asteroids"

    def given_api_key(self):
        # API key will be loaded from environment variable
        self.api_key = <load from env>

    def when_user_retrieves_asteroids(self):
        # Make a GET request to the NeoWs API
        self.response = httpx.get("https://api.nasa.gov/neo/rest/v1/feed", params={
            "api_key": self.api_key
        })

    def then_it_should_return_success_response(self):
        # Assert the success of the response
        assert self.response.status_code == 200

Reading Environment Variables in Python

To read environment variables in Python, you can use the os module. Here's how to modify the given_api_key method to load the API key from an environment variable:

from os import environ

class Scenario(vedro.Scenario):
    subject = "retrieve list of asteroids"

    def given_api_key(self):
        self.api_key = environ.get("API_KEY")

    ...

Defining Environment Variables

There are two primary ways to define environment variables for tests.

Method 1: Directly in the Environment

Define environment variables directly in the shell:

$ export API_KEY="DEMO_KEY"

Then run tests:

$ vedro run
Scenarios
*
 ✔ retrieve list of asteroids

# 1 scenario, 1 passed, 0 failed, 0 skipped (1.79s)

Method 2: Using a .env File

Alternatively, you can use a .env file:

API_KEY="DEMO_KEY"

Install the python-dotenv package to load variables from the .env file:

$ pip3 install python-dotenv

Then, load the environment variables in the vedro.cfg.py:

import vedro
import vedro_allure_reporter
from dotenv import load_dotenv

load_dotenv(".env")


class Config(vedro.Config):

    class Plugins(vedro.Config.Plugins):

        class AllureReporter(vedro_allure_reporter.AllureReporter):
            enabled = True

The configuration above includes the Allure reporter plugin for Vedro. This plugin generates comprehensive, interactive reports, enhancing test result visualization. For detailed integration instructions, refer to the blog post: How to Integrate Vedro with Allure.

Run tests again:

$ vedro run
Scenarios
*
 ✔ retrieve list of asteroids

# 1 scenario, 1 passed, 0 failed, 0 skipped (1.61s)

A significant advantage of using .env files in testing scenarios is their flexibility in managing multiple environment variables across different testing environments. For instance, separate env-files like .env-local for local development and .env-staging for staging environments can be used. This approach enables developers and testers to easily switch contexts without altering the code or the command line environment.

Regardless of the chosen method, environment variables can be utilized anywhere in your tests: within the test itself, in contexts, or in a separate configuration file.

How to Integrate Vedro with Allure

Mickey Streicher — Sat, 09 Sep 2023 14:07:49 +0000

In modern software development, a comprehensive reporting tool is as crucial as the test cases themselves. Allure has emerged as a highly flexible and insightful reporting tool that provides an in-depth look into what has been tested and what needs further attention. If you're using Vedro, a pragmatic testing framework, integrating it with Allure can significantly enhance your test reporting capabilities.

Laying the Groundwork

Before delving into the integration, ensure that you have Vedro installed:

pip install vedro

Consider the following simple Vedro scenario, which tests the retrieval of repositories for a GitHub user:

import vedro
import httpx

class Scenario(vedro.Scenario):
    subject = "retrieve user repos"

    def given_user(self):
        self.user = "gvanrossum"

    def when_guest_retrieves_repos(self):
        self.response = httpx.get(f"https://api.github.com/users/{self.user}/repos")

    def then_it_should_return_a_successful_response(self):
        assert self.response.status_code == 200

Next, let's integrate this with Allure.

Step 1: Install the Allure Plugin for Vedro

To get started with Allure in Vedro, first install the Allure Reporter plugin:

vedro plugin install vedro-allure-reporter

Step 2: Execute Tests and Generate Report Data

After installing the plugin, you can execute your tests and generate Allure report data:

vedro run -r rich allure

By default, this command saves the report data in the ./allure_reports directory. To specify a different directory, use:

vedro run -r rich allure --allure-report-dir ./custom_allure_reports

Step 3: Visualize the Report with the Allure CLI

To view the report, you'll need to install the Allure command-line tool first. Follow the installation instructions provided in the official Allure guide.

After you've installed the Allure CLI, serve the report:

allure serve ./allure_reports

This command will generate and open the report in your default web browser, providing a clear and interactive view of your test results.

Enhance Reports Through Categorization and Labeling

To improve report visualization and understanding, consider labeling your tests. For example, label the previously defined scenario to categorize it under "GitHub API Testing":

import vedro
from vedro_allure_reporter import allure_labels, Story, Epic, Feature

@allure_labels(Epic("GitHub API Testing"), Feature("User Repositories"))
class Scenario(vedro.Scenario):
    subject = "retrieve user repos"

    ...

Labeling is particularly useful when you have an extensive test suite and need to filter or group tests. For example, to run tests labeled under a specific epic, use:

vedro run --allure-labels epic="GitHub API Testing"

In summary, integrating Vedro with Allure not only enhances your testing workflow but also provides a comprehensive and interactive reporting experience, making it easier to track, filter, and understand your test results.