DEV Community: Artem Tregub

Building a Basic HTTP Server with Python: A Guide for Automation and Prototyping

Artem Tregub — Mon, 07 Aug 2023 19:40:01 +0000

What is this article about

This article presents a basic example of a Python-based HTTP server. It can be useful for personal needs, whether it be automation or prototyping. Moreover, it can be of use to software test (automation) engineers who wish to mock external services for System Testing or to remove environment instability or dependency.

Thanks to Python and its libraries, starting or editing this server (which is less than 200 lines of code) is straightforward. All the code is available in this repo. Feel free to experiment and use it for your goals. However, I strongly recommend not using it with sensitive information due to the absence of cyber security checks and coverage.

The implementation

The Base HTTP server is a standard library in Python, provided since version 2.7. It offers classes for the server itself and the request handler.

To get started, we'll need to import the necessary libraries and start our server. It's important to maintain the do_GET and do_POST functions as they are. From the outset, we have:

the Server class, which aids in listening to a specific address and port,
the RequestHandler class, tasked with managing server requests via the do_GET and do_POST methods, which will be automatically invoked based on the HTTP method,
the start_server function that acts as a wrapper to initiate our server,
and finally, the main block that includes a standard ArgumentParser, offering basic help and facilitating the required server address and port input.

import argparse
from http.server import BaseHTTPRequestHandler, HTTPServer
from json import dumps


class Server(HTTPServer):
    def __init__(self, address, request_handler):
        super().__init__(address, request_handler)


class RequestHandler(BaseHTTPRequestHandler):
    def __init__(self, request, client_address, server_class):
        self.server_class = server_class
        super().__init__(request, client_address, server_class)

    def do_GET(self):
        response = {"message": "Hello world"}
        self.send_response(200)
        self.send_header("Content-type", "application/json")
        self.send_header("Content-Length", str(len(dumps(response))))
        self.end_headers()
        self.wfile.write(str(response).encode('utf8'))

    def do_POST(self):
        response = {"message": "Hello world"}
        self.send_response(200)
        self.send_header("Content-type", "application/json")
        self.send_header("Content-Length", str(len(dumps(response))))
        self.end_headers()
        self.wfile.write(str(response).encode('utf8'))


def start_server(addr, port, server_class=Server, handler_class=RequestHandler):
    server_address = (addr, port)
    http_server = server_class(server_address, handler_class)
    print(f"Starting server on {addr}:{port}")
    http_server.serve_forever()


def main():
    parser = argparse.ArgumentParser(description="Run a simple HTTP server.")
    parser.add_argument(
        "-l",
        "--listen",
        default="0.0.0.0",
        help="Specify the IP address which server should listen",
    )
    parser.add_argument(
        "-p",
        "--port",
        type=int,
        default=80,
        help="Specify the port which server should listen",
    )
    args = parser.parse_args()
    start_server(addr=args.listen, port=args.port)


if __name__ == "__main__":
    main()

With this setup, we already have a functional HTTP server that will respond to any GET and POST HTTP requests with the json:

{"message": "Hello world"}

Good, but let's make it more useful. To enhance this server, we're going to establish it as a classic "mock server". This requires incorporating various HTTP path handling (including links and methods), data storage, and fundamental server responses.

Next, I suggest we work with json and xml to give a demonstration. It's also generally useful to have a heartbeat method in place. It is a type of function that routinely checks system or network connections to ensure they're operating correctly.

First up, let's add a straightforward class to encapsulate the paths our server will use:

class Path:
    JSON = "/json"
    XML = "/xml"
    HEALTH = "/health"

Basic init data responses:

class Response:
    HEALTH_GOOD_RESPONSE = dumps({"message": "I'm doing great!"})
    DEFAULT_RESPONSE = dumps({"message": "Hello there. This is a default server response."})
    GOOD_RESPONSE = dumps({"message": "I got your data"})
    INIT_JSON_DATA = dumps({"status": True, "data": {"id": "FEFE", "some_field": "some_field_data"}})
    INIT_XML_DATA = '<?xml version="1.0" encoding="utf-8"?>' \
                    '<status>True</status>' \
                    '<data>' \
                    '<id>FEFE</id>' \
                    '<some_field>some_field_data</some_field>' \
                    '</data>'

And storage that will work only during runtime:

class Storage:
    def __init__(self):
        self.json = Response.INIT_JSON_DATA
        self.xml = Response.INIT_XML_DATA

    def write_json(self, data):
        self.json = data

    def read_json(self):
        return self.json

    def write_xml(self, data):
        self.xml = data

    def read_xml(self):
        return self.xml

We then introduce new classes during the Server and RequestHandler initialization to highlight dependency and support future modifications. Additionally, the RequestHandler will have access to this new data.

class Server(HTTPServer):
    def __init__(self, address, request_handler, storage, paths, response):
        super().__init__(address, request_handler)
        self.storage = storage
        self.path = paths
        self.response = response


class RequestHandler(BaseHTTPRequestHandler):
    def __init__(self, request, client_address, server_class):
        self.server_class = server_class
        self.response_json_data = server_class.storage.read_json()
        self.response_xml_data = server_class.storage.read_xml()
        super().__init__(request, client_address, server_class)

Now, we need to adjust our RequestHandler.doGET and RequestHandler.doPOST methods. The logic behind this is straightforward:

check the method type (BaseHTTPRequestHandler will handle this for us),
verify the path that was used in the request,
invoke the requested method or function.

def do_GET(self):
    if self.path == self.server_class.path.JSON:
        return self.json_response()

    if self.path == self.server_class.path.XML:
        return self.xml_response()

    if self.path == self.server_class.path.HEALTH:
        return self.return_health()

    return self.default_response()


def do_POST(self):
    if self.path == self.server_class.path.JSON:
        return self.store_json_test_data()

    if self.path == self.server_class.path.XML:
        return self.store_xml_test_data()

    return self.default_response()

In order to respond with a minimal HTTP data we need to provide:

Data type (specified in the header)
Data length (also in the header)
Response code
Response data You can find more information about HTTP headers here.

def set_json_headers(self, success_response=None):
    self.send_response(200)
    if success_response is not None:
        self.send_header("Content-type", "application/json")
    self.send_header("Content-Length", str(len(success_response)))
    self.end_headers()


def set_response(self, response):
    self.wfile.write(str(response).encode('utf8'))


def return_health(self):
    """
    Implementation for health method and response.
    """
    self.set_json_headers(self.server_class.response.HEALTH_GOOD_RESPONSE)
    self.set_response(self.server_class.response.HEALTH_GOOD_RESPONSE)

This method enables us to get a response on the /health path with the GET method, along with data.

{"message": "I'm doing great!"}

Next, we'll add the rest of the response methods. The logic will be exactly the same as for the /health method above.

def set_xml_headers(self, success_response=None):
    self.send_response(200)
    if success_response is not None:
        self.send_header("Content-type", "text/plain")
        self.send_header("Content-Length", str(len(success_response)))
    self.end_headers()


def default_response(self):
    """
    Implementation for default server response.
    """
    self.set_json_headers(self.server_class.response.DEFAULT_RESPONSE)
    self.set_response(self.server_class.response.DEFAULT_RESPONSE)


def json_response(self):
    """
    Response with json test data stored in runtime.
    """
    self.set_json_headers(self.response_json_data)
    self.set_response(self.response_json_data)


def xml_response(self):
    """
    Response with xml test data stored in runtime.
    """
    self.set_xml_headers(self.response_xml_data)
    self.set_response(self.response_xml_data)

Perhaps the "trickiest" part of this server is the store methods. We want these methods to receive data and store it in runtime, so it can be sent back upon request. This method gives a full response, and additionally, we need to save data to our class Storage variables. Here, it's necessary to parse data length from the Content-Length and read the data itself.

def store_json_test_data(self):
    """
    Store test data in runtime. Use it to upload a new one.
    """
    success_response = self.server_class.response.GOOD_RESPONSE
    self.set_json_headers(success_response)
    self.set_response(success_response)

    if self.headers.get('Content-Length') is not None:
        self.server_class.storage.write_json(self.rfile.read(int(self.headers['Content-Length'])).decode())


def store_xml_test_data(self):
    """
    Store test data in runtime. Use it to upload a new one.
    """
    success_response = self.server_class.response.GOOD_RESPONSE
    self.set_json_headers(success_response)
    self.set_response(success_response)

    if self.headers.get('Content-Length') is not None:
        self.server_class.storage.write_xml(self.rfile.read(int(self.headers['Content-Length'])).decode())

We are now good to go. Launch the server and use the following links to open it in your browser. Doing so will trigger the method execution. Try sending data to it and receiving it back:
http://0.0.0.0/health
http://0.0.0.0/some_path
http://0.0.0.0/json
http://0.0.0.0/xml

Next, let's enhance our default_response and "help" message with a handy feature. I'd like for it to display all available paths in its response data. To achieve this, we need to add a method to our Path class. Fortunately, the Inspect library will do all the heavy lifting for us.
This method examines all class attributes, filters them by "__" symbols, and returns our paths as a list.

@staticmethod
def get_paths():
    return [attr_pair[1] for attr_pair in
            inspect.getmembers(Path, lambda attr_name: not (inspect.isroutine(attr_name))) if
            not (attr_pair[0].startswith('__') and attr_pair[0].endswith('__'))]

Next, we add new code in the DEFAULT_RESPONSE definition to incorporate our new data into the output strings.

DEFAULT_RESPONSE = dumps({"message": "Hello there. This is a default server response. Try valid URLs: {0}".format(
        Path.get_paths())})

And ArgumentParser

parser = argparse.ArgumentParser(description="Run a simple HTTP server. List of available paths: {0}".format(
    Path.get_paths()))

Now we have useful information from the default server response. It guides users to the right paths.
Try http://0.0.0.0/some_path in your browser

{"message": "Hello there. This is a default server response. Try valid URLs: ['/health', '/json', '/xml']"}

At this point, our server is up and running, and easy to use. All the code can be found in the repo with some basic unit tests as a bonus. This should help you safely modify it for your needs.

Priority and Severity of tasks and bugs

Artem Tregub — Mon, 03 Jul 2023 11:55:10 +0000

Priority and Severity of tasks and bugs

Priority and Severity of tasks and bugs identified during the testing life cycle are two essential notions of risk assessment that are often considered to have the same meaning. However, differentiation of these attributes is significant in the development process because allocating them to issues helps effectively determine the order in which release tasks are to be performed. This article will cover defining Priority and Severity, setting Priority and Severity for defects, creating a Priority/Severity matrix and giving some useful tips for implementing the matrix into your project.

Definition of Severity and Priority

Along with other properties such as release or fix version, sprint version, description of system behaviour, etc. of issues in a task tracking system, Priority and Severity are still the most practiced through the release management processes. To clearly understand them, definitions of terms will be given below.

Priority and its levels

Priority is the order in which a bug/task should be resolved. In other words, Priority shows the importance or urgency of fixing defects and implementing issues. This attribute depends on the Severity of the product systems and the business necessities.
Priority levels can be divided as follows:

Low - a defect/task can be fixed last or can not be fixed at all;
Medium - a defect/task should be resolved in the subsequent builds with consideration to the tasks queue;
High - a defect/task must be resolved as a first priority since it blocks overall or affects critical functionality of the product, but it has a temporary solution or workaround;
Blocker - a defect/task must be resolved immediately since it blocks overall or affects critical functionality of the product and it does not have a temporary solution or workaround.

Severity and its levels

Severity is the level of impact of a task/bug on a product. The Severity level indicates to what extent issues and glitches affect the functions, performance or public perception of the product.
Severity can be of following levels:

Low - a defect/task does not have an impact on the functionality or image of the product;
Medium - a defect/task has a small impact on the functionality or image of the product;
High - a defect/task has a significant impact on the functionality or image of the product, but it has a workaround or temporary solution. In addition, this affects the number of end users allowed (a certain percentage);
Blocker - a defect/task blocks overall or affects critical functionality or image of the product badly. It does not have a temporary solution or workaround. Moreover, this affects an unacceptable number of end users.

Priority and Severity setting

Typically, having detected certain bugs through the process of testing, a QA team assigns both Priority and Severity attributes to each defect to indicate the extent to which they affect the project's systems. Since Severity represents an eventual impact, QA engineers identify systems and their functionality affected by the bug, assess the presence of a workaround and evaluate the number of audiences under this effect. It helps determine the Severity level correctly. However, this level can be changed in the assessment procedure by stakeholders because they have an understanding of the system as a whole from a business point of view.
Although the QA team has already set the initial level of Priority during the test cycle, a Product Owner or a Project Manager defines the final order in which the bugs and tasks will be performed in a sprint by a development team.
It could seem that Priority completely depends on the level of Severity, however, Severity is the coefficient which may increase the level of Priority and decrease it as well. The following will show how easy it is to set the Severity for bugs.
Nevertheless, before setting the level it is necessary to recognize which components of the project are crucial from different points of view such as business, legislation, privacy, functionality, data, etc. This list of key systems should be allocated together with stakeholders and it should be fixed in the documentation of the project, for example, in a confluence page where every team member can see information about each component and its level of criticality.
As an example, the following critical systems can be distinguished:

Everything about money (e.g. order confirmation for an online marketplace);
Everything about data (e.g. payment data);
Everything about legislation or legal area (e.g. user agreement);
Any kind of core functionality (e.g. market data collection for brokers);
Functional safety (e.g. airbags for cars);
Cyber security (e.g.personal data).

Easy checklist to set Severity for a bug

There are several questions which should be asked in order to set the level properly:

Does it affect safety or privacy?
Is an impacted system crucial to a product?
Is there a workaround or temporary solution for an issue? Can any user use this solution?
How many users are affected?
Is influence critical to a product's image? Could this negatively affect the product or business?
Does it have an impact on business partners?
Does a problem increase the company's legal risks?

According to the list of declared critical systems and the list of questions, QA specialists can select the appropriate level of Severity for each defect and set it in a task-tracking system along with its Priority.

Priority/Severity matrix

Since the level of Severity influences Priority, the final level of urgency must be adjusted. Before the correction, stakeholders and a tech team should arrange the interconnection of levels of both attributes. Regardless of common levels of Priority and Severity that were mentioned previously, they might have other names than “high”, “low”, etc. and can be named as “crucial” or “trivial” because it directly depends on terminology in your project. However, the most convenient approach is if attributes will have the same name and degrees of criticality because it facilitates comprehension and assignment of the estimate. Moreover, to simplify the application, the definitive level of Priority of tasks or defects creates the matrix of mapping Priority and Severity. An instance of such a matrix is given below.

How to create your own Priority/Severity matrix

The following steps may help facilitate the creation of the matrix for your own project:

Take a default matrix in the first step;
Adapt it to your project Priority levels and names;
Decompose your project to systems;
Define the list of critical systems of your project.

Firstly, according to this approach, taking and adapting an already finished matrix allows you to focus on filling the list of urgency which is associated with your project instead of generating ideas about the representation form of the matrix.
Secondly, in cooperation with development and QA teams, you should determine all systems that are included in the project. Last but not least, note the most crucial components which require scrutiny in the processes of development and testing.
After implementing these steps, you get the matrix which reflects the dependence of Priority and Severity and the catalogue of the most important parts of your product. Both received artifacts allow your team to set proper levels of urgency for tasks and bugs correctly and will act as an argument in case of disagreements with stakeholders when choosing one or another level.

How to implement a new process to your project

Here is the way to include a new process for working with the matrix of Priority and Severity in your project:

Present the concept to your project manager. Your management should realize how a new process might improve the estimation of tasks and defects during a release procedure and facilitate interconnection with tech teams, especially, in cases of disputes about the priority of implementation issues or fixing bugs.
Show an adapted matrix and the list of critical systems for your project. It allows you to arrange with the management a choice of approach to evaluation and accept the scroll of components which will be under control.
Pursue the goal of improving bug/tasks management. Point specifically to the fact that this process will be in your area of responsibility. It is important to show your management that you will lead and keep under control the enhancement of an evaluation process in your team.
Discuss. Find an optimal formula for your project together. Frequently, any changes in the process face resistance from management and other team members. It requires time and negotiations to come to a common point of view on the process and accomplish success in building a new assessment procedure.
Create a wiki/confluence page to consolidate a final formula. Fixing agreements is a significant point that allows your team to avoid violations of the evaluation process in the future.
Create a Severity field in your task tracker system for your project. For example, Jira does not have a Severity field in basic settings and you should ask a relevant department to set this attribute and its values in the task tracker.

Priority and Severity are attributes on which the assessment of the impact of tasks and defects on the product is based. As was already mentioned, Priority is the position of an issue in the queue, while Severity affects Priority and may decrease or increase precedence and correct it. Furthermore, the Priority/Severity matrix is an instrument to manage bugs and tasks arrangement. This is especially useful for a project with plenty of bugs and tasks, and in this sense, understanding and the ability to arrange and incorporate the process suitable for your particular project is essential.