DEV Community: Daniel Medina

My findings for a pytest-django warning - Error when trying to teardown test databases

Daniel Medina — Mon, 08 Nov 2021 15:31:23 +0000

Background

I found myself having a bit of free time on Friday the 5th so I thought it would be helpful to my team to try and decrease the amount of warnings we received after running tox and having all tests pass.

Here was my environment:

Ubuntu - 20.04 LTS
django - 3.2.8
pytest - 6.2.5
pytest-django - 4.4.0
tox - 3.24.4

Warning and Resolution

I received a very strange / obscure warning whenever test databases would be torn down. The warning in full was:

Error when trying to teardown test databases: TypeError('object.__new__() takes exactly one argument (the type to instantiate)')

I ended up resolving this warning by:

Navigating to my pytest.ini file in my base directory.
On the addopts line, adding two switches to the end: --reuse-db --create-db .

The final version of this line looked like this:

[pytest]
// some other settings
addopts = [additional options here] --reuse-db --create-db

Findings

The source of this warning is from pytest-django's fixtures.py file. It can be found here on GitHub . The function being called is teardown_database which is defined on line 126.

According to pytest-django's official documentation, using the reuse-db and create-db switches together helps isolate and recreate databases throughout tests.

--reuse-db will not pick up schema changes between test runs. You must run the tests with --reuse-db --create-db to re-create the database according to the new schema. Running without --reuse-db is also possible, since the database will automatically be re-created.

(source: pytest-django docs)

My guess is that somewhere during the tests, not all of the test databases are properly created. This would result in pytest-django trying to tear down something that isn't there. This would also explain why it is looking for an object that can't be used when it is calling teardown_database .

Finally, in my time spent searching for a solution, I found some similar pages online. While they weren't directly related to my problem and solution, they may be helpful in forming a trail of breadcrumbs towards solving a different problem:

SO - Why is pytest throwing an "Attribute Error: 'NoneType' object has no attribute..."
GitHub - Ignoring migrations during Django testing (unmanaged databases, legacy databases, etc)

References and Special Thanks

I want to specifically thank Cris Ewing for their article on Medium . I relied heavily on their troubleshooting and debugging methodology. Without their post, I would never have found a solution or experimented using ideas from the official pytest-django documentation. Thank you Cris!

I also wish to thank Stefaan Lippens for their short blog post on how to disable PyTest's default log capturing. It was very useful as I was debugging. It can be found here .

References:

pytest-django documentation --reuse-db reuse the test database between runs
pytest-django source code

How to install the psycopg2 Python package in WSL 2

Daniel Medina — Fri, 30 Jul 2021 17:51:13 +0000

This post is a quick note to all about how to install psycopg2 in Linux. I used WSL 2 running Ubuntu 20.04.

Background

I was trying to set up a toy project to learn Django using PostgreSQL. When I tried to install the recommend psycopg2 package from pip, I received two error messages.

The first error I received was: error: invalid command 'bdist_wheel' . My second error was related to C / C++ and was in my log files:

In file included from psycopg/adapter_asis.c:28:

./psycopg/psycopg.h:35:10: fatal error: Python.h: No such file or directory
  35  |  #include <Python.h>
      |           ^~~~~~~~~~

compilation terminated.

The Steps

I installed the following dependencies: python-dev , python3-dev.
I installed wheels using pip3 install wheel .
I installed python[your version]-dev using Ubuntu's sudo command.

Explanation

The root cause of the problem was that psycopg2 was looking for headers for Python 3.8.5. Since I am using Python 3.9.5, the headers could not be located and psycopg2 could not install. After installing python3.9-dev , psycopg2 successfully installed on my system.

I hope that this short post saves people time and energy.

Additional references:

SO - Invalid command bdist wheel on Travis CI
SO - How to fix error: invalid command 'bdist_wheel'

Node.js server fails to render pages

Daniel Medina — Sat, 01 Feb 2020 23:57:00 +0000

Hello everyone! I am trying to render two pages in a simple toy application to better understand Node.js and TypeScript. The index page is supposed to display a simple message when going to localhost:3000 or / while the 404 page is supposed to display an error message whenever an invalid URL is entered.

Error Message

TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be one of type string, Buffer, or URL. Received type undefined at Object.open (fs.js:406:10)

Summary

The two issues I am encountering are properly using the fs module's readStream() method and absolute vs relative directory paths.

What I've already tried so far

I've done a lot of Googling and found a lot of StackOverflow posts. The majority that I have read are either relevant to other modules or are using readFile(). The only thing I've taken away from my searches is to use Node's path module for resolving directory / file names.

Additionally, I have tried using console.log() to log my requests to the server and the path that path.join() is reading.

Here is my project structure:

server_html (top level aka root directory)
-- node_modules
-- src
↳ index.ts
-- views
↳ 404.html
↳ index.html
package-lock.json
package.json
prettier.config.json
tsconfig.json
tslint.json

Code Snippets

Here are my imports, status codes, route map, and port:

import * as http from 'http';
import * as fs from 'fs';
import * as path from 'path';

const httpOkStatus: number = 200;
const httpNotFoundStatus: number = 404;

const routeMap: any = {
    "/": path.join(__dirname, "..", "views", "index.html"),
    "/index": path.join(__dirname, "..", "views", "index.html")
};

const port: number = 3000;

Here's the code for my server:

const app: http.Server = http
    .createServer((req: http.IncomingMessage, res: http.ServerResponse) => {

        res.writeHead(httpOkStatus, {
            "Content-Type": "text/html"
        });

        const pathToIndexView: string = routeMap[req.url as string];
        const sendToErrorPage: boolean = !routeMap[req.url as string];

        if (sendToErrorPage) {
            res.writeHead(httpNotFoundStatus, {
                "Content-Type": "text/html"
            });
            let errorPageFileStream: fs.ReadStream = fs.createReadStream(path.join(__dirname, "..", "views", "404.html") as string, "utf8");
            errorPageFileStream.pipe(res);
            res.end();
        }
        const src: fs.ReadStream = fs.createReadStream(pathToIndexView, "utf8");

        src.pipe(res);
        res.end();
    });

app.listen(port);
console.log(`The server has started and is listening on port number: ${port}`);

FYI, my Index View is boilerplate HTML with a simple <h1>Welcome!</h1>.

Since the error is coming from Object.open(fs.js:406:10), my guess is that this error is happening within one of the fs.createReadStream() methods. The questions I have are:

Why is the path to my Error page a type of undefined?

Why is my index page not rendering my message of "Welcome!"?

Here are my logged paths for both my error page and my index page:

Error:  C:\Users\Daniel\Documents\TypeScript\serve_html\views\404.html
Index:  C:\Users\Daniel\Documents\TypeScript\serve_html\views\index.html

Next Steps

I am going to go through Node's official API documentation to see what I can learn about the fs, path, and http modules.
I may need to dig into the network traffic to see where my requests are going to. I've already looked at some calls in Chrome's Network tab, but I am considering using separate tools to see all data paths.

I'm new to Node.js and would appreciate even glances at this problem. In the meantime, I will keep searching to find answers and / or stumble onto a working version of this.