DEV Community: toonarmycaptain

Unit testing with assertive mocks vs not testing implementation: Why not both?

toonarmycaptain — Tue, 30 Mar 2021 03:47:26 +0000

Often I will have a function that contains another function call, and want to write a unit test for it.

"""my_module"""

def function_under_test(arg):
    # do stuff
    var = called_function(arg2)
    # do stuff
    return something

The typical pytest way of testing function_under_test as a unit (rather than function_under_test and called_function) would be to mock out/monkeypatch called_function:

def test_function_under_test():

    def mock_called_function(*args):
        # I usually assign expected_args/return value 
        # based on test inputs/parametrize
        assert args = expected_args
        return appropriate_test_value

    monkeypatch.setattr(module_containing_called_function, "called_function", mock_called_function)

    assert function_under_test(arg) == expected_result

Now often I want to mock out the function call, as it might be writing to file, making network calls etc.
But this means if I change how I'm calling called_function, or the internals of called_function, the test breaks, even if the arguments to function_under_test and it's return value are exactly the same. So in a sense, it's not a pure unit test unless you get rid of the assert in the mock, it's still halfway to an integration test, provided your mock is accurate, because it cares what it was called by (although it can be simpler in terms of defining mocks to put a small amount of logic there, rather than specifying for each test case).

I have found (and it might just be me), that even when I run integration tests and pure(r) unit tests, I diagnose errors more easily when I have the assertions in my mocked function calls, because then it's easy to nail down the source of the error.
Yes, it can be a lot more work, because you might have to make changes to tests when you change the implementation (although you might have to anyway if the mocked function's return value would be different...).

But what if you simply did both?

@pytest.mark.parametrize[<sup>1</sup>](#footnote-1)('mock_assertions', [True, False])
def test_function_under_test(mock_assertions):

    def mock_called_function(*args):
        if mock_assertions:
            assert args = expected_args
        return appropriate_test_value

    monkeypatch.setattr(module_containing_called_function, "called_function", mock_called_function)

    assert function_under_test(arg) == expected_result

This way, you get instant feedback if you changed the behaviour of function_under_test, but you also get feedback if you changed the calls you might make to called_function in ways you did not expect. Sure you can make mistakes in your mock_called_function args and return values...but at least you need to think about them, particularly if the calls/needed return values change.

Integration tests are still necessary, because whatever you say called_function returns in your mocks, that might not be what it actually returns.
Yet I wonder if, despite some extra work to maintain, testing this way gets the best of a couple of worlds, helping us better pin down that our functions do what we expect, without flawed tests that pass because we returned what we'd want, rather than what we'd get, and helping us pinpoint why our integration tests failed (if we have them, and if they have the same exploration of edge cases as our unit tests).

Or is it better to have the integration tests and pure(r) unit tests, even if the integration tests force you to mock expensive/disk/network calls anyway?

What do you think?

¹ Do you use @pytest.mark.parametrize? It lets you specify a bunch of different inputs to your tests, and pass/fail the test run on each input. It is excellent.

Dropping local test runtime and scope with pytest

toonarmycaptain — Thu, 25 Feb 2021 20:26:31 +0000

I love pytest.

I have a project that I use as something of a learning platform (and that I don't want to break because it's at least occasionally used in production by my wife..). Most recently I've used it to learn how to abstract a database layer, implement a SQLite database, and now I'm similarly implementing SQLAlchemy.

Database tests can take a...while to run.
The cross product of a test suite and a variety of database backends via pytest.mark.parametrize takes...longer.

The other day I googled "pytest only run specific parameters" and happened across the python -m pytest -k some_string invocation.

Now, I see that the documentation refers to strings in quotation marks,¹ but I missed that, and my incovation python -m pytest -k sqlalchemy works just fine cut my test runtime down to 1/6th the full suite, running only tests specific to my SQLiteSQLAlchemyDatabase object in sqlite_sqlalchemy.py, and my common tests that run against every database backend, but only the parameters using my empty_sqlite_sqlalchemy_database fixture.²

So while I'm working on this backend, and assuming I'm not changing the database interface, I can run the full suite of tests, but only on the part of the code I'm working on. This way, I can ensure that the implementation works (or is getting closer to working), and that it's not breaking any thing that relies on it (because my common interface tests are running, and that one test for another component I would have forgotten that uses this code) as I go, without waiting for the entirety of the project's tests to run!

This is not only saving me time waiting, but means I'm not just running the tests more often, but receiving the feedback much closer to the writing of the code. You know, I'm actually checking the results of the tests before writing much/any subsequent code, that generally relies on the former actually working.

¹ Later I wondered how run my SQLiteDatabase tests without also catching my SQLiteSQLAlchemyDatabase tests (which is what would happen by using sqlite). The answer was to use a string with logic: python -m pytest -k "sqlite and not sqlalchemy". More options and ideas: SO: parametrize and running a single test in pytest, SO: Pytest select tests based on mark.parameterize value?, Working with custom markers.
² I should write a post about how I use fixtures as parameters in pytest.mark.parametrize.

What you need to learn that's not writing code.

toonarmycaptain — Fri, 04 Dec 2020 07:27:33 +0000

A mate told me they're trying to pick up coding. While I'm not yet employed with my skills, I thought I'd write down my thoughts about what they need to learn.

[They're planning on learning HTML/CSS/JS and Python for front end development, and have started with Python.]

What do they need to learn?

Version control - Git/GitHub

Use version control, preferably externally hosted. Git hosted on Github is the ubiquitous place to start.

It can be a bit of a head trip when you first try it, but at least get the basics of making a repo, cloning it to your machine, .gitignore, adding/tracking files, committing changes in sensible portions regularly, pushing, and the rest will come.

The rest being branching, making/merging PRs (Pull Requests), reset, and the-rest-you-will-likely-google-when-you-need-them-the-first-few-times-and-even-then-still-google-to-be-sure.

Seeing what files are tracked/ignored and what changes are committed is much easier when starting out with some sort of GUI, like an...
IDE - Pycharm

Become proficient in at least one full-featured IDE, and it's features.

PyCharm is a good one for python, and integrates with the rest of these well, and has useful features for HTML/CSS/JS.

Some call this cheating, but I believe learning how to use an IDE well is worthwhile both for productivity and understanding what is going on in the code you're writing.
The debugger in whatever language you're using is also a great skill, but when you're beginning, the GUI debugger in most IDEs is a good place to start.
Some testing skills - pytest

Whether or not you decide to go hard into TDD (Test Driven Development), as long as you write tests around the time you write the code, or figure out what you need the code to be doing/not doing, you'll be doing yourself a favour.

In tandem with git and automated testing, this will save you from your own assumptions, and force you to learn the basics of extremely useful tools (eg TravisCI, Appveyor).

Pytest, compatible with python's builtin unittest, and has lots of plugins for testing most things you'd care to test.
*While technically writing code, the concepts, practise, skill set of writing tests are separate from writing production code.
Documenting your code

When you go back in a few months and wonder what you were thinking...this is your chance to help yourself fix the mess you created when you had no idea what you were doing (for me this is generally yesterday). This includes comments as well as more formal documentation (ie package/module/class/function/method docstrings, README.md).
This also includes git commit messages that make enough sense for you (and preferably others) to follow what changes are being made.

Python has the helpful PEP 8 and PEP 257 general guides.
Learning practices

Keeping a journal or record of the things you're learning, or even a blog, are great practices. Even if it's as simple as text document you can search in, recording the things you learn and figure out gives you a reference later ("how did I do that obscure weird thing that other time and why") is invaluable. This can also function as a record of what you've learned and grown, for yourself, and to potentially to draw on to give current or future employers specific evidence of growth and experience.

Writing in public about what you're learning or doing also seems good advice; it's why I'm here; and the process of distilling knowledge for communicating or teaching others is a boon to your personal learning, if for no other reason than the gaps in your knowledge will be filled when you double-check your facts (I mean...research), or get corrected publicly. Which, if you've any humility, is more good than bad!

Speaking of which, please let me know what I'm missing, or wrong about!

I'll note that these are habits that will grow into well-developed skills when they're practiced as part of a workflow, rather than as an afterthought.
Or at least, I hope they will...

There are a few of almost-without-saying skills that are independent of any technology, but worth mentioning:

Problem solving, and a rough personal methodology, both when debugging code, and figuring out how you need to solve a problem (or, first, what the problem is).
...and the tool to find the details: Google and/or StackOverflow.
Learn to find, and how to navigate, the documentation for the technologies you're using.

Getting things done. Or, "how to solve problems".

toonarmycaptain — Tue, 24 Nov 2020 06:29:31 +0000

It's very easy for me to set myself tasks. Build that website. Write a blog post. Clean the house. Learn about that library or a new technology. Get a degree. Get the outside of the house ready for winter. Find a job. So easy. There, done.

Or...not?

One of the problems with this way of (not) planning is that it sets tasks or identifies problems, without a map to to a solution.

Breaking tasks down

One of the things I've learned to do deliberately since I've been coding is to break tasks down into their component elements. I've found myself doing this away from the keyboard, and it's been very, very helpful, particularly when getting motivated to tackle tasks I'm unlikely to complete in one session. Like the mountain of laundry that existed yesterday:

find place for baby clothes
sort into piles
think about ironing (then decide to put this off/not)
put away parents clothes
put away kids clothes
figure out organisation in spot for baby clothes
put away baby clothes
iron clothes (or not)

Recently for me this involved making and posting my first blog entry last week, and breaking down "make incomplete site presentable and deploy":

point site blog link at this blog
clean up CSS
remove "content coming soon" from home/landing and projects pages
bare bones projects content with links, minimal explanation from GitHub descriptions.
fix/insert titles on subpages
deploy at pythonanywhere.com/toonarmycaptain

This skill is really important to a lot of things, from isolating bugs by breaking down the possible causes, to writing simple and maintainable code by following the adage to keep functions/classes to one purpose. Of course, it's also helpful when attacking large projects. :)

Hello world!

toonarmycaptain — Tue, 17 Nov 2020 05:58:36 +0000

I've been planning on writing this for ... awhile.

There's a lot of advice around regarding getting into tech suggesting blogging and building your own website. I'm working on the latter.
On the former, I'm finally following the part of the advice that says "just start writing" and "use other services rather than building a blog yourself". So here we are.

I've been teaching myself python for a few years now on the side, recently transitioned (back) to a stay at home dad with another newborn, so my time is limited, but my goal is to get into a paid tech role in the next year or so.

In the shorter term, I plan to:

Have my embryonic personal site hosted on pythonanywhere. ✓
Implement recaptcha on my contact form.
Delegate contact form submissions to async functions. or/and celery, implement rate limiting.
Link to this blog from my website. ✓
Add social/github links to site.
Manage posts here from github
Migrate to hosting at Heroku, AWS, Digital Ocean, or similar.

My intention is to write about these things. Not that there's not great posts about them already around, but maybe my perspective can help someone.
I will likely also heavily edit this post. But that's the fun of being a picky writer and maintaining a blog.

DEV Community: toonarmycaptain

Unit testing with assertive mocks vs not testing implementation: Why not both?

Dropping local test runtime and scope with pytest

What you need to learn that's not writing code.

Version control - Git/GitHub

IDE - Pycharm

Some testing skills - pytest

Documenting your code

Learning practices

Speaking of which, please let me know what I'm missing, or wrong about!

Getting things done. Or, "how to solve problems".

Breaking tasks down

Hello world!