💪 UNBREAKABLE Code with ChatGPT!

TL;DR Video

If you're not using ChatGPT yet as a programmer, YOU ARE MISSING OUT!

I'll show you one of my favorite tricks for using ChatGPT: Using ChatGPT to write property-based tests!

Property-based tests, A.K.A PBTs, allow you to run hundreds of unit tests with just a single PBT. It does this by making you define a property of a function, then runs it with the full range of your inputs, and if your function fails the test, a PBT would find the smallest value that would fail that function.

Theoretically, if you were able to test all the properties of a function, then your function is now UNBREAKABLE! This is because the properties of your functions is the definition of your function: if your definition is perfect, then the function is perfect!

Oooookay so I think that's a bit confusing so let me show you an example instead!

Suppose we have an addition function:

def add(a, b):
  return a + b

How we'd usually test this is do something like:

def test_add_1(a, b):
  assert add(1, 1) = 2

But what if a weird dev used this implementation instead?

def add(a, b):
  return 2

Obviously this would pass the test, but would fail in other instances.

This is where unit tests fail. You can't possibly manually write every possible test for every possible input. With property-based tests, you can achieve this by testing the function properties themselves.

What you can do instead is to test the properties with Property-based Tests.

One of the properties of addition is the identity property. Whatever number you add to zero, it will return that number.

To test for that we can do something like this:

@given(st.floats())
def test_add_identity(a)
  assert add(a, 0) == a

---

What this will do is that for all float values, it will test if adding that to 0 will always return itself. With the weird implementation, if a is 2 then it works, but then the PBT will continue testing for other values until it encounters an issue.

Now this obviously still has loopholes, but if you manage to test for all properties, like the commutative, associative, etc. then you'll have an unbreakable function.

---

Now we understand how powerful these tests are. Let's use ChatGPT to do this for a more complex example!

Let's use this naive Pearson r implementation:

def pearson_correlation_coefficient(x_values, y_values):
    x_mean = sum(x_values) / len(x_values)
    y_mean = sum(y_values) / len(y_values)

    covariance = sum((x - x_mean) * (y - y_mean) for x, y in zip(x_values, y_values)) / len(x_values)
    stddev_x = (sum((x - x_mean) ** 2 for x in x_values) / len(x_values)) ** 0.5
    stddev_y = (sum((y - y_mean) ** 2 for y in y_values) / len(y_values)) ** 0.5

    return covariance / (stddev_x * stddev_y)

This has glaringly obvious bugs but assume that you don't know for now.
To write Property-based tests for these, we can do use this ChatGPT prompt:

Create property-based tests for the following code. Please be comprehensive and test all properties and cases, especially those that will make the function fail. Make sure to thoroughly document the properties being tested using docstrings. Use library as test library.

Code:
insert code

Together with the function we have, the complete prompt will look something like this:

This prompt generated these property-based tests for me:

import pytest
from hypothesis import given, strategies as st

@pytest.fixture()
def pearson_corr():
    return pearson_correlation_coefficient

@given(st.lists(st.floats()), st.lists(st.floats()))
def test_pearson_correlation_coefficient_valid_input(pearson_corr, x_values, y_values):
    """
    Test the pearson_correlation_coefficient function with valid input.
    It should return a value between -1 and 1, inclusive.
    """
    if len(x_values) < 2 or len(y_values) < 2:
        with pytest.raises(ZeroDivisionError):
            pearson_corr(x_values, y_values)
    else:
        result = pearson_corr(x_values, y_values)
        assert -1 <= result <= 1

@given(st.lists(st.floats(min_value=-100, max_value=100), min_size=2), st.lists(st.floats(min_value=-100, max_value=100), min_size=2))
def test_pearson_correlation_coefficient_identical_inputs(pearson_corr, x_values, y_values):
    """
    Test the pearson_correlation_coefficient function with identical inputs.
    It should return 1.
    """
    result = pearson_corr(x_values, x_values)
    assert result == 1

@given(st.lists(st.floats(min_value=-100, max_value=100), min_size=2), st.lists(st.floats(min_value=-100, max_value=100), min_size=2))
def test_pearson_correlation_coefficient_negative_identical_inputs(pearson_corr, x_values, y_values):
    """
    Test the pearson_correlation_coefficient function with identical inputs but with one list negated.
    It should return -1.
    """
    y_values_neg = [-y for y in y_values]
    result = pearson_corr(x_values, y_values_neg)
    assert result == -1

@given(st.lists(st.floats(min_value=-100, max_value=100), min_size=2), st.lists(st.floats(min_value=-100, max_value=100), min_size=2))
def test_pearson_correlation_coefficient_orthogonal_inputs(pearson_corr, x_values, y_values):
    """
    Test the pearson_correlation_coefficient function with orthogonal inputs.
    It should return 0.
    """
    x_values_orth = [y for y in y_values]
    result = pearson_corr(x_values, x_values_orth)
    assert result == 0

@given(st.lists(st.floats(), min_size=1), st.lists(st.floats(), min_size=1))
def test_pearson_correlation_coefficient_zero_division_error(pearson_corr, x_values, y_values):
    """
    Test the pearson_correlation_coefficient function with inputs that have zero variance.
    It should raise a ZeroDivisionError.
    """
    with pytest.raises(ZeroDivisionError):
        pearson_corr(x_values, y_values)

@given(st.lists(st.floats()), st.lists(st.floats()))
def test_pearson_correlation_coefficient_same_length(pearson_corr, x_values, y_values):
    """
    Test the pearson_correlation_coefficient function with inputs that have different lengths.
    It should raise a ValueError.
    """
    x_values.pop()
    with pytest.raises(ValueError):
        pearson_corr(x_values, y_values)

As you can see, with ChatGPT, it is VERY EASY to generate tests that will bug-proof your code. I use this almost everyday in my work as a data engineer. Now then, try this out with your own code and prepare to be amazed!

---

If you liked this, Like and Follow me here on Medium - Subscribing to my YouTube would help me a lot as well! Thank you!!! 💚