DEV Community: Kenichiro Nakamura

Azure Open AI in VNet

Kenichiro Nakamura — Thu, 12 Oct 2023 00:53:47 +0000

GPT models are hosted in multiple service vendor at the moment, and Microsoft Azure is one of them.

Even though the models themselves are the same, there are many differences including:

cost
functionalities
type of models and versions
geo location
security
support
etc.

One of the most important aspects when we use it in an Enterprise Environment is, of course, security.

By using Azure network security features with Azure Open AI, customers can consume the Open AI service from and within the VNet, therefore no information is flowing in public.

Sample Deployment

Azure Sample repo provides a sample bicep files to deploy Azure Open AI into VNet environment.

GitHub: openai-enterprise-iac

The key features the bicep uses are:

VNet
VNet integration for Web App
Private Endpoint for Azure Open AI
Private Endpoint for Cognitive Search
Private DNS Zone

By using these features, all the outbound traffic from the Web App only routed inside the VNet and all the names are resolved into private IP addresses. Open AI and Cognitive Search shut down the public IP address, thus there is not public interface endpoint available anymore.

Deploy

The bicep file will deploy following Azure Resources.

Let's deploy and confirm how it works. I create a resource group in East US region for my own test.



git clone https://github.com/Azure-Samples/openai-enterprise-iac
cd openai-enterprise-iac
az group create -n openaitest -l eastus
az deployment group create -g openaitest -f .\infra\main.bicep

Once I run the commend above, I see the deployment started.

Wait until the deployment completes.

Test

Let's see if the deployment was succeeded.

Azure Open AI

Let's try public access first.

I could create a deployment without any issue. But when I try from the Chat playground in my Azure Portal, I see the following error.

How about access via the Web API?

From an advanced tool of the App Service, I login to Bash session, and first I ping the service URL.

I see the private IP address assigned to the Private Endpoint is returend.

Then I use curl command to send request to the endpoint.

Azure ML Prompt flow: Use content safety before sending a request to LLM

Kenichiro Nakamura — Tue, 10 Oct 2023 09:57:33 +0000

Azure Machine Learning (Azure ML) offers an effective connector known as Azure Content Safety within its Prompt Flow feature. This article provides an in-depth exploration of how user input is scrutinized before it is directed towards LLM.

Prerequisites

Azure Subscription
Basic knowledge of Prompt flow
Azure Machine Learning
Azure AI Content Safety

Setting up Azure AI Content Safety and Establishing Connection

Create an Azure Content safety account (free is fine).
Create a connector in prompt flow. Use the endpoint and key information obtained from the previous step.

Constructing the Flow

The flow's construction is uncomplicated and includes the following steps:

Direct input toward content safety
Analyze the result
If the input is deemed safe, invoke LLM
Consolidate the result and transfer it to the output

[The following diagram shows 'Bypassed' when the input was considered to be unsafe.]

Content Safety Segment

I use the default sensitivity for all categories.

Result Extraction

Given that the content safety segment yields an object, I employ the Python segment to parse it.

from promptflow import tool

@tool
def my_python_tool(safety_result) -> str:
  return safety_result["suggested_action"]

LLM Segment

A standard LLM prompt is employed in this section.

But use the activate config to see the content safety result.

system:
You are an AI assistant reading the transcript of a conversation between an AI and a human. Given an input question and conversation history, infer user real intent.

The conversation history is provided just in case of a coreference (e.g. "What is this?" where "this" is defined in previous conversation).

{% for item in chat_history %}
user:
{{item.inputs.question}}
assistant:
{{item.outputs.answer}}
{% endfor %}

user:
{{question}}

Final Output

The final output is then produced by gathering the results from both content safety and LLM.

If the input is unsafe, a 'None' value is utilized for the LLM output as it does not provide an answer.

from promptflow import tool

@tool
def my_python_tool(safety_result, llm_answer=None) -> str:
  if safety_result["suggested_action"] == "Accept":
    return llm_answer
  else:
    return safety_result

Result

I prefer not to share the unsafe sentences. However, the general rule is that if the content safety identifies inappropriate input, it will refrain from sending the input to LLM

Conclusion

It's advisable to apply the content safety check to the LLM output as well to prevent any unwanted responses from LLM. For this purpose, the content filter feature within AOAI can also be employed.

Don't waste time to write README, use readme-ai instead

Kenichiro Nakamura — Wed, 27 Sep 2023 07:38:42 +0000

As a developer, most of us (or maybe just me?) are having fun writing code, but struggling to write README.md.

But README.md is obviously one of the most important artifacts of the repository and we cannot avoid it. So what shall we do? The answer is, AI.

readme-ai

One of my colleagues let me know about "readme-ai" today which was super easy to use.

readme-ai: GitHub

They provide local several ways to run the service.

Locally run from the source code
Use pip module
Use docker container
Use the hosted web application

I found the hosted one was the easiest to use.

Streamlit app

https://readmeai.streamlit.app/

There are only two steps.

Enter your Open AI API key.
Enter your GitHub repository URL.

That's it!

One caveat is that you may need to use the paid version of API as you may encounter API threshold.

Summary

I believe there maybe other solutions to write README.md on our behalf, so please let me know if you know some!

C#: Azure Open AI and Function Calling

Kenichiro Nakamura — Fri, 28 Jul 2023 08:12:00 +0000

As announced in this post, Azure Open AI now supports Function Calling feature.

I don't explain what it is, but I share my experiment result and C# code.

Prerequisites

Azure Subscription and Open AI account
Deploy model that supports Function Calling, e.g. gpt-35-turbo-16k

Test scenario

I wonder if LLM can chain functions if needed. How it behaves when it has multiple functions, etc. So, I tested.

Firstly, I added two functions.

GetWeatherFunction.cs

public class GetWeatherFunction
{
    static public string Name = "get_current_weather";

    // Return the function metadata
    static public FunctionDefinition GetFunctionDefinition()
    {
        return new FunctionDefinition()
        {
            Name = Name,
            Description = "Get the current weather in a given location",
            Parameters = BinaryData.FromObjectAsJson(
            new
            {
                Type = "object",
                Properties = new
                {
                    Location = new
                    {
                        Type = "string",
                        Description = "The city and state, e.g. San Francisco, CA",
                    },
                    Unit = new
                    {
                        Type = "string",
                        Enum = new[] { "Celsius", "Fahrenheit" },
                    }
                },
                Required = new[] { "location" },
            },
            new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }),
        };
    }

    // The function implementation. It always returns 31 for now.
    static public Weather GetWeather(string location, string unit)
    {
        return new Weather() { Temperature = 31, Unit = unit };
    }
}

// Argument for the function
public class WeatherInput
{
    public string Location { get; set; } = string.Empty;
    public string Unit { get; set; } = "Celsius";
}

// Return type
public class Weather
{
    public int Temperature { get; set; }
    public string Unit { get; set; } = "Celsius";
}

GetCapitalFunction.cs

public class GetCapitalFunction
{
    static public string Name = "get_capital";

    // Return the function metadata
    static public FunctionDefinition GetFunctionDefinition()
    {
        return new FunctionDefinition()
        {
            Name = Name,
            Description = "Get the capital of the location",
            Parameters = BinaryData.FromObjectAsJson(
            new
            {
                Type = "object",
                Properties = new
                {
                    Location = new
                    {
                        Type = "string",
                        Description = "The city, state or country, e.g. San Francisco, CA",
                    }
                },
                Required = new[] { "location" },
            },
            new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }),
        };
    }

    // The function implementation. It always return Tokyo for now.
    static public string GetCapital(string location)
    {
        return "Tokyo";
    }
}

// Argument for the function
public class CapitalInput
{
    public string Location { get; set; } = string.Empty;
}

Then I register them in Program.cs

Uri openAIUri = new("https://<your account>.openai.azure.com/");
string openAIApiKey = "<your key>";
string model = "gpt-35-turbo-16k";

// Instantiate OpenAIClient for Azure Open AI.
OpenAIClient client = new(openAIUri, new AzureKeyCredential(openAIApiKey));

ChatCompletionsOptions chatCompletionsOptions = new();
ChatCompletions response;
ChatChoice responseChoice;

// Add function definitions
FunctionDefinition getWeatherFuntionDefinition = GetWeatherFunction.GetFunctionDefinition();
FunctionDefinition getCapitalFuntionDefinition = GetCapitalFunction.GetFunctionDefinition();
chatCompletionsOptions.Functions.Add(getWeatherFuntionDefinition);
chatCompletionsOptions.Functions.Add(getCapitalFuntionDefinition);

I set user question like below.

string question = "What's the weather in the capital city of Japan?";
chatCompletionsOptions.Messages.Add(new(ChatRole.User, question));

Then I call the Completion in a loop to see the finish reason is function or stop.

If the finish reason is function call, then
- Get arguments value
- Call the function
Register responses and results to chatCompletionsOptions.Messages.
Call LLM again with the history.

while (responseChoice.FinishReason == CompletionsFinishReason.FunctionCall)
{
    // Add message as a history.
    chatCompletionsOptions.Messages.Add(responseChoice.Message);

    if (responseChoice.Message.FunctionCall.Name == GetWeatherFunction.Name)
    {
        string unvalidatedArguments = responseChoice.Message.FunctionCall.Arguments;
        WeatherInput input = JsonSerializer.Deserialize<WeatherInput>(unvalidatedArguments,
                new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase })!;
        var functionResultData = GetWeatherFunction.GetWeather(input.Location, input.Unit);
        var functionResponseMessage = new ChatMessage(
            ChatRole.Function,
            JsonSerializer.Serialize(
                functionResultData,
                new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }));
        functionResponseMessage.Name = GetWeatherFunction.Name;
        chatCompletionsOptions.Messages.Add(functionResponseMessage);
    }
    else if (responseChoice.Message.FunctionCall.Name == GetCapitalFunction.Name)
    {
        string unvalidatedArguments = responseChoice.Message.FunctionCall.Arguments;
        CapitalInput input = JsonSerializer.Deserialize<CapitalInput>(unvalidatedArguments,
                new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase })!;
        var functionResultData = GetCapitalFunction.GetCapital(input.Location);
        var functionResponseMessage = new ChatMessage(
            ChatRole.Function,
            JsonSerializer.Serialize(
                functionResultData,
                new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }));
        functionResponseMessage.Name = GetCapitalFunction.Name;
        chatCompletionsOptions.Messages.Add(functionResponseMessage);
    }

    // Call LLM again to generate the response.
    response =
        await client.GetChatCompletionsAsync(
            model,
            chatCompletionsOptions);

    responseChoice = response.Choices[0];
}

Console.WriteLine(responseChoice.Message.Content);

Result

I sent "What's the weather in the capital city of Japan?" to LLM.
LLM returns CompletionsFinishReason.FunctionCall to use get_capital.
Send back the message to LLM again with function result (Tokyo).
LLM returns CompletionsFinishReason.FunctionCall to use get_current_weather.
Send back the message to LLM again with function result (31 degree.)
LLM returns final response saying: "The current weather in the capital city of Japan, Tokyo, is 31 degrees Celsius.".

Conclusion

I see that LLM can chain functions depending on the user input. This behavior is similar with Semantic Kernel Planner, so I will compare them when I have time.

Visual Studio: Live Unit Test is awesome

Kenichiro Nakamura — Fri, 14 Apr 2023 03:21:29 +0000

I am okay writing unit test, but there are several things I don't like.

Keep checking the coverage report
Test failed after changing so many codes as a regression

Visual Studio Live Unit Testing feature helps me to solve these issues.

As the official document is very easy to read, I won't re-write it here but I just want to take note for several important things.

Prerequisites

Enterprise edition of Visual Studio
C# or VB.NET
Supports xUnit, NUnit and MSTest

What it does?

Basically, the Visual Studio runs existing unit tests background and let me know:

Which lines are covered by test
The test results
Which tests are covering the line

Every time I change the code, it runs automatically and shows me the results.

Caveat

It uses computer power, so recommend not to use it if you see any heavy performance decrease in your PC. Also be careful about the battery life if you work in cafe without AC.

Summary

I turn it on all the time as my PC is powerful enough to run it without any issues so far. Please try if you meet the prerequisites!

C#: Pass Lambda Expression to Select LINQ at Runtime

Kenichiro Nakamura — Wed, 29 Mar 2023 08:24:28 +0000

In the previous article, I explain how to pass the lambda string to Where clause. In this article, I do pass it to the Select extension method.

Create Classes

I used List<string> before, but let's create the class to test several different scenario.



public class MyClass
public class MyClass
{
    public int Id { get; set; }
    public string Name { get; set; } = string.Empty;
    public int Age { get; set; }
    public string Country { get; set; } = string.Empty;
}

Select LINQ for string

The Select method takes Func<TSource, TResult> delegate. I create List<MyClass> as an input, so the TSource is MyClass this time.

Then how about TResult? It's depends if I want to select a string property of an int property.



ParameterExpression param = Expression.Parameter(typeof(MyClass), "x");
Expression expression = new ExpressionParser(
    new ParameterExpression[] { param },
    "x.name", 
    null, 
    new ParsingConfig())
    .Parse(typeof(string));

Expression<Func<MyClass, string>> lambda = 
    Expression.Lambda<Func<MyClass, string>>(expression, param);

The input parameter is MyClass and I name it as x. Then the query is x.name to return name. Of course, the return type is string type.

Let's create a list and try the code.



List<MyClass> inputs = new() 
{
    new () { Id = 1, Name = "1" },
    new () { Id = 2, Name = "2" }
};

List<string> result = inputs.AsQueryable().Select(lambda).ToList();

The results is:

Select LINQ for object

If we don't know which property to select yet, we can specify object.



string query = Console.ReadLine();

ParameterExpression param = Expression.Parameter(typeof(MyClass), "x");
Expression expression = new ExpressionParser(
    new ParameterExpression[] { param },
    query, 
    null, 
    new ParsingConfig())
    .Parse(typeof(object));

Expression<Func<MyClass, object>> lambda = 
    Expression.Lambda<Func<MyClass, object>>(expression, param);

//List<string> inputs = new() { "1", "2" };
//var result = inputs.AsQueryable().Where(lambda).ToList();


List<MyClass> inputs = new() 
{
    new () { Id = 1, Name = "1" },
    new () { Id = 2, Name = "2" }
};

List<object> result = inputs.AsQueryable().Select(lambda).ToList();

Then we can run the application to test.

Summary

It's easy to create a lambda expression object based on input, output and its types.

C#: Pass Lambda Expression to Where LINQ at Runtime

Kenichiro Nakamura — Wed, 29 Mar 2023 06:47:28 +0000

I often use LINQ query in my projects, and I sometime needs to pass the lambda expression at the run time, rather than compile time. In this article, I show some samples to achieve this.

Install Library

I create a console application with .NET 7 this time. One library I need to include is System.Linq.Dynamic.Core.

Where LINQ

Let's write a code for Where LINQ query. The Where extension method takes Func<T, bool> delegate. The T changes depending on what you query against.

I use List<string> to simplify the sample for now.

Define input type and variable name as ParameterExpression
Define expression and return type
Define lambda



ParameterExpression param = Expression.Parameter(typeof(string), "x");
Expression expression = new ExpressionParser(
    new ParameterExpression[] { param }, 
    "x==\"1\"", 
    null, 
    new ParsingConfig())
    .Parse(typeof(bool));

Expression<Func<string, bool>> lambda = 
    Expression.Lambda<Func<string, bool>>(expression, param);

The first statement is to define the input type, which is string and I name it as x.

Then I create the expression as x=="1". The return type is always bool for Where method.

Then finally, create the lambda expression object.

Let's use it.



List<string> inputs = new() { "1", "2" };
List<string>result = inputs.AsQueryable().Where(lambda).ToList();

This is equivalent to inputs.Where(x=>x == "1").ToList().

The result is:

Now, change it to dynamic by using Console.ReadLine();



string query = Console.ReadLine();

ParameterExpression param = Expression.Parameter(typeof(string), "x");
Expression expression = new ExpressionParser(
    new ParameterExpression[] { param },
    query, 
    null, 
    new ParsingConfig())
    .Parse(typeof(bool));

Then, I can pass query from console.

Summary

I hope you get some idea how to dynamically create the lambda expression object. In the next article, I will show other LINQ examples.

python: unit test with fixture and patch decorators

Kenichiro Nakamura — Wed, 08 Mar 2023 02:57:41 +0000

In the [pervious article], I used multiple patch decorators to mock several functions. This time, I use Fixture with the decorators to see how they work together.

Fixture

When I have a reusable object across multiple unit tests, I can define a fixture and obtain it in the unit test function. I create one fixture this time to see how I can use it.

Structures and code

This is almost same as before, just modifying add_data and main method to take an argument, and

src/
    ├── my.py  
    ├── my_modules/
    │   ├── __init__.py
    │   └── util.py
    └── tests/
        ├── __init__.py
        ├── test_my.py
        ├── test_unit.py
        └── conftest.py

my.py

import sys
from my_modules.util import util, get_data


def main(argv: list[str]):
    data = get_data(argv)
    return util(data)

if __name__ == '__main__':
    main(sys.argv)

util.py

from datetime import datetime

def util(input: str) -> str:
    input = add_time(input)
    return f"util: {input}"

def add_time(input: str) -> str:
    return f"{datetime.now()}: {input}"

def get_data(input: list[str]) -> str:
    input = f"There are {len(input)} arguments"
    return add_time(input)

Add fixture

I can define fixture in conftest.py that automatically recognize via test frameworks. I define a list_mock fixture this time.

conftest.py

import pytest

@pytest.fixture
def list_mock():
    return ["input1", "input2"]

Add unit test for get_data

Let's add or modify the unit test code for get_data.

@patch('my_modules.util.add_time')
def test_get_data(add_time, list_mock):
    ct = datetime.now()
    expected = f"{ct}: There are {len(list_mock)} arguments"
    add_time.return_value = expected

    result = get_data(list_mock)

    assert expected == result

I patch the add_time function and receive the mock as the first argument of the test function. Then, I receive the list_mock fixture as the second argument.

I can specify the fixture function name as argument name, then it is automatically passed to the function. Very easy!!

Update main unit tests

As the main method also requires list[str] argument, let's update them all. Just receive the list_mock fixture as an argument and pass it to the main function.

test_my.py

from my import main
from unittest.mock import patch, Mock

@patch("my.util", Mock(return_value="dummy"))
@patch("my.get_data", Mock(return_value="some data"))
def test_main(list_mock):
    result = main(list_mock)
    assert result =='dummy'

@patch("my.util")
@patch("my.get_data")
def test_main_util_called_with_expected_parameter(get_data_mock, util_mock, list_mock):
    get_data_mock.return_value = 'some data'
    util_mock.return_value = 'dummy'
    result = main(list_mock)
    assert result =='dummy'
    util_mock.assert_any_call('some data')

def test_main_util_called_with_expected_parameter_with(list_mock):
    with patch("my.util") as util_mock:
        util_mock.return_value = 'dummy'
        with patch("my.get_data") as get_data_mock:
            get_data_mock.return_value = 'some data'
            result = main(list_mock)
    assert result =='dummy'
    util_mock.assert_any_call('some data')


@patch("my.util", Mock(return_value="dummy"))
@patch("my.get_data")
def test_main_get_data_called(get_data_mock, list_mock):
    get_data_mock.return_value = 'some data'
    result = main(list_mock)
    assert result =='dummy'
    assert get_data_mock.called

Summary

I defined


 as a fixture this time, but I can use any object including ``Mock`` object. When I write similar test objects in multiple unit test, we can move it to fixture.

See [the pytest fixtures official document](https://docs.pytest.org/en/7.1.x/how-to/fixtures.html) for more detail.

python: Mock variations

Kenichiro Nakamura — Tue, 07 Mar 2023 06:24:21 +0000

This is my note on how to create various mocks. Please let me know if you have trouble creating a mock in your test.

import pytest
from unittest.mock import Mock

# Create a mock with one attribute.
def test_mock_with_one_attribute():
    mock = Mock(one_attribute='one')
    assert "one" == mock.one_attribute

# Create a mock with multiple attributes.
def test_mock_with_multiple_attributes():
    mock = Mock(**{"one_attribute":"one","two_attribute":"two"})
    assert "one" == mock.one_attribute
    assert "two" == mock.two_attribute

# Create a mock with name attribute. As name is an argument of Mock, 
# we need to speficy it after create mock object.
def test_mock_name_attribute():
    mock = Mock()
    mock.name = "ken"
    assert "ken" == mock.name

# Set return value when the mock itself is called as function.
def test_mock_as_function():
    mock = Mock(return_value='result')
    assert "result" == mock()

# Create a mock with one method.
def test_mock_with_one_method():
    mock = Mock(**{"my_method.return_value":"result"})
    assert "result" == mock.my_method()

# Create a mock with several methods and attributes at once.
def test_mock_with_methods_and_attributes():
    mock = Mock(**{"my_method.return_value":"result_my",
                   "another_method.return_value":"result_another",
                   "one_attribute":"one",
                   "two_attribute":"two"})
    assert "result_my" == mock.my_method()
    assert "result_another" == mock.another_method()
    assert "one" == mock.one_attribute
    assert "two" == mock.two_attribute

# Create a method that returns Exception.
def test_mock_with_side_effect_exception():
    mock = Mock(side_effect=Exception("dummy"))
    with pytest.raises(Exception) as e:
        mock()
    assert "dummy" == str(e.value)

# Create a mock and inner mock as it's attribute
def test_mock_with_innter_mock_multiple_attributes():
    mock = Mock(data = Mock(**{"one_attribute":"one","two_attribute":"two"}))
    assert "one" == mock.data.one_attribute
    assert "two" == mock.data.two_attribute

python: use multiple patch decorators to mock functions

Kenichiro Nakamura — Tue, 07 Mar 2023 05:43:06 +0000

I wrote how to mock in the previous article. This time, I mock multiple functions in the test to see how I can handle them.

Structures and code

This is almost same as before, just adding one more function in util.py.

src/
    ├── my.py  
    ├── my_modules/
    │   ├── __init__.py
    │   └── util.py
    └── tests/
        ├── __init__.py
        ├── test_my.py
        └── test_unit.py

my.py

from my_modules.util import util, get_data

def main():
    data = get_data()
    return util(data)

if __name__ == '__main__':
    main()

util.py

from datetime import datetime

def util(input: str) -> str:
    input = add_time(input)
    return f"util: {input}"

def add_time(input: str) -> str:
    return f"{datetime.now()}: {input}"

def get_data() -> str:
    return "Return some data"

Add a unit test for main function

To test the main method in the my.py, I need to mock both util and get_data method. Let's do it.

Firstly, I added another patch and specify return_value. It doesn't change the test body though.

@patch("my.util", Mock(return_value="dummy"))
@patch("my.get_data", Mock(return_value="some data"))
def test_main():
    result = main()
    assert result =='dummy'

Let's receive the mock objects as the arguments.

@patch("my.util")
@patch("my.get_data")
def test_main_util_called_with_expected_parameter(get_data_mock, util_mock):
    get_data_mock.return_value = 'some data'
    util_mock.return_value = 'dummy'
    result = main()
    assert result =='dummy'
    util_mock.assert_any_call('some data')

The interesting part is the order of argument. As you see, I can get the mock from bottom up order of the patch decorators. I firstly though I can receive the mocks in the same order as the decorators, but I was wrong.

Finally, let's try with statement.

def test_main_util_called_with_expected_parameter_with():
    with patch("my.util") as util_mock:
        util_mock.return_value = 'dummy'
        with patch("my.get_data") as get_data_mock:
            get_data_mock.return_value = 'some data'
            result = main()
    assert result =='dummy'
    util_mock.assert_any_call('some data')

I am not 100% sure if this is the correct way to implement, but it works as expected anyway.

I just added one more example. This time, I specify the Mock object in the first decorator only. In this case, I can receive just one mock object as an argument.

@patch("my.util", Mock(return_value="dummy"))
@patch("my.get_data")
def test_main_get_data_called(get_data_mock):
    get_data_mock.return_value = 'some data'
    result = main()
    assert result =='dummy'
    assert get_data_mock.called

I check if get_data function is called.

Summary

I understand when I get the mock object as the arguments. It's a bit confusing but once I understand, it's very useful.

python: unit test with mock functions from different modules

Kenichiro Nakamura — Sun, 05 Mar 2023 15:14:14 +0000

I recently started learning python 3 and unit test with pytest and unittest.

As I struggle to figure out how to mock in several situations, I am taking note here so that anyone has same issue maybe find this useful someday.

Structures and code

Before writing tests, this is my folder and files structures.



src/
    ├── my.py  
    ├── my_modules/
    │   ├── __init__.py
    │   └── util.py
    └── tests/
        ├── __init__.py
        ├── test_my.py
        └── test_unit.py

my.py



from my_modules.util import util

def main():
    return util('my input')

if __name__ == '__main__':
    main()

util.py



from datetime import datetime

def util(input: str) -> str:
    input = add_time(input)
    return f"util: {input}"

def add_time(input: str) -> str:
    return f"{datetime.now()}: {input}"

Add a unit test for util function

For util method, I need to mock add_time that is from the same file. I found several ways to achieve this, and this is one of them.

test_unit.py



from unittest.mock import patch, Mock
from my_modules.util import util

@patch('my_modules.util.add_time', Mock(return_value='dummy'))
def test_util():
    expected = 'util: dummy'
    input = 'input'
    input = util(input)
    assert expected == input

I used unittest.mock.patch function as a decorator and specify the desired return value as part of the Mock object. The namespace for add_time is the same as the util function as they are in the same module.

Add a unit test for add_time function

To unit test add_time function, I need to mock datetime.now() function. I again use the unittest.mock.patch. This time, I need to create the Mock with a bit more code as I need to mock a function, rather than the simple return_value nor an attribute.



from datetime import datetime
from unittest.mock import patch, Mock
from my_modules.util import add_time

@patch('my_modules.util.datetime', Mock(**{"now.return_value": datetime(2023, 1, 1)}))
def test_add_time():
    expected = f'{datetime(2023, 1, 1)}: input'
    input = 'input'
    input = add_time(input)
    assert expected == input

I can pass a dictionary that contains attributes and methods information to the Mock object. As I mock now function, I use "now.return_value":<some date>.

If it's an attribute, I can simply pass it to the Mock like Mock(attribute_name=<value>) or as part of the dictionary like {"attribute":<value>}

The module name is a bit interesting as I need to specify my_modules.util.datetime. The reason is that as soon as the datetime is imported to the util.py, it becomes part of the same module, that was quite confusing to me.

I can show another sample of this in the next test.

Add a unit test for main function

To test the main method in the my.py, I just need to mock util method. Let's do it.

test_my.py



from my import main
from unittest.mock import patch, Mock

@patch("my.util", Mock(return_value="dummy"))
def test_main():
    result = main()
    assert result =='dummy'

Even though the util function comes from the my_modules module, in the test time, it becomes my.util namespace as I previously explained.

I specify the Mock object in the decorator, but I cannot if the util function is actually called with the expected parameters. So, let's accept the mock inside the test function.



@patch("my.util")
def test_main_util_called_with_expected_parameter(util_mock):
    util_mock.return_value = 'dummy'
    result = main()
    assert result =='dummy'
    util_mock.assert_any_call('my input')

This time, I use the decorator without passing the Mock object. Instead, I received it in the argument as util_mock, then I specify the return_value.

As I have access to the mock, I can then assert it the method is called with the expected arguments.

Lastly, I also learnt that I can use with statement as well to achieve the same.



def test_main_util_called_with_expected_parameter_with():
    with patch("my.util") as util_mock:
        util_mock.return_value = 'dummy'
        result = main()
    assert result =='dummy'
    util_mock.assert_any_call('my input')

By doing this, I don't need the decorator.

All the tests run successfully!

Summary

I actually don't know yet which is the best way to mock functions. I believe each one has pros and cons, so if any of you have any suggestions or opinions, please let me know in the comment! Thanks.

C# 11: New raw string literals feature makes my life easier!

Kenichiro Nakamura — Mon, 30 Jan 2023 05:32:39 +0000

Liquid syntax error: Variable '{{ ""key"":""{value}' was not properly terminated with regexp: /\}\}/