Azure ML Prompt flow: Use content safety before sending a request to LLM

Azure Machine Learning (Azure ML) offers an effective connector known as Azure Content Safety within its Prompt Flow feature. This article provides an in-depth exploration of how user input is scrutinized before it is directed towards LLM.

Prerequisites

• Azure Subscription
• Basic knowledge of Prompt flow
• Azure Machine Learning
• Azure AI Content Safety

Setting up Azure AI Content Safety and Establishing Connection

1. Create an Azure Content safety account (free is fine).

2. Create a connector in prompt flow. Use the endpoint and key information obtained from the previous step.

Constructing the Flow

The flow's construction is uncomplicated and includes the following steps:

• Direct input toward content safety
• Analyze the result
• If the input is deemed safe, invoke LLM
• Consolidate the result and transfer it to the output

[The following diagram shows 'Bypassed' when the input was considered to be unsafe.]

Content Safety Segment

I use the default sensitivity for all categories.

Result Extraction

Given that the content safety segment yields an object, I employ the Python segment to parse it.

from promptflow import tool

@tool
def my_python_tool(safety_result) -> str:
  return safety_result["suggested_action"]

LLM Segment

A standard LLM prompt is employed in this section.

But use the activate config to see the content safety result.

system:
You are an AI assistant reading the transcript of a conversation between an AI and a human. Given an input question and conversation history, infer user real intent.

The conversation history is provided just in case of a coreference (e.g. "What is this?" where "this" is defined in previous conversation).

{% for item in chat_history %}
user:
{{item.inputs.question}}
assistant:
{{item.outputs.answer}}
{% endfor %}

user:
{{question}}

Final Output

The final output is then produced by gathering the results from both content safety and LLM.

• If the input is unsafe, a 'None' value is utilized for the LLM output as it does not provide an answer.

from promptflow import tool

@tool
def my_python_tool(safety_result, llm_answer=None) -> str:
  if safety_result["suggested_action"] == "Accept":
    return llm_answer
  else:
    return safety_result

Result

I prefer not to share the unsafe sentences. However, the general rule is that if the content safety identifies inappropriate input, it will refrain from sending the input to LLM

Conclusion

It's advisable to apply the content safety check to the LLM output as well to prevent any unwanted responses from LLM. For this purpose, the content filter feature within AOAI can also be employed.