Build a Production LLM-Powered QC Visual Inspection System Using Azure AI Foundry

In many manufacturing and quality-control (QC) processes, visual inspection is a critical step. However, building an automated inspection system has traditionally required specialized machine vision tools and complex integrations. With Azure AI Foundry and its Prompt Flow feature, you can now orchestrate powerful large language models (LLMs)—like GPT-4 with Vision—to handle sophisticated visual inspection tasks with minimal friction.

In this post, we’ll walk through how to set up a new project in Azure AI Foundry, clone a ready-made quality-inspection sample, and then review how you can customize and deploy it for your own production needs.

1. Sign in to Azure AI Foundry

Head to the Azure AI Foundry portal (provided by your Azure environment).
Sign in using your Azure account credentials.
You’ll land on the Home page of Azure AI Foundry once authenticated.

2. Create a New Hub and Project

From the Home page, click + Create project.
Enter a Name for your project (for instance, “QC Visual Inspection”).
Choose an existing hub or click Create new hub to create one from scratch.
Give your new hub a memorable name (e.g., “ProductionQC”), then select Create.

Now your Azure AI Foundry workspace has a hub (which is like a container or environment) and a project (your application context).

3. Introduction to Prompt Flow

Prompt Flow is a development tool in Azure AI Foundry that helps you orchestrate LLMs, prompts, and Python tools in a visual, node-based interface. It speeds up the cycle of prototyping, experimenting, iterating, and deploying AI applications that rely on LLMs.

With Prompt Flow, you can:

Orchestrate: Combine Large Language Models, Python scripts, and prompts into a reproducible pipeline.
Visualize: Build flows via a drag-and-drop style interface—no heavy coding required.
Experiment & Iterate: Quickly test variations of prompts, compare outputs, and optimize your AI solution.

4. Clone the Quality Inspection Sample

Under Prompt flow in the Azure AI Foundry portal, you will see sample flows you can clone. For a visual inspection use case, the Detect Defects sample is an excellent starting point:

Find the Detect Defects tile (as shown in the “Create a new flow” screenshot).
Click Clone to create your own copy.
Name the flow (e.g., “QC_VisualInspection_Flow”).

5. Dive into the Flow Details

Inside the Detect Defects flow, you’ll see several key components:

Inputs:

reference_image: The “perfect” or expected product image.

test_image: The image of the item to be inspected.

system_message: A text prompt that instructs the LLM (e.g., “You’re a professional defect detector. Your job is to compare…”).

Resize Image (Python):

This step ensures the reference and test images are standardized in size/format before they’re fed into the LLM.

GPT4V_AOAI Node:

This node calls GPT-4 with Vision capabilities in Azure OpenAI to analyze whether the two images match. The model is instructed to respond with “No defect detected” or “Defect detected” along with a detailed reason.

Outputs:

The final result from GPT-4 is captured in the response field.

6. Testing & Iterating

Once your flow is cloned and configured:

Upload or drag-and-drop your reference_image and test_image.
Adjust your prompt in system_message if needed.
Click Start compute session (or Run once the session is ready).

View the logs and final output to see whether the system detected any defects.
If results aren’t accurate enough, you can:
- Fine-tune the prompt.
- Add additional Python logic in your flow to enhance the pre-processing or post-processing.

Prompt Flow makes it easy to iterate by adjusting nodes, re-running, and comparing performance.

7. Deployment Considerations

For a production QC system, you’ll likely want to:

Automate Input: Feed images from your production line into the flow automatically (e.g., from a camera or an upload process).
Scale: Use the Azure AI Foundry’s distributed compute capabilities if you have high throughput.
Monitor: Track performance metrics, run times, and defect-detection accuracy.
Integrate: Provide the resulting output to a database, a dashboard, or an alert system to ensure real-time responses to quality issues.

Conclusion

By following these steps, you’ll have a running LLM-powered QC Visual Inspection system in Azure AI Foundry:

Create a hub and project to organize your application.
Clone the Detect Defects flow (sample) to jumpstart your solution.
Customize prompts, inputs, or Python scripts to match your use case.
Test and iterate using Prompt Flow’s built-in debugging and evaluation features.
Deploy your final solution to production for real-time defect detection and automated QC.

Azure AI Foundry removes much of the complexity normally involved in building AI solutions—especially those requiring visual and language understanding. With GPT-4 (Vision) under the hood, you can quickly set up advanced visual inspections that would otherwise take a team of vision experts to perfect.

🚀 Happy building!