DEV Community

Kenichiro Nakamura
Kenichiro Nakamura

Posted on

3

Azure Open AI in VNet

GPT models are hosted in multiple service vendor at the moment, and Microsoft Azure is one of them.

Even though the models themselves are the same, there are many differences including:

  • cost
  • functionalities
  • type of models and versions
  • geo location
  • security
  • support
  • etc.

One of the most important aspects when we use it in an Enterprise Environment is, of course, security.

By using Azure network security features with Azure Open AI, customers can consume the Open AI service from and within the VNet, therefore no information is flowing in public.

Sample Deployment

Azure Sample repo provides a sample bicep files to deploy Azure Open AI into VNet environment.

GitHub: openai-enterprise-iac

The key features the bicep uses are:

  • VNet
  • VNet integration for Web App
  • Private Endpoint for Azure Open AI
  • Private Endpoint for Cognitive Search
  • Private DNS Zone

By using these features, all the outbound traffic from the Web App only routed inside the VNet and all the names are resolved into private IP addresses. Open AI and Cognitive Search shut down the public IP address, thus there is not public interface endpoint available anymore.

Deploy

The bicep file will deploy following Azure Resources.

architecture

Let's deploy and confirm how it works. I create a resource group in East US region for my own test.



git clone https://github.com/Azure-Samples/openai-enterprise-iac
cd openai-enterprise-iac
az group create -n openaitest -l eastus
az deployment group create -g openaitest -f .\infra\main.bicep


Enter fullscreen mode Exit fullscreen mode

Once I run the commend above, I see the deployment started.

Wait until the deployment completes.

deployment

Test

Let's see if the deployment was succeeded.

Azure Open AI

Let's try public access first.

I could create a deployment without any issue. But when I try from the Chat playground in my Azure Portal, I see the following error.

Chat failed with an error

How about access via the Web API?

From an advanced tool of the App Service, I login to Bash session, and first I ping the service URL.

Image description

I see the private IP address assigned to the Private Endpoint is returend.

Then I use curl command to send request to the endpoint.

👋 While you are here

Reinvent your career. Join DEV.

It takes one minute and is worth it for your career.

Get started

DEV (this website) is a community where over one million developers have signed up to keep up with what's new in software.

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

👋 Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay