Azure ChatGPT

Azure ChatGPT

Microsoft recently unveiled Azure ChatGPT, a cutting-edge chatbot built upon OpenAI’s framework. Notably, the code for Azure ChatGPT is open-sourced under the MIT Licence, offering developers transparency and adaptability. One of its standout features is its commitment to user privacy; the system is designed to be fully isolated from data managed by OpenAI, ensuring a secure user experience.

Moreover, organizations are given an unparalleled level of control, as the network traffic can be confined exclusively to their own network. This ensures a higher degree of security and adaptability. Further sweetening the deal, Azure ChatGPT is plug-n-play, which means that users can seamlessly incorporate their internal data sources or employ plug-ins for effortless integration with their internal services. This blend of privacy, control, and ease of use positions Azure ChatGPT as a remarkable tool for businesses and developers alike.

Prerequisites
Azure OpenAI: To deploy and run ChatGPT on Azure, you’ll need an Azure subscription with access to the Azure OpenAI service. Request access here. Once you have access, follow the instructions in this link to deploy the gpt-35-turbo or gpt-4 models.
Setup GitHub or Azure AD for Authentication: The add an identity provider section below shows how to configure authentication providers.💡Note: You can configure the authentication provider to your identity solution using NextAuth providers
Introduction
Azure ChatGPT is built with the following technologies.

Node.js 18: an open-source, cross-platform JavaScript runtime environment.

Next.js 13: enables you to create full-stack web applications by extending the latest React features

NextAuth.js: configurable authentication framework for Next.js 13

LangChain JS: AI orchestration layer to build intelligent apps

Tailwind CSS: is a utility-first CSS framework that provides a series of predefined classes that can be used to style each element by mixing and matching

shadcn/ui: re-usable components built using Radix UI and Tailwind CSS.

Azure Cosmos DB: fully managed platform-as-a-service (PaaS) NoSQL database to store chat history

Azure App Service: fully managed platform-as-a-service (PaaS) for hosting web applications, REST APIs, and mobile back ends.

Provision Azure resources
Click on the Deploy to Azure button and configure your settings in the Azure Portal as described in the Environment variables section.

⚠️ Note: This will only create azure resources. Follow the deploy to Azure section to build and deploy Azure ChatGPT.

Deploy to Azure
Run from your local machine
Clone this repository locally or fork to your Github account. Run all of the the steps below from the “src” directory:

https://github.com/microsoft/azurechatgpt/tree/main

Make sure you deploy an instance of Cosmos DB in your Azure Subscription
Create a new file named .env.local to store the environment variables add the following variables.
change the active directory to be src
Install npm packages by running npm install
Start the project by running npm run dev
You should now be prompted to login with your chosen OAuth provider. Once successfully logged in, you can start creating new conversations.
Please note:

Do not use double-quotes and do not delete any of the variables.
Make sure that NEXTAUTH_URL=http://localhost:3000 has no comments in the same line.

Azure OpenAI configuration

AZURE_OPENAI_API_KEY=
AZURE_OPENAI_API_INSTANCE_NAME=
AZURE_OPENAI_API_DEPLOYMENT_NAME=
AZURE_OPENAI_API_VERSION=

GitHub OAuth app configuration

AUTH_GITHUB_ID=
AUTH_GITHUB_SECRET=

Azure AD OAuth app configuration

AZURE_AD_CLIENT_ID=
AZURE_AD_CLIENT_SECRET=
AZURE_AD_TENANT_ID=

When deploying to production,

set the NEXTAUTH_URL environment variable to the canonical URL of your site.

More information: https://next-auth.js.org/configuration/options

NEXTAUTH_SECRET=
NEXTAUTH_URL=http://localhost:3000

AZURE_COSMOSDB_URI=
AZURE_COSMOSDB_KEY=

Deploy to Azure – GitHub Actions
The following steps describes how Azure ChatGPT can be deployed to Azure App service using GitHub Actions.

Fork the repository
Fork this repository to your own organisation so that you can execute GitHub Actions against your own Azure Subscription.

Configure secrets in your GitHub repository

1. AZURE_CREDENTIALS The GitHub workflow requires a secret named AZURE_CREDENTIALS to authenticate with Azure. The secret contains the credentials for a service principal with the Contributor role on the resource group containing the container app and container registry.

Create a service principal with the Contributor role on the resource group that contains the Azure App Service.az ad sp create-for-rbac --name --role contributor --scopes /subscriptions//resourceGroups/ --sdk-auth --output json
Copy the JSON output from the command.
In the GitHub repository, navigate to Settings > Secrets > Actions and select New repository secret.
Enter AZURE_CREDENTIALS as the name and paste the contents of the JSON output as the value.
Select Add secret.

1. AZURE_APP_SERVICE_NAME Under the same repository secrets add a new variable AZURE_APP_SERVICE_NAME to deploy to your Azure Web app. The value of this secret is the name of your Azure Web app e.g. my-web-app-name from the domain https://my-web-app-name.azurewebsites.net/

Run GitHub Actions
Once the secrets are configured, the GitHub Actions will be triggered for every code push to the repository. Alternatively, you can manually run the workflow by clicking on the “Run Workflow” button in the Actions tab in GitHub.

Add an identity provider
Once the deployment is complete, you will need to add an identity provider to authenticate your app.

⚠️ Note: Only one of the identity provider is required below.

GitHub Authentication provider
We’ll create two GitHub apps: one for testing locally and another for production.

🟡 Development app setup
Navigate to GitHub OAuth Apps setup https://github.com/settings/developers
Create a New OAuth App https://github.com/settings/applications/new
Fill in the following details
Application name: Azure ChatGPT DEV Environment
Homepage URL: http://localhost:3000
Authorization callback URL: http://localhost:3000/api/auth/callback/github
🟢 Production app setup
Navigate to GitHub OAuth Apps setup https://github.com/settings/developers
Create a New OAuth App https://github.com/settings/applications/new
Fill in the following details
Application name: Azure ChatGPT Production
Homepage URL: https://YOUR-WEBSITE-NAME.azurewebsites.net
Authorization callback URL: https://YOUR-WEBSITE-NAME.azurewebsites.net/api/auth/callback/github
⚠️ After completing app setup, ensure your environment variables locally and on Azure App Service are up to date.

# GitHub OAuth app configuration
AUTH_GITHUB_ID=
AUTH_GITHUB_SECRET=
Azure AD Authentication provider
🟡 Development app setup
Navigate to Azure AD Apps setup
Create a New Registration
Fill in the following details
Application name: Azure ChatGPT DEV Environment
Supported account types: Accounts in this organizational directory only
Redirect URI Platform: Web
Redirect URI: http://localhost:3000/api/auth/callback/azure-ad
🟢 Production app setup
Navigate to Azure AD Apps setup
Create a New Registration
Fill in the following details
Application name: Azure ChatGPT Production
Supported account types: Accounts in this organizational directory only
Redirect URI Platform: Web
Redirect URI: https://YOUR-WEBSITE-NAME.azurewebsites.net/api/auth/callback/azure-ad
After completing app setup, ensure your environment variables locally and on Azure App Service are up to date.

Azure AD OAuth app configuration

AZURE_AD_CLIENT_ID=
AZURE_AD_CLIENT_SECRET=
AZURE_AD_TENANT_ID=
Chatting with your file
Users can utilise this functionality to upload their PDF files through the portal and engage in chat discussions related to the content of those files.

Chat with your data utilises the following Azure Services:

Azure Document Intelligence for extracting information from documents.
Azure Cognitive Search for indexing and retrieving information.
Azure OpenAI Embeddings for embed content extracted from files
Azure OpenAI Embeddings
We use Azure OpenAI Embeddings to convert text to vectors and index it in Azure Cognitive Search.

update the OpenAI environment variables with the following:

AZURE_OPENAI_API_EMBEDDINGS_DEPLOYMENT_NAME=
When deploying to Azure, ensure to update the Azure App service app settings with AZURE_OPENAI_API_EMBEDDINGS_DEPLOYMENT_NAME

Setup Azure Cognitive Search index and Document Intelligence
Create Azure Cognitive Search using the following link
Create an index on Azure Cognitive Search with the following schema. You can use the Azure portal to create the following indexes
{
"name": "azure-chatgpt",
"fields": [
{
"name": "id",
"type": "Edm.String",
"key": true,
"filterable": true
},
{
"name": "user",
"type": "Edm.String",
"searchable": true,
"filterable": true
},
{
"name": "chatThreadId",
"type": "Edm.String",
"searchable": true,
"filterable": true
},
{
"name": "pageContent",
"searchable": true,
"type": "Edm.String"
},
{
"name": "metadata",
"type": "Edm.String"
},
{
"name": "embedding",
"type": "Collection(Edm.Single)",
"searchable": true,
"filterable": false,
"sortable": false,
"facetable": false,
"retrievable": true,
"analyzer": "",
"dimensions": 1536,
"vectorSearchConfiguration": "vectorConfig"
}
],
"vectorSearch": {
"algorithmConfigurations": [
{
"name": "vectorConfig",
"kind": "hnsw"
}
]
}
}
After the index has been created, proceed to modify the env.local file with the appropriate Azure Cognitive Search environment variables.

Azure cognitive search is used for chat over your data

AZURE_SEARCH_API_KEY=
AZURE_SEARCH_NAME=
AZURE_SEARCH_INDEX_NAME=
AZURE_SEARCH_API_VERSION="2023-07-01-Preview"
Create an instance of Azure Form Recognizer (also known as Azure Document Intelligence) using the following link. Please be aware that this resource might be called Form recognizer in Azure Portal.
After the Form Recognizer (Document Intelligence) resource has been created, proceed to modify the env.local file with appropriate environment variables. You can find values for these variables in your Form Recognizer resource (Resource Management blade > Keys and Endpoint). Please make sure that you don’t copy the endpoint from there, but only replace the region in the example below. For example, if your Form Recognizer resource is located in East US Azure region, your AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT variable would be https://eastus.api.cognitive.microsoft.com/.Please note that the file is only preserved for each chat thread:

Azure AI Document Intelligence to extract content from your data

AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT="https://REGION.api.cognitive.microsoft.com/"
AZURE_DOCUMENT_INTELLIGENCE_KEY=
At this point, you should be able to start new chat sessions with the File option.

Once the File chat option is selected, click the Choose File button to select your document and then click the Upload button to upload your file. Please note that the Form Recognizer service supports PDF (text or scanned), JPG and PNG input documents.
Once you receive a notification about a successful file upload, you should be able to start chatting with chatting with a chatbot.
Things to consider:
Central place maintain uploaded files (e.g a storage account with blob storage)
A way to delete indexed documents on Azure Cognitive Search if the chat thread is deleted
Environment variables
Below are the required environment variables, to be added to the Azure Portal or in the .env.local file.

App Setting Value Note
AZURE_OPENAI_API_KEY API keys of your Azure OpenAI resource
AZURE_OPENAI_API_INSTANCE_NAME the name of your Azure OpenAI resource
AZURE_OPENAI_API_DEPLOYMENT_NAME The name of your model deployment
AZURE_OPENAI_API_VERSION 2023-03-15-preview API version when using gpt chat
AUTH_GITHUB_ID Client ID of your GitHub OAuth application
AUTH_GITHUB_SECRET Client Secret of your GitHub OAuth application
NEXTAUTH_SECRET Used to encrypt the NextAuth.js JWT, and to hash email verification tokens. This set by default as part of the deployment template
NEXTAUTH_URL Current webs hosting domain name with HTTP or HTTPS. This set by default as part of the deployment template
AZURE_COSMOSDB_URI URL of the Azure CosmosDB
AZURE_COSMOSDB_KEY API Key for Azure Cosmos DB
AZURE_AD_CLIENT_ID The client id specific to the application
AZURE_AD_CLIENT_SECRET The client secret specific to the application
AZURE_AD_TENANT_ID The organisation Tenant ID
Azure Cognitive Search is optional for Azure ChatGPT. This is only required for chat over file feature.

AZURE_SEARCH_API_KEY API Key of Azure Cognitive search
AZURE_SEARCH_NAME https://AZURE_SEARCH_NAME.search.windows.net The deployment name of your Azure Cognitive Search
AZURE_SEARCH_INDEX_NAME The index name with vector search enabled
AZURE_SEARCH_API_VERSION 2023-07-01-Preview API version which supports vector search 2023-07-01-Preview
AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT https://REGION.api.cognitive.microsoft.com/ Endpoint url of the Azure document intelligence. The REGION is specific to your Azure resource location
AZURE_DOCUMENT_INTELLIGENCE_KEY API keys of your Azure Document intelligence resource
Azure ChatGPT Environment variables
Posted

Follow me for more such content!!!!

Image description

Nashet Ali - Senior Software Development Engineer - LTI - Larsen & Toubro Infotech https://in.linkedin.com/in/nashet-ali-464a65115
Cloudify with Nashet
Nashet Ali (@nashetking) / X https://twitter.com/NashetKing