DEV Community: Yoshio Terada

Securely Connecting Azure Container Apps to Azure OpenAI Using User Managed Identity

Yoshio Terada — Mon, 16 Sep 2024 02:15:16 +0000

Today, I want to share two key points from this entry:

Deploying Java Applications to Azure Container Apps Without Containerization
Secure Connection from Azure Container Apps to Azure OpenAI Using User Managed Identity

The source code used here can be found in the following GitHub repository.
GitHub: Secure-OpenAI-Java-App-on-Azure-Container-Apps

1. Deploying Java Applications to Azure Container Apps Without Containerization

On September 11, 2024, the announcement "Announcing the General Availability of Java Experiences on Azure Container Apps" was made.

As detailed in the Java on Azure Container Apps overview, support for Java on Azure Container Apps has been enhanced. For example, Azure Container Apps now supports the following Spring components as managed services:

Eureka Server for Spring
Config Server for Spring
Admin for Spring

Additionally, as explained in the Quickstart: Launch Your First Java Application in Azure Container Apps, Azure Container Apps now offers a new feature called Cloud Build Service. This allows you to deploy applications directly to Azure Container Apps from Java artifacts like JAR or WAR files. The service automatically creates and deploys container images from the specified Java artifacts, eliminating the need to manually write Dockerfile container definitions or handle container builds and pushes.

To achieve this with Azure Container Apps, you use the az containerapp up command and specify the Java artifact as an argument. Detailed steps are provided later in the guide (see: 2.8 Creating an Azure Container Apps Instance).

This significantly simplifies deploying Java applications to Azure Container Apps. Azure Container Apps can also scale the number of instances from zero as needed, making it a highly convenient service. We encourage you to try it.

2. Securely Connecting Azure Container Apps to Azure OpenAI Using User Managed Identity

In recent times, security measures have become increasingly important, and it is essential for businesses to build more secure systems. Microsoft recommends using Managed Identity for connections instead of password-based access when creating secure environments, such as production environments. This approach utilizes Microsoft Entra ID authentication.

This method allows you to grant specific permissions for resources within a defined scope, making security management more flexible. For more details on Managed Identity, refer to the article "What are Managed Identities for Azure Resources?". In this entry, I will clearly explain how to set up a User Managed Identity, step by step.

Following these steps will help you understand how to configure it, and they can also serve as a reference for setting up other resources.

Steps to Set Up User Managed Identity

To connect Azure Container Apps to Azure OpenAI using a User Managed Identity, follow these steps to set up the environment:

Set Environment Variables
Create a Resource Group
Create an Azure OpenAI Instance
Create a User Managed Identity
Assign Roles to the User Managed Identity for Azure OpenAI
Create an Azure Container Apps Environment
Develop a Spring Boot Web Application
Create an Azure Container Apps Instance
Assign the User Managed Identity to Azure Container Apps
Verify the Setup

2.1. Set Environment Variables

When setting up the environment, configure environment variables to avoid repetitive input.

export RESOURCE_GROUP=yoshio-OpenAI-rg
export LOCATION=eastus2
export AZURE_OPENAI_NAME=yt-secure-openai
export OPENAI_DEPLOY_MODEL_NAME=gpt-4o
export USER_MANAGED_IDENTITY_NAME=yoshio-user-managed-id
export SUBSCRIPTION=$(az account show --query id --output tsv)

Below are the environment variable names and their descriptions. If you need to change the names to suit your environment, please refer to the descriptions below to modify each resource name accordingly.

Environment Variable Name	Description
RESOURCE_GROUP	Name of the resource group to create
LOCATION	Location where the environment will be set up
USER_MANAGED_IDENTITY_NAME	Name of the User Managed Identity
AZURE_OPENAI_NAME	Name of the Azure OpenAI
OPENAI_DEPLOY_MODEL_NAME	Name of the AI model to be deployed
SUBSCRIPTION	Subscription ID to be used

2.2. Create a Resource Group

First, create a resource group in the Azure environment. Use the --location argument to specify the Azure region where it will be created.

az group create --name $RESOURCE_GROUP --location $LOCATION

2.3. Create an Azure OpenAI Instance

Next, create an Azure OpenAI instance by running the az cognitiveservices account create command. In this step, specify --kind OpenAI and --custom-domain for the instance.

az cognitiveservices account create \
  --name $AZURE_OPENAI_NAME \
  --resource-group $RESOURCE_GROUP \
  --kind OpenAI \
  --custom-domain $AZURE_OPENAI_NAME \
  --sku S0 \
  --location $LOCATION

Note:

It is crucial to specify --custom-domain when creating an Azure OpenAI instance.
If omitted, a region endpoint like https://eastus2.api.cognitive.microsoft.com/ will be automatically generated. As outlined in "Authenticate with Microsoft Entra ID", you cannot perform authentication using Microsoft Entra ID, in this case Managed Identity, without specifying this option. To enable Managed Identity authentication, you need to specify --custom-domain.

Next, deploy the OpenAI model to your newly created Azure OpenAI instance using the az cognitiveservices account deployment create command. Here, specify the model name with --model-name and the model version with --model-version. You should also define the service capacity using --sku-capacity and select the service plan with --sku-name.

az cognitiveservices account deployment create \
  --name $AZURE_OPENAI_NAME \
  --resource-group  $RESOURCE_GROUP \
  --deployment-name $OPENAI_DEPLOY_MODEL_NAME \
  --model-name $OPENAI_DEPLOY_MODEL_NAME \
  --model-version "2024-08-06"  \
  --model-format OpenAI \
  --sku-capacity "20" \
  --sku-name "GlobalStandard"

With this, the creation of the Azure OpenAI instance is complete. After creating the instance, store the necessary information for your Java program implementation and other operations in environment variables.

export OPEN_AI_RESOURCE_ID=$(az cognitiveservices account list \
                                    -g $RESOURCE_GROUP \
                                    --query "[0].id" \
                                    --output tsv)
export OPEN_AI_ENDPOINT=$(az cognitiveservices account show \
                                    -g $RESOURCE_GROUP \
                                    --name $AZURE_OPENAI_NAME \
                                    --query "properties.endpoint" \
                                    --output tsv)
export OPEN_AI_ACCESS_KEY=$(az cognitiveservices account keys list \
                                    -g $RESOURCE_GROUP \
                                    --name $AZURE_OPENAI_NAME \
                                    --query key1 --output tsv)

Here, we are setting the following environment variables:

Environment Variable Name	Description
OPEN_AI_RESOURCE_ID	Resource ID of OpenAI (Needed for role assignment scope)
OPEN_AI_ENDPOINT	Endpoint of OpenAI (Required for Java app connection)
OPEN_AI_ACCESS_KEY	Access key for OpenAI (Needed for Java app development locally)

2.4. Create a User Managed Identity

Now that the Azure OpenAI instance is set up, the next step is to create a User Managed Identity. Use the az identity create command.

az identity create -g $RESOURCE_GROUP -n $USER_MANAGED_IDENTITY_NAME

Once the User Managed Identity is created, retrieve the necessary information for future command execution and Java program usage, and assign it to the environment variables.

export USER_MANAGED_ID_CLIENT_ID=$(az identity list \
                                            -g $RESOURCE_GROUP \
                                            --query "[0].clientId" \
                                            -o tsv)
export USER_MANAGED_ID_PRINCIPAL_ID=$(az identity list \
                                            -g $RESOURCE_GROUP \
                                            --query "[0].principalId" \
                                            -o tsv)
export USER_MANAGED_ID_RESOURCE_ID=$(az identity list \
                                            -g $RESOURCE_GROUP \
                                            --query "[0].id" \
                                            -o tsv)

Below is an explanation of each environment variable's value and its usage:

Environment Variable Name	Description
USER_MANAGED_ID_CLIENT_ID	Client ID of the User Managed Identity (Needed for Java app implementation)
USER_MANAGED_ID_PRINCIPAL_ID	Principal ID of the User Managed Identity (Required for role assignment)
USER_MANAGED_ID_RESOURCE_ID	Resource ID of the User Managed Identity (Needed for assigning ID to Container Apps)

2.5. Assign Roles to User Managed Identity for Azure OpenAI

Use the az role assignment create command to assign a role to the User Managed Identity, allowing it to interact with the OpenAI resource using the Cognitive Services OpenAI User permission.

The $OPEN_AI_RESOURCE_ID represents the OpenAI resource ID, and the role is assigned specifically to that resource. This method grants only the necessary permissions for the app to run, enhancing security by avoiding unnecessary permissions.

az role assignment create --assignee $USER_MANAGED_ID_PRINCIPAL_ID \
                              --scope $OPEN_AI_RESOURCE_ID \
                              --role "Cognitive Services OpenAI User"

In addition to the roles mentioned, you can also assign the following roles:

Cognitive Services OpenAI User
Cognitive Services OpenAI Contributor
Cognitive Services Contributor
Cognitive Services Usages Reader

For detailed information about the capabilities of each role, please refer to the Role-based Access Control for Azure OpenAI Service.

2.6. Create an Azure Container Apps Environment

Next, create an Azure Container Apps Environment. To do this using the Azure CLI, you need to register additional extensions and providers. If you haven't executed the following commands before, please execute them now.

az upgrade
az extension add --name containerapp --upgrade -y
az provider register --namespace Microsoft.Web
az provider register --namespace Microsoft.App
az provider register --namespace Microsoft.OperationalInsights

Then, define the name of the Azure Container Apps Environment as an environment variable. Choose a name that suits your environment.

export CONTAINER_ENVIRONMENT=YTContainerEnv3

Finally, run the az containerapp env create command to set up the environment.

az containerapp env create --name $CONTAINER_ENVIRONMENT \
                               --enable-workload-profiles \
                               -g $RESOURCE_GROUP \
                               --location $LOCATION

2.7. Creating a Spring Boot Web Application

With the Azure OpenAI and Azure Container Apps Environment set up, we will now create a Java project to implement a simple app that invokes an OpenAI model from a Java Application. We will use Spring Boot for this implementation.

2.7.1 Creating a Spring Boot Project

Execute the following command to create a Spring Boot project. After creating and downloading the project, unzip the archive to extract its contents.

curl https://start.spring.io/starter.zip \
  -d dependencies=web \
  -d type=maven-project \
  -d language=java \
  -d bootVersion=3.3.3 \
  -d baseDir=Yoshio-AI-App-Spring-Boot \
  -d groupId=com.yoshio3 \
  -d artifactId=Yoshio-AI-App \
  -d name=myproject \
  -d packageName=com.yoshio3 \
  -o YoshioAIProject.zip

unzip YoshioAIProject.zip

The above command will generate a project with the following directory structure.

├── HELP.md
├── mvnw
├── mvnw.cmd
├── pom.xml
└── src
    ├── main
    │   ├── java
    │   │   └── com
    │   │       └── yoshio3
    │   │           └── MyprojectApplication.java
    │   └── resources
    │       ├── application.properties
    │       ├── static
    │       └── templates
    └── test
        └── java
            └── com
                └── yoshio3
                    └── MyprojectApplicationTests.java

2.7.2 Editing the pom.xml Project File

Add the following dependencies to the pom.xml file located in the root directory. This will include the necessary libraries for connecting to and authenticating with OpenAI.

<dependencies>
    ......
    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-ai-openai</artifactId>
        <version>1.0.0-beta.11</version>
    </dependency>
    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-identity</artifactId>
        <version>1.13.2</version>
    </dependency>
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-api</artifactId>
        <version>2.0.16</version>
    </dependency>
    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-core</artifactId>
        <version>1.5.8</version>
    </dependency>
    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-classic</artifactId>
        <version>1.5.8</version>
    </dependency>
</dependencies>

2.7.3 Implementing a RESTful Endpoint (Main Part)

Next, create an AIChatController.java file in the src/main/java/com/yoshio3 directory. Implement the following code to define a RESTful endpoint that queries OpenAI upon receiving a request.

package com.yoshio3;

import org.springframework.web.bind.annotation.RestController;
import com.azure.ai.openai.OpenAIClient;
import com.azure.ai.openai.OpenAIClientBuilder;
import com.azure.ai.openai.models.ChatCompletions;
import com.azure.ai.openai.models.ChatCompletionsOptions;
import com.azure.ai.openai.models.ChatRequestAssistantMessage;
import com.azure.ai.openai.models.ChatRequestMessage;
import com.azure.ai.openai.models.ChatRequestSystemMessage;
import com.azure.ai.openai.models.ChatRequestUserMessage;
import com.azure.core.credential.AzureKeyCredential;
import com.azure.identity.ManagedIdentityCredential;
import com.azure.identity.ManagedIdentityCredentialBuilder;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.util.ArrayList;
import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;


@RestController
public class AIChatController {

    private static final Logger LOGGER = LoggerFactory.getLogger(AIChatController.class);

    @Value("${USER_MANAGED_ID_CLIENT_ID}")
    private String userManagedIDClientId;

    @Value("${OPENAI_ENDPOINT}")
    private String openAIEndpoint;

    @Value("${OPENAI_KEY}")
    private String openAIKey;

    private static final String OPEN_AI_CHAT_MODEL = "gpt-4o";

    /**
     * This API is used to chat with OpenAI's GPT-4 model. And if user ask somethings, it will
     * return the message with Pirate language.
     * 
     * Ex. You can invoke the API by using the following command: curl -X POST
     * http://localhost:8080/askAI -H "Content-Type: application/json" -d '{"message":"Please tell
     * me about the appeal of Spring Boot in Japanese."}'
     * 
     * @param message RequestMessage
     * @return String Response from OpenAI
     */

    @PostMapping("/askAI")
    public String chat(@RequestBody RequestMessage message) {
        return getResponseFromOpenAI(message.getMessage());
    }

    /**
     * This method is used to get the response from OpenAI.
     * 
     * For production environment, you can use Managed Identity to authenticate with OpenAI. If you
     * want to use Managed Identity, please use the ManagedIdentityCredentialBuilder.
     * 
     * For local development, you can use AzureKeyCredential to authenticate with OpenAI.
     * 
     * @param message RequestMessage
     * @return String Response from OpenAI
     */

    private String getResponseFromOpenAI(String message) {
        try {
            // Create OpenAI client with User Managed Identity
            ManagedIdentityCredential credential =
                    new ManagedIdentityCredentialBuilder().clientId(userManagedIDClientId).build();
            OpenAIClient openAIClient = new OpenAIClientBuilder().credential(credential)
                    .endpoint(openAIEndpoint).buildClient();

            // Create OpenAI client without Managed Identity (For local development)
            // OpenAIClient openAIClient = new OpenAIClientBuilder().endpoint(openAIEndpoint)
            // .credential(new AzureKeyCredential(openAIKey)).buildClient();

            // Create Chat Request Messages
            List<ChatRequestMessage> chatMessages = new ArrayList<>();
            chatMessages.add(new ChatRequestSystemMessage("You are a helpful assistant. You will talk like a pirate."));
            chatMessages.add(new ChatRequestUserMessage("Can you help me?"));
            chatMessages.add(new ChatRequestAssistantMessage("Of course, me hearty! What can I do for ye?"));
            chatMessages.add(new ChatRequestUserMessage(message));
            ChatCompletionsOptions chatCompletionsOptions = new ChatCompletionsOptions(chatMessages);

            // Invoke OpenAI Chat API
            ChatCompletions chatCompletions = openAIClient.getChatCompletions(OPEN_AI_CHAT_MODEL, chatCompletionsOptions);
            StringBuilder response = new StringBuilder();
            chatCompletions.getChoices()
                    .forEach(choice -> response.append(choice.getMessage().getContent()));

            return response.toString();
        } catch (Exception e) {
            StackTraceElement[] stackTrace = e.getStackTrace();
            LOGGER.error(e.getMessage());
            LOGGER.error(e.getLocalizedMessage());
            Throwable cause = e.getCause();
            if (cause != null) {
                LOGGER.error(e.getCause().toString());
            }
            for (StackTraceElement element : stackTrace) {
                LOGGER.error(element.toString());
            }
            return e.getMessage();
        }
    }
}

Note that the OpenAIClient instance is described in two different ways. The current code uses ManagedIdentityCredential to connect to Azure OpenAI with a User Managed Identity. This code works only when running in an Azure environment and won't work in a local or non-Azure environment.

During development, you need to test and verify functionality locally. In such cases, a User Managed Identity cannot be used. Instead, connect using the OpenAI Access Key by uncommenting the line with AzureKeyCredential(openAIKey) to create the OpenAIClient instance.

Additionally, SLF4J and Logback are used for logging in the implementation. Configure them by creating a logback-spring.xml file in the /src/main/resources directory. While detailed logging configuration is not covered here, the original code is available on GitHub for reference if needed.

Finally, here's a brief overview of the code: When a question or message is received from a user, it responds in a pirate tone as defined by SYSTEM. Enjoy the pirate-style replies!

2.7.4 Defining the JSON Format for Endpoint Reception

Next, define the JSON data format to be sent to this RESTful service. To process messages like {"message":"What is the benefit of Spring Boot"} in the HTTP request body, define the following class.

package com.yoshio3;

public class RequestMessage {
    private String message;

    public void setMessage(String message) {
        this.message = message;
    }

    public String getMessage() {
        return message;
    }
}

With this, we have completed the minimal code necessary for operation verification.

2.7.5 Application Configuration

Next, configure the settings to connect to Azure OpenAI. During the systems setup from sections 2.1 to 2.6, all required information was stored in environment variables. Execute the following command to retrieve the necessary information in the Java program.

echo "USER_MANAGED_ID_CLIENT_ID" : $USER_MANAGED_ID_CLIENT_ID
echo "OPEN_AI_ENDPOINT" : $OPEN_AI_ENDPOINT
echo "OPEN_AI_ACCESS_KEY" : $OPEN_AI_ACCESS_KEY

Write the execution results above into the application.properties file located in the /src/main/resources/ directory.

spring.application.name=AI-Chatbot
logging.level.root=INFO
logging.level.org.springframework.web=INFO

USER_MANAGED_ID_CLIENT_ID=********-****-****-****-************
OPENAI_ENDPOINT=https://********.openai.azure.com/
OPENAI_KEY=********************************

Note:

OPENAI_KEY is the access key for OpenAI. Use it only during development in a development environment and avoid using it in a production environment.

2.7.6 (Optional): Verifying Operation in a Local Environment

To verify if the Java program works locally, swap the comments in the OpenAIClient instance creation part of the AIChatController class code and run it.

// ManagedIdentityCredential credential = new ManagedIdentityCredentialBuilder().clientId(userManagedIDClientId).build();
// OpenAIClient openAIClient = new OpenAIClientBuilder().credential(credential)
//                    .endpoint(openAIEndpoint).buildClient();

 OpenAIClient openAIClient = new OpenAIClientBuilder().endpoint(openAIEndpoint)
                       .credential(new AzureKeyCredential(openAIKey)).buildClient();

After making the changes, execute the following command.

mvn spring-boot:run

If the Spring Boot application starts successfully, it will be listening on port 8080. Use the following curl command to verify if you can query Azure OpenAI.

curl -X POST http://localhost:8080/askAI \
     -H "Content-Type: application/json" \
     -d '{"message":"What is the benefit of the Spring Boot?"}'

The response will vary each time you run the command, but it will reply in a pirate tone, for example.

Aye, Captain! Let me explain the benefits of Spring Boot.
Spring Boot is a framework that aids rapid application development. 
It allows you to create standalone applications with minimal configuration.

It also comes with auto-configuration, reducing manual setup. Managing dependencies is easy, and it includes commonly used libraries.
It's fast to start up, easy to deploy, and suitable for cloud operations. 

It's a powerful tool for boosting developer productivity.
Anything else I can assist with?

After completing local verification, revert the changes to generate the OpenAIClient instance using ManagedIdentityCredential.

2.7.7 Building the Application and Creating Artifacts

Once these steps are complete, build the application and create the artifacts.

mvn clean package

Upon building the project, the artifacts will be created in the target directory. Here, the Spring Boot application is created as Yoshio-AI-App-0.0.1-SNAPSHOT.jar.

> ls -l target 
total 40876
-rw-r--r-- 1 teradayoshio wheel 41847613  9 14 14:52 Yoshio-AI-App-0.0.1-SNAPSHOT.jar
-rw-r--r-- 1 teradayoshio wheel     7003  9 14 14:52 Yoshio-AI-App-0.0.1-SNAPSHOT.jar.original
drwxr-xr-x 5 teradayoshio wheel      160  9 14 20:34 classes
drwxr-xr-x 3 teradayoshio wheel       96  9 14 14:52 generated-sources
drwxr-xr-x 3 teradayoshio wheel       96  9 14 14:52 generated-test-sources
drwxr-xr-x 3 teradayoshio wheel       96  9 14 14:52 maven-archiver
drwxr-xr-x 3 teradayoshio wheel       96  9 14 14:52 maven-status
drwxr-xr-x 4 teradayoshio wheel      128  9 14 14:52 surefire-reports
drwxr-xr-x 3 teradayoshio wheel       96  9 14 20:34 test-classes

2.8. Creating an Azure Container Apps Instance

Now that the implementation of the Java program is complete, let's deploy this Java Application to Azure Container Apps.

Specify the name for the container application using the environment variable CONTAINER_APP_NAME. Define the path and filename of the Spring Boot artifact with JAR_FILE_PATH_AND_NAME.

export CONTAINER_APP_NAME=yoshio-ai-app
export JAR_FILE_PATH_AND_NAME=./target/Yoshio-AI-App-0.0.1-SNAPSHOT.jar

As explained in "1. Deploying Java Applications to Azure Container Apps Without Containerization", you no longer need to create and deploy your own container image when deploying to Azure Container Apps. We only have the Java artifact Yoshio-AI-App-0.0.1-SNAPSHOT.jar, and we will deploy based on this.

To do so, use the az containerapp up command. As you can see from the arguments, you only need to specify --artifact $JAR_FILE_PATH_AND_NAME, without any container image arguments. This command will automatically set up the build environment, build the container, and deploy it.

Please execute the following command.

az containerapp up \
  --name $CONTAINER_APP_NAME \
  --resource-group $RESOURCE_GROUP \
  --subscription $SUBSCRIPTION \
  --location $LOCATION \
  --environment $CONTAINER_ENVIRONMENT \
  --artifact $JAR_FILE_PATH_AND_NAME \
  --ingress external \
  --target-port 8080 \
  --env-vars AZURE_LOG_LEVEL=2 \
  --query properties.configuration.ingress.fqdn

Note:

You can slightly customize the container creation. Use the environment variables provided below as needed:
Build Environment Variables for Java in Azure Container Apps.

2.9. Assigning a User Managed Identity to Azure Container Apps

Having deployed the app to Azure Container Apps, the final step is to apply a User Managed Identity to the container we created. Execute the az containerapp identity assign command.

az containerapp identity assign --name $CONTAINER_APP_NAME \
                                    --resource-group $RESOURCE_GROUP \
                                    --user-assigned $USER_MANAGED_ID_RESOURCE_ID

With that, the User Managed Identity configuration is complete.

2.10. Verifying Operation

Now that all the settings are complete, let's verify the operation. First, obtain the FQDN host name assigned to the container's Ingress.

export REST_ENDPOINT=$(az containerapp show -n $CONTAINER_APP_NAME \
              -g $RESOURCE_GROUP \
              --query properties.configuration.ingress.fqdn --output tsv)

Next, append the RESTful endpoint URL to the obtained host name and connect to it.

curl -X POST https://$REST_ENDPOINT/askAI \
     -H "Content-Type: application/json" \
     -d '{"message":"What is Spring Boot? Please explain around 200 words?"}'

Each time it is executed, the response may vary, but you can expect results like those below.

Arrr, Spring Boot be a powerful framework fer buildin' Java applications with minimal fuss, savvy? 
It sails under the banner of the larger Spring ecosystem, providin' a swift and easy approach fer creatin' scalable and stand-alone web apps.

Ye see, it comes with embedded servers like Tomcat or Jetty, meanin' ye don't have the hassle of deployin' to external servers. 
Spring Boot simplifies the setup by makin' configurations automatic, allowin' developers to focus on writin' their code without worryin' about the boilerplate.

It also hoists the flag of convention over configuration, offerin' defaults that make gettin’ started as smooth as a voyage with favorable winds. 
Should ye need to adjust yer sails, customization be available through its easy-to-use properties.

And if ye treasure quick development, Spring Boot supports ye well with its rich assortment of pre-configured starters, integratin' smoothly with other technologies like databases and messaging systems.

In essence, Spring Boot be a trusty vessel fer any sea-farin' coder seekin' speed, efficiency, and a treasure chest of features in their Java web applications. 
Aye, it be a boon to all developers, whether seasoned or green as a fresh landlubber!

2.11. Reusing the Same User Managed Identity to Access Azure OpenAI from Other Azure Resources

Previously, we discussed how to apply a User Managed Identity to Azure Container Apps to connect to Azure OpenAI. However, this User Managed Identity can also be reused by other services.

In other words, applications deployed on other Azure resources can use the same User Managed Identity to connect to Azure OpenAI.

Some resources allow you to specify this identity during creation, while for others, you can assign it later.

Below, we demonstrate how to assign a User Managed Identity to existing Azure resources, except for Azure Container Instances.

For Azure VM

To assign a User Managed Identity to an Azure VM, execute the following command:

az vm identity assign -g $RESOURCE_GROUP \
                      -n $VM_NAME \
                      --identities $USER_MANAGED_ID_RESOURCE_ID

For Azure App Service

To assign a User Managed Identity to an Azure App Service, execute the following command:

az webapp identity assign -g $RESOURCE_GROUP \
                          --name $APP_SERVICE_NAME \
                          --identities $USER_MANAGED_ID_RESOURCE_ID

For Azure Functions

To assign a User Managed Identity to Azure Functions, execute the following command:

az functionapp identity assign -g $RESOURCE_GROUP \
                               -n $AZURE_FUNCTION_NAME \
                               --identities $USER_MANAGED_ID_RESOURCE_ID

For Azure Container Instances

For Azure Container Instances, you can assign a User Managed Identity during the instance creation:

az container create \
                        --resource-group $RESOURCE_GROUP  \
                        --name $CONTAINER_INSTANCE_NAME \
                        --image $CONTAINER_IMAGE \
                        --assign-identity $USER_MANAGED_ID_RESOURCE_ID

For Azure Kubernetes Service

To assign a User Managed Identity to Azure Kubernetes Service, execute the following command:

az aks update \
                    -g $RESOURCE_GROUP \
                    --name $AKS_CLUSTER_NAME \
                    --enable-managed-identity \
                    --assign-identity $USER_MANAGED_ID_RESOURCE_ID

Like this, by using a User Managed Identity, you can securely connect to target resources across different Azure services by reusing correctly configured access roles.

2.12. Comparing Access Keys and User Managed Identities from a Security Perspective

This section explains the security differences between using Access Keys and User Managed Identities for Azure OpenAI.

When using a User Managed Identity, you specify the client ID of the User Managed Identity when creating an instance of the ManagedIdentityCredential class in your Java Application. This client ID value is set as an environment variable $USER_MANAGED_ID_CLIENT_ID when the User Managed Identity is created.

At first glance, it might seem that this is the only critical piece of information needed for the connection, aside from the endpoint URL.

ManagedIdentityCredential credential =
        new ManagedIdentityCredentialBuilder()
                       .clientId(userManagedIDClientId)
                        .build();
OpenAIClient openAIClient = new OpenAIClientBuilder()
                                        .credential(credential)
                                        .endpoint(openAIEndpoint)
                                        .buildClient();

Let's consider the impact of a leaked client ID versus a leaked Access Key.

An Access Key is like a regular password. If it is leaked, anyone who knows the password can access the resource. In the case of Azure OpenAI, this means anyone could use AI models like GPT-4. If the network is publicly accessible, the impact could be even greater.

On the other hand, if the client ID is leaked, the impact is limited. This is because the client ID alone cannot connect to Azure OpenAI. To use a User Managed Identity, the service must be running on Azure. Even if Azure OpenAI is publicly accessible, you cannot connect from a local environment or over a network using a Java application.

Additionally, the following role assignment is configured for the User Managed Identity:

az role assignment create --assignee $USER_MANAGED_ID_PRINCIPAL_ID \
    --scope $OPEN_AI_RESOURCE_ID \
    --role "Cognitive Services OpenAI User"

This sets what actions can be performed using this user ID. Here, the Cognitive Services OpenAI User role is granted for Azure OpenAI services, limiting permissions to operations within Azure OpenAI.

This means that even with this ID, you cannot perform administrative actions like adding models, not just across Azure but even within Azure OpenAI.

Furthermore, to use this user ID, you must log in to Azure as an administrator, and only the resources specified by the administrator can be accessed within the scope of this ID's permissions.

In summary, compared to the impact of a leaked access key, a leaked client ID requires multiple steps to exploit, making it more secure.

For these reasons, User Managed Identities offer a more secure way to manage operations than access keys. We encourage you to use Managed Identities to build a secure environment.

3 Summary

In this guide, we walked through a detailed, step-by-step process for securely connecting Azure Container Apps to Azure OpenAI using a User Managed Identity. While setting up a User Managed Identity might seem cumbersome at first, it offers the significant advantage of reusability. If your systems are few or flexibility isn't a major concern, a System Managed Identity might be preferable.

However, as the number of systems increases, User Managed Identities become very handy. After creating a User Managed Identity and assigning roles, you can reuse the same ID across various services like Azure App Service, Azure Functions, and Azure Kubernetes Services. The more systems or services you manage, the more beneficial this becomes.

Please try using Managed Identities to effectively build secure environments.

Comparison of Two Methods for Deploying Azure Functions to Azure Container Apps

Yoshio Terada — Tue, 10 Sep 2024 15:51:51 +0000

Yesterday, I wrote an article titled "Deploying a Java Azure Function on Azure Container Apps."

In that entry, I mentioned using the Azure's integrated management capabilities and I want to clarify what that means and how it differs from previous methods in this article.

Old Method: Creating with `az containerapp create`

Azure Container Apps is one of Azure's container execution environments, allowing you to run any containerized service. Previously, if you wanted to run Azure Functions in Azure Container Apps, you would create an instance using the following command:



az containerapp create \
--name general-container-app \
--resource-group $RESOURCE_GROUP_NAME \
--environment $CONTAINER_ENVIRONMENT \
--registry-server $CONTAINER_REGISTRY_SERVER \
--image $CONTAINER_REGISTRY_SERVER/$C_IMAGE_NAME:$C_IMAGE_TAG \
--target-port 80 \
--ingress external \
--query properties.configuration.ingress.fqdn

After executing the command, you would see a message like this:



Container app created. Access your app at https://general-container-app.niceocean-********.eastus.azurecontainerapps.io/

You could then use a curl command to connect to your Azure Functions service:



curl https://general-container-app.niceocean-********.eastus.azurecontainerapps.io/api/httpexample?name=World

Upon accessing the Azure Container Apps Environment, you would see that the general-container-app is created as a Container App, and this management interface is available for any deployed containerized application.

https://raw.githubusercontent.com/yoshioterada/Azure-Functions-Deploy-To-Azure-Container-Apps/main/images/ACA-Instance-for-Azure-Functions.png

New Method: Creating with `az functionapp create`

The new method allows you to create Azure Functions in Azure Container Apps using the az functionapp create command instead of az containerapp create.



az functionapp create \
--name $AZURE_FUNCTION_NAME \
--resource-group $RESOURCE_GROUP_NAME \
--environment $CONTAINER_ENVIRONMENT \
--storage-account $STORAGE_NAME \
--workload-profile-name "Consumption" \
--max-replicas 15 \
--min-replicas 1 \
--functions-version 4 \
--runtime java \
--image $CONTAINER_REGISTRY_SERVER/$C_IMAGE_NAME:$C_IMAGE_TAG \
--assign-identity

With this command, your Azure Functions will be created in Azure Container Apps, and the management interface will clearly show that it is an Function App.

This means Azure Functions can now be managed through a dedicated Azure Functions management interface, making it difference from other container application.

However, there are some differences between the management features provided in Azure App Service and those available for Azure Functions on Container Apps. For instance, certain functionalities like diagnostic tools, deployment features, and others may not be available.

Reference: Azure Functions on App Service (Portal)

For comparison, here’s the management interface for Azure Functions deployed on Azure App Service.

The differences in management capabilities between App Service and Azure Container Apps can include:

Pick up:



- Diagnose and solve problems
- Microsoft Defender for Cloud
- Events (preview)
- Log stream
- Deployment
- App Service plan
- Development Tools
- Monitoring
- Support + troubleshooting

Some may think that the lack of certain features suggests missing functionality.

However, when deployed to Azure Container Apps, the operating environment is container-based, which changes deployment and management methods. Features not included in the Azure Functions management interface will need to be managed separately through the Azure Container Apps interface.

Management of Containers in Azure Container Apps

When using the az functionapp create command to create an Azure Functions instance on Azure Container Apps, a new resource group is automatically created that houses the container instance.

In my environment, the resource group name follows this convention:

$CONTAINER_ENVIRONMENT_FunctionApps_$UUID

You will see that an Azure Container Apps instance named after your specified $AZURE_FUNCTION_NAME has been generated.

When you click on this instance, you'll be directed to a management interface specific to Azure Container Apps, where the Azure Functions run as container instances.

Azure Container Apps provides different CI/CD and deployment methods than Azure App Service. It also allows for features offered at the container level, such as Dapr and Service Connector, which can be utilized.

Conclusion

Previously, it was possible to run Azure Functions by containerizing them in a container execution environment, but there was no dedicated management interface for Azure Functions.

With this new method, Azure Functions and Azure Container Apps have integrated, offering a container environment with an associated Azure Functions management interface.

I know some customers operate Azure Functions Container on Azure Kubernetes Service (AKS). Previously, they lacked a dedicated management interface. However, by deploying to Azure Container Apps, they can now use Azure Functions management while enjoying the simplicity of managing Azure Container Apps compared to managing operations on AKS.

The methods for deploying Azure Functions to Azure Container Apps are likely to evolve further. I look forward to seeing how this develops.

Deploying a Java Azure Function on Azure Container Apps

Yoshio Terada — Mon, 09 Sep 2024 13:52:38 +0000

Azure Functions provides integrated support for developing, deploying, and managing containerized function apps on Azure Container Apps. This makes it easier to run and manage Azure Functions using the integrated Azure management portal, compared to running Azure Functions independently in container environments like Azure Kubernetes Service (AKS). Additionally, by leveraging features provided by Azure Container Apps, you can easily utilize functionalities such as KEDA, Dapr, Envoy, scaling, monitoring, security, and access control for your Azure Functions.

[Reference]

Azure Container Apps hosting of Azure Functions

Create your first containerized functions on Azure Container Apps

Setting Environment Variables

Below are the environment variables related to creating Azure Container Apps resources. Here, you specify various names and installation locations for the resources you will create, as well as the container image name and tag.

# Azure Container Apps resource names
export LOCATION=eastus
export RESOURCE_GROUP_NAME=yoshio-rg
export CONTAINER_REGISTRY_NAME=cajava2411
export CONTAINER_ENVIRONMENT=YoshioContainerEnvironment
export STORAGE_NAME=yoshiojavastorage
export AZURE_FUNCTION_NAME=yoshiojavafunc

# Container image name and tag
export C_IMAGE_NAME=tyoshio2002/java-function-on-aca
export C_IMAGE_TAG=1.0

Creating and Testing a Java Azure Function Project

1. Create an Azure Functions for Java Maven Project

First, create a Maven project for Azure Functions for Java. This Maven project is designed for creating Azure Functions using Java 21. Use the mvn archetype:generate command to create the project, modifying parameters as needed.

mvn archetype:generate \
-DinteractiveMode=false \
-DarchetypeGroupId=com.microsoft.azure \
-DarchetypeArtifactId=azure-functions-archetype \
-DgroupId=com.yoshio3 \
-Dpackage=com.yoshio3 \
-DartifactId=yoshiojavafunc \
-DappName=Java-Azure-Functions \
-DappRegion=$LOCATION \
-DjavaVersion=21 \
-Dversion=1.0-SNAPSHOT \
-Ddocker

Executing the above command will automatically create a directory structure, and Function.java will contain sample code for an Azure Function with an HTTP trigger. A Dockerfile will also be created, which contains the configuration for running Azure Functions in a Docker container environment.

├── Dockerfile
├── host.json
├── local.settings.json
├── pom.xml
└── src
    ├── main
    │   └── java
    │       └── com
    │           └── yoshio3
    │               └── Function.java
    └── test
        └── java
            └── com
                └── yoshio3
                    ├── FunctionTest.java
                    └── HttpResponseMessageMock.java

2. Run Azure Function Locally

Build the Maven project and run the Azure Functions locally. Execute the following commands to start the Azure Functions with the HTTP trigger.

mvn clean package
mvn azure-functions:run

Once the Azure Functions are running, open another terminal and execute the following command to send a request to the HTTP trigger. You should receive a response saying "Hello, World".

curl "http://localhost:7071/api/HttpExample?name=World"
# Output: 
Hello, World

Creating and Testing the Azure Functions Container Image

1. Build and Test the Docker Image

Use the automatically generated Dockerfile to build the Azure Functions container image. Execute the following command to build the image.

docker build -t $C_IMAGE_NAME:$C_IMAGE_TAG -f ./Dockerfile .

After the build is complete, run the following command to check if the image has been created.

docker images | grep $C_IMAGE_NAME
# Output: 
tyoshio2002/java-function-on-aca    1.0    bcf471e6f774   9 hours ago     1.46GB

Once the image is created, run the following command to test the Azure Functions container image locally. The Azure Functions container internally uses HTTP port 80, so you will map it to port 8080 for local access. After the container starts, execute the curl command to send a request to the Azure Functions HTTP trigger. If everything is working correctly, you should receive "Hello, World".

docker run -p 8080:80 -it $C_IMAGE_NAME:$C_IMAGE_TAG
curl "http://localhost:8080/api/HttpExample?name=World"
# Output: 
Hello, World

Pushing the Container Image to Azure Container Registry

1. Log in to Azure CLI

First, log in to Azure using the Azure CLI. Execute the following command to log in.

az login

2. Create a Resource Group

Create a resource group in Azure. This resource group will be used to group resources related to Azure Container Registry and Azure Container Apps.

az group create --name $RESOURCE_GROUP_NAME --location $LOCATION

3. Create and Log in to Azure Container Registry

Create an Azure Container Registry and log in. Azure Container Registry is a private container registry for pushing container images.

az acr create --resource-group $RESOURCE_GROUP_NAME --name $CONTAINER_REGISTRY_NAME --sku Basic
az acr login --name $CONTAINER_REGISTRY_NAME
az acr update -n $CONTAINER_REGISTRY_NAME --admin-enabled true

4. Retrieve the Azure Container Registry Server Name

Retrieve the server name of the created Azure Container Registry. The server name will be in the format $CONTAINER_REGISTRY_NAME.azurecr.io.

CONTAINER_REGISTRY_SERVER=$(az acr show --name $CONTAINER_REGISTRY_NAME --query loginServer --output tsv)

5. Push the Image to Azure Container Registry

To push the locally created container image to Azure Container Registry, tag the image using the tag command. After tagging, use the push command to push the image.

docker tag $C_IMAGE_NAME:$C_IMAGE_TAG $CONTAINER_REGISTRY_SERVER/$C_IMAGE_NAME:$C_IMAGE_TAG
docker push $CONTAINER_REGISTRY_SERVER/$C_IMAGE_NAME:$C_IMAGE_TAG

Creating Azure Container Apps and Deploying the Java Azure Function

1. Register Extensions and Resource Providers in Azure CLI

To create and manage Azure Container Apps from Azure CLI, register the necessary extensions and resource providers.

az upgrade
az extension add --name containerapp --upgrade -y
az provider register --namespace Microsoft.Web
az provider register --namespace Microsoft.App
az provider register --namespace Microsoft.OperationalInsights

2. Create Azure Container Apps Environment

Create an environment for Azure Container Apps. This command sets up the configuration needed to host Azure Container Apps.

az containerapp env create --name $CONTAINER_ENVIRONMENT --enable-workload-profiles --resource-group $RESOURCE_GROUP_NAME --location $LOCATION

3. Create Storage Account for Azure Functions

Azure Functions requires a storage account when creating a Function App instance. Therefore, create a general-purpose storage account for Azure Functions.

az storage account create --name $STORAGE_NAME --location $LOCATION --resource-group $RESOURCE_GROUP_NAME --sku Standard_LRS

4. Create an Instance of Java Azure Function in Azure Container Apps

Create an instance of the Java Azure Function in Azure Container Apps. Execute the following command to create the instance. Since the Azure Function is created using Java 21, specify --runtime java.

az functionapp create --name $AZURE_FUNCTION_NAME \
--resource-group $RESOURCE_GROUP_NAME \
--environment $CONTAINER_ENVIRONMENT \
--storage-account $STORAGE_NAME \
--workload-profile-name "Consumption" \
--max-replicas 15 \
--min-replicas 1 \
--functions-version 4 \
--runtime java \
--image $CONTAINER_REGISTRY_SERVER/$C_IMAGE_NAME:$C_IMAGE_TAG \
--assign-identity

5. Assign Role for Azure Function to Access Azure Container Registry

Finally, configure secure access for Azure Functions to Azure Container Registry. Enable the system-managed identity for Azure Functions and assign the ACRPull role for access.

FUNCTION_APP_ID=$(az functionapp identity assign --name $AZURE_FUNCTION_NAME --resource-group $RESOURCE_GROUP_NAME --query principalId --output tsv)
ACR_ID=$(az acr show --name $CONTAINER_REGISTRY_NAME --query id --output tsv)
az role assignment create --assignee $FUNCTION_APP_ID --role AcrPull --scope $ACR_ID

6. Retrieve the URL of the Azure Function

Finally, retrieve the HTTP trigger function URL of the deployed Azure Function. Use the az functionapp function show command to get the details of the Azure Functions function.

az functionapp function show --resource-group $RESOURCE_GROUP_NAME --name $AZURE_FUNCTION_NAME --function-name HttpExample --query invokeUrlTemplate
# Output: "https://yoshiojavafunc.niceocean-********.eastus.azurecontainerapps.io/api/httpexample"

You can then send a request to the retrieved URL using curl command to confirm that the Azure Functions is working correctly.

curl "https://yoshiojavafunc.niceocean-********.eastus.azurecontainerapps.io/api/httpexample?name=World"
# Expected Output: 
Hello, World

If everything is set up correctly, you should receive a response saying "Hello, World", confirming that your Azure Function is functioning as expected.

Summary

In this guide, you learned how to:

Set up environment variables for your Azure resources.
Create a Java Azure Function project using Maven.
Run the Azure Function locally for testing.
Build a Docker image for the Azure Function.
Push the Docker image to Azure Container Registry.
Create an Azure Container Apps environment and deploy the Java Azure Function.
Assign necessary roles for secure access to Azure Container Registry.
Retrieve and test the URL of the deployed Azure Function.

By following these steps, you can successfully deploy a Java Azure Function on Azure Container Apps, leveraging the benefits of containerization and Azure's integrated management capabilities. If you have any further questions or need assistance with specific steps, feel free to ask!

Automated PDF-to-Text Conversion and Vector Search with Azure OpenAI and Blob Storage

Yoshio Terada — Wed, 14 Jun 2023 03:30:09 +0000

1. Introduction

Recently, I wrote a blog post how to use the Azure OpenAI Embedding model to find the most relevant documents. By utilizing this feature, you can easily locate the most pertinent documents. In this article, I will explain how to automatically convert PDF files to text by uploading them to Azure Blob Storage and perform vector searches using the Azure OpenAI Embedding model.

By using this service, you can effortlessly search for any document, be it internal company documents or various academic papers, as long as they are in PDF format. Simply upload the files to Azure Blob Storage, and the system will automatically make them searchable. When you actually perform a search, ChatGPT will summarize and display the relevant sections for you.

1.1 Overview of the Service

The service introduced in this article processes the data through the following steps:

Please obtain the source code of the application from the "Source Code is Here".

Upload a PDF file to Azure Blob Storage
Azure Functions' Blob Trigger detects the uploaded file and converts the PDF into text on a per-page basis
The converted text is vectorized by calling Azure OpenAI Embedding
The vectorized data is saved to Azure PostgreSQL Flexible Server.
The user enters a search query.
The input query is vectorized by calling Azure OpenAI Embedding.
Based on the vectorized data, similar data is retrieved from Azure PostgreSQL Flexible Server.
The relevant sections of the highly similar result documents are analyzed by Azure OpenAI ChatGPT and returned in a streaming.

1.2. Technologies Used

The service introduced in this article utilizes the following Azure services:

2. Setup Environment

Currently, there is no Azure CLI command available for creating an Azure OpenAI instance, so you will need to create one using the Azure Portal. Before proceeding with the steps below, please create an Azure OpenAI instance using the Azure Portal's GUI (as of June 13, 2023).

For other Azure resources, you can easily create them using the following commands:

> create-env-en.sh

Note :

Occasionally, even after enabling the vector search (pgvector) feature in Azure PostgreSQL Flexible Server multiple times, it may not be activated, and the execution of queries using vectors may fail. In such cases, please run the script above again to create a new PostgreSQL Flexible Server instance.

Before executing the commands above, please modify the variables within the script to match your environment.

####################### Setting Environment Variables for Creating Azure Resources #######################
# Configuring the Resource Group and location settings
export RESOURCE_GROUP_NAME=Document-Search-Vector1
export DEPLOY_LOCATION=japaneast

# Settings for Azure PostgreSQL Flexible Server (In my environment, there are construction restrictions, so it is set to eastus)
export POSTGRES_INSTALL_LOCATION=eastus
export POSTGRES_SERVER_NAME=documentsearch1
export POSTGRES_USER_NAME=azureuser
export POSTGRES_USER_PASS='!'$(head -c 12 /dev/urandom | base64 | tr -dc '[:alpha:]'| fold -w 8 | head -n 1)$RANDOM
export POSTGRES_DB_NAME=VECTOR_DB
export POSTGRES_TABLE_NAME=DOCUMENT_SEARCH_VECTOR
export PUBLIC_IP=$(curl ifconfig.io -4)

# Settings for Azure Blob Storage
export BLOB_STORAGE_ACCOUNT_NAME=documentsearch1

# Note: If you change the values below, you will also need to modify the implementation part of the BlobTrigger in Functions.java.
export BLOB_CONTAINER_NAME_FOR_PDF=pdfs

# Settings for Azure Cosmos DB
export COSMOS_DB_ACCOUNT_NAME=odocumentsearchstatus1
export COSMOS_DB_DB_NAME=documentregistrystatus
export COSMOS_DB_CONTAINER_NAME_FOR_STATUS=status

# Obtaining the Azure subscription ID (If you are using the default subscription, you do not need to make the changes below)
export SUBSCRIPTION_ID="$(az account list --query "[?isDefault].id" -o tsv)"
####################### Setting Environment Variables for Creating Azure Resources #######################

2.1 Setting Environment Variables

After executing the script above, if it is successful, you will see a message like below. Please update the local.settings.json file for Azure Functions (BlobUploadDetector) and the application.properties file for Spring Boot (PDF-Summarizer), respectively, according to the output content.

####################### Setting Environment Variables for Creating Azure Resources #######################
########## Please write the following content in the local.settings.json file ##########
-----------------------------------------------------------------------------
# Azure-related environment settings

"AzureWebJobsStorage": "****************************",
"AzurePostgresqlJdbcurl": "jdbc:postgresql://documentsearch1.postgres.database.azure.com:5432/VECTOR_DB?sslmode=require",
"AzurePostgresqlUser": "azureuser",
"AzurePostgresqlPassword": "********",
"AzurePostgresqlDbTableName": "DOCUMENT_SEARCH_VECTOR",
"AzureBlobstorageName": "documentsearch1",
"AzureBlobstorageContainerName": "pdfs",
"AzureCosmosDbEndpoint": "https://documentsearchstatus1.documents.azure.com:443/",
"AzureCosmosDbKey": "********************",
"AzureCosmosDbDatabaseName": "documentregistrystatus",
"AzureCosmosDbContainerName": "status",
"AzureOpenaiUrl": "https://YOUR_OPENAI.openai.azure.com",
"AzureOpenaiModelName": "gpt-4",
"AzureOpenaiApiKey": "YOUR_OPENAI_ACCESS_KEY",
-----------------------------------------------------------------------------

### Please write the following content in the application.properties file for Spring Boot ###
-----------------------------------------------------------------------------

# Settings for Azure PostgreSQL connection information

azure.postgresql.jdbcurl=jdbc:postgresql://documentsearch1.postgres.database.azure.com:5432/VECTOR_DB?sslmode=require
azure.postgresql.user=azureuser
azure.postgresql.password=**********
azure.postgresql.db.table.name=DOCUMENT_SEARCH_VECTOR

# The following Blob-related settings

azure.blobstorage.name=documentsearch1
azure.blobstorage.container.name=pdfs

# Settings for Azure Cosmos DB

azure.cosmos.db.endpoint=https://documentsearchstatus1.documents.azure.com:443
azure.cosmos.db.key=********************************************
azure.cosmos.db.database.name=documentregistrystatus
azure.cosmos.db.container.name=status

# Settings for Azure OpenAI

azure.openai.url=https://YOUR_OPENAI.openai.azure.com
azure.openai.model.name=gpt-4
azure.openai.api.key=********************************************
-----------------------------------------------------------------------------

# After creating PostgreSQL, please connect using the following command:
-----------------------------------------------------------------------------
> psql -U azureuser -d VECTOR_DB \
     -h documentsearch1.postgres.database.azure.com

Auto-generated PostgreSQL password: **********

# Once connected to PostgreSQL, please execute the following command:
-------------------------------------------------------------
VECTOR_DB=> CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
VECTOR_DB=> CREATE EXTENSION IF NOT EXISTS "vector";

# Finally, execute the following command to create the TABLE:

VECTOR_DB=> CREATE TABLE IF NOT EXISTS DOCUMENT_SEARCH_VECTOR
                 (id uuid, embedding VECTOR(1536),
                  origntext varchar(8192), fileName varchar(2048),
                  pageNumber integer, PRIMARY KEY (id));
-----------------------------------------------------------------------------

Note:

For the above OpenAI-related settings, please set the Connection URL, Model name, and API Key that were created in the Azure Portal, respectively.

2.1.1 Setting PostgreSQL Extensions

After generating the PostgreSQL instance, please check if it is accessible from your local environment by executing the following command:

psql -U azureuser -d VECTOR_DB \
     -h documentsearch1.postgres.database.azure.com

※ The password is displayed in the terminal.

Once connected to PostgreSQL, please execute the following command to add the extensions. In the example below, the UUID and pgvector extensions are enabled.

CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
CREATE EXTENSION IF NOT EXISTS "vector";

Lastly, please execute the following command to create a table that can utilize vectors.

CREATE TABLE IF NOT EXISTS DOCUMENT_SEARCH_VECTOR
                    (id uuid, embedding VECTOR(1536),
                    origntext varchar(8192), fileName varchar(2048),
                    pageNumber integer, PRIMARY KEY (id));

3. Running the Application

Once the environment setup is complete, follow the steps below to run the application:

3.1Running Azure Functions (BlobUploadDetector)

Since the environment variables have been changed, please build Azure Functions first and then execute it.

> cd BlobUploadDetector
> mvn clean package
> mvn azure-functions:run

3.2 Running Spring Boot

Since the environment variables have been changed, please build Spring Boot first and then execute it.

> cd SpringBoot
> mvn clean package
> mvn spring-boot:run

3.3 Uploading Files to Blob Storage

You can connect to the Azure Portal and upload files individually, but to make it easier to upload files, I will use Azure Storage Explorer.

Please download and install Azure Storage Explorer from Azure Storage Explorer Download.

When you launch the Azure Storage Explorer, you will be prompted to connect with your Azure account. Please connect, and once connected, you will see a screen like the one below, where you can drag and drop multiple files to upload them all at once.

3.4 Verifying the Spring Boot Application

When you access http://localhost:8080/, you will see a screen like the one below. Please enter the keyword you want to search for in the text area and click the Submit button. The search results will then be displayed in a streaming format.

Additionally, if you click on the Registered File List link, you will see a list of files registered in the database.

By clicking on the Failed Registration File List link, you will see a list of files that failed to register in the database.

4. Points to Note for Application Implementation

4.1 Key Points for Implementing Azure Functions (BlobUploadDetector)

First, I will explain the points to note for the implementation of BlobUploadDetector, which is implemented in Azure Functions.

4.1.1 Extending the Execution Time of Azure Functions

By default, Azure Functions have a runtime limit of 5 to 30 minutes. If this limit is exceeded, Azure Functions will time out. However, when analyzing and processing PDF files with a large number of pages, the processing may not be completed within this time limit. Therefore, I have extended the execution time of Azure Functions.

In this sample, I have added the following settings to host.json to extend the execution time of Azure Functions indefinitely. However, please note that only the Premium plan and Dedicated plan can be set to unlimited. The Consumption plan cannot be set to unlimited.

{
  "version": "2.0",
  "functionTimeout": "-1", 
}

Reference :

Function app timeout duration

4.1.2 Adjusting the Usage Interval of Azure OpenAI Embedding API

The maximum number of calls per minute for the Azure OpenAI text-embedding-ada-002 varies depending on the instance. If the limit is exceeded, the Azure OpenAI Embedding API will return a 400 error. Therefore, please adjust the call interval of the Azure OpenAI Embedding API according to your environment.

// Azure OpenAI Call Interval (milliseconds)
private final static int OPENAI_INVOCATION_INTERVAL = 20;

Reference:

The above setting is for when the number of calls per minute for the text-embedding-ada-002 model is set to the maximum allowable value of 240k Token/min (approximately 1440 req/min) in my environment. If the number of calls per minute is different, please adjust this setting accordingly. In some cases, it may be necessary to change the setting in units of 5-10 seconds instead of milliseconds.

You can check and change the number of calls per minute from the Quotas section in Azure OpenAI Studio.

4.1.3 Setting Environment Variables in Azure Functions

In Azure Functions, environment variables are set in the local.settings.json file for local environments. When deploying to Azure Functions, the settings are configured in the Configuration section of Azure Functions.

Note:
JBe aware that environment variables cannot be set in the property files under src/main/resources/, which are commonly used in Java.

The values of environment variables set in local.settings.json can be retrieved in the program by calling the System.getenv() method.

4.1.4 Points to Consider When Changing the Container Name Created in Azure Blob Storage

If you want to change the value of "AzureBlobstorageContainerName": "pdfs", in the environment variables, please also modify the following part of the Function.java source code. The value that can be specified for path requires a definition in constants, so it cannot be obtained from environment variables and set to path.

    @FunctionName("ProcessUploadedFile")
    @StorageAccount("AzureWebJobsStorage")
    public void run(
            @BlobTrigger(
                    name = "content", path = "pdfs/{name}", dataType = "binary") byte[] content,
            @BindingName("name") String fileName,
            @BlobInput(name = "inputBlob", path = "pdfs/{name}",
                    dataType = "binary") byte[] inputBlob,
            final ExecutionContext context) throws UnsupportedEncodingException {

4.1.5 Points to Consider in PDF Parsing Process (Especially When Using Models Other Than ChatGPT-4)

The following steps are performed when converting PDF files to text:

Split the PDF file by the number of pages
Convert each split page to text
If the converted text contains \n and multiple spaces, remove them and replace with a single white space character
Measure the size of the converted text
If the text size is within the range of (7200-7500) characters and characters such as ".", "?", "!" are found, split the text at that point
If the above delimiter characters are not found up to MAX_SEPARATE_TOKEN_LENGTH (7500), forcibly split the text
Send the split text to Azure OpenAI Embedding API to obtain vectors
Register the vectors data into the PostgreSQL vector database table
Register and update the status of each process in Cosmos DB as appropriate

Note:
The text-embedding-ada-002 and gpt-4 models have a maximum token limit of 8192 per request. In our experience, specifying a value close to the maximum may cause errors depending on the request conditions. Therefore, I have set the maximum number of characters per page to 7500. However, to avoid forcibly splitting the text in the middle of a sentence as much as possible, I have implemented the process so that if there is a delimiter character within the range of 7200 to 7500 characters, the text can be split there. If you use gpt-35 turbo, the MAX is 4000 tokens, so the current MAX_SEPARATE_TOKEN_LENGTH value is too large. Please change this value.

4.1.6 About the Data to Insert

Based on my experience, I believe that inserting pages with a certain number of characters into the PostgreSQL database is more effective. For example, if you insert a page with only one line written, such as Please refer to this for information on Azure Functions, into the database, the vector values obtained with text-embedding-ada-002 will have little difference from other data, resulting in a higher similarity. Therefore, by inserting pages with a certain number of characters, you can increase the difference in vector values.

In other words, to put it simply, if you search for the term Azure Functions, the above-mentioned page will be more likely to be found, regardless of what kind of text follows. Therefore, please register pages with a certain number of characters in the database.

4.2 Points to Consider in Spring Boot Implementation

Next, I will describe the points to consider when implementing a Spring Boot application.

4.2.1 Implementing as Server Sent Event

When sending requests to OpenAI, the Java SDK provides instances for both synchronous and asynchronous processing. In this case, I will use the OpenAIAsyncClient class for asynchronous processing. By using this, you can return results as a real-time stream.

Since the Server-Side returns results as a Realtime streaming, you can, of course, return the results to the client in Realtime using Server Sent Events.

To implement the Spring Boot application as a Server Sent Event, I will use Spring Boot's WebFlux. Additionally, when implementing SSE with WebFlux, I have added the following implementation to enable 1-to-1 communication:

The client's browser (JavaScript) generates a UUID and connects to /openai-gpt4-sse-stream with that UUID.

Below is the JavaScript code snipet.

        let userId;
        let eventSource;

        window.onload = function () {
            userId = generateUUID();
            setupEventSource();
        };

        function setupEventSource() {
            eventSource = new EventSource("/openai-gpt4-sse-stream?userId=" + userId);

Note :

The method of generating UUIDs in JavaScript is implemented very simply in this example.
If you plan to use this in a production environment, please consider a more secure method for generating UUIDs.

When connecting to the server, the server registers the client's UUID and Sinks.Many<String> in a 1 to 1 relationship using a Map.

Below is an excerpt from the Java code.

    private static Map<UUID, Sinks.Many<String>> userSinks;

    @GetMapping(path = "/openai-gpt4-sse-stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
    @ResponseBody
    public Flux<String> sseStream(@RequestParam UUID userId) {
        Sinks.Many<String> userSink = getUserSink(userId);
        if (userSink == null) {
            userSink = createUserSink(userId);
        }
        LOGGER.trace("USER ID IS ADDED: {}}", userId);
        return userSink.asFlux().delayElements(Duration.ofMillis(10));
    }

    private Sinks.Many<String> createUserSink(UUID userId) {
        Sinks.Many<String> userSink = Sinks.many().multicast().directBestEffort();
        userSinks.put(userId, userSink);
        LOGGER.debug("User ID: {} User Sink: {} is Added.", userId, userSink);
        return userSink;
    }

    // Get User Sinks
    private Sinks.Many<String> getUserSink(UUID userId) {
        return userSinks.get(userId);
    }

Then, by calling tryEmitNext on the Sinks.Many<String> associated with this UUID, you can return a string to the client's browser.

userSink.tryEmitNext(jsonMessage);

The above process is automatically performed when accessing the website with window.onload.

4.2.２ Processing when a string is entered and the Submit button is pressed

The process when a search string is entered in the browser and the Submit button is pressed is as follows:
At the time of Submit, you also pass the UUID to /openai-gpt4-sse-submit and access it via POST.

Below is an excerpt from the JavaScript code.

        function submitText() {
            let elements = document.querySelectorAll('#target *');
            elements.forEach(function (element) {
                element.remove();
            });
            const textFieldValue = document.getElementById("inputText").value;
            fetch("/openai-gpt4-sse-submit?userId=" + userId, {
                method: "POST",
                body: textFieldValue,
                headers: {
                    "Content-Type": "text/plain"
                }
            });
        }

On the server side, the user-input string received with @RequestBody is vectorized using text-embedding-ada-002, and then a vector search is performed in PostgreSQL. The search results from PostgreSQL are processed by calling OpenAI's ChatGPT for summarization. Finally, the results are returned to the client's browser using Server Sent Events.

    @PostMapping("/openai-gpt4-sse-submit")
    @ResponseBody
    public void openaiGpt4Sse(@RequestBody String inputText, @RequestParam UUID userId) {
        var userSink = getUserSink(userId);
        LOGGER.debug("InputText --------------: {}", inputText);
        // Receive user input and search for documents in the PostgreSQL Vector DB  
        findMostSimilarString(inputText).subscribe(findMostSimilarString -> {
            // Based on the search results of the documents, perform summarization using OpenAI and send the results to the client
            findMostSimilarString.forEach(docSummary -> {
                requestOpenAIToGetSummaryAndSendMessageToClient(docSummary, inputText, userSink);
            });
        });
    }

4.2.3 Points to Consider in Asynchronous Implementation

When implementing with Spring WebFlux's asynchronous processing, all internal processing must be implemented as asynchronous. If you block() during intermediate processing, an error will occur.

For example, when sending a query to PostgreSQL and returning the results, instead of returning them as a List<DocumentSummarizer>, you return them as a Mono<List<DocumentSummarizer>>.

Also, when calling OpenAI, the implementation is done as follows, without blocking any processing:

        // Send a request to OpenAI and send the results to the client
        client.getChatCompletionsStream(OPENAI_MODEL_NAME, new ChatCompletionsOptions(chatMessages))
                .doOnSubscribe(subscription -> {
                    // Send a request-event to create a DIV area in the HTML for displaying the link and result string
                    sendCreateAreaEvent(userSink, docSummary);
                    // Send a request-event in the HTML to display the link
                    sendCreateLinkEvent(userSink, docSummary);
                })
                .subscribe(chatCompletions -> {
                    // Send the results from OpenAI to the client via streaming
                    sendChatCompletionMessages(userSink, docSummary, chatCompletions, inputText);
                }, error -> {
                    LOGGER.error("Error Occurred: {}", error.getMessage());
                    userSink.tryEmitError(error);
                }, () -> {
                    LOGGER.debug("Completed");
                });

When processing asynchronously in a non-blocking manner, there is a possibility that multiple returned results may get mixed up if there are multiple answers. Therefore, to correctly identify which summary result corresponds to which search result, the UUID associated with the document (included in the docSummary) is also sent to the client.

For example, a request is sent in JSON format to create a display area associated with the documentID, in order to separate the display area for each document.

    // Send a request-event in the HTML to create a DIV area for displaying the link and result string
    private void sendCreateAreaEvent(Sinks.Many<String> userSink, DocumentSummarizer docSummary) {
        var documentID = docSummary.id().toString();
        var createArea = new CreateAreaInHTML("create", documentID);
        var gson = new Gson();
        var jsonCreateArea = gson.toJson(createArea);
        LOGGER.debug("jsonCreateArea: {}", jsonCreateArea);
        userSink.tryEmitNext(jsonCreateArea);
        // wait few mill seconds
        intervalToSendClient();
    }

The JavaScript implementation that receives the above JSON is as follows.
In the example below, when the JSON type is received as the string create, an area corresponding to the documentID is created.

                    const json = JSON.parse(data);
                    if (json.type === "create") {
                        var documentId = json.id;
                        // Add a child element under the target
                        addArea(documentId);
                        return;
                    } else if (json.type === "createLink") {
                        var documentId = json.id;
                        var link = json.link;
                        var fileName = json.fileName;
                        var pageNumber = json.pageNumber;
                        // Add a link to the link description part under the child
                        createLink(documentId, link, fileName, pageNumber);
                        return;
                    } else if (json.type === "addMessage") {
                        var documentId = json.id;
                        var content = json.content;
                        // Add text to the text description part under the child
                        addMessage(documentId, content);
                        return;
                    }

By implementing in this way, even when multiple search results are obtained, it is possible to display them separately for each search result.

4.2.4 Points to Consider in Cosmos DB Implementation

The following content is addressed in the create-env-en.sh script and is not particularly important, but it is mentioned here for the sake of information sharing.

To execute a query in Cosmos DB and perform sorting, the following SQL is executed:

"SELECT * FROM c WHERE c.status = 'COMPLETED' ORDER BY c.fileName ASC, c.pageNumber ASC

In Cosmos DB, when sorting with ORDER BY, it is necessary to create an index on the target property. If you do not create one, the query results will not be displayed correctly.

In this case, when creating the Cosmos DB container in the create-env-en.sh script, the cosmos-index-policy.json file is read, and indexes are created for the fileName and pageNumber properties.

The following command is executed in the script.

az cosmosdb sql container create --account-name $COSMOS_DB_ACCOUNT_NAME -g $RESOURCE_GROUP_NAME --database-name $COSMOS_DB_DB_NAME --name $COSMOS_DB_CONTAINER_NAME_FOR_STATUS --partition-key-path "/id"  --throughput 400  --idx @cosmos-index-policy.json

4.2.5 Points to Consider in Spring Data JPA Implementation

Initially, I considered implementing queries for PostgreSQL using Spring Data JPA, but I was unable to handle the pgvector type in PostgreSQL with Spring Data JPA. Therefore, I abandoned the implementation using Spring Data JPA for this project.

Despite trying various approaches, even implementing with Native Query resulted in errors, so I have implemented using standard JDBC.

Additional Notes

At this point, there are still some unimplemented features. For example, I have not yet implemented the deletion-related functions (deleting Blob files matching the UUID of failed file registrations, deleting entries in CosmosDB).

In Conclusion

In conclusion, by using this service, you can search for any internal documents, various research papers, or any PDF files by simply uploading them to Azure Storage. When you actually perform a search, ChatGPT will summarize and display the relevant sections for you.

Furthermore, not only PDFs but also Word and Excel files, as well as other text documents, can be handled by utilizing libraries.

If you are interested in this content, please feel free to give it a try.

How to use the Azure OpenAI Embedding model to find the most relevant documents

Yoshio Terada — Fri, 09 Jun 2023 04:29:21 +0000

1. Introduction

Initially, I had never worked with machine learning concepts like "Embedding", "Integrated", or "Embedded" before, so I was quite puzzled when I came across these terms. I didn't understand what could be done with this model. Therefore, I decided to research and actually use the model to grasp its usefulness. In this entry, I'd like to share my findings with you.

2. High-Accuracy and Affordable Models Have Arrived!

OpenAI has a model called text-embedding-ada-002 for text embedding purposes. By using this model, you can find the most relevant documents at a much lower cost.

There are other embedding models available, such as the Davinci model and a few others. However, text-embedding-ada-002 offers higher accuracy in most processes and is considerably more affordable compared to the others. Therefore, it is highly recommended to use this model for tasks like similarity search.

According to the Azure OpenAI Service Pricing, as of June 8, 2023, there were the following differences in pricing:

Embedding Models	Per 1,000 Tokens
Ada	$0.0004
Babbage	$0.005
Curie	$0.02
Davinci	$0.20

3. What is Embedding?

Before using embedding, it is essential to understand its basic concept; otherwise, we won't know how to use it effectively. In short, the idea of embedding is based on the concept of vectors, which we learned in high school.

During high school, we learned that two arrows are considered the same vector if they point in the same direction and have the same length. By applying this concept, we can find vectors with the closest possible length and direction, which implies that they are the most similar. In the illustration below, the blue line represents an identical vector. If we can find the vector "1" that is closest to this blue line, it would indicate the content most similar to it.

In practice, when you pass a natural language string to the text-embedding-ada-002 model, you will obtain a high-dimensional array (vector) consisting of 1536 floating-point numbers, as shown below.

Example: An excerpt of the result for the string "Please tell me about Azure Blob"



[-0.013197514, -0.025243968, 0.011384923, -0.015929632, -0.006410221, 0.031038966, 
-0.016921926, -0.010776317, -0.0019300125, -0.016300088, 0.01767607, -0.0047100903,
0.009691408, -0.014183193, -0.017001309, -0.014434575, 0.01902559, 0.010961545, 
0.013561356, -0.017371766, -0.007964816, 0.0026841562, 0.0019663966, -0.0019878964, 
-0.025614424, -0.0030298054, 0.020229574, -0.01455365, 0.022703694, -0.02033542, 
0.035696134, -0.002441044, -0.008057429, 0.0061191483, 0.004263558, -0.0025518502,
0.018046526, 0.011411385, 0.0063804523, -0.0021020102, 0.027572552, -0.017967142,
0.0077663567, 0.005361697, -0.0116693815, 0.004524862, -0.043581568, -0.01028017, 

.... Omitted

-0.0017265921, 0.083035186, -0.006205147, -0.008646191, 0.0070651355, -0.019052051, 
0.008374964, 0.024225213, 0.01522841, 0.019951731, -0.006516066, 0.017967142, 
0.0058082296, -0.0053253127, -0.009929558, -0.039109625, -0.031277116, -0.015863478, 
0.011040928, 0.012529369, 0.013012286, 0.022981536, -0.013706892, 0.012965979, 
0.011953839, -0.01903882, 0.015347485, 0.019052051, -0.0046538603, 0.012191989, 
-0.020983716, 0.0078722015, -0.0018605519, -0.02775778, -0.026739024, -0.010359553, 
-0.013918581, -0.011933993, 0.0066814483, 0.005196315, -0.0045744767, -2.7598185E-4, 
0.012251527, -0.018178832, -0.013276898, 0.011709073, -0.022928614, 0.002131779, 
-0.007462053, 0.0044554016]

The method using embeddings involves comparing the vector of the user's input string with pre-saved vectors in a database and searching for the closest ones.

There are several ways to find the closest match, but one of the most commonly known methods is calculating the cosine similarity. When you pass two vectors to calculate cosine similarity, you'll get a result ranging from -1 to 1. This calculation is performed alongside the data stored in the database.

Ultimately, the content with the result closest to 1 is considered the most similar.



Example of a user's input text:
AAA BBB CCC DDD EEE FFF GGG HHH III JJJ

Calculation results of cosine similarity with stored strings in the database:
AAA BBB CCC DDD EEE FFF GGG HHH III JJ  =       0.9942949478994986  <---- Closest to 1 (highest similarity)
AAA BBB KKK LLL MMM NNN OOO PPP QQQ RRR =       0.930036739659776   <---- Next closest to 1
Today, we are presenting at an IT event.=           0.7775105340227892  <---- Furthest from 1

In this way, the user's input query string is vectorized, and by combining it with the pre-saved vectors (arrays) in the database, a similarity search is performed.

4. Points to consider when handling strings

Token limit

The maximum number of tokens (roughly equivalent to the character count) that can be handled by text-embedding-ada-002 is 8192 tokens. Therefore, when dealing with texts exceeding approximately 8000 characters, splitting is necessary.

Preparing strings for processing

As mentioned in the "Replace newlines with a single space" section, it has been confirmed that "the presence of newline characters may result in unexpected outcomes."

Hence, it is recommended to replace newline characters (\n) with a space before sending the message to text-embedding-ada-002.

For example, if you have a text like the one below, replace all newline characters and convert it into a single line string (please refer to the sample code provided).



Visual Studio Code for Java is an open-source code editor provided by Microsoft.

It is an extension on Visual Studio Code that supports the Java programming language.

It provides an environment for Java developers to efficiently write, build, debug, test, and execute code. Visual Studio Code is a lightweight text editor that supports multiple languages and is known for its high extensibility. For Java developers, Visual Studio Code for Java can be used as an excellent development environment.

Visual Studio Code for Java can be used by installing the Java Development Kit (JDK), a basic toolset for compiling and running Java programs.

To start a Java project in Visual Studio Code, first install the JDK, and then install the Java Extension Pack from the Visual Studio Code extension marketplace.

The Java Extension Pack includes features such as Java language support, debugging, testing, and project management.

The main features of Visual Studio Code for Java are as follows:

1. Syntax highlighting: Color-coding for Java code to improve readability.
2. Code completion: Suggests possible code during input, allowing for efficient code writing.
3. Code navigation: Easy jumping to classes, methods, and variables, and searching for definitions.
4. Refactoring: Features to change code structure and names, improving code quality.
5. Debugging: Setting breakpoints, step execution, and variable monitoring.
6. Testing: Supports testing frameworks like JUnit and TestNG, creating, running, and displaying test results.
7. Project management: Supports build tools like Maven and Gradle, managing project configurations and dependencies.
8. Git integration: Integrated with Git for source code version control.

Visual Studio Code for Java offers a wealth of features to improve developer productivity and provides an environment suitable for Java project development. In addition, the Visual Studio Code extension marketplace offers various Java-related extensions that can be added as needed. With these extensions, Java developers can use Visual Studio Code as an integrated development environment.

4. Verification of Operation

4.1 Vector DB options available in Azure

As of June 8, 2023, there are several options for storing vectors in Azure. Please choose the appropriate database according to your needs.

4.2 Vector Search with Azure PostgreSQL Flexible Server

As mentioned earlier, there are various options for Vector DB, and you can choose the one that suits your needs. However, for verification purposes, we have decided to use PostgreSQL Flexible Server in this case. The steps to set up PostgreSQL Flexible Server to handle vectors are described below. Please give it a try if you're interested. If you choose a different persistence destination, please skip this section.

4.2.1 Setting environment variables

To create resources on Azure, please modify and set the following environment variables accordingly.



export RESOURCE_GROUP=PostgreSQL
export LOCATION=eastus
export POSTGRES_SERVER_NAME=yoshiopgsql3
export POSTGRES_USER_NAME=yoterada
export POSTGRES_USER_PASS='!'$(head -c 12 /dev/urandom | base64 | tr -dc '[:alpha:]'| fold -w 8 | head -n 1)$RANDOM
echo "GENERATED PASSWORD: " $POSTGRES_USER_PASS
export POSTGRES_DB_NAME=VECTOR_DB
export SUBSCRIPTION_ID=********-****-****-****-************
export PUBLIC_IP=$(curl ifconfig.io -4)

In the above configuration example, the password is automatically generated and output to the standard output. Please enter your own password or make a note of the generated password.

4.2.2 Installing Azure PostgreSQL Flexible Server

Please execute the following three commands. When you run these commands, the following tasks will be performed:

Install Azure PostgreSQL Flexible Server
Configure the firewall
Create a new database



az postgres flexible-server create --name $POSTGRES_SERVER_NAME \
    -g $RESOURCE_GROUP \
    --location $LOCATION \
    --admin-user $POSTGRES_USER_NAME \
    --admin-password $POSTGRES_USER_PASS \
    --public-access $PUBLIC_IP
    --yes
az postgres flexible-server firewall-rule create \
    -g $RESOURCE_GROUP \
    -n $POSTGRES_SERVER_NAME \
    -r AllowAllAzureIPs \
    --start-ip-address 0.0.0.0 \
    --end-ip-address 255.255.255.255
az postgres flexible-server db create \
    -g $RESOURCE_GROUP \
    -s $POSTGRES_SERVER_NAME \
    -d $POSTGRES_DB_NAME

4.2.3 Configuring Azure PostgreSQL Flexible Server for multi language support

Since the data to be persisted in this case includes Japanese strings, please perform the following settings to enable handling of Japanese UTF-8 within the database.



az postgres flexible-server parameter set \
    -g $RESOURCE_GROUP \
    --server-name $POSTGRES_SERVER_NAME \
    --subscription $SUBSCRIPTION_ID \
    --name lc_monetary --value "ja_JP.utf-8"



az postgres flexible-server parameter set \
    -g $RESOURCE_GROUP \
    --server-name $POSTGRES_SERVER_NAME \
    --subscription $SUBSCRIPTION_ID \
    --name lc_numeric --value "ja_JP.utf-8"



az postgres flexible-server parameter set \
    -g $RESOURCE_GROUP \
    --server-name $POSTGRES_SERVER_NAME \
    --subscription $SUBSCRIPTION_ID \
    --name timezone --value "Asia/Tokyo"

4.2.4 Installing extensions on Azure PostgreSQL Flexible Server

To enable handling of UUID and Vector in PostgreSQL, we will make use of extensions. Please execute the following command.

Note：

Do not leave any space between "VECTOR,UUID-OSSP".



az postgres flexible-server parameter set \
    -g $RESOURCE_GROUP \
    --server-name $POSTGRES_SERVER_NAME \
    --subscription $SUBSCRIPTION_ID \
    --name azure.extensions --value "VECTOR,UUID-OSSP"

4.3 Creating a table to handle Vector in PostgreSQL

Now that the PostgreSQL configuration is complete, execute the following command to connect.



> psql -U $POSTGRES_USER_NAME -d $POSTGRES_DB_NAME \
      -h $POSTGRES_SERVER_NAME.postgres.database.azure.com

Once connected successfully, enable the use of the extensions added earlier within PostgreSQL. Please execute the CREATE EXTENSION command for each, as shown below.



SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.

VECTOR_DB=>
VECTOR_DB=> CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
CREATE EXTENSION
VECTOR_DB=> CREATE EXTENSION IF NOT EXISTS "vector";
CREATE EXTENSION

Finally, create a table to store the Vector data. The Vector information will be stored in the embedding VECTOR(1536) part. For simplicity, we will also save the original text together, and display the original string of the most similar string. In practice, you may want to use a URL as the link destination, or you can join it with another table later if desired.



VECTOR_DB=> CREATE TABLE TBL_VECTOR_TEST(
    id uuid,
    embedding VECTOR(1536),
    origntext varchar(8192),
    PRIMARY KEY (id)
    );
CREATE TABLE

4.4 Creating a Java application

4.4.1 Adding dependencies to the Maven project

To use the Azure OpenAI library, connect to PostgreSQL, and perform data persistence, at least the following dependencies need to be added. Please add them to your pom.xml.



        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-ai-openai</artifactId>
            <version>1.0.0-beta.1</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.postgresql/postgresql -->
        <dependency>
            <groupId>org.postgresql</groupId>
            <artifactId>postgresql</artifactId>
            <version>42.6.0</version>
        </dependency>

4.4.2 Creating and setting the properties file

Please create an application.properties file in the src/main/resources directory and write the property settings information in it.



azure.openai.url=https://YOUR_OWN_AZURE_OPENAI.openai.azure.com
azure.openai.model.name=gpt-4
azure.openai.api.key=************************************

azure.postgresql.jdbcurl=jdbc:postgresql://YOUR_POSTGRESQL.postgres.database.azure.com:5432/VECTOR_DB
azure.postgresql.user=yoterada
azure.postgresql.password=************************************

logging.group.mycustomgroup=com.yoshio3
logging.level.mycustomgroup=DEBUG
logging.level.root=INFO

4.4.3 Implementing the Java program

Finally, implement the Java code.



package com.yoshio3;

import java.io.IOException;
import java.util.Arrays;
import java.util.List;
import java.util.Properties;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.UUID;
import java.util.concurrent.TimeUnit;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.azure.ai.openai.OpenAIClient;
import com.azure.ai.openai.OpenAIClientBuilder;
import com.azure.ai.openai.models.EmbeddingsOptions;
import com.azure.core.credential.AzureKeyCredential;

public class VectorTest {
    private final static Logger LOGGER = LoggerFactory.getLogger(VectorTest.class);

    // Azure OpenAI API KEY
    private String OPENAI_API_KEY = "";
    // Azure OpenAI Instance URL
    private String OPENAI_URL = "";

    private String POSTGRESQL_JDBC_URL = "";
    private String POSTGRESQL_USER = "";
    private String POSTGRESQL_PASSWORD = "";

    private final static List<String> INPUT_DATA = Arrays.asList(
            "Visual Studio Code for Java is an extension for the open-source code editor Visual Studio Code provided by Microsoft, which supports the Java programming language. It offers an environment for Java developers to efficiently write, build, debug, test, and execute code. Visual Studio Code is a lightweight text editor with multi-language support and high extensibility. For Java developers, Visual Studio Code for Java can be an excellent development environment. Visual Studio Code for Java can be used by installing the Java Development Kit (JDK), which is a basic toolset for compiling and running Java programs. To start a Java project in Visual Studio Code, install the JDK and then install the Java Extension Pack from the Visual Studio Code extension marketplace. The Java Extension Pack includes features such as Java language support, debugging, testing, and project management. The main features of Visual Studio Code for Java are as follows: 1. Syntax highlighting: Color-coding Java code to improve readability. 2. Code completion: Suggesting possible code while entering it, enabling efficient code writing.3. Code navigation: Easy jumping to classes, methods, variables, and finding definitions. 4. Refactoring: Changing the structure and names of code to improve its quality. 5. Debugging: Setting breakpoints, stepping through code, and monitoring variables. 6. Testing: Supporting testing frameworks such as JUnit and TestNG, allowing test creation, execution, and result display. 7. Project management: Supporting build tools such as Maven and Gradle, enabling project configuration and dependency management. 8. Git integration: Integrating with Git for source code version control. Visual Studio Code for Java offers a wealth of productivity-enhancing features and provides an environment suitable for Java project development. Additionally, the Visual Studio Code extension marketplace has various Java-related extensions, which can be added as needed. With these extensions, Java developers can use Visual Studio Code as an integrated development environment.",
            "Azure App Service for Java is a fully managed platform on Microsoft's cloud platform Azure, designed for hosting, deploying, and managing Java applications. Azure App Service supports the development and execution of web applications, mobile apps, APIs, and other backend applications, allowing Java developers to quickly deploy and scale their applications. By abstracting infrastructure management, developers can focus on their application code. Azure App Service for Java includes the Java Development Kit (JDK) and a web server, supporting Java runtimes such as Tomcat, Jetty, and JBoss EAP. Furthermore, Azure App Service provides features to support the entire lifecycle of Java applications, such as building CI/CD (Continuous Integration/Continuous Delivery) pipelines, setting up custom domains, managing SSL certificates, and monitoring and diagnosing applications.",
            "Azure Container Apps is a fully managed service on Microsoft's cloud platform Azure for deploying, managing, and scaling container-based applications. Azure Container Apps is suitable for applications that implement microservices architecture, web applications, backend services, and job execution. This service abstracts container orchestration platforms like Kubernetes, freeing developers from infrastructure management and scaling, allowing them to focus on application code. Azure Container Apps deploys applications using Docker container images and provides features such as automatic scaling, rolling updates, and auto-recovery. Moreover, Azure Container Apps is platform-independent, supporting any programming language or framework. Developers can easily deploy applications using the Azure portal, Azure CLI, or CI/CD pipelines such as GitHub Actions.In terms of security, Azure Container Apps offers features like network isolation, private endpoints, and Azure Active Directory (AAD) integration, ensuring the safety of your applications. Additionally, it provides features to support application monitoring and diagnostics, enabling the identification of performance and issues using tools like Azure Monitor and Azure Application Insights.",
            "Azure Cosmos DB is a global, distributed multi-model database service from Microsoft, offering low latency, high availability, and high throughput. Cosmos DB is a NoSQL database that supports multiple data models, including key-value, document, column-family, and graph. This fully managed service caters to various applications and can be used for the development of web, mobile, IoT, and gaming solutions. The key features of Azure Cosmos DB include: Global distribution: Automatically replicates data across multiple geographic regions, providing high availability and low latency. Horizontal scaling: Utilizes partition keys to split data across multiple physical partitions, enabling flexible scaling of throughput and storage capacity. Five consistency models: Choose from five consistency models, ranging from strong to eventual consistency, depending on the consistency requirements of your globally distributed applications. Real-time analytics: Integrates with Azure Synapse Analytics and Azure Functions for real-time data processing and analysis. Additionally, Azure Cosmos DB offers multiple APIs, such as SQL API, MongoDB API, Cassandra API, Gremlin API, and Table API, enabling developers to build applications using familiar APIs. In terms of data security, Cosmos DB provides features like encryption, network isolation, and access control to ensure data protection. Furthermore, Azure Monitor and Azure Application Insights can be utilized to identify database performance and issues.",
            "Azure Kubernetes Service (AKS) is a Kubernetes cluster management service provided by Microsoft that simplifies the deployment, scaling, and operation of containerized applications. As a managed Kubernetes service, AKS automates tedious infrastructure management and update tasks, allowing developers to focus on application development. AKS offers enterprise-grade security, monitoring, and operational management features, and it easily integrates with DevOps pipelines. Additionally, it can work with other Azure services to support flexible application development. The main features and benefits of AKS are as follows: Cluster provisioning and scaling: AKS automates cluster infrastructure management, allowing you to add or remove nodes as needed, ensuring efficient resource usage and operations. Security and access control: AKS provides built-in Azure Active Directory (AD) integration and enables secure cluster access management using role-based access control (RBAC). Integration with CI/CD pipelines: AKS integrates with CI/CD tools such as Azure DevOps and Jenkins, automating application build, test, and deployment processes. Monitoring and logging: AKS integrates with monitoring tools like Azure Monitor, Prometheus, and Grafana, allowing you to monitor cluster performance and resource usage. Logs can be centrally managed through Azure Log Analytics. Networking and storage: AKS uses Azure Virtual Networks (VNet) to run clusters within private networks. It also provides persistent data storage using Azure Storage and Azure Disks. Collaboration with other Azure services: AKS can work with other Azure services such as Azure Functions and Azure Cosmos DB, extending the functionality of your applications. These features enable AKS to reduce the burden of infrastructure management and operations for developers, allowing them to focus on application development. The provision of enterprise-grade security, monitoring, and operational management features ensures a reliable user experience.",
            "Azure Cognitive Services is a cloud-based service that integrates AI capabilities provided by Microsoft, allowing you to easily add artificial intelligence features to applications, websites, and bots. Even without deep learning or machine learning expertise, you can use these features through APIs. Azure Cognitive Services is divided into the following five categories: 1. Vision: Analyzes images and videos, providing features such as facial recognition, image recognition, and Optical Character Recognition (OCR). It includes Computer Vision, Custom Vision, Face API, Form Recognizer, and Video Indexer. 2. Speech: Provides speech-related features such as speech recognition, speech synthesis, and speech translation. It includes Speech Services, Speech Translation, and Speaker Recognition. 3. Language: Offers natural language processing (NLP) capabilities, allowing text analysis, machine translation, document summarization, and keyword extraction. It includes Text Analytics, Language Understanding (LUIS), QnA Maker, and Translation. 4. Decision: Provides features to support decision-making and recommendations, enabling personalized content and actions for individual users. It includes Personalization, Anomaly Detector, and Content Moderator. 5. Web Search: Utilizes Bing's search engine to provide web search, image search, video search, news search, and map search capabilities. It includes Bing Web Search API, Bing Image Search API, Bing Video Search API, Bing News Search API, and Bing Maps API. By combining these AI features, you can improve user experience and enhance the value of applications and services. In addition, Azure Cognitive Services is designed with privacy and security in mind, allowing businesses and developers to use it with confidence.",
            "Azure Container Instances (ACI) is a service provided by Microsoft that allows quick and easy deployment of containers. With ACI, you can run application containers without managing virtual machines or orchestration, reducing the workload on infrastructure management for developers. ACI offers per-second billing, with costs generated based on resource usage. The main features and benefits are as follows: 1. Simple and fast deployment: ACI can deploy containers quickly using Docker container images. Containers running on ACI are also compatible with Docker commands and Kubernetes clusters. 2. No need to manage the operating system: ACI eliminates the need to manage and update the host OS, allowing developers to focus on application development and operations. 3. Seamless scaling: ACI enables flexible scaling of the number of containers, allowing you to increase or decrease resources according to load. 4. Security: ACI uses Azure's security features to protect containers and data, and also offers network isolation. 5. Flexible resource allocation based on requirements: With ACI, you can allocate CPU and memory individually, optimizing resources according to application requirements. 6. Event-driven container execution: ACI can be integrated with services like Azure Functions and Logic Apps to enable event-driven container execution. These features make Azure Container Instances effective for various scenarios, such as short-term workloads, batch processing, and development and testing environments. Additionally, by combining with AKS, you can also achieve container management with orchestration capabilities.",
            "Azure Data Lake Storage (ADLS) is a large-scale data lake solution provided by Microsoft, designed to efficiently store, process, and analyze petabyte-scale data. ADLS is part of Azure Storage and centrally manages unstructured, semi-structured, and structured data, enabling advanced analytics such as big data analysis, machine learning, and real-time analysis. The main features and benefits of ADLS are as follows: 1. Scalability: ADLS provides high scalability that can store petabyte-scale data, flexibly adapting to the growth of data. 2. Performance: ADLS offers optimized performance for large-scale data reads and writes, making it suitable for big data processing and real-time analysis. 3. Security and Compliance: ADLS provides security features such as data encryption, access control, and audit logs, addressing corporate compliance requirements. 4. High Compatibility: ADLS is compatible with the Hadoop Distributed File System (HDFS) and can be integrated with existing Hadoop ecosystems as well as big data analytics tools like Apache Spark and Azure Databricks. 5. Hierarchical Storage: ADLS offers three storage tiers - hot, cool, and archive - providing optimal cost-performance based on data access frequency. 6. Integration of Data Lake and Object Storage: Azure Data Lake Storage Gen2 integrates ADLS and Azure Blob Storage, offering a solution that combines the benefits of large-scale data lakes and object storage. Azure Data Lake Storage is a powerful platform for enterprises to efficiently manage large amounts of data and achieve advanced data processing, such as big data analysis and machine learning. This enables businesses to fully leverage the value of their data, gaining business insights and improving decision-making.",
            "Azure Blob Storage is an object storage service provided by Microsoft, offering a cloud-based solution for storing and managing large amounts of unstructured data. It can securely and scalably store various types of data, such as text, binary data, images, videos, and log files. The main features and benefits are as follows: 1. Scalability: Azure Blob Storage provides high scalability that can store petabyte-scale data, flexibly adapting to the growth of data. 2. Performance: Azure Blob Storage offers high performance for data reads and writes, enabling rapid processing of large amounts of data. 3. Hierarchical Storage: Azure Blob Storage provides three storage tiers - hot, cool, and archive - offering optimal cost-performance based on data access frequency. 4. Security and Compliance: Azure Blob Storage offers security features such as data encryption, access control, and audit logs, addressing corporate compliance requirements. 5. Global Access: Azure Blob Storage leverages Microsoft's Azure data centers, allowing fast and secure data access from anywhere in the world. 6. Integration and Compatibility: Azure Blob Storage can be integrated with Azure Data Lake Storage Gen2, other Azure services, and on-premises systems, enabling centralized data management and analysis. Azure Blob Storage is effectively used in various scenarios, such as web applications, backups, archives, big data analytics, and IoT devices. This enables businesses to fully leverage the value of their data, gaining business insights and improving decision-making.");

    private OpenAIClient client;

    public VectorTest() throws IOException {
        Properties properties = new Properties();
        properties.load(this.getClass().getResourceAsStream("/application.properties"));
        OPENAI_API_KEY = properties.getProperty("azure.openai.api.key");
        OPENAI_URL = properties.getProperty("azure.openai.url");
        POSTGRESQL_JDBC_URL = properties.getProperty("azure.postgresql.jdbcurl");
        POSTGRESQL_USER = properties.getProperty("azure.postgresql.user");
        POSTGRESQL_PASSWORD = properties.getProperty("azure.postgresql.password");

        client = new OpenAIClientBuilder()
                .credential(new AzureKeyCredential(OPENAI_API_KEY))
                .endpoint(OPENAI_URL)
                .buildClient();
    }


    public static void main(String[] args) {
        VectorTest test;
        try {
            test = new VectorTest();
            // Execute the insertion into the database only once.
            // test.insertDataToPostgreSQL();

            // Retrieve similar documents using vector search for the data registered in the DB
            test.findMostSimilarString("Please tell me about Azure Blob Storage");
        } catch (IOException e) {
            LOGGER.error("Error : ", e);
        }
    }

    /**
     * Invoke Azure OpenAI (text-embedding-ada-002)
     */
    private List<Double> invokeTextEmbedding(String originalText) {
        EmbeddingsOptions embeddingsOptions = new EmbeddingsOptions(Arrays.asList(originalText));
        var result = client.getEmbeddings("text-embedding-ada-002", embeddingsOptions);
        var embedding = result.getData().stream().findFirst().get().getEmbedding();
        return embedding;
    }

    private void insertDataToPostgreSQL() {
        try (var connection = DriverManager.getConnection(POSTGRESQL_JDBC_URL, POSTGRESQL_USER, POSTGRESQL_PASSWORD)) {
            var insertSql = "INSERT INTO TBL_VECTOR_TEST (id, embedding, origntext) VALUES (?, ?::vector, ?)";

            for (String originText : INPUT_DATA) {
                // Call the text embedding and obtain the vector array
                List<Double> embedding = invokeTextEmbedding(originText);
                // Sleep for 10 seconds to prevent errors 
            // due to sending a large number of requests in a short period of time
                TimeUnit.SECONDS.sleep(10);

                PreparedStatement insertStatement = connection.prepareStatement(insertSql);
                insertStatement.setObject(1, UUID.randomUUID());
                insertStatement.setArray(2, connection.createArrayOf("double", embedding.toArray()));
                insertStatement.setString(3, originText);
                insertStatement.executeUpdate();
            }
        } catch (SQLException | InterruptedException e) {
            LOGGER.error("Connection failure." + e.getMessage());
        }
    }

    public void findMostSimilarString(String data) {
        try (var connection = DriverManager.getConnection(POSTGRESQL_JDBC_URL, POSTGRESQL_USER, POSTGRESQL_PASSWORD)) {
            // Create a vector array by calling the text embedding for the string the user wants to search
            List<Double> embedding = invokeTextEmbedding(data);
            String array = embedding.toString();
            LOGGER.info("Embedding: \n" + array);

            // Search with the vector array (find the closest string to the user's input)
            String querySql = "SELECT origntext FROM TBL_VECTOR_TEST ORDER BY embedding <-> ?::vector LIMIT 1;";
            PreparedStatement queryStatement = connection.prepareStatement(querySql);
            queryStatement.setString(1, array);
            ResultSet resultSet = queryStatement.executeQuery();
            while (resultSet.next()) {
                String origntext = resultSet.getString("origntext");
                LOGGER.info("Origntext: " + origntext);
            }
        } catch (SQLException e) {
            LOGGER.error("Connection failure." + e.getMessage());
        }
    }
}

Point 1

A list of strings called private final static List<String> INPUT_DATA is defined. This list contains descriptions of the following services, saved as a list of character arrays. As mentioned earlier, if newline characters are included, the results may not be accurate, so all newline characters are replaced with whitespace.

Visual Studio Code
Azure App Service for Java
Azure Container Apps
Azure Cosmos DB
Azure Kubernetes Service
Azure Cognitive Service
Azure Container Instances
Azure Data Lake Storage
Azure Blob Storage

Point 2

In the following invokeTextEmbedding method, the Azure OpenAI Embedded model is called. By providing a string to this method, a list of floating-point numbers is returned as the result.



    private List<Double> invokeTextEmbedding(String originalText) {
        EmbeddingsOptions embeddingsOptions = new EmbeddingsOptions(Arrays.asList(originalText));
        var result = client.getEmbeddings("text-embedding-ada-002", embeddingsOptions);
        var embedding = result.getData().stream().findFirst().get().getEmbedding();
        return embedding;
    }

Point 3

The following method takes one element at a time from the prepared list of strings (INPUT_DATA), calls Azure OpenAI's Embedded, receives a multidimensional array (vector), and then saves it to the database. This process is for inserting test data, so please execute it only once.



    private void insertDataToPostgreSQL() {
    private void insertDataToPostgreSQL() {
        try (var connection = DriverManager.getConnection(POSTGRESQL_JDBC_URL, POSTGRESQL_USER, POSTGRESQL_PASSWORD)) {
            var insertSql = "INSERT INTO TBL_VECTOR_TEST (id, embedding, origntext) VALUES (?, ?::vector, ?)";

            for (String originText : INPUT_DATA) {
                // Call the text embedding and obtain the vector array
                List<Double> embedding = invokeTextEmbedding(originText);
                // Sleep for 10 seconds to prevent errors 
            // due to sending a large number of requests in a short period of time
                TimeUnit.SECONDS.sleep(10);

                PreparedStatement insertStatement = connection.prepareStatement(insertSql);
                insertStatement.setObject(1, UUID.randomUUID());
                insertStatement.setArray(2, connection.createArrayOf("double", embedding.toArray()));
                insertStatement.setString(3, originText);
                insertStatement.executeUpdate();
            }
        } catch (SQLException | InterruptedException e) {
            LOGGER.error("Connection failure." + e.getMessage());
        }
    }

Point 4

Finally, compare the information stored in the database with the information entered by the user, and find the most relevant document.



    public static void main(String[] args) {
        VectorTest test;
        try {
            test = new VectorTest();
            // Retrieve similar documents using vector search for the data registered in the DB
            test.findMostSimilarString("Please tell me about Azure Blob Storage");

In this sample, the string "Please tell me about Azure Blob" is entered in the main() method as shown above. Please note that a multidimensional array is also created by calling invokeTextEmbedding(data); for this string.

That multidimensional array is passed to the following query:

SELECT origntext FROM TBL_VECTOR_TEST ORDER BY embedding <-> ?::vector LIMIT 1;"

Since LIMIT 1 is specified above, only the most relevant document is output. If you want to return multiple results, change this value.

Furthermore, the <-> notation can be changed. In PostgreSQL's pgvector, you can calculate similarity by specifying one of the following three operators. Change the operator as needed.

Operator	Description
<->	Euclidean distance: measures the straight-line distance between two vectors in n-dimensional space
<#>	Negative inner product
<=>	Cosine similarity: measures the cosine of the angle between two vectors

Euclidean distance measures the straight-line distance between two vectors in n-dimensional space, while cosine similarity measures the cosine of the angle between two vectors.

The query result is implemented to return the original text.



    public void findMostSimilarString(String data) {
        try (var connection = DriverManager.getConnection(POSTGRESQL_JDBC_URL, POSTGRESQL_USER, POSTGRESQL_PASSWORD)) {
            // Create a vector array by calling the text embedding for the string the user wants to search
            List<Double> embedding = invokeTextEmbedding(data);
            String array = embedding.toString();
            LOGGER.info("Embedding: \n" + array);

            // Search with the vector array (find the closest string to the user's input)
            String querySql = "SELECT origntext FROM TBL_VECTOR_TEST ORDER BY embedding <-> ?::vector LIMIT 1;";
            PreparedStatement queryStatement = connection.prepareStatement(querySql);
            queryStatement.setString(1, array);
            ResultSet resultSet = queryStatement.executeQuery();
            while (resultSet.next()) {
                String origntext = resultSet.getString("origntext");
                LOGGER.info("Origntext: " + origntext);
            }
        } catch (SQLException e) {
            LOGGER.error("Connection failure." + e.getMessage());
        }
    }

By using multidimensional vector arrays in this way, you can find highly relevant documents.
If you have such use cases, please give it a try.

Reference Information

The reference information is listed below. Please take a look as needed.

MicroProfile Config in Azure Kubernetes Service

Yoshio Terada — Thu, 26 Mar 2020 05:19:42 +0000

1. MicroProfile Config in Azure Kubernetes Service

The application should separates the external system configuration such like the connection information (DB and external HTTP endpoint). Because there is the difference in environment settings between development environment, test environment, and production environment.
If you separate the configuration from the program source code, you can easily change the setting and you can write the settings in a setting file.
By writing to an external setting file, you do not need to edit the application source code to switch the connection destination, and you can use the same source code without the test.

Excluding configuration information is very important for building cloud-native applications.

「Refer：The Twelve Factors III. Config : Store config in the environment」

Twelve-Factor strict separation of config from code.

With MicroProfile Config, the configuration information can be obtained from various locations.
These configuration locations are called ConfigSources, and if the same property is defined in more than one ConfigSource, apply a policy to specify which values are valid.

From Java VM System Properties
OS environment variables
From external configuration files (.properties, .xml)
External data sources such as LDAP, DB, and key-value stores

In some situations, you may want to switch some data sources dynamically. Then, the changed value must use the updated content from the program without restarting the application.

To meet these requirment, MicroProfile Config makes the configured values available immediately after modification.

About Runtime for MicroProfile Config

Microprofile Config specifies only API and it does not include implementation.

The MicroProfile Config implementation is provided separately by each MicroProfile implementation provider like follow.

Overview of MicroProfile Config

MicroProfile Config consists of a few APIs.

List of MicroProfile Config API 1.4

Priority of ConfigSource

MicroProfile Config consists of org.eclipse.microprofile.config.spi.ConfigSource. The ConfigSources has a priority and are sorted according to order. This allows you to overwrite the settings from outside.

By default, there are three default ConfigSources.

System.getProperties() (Priority = 400)
System.getenv() (Priority = 300)
META-INF/microprofile-config.properties file on ClassPath (default priority = 100, priority can be set individually by setting config_ordinal property in each file)

For example,at first you can be specified the default values in the property file when packaging the application, and the values can be overridden later for each deployment in the environment value. *** "The higher the priority, the higher the priority." ***

Obtain the Configuration Value

The MicroProfile Config specification provides two methods for obtain the setting value.

Obtain the setting value programmatic
Obtain the setting value using annotations

1. Obtain the setting value programmatic

The following is an example of creating a Config instance programmatically and obtaining a configuration information.

public class  MyAppWithGetConfigFromProgram {

    public Response invokeMicroserviceWithConfig() {
        // Create Config Instnce
        Config config = ConfigProvider.getConfig();
        // Obtain the URL Endpoint from Config for Microservices A
        String microserviceA = config.getValue("URL_OF_MICROSERVICE_A", String.class);
        // Invoke Microservices A
        return invokeMicroservice(microserviceA);
    }
}

To obtain the configuration information, at first you must get a Config instance. To get a Config instance programmatically, you can get it by calling ConfigProvider#getConfig() method.

(An instance of the Config class is registered with the context class loader after it is created.)

2. Obtain the setting value using annotations (Recommend)

The following is an example of obtaining a Config instance using annotations and get configuration information with @ConfigProperty.

@ApplicationScoped
public class MyAppWithGetConfigFromAnnotation {

    @Inject
    private Config config;

    //The property url.microserviceA must exist in one of the configsources, otherwise a
    //DeploymentException will be thrown.
    @Inject
    @ConfigProperty(name="url.microserviceA")
    private String urlOfMicroserviceA;

    //The following code injects an Optional value of port.microserviceA property.
    @Inject
    @ConfigProperty(name="port.microserviceA")
    private Optional<Integer> portOfMicroserviceA;
}

Sample Application of MicroProfile Config

1. Creating the MicroProfile Config sample project

Please access to the following URL and create the MicroProfile Project?

MicroProfile Starter

After enter the value to the form, you can get the MPConfigSample.zip file by pushing the "DOWNLOAD" button. After unarchive the zip file, you can see like the following directory structure.

.
├── pom.xml
├── readme.md
└── src
    └── main
        ├── java
        │   └── com
        │       └── yoshio3
        │           └── MPConfigSample
        │               ├── HelloController.java
        │               ├── MPConfigSampleRestApplication.java
        │               └── config
        │                   └── ConfigTestController.java
        ├── resources
        │   └── META-INF
        │       └── microprofile-config.properties
        └── webapp
            ├── WEB-INF
            │   └── beans.xml
            └── index.html
11 directories, 8 files

And the sample code of MicroProfile Config is wrote in ConfigTestController.java as follows.

package com.yoshio3.MPConfigSample.config;

import org.eclipse.microprofile.config.Config;
import org.eclipse.microprofile.config.ConfigProvider;
import org.eclipse.microprofile.config.inject.ConfigProperty;

import javax.enterprise.context.RequestScoped;
import javax.inject.Inject;
import javax.ws.rs.GET;
import javax.ws.rs.Path;

@Path("/config")
@RequestScoped
public class ConfigTestController {

    @Inject
    @ConfigProperty(name = "injected.value")
    private String injectedValue;

    @Path("/injected")
    @GET
    public String getInjectedConfigValue() {
        return "Config value as Injected by CDI " + injectedValue;
    }

    @Path("/lookup")
    @GET
    public String getLookupConfigValue() {
        Config config = ConfigProvider.getConfig();
        String value = config.getValue("value", String.class);
        return "Config value from ConfigProvider " + value;
    }
}

The above code is very simple that returns the value in the property as an HTTP response.
If you invoke the HTTP Endpoint like belows, the character string which you specify in Config value will be returned.

$ curl -X GET http://localhost:8080/data/config/injected
$ curl -X GET http://localhost:8080/data/config/lookup

The actual settings are configured in the microprofile-config.properties file under the META_INF directory.

# Location of the Property file
└── src
    └── main
        ├── resources
        │   └── META-INF
        │       └── microprofile-config.properties

The following properties are set by default.

injected.value=Injected value
value=lookup value

2. Build and run the sample project

Build the project and launch the application to confirm the MicroProfile Config behavior.

# Build the Project
$ mvn clean package

# Launch the Application
$ java -jar target/MPConfigSample-microbundle.jar 

......
Payara Micro URLs:
http://192.168.100.7:8080/

'ROOT' REST Endpoints:
GET     /data/application.wadl
GET     /data/config/injected
GET     /data/config/lookup
GET     /data/hello
GET     /openapi/
GET     /openapi/application.wadl
]]
[2020-03-10T22:19:06.610+0900] [] [INFO] [] [PayaraMicro] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1583846346610] [levelValue: 800] Payara Micro  5.194 #badassmicrofish (build 327) ready in 32,755 (ms)
[2020-03-10T22:19:33.646+0900] [] [INFO] [] [javax.enterprise.system.container.web.com.sun.web.security] [tid: _ThreadID=29 _ThreadName=http-thread-pool::http-listener(1)] [timeMillis: 1583846373646] [levelValue: 800] Context path from ServletContext:  differs from path from bundle: /

After the application is started as described above, execute the curl command to confirm the behavior.

If it works correctly, the string of the setting (Injected value, value) obtained from the property file is displayed as shown below.

# Invoke endpoint to the destination which is implemented as annotation
$ curl localhost:8080/data/config/injected
Config value as Injected by CDI Injected value

# Invoke endpoint to the destination which is implemented as programatic
$ curl localhost:8080/data/config/lookup
Config value from ConfigProvider lookup value

In MicroProfile, the setting value of the property file can be overwritten by the system property or environment value. Then you can overwrite the value set in the "microprofile-config.properties" file and confirm that the value set in the environment variable is displayed.

# Configure the Environment Value [Replace the character from .(dot) to _(under bar)]
$ export injected_value="Environment Value"

# Run the app with Java system properties which is configured on the environment variables
$ java -D"$injected_value" -jar target/MPConfigSample-microbundle.jar

# Confirm the Application Behavior
$ curl http://localhost:8080/data/config/injected
Config value as Injected by CDI Environment Value

Note: Although the property file is written in. (Dot) notation, environment variables cannot use. (Dot) notation in some OS environment. Therefore, in the environment variable settings, replace the .(Dot) notation with _(under bar). Conversion is performed automatically inside the implementation.

3. Run the Application on Local Docker Environment

Now the application has been verified in the local environment, next you can run MicroProfile in the local Docker environment.

To create a Docker image of Payara Micro's, please create the following Dockerfile.

FROM payara/micro:5.201

USER payara
WORKDIR ${PAYARA_HOME}

# Deploy Artifact
COPY ./target/MPConfigSample.war $DEPLOY_DIR

CMD ["--nocluster","--deploymentDir", "/opt/payara/deployments", "--contextroot", "app"]

Next, please build and create an image of the container using the above Dockerfile?

Please execute following command?

$ docker build -t tyoshio2002/payara-config-sample:1.0 .

# Console output will be look like follows
Sending build context to Docker daemon  151.2MB
Step 1/5 : FROM payara/micro:5.201
5.201: Pulling from payara/micro
050382585609: Already exists 
59f5185426ac: Already exists 
4d95208cd9c0: Pull complete 
c1409397cf71: Pull complete 
Digest: sha256:3ff92627d0d9b67454ee241cc7d5f2e485e46db81a886c87cf16035df7c80cc8
Status: Downloaded newer image for payara/micro:5.201
 ---> a11a548b0a25
Step 2/5 : USER payara
 ---> Running in cb755e484e79
Removing intermediate container cb755e484e79
 ---> 564283252ae4
Step 3/5 : WORKDIR ${PAYARA_HOME}
 ---> Running in f26dd5cd172c
Removing intermediate container f26dd5cd172c
 ---> f2bf88b18a77
Step 4/5 : COPY ./target/MPConfigSample.war $DEPLOY_DIR
 ---> 1b54373fe95a
Step 5/5 : CMD ["--nocluster","--deploymentDir", "/opt/payara/deployments", "--contextroot", "app"]
 ---> Running in 3eb731eb77c3
Removing intermediate container 3eb731eb77c3
 ---> 1d11549e99b8
Successfully built 1d11549e99b8
Successfully tagged tyoshio2002/payara-config-sample:1.0

After the container image is created, please start the container?

Please execute the following command to start the container?

$ docker run -p 8080:8080 -e injected_value=FOOBAR -it tyoshio2002/payara-config-sample:1.0

# Console output will be look like follows
..... 
[2020-03-11T07:46:59.119+0000] [] [INFO] [] [PayaraMicro] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1583912819119] [levelValue: 800] [[ 
{
    "Instance Configuration": {
        "Host": "3877abb54d57",
        "Http Port(s)": "8080",
        "Https Port(s)": "",
        "Instance Name": "payara-micro",
        "Instance Group": "no-cluster",
        "Deployed": [
            {
                "Name": "MPConfigSample",
                "Type": "war",
                "Context Root": "/app"
            }
        ]
    }
}]]
[2020-03-11T07:46:59.131+0000] [] [INFO] [] [PayaraMicro] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1583912819131] [levelValue: 800] [[
Payara Micro URLs:
http://3877abb54d57:8080/app
'MPConfigSample' REST Endpoints:
GET /app/data/application.wadl
GET /app/data/config/injected
GET /app/data/config/lookup
GET /app/data/hello
]]
[2020-03-11T07:46:59.131+0000] [] [INFO] [] [PayaraMicro] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1583912819131] [levelValue: 800] Payara Micro  5.201 #badassmicrofish (build 512) ready in 31,286 (ms)

Now that the created container has been started, please access to the application running on the container?

This time, the port 8080 inside the container is mapped to the local port 8080 at startup, so you can connect to the container application by accessing port 8080 in the local environment.
Please execute the following command?

$ curl http://localhost:8080/app/data/config/injected
Config value as Injected by CDI FOOBAR

Since we gave an startup argument of the environment variable (-e injected_value = FOOBAR), the character string will be displayed.

4. Run the Application on Azure Kubernetes Service

The basic operation was confirmed in the local Docker environment, so we will also confirm in the Azure Kubernetes Service environment.

Following is the procedure to confirm.

Create a resource group for Azure Container Registry
Create Azure Container Registry
Confirm Azure Container Registry password
Log in to Azure Container Registry and push the image
Confirm the image which is pushed to Azure Container Registrys
Create resource group for Azure Kubernetes Service
Create service principal for Azure Kubernetes Service
Grant ACR read permission so that AKS can reference ACR resources
Create AKS
Install kubectl command and get AKS connection information
Create Deployment YAML
Apply Deployment YAML
Confirm the Pod behavior
Set and update environment variables
Getting configuration information from Kubernetes Config Map
Update Config Map values and restart pod

4.1. Create a resource group for Azure Container Registry

First, please create Azure Container Registry and upload the image of Docker image which is created on local?

So, please create a resource group for the Azure Container Registry.

$ az group create --name WebApp-Containers --location "Japan East"
{
  "id": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp-Containers",
  "location": "japaneast",
  "managedBy": null,
  "name": "WebApp-Containers",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}

4.2. Create Azure Container Registry

Next, Pleasr create the Azure Container Registry with following options?

--name: Specify the name of Container Registry
--resource-group: Specify the name of the Resouce Group which is created in the above step
--sku: Specify one of the following "Basic", "Standard", "Premium"
--admin-enabled: If you specify "true", you can login to the registry by docker command.

$ az acr create --name containerreg4yoshio --resource-group WebApp-Containers --sku Basic --admin-enabled true
{
  "adminUserEnabled": true,
  "creationDate": "2020-03-12T02:27:59.357654+00:00",
  "id": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp-Containers/providers/Microsoft.ContainerRegistry/registries/containerreg4yoshio",
  "location": "japaneast",
  "loginServer": "containerreg4yoshio.azurecr.io",
  "name": "containerreg4yoshio",
  "networkRuleSet": null,
  "policies": {
    "quarantinePolicy": {
      "status": "disabled"
    },
    "retentionPolicy": {
      "days": 7,
      "lastUpdatedTime": "2020-03-12T02:28:01.654662+00:00",
      "status": "disabled"
    },
    "trustPolicy": {
      "status": "disabled",
      "type": "Notary"
    }
  },
  "provisioningState": "Succeeded",
  "resourceGroup": "WebApp-Containers",
  "sku": {
    "name": "Basic",
    "tier": "Basic"
  },
  "status": null,
  "storageAccount": null,
  "tags": {},
  "type": "Microsoft.ContainerRegistry/registries"
}

4.3. Confirm Azure Container Registry password

Next, please confirm the password to connect to Azure Container Registry?

--name: Specify container registry name
--resource-group: Specify the resource group name which is created above

$ az acr credential show --name containerreg4yoshio --resource-group WebApp-Containers
{
  "passwords": [
    {
      "name": "password",
      "value": "4zaIiLk*************+H1XO4AlYFvN"
    },
    {
      "name": "password2",
      "value": "fT03XPs*************Oq2cAZiVHV+L"
    }
  ],
  "username": "containerreg4yoshio"
}

4.4. Log in to Azure Container Registry and push the image

Next, please execute the docker login command to connect to Azure Container Registry?
(For the password, enter the password obtained above.)

Once you have logged in, tag your images with the docker tag command. The new tag is used locally created Docker container image name with the "loginServer" name of the container registry (for example, "containerreg4yoshio.azurecr.io").

Finally, run the docker push command to push the image to Azure Container Registry.

# Login to the Azure Container Registry 
$ docker login containerreg4yoshio.azurecr.io -u containerreg4yoshio
Password: 
Login Succeeded

# Tag of Docker Image
$ docker tag tyoshio2002/payara-config-sample:1.0 containerreg4yoshio.azurecr.io/tyoshio2002/payara-config-sample:1.0

# Push the Docker Image to Azure Container Registry
$ docker push containerreg4yoshio.azurecr.io/tyoshio2002/payara-config-sample:1.0

The push refers to repository [containerreg4yoshio.azurecr.io/tyoshio2002/payara-config-sample]
bbd197848553: Pushed 
ec40a5d738cc: Pushed 
f95fe3528c56: Pushed 
bded2364df91: Pushed 
1bfeebd65323: Pushed 
1.0: digest: sha256:689dbacc212d37afe09c43417bc79d8e241c3fa7b5cf71c27097ef535cf77f76 size: 1368

4.5. Confirm the image which is pushed to Azure Container Registry

Please verify that the image has been successfully pushed to Azure Container Registry?

$ az acr repository list -n containerreg4yoshio -g WebApp-Containers
Argument 'resource_group_name' has been deprecated and will be removed in a future release.
[
  "tyoshio2002/payara-config-sample"
]

$ az acr repository show-tags --name containerreg4yoshio  --repository tyoshio2002/payara-config-sample
[
  "1.0"
]

4.6. Create Resource Group for Azure Kubernetes Service

Next, please create resource group for Azure Kubernetes Service (AKS)?

$ az group create --name MC_yoshio-aks --location "Japan East"
{
  "id": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/MC_yoshio-aks",
  "location": "japaneast",
  "managedBy": null,
  "name": "MC_yoshio-aks",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}
$

4.7. Create service principal for Azure Kubernetes Service

Then please create Service Principal. In this time, there was no assignment for role.

$ az ad sp create-for-rbac --skip-assignment
{
  "appId": "884ac0da-****-****-****-c9e2d3fce495",
  "displayName": "azure-cli-2020-03-13-03-12-59",
  "name": "http://azure-cli-2020-03-13-03-12-59",
  "password": "03a0e079-****-****-****-a760333af0b0",
  "tenant": "72f988bf-****-****-****-2d7cd011db47"
}

4.8. Grant ACR read permission so that AKS can reference ACR resources

When you are using Azure, do not register the connection information to the container registry in Kubernetes (usually register and use the connection information in Secret), use the service principal of Azure Active Directory (Azure AD) of Azure , You can get images of containers that exist in the Azure Container Registry.

Refer: Azure Container Registry authentication by service principal

First, you need to get the ACR resource ID with the az acr show command to grant permissions to the ACR.

Next, use the az role assignment create command to grant acrpull to the service principal (created above) used by AKS for ACR.

--assignee: specifiy appId ("appId": "884ac0da-*--***-c9e2d3fce495",)
--scope: specifiy the ID of the above ACR
--role: acrpull role allows you to pull images from ACR (no security settings in YAML)

$ az acr show --resource-group WebApp-Containers --name containerreg4yoshio --query "id" --output tsv
/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp-Containers/providers/Microsoft.ContainerRegistry/registries/containerreg4yoshio

$ az role assignment create --assignee 884ac0da-****-****-****-c9e2d3fce495 --scope /subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp-Containers/providers/Microsoft.ContainerRegistry/registries/containerreg4yoshio --role acrpull
{
  "canDelegate": null,
  "id": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp-Containers/providers/Microsoft.ContainerRegistry/registries/containerreg4yoshio/providers/Microsoft.Authorization/roleAssignments/72cbc68a-****-****-****-1a66c8568cba",
  "name": "72cbc68a-****-****- ****-1a66c8568cba",
  "principalId": "33472555-****-****-****-31e2064fb702",
  "principalType": "ServicePrincipal",
  "resourceGroup": "WebApp-Containers",
  "roleDefinitionId": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/providers/Microsoft.Authorization/roleDefinitions/7f951dda-****-****-****-43fe172d538d",
  "scope": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp-Containers/providers/Microsoft.ContainerRegistry/registries/containerreg4yoshio",
  "type": "Microsoft.Authorization/roleAssignments"
}

4.9. Create AKS

Finally you can create the AKS with following options.

--resource-group: specify the resource group name created in 4.6
--name: specify the name of the AKS cluster
--kubernetes-version: specify the Kubernetes version of the AKS cluster
--node-vm-size: specifies the VM size
--node-count: specifies the number of Kubernetes worker nodes
--location: specifies East Japan as location
--service-principal: specifies appId which is created on the step 4.7
--client-secret: specifies the password which is created on the step 4.7
--generate-ssh-keys: creates SSH keys (Public, Private) in .ssh directory in home directory.

$ az aks create \
    --resource-group MC_yoshio-aks \
    --name yoshioAKSCluster1164 \
    --kubernetes-version 1.16.4 \
    --node-vm-size Standard_DS2_v2 \
    --node-count 3 \
    --location japaneast \
    --service-principal "884ac0da-****-****-****-c9e2d3fce495" \
    --client-secret  "03a0e079-****-****-****-a760333af0b0" \
    --generate-ssh-keys

Note: The above commands is for development environment and verification environment. In order to build the production environment, you need more options like virtual network, subnet in association with external service, availability zone for availability. Many options need to be specified. See below for details on the az aks create command.

Refer：detail of az aks command

4.10. Install kubectl command and get AKS connection information

In order to manage the AKS, you need to use the kubectl command. If you are working with Kubernetes for the first time, you may not have the kubectl command in your operating environment. If the kubectl command is not installed, execute the az aks install-cli command to obtain it.

After got the kubectl command, you can manage the AKS by using the kubectl command. To connect to the created AKS, you need to get the connection information. Therefore, execute the az aks get-credentials command to obtain connection information (credentials) for AKS.

# Install the kubectl command
$ az aks install-cli
Downloading client to "/usr/local/bin/kubectl" from "https://storage.googleapis.com/kubernetes-release/release/v1.17.4/bin/darwin/amd64/kubectl"
Please ensure that /usr/local/bin is in your search PATH, so the `kubectl` command can be found.
$ which kubectl 
/usr/local/bin/kubectl

# Obtain the connection infor(credential) for AKS
$ az aks get-credentials --resource-group MC_yoshio-aks --name yoshioAKSCluster1164
Merged "yoshioAKSCluster1164" as current context in /Users/yoterada/.kube/config

# Confirm which version of kubernetes is running
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4", GitCommit:"8d8aa39598534325ad77120c120a22b3a990b5ea", GitTreeState:"clean", BuildDate:"2020-03-12T21:03:42Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.4", GitCommit:"224be7bdce5a9dd0c2fd0d46b83865648e2fe0ba", GitTreeState:"clean", BuildDate:"2019-12-13T20:40:52Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}

4.11. Create Deployment YAML

After evaluated the AKS running, in order to run the container on AKS, please create a YAML file for Deployment?

Create the following YAML file and save it as deployment.yaml.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payara-config-service
spec:
  replicas: 2
  selector:
    matchLabels:
      app: payara-config-service
  template:
    metadata:
      labels:
        app: payara-config-service
        version: v1
        stage: develop
    spec:
      securityContext:
        runAsUser: 1000 
      containers:
      - name: payara-config-service
        image: containerreg4yoshio.azurecr.io/tyoshio2002/payara-config-sample:1.0
        resources:
          requests:
            cpu: 100m
            memory: 1Gi
          limits:
            cpu: 200m
            memory: 1Gi

4.12. Apply Deployment YAML

After created the Deployment YAML file, please apply the deployment yaml file to the AKS environment? Please execute the following command to apply the YAML file.

$ kubectl apply -f deployment.yaml 
deployment.apps/payara-config-service created

4.13. Confirm the Pod behavior

If you execute the "kubectl get po" command, you can verify the STATUS is Running when it was deploied correctly.

$ kubectl get po
NAME                                     READY   STATUS    RESTARTS   AGE
payara-config-service-85f5cdd768-42sh8   1/1     Running   0          28s
payara-config-service-85f5cdd768-fhjrh   1/1     Running   0          28s

If the Running status is not showed, please execute the following command to confirm the cause?

$ kubectl describe po payara-config-service-85f5cdd768-42sh8  

Or
$ kubectl logs payara-config-service-85f5cdd768-42sh8

If the status is Running, the application is running. however it is impossible to access to the application from outside of the AKS cluster. Because there is no public endpoint nor public IP addoress like follows (10.244.2.3 is private IP addoress).

$ kubectl get po -o wide
NAME                                     READY   STATUS    RESTARTS   AGE   IP           NODE                                NOMINATED NODE   READINESS GATES
payara-config-service-85f5cdd768-42sh8   1/1     Running   0          12d   10.244.2.3   aks-nodepool1-14395993-vmss000002   <none>           <none>
payara-config-service-85f5cdd768-fhjrh   1/1     Running   0          12d   10.244.1.5   aks-nodepool1-14395993-vmss000000   <none>           <none>

So In order to access to the application, please execute the following port-forard command? Then you can access to the application from localhost.

$ kubectl port-forward payara-config-service-85f5cdd768-42sh8 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

After executed the port-forward command, please open another new terminal and start the curl command (or open new browser) to access the application.
If the text is displayed as shown like below, the application is working properly.

$ curl localhost:8080/app/data/config/injected
Config value as Injected by CDI Injected value

4.14. Set and update environment variables

We could confirm the application is running in the above steps, but since the environment variables have not been set at this time, in this time the value is took from the properties file.
So we will modify the Kubernetes' Deployment YAML to take values from environment variables outside the application without redeploy.

The YAML settings to be added are as follows.

        env:
          - name: injected_value
            value: ENVIRONMENT VALUE

The modified all of YAML file is as follows.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payara-config-service
spec:
  replicas: 2
  selector:
    matchLabels:
      app: payara-config-service
  template:
    metadata:
      labels:
        app: payara-config-service
        version: v1
        stage: develop
    spec:
      securityContext:
        runAsUser: 1000 
      containers:
      - name: payara-config-service
        image: containerreg4yoshio.azurecr.io/tyoshio2002/payara-config-sample:1.0
        env:
          - name: injected_value
            value: ENVIRONMENT VALUE
        resources:
          requests:
            cpu: 100m
            memory: 1Gi
          limits:
            cpu: 200m
            memory: 1Gi

After modified the above YAML file, please apply the YAML file.

$ kubectl apply -f  deployment.yaml 
deployment.apps/payara-config-service configured

After applyed the YAML file, it will launch a new pod with a rolling upgrade. Obtain the name of new Pod and execute the port-foward command to access from locally.

# List of all Pods
$ kubectl get po
NAME                                    READY   STATUS    RESTARTS   AGE
payara-config-service-58cdfd6c7-gfvcl   1/1     Running   0          37s
payara-config-service-58cdfd6c7-hdm6c   1/1     Running   0          48s

# Port Forawrd for local access
$ kubectl port-forward payara-config-service-58cdfd6c7-gfvcl 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

After por-forward for new pod, you can access to the endpoint via curl command or browser.

Then, the environment variables defined in the YAML file will be displayed.

$ curl http://localhost:8080/app/data/config/injected
Config value as Injected by CDI ENVIRONMENT VALUE

4.15. Getting configuration information from Kubernetes Config Map

In Kubernetes, we can manage centrally the configuration separately from the container management (deployment.yaml).
Config Map

Therefore, we can configure the environment variables in the Config Map.

Please create the following Config Map YAML file and save it as configmap.yaml.

apiVersion: v1
kind: ConfigMap
metadata:
  name: microprofile-service-config
data:
  injected_value: "CONFIG MAP VALUE"

Next, Please apply the Config Map?

# Apply the Config Map
$ kubectl apply -f configmap.yaml 
configmap/microprofile-service-config createds

# Confirm the applied Config Map
$ kubectl get cm
NAME                          DATA   AGE
microprofile-service-config   1      37m

Next, Please modify the Deployment YAML file to refer to the value of the Config Map from the Kubernetes Pod.

The modifications point is as follows.

        env:
          - name: injected_value
            valueFrom:
              configMapKeyRef:
                name: microprofile-service-config
                key: injected_value

The modified Deployment YAML is as follows.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payara-config-service
spec:
  replicas: 2
  selector:
    matchLabels:
      app: payara-config-service
  template:
    metadata:
      labels:
        app: payara-config-service
        version: v1
        stage: develop
    spec:
      securityContext:
        runAsUser: 1000 
      containers:
      - name: payara-config-service
        image: containerreg4yoshio.azurecr.io/tyoshio2002/payara-config-sample:1.0
        env:
          - name: injected_value
            valueFrom:
              configMapKeyRef:
                name: microprofile-service-config
                key: injected_value
        resources:
          requests:
            cpu: 100m
            memory: 1Gi
          limits:
            cpu: 200m
            memory: 1Gi

Please apply after modifying Deployment YAML?

After applying, please confirm that STATUS is Running?

Please execute the port-forward to access to the pod.

# Apply the Deployment YAML 
$ kubectl apply -f  deployment.yaml 
deployment.apps/payara-config-service configured

# List of all Pods 
$ kubectl get po
NAME                                     READY   STATUS    RESTARTS   AGE
payara-config-service-68bb7c8dfb-8tcbx   1/1     Running   0          22s
payara-config-service-68bb7c8dfb-f67x6   1/1     Running   0          20s

# Execute the port-forward to access to the Pod from local
$ kubectl port-forward payara-config-service-68bb7c8dfb-8tcbx 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

After executed the port-forward command, please access to the application endpoint. Then you can confirm that the value specified in Config Map (CONFIG MAP VALUE) is displayed.

$ curl http://localhost:8080/app/data/config/injected
Config value as Injected by CDI CONFIG MAP VALUE

Then we could confirm that we can get the value from Config Map of Kubernetes.

4.16. Update Config Map values and restart pod

Finally, please confirm how to update the value of the Config.

Update the value of Config Map like follows which is configured on the step 4.15.

apiVersion: v1
kind: ConfigMap
metadata:
  name: microprofile-service-config
data:
  injected_value: "CONFIG MAP VALUE NEW"

After saved the file, Please apply the updated Config Map?
After applied it, please execute the describe command to confirm the value had change correctly?

# Apply the new Config Map
$ kubectl apply -f configmap.yaml 
configmap/microprofile-service-config configured

# Confirm the value had modified correctly on the Config Map
$ kubectl describe cm microprofile-service-config
Name:         microprofile-service-config
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"v1","data":{"injected_value":"CONFIG MAP VALUE NEW"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"microprofile-...

Data
====
injected_value:
----
CONFIG MAP VALUE NEW
Events:  <none>

After applied the new Config Map, please access the application endpoint? In this time, you may confirm that the new settings are not reflected. Because the Kubernetes Pod read the environment value during the startup time and not automatically reloading. As a result it will show the old configuration information.

curl http://localhost:8080/app/data/config/injected
Config value as Injected by CDI CONFIG MAP VALUE

So you need to restart the pod to reflect the setting information.
In order to restart the pod, plese delete the existing pod? The Kubernetes Cluster will automatically create and launched the new pod. (Because the Pod is guaranteed to start for the number of replicas in Deployment YAML)

After getting the list of all pods and please delete one of the pod and wait for the new pod to be restarted.

# List of all Pod 
$ kubectl get po
NAME                                     READY   STATUS    RESTARTS   AGE
payara-config-service-68bb7c8dfb-8tcbx   1/1     Running   0          4m24s
payara-config-service-68bb7c8dfb-f67x6   1/1     Running   0          4m22s

# Delete one of the Pod
$ kubectl delete po payara-config-service-68bb7c8dfb-8tcbx
pod "payara-config-service-68bb7c8dfb-8tcbx" deleted

# List of all Pod （payara-config-service-68bb7c8dfb-dx7sg is new created pod）
$ kubectl get po
NAME                                     READY   STATUS    RESTARTS   AGE
payara-config-service-68bb7c8dfb-dx7sg   1/1     Running   0          26s
payara-config-service-68bb7c8dfb-f67x6   1/1     Running   0          5m27s

Please execute the port-forwad command to accessed from locally?

$ kubectl port-forward payara-config-service-68bb7c8dfb-dx7sg 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080

When you access to the application endpoint, you can confirm that the updated value (CONFIG MAP VALUE NEW) is displayed .

$ curl http://localhost:8080/app/data/config/injected
Config value as Injected by CDI CONFIG MAP VALUE NEW

Now we could confirm the MicroPrifile Config with AKS.

By taking out the configuration and managing them, we can dynamically change the external connection destination without modifying the source code. This greatly reduces the cost of testing and managing applications, and brings you one step closer to cloud-native apps.

Please enjoy!!

MicroProfile Fault Tolerance について

Yoshio Terada — Tue, 17 Mar 2020 14:27:25 +0000

マイクロサービスでサービスを構築する際、耐障害性を考慮して実装する事がとても重要です。リトライ・ポリシー、バルクヘッド、サーキットブレーカーなどは、マイクロサービスのデザイン・パターンにも定義される、とても重要な概念です。

MicroProfile の Fault Tolerance はこうした、耐障害性のあるサービスを構築するために必要な機能を提供しています。実装は CDI によるアノテーション・ベースで容易に開発ができ、CDI のインターセプターを利用して動作しています (クラスは CDI の Bean として実装しなければなりません)。
これにより、ビジネス・ロジックと Fault Torerance 用の冗長的なコードを分離し、かんたんに実装できるようになっています。

MicroProfile の Fault Tolerance のポリシーは外部設定に外だしして管理することが可能になっており、MicroProfile Config を利用してポリシー管理を行うこともできます。

Fault Tolerance の仕様に含まれる主な機能

Fault Tolerance で提供する機能	使用するアノテーションと概要説明
1. タイムアウト:	@Timeout アノテーションを利用。処理に要する最大時間を定義します
2. リトライ:	@Retryアノテーションを利用。処理に失敗した際のリトライ（再試行）の動作を設定します
3. フォールバック：	@Fallback アノテーションを利用。処理に失敗した際の代替の方法を提供（別メソッドの呼び出し）します
4. バルクヘッド(隔壁)：	@Bulkhead アノテーションを利用。同時実行数を制限します。これにより、高負荷時に単一の処理に負荷が集中してレスポンスが低下し、これを起因としたシステム全体への連鎖的な障害を防ぎます
5. サーキット・ブレーカー:	@CircuitBreaker アノテーションを利用。処理が繰り返して失敗する場合、その処理呼び出しを自動的に即時に失敗しるようにします
6. 非同期：	@Asynchronous アノテーションを利用。処理を非同期にします

基本的に、上記のいずれかのポリシーを適用したい場合（複数の指定も可）、実装するクラス、もしくはメソッドにアノテーションを付加するだけで設定できます。

1. タイムアウト (@Timeout) ポリシー

タイムアウトを設定する事により、処理の完了を待ち続けるのを防ぎます。仮にタイムアウトを設定しない場合、ネットワーク障害や、接続先が高負荷でレスポンスをただちに返せないような場合、呼び出し元の接続プールのワーカー・スレッドが枯渇するなど、呼び出し元にも負荷をかけてしまいかねません。

そこで、複数のマイクロサービスを実装する際、もしくは外部サービスを呼び出すような場合、各サービス間の連携においてタイムアウトを設定します。

@Timeout(400) // 接続タイムアウト値 400ms (0.4 sec)
public Connection getConnectionForServiceA() {
   Connection conn = connectionService();
   return conn;
}

@Timeout のアノテーションはクラス、もしくはメソッドレベルで付加できます。タイムアウト値に達した場合、TimeoutException が送出されます。

2. リトライ (@Retry) ポリシー

軽いネットワーク障害や、接続先からの返信が返ってこないような場合、@Retry アノテーションを使用して、処理呼び出しを再試行できます。

リトライポリシーでは以下を構成できます。

パラメータ	説明
maxRetries:	最大のリトライ回数
delay:	リトライ間隔
delayUnit:	delay のユニット
maxDuration:	再試行を実行する最大期間
durationUnit:	duration ユニット
jitter:	再試行遅延のランダムな変化 (クロック信号のタイミング（もしくは周期）のズレ)
jitterDelayUnit:	jitter ユニット
retryOn:	再試行する失敗 (Exception, Error) を指定
abortOn:	中止する失敗 (Exception, Error) を指定

@Retry アノテーションはクラスレベル、もしくはメソッドレベルで付加可能で、クラスに付加した場合、クラス内に存在する全メソッドに適用されます。メソッドに付加した場合、指定したメソッドだけが対象になります。クラスでアノテーションを付加し、メソッドにも付加した場合は、メソッドで指定した設定が有効になります。

正常に処理が終了した場合、結果を正常に返します。
送出された例外を abortOn で指定した場合、スローされた例外を再送出します
送出された例外を retryOn で指定した場合、メソッド呼び出しが再試行されます
それ以外の場合、送出された例外を再送出します

また、他の Fault Tolerance のアノテーションと共に併用できます。

    /**
     * serviceA() メソッド呼び出しで、例外が送出された場合に、
     * 例外が IOException でない場合は再試行します。
     */
    @Retry(retryOn = Exception.class, abortOn = IOException.class)
    public void invokeService() {
        callServiceA();
    }

    /**
     * 最大再試行回数は90、再試行を実行する最大期間は 1000 ミリ秒に設定
     * 再試行の最大期間に達すると、最大再試行回数に達していない場合でも、再試行は実行されない。
     */
    @Retry(maxRetries = 90, maxDuration= 1000)
    public void serviceB() {
        callServiceB();
    }

    /**
    * クロック周波数のズレ(jitter)を 400ms と仮定した場合、-400ms 〜 400ms つまり
    * 0 (delay - jitter) 〜 800ms (delay + jitter )の差で再試行が行われることが予想されます。
    * 最大遅延が発生した場合を想定し、3200/800=4 で、最低試行回数は４回以上、
    * 最大でも 10 回を超えない試行回数を設定します
    */
    @Retry(delay = 400, maxDuration= 3200, jitter= 400, maxRetries = 10)
    public Connection serviceA() {
        return getConnectionForServiceA();
    }

3. フォールバック (@Fallback) ポリシー

@Fallback アノテーションはメソッドレベルで指定できます。アノテーションが付加されたメソッドで例外が発生し終了した場合、フォールバックメソッドで指定したメソッドが呼び出されます。

@Fallback アノテーションは、単体もしくは他の Fault Tolerance アノテーションと一緒に使用できます。他のアノテーションと併用した場合、フォールバックは、他のすべての Fault Tolerance 処理が行われた後に呼び出されます。

たとえば、@Retry が定義されている場合、リトライが最大試行回数を超えた場合にフォールバックの処理が実行されます。
また、@CircuitBreaker が共に定義されている場合、メソッド呼び出しが失敗した場合に直ちに呼び出されます。そしてサーキットがオープンしている場合は常に、フォールバック・メソッドが呼び出されます。

3.1 FallbackHandler の実装によるフォールバック処理の実装例

FallbackHandler インタフェースを実装した FallbackHandler のクラス (ServiceInvocationAFallbackHandler) を定義します。そして handle メソッド内で代替の処理を実装します。

ここでは、MicroProfile Config を利用して app.serviceinvokeA.FallbackReplyMessage のプロパティ、もしくは環境変数などで定義した文字列を返信するように実装しています。

@Dependent
public class ServiceInvocationAFallbackHandler implements FallbackHandler<String> {

    @ConfigProperty(name="app.serviceinvokeA.FallbackReplyMessage", defaultValue = "Unconfigured Default Reply")
    private String replyString;

    @Override
    public String handle(ExecutionContext ec) {
        return replyString;
    }
}

下記の、BusinessLogicServiceBean#invokeServiceA() メソッドが呼び出されると、ここでは内部的に RuntimeException が発生しますが 3 回処理の再試行します、全ての再試行に失敗したのち、ServiceInvocationAFallbackHandler#handle() が呼び出されます。

@RequestScoped
public class BusinessLogicServiceBean {

    // FallbackHandler の実装クラスを指定し @Fallback アノテーションを付加
    // 最大のリトライ回数 (3回)を超えた場合、FallbackHandler の handle() メソッドが呼ばれる
    @Retry(maxRetries = 3)
    @Fallback(ServiceInvocationAFallbackHandler.class)
    public String invokeServiceA() {
        throw new RuntimeException("Connection failed");
        return null;
    }   
}

3.2 fallbackMethod を指定したフォールバック処理の実装例

@Fallback アノテーション内で直接、代替で呼び出すメソッド名を記述します。

ここでは、fallbackForServiceB() メソッドを代替メソッドとして定義しています。

@RequestScoped
public class BusinessLogicServiceBean {

    @Retry(maxRetries = 3)
    @Fallback(fallbackMethod= "fallbackForServiceB")
    public String invokeServiceB() {
        counterForInvokingServiceB++;
       return nameService();
    }


    @ConfigProperty(name="app.serviceinvokeB.FallbackReplyMessage", defaultValue = "Unconfigured Default Reply")
    private String replyString;

    private String fallbackForInvokeServiceB() {
        return replyString;
    }

4. バルクヘッド (@Bulkhead) ポリシー

バルクヘッド・パターンは、システムの一部の障害がシステム全体に伝播し、システム全体がダウンするのを防ぐために利用します。MicroProfile の実装では、インスタンスにアクセスする同時リクエスト数を制限します。

Bulkheadパターンは、大量に呼び出される可能性のあるコンポーネントや、高負荷時にレスポンス低下を招くようなサービスに対して適用すると効果的です。

@Bulkhead アノテーションはクラスレベル、もしくはメソッドレベルで付加可能で、クラスに付加した場合、クラス内に存在する全メソッドに適用されます。メソッドに付加した場合、指定したメソッドだけが対象になります。クラスでアノテーションを付加し、メソッドにも付加した場合は、メソッドで指定した設定が有効になります。

バルクヘッドには下記の2種類の方法で設定可能です。

スレッド・プールの分離 : (@Asynchronous アノテーションと併用した場合) スレッド・プール内の待機中のキューサイズで最大同時リクエスト数を設定します。
セマフォの分離：(@Asynchronous アノテーションと併用しない場合) 同時リクエスト数の設定のみが許可されます。

4.1 スレッド・プールによる分離例

@Asynchronous アノテーションと併用した場合、スレッド・プールの分離が適用されます。下記の例では、最大で５つの同時リクエストが許可され、８つのリクエストが待機キューで保持されます。

// 最大5つの同時リクエストが許可され、最大 8つのリクエストが待機キューで許可される
@Asynchronous
@Bulkhead(value = 5, waitingTaskQueue = 8)
public Future<Connection> invokeServiceA() {
   Connection conn = null;
   counterForInvokingServiceA++;
   conn = connectionService();
   return CompletableFuture.completedFuture(conn);
}

4.2 セマフォによる分離例

@Asynchronous アノテーションを併用しない場合は、単に同時リクエスト数を定義します。

@Bulkhead(5) // 最大5つの同時要求が許可されます
public Connection invokeServiceA() {
   Connection conn = null;
   counterForInvokingServiceA++;
   conn = connectionService();
   return conn;
}

5. サーキット・ブレーカー (@CircuitBreaker) ポリシー

サーキット・ブレーカーは、障害のあるサービスに対して繰り返しの呼び出しを防いで、障害のあるサービスもしくは API 呼び出しで直ちに失敗するようにします。サービス呼び出しが頻繁に失敗する場合、サーキットブレーカーがオープンし、一定の時間が経過するまでそのサービスへの呼び出しは試行されません。

@CircuitBreaker アノテーションはクラスレベル、もしくはメソッドレベルで付加可能で、クラスに付加した場合、クラス内に存在する全メソッドに適用されます。メソッドに付加した場合、指定したメソッドだけが対象になります。クラスでアノテーションを付加し、メソッドにも付加した場合は、メソッドで指定した設定が有効になります。

サーキットブレーカーの3つの状態

クローズド：　(通常時)

通常サーキットブレーカーは閉じています。サーキットブレーカーは、各呼び出しが成功したか失敗したかを記録しており最新の結果を追跡します。障害の割合が failureRatio を超えると、サーキット・ブレーカーがオープンします。

オープン： (障害発生時)

サーキットブレーカーが開いている場合、サーキットブレーカーで動作しているサービスへの呼び出しは、CircuitBreakerOpenException で直ちに失敗します。しばらくした後（設定可能）、サーキットブレーカーはハーフ・オープン状態に移行します。

ハーフ・オープン： (障害復旧の確認中)

ハーフオープン状態では、サービス呼び出しの試行が始まります(設定可能な数)。仮にいずれかの呼び出しで障害が発生した場合、再度サーキットブレーカはオープン状態に戻ります。すべての試行が成功した場合、サーキットブレーカーはクローズド状態に移行します。

サーキット・ブレーカの実装例1

@CircuitBreaker(successThreshold = 10, requestVolumeThreshold = 4, failureRatio=0.5, delay = 1000)
public Connection serviceA() {
   Connection conn = null;
   counterForInvokingServiceA++;
   conn = connectionService();
   return conn;
}

パラメータ	説明
requestVolumeThreshold：	サーキットブレーカーが「クローズ」のときに使用するローリングウィンドウ(障害比率を計算するための分母の数)のサイズ
failureRatio：	サーキットブレーカーを「オープン」にするための、ローリングウィンドウ内の障害比率
successThreshold：	サーキットブレーカーが「ハーフ・オープン」の時、クローズドに移行するための試行回数
delayおよびdelayUnit：	サーキットブレーカーを「オープン」にしつづける時間

上記では、requestVolumeThreshold で指定したローリングウィンドウ数である 4 回の連続した呼び出し中に 2 回（4 x 0.5）の障害が発生すると、サーキットが「オープン」します。サーキットは 1,000 ミリ秒間「オープン」のままになり、その後「ハーフ・オープン」に移ります。「ハーフ・オープン」で 10 回呼び出しが成功すると、サーキットは再び「クローズ」になります。

リクエスト1-成功
リクエスト2-失敗
リクエスト3-成功
リクエスト4-成功
リクエスト5-失敗
リクエスト6-CircuitBreakerOpenException

上記のリクエストの場合、最後の4つのリクエストのうち2つが失敗し、failureRatio が 0.5 に達するため、「リクエスト5」でサーキットが「オープン」になり CircuitBreakerOpenException が送出されます。

成功/失敗とみなす例外定義を追加

failOn パラメーターと skipOn パラメーターは、サーキットブレーカーを「オープン」にするか否かを決定するため、どの例外を失敗と見なすかを定義するために使用します。

@CircuitBreaker(successThreshold = 10, requestVolumeThreshold = 4, failureRatio=0.5, delay = 1000,failOn = {ExceptionA.class, ExceptionB.class}, skipOn = ExceptionBSub.class))
public Connection serviceA() {
   Connection conn = null;
   counterForInvokingServiceA++;
   conn = connectionService();
   return conn;
}

failOn に指定した例外が発生した場合は、失敗とみなします
skipOn に指定した例外が発生した場合は、成功とみなします

6. 非同期 (@Asynchronous) ポリシー

Fault Tolerance の主な機能は Architecture に記述されているように、上記 1-5 までにあげた機能です。そこで、非同期処理は直接 Fault Tolerance と関連するわけではありません。しかし、分散処理において非同期処理はとても重要で Fault Tolerance の各種機能と組み合わせる事により、より有効的に働くため仕様内に取り込まれました。

引用元：Architecture

As mentioned above, the Fault Tolerance specification is to focus on the following aspects:

Timeout: Define a duration for timeout

Retry: Define a criteria on when to retry

Fallback: provide an alternative solution for a failed execution.

CircuitBreaker: offer a way of fail fast by automatically failing execution to prevent the system overloading and indefinite wait or timeout by the clients.

Bulkhead: isolate failures in part of the system while the rest part of the system can still function.

@Asynchronousアノテーションはクラスレベル、もしくはメソッドレベルで付加可能で、クラスに付加した場合、クラス内に存在する全メソッドに適用されます。メソッドに付加した場合、指定したメソッドだけが対象になります。クラスでアノテーションを付加し、メソッドにも付加した場合は、メソッドで指定した設定が有効になります。

@Asynchronous アノテーションが付加されたメソッドが呼び出されると、すぐに Future もしくは CompletionStage を返します。残りのメソッド本体の処理は別スレッドで実行されます。非同期処理が完了するまで、返却された Future もしくは CompletionStage は正しい値を持ちません。仮に処理中に例外が発生した場合は、Future または CompletionStage はその例外で終了します。
処理が正常に完了した場合、Future もしくは CompletionStage は戻り値（それ自体がFutureまたはCompletionStage）を返します。

@Asynchronous
public CompletionStage <Connection> serviceA（）{
   Connection conn = null;
   counterForInvokingServiceA ++;
   conn = connectionService（）;
   return CompletableFuture.completedFuture（conn）;
}

上記の例では、serviceA メソッドへの呼び出しが非同期処理になります。serviceA の呼び出しはCompletionStage を返し、メソッド本体の実行は別スレッドで実行されます。

注意：

CDI の RequestScope から @Asynchronous を呼び出す場合、非同期メソッド呼び出し中 RequestScope がアクティブでなければなりません。@Asynchronousアノテーションが付加されたメソッドは、java.util.concurrentパッケージの Future もしくは CompletionStage を返す必要があります。そうでない場合、FaultToleranceDefinitionExceptionが発生します。

ソースコードに記載した設定値の上書き方法

各節で確認したように、Fault Tolerance のポリシーは一部を除いてほとんどの場合、アノテーションを使用して適用できます。
ソースコードの実装後、仮にアノテーションで実装した値を変更したい場合は、MicroProfile Config を使用して設定値を上書きすることもできます。

アノテーション内のパラメーターは、次の命名規則を使用して、設定プロパティーで上書きできます：

<classname>/<methodname>/<annotation>/<parameter>

たとえば、ある特定のメソッドで指定した Timeout や Retry のアノテーションで指定したパラメータを外部で上書き設定したい場合、MicroProfile Config で下記のように記述します。

com.yoshio3.FaultToleranceService.resilient.ResilienceController/checkTimeout/Timeout/value=2000
com.yoshio3.FaultToleranceService.resilient.ResilienceController/checkTimeout/Retry/maxDuration=3000

仮に、クラス全体に適用したい場合は下記のように、メソッド名の部分を削除してクラス全体に適用することもできます。

com.yoshio3.FaultToleranceService.resilient.ResilienceController/Timeout/value=2000
com.yoshio3.FaultToleranceService.resilient.ResilienceController/Retry/maxDuration=3000

そして、プロジェクト内の全コードに対して同一ルールを適用したい場合は、アノテーションとパラメータ設定だけを記載することもできます。

Timeout/value=2000
Retry/maxDuration=3000

さいごに

ここでは、MicroProfile Fault Tolerance を利用して耐障害性を高めるアプリケーションを構築するためのコードを確認しました。次は、Fault Tolerance を利用したアプリケーションを実際に構築し、Azure 上で耐障害性を持つ複数のサービスを連携をしてみたいと思います。

Azure Kubernetes Service 環境で MicroProfile Config 動作検証

Yoshio Terada — Fri, 13 Mar 2020 06:52:17 +0000

1. Azure Kubernetes Service 環境で MicroProfile Config 動作検証

アプリケーションは、外部システムとの連携のための接続情報 (DB や、外部の HTTP エンドポイント)や、開発環境、テスト環境、本番環境などの環境設定の差を、プログラム・ソースコードから切り離し、外部の設定ファイル等に書き出すことで容易に接続先や設定を切り替えることができます。
また、外部の設定ファイルなどに書き出しておくことで、接続先を切り替えるためにアプリケーションのソースコードの編集やビルドは不要で、同一のソースコードや実行ライブラリを利用できます。

クラウド・ネィティブなアプリケーションを構築していくために、設定情報の外だしはとても重要です。
「参照：The Twelve Factors のIII. 設定 : 設定を環境変数に格納する」

Twelve-Factorは 設定をコードから厳密に分離すること を要求する。

MicroProfile Config を利用すると、設定情報を下記のようなさまざまな場所から取得できます。
これらの設定場所を ConfigSources と呼び、同じプロパティが複数の ConfigSource で定義されている場合、ポリシーを適用しどの値が有効かを指定します。

Java VM のシステム・プロパティから
OS の環境変数
外部構成ファイル (.properties, .xml)から
LDAP, DB, Key-Value ストアなどの外部データそ＝す

また状況によっては、一部のデータソースを動的に切り替えたい場合があります。そして変更した値は、アプリケーションを再起動することなく、プログラム上から更新した内容を利用する必要があります。こうしたニーズに応えるため、MicroProfile Config では、構成した値を変更直後から利用できるようになっています。

MicroProfile Config の実装について

Microprofile Config は API のみを規定しており実装は含まれていません。
MicroProfile Config の実装は、各 MicroProfile の実装プロバイダから個別に提供されています。

MicroProfile Config の概要

MicroProfile Config は数少ない API から構成されています。

MicroProfile Config API 1.4 の一覧

ConfigSource の優先順位

Configは、登録されたorg.eclipse.microprofile.config.spi.ConfigSourceから収集された情報で構成されます。これらのConfigSourceは、順序に従ってソートされます。これにより、外部から重要度の低い設定を上書きできます。

デフォルトでは、3つのデフォルトConfigSourceがあります。

System.getProperties()（優先順位 = 400）
System.getenv()（優先順位= 300）
ClassPath上の META-INF/microprofile-config.properties ファイル（デフォルト優先順位 = 100、各ファイル中に config_ordinal プロパティを設定して個別に優先順位を設定可能）

デフォルト値は、アプリケーションのパッケージ時にファイル内で指定でき、値はデプロイメントごとに後から上書きできます。 「優先順位は値が大きいほど優先されます。」

設定情報の取得例

MicroProfile Config 仕様では、設定値を読み取るために 2種類の方法を用意しています。

プログラム的な設定情報の取得
アノテーションを利用した設定情報の取得

1. プログラム的な設定情報の取得

プログラム的に Config インスタンスを取得し設定情報を取得するサンプルを下記に示します。

public class  MyAppWithGetConfigFromProgram {

    public Response invokeMicroserviceWithConfig() {
        // Config インスタンスの取得
        Config config = ConfigProvider.getConfig();
        // マイクロサービス A の URL を取得
        String microserviceA = config.getValue("URL_OF_MICROSERVICE_A", String.class);
        // マイクロサービス A の呼び出し
        return invokeMicroservice(microserviceA);
    }
}

設定情報を取得するためには、最初に Config インスタンスを取得しなければなりません。
プログラム的に Config インスタンスを取得するためには、ConfigProvider#getConfig() を呼び出して取得できます。

（Config クラスのインスタンスは、生成されたのちコンテキストクラスローダーに登録されます。）

2. アノテーションを利用した設定情報の取得 (推奨)

アノテーションを利用し Config インスタンスを取得し、@ConfigProperty で設定情報を取得するサンプルを下記に示します。

@ApplicationScoped
public class MyAppWithGetConfigFromAnnotation {

    @Inject
    private Config config;

    //The property myprj.some.url must exist in one of the configsources, otherwise a
    //DeploymentException will be thrown.
    @Inject
    @ConfigProperty(name="myprj.some.url")
    private String someUrl;

    //The following code injects an Optional value of myprj.some.port property.
    //Contrary to natively injecting the configured value, this will not lead to a
    //DeploymentException if the value is missing.
    @Inject
    @ConfigProperty(name="myprj.some.port")
    private Optional<Integer> somePort;
}

MicroProfile Config サンプル・アプリケーション

1. MicroProfile Config サンプル・プロジェクトの作成

MicroProfile Starter にアクセスし、MicroProfile のプロジェクトを作成します。

(DOWNLOAD) のリンクを押下すると MPConfigSample.zip ファイルがダウンロードできます。ファイルを展開すると下記のようなファイル・ディレクトリ構成が自動的に生成されています。

.
├── pom.xml
├── readme.md
└── src
    └── main
        ├── java
        │   └── com
        │       └── yoshio3
        │           └── MPConfigSample
        │               ├── HelloController.java
        │               ├── MPConfigSampleRestApplication.java
        │               └── config
        │                   └── ConfigTestController.java
        ├── resources
        │   └── META-INF
        │       └── microprofile-config.properties
        └── webapp
            ├── WEB-INF
            │   └── beans.xml
            └── index.html
11 directories, 8 files

そして、MicroProfile Config のサンプルコードが ConfigTestController.java に下記のように記載されています。

package com.yoshio3.MPConfigSample.config;

import org.eclipse.microprofile.config.Config;
import org.eclipse.microprofile.config.ConfigProvider;
import org.eclipse.microprofile.config.inject.ConfigProperty;

import javax.enterprise.context.RequestScoped;
import javax.inject.Inject;
import javax.ws.rs.GET;
import javax.ws.rs.Path;

@Path("/config")
@RequestScoped
public class ConfigTestController {

    @Inject
    @ConfigProperty(name = "injected.value")
    private String injectedValue;

    @Path("/injected")
    @GET
    public String getInjectedConfigValue() {
        return "Config value as Injected by CDI " + injectedValue;
    }

    @Path("/lookup")
    @GET
    public String getLookupConfigValue() {
        Config config = ConfigProvider.getConfig();
        String value = config.getValue("value", String.class);
        return "Config value from ConfigProvider " + value;
    }
}

上記のコードでは、プロパティに記載された値を HTTP のレスポンスとして返す簡単なコードです。

下記のように HTTP の GET メソッドで呼び出すと return 文で記載される文字列が返ってきます。

$ curl -X GET http://localhost:8080/data/config/injected
$ curl -X GET http://localhost:8080/data/config/lookup

実際の設定内容は、META_INF ディレクトリ配下の microprofile-config.properties ファイルに記載されています。

# プロパティ・ファイルの場所
└── src
    └── main
        ├── resources
        │   └── META-INF
        │       └── microprofile-config.properties

デフォルトで下記のプロパティが設定されています。

# プロパティ・ファイルに設定された値
injected.value=Injected value
value=lookup value

2. サンプル・プロジェクトのビルドと実行

MicroProfile Config の動作確認を行うため、プロジェクトをビルドし、アプリケーションを起動します。

# プロジェクトのビルド
$ mvn clean package

# アプリケーションの実行
$ java -jar target/MPConfigSample-microbundle.jar 

......
Payara Micro URLs:
http://192.168.100.7:8080/

'ROOT' REST Endpoints:
GET     /data/application.wadl
GET     /data/config/injected
GET     /data/config/lookup
GET     /data/hello
GET     /openapi/
GET     /openapi/application.wadl
]]
[2020-03-10T22:19:06.610+0900] [] [情報] [] [PayaraMicro] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1583846346610] [levelValue: 800] Payara Micro  5.194 #badassmicrofish (build 327) ready in 32,755 (ms)
[2020-03-10T22:19:33.646+0900] [] [情報] [] [javax.enterprise.system.container.web.com.sun.web.security] [tid: _ThreadID=29 _ThreadName=http-thread-pool::http-listener(1)] [timeMillis: 1583846373646] [levelValue: 800] Context path from ServletContext:  differs from path from bundle: /

上記のようにアプリケーションが起動したのち、curl コマンドを実行し動作確認を行います。
正しく動作している場合、下記のようにプロパティ・ファイルから取得した設定 (Injected value, value) の文字列が表示されます。

# アノテーションで実装されたエンドポイントへの呼び出し
$ curl localhost:8080/data/config/injected
Config value as Injected by CDI Injected value

# プログラムで実装されたエンドポイントへの呼び出し
$ curl localhost:8080/data/config/lookup
Config value from ConfigProvider lookup value

MicroProfile ではプロパティ・ファイルの設定値をシステム・プロパティで上書き設定することができます。そこで環境変数を設定し、環境変数の値を Java のシステム・プロパティに代入して実行します。すると "microprofile-config.properties" ファイルに設定した値を上書きし、環境変数に設定した値が表示されている事を確認できます。

　

# 環境変数の設定 [.(ドット）を _(アンダーバー）に置き換えて設定]
$ export injected_value="Environment Value"

# 環境変数を Java のシステム・プロパティに設定してアプリを実行
$ java -D"$injected_value" -jar target/MPConfigSample-microbundle.jar

# アプリケーションの動作確認
$ curl http://localhost:8080/data/config/injected
Config value as Injected by CDI Environment Value

ご注意：　properties ファイル中では . (ドット）表記で記載していますが、環境変数は OS によっては . (ドット）表記が使えません。そこで、環境変数の設定では . (ドット）表記箇所を _ (アンダーバー）に置き換えて設定してください。実装内部で自動的に変換をしています。

3. ローカルの Docker 環境での実行

ローカルの環境でアプリケーションの動作確認ができたので、次にローカルの Docker 環境で MicroProfile を動作させます。Payara Micro の Docker コンテナのイメージを作成するため、下記のような Dockerfile を作成してください。

FROM payara/micro:5.201

USER payara
WORKDIR ${PAYARA_HOME}

# Deploy Artifact
COPY ./target/MPConfigSample.war $DEPLOY_DIR

CMD ["--nocluster","--deploymentDir", "/opt/payara/deployments", "--contextroot", "app"]

次に、この Dockerfile を利用してコンテナのイメージを作成します。docker build コマンドを実行しコンテナのイメージを作成してください。

$ docker build -t tyoshio2002/payara-config-sample:1.0 .

# コマンド実行時のコンソール出力例
Sending build context to Docker daemon  151.2MB
Step 1/5 : FROM payara/micro:5.201
5.201: Pulling from payara/micro
050382585609: Already exists 
59f5185426ac: Already exists 
4d95208cd9c0: Pull complete 
c1409397cf71: Pull complete 
Digest: sha256:3ff92627d0d9b67454ee241cc7d5f2e485e46db81a886c87cf16035df7c80cc8
Status: Downloaded newer image for payara/micro:5.201
 ---> a11a548b0a25
Step 2/5 : USER payara
 ---> Running in cb755e484e79
Removing intermediate container cb755e484e79
 ---> 564283252ae4
Step 3/5 : WORKDIR ${PAYARA_HOME}
 ---> Running in f26dd5cd172c
Removing intermediate container f26dd5cd172c
 ---> f2bf88b18a77
Step 4/5 : COPY ./target/MPConfigSample.war $DEPLOY_DIR
 ---> 1b54373fe95a
Step 5/5 : CMD ["--nocluster","--deploymentDir", "/opt/payara/deployments", "--contextroot", "app"]
 ---> Running in 3eb731eb77c3
Removing intermediate container 3eb731eb77c3
 ---> 1d11549e99b8
Successfully built 1d11549e99b8
Successfully tagged tyoshio2002/payara-config-sample:1.0

コンテナのイメージが作成できたのち、コンテナを起動します。下記のコマンドを実行してコンテナを起動してください。

$ docker run -p 8080:8080 -e injected_value=hogehoge -it tyoshio2002/payara-config-sample:1.0

# コマンド実行時のコンソール出力例
..... (中略)
[2020-03-11T07:46:59.119+0000] [] [INFO] [] [PayaraMicro] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1583912819119] [levelValue: 800] [[ 
{
    "Instance Configuration": {
        "Host": "3877abb54d57",
        "Http Port(s)": "8080",
        "Https Port(s)": "",
        "Instance Name": "payara-micro",
        "Instance Group": "no-cluster",
        "Deployed": [
            {
                "Name": "MPConfigSample",
                "Type": "war",
                "Context Root": "/app"
            }
        ]
    }
}]]
[2020-03-11T07:46:59.131+0000] [] [INFO] [] [PayaraMicro] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1583912819131] [levelValue: 800] [[
Payara Micro URLs:
http://3877abb54d57:8080/app
'MPConfigSample' REST Endpoints:
GET /app/data/application.wadl
GET /app/data/config/injected
GET /app/data/config/lookup
GET /app/data/hello
]]
[2020-03-11T07:46:59.131+0000] [] [INFO] [] [PayaraMicro] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1583912819131] [levelValue: 800] Payara Micro  5.201 #badassmicrofish (build 512) ready in 31,286 (ms)

作成したコンテナの起動が完了したので、コンテナ上で動作するアプリケーションに対して接続をします。

今回は、起動時にコンテナ内部の 8080番ポートを、ローカルの 8080番ポートにマッピングしていますので、ローカル環境の 8080 番ポートにアクセスすることで、コンテナのアプリケーションに接続できます。
下記のコマンドを実行してください。

$ curl http://localhost:8080/app/data/config/injected
Config value as Injected by CDI hogehoge

コンテナの起動時に引数として環境変数 (-e injected_value=hogehoge) を与えているため、起動時に入力した文字列が表示されます。

4. Azure Kubernetes Service 環境での実行

ローカルの Docker 環境で動作確認ができたので、Azure Kubernetes Service 環境で動作確認を行います。下記の手順に従い動作確認を行います。

Azure Container Registry 用のリソース・グループを作成
Azure Container Registry を作成
Azure Container Registry のパスワードの確認
Azure Container Registry にログインしイメージを Push
Azure Container Registry に Push したイメージの確認
Azure Kubernetes Service 用のリソース・グループを作成
Azure Kubernetes Service 用のサービス・プリンシパル作成
AKS から ACR のリソースを参照できるように ACR に参照権限を付与
AKS の作成
kubectl コマンドのインストールと AKS 接続情報の取得
Deployment YAML の作成
Deployment YAML の適用
Pod の動作確認
環境変数を設定し更新
Kubernetes の Config Map からの設定情報の取得
Config Map 値の更新と Pod の再起動

4.1. Azure Container Registry 用のリソース・グループを作成

まずは、Azure Container Registry を作成し、ローカルで作成した Docker コンテナのイメージをアップロードします。そこで、Azure Container Registry を作成するためのリソース・グループを作成します。

$ az group create --name WebApp-Containers --location "Japan East"
{
  "id": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp-Containers",
  "location": "japaneast",
  "managedBy": null,
  "name": "WebApp-Containers",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}

4.2. Azure Container Registry を作成

次に、Azure Container Registry を作成します。

--name にコンテナ・レジストリ名を指定します
--resource-group に上記で作成したリソース・グループ名を指定します
--sku は "Basic", "Standard", "Premium" の何れかを指定します
--admin-enabled true に設定する事で、コンテナ・レジストリに docker コマンドでアクセスできるようにします

$ az acr create --name containerreg4yoshio --resource-group WebApp-Containers --sku Basic --admin-enabled true
{
  "adminUserEnabled": true,
  "creationDate": "2020-03-12T02:27:59.357654+00:00",
  "id": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp-Containers/providers/Microsoft.ContainerRegistry/registries/containerreg4yoshio",
  "location": "japaneast",
  "loginServer": "containerreg4yoshio.azurecr.io",
  "name": "containerreg4yoshio",
  "networkRuleSet": null,
  "policies": {
    "quarantinePolicy": {
      "status": "disabled"
    },
    "retentionPolicy": {
      "days": 7,
      "lastUpdatedTime": "2020-03-12T02:28:01.654662+00:00",
      "status": "disabled"
    },
    "trustPolicy": {
      "status": "disabled",
      "type": "Notary"
    }
  },
  "provisioningState": "Succeeded",
  "resourceGroup": "WebApp-Containers",
  "sku": {
    "name": "Basic",
    "tier": "Basic"
  },
  "status": null,
  "storageAccount": null,
  "tags": {},
  "type": "Microsoft.ContainerRegistry/registries"
}

4.3. Azure Container Registry のパスワードの確認

次に、Azure Container Registry に接続するためのパスワードを確認します。

--name にコンテナ・レジストリ名を指定します
--resource-group に上記で作成したリソース・グループ名を指定します

$ az acr credential show --name containerreg4yoshio --resource-group WebApp-Containers
{
  "passwords": [
    {
      "name": "password",
      "value": "4zaIiLk*************+H1XO4AlYFvN"
    },
    {
      "name": "password2",
      "value": "fT03XPs*************Oq2cAZiVHV+L"
    }
  ],
  "username": "containerreg4yoshio"
}

4.4. Azure Container Registry にログインしイメージを Push

次に、docker login コマンドを実行し Azure Container Registry に接続します。

(パスワードは上記で取得したパスワードを入力してください。)

ログインが完了すると、docker tag コマンドでイメージのタグ付けを行います。ローカルで作成した Docker コンテナのイメージ名に、コンテナ・レジストリの "loginServer" 名 ( 例："containerreg4yoshio.azurecr.io") を付け加えた名前でタグ付けします。

最後に、docker push コマンドを実行し、Azure Container Registry にイメージを Push します。

# Azure Container Registry にログイン
$ docker login containerreg4yoshio.azurecr.io -u containerreg4yoshio
Password: 
Login Succeeded

# Docker コンテナのタグ付け
$ docker tag tyoshio2002/payara-config-sample:1.0 containerreg4yoshio.azurecr.io/tyoshio2002/payara-config-sample:1.0

# Azure Container Registry にタグ付けしたイメージを Push
$ docker push containerreg4yoshio.azurecr.io/tyoshio2002/payara-config-sample:1.0

The push refers to repository [containerreg4yoshio.azurecr.io/tyoshio2002/payara-config-sample]
bbd197848553: Pushed 
ec40a5d738cc: Pushed 
f95fe3528c56: Pushed 
bded2364df91: Pushed 
1bfeebd65323: Pushed 
1.0: digest: sha256:689dbacc212d37afe09c43417bc79d8e241c3fa7b5cf71c27097ef535cf77f76 size: 1368

4.5. Azure Container Registry に Push したイメージの確認

Azure Container Registry に正しくイメージが Push されていることを確認します。

$ az acr repository list -n containerreg4yoshio -g WebApp-Containers
Argument 'resource_group_name' has been deprecated and will be removed in a future release.
[
  "tyoshio2002/payara-config-sample"
]

$ az acr repository show-tags --name containerreg4yoshio  --repository tyoshio2002/payara-config-sample
[
  "1.0"
]

4.6. Azure Kubernetes Service 用のリソース・グループを作成

次に Azure Kubernetes Service (AKS) を作成します。まず、AKS を作成するリソース・グループを作成します。

$ az group create --name MC_yoshio-aks --location "Japan East"
{
  "id": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/MC_yoshio-aks",
  "location": "japaneast",
  "managedBy": null,
  "name": "MC_yoshio-aks",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}
$

4.7. Azure Kubernetes Service 用のサービス・プリンシパル作成

まず AKS で利用するサービスプリンシパルを作成します。作成時には既定のロールはアサインしていません。

$ az ad sp create-for-rbac --skip-assignment
{
  "appId": "884ac0da-****-****-****-c9e2d3fce495",
  "displayName": "azure-cli-2020-03-13-03-12-59",
  "name": "http://azure-cli-2020-03-13-03-12-59",
  "password": "03a0e079-****-****-****-a760333af0b0",
  "tenant": "72f988bf-****-****-****-2d7cd011db47"
}

4.8. AKS から ACR のリソースを参照できるように ACR に参照権限を付与

Azure を利用する場合、Kubernetes 内でコンテナ・レジストリへの接続情報を登録せず (通常は Secret 内に接続情報を登録し利用)、Azure の Azure Active Directory (Azure AD) のサービスプリンシパルを利用して、Azure Container Registry 内に存在するコンテナのイメージを取得することができます。

参照：サービスプリンシパルによる Azure Container Registry 認証

まず、ACR に権限を付与するため、az acr show コマンドで ACR のリソース ID を取得します。

次に、az role assignment create コマンドで ACR に対して、AKS で利用するサービスプリンシパル(上記で作成)に対して acrpull を許可します。

--assignee は appId を指定 ("appId": "884ac0da-*--***-c9e2d3fce495",)
--scope は上記 ACR の ID を指定
--role acrpull ロールを指定すると ACR からイメージをプル可能 (YAML 内でセキュリティ設定不要)

$ az acr show --resource-group WebApp-Containers --name containerreg4yoshio --query "id" --output tsv
/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp-Containers/providers/Microsoft.ContainerRegistry/registries/containerreg4yoshio

$ az role assignment create --assignee 884ac0da-****-****-****-c9e2d3fce495 --scope /subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp-Containers/providers/Microsoft.ContainerRegistry/registries/containerreg4yoshio --role acrpull
{
  "canDelegate": null,
  "id": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp-Containers/providers/Microsoft.ContainerRegistry/registries/containerreg4yoshio/providers/Microsoft.Authorization/roleAssignments/72cbc68a-****-****-****-1a66c8568cba",
  "name": "72cbc68a-****-****- ****-1a66c8568cba",
  "principalId": "33472555-****-****-****-31e2064fb702",
  "principalType": "ServicePrincipal",
  "resourceGroup": "WebApp-Containers",
  "roleDefinitionId": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/providers/Microsoft.Authorization/roleDefinitions/7f951dda-****-****-****-43fe172d538d",
  "scope": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp-Containers/providers/Microsoft.ContainerRegistry/registries/containerreg4yoshio",
  "type": "Microsoft.Authorization/roleAssignments"
}

4.9. AKS の作成

最後に、AKS を作成します。

--resource-group は 4.6 で作成したリソース・グループ名を指定します
--name は AKS クラスターの名前を指定します
--kubernetes-version は AKS クラスターの Kubernetes のバージョンを指定します
--node-vm-size は VM サイズを指定します
--node-count は Kubernetes のワーカー・ノードの数を指定します
--location は東日本を指定します
--service-principal は 4.7 のサービスプリンシパル作成時の appId を指定します
--client-secret は 4.7 のサービスプリンシパル作成時の password を指定します
--generate-ssh-keys は作成時に SSH キー(Public,Private)がユーザーディレクトリの .ssh に作成されます

az aks create \
    --resource-group MC_yoshio-aks \
    --name yoshioAKSCluster1164 \
    --kubernetes-version 1.16.4 \
    --node-vm-size Standard_DS2_v2 \
    --node-count 3 \
    --location japaneast \
    --service-principal "884ac0da-****-****-****-c9e2d3fce495" \
    --client-secret  "03a0e079-****-****-****-a760333af0b0" \
    --generate-ssh-keys

注意：上記の AKS の作成時のコマンドは開発環境用、検証用として作成しています。AKS の本番環境を構築するためには、ネットワーク周りで仮想ネットワークを利用したり、外部サービス連携用のサブネットを切ったり、アベイラビリティ・ゾーンとして構築する必要があるため、本番環境構築のためにはより多くのオプション指定が必要になります。az aks create コマンドの詳細は下記をご参照ください。

参考：az aks コマンドの詳細

4.10. kubectl コマンドのインストールと AKS 接続情報の取得

AKS を操作するためには、kubectl コマンドを利用します。初めて Kubernetes を操作する場合、操作環境に kubectl コマンドがインストールされていないかもしれません。kubectl コマンドがインストールされていない場合は az aks install-cli コマンドを実行して入手します。

kubectl コマンドを入手後、kubectl コマンドを利用して AKS を操作します。作成した AKS に接続するためには、AKS への接続情報が必要になります。そこで AKS への接続情報（クレデンシャル）を az aks get-credentials コマンドを実行して取得してください。

# kubectl コマンドのインストール
$ az aks install-cli
Downloading client to "/usr/local/bin/kubectl" from "https://storage.googleapis.com/kubernetes-release/release/v1.17.4/bin/darwin/amd64/kubectl"
Please ensure that /usr/local/bin is in your search PATH, so the `kubectl` command can be found.
$ which kubectl 
/usr/local/bin/kubectl

# AKS の接続情報（クレデンシャル）の取得
$ az aks get-credentials --resource-group MC_yoshio-aks --name yoshioAKSCluster1164
Merged "yoshioAKSCluster1164" as current context in /Users/yoterada/.kube/config

# kubernetes のクライアントとサーバのバージョン確認
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4", GitCommit:"8d8aa39598534325ad77120c120a22b3a990b5ea", GitTreeState:"clean", BuildDate:"2020-03-12T21:03:42Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.4", GitCommit:"224be7bdce5a9dd0c2fd0d46b83865648e2fe0ba", GitTreeState:"clean", BuildDate:"2019-12-13T20:40:52Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}

4.11. Deployment YAML の作成

次に、AKS 上で作成したコンテナを動作させるために、Deployment の YAML を作成します。
下記の YAML ファイルを作成し deployment.yaml として保存してください。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payara-config-service
spec:
  replicas: 2
  selector:
    matchLabels:
      app: payara-config-service
  template:
    metadata:
      labels:
        app: payara-config-service
        version: v1
        stage: develop
    spec:
      securityContext:
        runAsUser: 1000 
      containers:
      - name: payara-config-service
        image: containerreg4yoshio.azurecr.io/tyoshio2002/payara-config-sample:1.0
        resources:
          requests:
            cpu: 100m
            memory: 1Gi
          limits:
            cpu: 200m
            memory: 1Gi

4.12. Deployment YAML の適用

Deployment の YAML ファイルを作成したのち、AKS 環境に Deploy します。下記のコマンドを実行して YAML ファイルを適用してください。

$ kubectl apply -f deployment.yaml 
deployment.apps/payara-config-service created

4.13. Pod の動作確認

正しく、Deployment YAML ファイルが適用されると kubectl get po のコマンドで STATUS が Running になっていることを確認できます。

$ kubectl get po
NAME                                     READY   STATUS    RESTARTS   AGE
payara-config-service-85f5cdd768-42sh8   1/1     Running   0          28s
payara-config-service-85f5cdd768-fhjrh   1/1     Running   0          28s

仮に Running のステータスが表示されていない場合は、下記のコマンドを実行して原因を確認してください

$ kubectl describe po payara-config-service-85f5cdd768-42sh8  

もしくは
$ kubectl logs payara-config-service-85f5cdd768-42sh8

ステータスが Running になっている場合、アプリケーションが動作しています。ローカルの環境からアプリケーションの動作確認を行うために、下記の port-forard コマンドを実行してください。

$ kubectl port-forward payara-config-service-85f5cdd768-42sh8 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

port-forward コマンドを実行した後、別のターミナルを開き curl コマンド、もしくはブラウザを起動しアプリケーションのエンドポイントにアクセスしてください。

下記のように文字列が表示されていればアプリケーションが正しく動作しています。

$ curl localhost:8080/app/data/config/injected
Config value as Injected by CDI Injected value

4.14. 環境変数を設定し更新

上記で、アプリケーションの動作確認ができましたが現時点で環境変数を設定していないため、アプリケーション内の properties ファイルに設定した値が表示されています。

アプリケーション外部の環境変数から値をとるために、Kubernetes の Deployment YAML を修正し再度デプロイしてください。

追記する YAML の設定は下記の箇所です。

        env:
          - name: injected_value
            value: ENVIRONMENT VALUE

修正後の YAML ファイルは下記のようになります。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payara-config-service
spec:
  replicas: 2
  selector:
    matchLabels:
      app: payara-config-service
  template:
    metadata:
      labels:
        app: payara-config-service
        version: v1
        stage: develop
    spec:
      securityContext:
        runAsUser: 1000 
      containers:
      - name: payara-config-service
        image: containerreg4yoshio.azurecr.io/tyoshio2002/payara-config-sample:1.0
        env:
          - name: injected_value
            value: ENVIRONMENT VALUE
        resources:
          requests:
            cpu: 100m
            memory: 1Gi
          limits:
            cpu: 200m
            memory: 1Gi

上記の YAML ファイルを編集した後、YAML ファイルを適用してください。

$ kubectl apply -f  deployment.yaml 
deployment.apps/payara-config-service configured

YAML を適用すると、ローリング・アップグレードで新しい Pod が起動されます。新しく起動した Pod 名を取得し port-foward コマンドを実行し、ローカルからアクセスできるようにしてください。

# Pod の一覧を取得
$ kubectl get po
NAME                                    READY   STATUS    RESTARTS   AGE
payara-config-service-58cdfd6c7-gfvcl   1/1     Running   0          37s
payara-config-service-58cdfd6c7-hdm6c   1/1     Running   0          48s

# Port Forawrd で k8s 上で動作する Pod の Port をローカルにフォワード
$ kubectl port-forward payara-config-service-58cdfd6c7-gfvcl 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

新しい Pod をローカルにフォワードした後、curl コマンドもしくはブラウザからエンドポイントにアクセスしてください。

すると、YAML ファイル中に定義した環境変数が表示されるようになります。

$ curl http://localhost:8080/app/data/config/injected
Config value as Injected by CDI ENVIRONMENT VALUE

4.15. Kubernetes の Config Map からの設定情報の取得

Kubernetes では、コンテナーの管理 (deployment.yaml) から設定を切り離して、一元管理できる Config Map という仕組みが用意されています。そこで、Deployment YAML 中に設定した環境変数の設定を Config Map に外だしして Config Map 内で設定情報を管理するようにします。

下記の Config Map の YAML ファイルを作成し configmap.yaml として保存してください。

apiVersion: v1
kind: ConfigMap
metadata:
  name: microprofile-service-config
data:
  injected_value: "CONFIG MAP VALUE"

次に、Config Map の設定を適用します。

# Config Map を適用
$ kubectl apply -f configmap.yaml 
configmap/microprofile-service-config createds

# Config Map の設定が追加されている事の確認
$ kubectl get cm
NAME                          DATA   AGE
microprofile-service-config   1      37m

次に、Kubernetes の Pod から Config Map の値を参照するように Deployment YAML ファイルの内容を修正します。

修正箇所は下記になります。

        env:
          - name: injected_value
            valueFrom:
              configMapKeyRef:
                name: microprofile-service-config
                key: injected_value

修正した Deployment YAML は下記になります。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payara-config-service
spec:
  replicas: 2
  selector:
    matchLabels:
      app: payara-config-service
  template:
    metadata:
      labels:
        app: payara-config-service
        version: v1
        stage: develop
    spec:
      securityContext:
        runAsUser: 1000 
      containers:
      - name: payara-config-service
        image: containerreg4yoshio.azurecr.io/tyoshio2002/payara-config-sample:1.0
        env:
          - name: injected_value
            valueFrom:
              configMapKeyRef:
                name: microprofile-service-config
                key: injected_value
        resources:
          requests:
            cpu: 100m
            memory: 1Gi
          limits:
            cpu: 200m
            memory: 1Gi

Deployment YAML を修正した後適用してください。

適用した後 STATUS が Running になっていることを確認してください。

再起動した Pod をローカルからアクセスできるように Port Forward してください。

# 修正した Deployment YAML の適用
$ kubectl apply -f  deployment.yaml 
deployment.apps/payara-config-service configured

# 再起動した Pod の一覧を取得
$ kubectl get po
NAME                                     READY   STATUS    RESTARTS   AGE
payara-config-service-68bb7c8dfb-8tcbx   1/1     Running   0          22s
payara-config-service-68bb7c8dfb-f67x6   1/1     Running   0          20s

# ローカルからアクセスするためにポート・フォワードを実行
$ kubectl port-forward payara-config-service-68bb7c8dfb-8tcbx 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

ポート・フォワードを実行した後、アプリケーションのエンドポイントにアクセスしてください。すると Config Map で指定した値 (CONFIG MAP VALUE) が表示されていることを確認できます。

$ curl http://localhost:8080/app/data/config/injected
Config value as Injected by CDI CONFIG MAP VALUE

上記で、Config Map から設定値を取り出しアプリケーションから参照できるようになりました。

4.16. Config Map 値の更新と Pod の再起動

最後に、設定値を変更する方法について確認します。

4.15 で設定した Config Map の値を更新 (CONFIG MAP VALUE NEW") します。

apiVersion: v1
kind: ConfigMap
metadata:
  name: microprofile-service-config
data:
  injected_value: "CONFIG MAP VALUE NEW"

ファイルを保存した後、更新した Config Map を再度適用します。

適用した後 describe コマンドで正しく修正されているかを確認します。

# 修正した Config Map を適用
$ kubectl apply -f configmap.yaml 
configmap/microprofile-service-config configured

# Config Map の値が変わっているか否かの確認
$ kubectl describe cm microprofile-service-config
Name:         microprofile-service-config
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"v1","data":{"injected_value":"CONFIG MAP VALUE NEW"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"microprofile-...

Data
====
injected_value:
----
CONFIG MAP VALUE NEW
Events:  <none>

Config Map の設定を反映した直後、アプリケーションのエンドポイントにアクセスしてみてください。

すると、新しい設定は反映されていない事が確認できます。

これは、Kubernetes の Pod は Config Map に設定した値を起動時にのみ読みにいくためです。Config Map の変更をチェックし自動的に再読み込みは行われないため、すでに起動している Pod にアクセスすると古い設定情報が表示されます。

curl http://localhost:8080/app/data/config/injected
Config value as Injected by CDI CONFIG MAP VALUE

つまり、設定情報を反映するためには Pod の再起動が必要になります。　　
Deployment で Pod をインストールしている場合、Pod の再起動を行うためには対象の Pod を削除すれば自動的に新しい Pod が起動されます。(Deployment YAML 中の replicas の数分だけ Pod は起動することが保証されるため)

そこで、Pod の一覧を取得した後、Pod を delete して新しい Pod が再起動されるのを待ちます。

# Pod の一覧を取得
$ kubectl get po
NAME                                     READY   STATUS    RESTARTS   AGE
payara-config-service-68bb7c8dfb-8tcbx   1/1     Running   0          4m24s
payara-config-service-68bb7c8dfb-f67x6   1/1     Running   0          4m22s

# 特定の Pod を削除
$ kubectl delete po payara-config-service-68bb7c8dfb-8tcbx
pod "payara-config-service-68bb7c8dfb-8tcbx" deleted

# Pod の一覧を取得（payara-config-service-68bb7c8dfb-dx7sg が新しい Pod）
$ kubectl get po
NAME                                     READY   STATUS    RESTARTS   AGE
payara-config-service-68bb7c8dfb-dx7sg   1/1     Running   0          26s
payara-config-service-68bb7c8dfb-f67x6   1/1     Running   0          5m27s

再起動した Pod を Port Forwad してローカルからアクセスできるようにします。

$ kubectl port-forward payara-config-service-68bb7c8dfb-dx7sg 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080

Port Foward 後にアプリケーションのエンドポイントにアクセスすると Config Map で更新した値 (CONFIG MAP VALUE NEW) が表示されていることを確認できます。

$ curl http://localhost:8080/app/data/config/injected
Config value as Injected by CDI CONFIG MAP VALUE NEW

以上で、AKS 環境での MicroProfile Config の動作確認ができました。

設定を外部に出して管理する事で、ソースコードの修正なしに外部の接続先などを動的に変更することができます。これによりアプリケーションのテストや管理などコストが大幅に削減され、クラウド・ネィティブのアプリに一歩近づきます。ぜひ、ご適用ください。

Azure Web App for Containers 環境で MicroProfile Config 動作検証

Yoshio Terada — Thu, 12 Mar 2020 05:51:39 +0000

1. Azure Web App for Containers 環境で MicroProfile Config 動作検証

Twelve-Factorは 設定をコードから厳密に分離すること を要求する。

Java VM のシステム・プロパティから
OS の環境変数
外部構成ファイル (.properties, .xml)から
LDAP, DB, Key-Value ストアなどの外部データそ＝す

MicroProfile Config の実装について

MicroProfile Config の概要

MicroProfile Config は数少ない API から構成されています。

MicroProfile Config API 1.4 の一覧

ConfigSource の優先順位

デフォルトでは、3つのデフォルトConfigSourceがあります。

System.getProperties()（優先順位 = 400）
System.getenv()（優先順位= 300）
ClassPath上の META-INF/microprofile-config.properties ファイル（デフォルト優先順位 = 100、各ファイル中に config_ordinal プロパティを設定して個別に優先順位を設定可能）

設定情報の取得例

MicroProfile Config 仕様では、設定値を読み取るために 2種類の方法を用意しています。

プログラム的な設定情報の取得
アノテーションを利用した設定情報の取得

1. プログラム的な設定情報の取得

プログラム的に Config インスタンスを取得し設定情報を取得するサンプルを下記に示します。

public class  MyAppWithGetConfigFromProgram {

    public Response invokeMicroserviceWithConfig() {
        // Config インスタンスの取得
        Config config = ConfigProvider.getConfig();
        // マイクロサービス A の URL を取得
        String microserviceA = config.getValue("URL_OF_MICROSERVICE_A", String.class);
        // マイクロサービス A の呼び出し
        return invokeMicroservice(microserviceA);
    }
}

2. アノテーションを利用した設定情報の取得 (推奨)

アノテーションを利用し Config インスタンスを取得し、@ConfigProperty で設定情報を取得するサンプルを下記に示します。

@ApplicationScoped
public class MyAppWithGetConfigFromAnnotation {

    @Inject
    private Config config;

    //The property myprj.some.url must exist in one of the configsources, otherwise a
    //DeploymentException will be thrown.
    @Inject
    @ConfigProperty(name="myprj.some.url")
    private String someUrl;

    //The following code injects an Optional value of myprj.some.port property.
    //Contrary to natively injecting the configured value, this will not lead to a
    //DeploymentException if the value is missing.
    @Inject
    @ConfigProperty(name="myprj.some.port")
    private Optional<Integer> somePort;
}

MicroProfile Config サンプル・アプリケーション

1. MicroProfile Config サンプル・プロジェクトの作成

MicroProfile Starter にアクセスし、MicroProfile のプロジェクトを作成します。

.
├── pom.xml
├── readme.md
└── src
    └── main
        ├── java
        │   └── com
        │       └── yoshio3
        │           └── MPConfigSample
        │               ├── HelloController.java
        │               ├── MPConfigSampleRestApplication.java
        │               └── config
        │                   └── ConfigTestController.java
        ├── resources
        │   └── META-INF
        │       └── microprofile-config.properties
        └── webapp
            ├── WEB-INF
            │   └── beans.xml
            └── index.html
11 directories, 8 files

そして、MicroProfile Config のサンプルコードが ConfigTestController.java に下記のように記載されています。

package com.yoshio3.MPConfigSample.config;

import org.eclipse.microprofile.config.Config;
import org.eclipse.microprofile.config.ConfigProvider;
import org.eclipse.microprofile.config.inject.ConfigProperty;

import javax.enterprise.context.RequestScoped;
import javax.inject.Inject;
import javax.ws.rs.GET;
import javax.ws.rs.Path;

@Path("/config")
@RequestScoped
public class ConfigTestController {

    @Inject
    @ConfigProperty(name = "injected.value")
    private String injectedValue;

    @Path("/injected")
    @GET
    public String getInjectedConfigValue() {
        return "Config value as Injected by CDI " + injectedValue;
    }

    @Path("/lookup")
    @GET
    public String getLookupConfigValue() {
        Config config = ConfigProvider.getConfig();
        String value = config.getValue("value", String.class);
        return "Config value from ConfigProvider " + value;
    }
}

$ curl -X GET http://localhost:8080/data/config/injected
$ curl -X GET http://localhost:8080/data/config/lookup

実際の設定内容は、META_INF ディレクトリ配下の microprofile-config.properties ファイルに記載されています。

# プロパティ・ファイルの場所
└── src
    └── main
        ├── resources
        │   └── META-INF
        │       └── microprofile-config.properties

デフォルトで下記のプロパティが設定されています。

# プロパティ・ファイルに設定された値
injected.value=Injected value
value=lookup value

2. サンプル・プロジェクトのビルドと実行

MicroProfile Config の動作確認を行うため、プロジェクトをビルドし、アプリケーションを起動します。

# プロジェクトのビルド
$ mvn clean package

# アプリケーションの実行
$ java -jar target/MPConfigSample-microbundle.jar 

......
Payara Micro URLs:
http://192.168.100.7:8080/

'ROOT' REST Endpoints:
GET     /data/application.wadl
GET     /data/config/injected
GET     /data/config/lookup
GET     /data/hello
GET     /openapi/
GET     /openapi/application.wadl
]]
[2020-03-10T22:19:06.610+0900] [] [情報] [] [PayaraMicro] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1583846346610] [levelValue: 800] Payara Micro  5.194 #badassmicrofish (build 327) ready in 32,755 (ms)
[2020-03-10T22:19:33.646+0900] [] [情報] [] [javax.enterprise.system.container.web.com.sun.web.security] [tid: _ThreadID=29 _ThreadName=http-thread-pool::http-listener(1)] [timeMillis: 1583846373646] [levelValue: 800] Context path from ServletContext:  differs from path from bundle: /

# アノテーションで実装されたエンドポイントへの呼び出し
$ curl localhost:8080/data/config/injected
Config value as Injected by CDI Injected value

# プログラムで実装されたエンドポイントへの呼び出し
$ curl localhost:8080/data/config/lookup
Config value from ConfigProvider lookup value

# 環境変数の設定 [.(ドット）を _(アンダーバー）に置き換えて設定]
$ export injected_value="Environment Value"

# 環境変数を Java のシステム・プロパティに設定してアプリを実行
$ java -D"$injected_value" -jar target/MPConfigSample-microbundle.jar

# アプリケーションの動作確認
$ curl http://localhost:8080/data/config/injected
Config value as Injected by CDI Environment Value

3. ローカルの Docker 環境での実行

FROM payara/micro:5.201

USER payara
WORKDIR ${PAYARA_HOME}

# Deploy Artifact
COPY ./target/MPConfigSample.war $DEPLOY_DIR

CMD ["--nocluster","--deploymentDir", "/opt/payara/deployments", "--contextroot", "app"]

次に、この Dockerfile を利用してコンテナのイメージを作成します。docker build コマンドを実行しコンテナのイメージを作成してください。

$ docker build -t tyoshio2002/payara-config-sample:1.0 .

# コマンド実行時のコンソール出力例
Sending build context to Docker daemon  151.2MB
Step 1/5 : FROM payara/micro:5.201
5.201: Pulling from payara/micro
050382585609: Already exists 
59f5185426ac: Already exists 
4d95208cd9c0: Pull complete 
c1409397cf71: Pull complete 
Digest: sha256:3ff92627d0d9b67454ee241cc7d5f2e485e46db81a886c87cf16035df7c80cc8
Status: Downloaded newer image for payara/micro:5.201
 ---> a11a548b0a25
Step 2/5 : USER payara
 ---> Running in cb755e484e79
Removing intermediate container cb755e484e79
 ---> 564283252ae4
Step 3/5 : WORKDIR ${PAYARA_HOME}
 ---> Running in f26dd5cd172c
Removing intermediate container f26dd5cd172c
 ---> f2bf88b18a77
Step 4/5 : COPY ./target/MPConfigSample.war $DEPLOY_DIR
 ---> 1b54373fe95a
Step 5/5 : CMD ["--nocluster","--deploymentDir", "/opt/payara/deployments", "--contextroot", "app"]
 ---> Running in 3eb731eb77c3
Removing intermediate container 3eb731eb77c3
 ---> 1d11549e99b8
Successfully built 1d11549e99b8
Successfully tagged tyoshio2002/payara-config-sample:1.0

コンテナのイメージが作成できたのち、コンテナを起動します。下記のコマンドを実行してコンテナを起動してください。

$ docker run -p 8080:8080 -e injected_value=hogehoge -it tyoshio2002/payara-config-sample:1.0

# コマンド実行時のコンソール出力例
..... (中略)
[2020-03-11T07:46:59.119+0000] [] [INFO] [] [PayaraMicro] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1583912819119] [levelValue: 800] [[ 
{
    "Instance Configuration": {
        "Host": "3877abb54d57",
        "Http Port(s)": "8080",
        "Https Port(s)": "",
        "Instance Name": "payara-micro",
        "Instance Group": "no-cluster",
        "Deployed": [
            {
                "Name": "MPConfigSample",
                "Type": "war",
                "Context Root": "/app"
            }
        ]
    }
}]]
[2020-03-11T07:46:59.131+0000] [] [INFO] [] [PayaraMicro] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1583912819131] [levelValue: 800] [[
Payara Micro URLs:
http://3877abb54d57:8080/app
'MPConfigSample' REST Endpoints:
GET /app/data/application.wadl
GET /app/data/config/injected
GET /app/data/config/lookup
GET /app/data/hello
]]
[2020-03-11T07:46:59.131+0000] [] [INFO] [] [PayaraMicro] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1583912819131] [levelValue: 800] Payara Micro  5.201 #badassmicrofish (build 512) ready in 31,286 (ms)

$ curl http://localhost:8080/app/data/config/injected
Config value as Injected by CDI hogehoge

コンテナの起動時に引数として環境変数 (-e injected_value=hogehoge) を与えているため、起動時に入力した文字列が表示されます。

4. Azure Web App for Containers 環境での実行

ローカルの Docker 環境で動作確認ができたので、Web App for Containers 環境で動作確認を行います。下記の手順に従い動作確認を行います。

Azure Container Registry 用のリソース・グループを作成
Azure Container Registry を作成
Azure Container Registry のパスワードの確認
Azure Container Registry にログインしイメージを Push
Azure Container Registry に Push したイメージの確認
Web App for Containers 用のリソース・グループを作成
Web App for Containers 用の AppService プランの作成
コンテナ・イメージを指定し Web App for Containers を作成
デプロイしたアプリケーションの動作確認
Web App for Containers のアプリケーション設定の追加
設定変更後のアプリケーションの動作確認

4.1. Azure Container Registry 用のリソース・グループを作成

$ az group create --name WebApp-Containers --location "Japan East"
{
  "id": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp-Containers",
  "location": "japaneast",
  "managedBy": null,
  "name": "WebApp-Containers",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}

4.2. Azure Container Registry を作成

次に、Azure Container Registry を作成します。

--name にコンテナ・レジストリ名を指定します
--resource-group に上記で作成したリソース・グループ名を指定します
--sku は "Basic", "Standard", "Premium" の何れかを指定します
--admin-enabled true に設定する事で、コンテナ・レジストリに docker コマンドでアクセスできるようにします

$ az acr create --name containerreg4yoshio --resource-group WebApp-Containers --sku Basic --admin-enabled true
{
  "adminUserEnabled": true,
  "creationDate": "2020-03-12T02:27:59.357654+00:00",
  "id": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp-Containers/providers/Microsoft.ContainerRegistry/registries/containerreg4yoshio",
  "location": "japaneast",
  "loginServer": "containerreg4yoshio.azurecr.io",
  "name": "containerreg4yoshio",
  "networkRuleSet": null,
  "policies": {
    "quarantinePolicy": {
      "status": "disabled"
    },
    "retentionPolicy": {
      "days": 7,
      "lastUpdatedTime": "2020-03-12T02:28:01.654662+00:00",
      "status": "disabled"
    },
    "trustPolicy": {
      "status": "disabled",
      "type": "Notary"
    }
  },
  "provisioningState": "Succeeded",
  "resourceGroup": "WebApp-Containers",
  "sku": {
    "name": "Basic",
    "tier": "Basic"
  },
  "status": null,
  "storageAccount": null,
  "tags": {},
  "type": "Microsoft.ContainerRegistry/registries"
}

4.3. Azure Container Registry のパスワードの確認

次に、Azure Container Registry に接続するためのパスワードを確認します。

--name にコンテナ・レジストリ名を指定します
--resource-group に上記で作成したリソース・グループ名を指定します

$ az acr credential show --name containerreg4yoshio --resource-group WebApp-Containers
{
  "passwords": [
    {
      "name": "password",
      "value": "4zaIiLk*************+H1XO4AlYFvN"
    },
    {
      "name": "password2",
      "value": "fT03XPs*************Oq2cAZiVHV+L"
    }
  ],
  "username": "containerreg4yoshio"
}

4.4. Azure Container Registry にログインしイメージを Push

次に、docker login コマンドを実行し Azure Container Registry に接続します。

(パスワードは上記で取得したパスワードを入力してください。)

最後に、docker push コマンドを実行し、Azure Container Registry にイメージを Push します。

# Azure Container Registry にログイン
$ docker login containerreg4yoshio.azurecr.io -u containerreg4yoshio
Password: 
Login Succeeded

# Docker コンテナのタグ付け
$ docker tag tyoshio2002/payara-config-sample:1.0 containerreg4yoshio.azurecr.io/tyoshio2002/payara-config-sample:1.0

# Azure Container Registry にタグ付けしたイメージを Push
$ docker push containerreg4yoshio.azurecr.io/tyoshio2002/payara-config-sample:1.0

The push refers to repository [containerreg4yoshio.azurecr.io/tyoshio2002/payara-config-sample]
bbd197848553: Pushed 
ec40a5d738cc: Pushed 
f95fe3528c56: Pushed 
bded2364df91: Pushed 
1bfeebd65323: Pushed 
1.0: digest: sha256:689dbacc212d37afe09c43417bc79d8e241c3fa7b5cf71c27097ef535cf77f76 size: 1368

4.5. Azure Container Registry に Push したイメージの確認

Azure Container Registry に正しくイメージが Push されていることを確認します。

$ az acr repository list -n containerreg4yoshio -g WebApp-Containers
Argument 'resource_group_name' has been deprecated and will be removed in a future release.
[
  "tyoshio2002/payara-config-sample"
]

$ az acr repository show-tags --name containerreg4yoshio  --repository tyoshio2002/payara-config-sample
[
  "1.0"
]

4.6. Web App for Containers 用のリソース・グループを作成

Azure Conginer Registry を作成したので、次に Web App for Containers を作成します。まず、Web App for Containers を作成するリソース・グループを作成します。

$ az group create --name WebApp --location "Japan East"
{
  "id": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp",
  "location": "japaneast",
  "managedBy": null,
  "name": "WebApp",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}

4.7. Web App for Containers 用の AppService プランの作成

次に、Linux 用の AppService プランを作成します。今回は検証環境での動作確認のため、SKU は B1 を利用していますが、環境に応じて適宜、ご選択ください。

--name にAppService プラン名を指定します
--resource-group に上記で作成したリソース・グループ名を指定します
--sku に F1, B1, P1V2, P2V2, P3V2, I1, I2, I3 など稼働させるマシン・価格など適切な SKU (Stock Keeping Unit)を指定します
--is-linux は Linux 環境での構築を指定します。

$ az appservice plan create --name webapp4container --resource-group WebApp --sku B1 --is-linux
{
  "freeOfferExpirationTime": "2020-04-11T02:38:56.873333",
  "geoRegion": "Japan East",
  "hostingEnvironmentProfile": null,
  "hyperV": false,
  "id": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp/providers/Microsoft.Web/serverfarms/webapp4container",
  "isSpot": false,
  "isXenon": false,
  "kind": "linux",
  "location": "Japan East",
  "maximumElasticWorkerCount": 1,
  "maximumNumberOfWorkers": 3,
  "name": "webapp4container",
  "numberOfSites": 0,
  "perSiteScaling": false,
  "provisioningState": "Succeeded",
  "reserved": true,
  "resourceGroup": "WebApp",
  "sku": {
    "capabilities": null,
    "capacity": 1,
    "family": "B",
    "locations": null,
    "name": "B1",
    "size": "B1",
    "skuCapacity": null,
    "tier": "Basic"
  },
  "spotExpirationTime": null,
  "status": "Ready",
  "subscription": "f77aafe8-****-****-****-d0c37687ef70",
  "tags": null,
  "targetWorkerCount": 0,
  "targetWorkerSizeId": 0,
  "type": "Microsoft.Web/serverfarms",
  "workerTierName": null
}

4.8. コンテナ・イメージを指定し Web App for Containers を作成

次に、Azure Container Registry に Push したイメージを利用して Web App for Containers を作成します。

--name に Web App for Containers の名前を指定します
--resource-group に上記で作成したリソース・グループ名を指定します
--plan に上記で作成した AppService プラン名を指定します
--deployment-container-image-name に Azure Container Registry に Push したイメージ名を指定します。

$ az webapp create --resource-group WebApp \ 
                               --plan webapp4container \
                               --name yoshiowebapp \
                               --deployment-container-image-name containerreg4yoshio.azurecr.io/tyoshio2002/payara-config-sample:1.0

No credential was provided to access Azure Container Registry. Trying to look up...
{
  "availabilityState": "Normal",
  "clientAffinityEnabled": true,
  "clientCertEnabled": false,
  "clientCertExclusionPaths": null,
  "cloningInfo": null,
  "containerSize": 0,
  "dailyMemoryTimeQuota": 0,
  "defaultHostName": "yoshiowebapp.azurewebsites.net",
  "enabled": true,
  "enabledHostNames": [
    "yoshiowebapp.azurewebsites.net",
    "yoshiowebapp.scm.azurewebsites.net"
  ],
  "ftpPublishingUrl": "ftp://waws-prod-ty1-***.ftp.azurewebsites.windows.net/site/wwwroot",
  "geoDistributions": null,
  "hostNameSslStates": [
    {
      "hostType": "Standard",
      "ipBasedSslResult": null,
      "ipBasedSslState": "NotConfigured",
      "name": "yoshiowebapp.azurewebsites.net",
      "sslState": "Disabled",
      "thumbprint": null,
      "toUpdate": null,
      "toUpdateIpBasedSsl": null,
      "virtualIp": null
    },
    {
      "hostType": "Repository",
      "ipBasedSslResult": null,
      "ipBasedSslState": "NotConfigured",
      "name": "yoshiowebapp.scm.azurewebsites.net",
      "sslState": "Disabled",
      "thumbprint": null,
      "toUpdate": null,
      "toUpdateIpBasedSsl": null,
      "virtualIp": null
    }
  ],
  "hostNames": [
    "yoshiowebapp.azurewebsites.net"
  ],
  "hostNamesDisabled": false,
  "hostingEnvironmentProfile": null,
  "httpsOnly": false,
  "hyperV": false,
  "id": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp/providers/Microsoft.Web/sites/yoshiowebapp",
  "identity": null,
  "inProgressOperationId": null,
  "isDefaultContainer": null,
  "isXenon": false,
  "kind": "app,linux,container",
  "lastModifiedTimeUtc": "2020-03-12T02:39:50.356666",
  "location": "Japan East",
  "maxNumberOfWorkers": null,
  "name": "yoshiowebapp",
  "outboundIpAddresses": "13.**.***.96,13.**.**.49,13.**.**.66,13.**.**.140,13.**.**.186",
  "possibleOutboundIpAddresses": "13.**.**.96,13.**.**.49,13.**.**.66,13.**.**.140,13.**.**.186,13.**.**.30,13.**.**.70,13.**.**.101,13.**.**.163,13.**.**.200",
  "redundancyMode": "None",
  "repositorySiteName": "yoshiowebapp",
  "reserved": true,
  "resourceGroup": "WebApp",
  "scmSiteAlsoStopped": false,
  "serverFarmId": "/subscriptions/f77aafe8-****-****-****-d0c37687ef70/resourceGroups/WebApp/providers/Microsoft.Web/serverfarms/webapp4container",
  "siteConfig": null,
  "slotSwapStatus": null,
  "state": "Running",
  "suspendedTill": null,
  "tags": null,
  "targetSwapSlot": null,
  "trafficManagerHostNames": null,
  "type": "Microsoft.Web/sites",
  "usageState": "Normal"
}

4.9. デプロイしたアプリケーションの動作確認

Web App for Containers を作成したのち、Web App for Containers のエンドポイントにアクセスし、正しくアプリケーションが動作しているか否かを確認します。

ここでは、環境変数を設定していないため、プロパティで設定した値 (Injected value) が表示されます。

$ curl https://yoshiowebapp.azurewebsites.net/app/data/config/injected
Config value as Injected by CDI Injected value

4.10. Web App for Containers のアプリケーション設定の追加

次に、Web App Config のアプリケーション設定を追加し、injected_value に "Value from Server App Setting" という文字列を設定します。

$ az webapp config appsettings set --resource-group WebApp --name yoshiowebapp --settings injected_value="Value from Server App Setting"
[
  {
    "name": "WEBSITES_ENABLE_APP_SERVICE_STORAGE",
    "slotSetting": false,
    "value": "false"
  },
  {
    "name": "DOCKER_REGISTRY_SERVER_URL",
    "slotSetting": false,
    "value": "containerreg4yoshio.azurecr.io/tyoshio2002"
  },
  {
    "name": "DOCKER_REGISTRY_SERVER_USERNAME",
    "slotSetting": false,
    "value": "containerreg4yoshio"
  },
  {
    "name": "DOCKER_REGISTRY_SERVER_PASSWORD",
    "slotSetting": false,
    "value": null
  },
  {
    "name": "WEBSITES_PORT",
    "slotSetting": false,
    "value": "8080"
  },
  {
    "name": "injected_value",
    "slotSetting": false,
    "value": "Value from Server App Setting"
  }
]

4.11. 設定変更後のアプリケーションの動作確認

最後に、アプリケーション設定で追加した設定が反映されているかを確認します。

$ curl https://yoshiowebapp.azurewebsites.net/app/data/config/injected
Config value as Injected by CDI Value from Server App Setting

上記 4.10 の設定後は、明示的にコンテナを再起動しなくても内部的に再起動が行われ設定が反映されます

以上で、MicroProfile Config を使用したアプリケーションを Azure Web App for Containers 環境で動作させることができました。また Web App for Containers のアプリケーション設定（外部の設定値）をアプリケーションで読み取ることができました。

Use DaemonSet as a cache when referencing large files from each pod

Yoshio Terada — Tue, 21 Jan 2020 09:31:52 +0000

Assuming that the problem that came up when I met a certain customer this time and conducted a hackfest and the solution that I proposed to the customer might be effective if there were similar requirements elsewhere, Share that way here.

Following Use Case will be fit for this approach

It is necessary to read the same huge file (GB class file) from each pod
File is read only, no write (If writing is required, another method is available.)
I want to read a huge file as soon as possible

Note: Please consider the appropriate way by yourself depends on your requirements.

If you encounter the above scenario ,please consider to use the DaemonSet as a cache? Use the DaemonSet to launch one pod on each Node, and each pod in the DaemonSet have a copy of the file. Pods who want to refer to and acquire files is possible for not only reduce the communication across networks by acquiring and referencing copy files from the pod of the DaemonSet, but also can get files in a short time because the file is transfered by local network. (explaining the obvious things and evaluated it).

General Kubernetes volume mounting method

When dealing with persistent volumes in each pods, there is some way to mount the volume on each pods. And many cloud providers provide plugins to handle Persistence Volume on each cloud storage, and there are methods to mount disks in storage from each pod (3 way: ReadWriteOnce, ReadOnlyMany) , ReadWriteMany).

Please refer to the Kubernetes documentation

If you want to mount a disk one-to-one from each pod in Azure (ReadWriteOnce),you can use the Azure Disk. If you want to share the same Disk from multiple pods (ReadOnlyMany, ReadWriteMany), you can use the Azure Files. In this use case, we want to refer to large files from multiple pods, so I selected Azure Files (ReadOnlyMany).

Specifically, you can create Azure Files by following procedure and use Azure Files as a Kubernetes Persistence Volume.

$ export AKS_PERS_STORAGE_ACCOUNT_NAME=myfilestorageaccount
$ export AKS_PERS_RESOURCE_GROUP=Yoshio-Storage
$ export AKS_PERS_LOCATION=japaneast
$ export AKS_PERS_SHARE_NAME=aksshare

$ az storage account create -n $AKS_PERS_STORAGE_ACCOUNT_NAME -g $AKS_PERS_RESOURCE_GROUP -l $AKS_PERS_LOCATION --sku Standard_LRS

$ export AZURE_STORAGE_CONNECTION_STRING=`az storage account show-connection-string -n $AKS_PERS_STORAGE_ACCOUNT_NAME -g $AKS_PERS_RESOURCE_GROUP -o tsv`

$ az storage share create -n $AKS_PERS_SHARE_NAME

$ STORAGE_KEY=$(az storage account keys list --resource-group $AKS_PERS_RESOURCE_GROUP --account-name $AKS_PERS_STORAGE_ACCOUNT_NAME --query "[0].value" -o tsv)

$ kubectl create secret generic azure-secret --from-literal=azurestorageaccountname=$AKS_PERS_STORAGE_ACCOUNT_NAME --from-literal=azurestorageaccountkey=$STORAGE_KEY

Next, please create the Deployment artifact for mounting the volumes in each pods?

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ubuntu
spec:
  replicas: 2
  selector:
    matchLabels:
      app: ubuntu
  template:
    metadata:
      labels:
        app: ubuntu
        version: v1
    spec:
      containers:
      - name: ubuntu
        image: ubuntu
        command:
          - sleep
          - infinity
        volumeMounts:
          - mountPath: "/mnt/azure"
            name: volume
        resources:
          limits:
            memory: 4000Mi
          requests:
            cpu: 1000m
            memory: 4000Mi
        env:
          - name: NODE_IP
            valueFrom:
              fieldRef:
                fieldPath: status.hostIP
      volumes:
        - name: volume
          azureFile:
            secretName: azure-secret
            shareName: aksshare
            readOnly: true

In this time, I configured the memory as 4GB because it copy large files.

Please save the above file as deployment.yaml and execute the following command?

$ kubectl apply -f deployment.yaml

After the pod launches successfully, you can execute the following command to confirm the mount status. And you can see the Azure Files is mounted at /mnt/azure on individual pods.

$ kubectl exec -it ubuntu-884df4bfc-7zgkz mount |grep azure
//**********.file.core.windows.net/aksshare on /mnt/azure 
type cifs (rw,relatime,vers=3.0,cache=strict,username=********,domain=,uid=0,noforceuid,gid=0,noforcegid,addr=40.***.***.76,file_mode=0777,dir_mode=0777,soft,persistenthandles,nounix,serverino,mapposix,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1)

According to the above result, the actual file exists on Files Share (***************.file.core.windows.net/aksshare) of Azure Storage.

In order to download or refer to the files from the /aksshare, it will use the the Samba Protocol.
You can confirm it by using following command.

# date
Wed Jan 15 16:29:03 UTC 2020
# cp /mnt/azure/Kuberenetes-Operation-Demo.mov ~/mount.mov
# date
Wed Jan 15 16:29:54 UTC 2020 (51 Seconds)

According to the above, you can see that it took about 51 seconds to copy about 3 Gb video file into the pod. If multiple pods start up and refer to the same file, the file copy will be repeated over the network. It is inefficiency but normal usage.

In this time, we will use DaemonSet to improve the number of network transfers and the speed of this file sharing.

How to create the DaemonSet as a cache

In this time, I used the Nginx Server for container which running on DaemonSet. I copy the file to under the context root (/ app) of Nginx from Azure Blog Storage so that you can retrieve the file by HTTP Protocol.
I configured the below so that the contents in /app can be obtained via http://podIP/.

FROM alpine:3.6

RUN apk update && \
    apk add --no-cache nginx

RUN apk add curl

ADD default.conf /etc/nginx/conf.d/default.conf

EXPOSE 80
RUN mkdir /app
RUN mkdir -p /run/nginx

WORKDIR /app
CMD nginx -g "daemon off;"

Please Save the above as a Dockerfile?

server {
  listen 80 default_server;
  listen [::]:80 default_server;

  root /app;

  location / {
  }
}

Please Save the above as default.conf?

After that, Please Build this image and push it to the Azure Container Registry?

$ docker build -t tyoshio2002/nginxdatamng:1.1 .
$ docker tag tyoshio2002/nginxdatamng:1.1 yoshio.azurecr.io/tyoshio2002/nginxdatamng:1.1
$ docker login -u yoshio yoshio.azurecr.io 
Password: 
$ docker push yoshio.azurecr.io/tyoshio2002/nginxdatamng:1.1

Next, Please create a secret to connect to Azure Container Registry from Kubernetes Cluster?

kubectl create secret docker-registry docker-reg-credential --docker-server=yoshio.azurecr.io --docker-username=yoshio  --docker-password="***********************" --docker-email=foo-bar@microsoft.com

Now that we have pushed the image to the Azure Container Registry, we will write a manifest for the Daemonset.

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: mydaemonset
  labels:
    app: mydaemonset
spec:
  selector:
    matchLabels:
      name: mydaemonset
  template:
    metadata:
      labels:
        name: mydaemonset
    spec:
      tolerations:
      - key: node-role.kubernetes.io/master
        effect: NoSchedule
      imagePullSecrets:
        - name: docker-reg-credential
      containers:
      - name: nginx
        image: yoshio.azurecr.io/tyoshio2002/nginxdatamng:1.1
        resources:
          limits:
            memory: 4000Mi
          requests:
            cpu: 1000m
            memory: 4000Mi
        lifecycle:
          postStart:
            exec:
              command:
                - sh
                - -c
                - "curl https://myfilestorageaccount.blob.core.windows.net/fileshare/Kuberenetes-Operation-Demo.mov -o /app/aaa.mov"
      terminationGracePeriodSeconds: 30

In this case, a single file (Kuberenetes-Operation-Demo.mov) is downloaded under /app in the postStart phase to simplify verification.

When you need to use the multiple files, you need to implement the mechanism separately to download multiple files in the container image.

It is also assumed that it is possible to customize Nginx or use other web servers and App servers to improve the efficiency such as enabling content caching.

Please Save the above manifest file as daemonset.yaml and execute the following command?

$ kubectl apply -f daemonset.yaml

After creating a DaemonSet, Please create a Service for the DaemonSet? By creating a Service, you can access port 30001 of each node and access Nginx with the Node IP address with port number.

apiVersion: v1
kind: Service
metadata:
  labels:
    app: daemonset-service
  name: daemonset-service
spec:
  ports:
  - port: 80
    name: http
    targetPort: 80
    nodePort: 30001
  selector:
    name: mydaemonset
  sessionAffinity: None
  type: NodePort

Please Save the above file as service.yaml and execute the following command?

$ kubectl apply -f service.yaml

After deploying Nginx DaemonSet and Ubuntu Deployment respectively, execute the following command to check which pod is running on which node.

$ kubectl get no -o wide
NAME                                STATUS   ROLES   AGE    VERSION                         INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
aks-agentpool-41616757-vmss000000   Ready    agent   176m   v1.15.7                         10.240.0.4            Ubuntu 16.04.6 LTS   4.15.0-1064-azure   docker://3.0.8
aks-agentpool-41616757-vmss000001   Ready    agent   176m   v1.15.7                         10.240.0.35           Ubuntu 16.04.6 LTS   4.15.0-1064-azure   docker://3.0.8
aks-agentpool-41616757-vmss000002   Ready    agent   176m   v1.15.7                         10.240.0.66           Ubuntu 16.04.6 LTS   4.15.0-1064-azure   docker://3.0.8
virtual-node-aci-linux              Ready    agent   175m   v1.14.3-vk-azure-aci-v1.1.0.1   10.240.0.48                                  
$ kubectl get po -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP            NODE                                NOMINATED NODE   READINESS GATES
mydaemonset-gqlrw        1/1     Running   0          72m   10.240.0.24   aks-agentpool-41616757-vmss000000              
mydaemonset-nnn5l        1/1     Running   0          72m   10.240.0.50   aks-agentpool-41616757-vmss000001              
mydaemonset-pzvzx        1/1     Running   0          72m   10.240.0.82   aks-agentpool-41616757-vmss000002              
ubuntu-884df4bfc-7zgkz   1/1     Running   0          64m   10.240.0.69   aks-agentpool-41616757-vmss000002              
ubuntu-884df4bfc-gd26h   1/1     Running   0          63m   10.240.0.33   aks-agentpool-41616757-vmss000000              
ubuntu-884df4bfc-vh7rg   1/1     Running   0          63m   10.240.0.49   aks-agentpool-41616757-vmss000001

According to the above, ubuntu-884df4bfc-vh7rg is running on the node aks-agentpool-41616757-vmss000001, and on the node aks-agentpool-41616757-vmss000001 the DaemonSet Pod of mydaemonset-nnn5l is running.
And you can see that the node IP address of aks-agentpool-41616757-vmss000001 is 10.240.0.35.

Inside of the Ubuntu pod, the IP address of the Node where the pod is running is got by the environment variable $ NODE_IP.

$ kubectl exec -it ubuntu-884df4bfc-vh7rg env|grep NODE_IP
NODE_IP=10.240.0.35

In Addition
Lastly, we will skip the detailed explanation of how to create each of Standard and Premium (high speed) Azure Blob Storage.

After you created both of them, please copy the same video file in each storage?

Premium Storage of Azure Blob is connected to AKS VNET and files are obtained via VNET.

Other Reference information

Verification Details

In this time, I evaluated the following four points.

Time required to copy files in the directory where Azure Files is mounted to the pod
Time required to retrieve files existing in Azure Blob (Standard) by pod
Time required to retrieve files existing in Azure Blob (Premium) by pod
Time required to get from files copied in DaemonSet

First Try (The video of actual verification contents)

Video: https://www.youtube.com/watch?v=fe4QSfe9n-A

Confirmed by executing the following command

kubectl exec -it ubuntu-884df4bfc-vh7rg

## File Put on Azure Files (File shares : Samba)
$ date
$ cp /mnt/azure/Kuberenetes-Operation-Demo.mov ~/mount.mov
$ date

## File Put on Azure Blob (Containers)
$ curl https://myfilestorageaccount.blob.core.windows.net/fileshare/Kuberenetes-Operation-Demo.mov -o ~/direct.mov
$ curl https://mypremiumstorageaccount.blob.core.windows.net/fileshare/Kuberenetes-Operation-Demo.mov -o ~/premium2.mov
$ curl http://$NODE_IP:30001/aaa.mov -o ~/fromNodeIP.mov

The result of Third Try:

	Copy from PV(Samba)	Get file from Standard Storage	Get file from Premium Storage	Get file from DaemonSet Cache
１	51 Sec	69 Sec	27 Sec	20 Sec
２	60 Sec	60 Sec	23 Sec	11 Sec
３	53 Sec	57 Sec	20 Sec	8 Sec

By the way, even when using Nginx of DaemonSet, it takes time for the first access. This assumes that it is taking a long time to get, probably because it is not in Nginx memory. In other environments, it is faster after the second access.

Result Summary of verification

From the results of the above three executions, it was found that it is the fastest to get the file into the DaemonSet and *** get the file from the DaemonSet's cache***.
By using Premium storage, you can transfer files faster than Standard storage, but Premium costs more than Standard. Also, in the case of acquisition from Storage, it is necessary to download files via the network each time. Therefore, considering that, it is most efficient to first get the file on the same node (VM) and get it from there. It is natural to be quick and quick.

Advantages of using DaemonSet

Reduce network traffic
Faster file acquisition speed
If the implementation of the container in the DaemonSet is made richer, it can be used for other purposes (for example, writing support).

Concerns

Currently, a specific single file is acquired, but when handling multiple files or when there is an update of the file, a mechanism to periodically check for updates inside the DaemonSet and download the updated part Need to implement
The file size in the DaemonSet pod may be bloated and needs to be cleaned up
Regarding file writing, it is difficult to respond if this verification method is used as it is, but it is assumed that it will be supported if you implement a function like the In Memory Grid product in the container inside the DaemonSet.

Finally:
In addition, I conducted some other internal verifications, but because the explanation contents increased and the viewpoint was likely to be blurred, we focused on only the points that were particularly effective. Also, in order to make it as simple as possible, I used Nginx for DaemonSet, but I think that further performance improvement is needed, you can tune up the Nginx configuration or changing another implementation.

I would be very grateful if you could give further tuning based on this information.

Please don't move from Monolithic Application to Monolithic Kubernetes Cluster?

Yoshio Terada — Tue, 04 Jun 2019 18:34:13 +0000

This is my proposal message from experience.

I recommend you to not create the huge size of node on one Kubernetes Cluster.

According to the following document, If you use the Azure Kubernetes Service(AKS), the number of max node is 100.
Quotas, virtual machine size restrictions, and region availability in Azure Kubernetes Service (AKS)

However, I don't recommend you to create the cluster with max node size.
Because it is same as creating a new Monolith in Container　Environment(Kubernetes) again.

As you can imagine, if you created it, you may face the following issue.

Increase in Impact
If you created 150 microservices on 100 nodes and there are system failures or other performance impacts, all 150 services may be affected.
The scope of this impact compared to a monolithic application(10-20 services) is a much worse situation with a monolithic kubernetes cluster(150 services).
Increased time to upgrade
There is a kubernetes release every 3-4 months.
And AKS(Kubernetes) has an end of life policy that requires updating or reinstalling the kubernetes cluster approximately every 1 or 2 year at minimum.
Exam: Azure Kubernetes Service: Kubernetes 1.9 end-of-life notice
In order to update or create the new Kubernetes Cluster very quickly, the number of node size is very important.For example, if you created 100 nodes of cluster, it will take a long time than 30 nodes create.
Difficult to maintain
I would recommend you to create very simple structure and configuration (simple is the Best!!).
If you created the simple structure and configuration, it is easy for you to create the same environment.However if you created the complex structure, sometimes it may be difficult to create the same environment even though you created the Infrastructure as Code.
I recommend that you should be easy to create or destroy the environment at any time.

Finaly
I recommend you to create the Kubernetes Cluster for Mini Service level like 20-30 node. However, in fact, sizing depends on your system. For example, you need create a chunk of relevant microservices and deploy it to the the cluster.

Experience of Kubernetes Hackathon with Customers

Yoshio Terada — Fri, 10 May 2019 09:27:25 +0000

TL;DR

History :

At first, I would like to explain why I had been opening the k8s Hackathon many times since last year.

In fact, k8s is not only for Infrastructure engineer but also for Developer engineer. In order to use the k8s appropriately, both technical knowledge is needed.

For me, I took few month to understand the k8s of all. However it is too long for everyone and they would like to catch up it more smoothly.

In the past, as an ex-Evangelist and now Developer Advocate, of course I had done many presentation about k8s. However I thought that it is very difficult for me to explain the important topic of k8s for only one hour in one presentation.

Because sometimes I need to explain 12 Factor App, CI/CD, Microservices depends on the audience level.

And in tha past, I created so many Technical Hands on Lab(HoL) contents. Yes, the HoL is very useful for first contact. However if the user face the trouble, they can't solve the issue from the HoL contents. Thus, I thought that the attendee should consider more during the HoL or Hackathon.

Since last year, I started the k8s Hackathon in Japan(Tokyo, Sapporo, actual customer) and Australia. During the Hackathon, I will gather the requirements of the attendee at first. And depends on the requirements, I will help and advise the attendee to achieve the requirement.

About Hackathon Event

In general, every times the technical skill of the attendee is so difference. Sometimes members doesn't have an experience of Azure even though they had an experience of AWS. Some of the engineer is not developer. Some of the engineer is not familiar with the Infrastructure.

In this situation, I will inform them since the basic concept of Container, Microservices, k8s and Azure at first.

For Community Event, it is very difficult to create production ready contents. Because almost of the community event is planning as only 2days. So almost of the attendee may only catch up how to use, troubleshoot and consider the architecture in the day.

For Customer Event, of course depends on the customer and their requirements, they will be able to do more things.

For Example, if they can prepare the one-week Hackathon and they would like to migrate their application to k8s, it is possible in the week.

Date:

Community Event : 2 Day is needed (Saturday, Sunday AM:10:00-18:00)
Customer Event : 3-5 Day is needed (Monday to Friday:AM 10:00-18:00)

Activity

1.Presentation (Introduction) of Basic Concept (1.5 Hour)

The Change the World of the IT Industry
Improve the Development Work Flow by using Value Stream Mapping
Importance of Cloud Native Application
12 Factor App
Microservices
Why the container and k8s is trend and important
Suitable and Not Suitable application/system/architecture of k8s
Basic of Container (Docker)

At first, I will explain why Container and k8s is important now. And what kind of the situation the Container or k8s is useful. Also I inform them what kind of the situation you shouldn't use the Container and k8s.
Because Container and k8s is not the Silver Bullet. If the user use them in the wrong spaces, they may distress their management.

2.Hackathon (1.5 Days)

This is not the HoL but Hackathon. So I will gather the information of what kind of work they would like to do or solve in this two days.

After that I created the group and proceeded as Mob Programing Style.

Basic operation:

Create Linux VM on Azure
Install docker client on Linux VM
Create Sample Application (Java/PHP/Python/.Net Core)
Install Azure Container Registry
Create Docker Image and push it to Registry
Install az command
Install AKS on Azure
Create Deployment YAML
Create Service YAML
Create Ingress YAML
Evaluate the Self-healing by Readiness/Liveness Probe
Evaluate the Rolling Upgrade
Create the Virtual Node
Create CI/CD environment by Azure DevOps
Create Monitoring
Istio
.... Depends on the Attendee ...

Venue:

By the way, Venue and Facility is very important to open the Hackathon. Every times, I'm opening the Hackathon by using Mob Programing Style. In order to do it, Big screen(monitor) is needed for every teams. Also we need high band wide network to connect to Azure.

Fortunately, Every Meeting room in Microsoft Office has big screen monitor, so it is easy too open the Hacakathon. However outside of the office, sometimes we need to take care of it.

Result of Hackahon:

According to the feedback from the attendee, almost of the attendee satisfied the k8s Hackathon. In fact, they could learn and operate the k8s for only two days even though they were beginners.
And even though, they were not familiar with the Azure, they felt that the good parts of Azure like AKS, Azure DevOps, Virtual Node and Azure Dev Spaces.

Refer to following Blogs.

FeedBack & Blog Post from the Attendees

Note:

Following contents is directly I engaged and it was wrote by attendee by Japanese language, if you refer to the following blog entry, please use the Translator like Bing Translator or Google Translator?

2018/05/15 Oisix ra Daichi

2018/7/7日 - 7/8 : Japan Java Users Group 2 day k8s Hack

2018/8/7-8/10: Pnop : Partner Company (and MVPs)

2018/11/17(土)～18(日) Sapporo JavaDo (Java Users Group for Sapporo)

I tried a Hackahon of Kubernetes in Java Do Community

2019/01/21-1/25 Customer: HeartBeat

2019/02/23-/2/24 : Kubernetes Hackathon at KanJava(Java Community in West side of Japan Area)

"Azure" What is For Kubernetes? I went to hackathon, so let's go over and review

2019/03/19 : OiStudy #1 Azure & AKS: Community Event

2019/04/13-4/14 KitaAzu: JAZUG Sapporo 22nd Study

2019/04/2-4/4 Customer Mercari: Selling App

Automation UI Test (Selenium Grid/Zalenium) of Mercari Web by using Azure Kubernetes Service (AKS)

I'm very glad that there is a lot of AKS related Technical Contents outside of Microsoft. AKS usage and Japan community is growing up now !!

Others:

I got many direct feedback both products and events from Slack Channel which is used during the Hackathon.

Finally

If you would like to open the Hackathon with me, if you can accept one of following rule, depends on my schedule, I can engage with you. Even though it is outside of Japan, it is OK for me.

Your company can open the Microsof Customer Story like Oisix
You are considering to use the AKS for production like Oisix/HeartBeats
Provide us the Products FeedBack to us during the Hackathon
Some of engineer can write the blog about your experiences in Community Event
For Community: You need to activate the Azure

DEV Community: Yoshio Terada

Securely Connecting Azure Container Apps to Azure OpenAI Using User Managed Identity

1. Deploying Java Applications to Azure Container Apps Without Containerization

2. Securely Connecting Azure Container Apps to Azure OpenAI Using User Managed Identity

Steps to Set Up User Managed Identity

2.1. Set Environment Variables

2.2. Create a Resource Group

2.3. Create an Azure OpenAI Instance

2.4. Create a User Managed Identity

2.5. Assign Roles to User Managed Identity for Azure OpenAI

2.6. Create an Azure Container Apps Environment

2.7. Creating a Spring Boot Web Application

2.7.1 Creating a Spring Boot Project

2.7.2 Editing the pom.xml Project File

2.7.3 Implementing a RESTful Endpoint (Main Part)

2.7.4 Defining the JSON Format for Endpoint Reception

2.7.5 Application Configuration

2.7.6 (Optional): Verifying Operation in a Local Environment

2.7.7 Building the Application and Creating Artifacts

2.8. Creating an Azure Container Apps Instance

2.9. Assigning a User Managed Identity to Azure Container Apps

2.10. Verifying Operation

2.11. Reusing the Same User Managed Identity to Access Azure OpenAI from Other Azure Resources

For Azure VM

For Azure App Service

For Azure Functions

For Azure Container Instances

For Azure Kubernetes Service

2.12. Comparing Access Keys and User Managed Identities from a Security Perspective

3 Summary

Comparison of Two Methods for Deploying Azure Functions to Azure Container Apps

Old Method: Creating with az containerapp create

New Method: Creating with az functionapp create

Reference: Azure Functions on App Service (Portal)

Management of Containers in Azure Container Apps

Conclusion

Deploying a Java Azure Function on Azure Container Apps

Setting Environment Variables

Creating and Testing a Java Azure Function Project

1. Create an Azure Functions for Java Maven Project

2. Run Azure Function Locally

Creating and Testing the Azure Functions Container Image

1. Build and Test the Docker Image

Pushing the Container Image to Azure Container Registry

1. Log in to Azure CLI

2. Create a Resource Group

3. Create and Log in to Azure Container Registry

4. Retrieve the Azure Container Registry Server Name

5. Push the Image to Azure Container Registry

Creating Azure Container Apps and Deploying the Java Azure Function

1. Register Extensions and Resource Providers in Azure CLI

2. Create Azure Container Apps Environment

3. Create Storage Account for Azure Functions

4. Create an Instance of Java Azure Function in Azure Container Apps

5. Assign Role for Azure Function to Access Azure Container Registry

6. Retrieve the URL of the Azure Function

Summary

Automated PDF-to-Text Conversion and Vector Search with Azure OpenAI and Blob Storage

1. Introduction

1.1 Overview of the Service

1.2. Technologies Used

2. Setup Environment

2.1 Setting Environment Variables

2.1.1 Setting PostgreSQL Extensions

3. Running the Application

3.1Running Azure Functions (BlobUploadDetector)

3.2 Running Spring Boot

3.3 Uploading Files to Blob Storage

3.4 Verifying the Spring Boot Application

4. Points to Note for Application Implementation

4.1 Key Points for Implementing Azure Functions (BlobUploadDetector)

4.1.1 Extending the Execution Time of Azure Functions

4.1.2 Adjusting the Usage Interval of Azure OpenAI Embedding API

4.1.3 Setting Environment Variables in Azure Functions

4.1.4 Points to Consider When Changing the Container Name Created in Azure Blob Storage

4.1.5 Points to Consider in PDF Parsing Process (Especially When Using Models Other Than ChatGPT-4)

4.1.6 About the Data to Insert

4.2 Points to Consider in Spring Boot Implementation

4.2.1 Implementing as Server Sent Event

4.2.２ Processing when a string is entered and the Submit button is pressed

Old Method: Creating with `az containerapp create`

New Method: Creating with `az functionapp create`