DEV Community: Suji Matts

Analyze Images with the GCP Cloud Vision API

Suji Matts — Thu, 17 Oct 2024 12:29:56 +0000

Overview

This article explores usage of multiple Google Cloud ML APIs, including the Cloud Vision API, Translation API, and Natural Language API. You’ll start by using Optical Character Recognition (OCR) to extract text from an image, then translate that text, and finally analyze it for deeper insights.

Key Steps

Creating an API Key: Begin by generating an API key to authenticate your requests to the Vision API.

Uploading an Image: You will create a Cloud Storage bucket to store your images and upload a sample image for text extraction.

Making a Vision API Request: Using curl, you'll construct a request to the Vision API to perform text detection on the uploaded image.

Translating the Text: With the extracted text, you’ll use the Translation API to convert it from French to English.

Analyzing the Text: Finally, leverage the Natural Language API to extract entities and analyze the translated text for additional insights.

1. Creating an API Key

To get started, you need an API key to authenticate your requests to the Vision API. Here’s how you do it:

Navigate to the API Credentials Sectio
n: In the Google Cloud Console, go to APIs & Services and then Credentials.

Generate an API Key
: Click on Create Credentials and select API Key from the dropdown menu. This key will allow you to make secure requests to the Vision API.

Store the API Key
: After generating the key, copy it and store it as an environment variable in your Cloud Shell. This simplifies the process of including the key in your requests, making it easier to manage.

2. Uploading an Image

To process an image with the Vision API, you need to upload it to a Cloud Storage bucket:

Create a Cloud Storage Bucket
Navigate to the Cloud Storage browser in the Google Cloud Console and click Create Bucket. Choose a unique name for your bucket and configure access settings.

Set Access Permissions
Uncheck the box for Enforce public access prevention and select Fine-grained access control.

Upload the Image
After creating the bucket, you can upload your sample image (e.g.https://cdn.qwiklabs.com/cBoI5P4dZ6k%2FAr5Mv7eME%2F0fCb4G6nIGB0odCXzpEa4%3D). Ensure that the image has public access so the Vision API can retrieve it for processing.
Save this image as sign.jpg

3. Create your Cloud Vision API request

create a file with ocr-request.json below content

{
  "requests": [
      {
        "image": {
          "source": {
              "gcsImageUri": "gs://my-bucket-name/sign.jpg"
          }
        },
        "features": [
          {
            "type": "TEXT_DETECTION",
            "maxResults": 10
          }
        ]
      }
  ]
}

You're going to use the TEXT_DETECTION feature of the Cloud Vision API. This will run optical character recognition (OCR) on the image to extract text.

4. Call the text detection method

curl -s -X POST -H "Content-Type: application/json" --data-binary @ocr-request.json  https://vision.googleapis.com/v1/images:annotate?key=${API_KEY}

The first part of your response should look like the following:

{
  "responses": [
    {
      "textAnnotations": [
        {
          "locale": "fr",
          "description": "LE BIEN PUBLIC\nles dépêches\nPour Obama,\nla moutarde\nest\nde Dijon\n",
          "boundingPoly": {
            "vertices": [
              {
                "x": 138,
                "y": 40
              },
              {
                "x": 622,
                "y": 40
              },
              {
                "x": 622,
                "y": 795
              },
              {
                "x": 138,
                "y": 795
              }
            ]
          }
        },
        {
          "description": "LE",
          "boundingPoly": {
            "vertices": [
              {
                "x": 138,
                "y": 99
              },
              {
                "x": 274,
                "y": 82
              },
              {
                "x": 283,
                "y": 157
              },
              {
                "x": 147,
                "y": 173
              }
            ]
          }
        },
        {
          "description": "BIEN",
          "boundingPoly": {
            "vertices": [
              {
                "x": 291,
                "y": 79
              },
              {
                "x": 413,
                "y": 64
              },
              {
                "x": 422,
                "y": 139
              },
              {
                "x": 300,
                "y": 154
              }
            ]
          }
            ...
      ]
}]
}

The OCR method is able to extract lots of text from the image.

The first piece of data you get back from textAnnotations is the entire block of text the API found in the image. This includes:

the language code (in this case fr for French)
a string of the text
a bounding box indicating where the text was found in the image
Then you get an object for each word found in the text with a bounding box for that specific word.

Run the following curl command to save the response to an ocr-response.json file so it can be referenced later:

curl -s -X POST -H "Content-Type: application/json" --data-binary @ocr-request.json  https://vision.googleapis.com/v1/images:annotate?key=${API_KEY} -o ocr-response.json

5. Send text from the image to the Translation API

The Translation API can translate text into 100+ languages. It can also detect the language of the input text. To translate the French text into English, pass the text and the language code for the target language (en-US) to the Translation API.

First, create a translation-request.json file and add the following to it:

{
  "q": "your_text_here",
  "target": "en"
}

Run this Bash command in Cloud Shell to extract the image text from the previous step and copy it into a new translation-request.json (all in one command):

STR=$(jq .responses[0].textAnnotations[0].description ocr-response.json) && STR="${STR//\"}" && sed -i "s|your_text_here|$STR|g" translation-request.json

Now you're ready to call the Translation API. This command will also copy the response into a translation-response.json file:

curl -s -X POST -H "Content-Type: application/json" --data-binary @translation-request.json https://translation.googleapis.com/language/translate/v2?key=${API_KEY} -o translation-response.json

Run this command to inspect the file with the Translation API response

cat translation-response.json

Now you can understand more of what the sign said!

{
  "data": {
    "translations": [
      {
        "translatedText": "TO THE PUBLIC GOOD the dispatches For Obama, the mustard is from Dijon",
        "detectedSourceLanguage": "fr"
      }
    ]
  }
}

6. Analyzing the image's text with the Natural Language API

The Natural Language API helps you understand text by extracting entities, analyzing sentiment and syntax, and classifying text into categories. Use the analyzeEntities method to see what entities the Natural Language API can find in the text from your image.

To set up the API request, create a nl-request.json file with the following

{
  "document":{
    "type":"PLAIN_TEXT",
    "content":"your_text_here"
  },
  "encodingType":"UTF8"
}

STR=$(jq .data.translations[0].translatedText  translation-response.json) && STR="${STR//\"}" && sed -i "s|your_text_here|$STR|g" nl-request.json

Call the `analyzeEntities `endpoint of the Natural Language API with this curl request:
curl "https://language.googleapis.com/v1/documents:analyzeEntities?key=${API_KEY}" \
  -s -X POST -H "Content-Type: application/json" --data-binary @nl-request.json

If you scroll through the response you can see the entities the Natural Language API found:

{
  "entities": [
    {
      "name": "dispatches",
      "type": "OTHER",
      "metadata": {},
      "salience": 0.3560996,
      "mentions": [
        {
          "text": {
            "content": "dispatches",
            "beginOffset": 23
          },
          "type": "COMMON"
        }
      ]
    },
    {
      "name": "mustard",
      "type": "OTHER",
      "metadata": {},
      "salience": 0.2878307,
      "mentions": [
        {
          "text": {
            "content": "mustard",
            "beginOffset": 38
          },
          "type": "COMMON"
        }
      ]
    },
    {
      "name": "Obama",
      "type": "PERSON",
      "metadata": {
        "mid": "/m/02mjmr",
        "wikipedia_url": "https://en.wikipedia.org/wiki/Barack_Obama"
      },
      "salience": 0.16260329,
      "mentions": [
        {
          "text": {
            "content": "Obama",
            "beginOffset": 31
          },
          "type": "PROPER"
        }
      ]
    },
    {
      "name": "Dijon",
      "type": "LOCATION",
      "metadata": {
        "mid": "/m/0pbhz",
        "wikipedia_url": "https://en.wikipedia.org/wiki/Dijon"
      },
      "salience": 0.08129317,
      "mentions": [
        {
          "text": {
            "content": "Dijon",
            "beginOffset": 54
          },
          "type": "PROPER"
        }
      ]
    }
  ],
  "language": "en"
}

For entities that have a wikipedia page, the API provides metadata including the URL of that page along with the entity's mid. The mid is an ID that maps to this entity in Google's Knowledge Graph. To get more information on it, you could call the Knowledge Graph API, passing it this ID. For all entities, the Natural Language API tells us the places it appeared in the text (mentions), the type of entity, and salience (a [0,1] range indicating how important the entity is to the text as a whole). In addition to English, the Natural Language API also supports the languages listed in the Language Support reference.

Configure Google Cloud Speech-to-Text API

Suji Matts — Mon, 14 Oct 2024 12:19:32 +0000

Overview

The Speech-to-Text API enables easy integration of Google speech recognition technologies into developer applications. It allows you to send audio and receive a text transcription from the service.

What we'll cover
In this lab, you will learn how to:

Create an API key
Create a Speech-to-Text API request
Call the Speech-to-Text API

Step 1: Create an API Key

In the Google Cloud Console, navigate to Navigation menu > APIs & services > Credentials.
Click on Create credentials and select API key.
Copy the generated key and click Close.

Save API Key as Environment Variable

Connect to your VM instance via SSH.
In the command line, set the environment variable

export API_KEY=<YOUR_API_KEY>

Step 2: Create Your Speech-to-Text API Request

Create a new file named request.json:

touch request.json

Open the file in a text editor and add the following JSON configuration, specifying the audio file’s URI:

{
  "config": {
    "encoding": "FLAC",
    "languageCode": "en-US"
  },
  "audio": {
    "uri": "gs://cloud-samples-tests/speech/brooklyn.flac"
  }
}

Step 3: Call the Speech-to-Text API

curl -s -X POST -H "Content-Type: application/json" --data-binary @request.json "https://speech.googleapis.com/v1/speech:recognize?key=${API_KEY}"

The response will include the transcript and a confidence score.

Save Response to a File
curl -s -X POST -H "Content-Type: application/json" --data-binary @request.json "https://speech.googleapis.com/v1/speech:recognize?key=${API_KEY}" > result.json

Conclusion

Congratulations! You have successfully used the Speech-to-Text API to transcribe an audio file. This hands-on lab demonstrated how to create an API key, construct a request, and call the Speech-to-Text service.

Building a Spring Boot Application with Maven and Deploying on Kind K8s Cluster Using Helm

Suji Matts — Fri, 11 Oct 2024 05:59:52 +0000

In this article, we will walk through the process of setting up a Kubernetes environment using Kind, building a simple Java Hello World application, and deploying it using Docker and Helm. This guide is perfect for developers looking to streamline their Kubernetes development workflow.

We will go through two phases here: the first phase focuses on setting up a local Kubernetes environment and deploying a Java Hello World application, while the second phase covers modifying the application and redeploying it to reflect those changes.

Prerequisites

Before you begin, ensure you have the following installed on your machine:

Docker
Kind
kubectl
Helm
Maven

Step 1: Create a Kind Cluster

Kind (Kubernetes IN Docker) allows you to run Kubernetes clusters in Docker containers. To create a Kind cluster, use the following command:

kind create cluster

Verify that your cluster is running:

kubectl cluster-info --context kind-kind

kubectl cluster-info --context kind-kind
Kubernetes control plane is running at https://127.0.0.1:42723
CoreDNS is running at https://127.0.0.1:42723/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Step 2: Create a Java Hello World Application

Create a folder structure like below and create two files as mentioned

└── src
    └── main
        ├── java
        │   └── com
        │       └── example
        │           └── HelloWorldApplication.java
        └── resources
            └── application.properties

*HelloWorldApplication.java *

package com.example;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@SpringBootApplication
@RestController
public class HelloWorldApplication {

    public static void main(String[] args) {
        SpringApplication.run(HelloWorldApplication.class, args);
    }

    @GetMapping("/")
    public String hello() {
        return "Hello, World!";
    }
}

application.properties

server.port=8081

Create a pom.xml file in the root folder [same folder of src]

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>hello-world-app</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <properties>
        <java.version>11</java.version>
        <spring.version>2.6.3</spring.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
            <version>${spring.version}</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <version>${spring.version}</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>repackage</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.1</version>
                <configuration>
                    <source>${java.version}</source>
                    <target>${java.version}</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

Step 3: Build your application using maven

mvn clean package

This will create a jar file inside the target folder

$ls
pom.xml  src  target
$ls target/
classes  generated-sources  hello-world-app-1.0-SNAPSHOT.jar  hello-world-app-1.0-SNAPSHOT.jar.original  maven-archiver  maven-status

Step 4: Build the Docker Image

Create a Dockerfile in the root directory [same dir of src]

# Use the official OpenJDK image as the base image
FROM openjdk:11-jre-slim

# Set the working directory
WORKDIR /app

# Copy the JAR file into the container
COPY target/hello-world-app-1.0-SNAPSHOT.jar app.jar

# Expose the port the app runs on
EXPOSE 8081

# Run the application
ENTRYPOINT ["java", "-jar", "app.jar"]

Build your application and create the Docker image:

docker build -t <your-dockerhub-username>/hello-world-app:latest .

Step 5: Push the Docker Image to Docker Hub

docker login

Push your image to Docker Hub:

docker push <your-dockerhub-username>/hello-world-app:latest

Step 6: Create Helm Charts

Generate a new Helm chart:

helm create hello-world-java

This will create below structure

├── hello-world-java
│   ├── charts
│   ├── Chart.yaml
│   ├── templates
│   │   ├── deployment.yaml
│   │   ├── _helpers.tpl
│   │   ├── hpa.yaml
│   │   ├── ingress.yaml
│   │   ├── NOTES.txt
│   │   ├── serviceaccount.yaml
│   │   ├── service.yaml
│   │   └── tests
│   │       └── test-connection.yaml
│   └── values.yaml

We need only two files from these. Those are

Chart.yaml
values.yaml
templates/deployment.yaml

Edit the values.yaml file to specify your Docker image and Service

image:
  repository: <your-dockerhub-username>/hello-world-app
  tag: latest

service:
  type: NodePort
  port: 8081

Make sure the deployment.yaml file under templates is configured correctly:

containers:
        - name: {{ .Chart.Name }}
          securityContext:
            {{- toYaml .Values.securityContext | nindent 12 }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          ports:
            - name: http
              containerPort: {{ .Values.service.port }}
              protocol: TCP

Step 7: Deploy the Application Using Helm

helm install hello-world-java ./hello-world-java

Verify that your application is running:

kubectl get pods

NAME                               READY   STATUS    RESTARTS   AGE
hello-world-java-54fb55948-597hk   1/1     Running   0          144m

Step 8: Access Your Application

To access your application, you can use port forwarding since you're running in a Kind cluster:

kubectl port-forward svc/hello-world-java 8081:8081

You should see output indicating that the connection is being forwarded. Open your browser and go to:

http://localhost:8081

Now you can successfully access the Hello World page from your browser. Let’s explore how to make a change and redeploy it.

Step 9: Modify Application

First, navigate to the HelloWorldApplication.javafile and change "Hello World" to "Hi World."

Step 10: Rebuild your application using Maven

mvn clean package

Step 11: Re-build your Docker image

docker build -t <your-dockerhub-username>/hello-world-app:latest .

Step 12: Push the updated image to Docker Hub

docker push <your-dockerhub-username>/hello-world-app:latest

Before deploying it with Helm, ensure the image pull policy is set to Always in values.yaml; otherwise, it wont download the latest image.

image:
  repository: your-docker-hub-username/hello-world-java
  pullPolicy: Always
  tag: "latest"

Step 13: Helm Upgrade

Now you can upgrade your Helm release:

helm upgrade hello-world-java ./hello-world-java

Finally, set up port forwarding to access your application:

kubectl port-forward svc/hello-world-java 8081:8081

Step 14:Access Your Application

Now you can navigate to http://localhost:8081 in your browser and see your changes reflected.You will now see the "Hi World" page instead of "Hello World."