DEV Community: Abinash S

Build Intelligent ChatBots with Language Processing

Abinash S — Tue, 01 Apr 2025 08:52:34 +0000

Blog Co-Authored by @madhusri-j

Ever Wondered How Chatbots Just Know What You’re Asking For?

Imagine typing a question into a chatbot and instantly receiving a relevant and accurate response. How does the chatbot understand your intent so effectively?

It’s all thanks to intent classification - the magic behind a chatbot's ability to figure out whether you’re asking for an answer, a summary, or just chatting. Without it, bots would be lost in translation, and you'd end up frustrated.

In this blog, let’s unravel how intent classification works, exploring practical methods to get it right. Whether you're building a bot or tweaking an existing one, there’s something here for everyone.

Why Does Intent Classification Matter?

If you’re training your bot to handle different situations. Maybe you want it to recognize:

A direct question (e.g., What’s the time in New York?).
A request for a summary (e.g., Can you summarize this article?).
A comparison task (e.g., What’s better, A or B?).
Or even casual banter (e.g., Tell me a joke!).

The better the bot is at understanding these, the smarter and more useful it feels. Let’s explore the different ways you can achieve this.

Approaches to Intent Classification

There’s no one-size-fits-all solution, but here are four popular methods to get you started:

1. Machine Learning / Deep Learning Models

What’s Needed? A labeled dataset.

This approach involves teaching your chatbot to identify patterns in user inputs, using machine learning algorithms.

How It Works: You collect examples of what users might say (a dataset), convert those into numbers the machine understands, and train a model to predict intents based on this data.

So Intent classification can be achieved by building machine learning models such as Random Forest , Naive Bayes or by training a Multi Layered Perceptron neral nets, using a labeled training dataset. The training dataset must be gathered with multiple utterance samples for each intents. These utterances has to be converted numercial form using any of the tokenizers/vectorizers (we have used basic TfidfVectorizer). Once the model is trained, the resulting weights are saved in a pickle file, which can be loaded for inference in your application.

Here’s the gist:

Prepare Data → Create Pipeline → Train Model → Evaluate Model → Export Weights

Prepare Data: Ensure you have a dataset containing the input and its desired classification.
Create Pipeline: Here, we convert the text data using TfidfVectorizer and then pass to our Naive Bayes model, MultinomialNB.
Train Model: Fit the model on the dataset.
Evaluate Model: Validate the model’s performance on test data.
Export Weights: Save the trained model and label encoder to pickle files.

Here’s how you might integrate it:

Sample dataset: dataset.csv
- Format: Contains the input text and corresponding classification labels.
Example notebook: naive_bayes.ipynb
- Running this Jupyter notebook will produce two output files:
  - intent_classifier.pkl (trained model weights)
  - label_encoder.pkl (for decoding classification labels)

import joblib
from loguru import logger


def classify_question_intent(prompt: str) -> tuple[str, float]:
    """
    Classify the intent of a given prompt using a pre-trained model.

    Args:
        prompt (str): The input text to classify.

    Returns:
        tuple[str, float]: The predicted intent and its confidence score.
    """
    logger.info("Classifying Intent")
    file_path = "app/internal/artifacts"  # Path to saved model files

    # Load the pipeline and label encoder
    pipeline = joblib.load(f"{file_path}/intent_classifier.pkl")
    label_encoder = joblib.load(f"{file_path}/label_encoder.pkl")

    # Predict intent
    intent_index = pipeline.predict([prompt])[0]
    intent_confidence = max(pipeline.predict_proba([prompt])[0])
    intent = label_encoder.inverse_transform([intent_index])[0]

    logger.debug(f"Intent: {intent}, Confidence: {intent_confidence}")
    return intent, intent_confidence

Why Choose This?

It’s quick and efficient once trained.
You have full control over the process, making it easy to customize for your needs.
Works great if you’ve got a solid, clean dataset to start with.

Challenges:

You’ll need enough labeled training data, which can take time to prepare.
Training and deploying the model requires extra steps like validation and exporting weights.

2. Large Language Models (LLMs)

What’s Needed? Just a good prompt.

For scenarios where creating datasets is not feasible, Large Language Models offer a convenient alternative. Large language models like GPT can handle intent classification right out of the box.

How It Works:

You give the LLM a structured prompt and a list of possible intents. Optionally, throw in a few examples, and it’ll figure out what the user wants.

Classify the intent of the user request and respond with one of the following intent [RAG, Compare, AI, Summay]

Question:
User: "Can you summarize this?" → Intent: Summary

Why Choose This?

No datasets or pre-training required—you’re ready to roll with just a good prompt.
Super easy to implement, even for those new to AI.

Challenges:

It might be a bit slower since it requires multiple API calls for each query.
Costs can add up quickly if you’re handling a lot of user queries.

3. Azure Language Service

What’s Needed? A labeled dataset and access to Azure.

AWS Alternative: Comprehend Custom Classification

If you’re dealing with enterprise-scale systems, Azure’s Custom Text Classification can do the heavy lifting.

Steps to Set Up in Azure:
1. Provision a Custom Text Classification project in Azure Language Studio.
2. Upload a labeled dataset (text with corresponding intent labels).
3. Train the model on the dataset.
4. Deploy the trained model for API-based predictions.
Sample Schema:
- Intents: compare, llm, rag, summary, None.
- Example Utterances:
  - "Compare and contrast these documents" → Intent: compare (Confidence: 96.67%, as seen in the second image).
Integration:
- Deploy the model as an endpoint in Azure.
- Call the endpoint using Azure API to classify user input.

Why Choose This?

Pre-built infrastructure means less manual coding.
Perfect for scaling and integrating with cloud-based systems.
Azure provides a user-friendly interface to train and deploy models without heavy lifting.

Challenges:

Requires labeled data to train the model.
Text length is capped at 1,000 characters, which can be limiting for complex queries.
Cloud services aren’t always cheap, though free tiers are available.

### In-Memory Vector Store

What’s Needed? A dataset and embedding tools.

This method leverages a vector store to indentify the closest matching intents. Sample user queries are embedded as vectors, with metadata tagging each vector to a specific intent. This data is stored in a vector store. When a user asks a questions, querying the vector store with the current user question, the top result (based on highest similarity) provides the predicted intent. The efficiency of this approach depends on various factors like the quality of the source dataset, vector retrival technique and embedding algorithm

How does it work?

Prepare Data → Load Data to VectorStore → User Query → Pick the Intent

Prepare Data:
- Create a dataset where each entry consists of a user query (text) and its corresponding intent label.
- Example:

User Query	Intent
"Summarize this document"	summary
"Compare and contrast these"	compare

Load Data to Vector Store:
- Convert the user queries into embeddings (using tools like OpenAI Embeddings, Sentence Transformers, or similar).
- Store the embeddings in an in-memory vector store (e.g., FAISS, ChromaDB) with intent labels stored as metadata.
User Query:
- Convert the incoming user query into an embedding.
- Query the vector store to find the most similar vector (using cosine similarity).
Pick the Intent:
- Fetch the metadata (intent) of the top result from the vector store.

Why Choose This?

Lightning-fast when dealing with small-to-medium datasets.
Uses semantic similarity, making it a great fit for Retrieval-Augmented Generation workflows.
Scalable with tools like FAISS or ChromaDB for larger use cases.

Challenges:

You’ll need embeddings generated by models like OpenAI’s or Sentence Transformers.
Memory constraints can be a bottleneck if you’re storing everything in-memory.
A well-prepared dataset is key to making this method shine.

Which Method Should You Pick?

Let’s simplify this:

New to this? Start with LLMs—they’re easy and flexible.
Tight on budget? Try machine learning—it’s cost-effective.
Scaling for enterprise? Azure or AWS services are great.
Need speed? Vector-based methods are a solid choice.

Intent classification isn’t just tech—it’s the art of making your chatbot feel human, smart, and intuitive. Whether you’re building something from scratch or optimizing an existing bot, there’s a method here for every stage of your journey.

Start simple, experiment, and watch your bot evolve into the assistant users can’t live without!

What’s your favorite approach? Let’s discuss in the comments!

Visualizing Deep Learning Annotations - Interactive Video Player

Abinash S — Sat, 17 Feb 2024 18:40:49 +0000

In the intricate world of video analytics, the art of visualization is as essential as the core analysis itself. It's a realm where pixels and algorithms weave a visual narrative, and every annotation is a pivotal character. Traditional HTML video tags, adept for basic video playback, find themselves outpaced in terms of annotations, particularly in the world of object tracking and detection.

The Why Behind the What: Annotations

Before we delve into the realm of visualizations, let's explore the concept of annotations. Picture a busy street scene captured on camera – each person and vehicle, a distinct element in a larger visual story. Systems skilled in Person or Object detection and tracking take the lead in this visual waltz.

Models like Yolo step in. They take a video and transform it into a series of bounding boxes, each framing an object or a person. These boxes are given in a data frame, capturing frame numbers, object classes, confidence scores, and respective coordinates. These coordinates mark the boundaries of each bounding box, neatly framing every key element in our scene.

Imagine a scenario where you want precise control over your video annotations. Sometimes, you might need to highlight specific objects, display all annotations, or perhaps none at all, especially when integrating these features into a website or mobile app. Constantly relying on a Python backend to generate and retrieve a new video for each user action is far from ideal. This kind of agility needs to be at the client's fingertips. The twist? The humble HTML <video> tag is not equipped to handle these dynamic, on-the-fly annotations. This is the cue for our custom solution to take center stage – a harmonious blend of a canvas element and a bespoke video player.

Custom Video Player

For this task, Vue was my go-to for its reactive capabilities, complemented by Vuetify's sleek UI components. The result? A video player that not only plays the video but also seamlessly overlays machine learning annotations, synchronized with the video frames and equipped with custom video controls.

Workflow

The heart of our solution lies in integrating the <canvas> element with video playback, allowing us to paint annotations directly over the video. Here is an overview,

Video Tag Creation: Start by creating a video tag as the source for your video.
Setting Up the Canvas: Position a canvas with identical dimensions as the video.
Rendering Video on Canvas: Keep the video tag hidden, and instead, render the video's frames on the canvas, refreshing in sync with the browser's repaint.
Developing Custom Video Controls: Craft a set of video controls using icons and CSS to programmatically manage video playback.

Note: The scope of the blog is to explore building basic canvas and not Vue 3 setup and installation. There are other great blogs out there for the same.

Here's a breakdown of the key components:

Vue Template

Let’s create the template part of our video player component. We will have the hidden video element, the canvas where are going to display video and annotations, a control button to enable and disable the annotations and video controls.



<template>
  <v-container class="mx-auto pa-0" id="canvas-container" style="max-width: 1280px; max-height: 720px">
    <div>
      <video ref="video" style="display: none;">
        <source :src="videoData.source" :type="videoData.type">
      </video>
      <canvas ref="canvas" id="canvas" width="1280" height="720"></canvas>
      <VideoControls @previousFrame="previousFrame" @playVideo="playVideo"
                         @pauseVideo="pauseVideo" @nextFrame="nextFrame" @playFram="playFrame" :isPlaying="isPlaying" :currentFrame="videoData.currentFrame" :currentDuration="videoData.currentDuration" :duration="videoData.duration"/>
            <button class="mx-auto text-center ma-3" @click="showBoundingBoxes = !showBoundingBoxes">
        BBOX
      </button>
    </div>
  </v-container>
</template>

Component/Tag References are important as only through which we will be able to get data and control it in Vue

Annotations

We should have a backend which sends annotations in a format that javascript can handler, preferably json. When loading the video, we should also get the annotations for the video along with it. A sample annotation would look like below,



const annotations = 
{ 
    0: [
            {
          "frameNo": 0,
          "bbox": [295, 40, 750, 50],
          "class": "scoreboard",
          "fillColor": [152, 255, 255],
                "confidence": 0.89
        },
            {
          "frameNo": 0,
          "bbox": [595, 151, 227, 569],
          "class": "player",
          "fillColor": [255, 255, 255]
                "confidence": 0.60
        },
        ],
    // Other frames and their detections
}

Getting started with States

Vue 3's Composition API introduces a more flexible way to manage state in your components. We start by declaring our reactive states using ref



const annotations = ref({
  // ... the annotations constant or load from from http request
});
const showBoundingBoxes = ref(false);
const video = ref(null);
const canvas = ref(null);
const isPlaying = ref(false);
const requestedFrame = ref(0);
const frameRequestId = ref(null);
const ctx = ref(null);
const videoData = ref({
  source: "samplevideo.mp4",
  type: "video/mp4",
  fps: 60,
  duration: 0,
  currentDuration: 0,
  currentFrame: 0
});

Frame Calculation and Rendering

An essential aspect of our player is calculating the current frame based on the duration, as the HTML5 video tag doesn’t provide this directly



const estimateCurrentFrame = (currentTime) => {
  return Math.floor(currentTime * videoData.value.fps) ?? 1;
}

To play the video in canvas, we have to take every frame in the video and draw that in the canvas and check if the frame has any annotations associated. If so, we have to draw boxes in the canvas with the help of bbox coordinates. Let’s create the functions to do these.



const drawFrame = () => {
  videoData.value.currentFrame = estimateCurrentFrame(video.value.currentTime);
  videoData.value.currentDuration = video.value.currentTime;
  // Draw the current frame from the video onto the canvas
  ctx.value.drawImage(video.value, 0, 0, canvas.value.width, canvas.value.height);
  // Continue drawing the next frame
  frameRequestId.value = requestAnimationFrame(drawFrame);
  if (showBoundingBoxes) drawBoundingBoxes(estimateCurrentFrame(video.value.currentTime));
}



const drawBoundingBox = (frameNo, label, x, y, width, height, color) => {
  // Set the color and border width for the bounding box
  ctx.value.strokeStyle = color;
  ctx.value.lineWidth = 2;
  // Draw the bounding box
  ctx.value.beginPath();
  ctx.value.rect(x, y, width, height);
  ctx.value.stroke();
  // Add the label text above the bounding box
  ctx.value.fillStyle = color;
  ctx.value.font = '12px Arial';
  ctx.value.fillText(`${label}`, x, y - 5);
}



const drawBoundingBoxes = (frameNo) => {
  const boundingBoxes = annotations.value[frameNo];
    // Iterate annotations as there can be multiple detections in a single frame
  if (boundingBoxes.length > 0) {
    for (const item of boundingBoxes) {
      let label = item.class;
      const color = `rgb(${item.fillColor.join(',')})`;
      const [x, y, width, height] = item.bbox;
      drawBoundingBox(frameNo, label, x, y, width, height, color);
    }
  }
}

The drawFrame method is pivotal, rendering each frame onto the canvas and overlaying it with annotations. requestAnimationFrame is a JavaScript method used for creating smooth, high-performance animations in web browsers. It tells the browser that you wish to perform an animation and requests that the browser calls a specified function to update an animation before the next repaint.

Key Considerations for Optimal Video Playback

Video Dimensions: Knowing the video's height and width is crucial. These dimensions are essential for correctly sizing the canvas and accurately positioning the bounding boxes. A mismatch in dimensions could lead to improper object tagging.
Frame Rate: Understanding the video's exact frame rate is vital for accurately calculating the current frame during playback. This is essential for synchronizing the bounding boxes with the video. Unfortunately, the HTML5 <video> tag doesn't provide this information directly. However, we can circumvent this by having the server send the frame rate as part of the metadata along with the annotations.
Video Control Actions

To implement the playback controls, we can use the video tag’s default functions.



// Function to play the video
const playVideo = () => {
  video.value.play();
  // Draw the current frame
  drawFrame();
}

// Function to move to the next frame
const nextFrame = () => {
  // Move one frame forward
  video.value.currentTime += 1 / videoData.value.fps;
  drawFrame();
}

// Other required functions

These are the functions we are going to call from the video controls component. By setting video.value.currentTime we can do many things by taking the video to a particular frame, or forward or backward 30 seconds functionality.

Events

In order to capture the video events like play, pause, load, we need to register the events when the component mounts. Let’s do that. We can use loadedmetadata event to know if the video is ready to be played. This can be useful to get the duration, other meta data as well as showing loaders if required. We need to call the drawFrame() here as it is always good to see the first frame of the video before playing it ;)



onMounted(() => {
  ctx.value = canvas.value.getContext('2d');

  video.value.addEventListener('loadedmetadata', () => {
    // The 'loadedmetadata' event is fired when the video's metadata, including the duration, is loaded.
    videoData.value.duration = video.value.duration;
  });

  video.value.addEventListener('play', () => {
    videoData.value.currentDuration = video.value.currentTime;
    videoData.value.currentFrame = estimateCurrentFrame(video.value.currentTime);
    isPlaying.value = true;
    drawFrame();
  });

  video.value.addEventListener('pause', () => {
    isPlaying.value = false;
  });

  video.value.addEventListener('ended', () => {
    isPlaying.value = false;
  });

  drawFrame();
});

Video Playback Controls

As we hid the video and playing with the canvas, we are the ones to design controls for our video. Fortunately, HTML's video tag comes packed with programmable functions, which allowed us to craft our own set of controls tailored to our needs which we have already implemented in our parent component. We just need to call them here.

Lets create a component for the video controls and start with the video slider for video progress and other playback controls,



<template>
    <!-- Custom Slider for Video Progress -->
  <v-row class="pa-2" justify="center">
    <v-slider
        :model-value="currentDuration"
        v-model="thumb"
        @update:modelValue="emit('playFrame', thumb)"
        :max="duration"
        :step="1"
        color="white"
        thumb-color="red"
        hide-details
        rounded
        class="ma-1"
    ></v-slider>
  </v-row>
    <!-- Playback Controls -->
  <v-row class="pa-2 mb-2" justify="center">
    <!-- Play/Pause Buttons -->
    <v-col cols="1" v-if="!isPlaying">
      <v-btn icon="mdi-play-outline" @click="emit('playVideo')"></v-btn>
    </v-col>
        <!-- .... Other Required controls -->
    </v-row>
</template>

Now, as the template is ready, lets do the script part,



<script setup>
import { ref, watch, computed } from 'vue';

// Define props
const props = defineProps({
  isPlaying: Boolean,
  currentFrame: Number,
  currentDuration: Number,
  duration: Number
});

// Define emits
const emit = defineEmits(["previousFrame", "playVideo", "pauseVideo", "nextFrame", "playFrame"]);

const thumb = ref(0);

// Watchers
watch(() => props.currentDuration, (newVal) => {
  thumb.value = newVal;
});

</script>

Here, we show the playing duration, overall duration, play/pause, next frame and previous frame. There is also a slider, sliding on which it will calculate the duration and move the video playback to the relative duration

Our Output

With all the efforts, the result will look like this,

This UI isn't just a static entity; it's a canvas of possibilities, offering vast flexibility. One of the hidden gems (hint: check out the GitHub Gist) is the ability to display video captions, adding another layer of engagement.

And remember, this innovation isn't confined to the realms of Vue. The principles and processes we've explored can be adapted to various web frameworks, showcasing the universality of our approach.While this post has highlighted key aspects and challenges of our project, the full depth and scope are best experienced through the complete code. For the curious minds and avid coders, the scripts await on GitHub (link provided below). Dive in, explore, adapt it to your projects, and maybe even enhance it with your unique touch. I eagerly anticipate seeing how this project grows and transforms with your contributions and insights.

Github Gist: https://gist.github.com/s-abinash/4a3c7afaba94ab9dd74c551f0fe898fc

Going Serverless with Go: A Guide to Lambda Functions

Abinash S — Sat, 17 Feb 2024 06:02:38 +0000

Have you ever thought, “I wish I could just write code without all the extra fuss”? Well, you’re in the right place. Our journey is all about using Go to code without worrying about servers. It’s like having a self-driving car; you just sit back and enjoy the ride.

In this blog, we’ll ease into setting up a REST API using Go in a Serverless fashion to get the user data and save it to DynamoDB whilst keeping it light, friendly, and jargon-free.

Though there are a couple of good blogs for Go Serverless setup, many lack the local development and enhancing the development experience of writing go lambdas. We will touch up on that too ;)

The Serverless Framework

Let's unwrap the concept of 'serverless' with an analogy that's both fun and enlightening. Imagine you're planning a grand celebration (that's your application). In the old days, you'd need to own a venue (a server) to host this bash. Sounds like a heap of responsibility, doesn't it?

Enter the world of serverless – it's akin to hiring an expert party planner. This planner takes care of the venue, the setup, the cleanup – everything but the party itself. That's your job. You bring the code (the life of the party) and let the experts handle the where and how of its execution.

In the realm of technology, serverless computing lets you build and run applications without the burden of server management. It's all about writing your code and letting a cloud provider run it. This provider handles the heavy lifting – scaling up or down as needed, maintaining the servers, and ensuring uptime. It's like having an invisible but incredibly efficient helper ensuring your app can handle any amount of guests (users) without breaking a sweat.

This open-source wonder is not just a tool; it's a wizard in your coding journey. Serverless Framework streamlines the process of deploying serverless applications, taking the tediousness out of server management. What sets the Serverless Framework apart from other tools in its league? Its ability to play nice with multiple cloud providers. Though it began with a focus on AWS, it's grown to support Azure, GCP, and many others. This multi-cloud provider support means you're not putting all your eggs in one basket. You have the flexibility to choose or switch your cloud provider based on your needs, costs, or even whims!

Serverless Configuration

Equip Yourself with the Right Tools

Before we embark on this adventure, make sure you have two key items in your toolkit:

Go - If Go isn't already part of your developer toolkit, now's the time to welcome it aboard. Head over to the official Go website and grab the latest version. We're focusing on Go 1.21 and above in this blog, so make sure you're up-to-date. The installation is a breeze – just follow the instructions for your operating system, and you're set!
Serverless Framework - This nifty tool is like having a personal assistant for your serverless adventure. To get this assistant on board, simply run npm install -g serverless in your command line. This requires Node.js, so if you don't have it yet, a quick trip to the Node.js website will sort that out. As a bonus, we'll sprinkle in some nodemon magic for auto-reloading our functions, making our development process smoother.
Boiler Plate Code - Let's lay the foundation with some boilerplate code. Fire up your command line and execute serverless create --template aws-go --path go-serverless-api. Once done, step into your new project with cd go-serverless-api. Though it's optional (since we'll be customizing the Go file anyway), it's a great starting point and sets the tone for our serverless application.

To ensure our serverless function can smoothly chat with DynamoDB, we need to grant it the right permissions. Let's add an IAM Role statement to our serverless.yml file that allows our function to access DynamoDB:

service: go-serverless-api

frameworkVersion: "3"

useDotenv: true
provider: 
  name: aws
  runtime: go1.x # Change to `provided.al2` while deployment
  stage: ${opt:stage, 'dev'}
  timeout: 120 # Change to 30 while deployment
  memorySize: 256
  versionFunctions: false
  iamRoleStatements: 
    - Effect: Allow
      Action: 
        - dynamodb:Query
        - dynamodb:Scan
        - dynamodb:GetItem
        - dynamodb:BatchGetItem
      Resource: "arn:aws:dynamodb:us-east-1:${env:ACCOUNT_ID}:table/${env:TABLE_NAME}"

We will install three plugins to help the development,

serverless-offline, lets us emulate AWS Lambda and API Gateway locally, making testing a breeze
serverless-dotenv-plugin, load your .env file variables automatically. It's optional, though, if you prefer the manual approach
serverless-go-plugin, automatically compiles go functions on sls deploy. Say goodbye to manual build commands

Install all the plugins using your favourite package managers (mine is bun) and we will be adding them to the plugin section of serverless.yaml like this,

plugins: 
  - serverless-offline
  - serverless-go-plugin
  - serverless-dotenv-plugin

When it comes to adding functions to our serverless.yaml, it's all about personal preference. However, as the number of functions grows, so does the complexity. To keep things neat and tidy, let's define our functions in a separate .yaml file and reference them in our main serverless.yaml

functions: 
  - ${file(functions/functions.yml):functions}

We need a DynamoDB db to work with. You can either use the local DynamoDB client (localhost url has to be mentioned while creating dynamo client) or create a dynamo table in AWS with default settings and name the primary and sort keys as pk and sk

With these tools in hand, we are now equipped, so let's turn the page to the next chapter of our adventure and start crafting some go magic!

Serverless Function

Utilities

Logging with Zerolog

Effective logging is crucial for any function to gain insights and debug issues. We're going to integrate zerolog, an excellent logging library known for its simplicity and performance. Here's how we set it up as both a utility:

func init() {
    once.Do(func() {
        env := config.GetConfig() // To use env from go-envconfig
        var output io.Writer
        zerolog.ErrorStackMarshaler = pkgerrors.MarshalStack
        zerolog.TimeFieldFormat = time.RFC3339
        logLevel, err := strconv.Atoi(env.LogLevel)
        if err != nil {
            logLevel = int(zerolog.InfoLevel)
        }
        Logger = zerolog.New(os.Stdout).Level(zerolog.Level(logLevel)).With().Timestamp().Caller().Logger()
    })
}

Env management with go-envconfig

For managing environment variables, go-envconfig is our go-to solution. It simplifies loading and parsing environment configurations. The setup code can be found in the GitHub repository linked at the end.

Defining Structs

We'll define a User struct with both json and dynamodbav tags. The dynamodbav tags specify column names in DynamoDB, while json tags are used for parsing data into this struct.

type primaryKey struct {
    Pk string `json:"pk" dynamodbav:"pk"`
    Sk string `json:"sk" dynamodbav:"sk"`
} 

type User struct {
    primaryKey
    ID        string `json:"id" dynamodbav:"id"`
    Name      string `json:"name" dynamodbav:"name"`
    Email     string `json:"email" dynamodbav:"email"`
    Age       int    `json:"age" dynamodbav:"age"`
    IsActive  bool   `json:"isActive" dynamodbav:isActive""`
}

Dynamo Wrappers

Establishing a connection to DynamoDB is essential. Here’s a function to get a DynamoDB client:

type DB struct {
    client    *dynamodb.Client
    table     string
}

func GetDynamoDBClient(region string) (*dynamodb.Client, error) {
    cfg, err := config.LoadDefaultConfig(context.Background(), config.WithRegion(region))
    if err != nil {
        log.Error().Err(err).Msg(DBError.DBFailedToLoad)
        return nil, err
    }
    dynamodbClient := dynamodb.NewFromConfig(cfg)
    return &DB{
        client:    dynamodbClient,
        tableName: tableName,
    }, err
}

For storing data, we have a wrapper function that accepts any data type and saves it to DynamoDB:

func putItemWrapper(ctx context.Context, db *DB, inputItem interface{}) (*dynamodb.PutItemOutput, error) {
    item, err := attributevalue.MarshalMap(inputItem)
    if err != nil {
        return nil, err
    }

    response, err := db.client.PutItem(ctx, &dynamodb.PutItemInput{
        TableName: aws.String(db.tableName),
        Item:      item,
    })
    if err != nil {
        return nil, err
    }
    return response, nil
}

Go Handler

Now, we reach the heart of our application - the Go handler function itself. This is where the magic happens, turning data into action. We're using aws-lambda-go, the official AWS package for Go Lambda functions, to make this happen. Here's a quick note on structure:

The function should reside in the main package.
Only one handler file should exist at the same folder level (for serverless-offline and serverless-go-plugin to work properly).
Feel free to organize utilities, types, and other components in separate files and import them into your handler.

Our handler would parse the incoming requests body and un marshals into our user struct which we then send to DynamoDB.

func Handler(ctx context.Context, request Request) (Response, error) {
    userItem := User{}
    err := json.Unmarshal([]byte(req.Body), &user)
    if err != nil {
        return JSONResponse(http.StatusBadRequest, err)
    }
    db, _ := dao.GetDynamoDBClient(env.DynamoTable, env.AwsRegion)
    err = dao.PutUserInfo(ctx, db, user)
    if err != nil {
        return NewJSONErrorResponse(http.StatusInternalServerError)
    }
    return JSONResponse(http.StatusCreated, map[string]string{"msg":"User Created"}
}

Note here the JSONResponse utility makes it easy task for us to send back JSON responses, whether it's an error, a struct response, or a map response.

I have considered the simplest access patterns for this project,

Save user with their details
Retrieve user given user id

You can expand access patterns using Global Secondary Indexes (GSIs). With support for up to 20 GSIs per table, DynamoDB offers plenty of flexibility for various query requirements.

We might need a function to return us a struct which has both primary key and the user info. This function can be used in both put and get operations of DynamoDB

func GetUserDBItem(ctx context.Context, user UserInfo) UserDBItem {
    primaryKey := primaryKey{
        Pk: fmt.Sprintf("userId#%s", user.ID),
        Sk: fmt.Sprintf("userId#%s", user.ID),
    }
    userInfoDBItem := UserDBItem{
        primaryKey: primaryKey,
        UserInfo:   user,
    }
    return userInfoDBItem
}

And giving the output of this function to our previously written putItemWrapper would create the user for us in DynamoDB 🎉

userDBItem := GetUserDBItem(ctx, user)
_, err := putItemWrapper(db, userDBItem)

And as a last touch, let’s not forget to add our function definition in the function.yaml file to make sure serverless considers our lambda for deployments

functions:
  save-user:
    name: save-user
    handler: functions/save-user/main.go
    events:
      - http:
          path: users
          method: post

We can test our lambda’s locally using sls offline command, with nodemon for hot reloading. Make sure to checkout the repo mentioned below 😉

With all the configuration in place, a simple sls deploy command in your terminal breathes life into your function. Hit Enter, and voilà - our serverless function is up and running, ready to save users to your DynamoDB table! 🎉

Dive deeper into this go serverless world with more utilities and comprehensive implementations. Check out the complete codebase in this GitHub Repo - https://github.com/s-abinash/go-serverless-api

DEV Community: Abinash S

Build Intelligent ChatBots with Language Processing

Why Does Intent Classification Matter?

Approaches to Intent Classification

1. Machine Learning / Deep Learning Models

Here’s the gist:

Here’s how you might integrate it:

2. Large Language Models (LLMs)

3. Azure Language Service

How does it work?

Which Method Should You Pick?

Visualizing Deep Learning Annotations - Interactive Video Player

The Why Behind the What: Annotations

Custom Video Player

Workflow

Vue Template

Annotations

Getting started with States

Frame Calculation and Rendering

Video Control Actions

Events

Video Playback Controls

Our Output

Going Serverless with Go: A Guide to Lambda Functions

The Serverless Framework

Serverless Configuration

Equip Yourself with the Right Tools

Serverless Function

Utilities

Go Handler