DEV Community: BERAT DİNÇKAN

Building Multi-Agent Systems: @openai/agents Handoffs with NestJS and TypeScript

BERAT DİNÇKAN — Tue, 15 Jul 2025 21:30:17 +0000

🤖 AI Agent Development with NestJS

A comprehensive guide and example project demonstrating how to build an intelligent photo recognition system for identifying cat-safe plants using NestJS framework.

📖 About This Project

This project is a photo recognition system designed to identify whether plant leaves are harmful, poisonous, or safe for cats. Using AI-powered image analysis, the system can analyze photos submitted by clients and determine the potential danger level of various plants to feline companions. The application leverages OpenAI's Agent SDK integrated with NestJS to provide intelligent plant identification and safety assessment.

🔧 Implementation

Step 1: Install @openai/agents

First you need to install the @openai/agents package:

npm install @openai/agents

Step 2: Create Agent Module

Create a module for the AI agent:

nest g module agent

Step 3: Create Agent Service

Create an agent service:

nest g service agent

📁 Project Structure

src/
├── agent/
│   ├── agent.module.ts
│   └── agent.service.ts
├── app.controller.ts
├── app.module.ts
├── app.service.ts
└── main.ts

Step 4: Initialize Agent Service

Initialize the agent service with OpenAI API key. The setDefaultOpenAIKey function configures the global API key for all OpenAI agent operations:

import { Injectable } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { setDefaultOpenAIKey } from '@openai/agents';

@Injectable()
export class AgentService {
  constructor(private configService: ConfigService) {
    const openaiApiKey = this.configService.get<string>('OPENAI_API_KEY');
    if (!openaiApiKey) {
      throw new Error('OPENAI_API_KEY is required but not provided');
    }
    setDefaultOpenAIKey(openaiApiKey);
  }
}

Step 5: Configure Environment

Set up the ConfigModule in your app module:

import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import { AgentModule } from './agent/agent.module';

@Module({
  imports: [ConfigModule.forRoot({ isGlobal: true }), AgentModule],
})
export class AppModule {}

🚀 Getting Started

Create first agent to check if the plant is harmful

import { Agent, webSearchTool } from '@openai/agents';
import { harmfulPrompt } from '../prompts/harmful.prompt';

export const HarmfulAgent = new Agent({
  name: 'Harmful Plant Identifier',
  instructions: harmfulPrompt.join('\n'),
  tools: [webSearchTool()],
  model: 'gpt-4.1-mini',
});

Agent Configuration Explained

The agent configuration consists of several key components:

name: Sets a descriptive name for the agent that identifies its purpose (in this case, "Harmful Plant Identifier")
instructions: Contains the prompt that tells the agent how to behave and what tasks to perform
tools: An array of tools the agent can use. Here we include webSearchTool() which enables the agent to search the internet for real-time information about plants
model: Specifies which OpenAI model to use. We're using gpt-4.1-mini which provides a good balance of performance and cost for this type of analysis

So what about the prompt? Let's take a look at how we structure our prompt:

export const harmfulPrompt: string[] = [
  'you are a veterinarian who is specialized in cat and at the same time you are a plant expert',
  'you search the internet for the plant name and the plant description',
  'Your task is to:',
  '1- identify the plant on the photo',
  '2- provide the which part of the plant is harmful to cats or which part of the plant is safe for cats',
  '3- How much of a plant would harm a cat?',
  '4- provide the plant description',
  '6- provide the plant name',
  '7- provide comperensive chemicals list of the plant',
  '',
  'IMPORTANT: Always set agentType to "HARMFUL" in your response.',
];

Why Use Prompt Arrays?

Instead of writing one long string, we use an array of strings for several advantages:

Better Readability: Each instruction is on its own line, making the prompt easier to read and understand
Easier Maintenance: You can easily add, remove, or reorder instructions without dealing with complex string concatenation
Version Control Friendly: Changes to individual instructions are clearly visible in git diffs
Modular Structure: You can break down complex prompts into logical sections

The join('\n') method converts the array back to a single string with newlines between each instruction, which is the format the agent expects.

Step 6: Implementing the Harmful Agent to the Agent Service

import { Injectable } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import {
  Agent,
  AgentInputItem,
  run,
  setDefaultOpenAIKey,
} from '@openai/agents';
import { HarmfulAgent } from './agents/harmful.agent';

@Injectable()
export class AgentService {
  private harmfulAgent: Agent;
  constructor(private configService: ConfigService) {
    const openaiApiKey = this.configService.get<string>('OPENAI_API_KEY');
    if (!openaiApiKey) {
      throw new Error('OPENAI_API_KEY is required but not provided');
    }
    setDefaultOpenAIKey(openaiApiKey);
    this.harmfulAgent = HarmfulAgent;
  }

  async checkThePlantIsHarmful(base64Image: string) {
    const imageUrl = `data:image/jpeg;base64,${base64Image}`;
    const agentInputItem: AgentInputItem = {
      role: 'user',
      content: [
        {
          type: 'input_image',
          image: imageUrl,
        },
      ],
    };
    const result = await run(this.harmfulAgent, [agentInputItem]);
    return result.finalOutput;
  }
}

Understanding the `checkThePlantIsHarmful` Function

This function demonstrates the proper way to handle image inputs when working with OpenAI agents. Here's what's happening:

Function Purpose: The function takes a base64-encoded image string as input and uses the harmful agent to analyze whether the plant in the image is safe for cats.
Base64 to Data URL Conversion: The function converts the base64 image data to a data URL format that the OpenAI agent can process:

   const imageUrl = `data:image/jpeg;base64,${base64Image}`;

AgentInputItem Structure: Instead of passing the image URL as a plain string, we create an AgentInputItem object with:
- role: 'user' - Indicates this is a user input
- content - An array containing the actual input data
- type: 'input_image' - Specifies that this is an image input
- image: imageUrl - The data URL of the image to analyze
Why Use AgentInputItem Instead of Plain Strings?

❌ Problem with String Approach:

   // DON'T DO THIS - Can cause token limit issues
   const result = await run(this.harmfulAgent, [
     { role: 'user', content: `Analyze this image: ${imageUrl}` },
   ]);

✅ Correct Approach (What We're Using):

   // DO THIS - Efficient and avoids token limits
   const agentInputItem: AgentInputItem = {
     role: 'user',
     content: [{ type: 'input_image', image: imageUrl }],
   };

Token Limit Benefits: When you use the input_image type, the agent processes the image directly without converting it to text tokens. This means:
- You avoid consuming large amounts of tokens for image data
- The agent can process high-resolution images efficiently
- You stay within OpenAI's token limits even with complex images
- Better performance and cost-effectiveness
Agent Execution: The function then runs the harmful agent with the properly formatted input and returns the analysis result.
Integration with File Upload: This function is designed to work with file upload middleware (like Multer) that provides base64-encoded image data, making it easy to integrate with web forms and mobile applications.

Now let's create an endpoint to run our agent and analyze plant images.

Created a new controller to handle the request and response:

Create cat-safe module

nest g module cat-safe

Create cat-safe controller

nest g controller cat-safe

Create cat-safe service

nest g service cat-safe

📁 Updated Project Structure


src/
├── agent/
│ ├── agent.module.ts
│ ├── agent.service.ts
│ ├── agents/
│ │ └── harmful.agent.ts
│ └── prompts/
│ └── harmful.prompt.ts
├── cat-safe/
│ ├── cat-safe.controller.ts
│ ├── cat-safe.module.ts
│ └── cat-safe.service.ts
├── app.controller.ts
├── app.module.ts
├── app.service.ts
└── main.ts

Implement the agent service to the cat-safe service

import { Injectable } from '@nestjs/common';
import { AgentService } from '../agent/agent.service';

@Injectable()
export class CatSafeService {
  constructor(private readonly agentService: AgentService) {}

  async checkThePlantIsHarmful(image: Express.Multer.File) {
    const base64Image = image.buffer.toString('base64');
    const result = await this.agentService.checkThePlantIsHarmful(base64Image);
    return result;
  }
}

Now We can use the cat-safe service in the cat-safe controller

import {
  Body,
  Controller,
  HttpStatus,
  ParseFilePipeBuilder,
  Post,
  UploadedFile,
  UseInterceptors,
} from '@nestjs/common';
import { CatSafeService } from './cat-safe.service';
import { FileInterceptor } from '@nestjs/platform-express';
import { ApiBody, ApiConsumes } from '@nestjs/swagger';

@Controller('cat-safe')
export class CatSafeController {
  constructor(private readonly catSafeService: CatSafeService) {}

  @Post('check-harmful')
  @UseInterceptors(FileInterceptor('image'))
  @ApiConsumes('multipart/form-data')
  @ApiBody({
    schema: {
      type: 'object',
      properties: {
        image: {
          type: 'string',
          format: 'binary',
          description: 'the image of the user',
        },
      },
    },
  })
  async checkHarmful(
    @UploadedFile(
      new ParseFilePipeBuilder()
        .addFileTypeValidator({
          fileType: /(jpg|jpeg|png)$/,
        })
        .addMaxSizeValidator({
          maxSize: 5 * 1024 * 1024, // 5MB
        })
        .build({
          errorHttpStatusCode: HttpStatus.UNPROCESSABLE_ENTITY,
        }),
    )
    image: Express.Multer.File,
  ) {
    return this.catSafeService.checkThePlantIsHarmful(image);
  }
}

Implement the swagger documentation to main.ts

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { DocumentBuilder, SwaggerModule } from '@nestjs/swagger';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  const config = new DocumentBuilder()
    .setTitle('Cat Safe API')
    .setDescription('API for checking if a plant is harmful to cats')
    .setVersion('1.0')
    .build();
  const document = SwaggerModule.createDocument(app, config);
  SwaggerModule.setup('swagger', app, document);
  await app.listen(3000);
}
bootstrap();

Now We can run the application and test the api using the swagger ui

npm run start:dev

We can go to http://localhost:3000/swagger to see the swagger ui

When I test the api with the image of Spider Plant 🕷️, I got the following result:

⚠️ IMPORTANT: Response Format Issue

As we can see, the agent is able to identify the plant and provide the information about the plant. However, there's a critical issue with the response format that needs immediate attention:

🚨 Current Problem:

The response body is not in a good format - it's raw, unstructured text that's difficult to parse and understand programmatically.

🎯 Why This Matters:

API Integration: Other applications can't easily consume unstructured responses
User Experience: Frontend applications need structured data to display information properly
Error Handling: Without structured responses, it's hard to handle different types of responses
Scalability: Unstructured responses make it difficult to add features like response caching or analytics

🔧 What We Need to Fix:

We need to improve the response body to be more readable, structured, and easy to understand. This is a critical improvement that will make our API production-ready.

Next Steps: We'll implement structured response formatting to ensure consistent, parseable responses.

🔍 Solution: Structured Response Formatting

We'll implement structured response formatting to ensure consistent, parseable responses using outputType in the harmful agent.

🎯 What is `outputType`?

outputType is a powerful function that takes a Zod schema and returns a function that can be used to parse and validate the response body. This ensures:

✅ Consistent Response Structure: Every response follows the same format
✅ Type Safety: TypeScript can infer the response type
✅ Validation: Automatic validation of response data
✅ Error Handling: Clear error messages for invalid responses

🔧 How `outputType` Works:

Define a Zod Schema: Create a schema that describes your expected response structure
Apply to Agent: Use outputType(schema) when creating your agent
Automatic Parsing: The agent will automatically format responses according to your schema
Type Safety: TypeScript will know exactly what structure your response will have

📋 Benefits of Using `outputType`:

Benefit	Description
🏗️ Structure	Consistent JSON responses instead of raw text
🔒 Validation	Automatic validation of response data
📝 Type Safety	TypeScript can infer the response type
🐛 Debugging	Clear error messages for malformed responses
⚡ Performance	Faster parsing and processing
🔄 Integration	Easy integration with frontend applications

🚀 Implementation Steps:

Define Response Schema using Zod
Update Agent Configuration with outputType
Test Structured Responses

Let's implement this step by step!

Define the response schema using zod. but before that, we need to create a new enum for the agent type.

agent.enum.ts

export enum AgentType {
  HARMFUL = 'HARMFUL',
  POISONOUS = 'POISONOUS',
}

In agent.enum.ts, we define the agent type.

In harmful.schema.ts, we use the agent type enum to define the agent type.

harmful.schema.ts

import { z } from 'zod';
import { AgentType } from '../agent.enum';

export const HarmfulSchema = z.object({
  isHarmful: z.boolean(),
  plantName: z.string(),
  plantScientificName: z.string(),
  chemicalComposition: z.string(),
  chemicalList: z.array(
    z.object({
      name: z.string(),
      description: z.string(),
      toxicity: z.boolean(),
      toxicityLevel: z.number().min(0).max(10),
      toxicityDescription: z.string(),
      toxicitySymptoms: z.array(z.string()),
      toxicityTreatment: z.string(),
    }),
  ),
  toxicityThreshold: z.string(),
  agentType: z.nativeEnum(AgentType),
});

export type HarmfulSchemaType = z.infer<typeof HarmfulSchema>;

Update the harmful agent and agent service to use the harmful schema

In harmful.agent.ts, we need to update the outputType to use the harmful schema.

import { Agent, webSearchTool } from '@openai/agents';
import { harmfulPrompt } from '../prompts/harmful.prompt';
import { HarmfulSchema } from '../schemas/harmful.schema';

export const HarmfulAgent = new Agent({
  name: 'Harmful Plant Identifier',
  instructions: harmfulPrompt.join('\n'),
  tools: [webSearchTool()],
  model: 'gpt-4.1-mini',
  outputType: HarmfulSchema,
});

In agent.service.ts, we need to update the harmfulAgent to use the harmful schema. This is done by using the Agent<unknown, typeof HarmfulAgent.outputType> type. This facilitates the type safety and validation of the response body.

private harmfulAgent: Agent<unknown, typeof HarmfulAgent.outputType>;

When we run the application, we can see the response body is in a good format like this:

{
  "isHarmful": false,
  "plantName": "Spider Plant",
  "plantScientificName": "Chlorophytum comosum",
  "chemicalComposition": "Spider plants are known for their air-purifying properties, effectively removing toxins like formaldehyde, xylene, and carbon monoxide from indoor environments. However, specific chemical compositions of the plant are not well-documented in available sources.",
  "chemicalList": [
    {
      "name": "Formaldehyde",
      "description": "A common indoor air pollutant that spider plants can absorb, improving air quality.",
      "toxicity": true,
      "toxicityLevel": 2,
      "toxicityDescription": "Formaldehyde is a toxic compound that can cause respiratory issues and other health problems in humans and animals.",
      "toxicitySymptoms": ["Respiratory irritation", "Headaches", "Dizziness"],
      "toxicityTreatment": "Remove the source of exposure and seek medical attention if symptoms persist."
    },
    {
      "name": "Xylene",
      "description": "An organic compound found in indoor air that spider plants can help reduce.",
      "toxicity": true,
      "toxicityLevel": 3,
      "toxicityDescription": "Xylene exposure can lead to headaches, dizziness, and nausea in humans and animals.",
      "toxicitySymptoms": ["Headaches", "Dizziness", "Nausea"],
      "toxicityTreatment": "Ensure fresh air circulation and consult a healthcare professional if symptoms continue."
    },
    {
      "name": "Carbon Monoxide",
      "description": "A colorless, odorless gas that spider plants can help mitigate in indoor spaces.",
      "toxicity": true,
      "toxicityLevel": 5,
      "toxicityDescription": "Carbon monoxide is highly toxic and can be fatal at high concentrations.",
      "toxicitySymptoms": [
        "Headaches",
        "Dizziness",
        "Nausea",
        "Confusion",
        "Loss of consciousness"
      ],
      "toxicityTreatment": "Move to fresh air immediately and seek emergency medical care."
    }
  ],
  "toxicityThreshold": "Spider plants are non-toxic to cats and are safe for them to ingest. There is no known harmful threshold for cats consuming spider plants.",
  "agentType": "HARMFUL"
}

What would happen if the plant was poisonous? For this reason, we need to create a new agent for the poisonous plants.

🌿 Building the Poisonous Plant Analysis System

🔄 Step-by-Step Development Process

We'll follow a systematic approach to build the poisonous plant agent:

📝 Create Prompts - Define the AI's behavior and analysis framework
🏗️ Create Schema - Structure the response data format
🤖 Create Agent - Combine prompts and schema into a working agent

📝 Step 1: Create the Poisonous Plant Prompt

First, we need to create a comprehensive prompt that guides the AI to perform detailed toxicological analysis.

Create src/agent/prompts/poisonous.prompt.ts:

export const poisonousPrompt: string[] = [
  'You are an expert veterinary toxicologist and botanist specializing in feline health and plant identification.',
  'You have extensive knowledge of plant toxicology, chemical compounds, and their effects on cats.',
  'IMPORTANT: You MUST use web search throughout your analysis to find current and accurate information.',
  '',
  'CONTEXT:',
  'You will be provided with images of plants and cats. Your role is to assess potential toxicity risks.',
  '',
  'YOUR ANALYSIS MUST INCLUDE:',
  '',
  '1. PLANT IDENTIFICATION:',
  '   - Scientific name (genus and species)',
  '   - Common names',
  '   - Plant family classification',
  '   - Key identifying characteristics visible in the image',
  '',
  '2. TOXICITY ASSESSMENT:',
  '   - Overall toxicity level to cats (Non-toxic, Mildly toxic, Moderately toxic, Highly toxic, Lethal)',
  '   - Specific plant parts that are toxic (leaves, stems, flowers, roots, seeds, sap, etc.)',
  '   - Parts that are safe (if any)',
  '',
  '3. TOXIC COMPOUNDS:',
  '   - Identify specific toxic chemicals/compounds present',
  '   - Chemical classification (alkaloids, glycosides, saponins, oxalates, etc.)',
  '   - Mechanism of action (how these compounds affect cat physiology)',
  '   - Target organs/systems affected',
  '',
  '4. DOSAGE AND SEVERITY:',
  '   - Minimum toxic dose (if known)',
  '   - Relationship between amount consumed and severity of symptoms',
  '   - Factors affecting toxicity (cat size, age, health status)',
  '',
  '5. CLINICAL PRESENTATION:',
  '   - Early symptoms and signs',
  '   - Progressive symptoms if untreated',
  '   - Timeline of symptom onset',
  '   - Potential complications',
  '',
  '6. EMERGENCY RESPONSE:',
  '   - Immediate first aid measures',
  '   - When to seek veterinary care urgently',
  '   - What information to provide to the veterinarian',
  '',
  '7. PREVENTION RECOMMENDATIONS:',
  '   - Safe placement strategies if keeping the plant',
  '   - Cat-safe alternatives with similar appearance',
  '   - Environmental modifications to prevent access',
  '',
  '8. EMERGENCY VETERINARY LOCATOR:',
  '   - If the plant is poisonous and longitude/latitude coordinates are provided',
  '   - MUST use online web search to find and list the nearest 10 veterinary clinics or emergency animal hospitals',
  '   - Search online veterinary directories, Google Maps, emergency vet websites',
  '   - Include clinic names, addresses, phone numbers, and distance from location',
  '   - Prioritize 24-hour emergency clinics when available',
  '   - Verify clinic information is current through online search',
  '',
  'FORMAT YOUR RESPONSE:',
  'Structure your analysis clearly with headers for each section above.',
  'Use bullet points for easy reading.',
  'Highlight critical information about severe toxicity.',
  'Provide specific, actionable advice.',
  '',
  'IMPORTANT NOTES:',
  '- If plant identification is uncertain, mention possible alternatives',
  '- Always err on the side of caution regarding toxicity',
  '- Include disclaimer about seeking professional veterinary advice',
  '- Mention that individual cats may react differently',
  'you search the internet for the plant name and the plant description',
  '',
  'IMPORTANT: Always set agentType to "POISONOUS" in your response.',
];

🎯 Why This Prompt Structure Works:

Comprehensive Analysis: Covers 8 key areas from plant ID to emergency response
Web Search Integration: Explicitly requires internet research for current information
Emergency Focus: Includes veterinary locator for poisoning emergencies
Structured Output: Clear formatting guidelines for consistent responses
Safety First: Emphasizes caution and professional veterinary consultation

🏗️ Step 2: Create the Poisonous Plant Schema

Next, we define a Zod schema that structures the response data to match our prompt requirements.

Create src/agent/schemas/poisonous.schema.ts:

import { z } from 'zod';
import { AgentType } from '../agent.enum';

export const PoisonousSchema = z.object({
  isPoisonous: z.boolean(),
  plantName: z.string(),
  plantScientificName: z.string(),
  plantDescription: z.string(),
  toxicityLevel: z.string(), // Non-toxic, Mildly toxic, Moderately toxic, Highly toxic, Lethal
  toxicPlantParts: z.array(z.string()), // leaves, stems, flowers, roots, etc.
  safePlantParts: z.array(z.string()),
  toxicCompounds: z.array(
    z.object({
      name: z.string(),
      chemicalClassification: z.string(), // alkaloids, glycosides, saponins, etc.
      mechanismOfAction: z.string(),
      targetOrgans: z.array(z.string()),
      toxicityDescription: z.string(),
    }),
  ),
  clinicalSigns: z.object({
    earlySymptoms: z.array(z.string()),
    progressiveSymptoms: z.array(z.string()),
    onsetTimeline: z.string(),
  }),
  emergencyResponse: z.object({
    firstAidSteps: z.array(z.string()),
    whenToSeekHelp: z.string(),
    vetInformation: z.array(z.string()),
  }),
  nearestVets: z.array(
    z.object({
      name: z.string(),
      address: z.string(),
      phoneNumber: z.string(),
      distance: z.string(),
      isEmergency24h: z.boolean(),
    }),
  ),
  locationCoordinates: z.object({
    latitude: z.number(),
    longitude: z.number(),
  }),
  toxicityThreshold: z.string(),
  agentType: z.nativeEnum(AgentType),
});

export type PoisonousSchemaType = z.infer<typeof PoisonousSchema>;

🔍 Schema Design Principles:

Follows Harmful Pattern: Consistent with existing harmful.schema.ts structure
Comprehensive Data: Captures all 8 sections from the prompt
Emergency Ready: Requires vet locator and location coordinates for complete emergency response
Type Safety: Full TypeScript support with inferred types
Validation: Automatic data validation and error handling
No Optional Fields: All fields are required to ensure complete and consistent responses

🤖 Step 3: Create the Poisonous Plant Agent

Finally, we combine the prompt and schema into a working agent.

Create src/agent/agents/poisonous.agent.ts:

import { Agent, webSearchTool } from '@openai/agents';
import { poisonousPrompt } from '../prompts/poisonous.prompt';
import { PoisonousSchema } from '../schemas/poisonous.schema';

export const PoisonousAgent = new Agent({
  name: 'Poisonous Plant Analyzer',
  instructions: poisonousPrompt.join('\n'),
  tools: [webSearchTool()],
  model: 'gpt-4.1-mini',
  outputType: PoisonousSchema,
});

Now We can use the poisonous agent in the cat-safe service and controller but something is missing. Do we have to send two separate requests to understand if a plant is harmful AND get poisonous plant details? This is inefficient and creates a poor user experience.

🔄 Agent Handoffs: The Solution

The problem with our current implementation is that we need to make two separate API calls:

One to HarmfulAgent to check if the plant is harmful
Another to PoisonousAgent to get detailed toxicity information

Agent handoffs solve this by allowing agents to coordinate and pass control to each other within a single conversation thread.

🎯 How Agent Handoffs Work

Agent handoffs enable:

Single Request: User makes one API call
Agent Coordination: Agents automatically decide when to hand off to specialists
Contextual Transfer: Full conversation context passes between agents
Intelligent Routing: Agents choose the best specialist for each task

🏗️ Implementing Agent Handoffs with Cat-Safe Agent

Let's create a Cat-Safe Agent that implements handoff logic internally through intelligent prompt design.

🧠 Create the Cat-Safe Prompt with Handoff Logic

Create src/agent/prompts/cat-safe.prompt.ts:

export const catSafePrompt: string[] = [
  'You are a plant safety coordinator responsible for analyzing plant images to ensure cat safety.',
  'Your primary job is to examine plant images and determine which specialist agent should handle the analysis.',
  '',
  'ANALYSIS PROCESS:',
  '',
  '1. Carefully examine the provided plant image',
  '2. Identify the plant species if possible',
  '3. Determine if the plant is poisonous to cats',
  '4. IMMEDIATELY hand off to the appropriate specialist agent',
  '',
  'DECISION LOGIC:',
  '',
  'If the plant is POISONOUS (toxic to cats):',
  '- Examples: Lilies, Azaleas, Oleander, Foxglove, Sago Palm, Tulips, Daffodils',
  '- IMMEDIATELY hand off to PoisonousAgent',
  '- Do NOT provide detailed analysis yourself',
  '',
  'If the plant is NOT POISONOUS (not toxic for cats but harmful for cats):',
  '- Examples: Spider Plant, Boston Fern, Cat Grass, Catnip, Rosemary, Basil',
  '- IMMEDIATELY hand off to HarmfulAgent',
  '- Do NOT provide detailed analysis yourself',
  '',
  'HANDOFF RULES:',
  '',
  '- You are ONLY a coordinator, not an analyzer',
  '- Your job is to quickly identify and route, not to provide detailed information',
  '- Always hand off within your first response',
  '- Never provide comprehensive plant analysis yourself',
  '',
  'SAFETY PRIORITY:',
  'If you are uncertain about a plants toxicity, hand off to PoisonousAgent to be safe.',
];

🤖 Create the Cat-Safe Agent

Create src/agent/agents/cat-safe.agent.ts:

import { Agent } from '@openai/agents';
import { catSafePrompt } from '../prompts/cat-safe.prompt';
import { HarmfulAgent } from './harmful.agent';
import { PoisonousAgent } from './poisonous.agent';

export const CatSafeAgent = Agent.create({
  name: 'Cat Safe Plant Analysis Coordinator',
  instructions: catSafePrompt.join('\n'),
  model: 'gpt-4.1-mini',
  handoffs: [HarmfulAgent, PoisonousAgent],
});

⚠️ IMPORTANT: Agent.create vs new Agent()

When you use handoffs that result in different output types, you MUST create the agent with Agent.create() instead of new Agent().

Why this matters:

HarmfulAgent uses HarmfulSchema (with fields like isHarmful, chemicalList, etc.)
PoisonousAgent uses PoisonousSchema (with fields like isPoisonous, clinicalSigns, emergencyResponse, etc.)

❌ Wrong approach:

// DON'T DO THIS - Will cause type conflicts
export const CatSafeAgent = new Agent({
  name: 'Cat Safe Plant Analysis Coordinator',
  instructions: catSafePrompt.join('\n'),
  handoffs: [HarmfulAgent, PoisonousAgent], // Different output types!
  model: 'gpt-4.1-mini',
});

✅ Correct approach:

// DO THIS - Handles multiple output types correctly
export const CatSafeAgent = Agent.create({
  name: 'Cat Safe Plant Analysis Coordinator',
  instructions: catSafePrompt.join('\n'),
  model: 'gpt-4.1-mini',
  handoffs: [HarmfulAgent, PoisonousAgent], // Different output types handled properly
});

Key Benefits of Agent.create():

✅ Type Safety: Properly handles multiple output schemas
✅ Flexible Responses: Each handoff can return its appropriate schema

🔧 Update Agent Service with Location Integration

Update agent.service.ts to implement handoff logic with location data:

import { Injectable } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import {
  Agent,
  AgentInputItem,
  run,
  setDefaultOpenAIKey,
} from '@openai/agents';
import { CatSafeAgent } from './agents/cat-safe.agent';

@Injectable()
export class AgentService {
  private catSafeAgent: Agent<unknown, typeof CatSafeAgent.outputType>;
  constructor(private configService: ConfigService) {
    const openaiApiKey = this.configService.get<string>('OPENAI_API_KEY');
    if (!openaiApiKey) {
      throw new Error('OPENAI_API_KEY is required but not provided');
    }
    setDefaultOpenAIKey(openaiApiKey);
    this.catSafeAgent = CatSafeAgent;
  }

  async checkThePlantIsHarmful(
    base64Image: string,
    longitude: number,
    latitude: number,
  ) {
    const imageUrl = `data:image/jpeg;base64,${base64Image}`;
    const agentInputItem: AgentInputItem = {
      role: 'user',
      content: [
        {
          type: 'input_text',
          text: `The plant is located at ${longitude}, ${latitude}`,
        },
        {
          type: 'input_image',
          image: imageUrl,
        },
      ],
    };
    const result = await run(this.catSafeAgent, [agentInputItem]);
    return result.finalOutput;
  }
}

🎮 Update Cat-Safe Service with Handoff Implementation

Update cat-safe.service.ts:

import { Injectable } from '@nestjs/common';
import { AgentService } from '../agent/agent.service';

@Injectable()
export class CatSafeService {
  constructor(private readonly agentService: AgentService) {}

  async checkThePlantIsHarmful(
    image: Express.Multer.File,
    longitude: number,
    latitude: number,
  ) {
    const base64Image = image.buffer.toString('base64');
    const result = await this.agentService.checkThePlantIsHarmful(
      base64Image,
      longitude,
      latitude,
    );
    return result;
  }
}

📍 Adding Location Information to the Controller

The controller needs to be updated to accept and process location coordinates for emergency veterinary services. This is crucial for the agent to provide nearby emergency vet information when poisonous plants are detected.

🔧 Why Location Information is Needed:

Emergency Response: If a plant is poisonous, users need immediate access to nearby veterinary clinics
Geographic Accuracy: Vet recommendations should be relevant to the user's location
Time-Critical Situations: In poisoning emergencies, distance to veterinary care matters
Complete Analysis: Location enables comprehensive emergency response planning

🎯 Controller Updates Required:

The cat-safe.controller.ts needs modification to:

Accept Location Parameters: Add longitude and latitude fields to the request body
Update Swagger Documentation: Include location fields in API documentation
Pass Location Data: Forward coordinates to the service layer

📝 Updated Controller Implementation:

Update cat-safe.controller.ts:

import {
  Body,
  Controller,
  HttpStatus,
  ParseFilePipeBuilder,
  Post,
  UploadedFile,
  UseInterceptors,
} from '@nestjs/common';
import { CatSafeService } from './cat-safe.service';
import { FileInterceptor } from '@nestjs/platform-express';
import { ApiBody, ApiConsumes } from '@nestjs/swagger';

@Controller('cat-safe')
export class CatSafeController {
  constructor(private readonly catSafeService: CatSafeService) {}

  @Post('check-harmful')
  @UseInterceptors(FileInterceptor('image'))
  @ApiConsumes('multipart/form-data')
  @ApiBody({
    schema: {
      type: 'object',
      properties: {
        image: {
          type: 'string',
          format: 'binary',
          description: 'the image of the user',
        },
        longitude: {
          type: 'number',
          description: 'the longitude of the plant',
          example: -73.968285,
        },
        latitude: {
          type: 'number',
          description: 'the latitude of the plant',
          example: 40.785091,
        },
      },
    },
  })
  async checkHarmful(
    @UploadedFile(
      new ParseFilePipeBuilder()
        .addFileTypeValidator({
          fileType: /(jpg|jpeg|png)$/,
        })
        .addMaxSizeValidator({
          maxSize: 5 * 1024 * 1024, // 5MB
        })
        .build({
          errorHttpStatusCode: HttpStatus.UNPROCESSABLE_ENTITY,
        }),
    )
    image: Express.Multer.File,
    @Body()
    body: { longitude: number; latitude: number },
  ) {
    return this.catSafeService.checkThePlantIsHarmful(
      image,
      body.longitude,
      body.latitude,
    );
  }
}

🚀 How Handoffs Improve the Experience

Before (Multiple Requests):

// Client needs to make multiple calls
const harmfulResult = await fetch('/cat-safe/harmful', { ... });
const poisonousResult = await fetch('/cat-safe/poisonous', { ... });
// Client must combine results manually

After (Single Request with Handoffs):

// Single call, agents coordinate internally
const completeResult = await fetch('/cat-safe/check-harmful', { ... });
// Get comprehensive analysis in one response

🧠 Handoff Decision Logic

The cat-safe agent uses intelligent decision-making:

1. 📸 Receives plant image + location coordinates
2. 🔍 Performs initial analysis to determine if plant is poisonous
3. 🤔 Makes routing decision:
   - If plant IS POISONOUS → Routes to PoisonousAgent for detailed toxicity analysis
   - If plant IS NOT POISONOUS → Routes to HarmfulAgent for general safety confirmation
4. 🔄 Internally applies appropriate agent logic based on decision
5. 📊 Gets result: Either comprehensive toxicity info OR general safety assessment
6. 📋 Returns complete safety analysis in one response

🎯 Benefits of Agent Handoffs

✅ Single API Call: One request handles everything
✅ Intelligent Routing: Right analysis level for each plant type
✅ Better Performance: No multiple round trips

This handoff pattern transforms your API from requiring multiple requests into a single, intelligent conversation that automatically provides the right level of plant safety analysis based on toxicity and user location!

🧪 Let's test the new implementation:
When I send a request to the cat-safe/check-harmful endpoint with Lily plant image 🌸, I get the following response:

{
  "isPoisonous": true,
  "plantName": "Lily",
  "plantScientificName": "Lilium species",
  "plantDescription": "Lilies are flowering plants known for their large, prominent flowers and are commonly found in gardens and as ornamental plants. They belong to the Liliaceae family and are characterized by their trumpet-shaped flowers, long leaves, and bulbous roots.",
  "toxicityLevel": "Highly toxic",
  "toxicPlantParts": ["leaves", "stems", "flowers", "pollen", "roots", "seeds"],
  "safePlantParts": [],
  "toxicCompounds": [
    {
      "name": "Unknown toxin",
      "chemicalClassification": "Unknown",
      "mechanismOfAction": "The specific toxin affecting cats has not been identified, but it leads to severe kidney damage upon ingestion.",
      "targetOrgans": ["kidneys"],
      "toxicityDescription": "Even small amounts ingested can cause acute kidney failure in cats."
    }
  ],
  "clinicalSigns": {
    "earlySymptoms": ["vomiting", "loss of appetite", "lethargy", "drooling"],
    "progressiveSymptoms": [
      "increased thirst",
      "increased urination",
      "dehydration",
      "tremors",
      "seizures",
      "kidney failure",
      "death"
    ],
    "onsetTimeline": "Symptoms can appear within 1-3 hours of ingestion, with severe kidney damage developing within 12-24 hours."
  },
  "emergencyResponse": {
    "firstAidSteps": [
      "Immediately remove the cat from the area with the lily plant.",
      "If possible, bring a sample or photo of the plant to the veterinarian.",
      "Do not induce vomiting unless instructed by a veterinarian."
    ],
    "whenToSeekHelp": "Seek veterinary care immediately if you suspect your cat has ingested any part of a lily plant.",
    "vetInformation": [
      "Contact your veterinarian or an emergency animal hospital as soon as possible."
    ]
  },
  "nearestVets": [
    {
      "name": "Animal Medical Center",
      "address": "510 E 62nd St, New York, NY 10065",
      "phoneNumber": "(212) 838-7053",
      "distance": "0.5 miles",
      "isEmergency24h": true
    },
    {
      "name": "BluePearl Pet Hospital",
      "address": "410 E 55th St, New York, NY 10022",
      "phoneNumber": "(212) 988-1500",
      "distance": "0.7 miles",
      "isEmergency24h": true
    },
    {
      "name": "Animal Care Centers of NYC (ACC) - Manhattan",
      "address": "326 E 110th St, New York, NY 10029",
      "phoneNumber": "(212) 788-4000",
      "distance": "1.5 miles",
      "isEmergency24h": true
    },
    {
      "name": "VCA West Chelsea Animal Hospital",
      "address": "408 W 15th St, New York, NY 10011",
      "phoneNumber": "(212) 924-9119",
      "distance": "1.7 miles",
      "isEmergency24h": true
    },
    {
      "name": "East River Veterinary Hospital",
      "address": "510 E 62nd St, New York, NY 10065",
      "phoneNumber": "(212) 838-7053",
      "distance": "0.5 miles",
      "isEmergency24h": true
    },
    {
      "name": "Animal Hospital of the Rockaways",
      "address": "114-04 Beach Channel Dr, Rockaway Park, NY 11694",
      "phoneNumber": "(718) 474-0500",
      "distance": "11.5 miles",
      "isEmergency24h": true
    },
    {
      "name": "VCA Queens Animal Hospital",
      "address": "92-15 Queens Blvd, Rego Park, NY 11374",
      "phoneNumber": "(718) 459-4444",
      "distance": "8.5 miles",
      "isEmergency24h": true
    },
    {
      "name": "Animal Hospital of the Village",
      "address": "64 W 12th St, New York, NY 10011",
      "phoneNumber": "(212) 924-9119",
      "distance": "1.5 miles",
      "isEmergency24h": true
    },
    {
      "name": "VCA Brooklyn Veterinary Referral & Emergency Center",
      "address": "2205 44th Ave, Long Island City, NY 11101",
      "phoneNumber": "(718) 786-8000",
      "distance": "3.5 miles",
      "isEmergency24h": true
    },
    {
      "name": "Animal Hospital of the Bronx",
      "address": "1000 E 138th St, Bronx, NY 10454",
      "phoneNumber": "(718) 665-0100",
      "distance": "4.5 miles",
      "isEmergency24h": true
    }
  ],
  "locationCoordinates": {
    "latitude": 40.785091,
    "longitude": -73.968285
  },
  "toxicityThreshold": "Even small amounts ingested can cause severe kidney damage in cats.",
  "agentType": "POISONOUS"
}

So lets try it with a spider plant 🕷️. The output is:

{
  "isHarmful": false,
  "plantName": "Spider Plant",
  "plantScientificName": "Chlorophytum comosum",
  "chemicalComposition": "",
  "chemicalList": [
    {
      "name": "Saponins",
      "description": "Naturally occurring compounds found in spider plants that can cause mild gastrointestinal upset if ingested by pets.",
      "toxicity": true,
      "toxicityLevel": 1,
      "toxicityDescription": "Mild gastrointestinal upset in pets.",
      "toxicitySymptoms": ["Vomiting", "Diarrhea"],
      "toxicityTreatment": "Generally self-limiting; consult a veterinarian if symptoms persist."
    }
  ],
  "toxicityThreshold": "Ingestion of a small amount may cause mild symptoms; larger amounts can lead to more pronounced effects.",
  "agentType": "HARMFUL"
}

🎯 Results: Single Request, Multiple Output Types

The beauty of our handoff implementation is that one API endpoint can return different output schemas based on the plant analysis. Here's what we achieved:

📡 Single API Endpoint:

POST /cat-safe/check-harmful
Content-Type: multipart/form-data
Body: { image: file, longitude: number, latitude: number }

🚀 Key Achievements:

✅ Smart Routing: Cat-safe agent correctly identifies plant toxicity and routes to appropriate specialist

✅ Dynamic Schemas: Same endpoint returns different output structures based on plant type

✅ Type Safety: Each response follows its respective schema (PoisonousSchema vs HarmfulSchema)

✅ Agent Identification: agentType field clearly shows which specialist handled the request

✅ Single Request: No need for multiple API calls - everything handled in one request

✅ Location Integration: Emergency vet information included when needed (poisonous plants)

🎭 The Magic of Agent Handoffs:

🌸 Lily Image → Cat-Safe Agent → "This is poisonous!" → Hands off to PoisonousAgent → Detailed toxicity analysis
🕷️ Spider Plant → Cat-Safe Agent → "This is not poisonous!" → Hands off to HarmfulAgent → General safety assessment

This demonstrates the power of intelligent agent coordination - one endpoint, multiple specialists, appropriate responses for each scenario! 🎯

📋 Summary

This project showcases a comprehensive AI-powered plant safety system built with NestJS and OpenAI Agents that helps cat owners determine if plants are safe for their feline companions.

🎯 What We Built:

🤖 Intelligent Agent System: Multi-agent coordination with handoffs between specialized plant analysis agents
📱 Single API Endpoint: One request handles all plant safety analysis scenarios
🏥 Emergency Response: Location-based veterinary clinic finder for poisoning emergencies
📊 Structured Responses: Type-safe, validated responses using Zod schemas
🌿 Dual Analysis Types: Specialized agents for harmful vs poisonous plant analysis

🚀 Key Technical Achievements:

Agent Handoffs: Implemented intelligent routing between HarmfulAgent and PoisonousAgent
Dynamic Schema Responses: Same endpoint returns different output structures based on plant toxicity
Real-time Web Search: Agents use live internet data for accurate plant identification
Location Integration: Emergency veterinary services based on GPS coordinates
Type Safety: Full TypeScript support with Zod schema validation
File Upload: Seamless image processing with Multer integration

🔗 Connect & Collaborate

👨‍💻 Find Me Online:

🐦 X (Twitter): @berat_dinckan
💼 LinkedIn: Cengiz Berat Dinckan
🧑‍💻 GitHub: pandashavenobugs

📂 Code Repository:

This complete implementation is available on GitHub:

🌟 nest-ai-agent-example

🤝 Get Involved:

⭐ Star the repo if this helped you build AI agents
🍴 Fork it to build your own agent systems
🐛 Report issues or suggest improvements
💡 Contribute new agent features or optimizations

📧 Contact:

Have questions about AI agents or want to collaborate? Reach out through any of the platforms above!

Built with ❤️ using NestJS and OpenAI Agents

Made for keeping our feline friends safe! 🐱🌿

ESP32 to AWS: Complete IoT Solution with IoT Core, DynamoDB, and Lambda Functions in Golang

BERAT DİNÇKAN — Mon, 29 Jan 2024 22:54:48 +0000

Introduction

In this blog post, We're going to connect an ESP32 microcontroller to AWS IoT Core. We'll use DynamoDB for data storage and trigger AWS Lambda functions, all programmed in Golang. This guide is perfect for anyone interested in building IoT projects using ESP32 and AWS. We'll walk through each step, making it easy to follow along, even if you're new to these technologies. Let’s get started!

Technologies and Techniques Used

In this project, We utilize a range of technologies and techniques to achieve seamless integration between the ESP32 microcontroller and AWS services. Here's a breakdown of what We use:

Hardware Side:

PlatformIO: An open-source ecosystem for IoT development.
Arduino: The programming platform for our ESP32 microcontroller.
C++ (CPP): The programming language used for ESP32 development.
knolleary/pubsubclient: A client library for MQTT messaging.
LittleFS: A file system for handling files on the ESP32.

Cloud and Backend Side:

AWS IoT Core: Manages and connects IoT devices.
DynamoDB: AWS's NoSQL database service for handling device data.
Lambda Function: AWS service for running backend code in response to events.
SAM CLI: AWS CLI tool for managing aws resources.
Golang: The programming language used for writing AWS Lambda functions.

Creating a PlatformIO Project: Your First Step

First, let's create a folder for our project. We'll use PlatformIO to create a new project inside that folder. We'll call our project esp32_to_aws_hardware_example. Then We'll open the project in VSCode.



mkdir esp32_to_aws_hardware_example
cd esp32_to_aws_hardware_example
code .

Press F1 to open the command palette and type PlatformIO: New Terminal. This will open a new terminal in VSCode. We'll use this terminal to run all the commands in this guide.

So We can create a new project using the following commands:

First We need to delete all cache files. Then We can create a new project. This command is optional you can skip it.



pio system prune -f

Project initialization:



platformio init

After initialization, We can see the following files and folders in our project:

Adding Libraries and Board Informations to PlatformIO

In platformio.ini file We can add libraries and board informations. We need to add the following libraries to our project:



[env:esp32doit-devkit-v1]
platform = espressif32
board = esp32doit-devkit-v1
framework = arduino
monitor_speed=115200
upload_port=/dev/cu.usbserial-0001
lib_extra_dirs = lib
board_build.filesystem = littlefs
lib_deps =
    bblanchon/ArduinoJson@^6.18.5
    knolleary/pubsubclient@^2.8

At this point, I want to talk about the lib_extra_dirs and board_build.filesystem parameters. We need to add the lib_extra_dirs parameter to the platformio.ini file to use the libraries in the lib folder. We need to add the board_build.filesystem parameter to the platformio.ini file to use the LittleFS file system.

Installing Packages

To install the libraries, We need to run the following command:



pio pkg install

Exit the project and restart the VSCode. Then We can see the platformIO tasks in our project:

Initialization is complete. Now We can start writing our code.

note:

When I tried to use interface of the platformIO to create a new project, It didn't work properly. So I used the command line to create the project. I didn't understand why it didn't work. If you know the reason, please let me know in the comments.

Setting Up Your First 'Thing' in AWS IoT Core with SSL Certificates

In this part, We're going to set up a 'thing' in AWS IoT Core. This 'thing' represents our ESP32 microcontroller in the AWS cloud. We'll also download the SSL certificates required for secure communication. Here's how We do it:

1. Log in to AWS IoT Core:

First, log in to your AWS Management Console and navigate to IoT Core.

2. Create a Policy:

In the AWS IoT Core console, navigate to 'Secure', and then 'Policies'.
Click on ‘Create a policy’.
Name your policy, for example, iot_policy.
Set the policy document with the following JSON structure:



{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["iot:*"],
      "Resource": "*"
    }
  ]
}

This policy allows your 'thing' to perform all actions (iot:*) on all resources (*).

3. Create a 'Thing':

Go to the 'Manage' section and click on 'Things'.
Choose 'Create' to start creating a new 'thing'.
Select 'Create a single thing'.
Click on 'next'.
Follow the guided steps to name your 'thing' and complete its creation. For this guide, We'll name our 'thing' esp32_test1.
Click on 'next'.
Select 'Auto-generate a new certificate (recommended)'.
Select the policy you created in the previous step (iot_policy).
Click on 'Create thing'.

4. Download SSL Certificates:

Once your thing is created, AWS IoT Core will prompt you to create certificates. Click on 'Create certificates'.
Download the following three essential files for your thing:
- A certificate file (blablabla-certificate.pem.crt).
- A private key file (blablabla-private.pem.key).
- The Amazon Root CA 1 (AmazonRootCA1.pem) file.
Make sure to save these files securely, as they are vital for your ESP32 to communicate securely with AWS IoT Core.

The file names are too long and you can change the names. So do I. I changed the names as follows:

blalblablabla-certificate.pem.crt to certificate.pem.crt
blalblablabla-private.pem.key to private.pem.key
AmazonRootCA1.pem to aws_cert_ca.pem

After changing the name of the files We have 3 files to use in our project:

certificate.pem.crt
private.pem.key
aws_cert_ca.pem

Configuring ESP32 Filesystem for AWS IoT Core Connection

In this part, We're going to configure the ESP32 filesystem to connect to AWS IoT Core. We'll use the LittleFS file system to store the SSL certificates and the AWS IoT Core endpoint. Here's how We do it:

1. Create a 'data' Folder:

In the root directory of your project, create a folder called data. This folder will contain all the files We need to store in the ESP32 filesystem.

Note: The data folder must be in the root directory of your project. Otherwise, the ESP32 will not be able to find the files.

The mqtt_config.json file will contain the AWS IoT Core endpoint, port, clientId, and topic to publish data.



{
  "port": 8883,
  "host": "YOUR_AWS_IOT_CORE_ENDPOINT",
  "clientId": "esp_test1",
  "publishTopic": "esp32/sensor/test"
}

To get host information We need to go to the AWS IoT Core console and click on the Settingstab. Then We can see the endpoint information.

While connecting to AWS IoT Core, port 8883 is used for secure connection.

clientId is the name of the thing We created in AWS IoT Core.

publishTopic is the topic name that We will publish data to.

The wifi_config.json file will contain ssid and password.



{
  "ssid": "your_wifi_name",
  "password": "your_wifi_password"
}

Writing the ESP32 Code for AWS IoT Core Connectivity

In this part, We're going to write the ESP32 code for connecting to AWS IoT Core. We'll use the PubSubClient library to connect to AWS IoT Core using MQTT.

First We need to create the models for the wifi, mqtt and certificate configurations. We'll create the following models in the model folder in the lib folder.

WifiCredentialModel.h



// lib/model/WifiCredentialModel.h
// WifiCredentialModel class definition.

#ifndef WIFICREDENTIALMODEL_H
#define WIFICREDENTIALMODEL_H

#include <Arduino.h>

class WifiCredentialModel
{
public:
    String ssid;
    String password;
    WifiCredentialModel() : ssid(""), password(""){};
    WifiCredentialModel(String ssid, String password) : ssid(ssid), password(password){};

    bool isEmpty()
    {
        return ssid == "" || password == "";
    }
};

#endif

MqttCredentialModel.h



// lib/model/MqttCredentialModel.h
// MqttCredentialModel class definition.

#ifndef MQTTCREDENTIALMODEL_H
#define MQTTCREDENTIALMODEL_H

#include <Arduino.h>

class MqttCredentialModel
{
public:
    int port;
    String host;
    String clientId;
    String publishTopic;

    MqttCredentialModel() : port(0), host(""), clientId(""), publishTopic(""){};
    MqttCredentialModel(int port, String host, String clientId, String publishTopic) : port(port), host(host), clientId(clientId), publishTopic(publishTopic){};

    bool isEmpty()
    {
        return port == 0 || host == "" || clientId == "" || publishTopic == "";
    }
};

#endif

CertificateCredentialModel.h



// lib/model/CertificateCredentialModel.h
// CertificateCredentialModel definition.

#ifndef CERTIFICATECREDENTIALMODEL_H
#define CERTIFICATECREDENTIALMODEL_H

#include <Arduino.h>

class CertificateCredentialModel
{
public:
    String ca;
    String certificate;
    String privateKey;

    CertificateCredentialModel() : ca(""), certificate(""), privateKey(""){};
    CertificateCredentialModel(String ca, String certificate, String privateKey) : ca(ca), certificate(certificate), privateKey(privateKey){};

    bool isEmpty()
    {
        return ca == "" || certificate == "" || privateKey == "";
    }
};

#endif

Now We can create a service that will read the wifi, mqtt and certificate configurations from the files and return the models. We'll create the following service in the service folder in the lib folder.

ConfigService.h



// lib/service/ConfigService.h
// ConfigService class definition.

#ifndef CONFIGSERVICE_H
#define CONFIGSERVICE_H

#include <Arduino.h>
#include "../model/CertificateCredentialModel.h"
#include "../model/MqttCredentialModel.h"
#include "../model/WifiCredentialModel.h"
#include <FS.h>

class ConfigService
{
private:
    fs::FS &fileSystem;

public:
    ConfigService(fs::FS &fileSystem) : fileSystem(fileSystem){};

    WifiCredentialModel getWifiCredential();
    MqttCredentialModel getMqttCredential();
    CertificateCredentialModel getCertificateCredential();
};

#endif

ConfigService.cpp



// lib/service/ConfigService.cpp
// ConfigService class implementation
#include "ConfigService.h"
#include <ArduinoJson.h>
#include <FS.h>

String readFile(fs::FS &fs, const char *path)
{
    Serial.printf("Reading file: %s\r\n", path);

    File file = fs.open(path, FILE_READ);
    if (!file)
    {
        Serial.println("Failed to open file for reading");
        return "";
    }
    return file.readString();
}

CertificateCredentialModel ConfigService::getCertificateCredential()
{
    String ca = readFile(fileSystem, "/certs/aws_cert_ca.pem");
    String certificate = readFile(fileSystem, "/certs/certificate.pem.crt");
    String privateKey = readFile(fileSystem, "/certs/private.pem.key");
    return CertificateCredentialModel(ca, certificate, privateKey);
}

WifiCredentialModel ConfigService::getWifiCredential()
{
    File wifiConfigFile = fileSystem.open("/wifi_config.json", "r");
    if (!wifiConfigFile)
    {
        Serial.println("Failed to open wifi_config.json file");
        return WifiCredentialModel();
    }
    DynamicJsonDocument doc(1024);
    DeserializationError error = deserializeJson(doc, wifiConfigFile);
    if (error)
    {
        Serial.println("Failed to read file, using default configuration");
        wifiConfigFile.close();
        return WifiCredentialModel();
    }
    String ssid = doc["ssid"];
    String password = doc["password"];
    wifiConfigFile.close();
    return WifiCredentialModel(ssid, password);
}

MqttCredentialModel ConfigService::getMqttCredential()
{
    File mqttConfigFile = fileSystem.open("/mqtt_config.json", "r");
    if (!mqttConfigFile)
    {
        Serial.println("Failed to open mqtt_config.json file");
        return MqttCredentialModel();
    }
    DynamicJsonDocument doc(1024);
    DeserializationError error = deserializeJson(doc, mqttConfigFile);
    if (error)
    {
        Serial.println("Failed to read file, using default configuration");
        mqttConfigFile.close();
        return MqttCredentialModel();
    }
    String host = doc["host"];
    int port = doc["port"];
    String clientId = doc["clientId"];
    String publishTopic = doc["publishTopic"];
    mqttConfigFile.close();
    return MqttCredentialModel(port, host, clientId, publishTopic);
}

We read the files in the data folder. Then We deserialize the JSON data using the ArduinoJson library. We return the models We created using the data We read.

Bringing It All Together in main.cpp: Finalizing the ESP32-AWS Connection

In this part, We're going to write the main.cpp file. We'll use the PubSubClient library to connect to AWS IoT Core using MQTT. We'll use the ConfigService We created to read the wifi, mqtt and certificate configurations from the files. We'll use the models We created to store the configurations.

main.cpp



#define LED_PIN 2
#include <Arduino.h>
#include <LittleFS.h>
#include <PubSubClient.h>
#include <WiFiClientSecure.h>
#include "../service/ConfigService.h"

WiFiClientSecure espClient;
PubSubClient client(espClient);
MqttCredentialModel mqttCredential;
WifiCredentialModel wifiCredential;
CertificateCredentialModel certificateCredential;

void setup()
{
    Serial.begin(115200);
    Serial.println("Starting...");
    if (!LittleFS.begin())
    {
        Serial.println("An Error has occurred while mounting LittleFS");
        return;
    }

    ConfigService configService(LittleFS);
    wifiCredential = configService.getWifiCredential();
    if (wifiCredential.isEmpty())
    {
        Serial.println("Wifi credential is empty");
        return;
    }

    mqttCredential = configService.getMqttCredential();
    if (mqttCredential.isEmpty())
    {
        Serial.println("Mqtt credential is empty");
        return;
    }

    certificateCredential = configService.getCertificateCredential();
    if (certificateCredential.isEmpty())
    {
        Serial.println("Certificate credential is empty");
        return;
    }

    WiFi.begin(wifiCredential.ssid.c_str(), wifiCredential.password.c_str());
    while (WiFi.status() != WL_CONNECTED)
    {
        digitalWrite(LED_PIN, HIGH);
        delay(50);
        digitalWrite(LED_PIN, LOW);
        delay(50);
        digitalWrite(LED_PIN, HIGH);
        delay(50);
        digitalWrite(LED_PIN, HIGH);
        delay(1000);
        Serial.print(".");
    }
    Serial.println("WiFi connected");

    // Set the certificates to the client
    espClient.setCACert(certificateCredential.ca.c_str());
    espClient.setCertificate(certificateCredential.certificate.c_str());
    espClient.setPrivateKey(certificateCredential.privateKey.c_str());

    client.setServer(mqttCredential.host.c_str(), mqttCredential.port);

    while (!client.connected())
    {
        Serial.println("Connecting to AWS IoT...");

        if (client.connect(mqttCredential.clientId.c_str()))
        {
            Serial.println("Connected to AWS IoT");
        }
        else
        {
            Serial.print("failed, rc=");
            Serial.print(client.state());
            Serial.println(" try again in 5 seconds");
            delay(5000);
        }
    }
}
void loop() {}

Testing the ESP32-AWS Connection

To test esp32-aws connection, We need to build and upload the code and folders to the esp32. We can do this using the platformIO tasks. We need to run the following tasks

Build Filesystem Image
Upload Filesystem Image
Build Code
Upload Code
Monitor

To build Filesystem Image We need to run Build Filesystem Image task:

After the Clicking the Build Filesystem Image task, We can see the following output in the terminal:

Now You can upload the filesystem image to the esp32. To do this, We need to run the Upload Filesystem Image task:

The output of the Upload Filesystem Image task is as follows:

Now We can build the code. To do this, We need to run the Build task:

ESP32 is ready to upload the code. To do this, We need to run the Upload Code task but I go with the Upload and Monitor task to see the output after uploading the code.

After clicking the Upload and Monitor task, We can see the following output in the terminal:

As you can see, the esp32 is connected to AWS IoT Core. Now We can send data to AWS IoT Core.

Sending Data to AWS IoT Core

In this part, We're going to send data to AWS IoT Core. I have a dht11 sensor and I want to send the temperature and humidity data to AWS IoT Core. I'll use the following libraries to read the data from the sensor:

Adafruit Unified Sensor
DHT sensor library

We need to add the following libraries to our project:



lib_deps =
    bblanchon/ArduinoJson@^6.18.5
    knolleary/pubsubclient@^2.8
    adafruit/DHT sensor library@^1.4.6
    adafruit/Adafruit Unified Sensor@^1.1.14

After adding these libraries, platformIO will download the libraries and add them to the lib folder. If not, you need to open platformIO terminal and run the following command:



pio pkg install

First, We need to create a model for the sensor data. We'll create the payload model in the model folder in the lib folder.

PayloadModel.h



// lib/model/PayloadModel.h
// PayloadModel class definition.

#ifndef PAYLOADMODEL_H
#define PAYLOADMODEL_H

#include <Arduino.h>
#include <ArduinoJson.h>

class PayloadModel
{
private:
    String clientId;
    bool isClientIdValid;
    float humidity;
    bool isHumidityValid;
    float temperature;
    bool isTemperatureValid;

public:
    PayloadModel()
    {
        clientId = "";
        isClientIdValid = false;
        humidity = 0;
        isHumidityValid = false;
        temperature = 0;
        isTemperatureValid = false;
    };
    void setClientId(String clientId, bool isClientIdValid)
    {
        this->clientId = clientId;
        this->isClientIdValid = isClientIdValid;
    };
    void setHumidity(float humidity, bool isHumidityValid)
    {
        this->humidity = humidity;
        this->isHumidityValid = isHumidityValid;
    };
    void setTemperature(float temperature, bool isTemperatureValid)
    {
        this->temperature = temperature;
        this->isTemperatureValid = isTemperatureValid;
    };

    char *toJson()
    {
        static char buffer[512];
        DynamicJsonDocument doc(256);
        if (this->isClientIdValid)
        {
            doc["clientId"] = this->clientId;
        }
        else
        {
            doc["clientId"] = nullptr;
        }
        if (this->isHumidityValid)
        {
            doc["humidity"] = this->humidity;
        }
        else
        {
            doc["humidity"] = nullptr;
        }
        if (this->isTemperatureValid)
        {
            doc["temperature"] = this->temperature;
        }
        else
        {
            doc["temperature"] = nullptr;
        }

        serializeJson(doc, buffer);
        return buffer;
    };
};
#endif

For the payloadModel We need to create a json string. We'll use the ArduinoJson library to create the json string.

Now We can implement the payloadModel and dht11 sensor in the main.cpp file.

main.cpp



#define LED_PIN 2
#define DHT_PIN 15
#define DHT_TYPE DHT11

unsigned long previousSensorMillis = 0;
const long sensorInterval = 10000;

#include <Arduino.h>
#include <LittleFS.h>
#include <PubSubClient.h>
#include <WiFiClientSecure.h>
#include "../service/ConfigService.h"
#include "../model/PayloadModel.h"
#include <DHT.h>
WiFiClientSecure espClient;
PubSubClient client(espClient);
MqttCredentialModel mqttCredential;
WifiCredentialModel wifiCredential;
CertificateCredentialModel certificateCredential;
PayloadModel payloadModel;
char *payload;
DHT dht(DHT_PIN, DHT_TYPE);

void setup()
{
    Serial.begin(115200);
    Serial.println("Starting...");
    if (!LittleFS.begin())
    {
        Serial.println("An Error has occurred while mounting LittleFS");
        return;
    }

    ConfigService configService(LittleFS);
    wifiCredential = configService.getWifiCredential();
    if (wifiCredential.isEmpty())
    {
        Serial.println("Wifi credential is empty");
        return;
    }

    mqttCredential = configService.getMqttCredential();
    if (mqttCredential.isEmpty())
    {
        Serial.println("Mqtt credential is empty");
        return;
    }

    certificateCredential = configService.getCertificateCredential();
    if (certificateCredential.isEmpty())
    {
        Serial.println("Certificate credential is empty");
        return;
    }

    WiFi.begin(wifiCredential.ssid.c_str(), wifiCredential.password.c_str());
    while (WiFi.status() != WL_CONNECTED)
    {
        digitalWrite(LED_PIN, HIGH);
        delay(50);
        digitalWrite(LED_PIN, LOW);
        delay(50);
        digitalWrite(LED_PIN, HIGH);
        delay(50);
        digitalWrite(LED_PIN, HIGH);
        delay(1000);
        Serial.print(".");
    }
    Serial.println("WiFi connected");

    // Set the certificates to the client
    espClient.setCACert(certificateCredential.ca.c_str());
    espClient.setCertificate(certificateCredential.certificate.c_str());
    espClient.setPrivateKey(certificateCredential.privateKey.c_str());

    client.setServer(mqttCredential.host.c_str(), mqttCredential.port);

    while (!client.connected())
    {
        Serial.println("Connecting to AWS IoT...");

        if (client.connect(mqttCredential.clientId.c_str()))
        {
            Serial.println("Connected to AWS IoT");
        }
        else
        {
            Serial.print("failed, rc=");
            Serial.print(client.state());
            Serial.println(" try again in 5 seconds");
            delay(5000);
        }
    }
    // set the payload model
    payloadModel = PayloadModel();
    payloadModel.setClientId(mqttCredential.clientId, true);
}
void loop()
{
    unsigned long currentMillis = millis();
    if (currentMillis - previousSensorMillis >= sensorInterval)
    {
        digitalWrite(LED_PIN, HIGH);
        previousSensorMillis = currentMillis;
        float humidity = dht.readHumidity();
        float temperature = dht.readTemperature();
        payloadModel.setHumidity(humidity, !isnan(humidity));
        payloadModel.setTemperature(temperature, !isnan(temperature));
        payload = payloadModel.toJson();
        Serial.println("Publish message: ");
        Serial.println(payload);
        client.publish(mqttCredential.publishTopic.c_str(), payload);
    }
    digitalWrite(LED_PIN, LOW);
}

This code reads the temperature and humidity data from the sensor and sends it to AWS IoT Core every 10 seconds as json data.

To test the code run the Upload and Monitor task. After uploading the code, We can see the following output in the terminal:

As We can see, the esp32 is sending data to AWS IoT Core.

We can check if the esp32 sends data to AWS IoT Core. To do this, We need to go to the AWS IoT Core console and click on the Test tab. Then We can subscribe to the topic We created in the mqtt_config.json file.

After subscribing to the topic, We can see the data sent by the esp32.

The data we can see is as follows:



{
  "clientId": "esp_test1",
  "humidity": 55,
  "temperature": 27.60000038
}

Everything is working properly. So What is next?

Building a SAM CloudFormation Template: Integrating DynamoDB, IoT Core Rule, and Lambda

In this part, We're going to build a SAM CloudFormation template. We'll use the template to create a DynamoDB table, an IoT Core rule, and a Lambda function. Here's how We do it:

1. Create a Role For The Lambda Function:



LambdaFunctionRole:
  Type: AWS::IAM::Role
  Properties:
    AssumeRolePolicyDocument:
      Version: "2012-10-17"
      Statement:
        - Effect: Allow
          Principal:
            Service:
              - lambda.amazonaws.com
          Action:
            - sts:AssumeRole
    Policies:
      - PolicyName: "lambda-function-policy"
        PolicyDocument:
          Version: "2012-10-17"
          Statement:
            - Effect: Allow
              Action:
                - logs:CreateLogGroup
                - logs:CreateLogStream
                - logs:PutLogEvents
              Resource: "*"
      - PolicyName: DynamoDBCRUDPolicy
        PolicyDocument:
          Version: "2012-10-17"
          Statement:
            - Effect: Allow
              Action:
                - dynamodb:PutItem
              Resource: "*"

We added 2 policies to the role. The first policy is for logging. The second policy is for DynamoDB. The lambda function will write data to DynamoDB. So We need to add the DynamoDB policy to the role. For now dynamodb:PutItem is enough for us but We can add other dynamodb actions to the policy.

2. Create a Lambda Function:



MQTTSubscribeHandler:
  Type: AWS::Serverless::Function
  Properties:
    Role: !GetAtt LambdaFunctionRole.Arn
    FunctionName: "esp32_to_aws_mqtt-subscribe-handler"
    CodeUri: ./functions/mqtt-subscribe-handler
    Handler: app.lambda_handler
    Runtime: go1.x
    Architectures:
      - x86_64

This function will be triggered by the IoT Core rule. The function will write the data to DynamoDB.

3. Create a DynamoDB Table:



ThingData:
  Type: AWS::DynamoDB::Table
  Properties:
    TableName: "ThingDataTable"
    AttributeDefinitions:
      - AttributeName: clientId
        AttributeType: S
      - AttributeName: createdAt
        AttributeType: S
    KeySchema:
      - AttributeName: clientId
        KeyType: HASH
      - AttributeName: createdAt
        KeyType: RANGE
    BillingMode: PAY_PER_REQUEST

We'll use this table to store the data sent by the esp32.

clientId is the partition key and createdAt is the sort key.

clientId + createdAt is the composite primary key. We can use this key to save the data that have same clientId but different createdAt.

4. Create an IoT Core Rule:



IoTTopicRule:
  Type: AWS::IoT::TopicRule
  Properties:
    TopicRulePayload:
      RuleDisabled: false
      Sql: "SELECT * FROM 'esp32/sensor/test'"
      Actions:
        - Lambda:
            FunctionArn: !GetAtt MQTTSubscribeHandler.Arn

This rule will trigger the lambda function when the data is sent to the esp32/sensor/test topic.

5. Create a Permission to Invoke The Lambda Function:



MQTTSubscribeHandlerPermission:
  Type: AWS::Lambda::Permission
  Properties:
    Action: lambda:InvokeFunction
    FunctionName: !GetAtt MQTTSubscribeHandler.Arn
    Principal: iot.amazonaws.com
    SourceArn: !GetAtt IoTTopicRule.Arn

This permission will allow the IoT Core rule to invoke the lambda function.

6. Finalize The SAM Template:



AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: >
  esp32-to-aws-cloud-example

  Sample SAM Template for esp32-to-aws-cloud-example

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 5
    MemorySize: 128
    Environment:
      Variables:
        ThingDataTable: "ThingDataTable"

Resources:
  LambdaFunctionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      Policies:
        - PolicyName: "lambda-function-policy"
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - logs:CreateLogGroup
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                Resource: "*"
        - PolicyName: DynamoDBCRUDPolicy
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - dynamodb:PutItem
                Resource: "*"

  MQTTSubscribeHandler:
    Type: AWS::Serverless::Function
    Properties:
      Role: !GetAtt LambdaFunctionRole.Arn
      FunctionName: "esp32_to_aws_mqtt-subscribe-handler"
      CodeUri: ./functions/mqtt-subscribe-handler
      Handler: app.lambda_handler
      Runtime: go1.x
      Architectures:
        - x86_64

  ThingData:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: "ThingDataTable"
      AttributeDefinitions:
        - AttributeName: clientId
          AttributeType: S
        - AttributeName: createdAt
          AttributeType: S
      KeySchema:
        - AttributeName: clientId
          KeyType: HASH
        - AttributeName: createdAt
          KeyType: RANGE
      BillingMode: PAY_PER_REQUEST

  IoTTopicRule:
    Type: AWS::IoT::TopicRule
    Properties:
      TopicRulePayload:
        RuleDisabled: false
        Sql: "SELECT * FROM 'esp32/sensor/test'"
        Actions:
          - Lambda:
              FunctionArn: !GetAtt MQTTSubscribeHandler.Arn

  MQTTSubscribeHandlerPermission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: lambda:InvokeFunction
      FunctionName: !GetAtt MQTTSubscribeHandler.Arn
      Principal: iot.amazonaws.com
      SourceArn: !GetAtt IoTTopicRule.Arn

Building an AWS IoT Lambda Function with Golang

In this part, We're going to write the lambda function code. We'll use the AWS SDK for Go to write the lambda function.

First, We need to create a folder for the lambda function. We'll call our folder mqtt-subscribe-handler. Then We'll create a main.go file in the folder.

The project folder structure is as follows:

mqtt-subscribe-handler lambda function is under the functions folder.

lib folder contains the models and services We created.

Note : We will use the go.work so that we can use the models, services and libraries in the lambda function. lib folder has the its own go.mod module so If we want to use it in the lambda functions it must be in the same root level with the lambda functions but it is not. To solve this problem We need to add the lib and lambda functions to the go.work file. Keep in mind that information If you want to refactor the services and models for the aws lambda functions in golang.



go 1.21.4

use (
    ./functions/mqtt-subscribe-handler
    ./lib
)

Lets start writing the code.

lib/constants/dbNames.go



package constants

const THING_TABLE_NAME_KEY = "ThingDataTable"

While using dynamo db we need the table name. So We need to create a constant for the table name. It is related to Globals in the template file.



Globals:
  Function:
    Timeout: 5
    MemorySize: 128
    Environment:
      Variables:
        ThingDataTable: "ThingDataTable"

lib/helper/logger.go



package helper

import (
    "encoding/json"
    "log"
)

func EventLogger(event interface{}) {
    eventJSON, err := json.Marshal(event)
    if err != nil {
        log.Printf("Error marshalling event: %v", err)
    }
    log.Printf("Event:\n%s", string(eventJSON))
}

This helper function will be used to log events in json format.

lib/model/thingModel.go



package model

type ThingPayload struct {
    ClientId    string  `json:"clientId"`
    Temperature float64 `json:"temperature"`
    Humidity    float64 `json:"humidity"`
}

type ThingData struct {
    ThingPayload
    CreatedAt string `json:"createdAt"`
}

ThingPayload is the payload model We will use it as event in our lambda function. ThingData is for dynamo db. We'll use ThingData to save the data to DynamoDB.

ThingPayload and ThingData have properties in common so We can use ThingPayload as an embedded struct in ThingData.

lib/service/thingDataDbService.go



package service

import (
    "lib/constants"
    "lib/model"
    "os"

    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/dynamodb"
    "github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"
)

type ThingDataDbService struct {
    tableName        string
    dynamoDbProvider *dynamodb.DynamoDB
}

func (s ThingDataDbService) CreateThing(thing model.ThingData) error {
    attributeValue, err := dynamodbattribute.MarshalMap(thing)
    if err != nil {
        return err
    }
    input := &dynamodb.PutItemInput{
        Item:      attributeValue,
        TableName: aws.String(s.tableName),
    }
    _, err = s.dynamoDbProvider.PutItem(input)
    return err
}
func NewThingDataDbService(session *session.Session) ThingDataDbService {
    dynamoDBProvider := dynamodb.New(session)
    return ThingDataDbService{
        tableName:        os.Getenv(constants.THING_TABLE_NAME_KEY),
        dynamoDbProvider: dynamoDBProvider,
    }
}

CreateThing function will be used to save the data to DynamoDB.
As You can see we used the os.Getenv(constants.THING_TABLE_NAME_KEY) to get the table name. This is related to Globals in the template file.

After creating the models and services We can start writing the lambda function.

functions/mqtt-subscribe-handler/main.go




package main

import (
    "fmt"
    "lib/helper"
    "lib/model"
    "lib/service"
    "time"

    "github.com/aws/aws-lambda-go/lambda"
    "github.com/aws/aws-sdk-go/aws/session"
)

func handler(event model.ThingPayload) {
    helper.EventLogger(event)

    currentTime := time.Now().UTC().Format(time.RFC3339)
    sess, err := session.NewSession()
    if err != nil {
        fmt.Println("Error creating session: ", err)
        return
    }
    thingDataDbService := service.NewThingDataDbService(sess)
    thingData := model.ThingData{
        ThingPayload: event,
        CreatedAt:    currentTime,
    }
    err = thingDataDbService.CreateThing(thingData)
    if err != nil {
        fmt.Println("Error creating thing: ", err)
        return
    }

}

func main() {
    lambda.Start(handler)
}

First we log the event. Then We get the current time. After that We create a session. Then We create a thingDataDbService. Finally We create a thingData and save it to DynamoDB.

Deploying the SAM Template

In this part, We're going to deploy the SAM template. We'll use the AWS CLI to deploy the SAM template. We will use Makefile to run the AWS CLI commands. Here's how We do it:

1. Create a Makefile:



.PHONY: build

build:
    sam build

clean-deploy:
    rm -rf .aws-sam/cache
    sam build
    sam deploy --no-confirm-changeset

I create clean-deploy target to clean the cache and deploy the template because sometimes the changes in lib folder are not reflected in the lambda function. So We need to clean the cache and deploy the template again.

2. Deploy the SAM Template:



make clean-deploy

After running this command, We can see the following output in the terminal:

As We can see, the stack is created successfully.

Testing Our ESP32-AWS Architecture

To test it, I think it is enough to check the DynamoDB table. So We need to go to the DynamoDB console and click on the Tables tab. Then We can see the ThingDataTable table.

As We can see, the data sent by the esp32 is saved to DynamoDB.
Everything looks working properly.

Conclusion

In this article, We learned how to connect the esp32 to AWS IoT Core. We learned how to send data to AWS IoT Core. We learned how to create a SAM template to create a DynamoDB table, an IoT Core rule, and a Lambda function. We learned how to write a lambda function to save the data to DynamoDB. We learned how to deploy the SAM template. We learned how to test our architecture.

I hope you enjoyed this article. If you have any questions or suggestions, please feel free to ask me in the comments.

Repository

esp32 hardware code example for this blogpost (github)

cloudformation template and lambda function code example for this blogpost (github)

Connect with me

GitHub

Twitter

The Solution to the Issue of Signed URL for an S3 Object Expiring Before the Desired Time

BERAT DİNÇKAN — Fri, 27 Oct 2023 09:29:56 +0000

In my current project, my team was instructed to create a signed URL for an s3 object created programmatically so that they were able to download the s3 object without any AWS account.

Writing The Code

We created a lambda function. This lambda function used typescript and javascript aws sdk v3.

import { Context } from "aws-lambda";
import {
  GetObjectCommand,
  PutObjectCommand,
  S3Client,
  S3ClientConfig,
} from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
export const lambdaHandler = async (event: object, context: Context) => {
  const maxExpireTime = 604800; //7 days maximum expire time
  const region = "us-west-2";
  const s3client = new S3Client({
    region,
  } as S3ClientConfig);
  const bucketName = "my-bucket-name";
  const key = "my-key";
  const body = "my-body";

  const putObjectCommand = new PutObjectCommand({
    Bucket: bucketName,
    Key: key,
    Body: body,
  });
  await s3client.send(putObjectCommand);

  const getObjectCommand = new GetObjectCommand({
    Bucket: bucketName,
    Key: key,
  });
  const signedURL = await getSignedUrl(s3client, getObjectCommand, {
    expiresIn: maxExpireTime,
  });
  return {
    statusCode: 200,
    body: {
      signedURL,
    },
  };
};

According to this link, the max expiration time is 604800 seconds which means 7 days.

The Problem

So far so good. Lambda worked properly and gave us the signed URL but after 12 hours the signed URL for the s3 object expired and threw an error like below.

After investigating, we found out that if the token expires the signed URL expires.

The Solution

if you want a token that expires in 7 days which means more than 12 hours you need to use IAM user credentials with signature version 4.

We created IAM user access keys using the AWS console and implemented these keys with signature version 4 in our lambda function.

import { Context } from "aws-lambda";
import {
  GetObjectCommand,
  PutObjectCommand,
  S3Client,
  S3ClientConfig,
} from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
export const lambdaHandler = async (event: object, context: Context) => {
  const maxExpireTime = 604800; //7 days maximum expire time
  const region = "us-west-2";

  const accessKeyId = "MY_ACCESS_KEY_ID";
  const secretAccessKey = "MY_SECRET_ACCESS_KEY";

  const s3client = new S3Client({
    region,
    credentials: {
      accessKeyId,
      secretAccessKey,
    },
    signatureVersion: "v4",
  } as S3ClientConfig);
  const bucketName = "my-bucket-name";
  const key = "my-key";
  const body = "my-body";

  const putObjectCommand = new PutObjectCommand({
    Bucket: bucketName,
    Key: key,
    Body: body,
  });
  await s3client.send(putObjectCommand);

  const getObjectCommand = new GetObjectCommand({
    Bucket: bucketName,
    Key: key,
  });
  const signedURL = await getSignedUrl(s3client, getObjectCommand, {
    expiresIn: maxExpireTime,
  });
  return {
    statusCode: 200,
    body: {
      signedURL,
    },
  };
};

The Result

After using the IAM user access keys with signature version 4, the generated signed URL didn't expire after 12 hours as expected.

The Sources

Contact me:

Testing Mongoose with Ts-Jest

BERAT DİNÇKAN — Wed, 20 Apr 2022 12:51:24 +0000

If you want to learn MongoDB with mongoose, learning by testing is just for you. In this blog post, I talk about how to install ts-jest , how to create models and fake data using typescript and @faker-js/faker, and how to use jest to test them.

Why testing is important ?

Testing the code that We write makes us aware of the possible problems that occur in the future or gives us an idea about the behavior of the code. For instance, We have a car model and the car model has a field named age.The age field can not be negative. At this point, We need to be sure what happens when the age is a negative value. We give a negative input for the age field to the car model then we expect the car model throws an error in a testing module. So we can be sure if the car model works in line with the purpose before deploying the project.

What is jest?

Jest is a javascript testing framework. I will test all models by using jest. The reason I use the jest framework is that it requires minimum configuration for testing.

Creating the project and installing the packages

Creating the package.json



npm init -y

I will use ts-jest package in this blog post beacause ts-jest lets me use jest to test projects written in typescript.

Installing the packages.



npm install -D jest typescript ts-jest @types/jest ts-node @types/node

While installing mongoose we don't need the @types/mongoose because the mongoose package has built-in Typescript declarations.



npm install mongoose

Giving data to inputs by myself is hard so I install the @faker-js/faker package. @faker-js/faker helps me create random data for the models.



npm install -D @faker-js/faker

Creating tsconfig.json



tsc --init

Changing properties in tsconfig.json for the project



 "rootDir": "./src",
 "moduleResolution": "node",
 "baseUrl": ".",
 "outDir": "./build",

Adding include and exclude sides in tsconfig.json.



"include": ["src/**/*.ts"],
"exclude": ["node_modules","build"]

Creating config file for testing



npx ts-jest config:init

After that, You could see jest.config.js in the project folder. And that's it. We are ready to go.

Project Structure

I create two main folders named src and test because I accept this project as a real one. Model files will be in models folder in the src but tests of the models will be in the test.

Connecting the MongoDB

I Create the connectDBForTesting.ts in the test folder. My MongoDB runs on localhost:27018 if you have different options you could add or change connection options while you connect to MongoDB.



touch test/connectDBForTesting.ts

test/connectDBForTesting.ts



import mongoose from "mongoose";

export async function connectDBForTesting() {
  try {
    const dbUri = "mongodb://localhost:27018";
    const dbName = "test";
    await mongoose.connect(dbUri, {
      dbName,
      autoCreate: true,
    });
  } catch (error) {
    console.log("DB connect error");
  }
}

export async function disconnectDBForTesting() {
  try {
    await mongoose.connection.close();
  } catch (error) {
    console.log("DB disconnect error");
  }
}

Creating mongoose model

Models in mongoose are used for creating, reading, deleting, and updating the Documents from the MongoDB database. Let's create and test a Person model.



touch src/models/person.model.ts

src/models/person.model.ts



import mongoose, { Types, Schema, Document } from "mongoose";

export interface PersonInput {
  name: string;
  lastName: string;
  address: string;
  gender: string;
  job: string;
  age: number;
}

export interface PersonDocument extends PersonInput, Document {
  updatedAt: Date;
  createdAt: Date;
}

const PersonSchema = new mongoose.Schema<PersonDocument>(
  {
    name: { type: String, required: [true, "name required"] },
    lastName: { type: String },
    address: { type: String, required: [true, "address required"] },
    gender: { type: String, required: [true, "gender is required"] },
    job: { type: String },
    age: { type: Number, min: [18, "age must be adult"] },
  },
  {
    timestamps: true, // to create updatedAt and createdAt
  }
);

const personModel = mongoose.model("Person", PersonSchema);
export default personModel;

We have 2 important things here, PersonInput and PersonDocument interfaces. The PersonInput interface is used to create the personModel and the PersonDocument interface describes the object that is returned by the personModel. You will see clearly in the test section of the personModel.

Creating test for the personModel



touch test/person.model.test.ts

test/person.model.test.ts



import {
  connectDBForTesting,
  disconnectDBForTesting,
} from "../connectDBForTesting";

import personModel, {
  PersonDocument,
  PersonInput,
} from "../../src/models/person.model";
import faker from "@faker-js/faker";
describe("personModel Testing", () => {
  beforeAll(async () => {
    await connectDBForTesting();
  });

  afterAll(async () => {
    await personModel.collection.drop();
    await disconnectDBForTesting();
  });
});

First of all, the describe creates a block that includes test sections. You can add some global objects in the describe block to use them.

beforeAll runs a function before all tests in the describe block run. In the beforeAll, I connect the MongoDB server.

afterAll runs a function after all tests in the describe block have complated. In the afterAll, I disconnect the MongoDB server and drop the personModel collection.

PersonModel Create Test



test("personModel Create Test", async () => {
  const personInput: PersonInput = {
    name: faker.name.findName(),
    lastName: faker.name.lastName(),
    age: faker.datatype.number({ min: 18, max: 50 }),
    address: faker.address.streetAddress(),
    gender: faker.name.gender(),
    job: faker.name.jobTitle(),
  };
  const person = new personModel({ ...personInput });
  const createdPerson = await person.save();
  expect(createdPerson).toBeDefined();
  expect(createdPerson.name).toBe(person.name);
  expect(createdPerson.lastName).toBe(person.lastName);
  expect(createdPerson.age).toBe(person.age);
  expect(createdPerson.address).toBe(person.address);
  expect(createdPerson.gender).toBe(person.gender);
  expect(createdPerson.job).toBe(person.job);
});

Note : When a new personModel is declared it returns a PersonDocument type object. So I can use the mongoose.Document properties, validates, and middlewares.

I create a person object using personInput. The person.save() method inserts a new document into the database and return PersonDocument type object.

expect checks if the given data matches the certain conditions or not. If the given data matches the certain conditions the test passes. If not so, the test fails.

The last state of the test/models/person.model.test.ts



import {
  connectDBForTesting,
  disconnectDBForTesting,
} from "../connectDBForTesting";

import personModel, {
  PersonDocument,
  PersonInput,
} from "../../src/models/person.model";
import faker from "@faker-js/faker";
describe("personModel Testing", () => {
  beforeAll(async () => {
    await connectDBForTesting();
  });
  afterAll(async () => {
    await personModel.collection.drop();
    await disconnectDBForTesting();
  });

  test("personModel Create Test", async () => {
    const personInput: PersonInput = {
      name: faker.name.findName(),
      lastName: faker.name.lastName(),
      age: faker.datatype.number({ min: 18, max: 50 }),
      address: faker.address.streetAddress(),
      gender: faker.name.gender(),
      job: faker.name.jobTitle(),
    };
    const person = new personModel({ ...personInput });
    const createdPerson = await person.save();
    expect(createdPerson).toBeDefined();
    expect(createdPerson.name).toBe(person.name);
    expect(createdPerson.lastName).toBe(person.lastName);
    expect(createdPerson.age).toBe(person.age);
    expect(createdPerson.address).toBe(person.address);
    expect(createdPerson.gender).toBe(person.gender);
    expect(createdPerson.job).toBe(person.job);
  });
});

Running the jest

I add a command to the scripts in package.json to run the jest.



"scripts": {
    "test": "npx jest --coverage "
  },

coverage options indicates that test coverage information should be collected and reported in the output. But you can ignore it.

I run the test.



npm run test

The test result

To see what happens when a test fails I change a expect side with a wrong data on purpose.



expect(createdPerson.job).toBe(person.name);

The result of the test failing

The reason the test fails is that the jest expects the createdPerson.job and createdPerson.name to have the same data.

PersonModel Read Test



test("personModel Read Test", async () => {
  const personInput: PersonInput = {
    name: faker.name.findName(),
    lastName: faker.name.lastName(),
    age: faker.datatype.number({ min: 18, max: 50 }),
    address: faker.address.streetAddress(),
    gender: faker.name.gender(),
    job: faker.name.jobTitle(),
  };
  const person = new personModel({ ...personInput });
  await person.save();
  const fetchedPerson = await personModel.findOne({ _id: person._id });
  expect(fetchedPerson).toBeDefined();
  expect(fetchedPerson).toMatchObject(personInput);
});

I create a personModel and save it then fetch the person by _id. The fetchedPerson has to be defined and its properties have to be the same as the personInput has. I can check if the fetchPerson properties match the personInput properties using the expect.tobe() one by one but using expect.toMatchObject() is a little bit more easy.

expect.toMatchObject() checks if a received javascript object matches the properties of an expected javascript object.

Something is missing

For the each test, I created person model over and over again.It was not much efficient Therefore I declare the personInput and personModel top of the describe.



describe("personModel Testing", () => {}
const personInput: PersonInput = {
    name: faker.name.findName(),
    lastName: faker.name.lastName(),
    age: faker.datatype.number({ min: 18, max: 50 }),
    address: faker.address.streetAddress(),
    gender: faker.name.gender(),
    job: faker.name.jobTitle(),
  };
  const person = new personModel({ ...personInput });
)

So I can use the personInput and person objects in all tests.

PersonModel Update Test



test("personModel Update Test", async () => {
  const personUpdateInput: PersonInput = {
    name: faker.name.findName(),
    lastName: faker.name.lastName(),
    age: faker.datatype.number({ min: 18, max: 50 }),
    address: faker.address.streetAddress(),
    gender: faker.name.gender(),
    job: faker.name.jobTitle(),
  };
  await personModel.updateOne({ _id: person._id }, { ...personUpdateInput });
  const fetchedPerson = await personModel.findOne({ _id: person._id });
  expect(fetchedPerson).toBeDefined();
  expect(fetchedPerson).toMatchObject(personUpdateInput);
  expect(fetchedPerson).not.toMatchObject(personInput);
});

Even if I use the same schema, I can create personUpdateInput that is different from personInput because @faker-js/faker creates data randomly. The properties of fetchedPerson is expected to match the personUpdateInput at the same time is expect to not match the personInput.

PersonModel Delete Test



test("personModel Delete Test", async () => {
  await personModel.deleteOne({ _id: person._id });
  const fetchedPerson = await personModel.findOne({ _id: person._id });
  expect(fetchedPerson).toBeNull();
});

I delete a mongoose document by using person._id. After that, The fetchedPerson fetched from MongoDB by using is expected to be null.

The last State of the test/models/person.model.test.ts



import {

  connectDBForTesting,

  disconnectDBForTesting,

} from "../connectDBForTesting";

import personModel, {

  PersonDocument,

  PersonInput,

} from "../../src/models/person.model";

import faker from "@faker-js/faker";

describe("personModel Testing", () => {

  const personInput: PersonInput = {

    name: faker.name.findName(),

    lastName: faker.name.lastName(),

    age: faker.datatype.number({ min: 18, max: 50 }),

    address: faker.address.streetAddress(),

    gender: faker.name.gender(),

    job: faker.name.jobTitle(),

  };

  const person = new personModel({ ...personInput });

beforeAll(async () => {

    await connectDBForTesting();

  });

  afterAll(async () => {

    await personModel.collection.drop();

    await disconnectDBForTesting();

  });

test("personModel Create Test", async () => {

    const createdPerson = await person.save();

    expect(createdPerson).toBeDefined();

    expect(createdPerson.name).toBe(person.name);

    expect(createdPerson.lastName).toBe(person.lastName);

    expect(createdPerson.age).toBe(person.age);

    expect(createdPerson.address).toBe(person.address);

    expect(createdPerson.gender).toBe(person.gender);

    expect(createdPerson.job).toBe(person.job);

  });

test("personModel Read Test", async () => {

    const fetchedPerson = await personModel.findOne({ _id: person._id });

    expect(fetchedPerson).toBeDefined();

    expect(fetchedPerson).toMatchObject(personInput);

  });

  test("personModel Update Test", async () => {

    const personUpdateInput: PersonInput = {

      name: faker.name.findName(),

      lastName: faker.name.lastName(),

      age: faker.datatype.number({ min: 18, max: 50 }),

      address: faker.address.streetAddress(),

      gender: faker.name.gender(),

      job: faker.name.jobTitle(),

    };

    await personModel.updateOne({ _id: person._id }, { ...personUpdateInput });

    const fetchedPerson = await personModel.findOne({ _id: person._id });

    expect(fetchedPerson).toBeDefined();

    expect(fetchedPerson).toMatchObject(personUpdateInput);

    expect(fetchedPerson).not.toMatchObject(personInput);

  });

test("personModel Delete Test", async () => {

    await personModel.deleteOne({ _id: person._id });

    const fetchedPerson = await personModel.findOne({ _id: person._id });

    expect(fetchedPerson).toBeNull();

  });

});

Testing all



npm run test

Result

That's it. This is usually how to test mongoose models:

create a mongoose model.
create a test for the mongoose model.
apply the CRUD operations for the mongoose model in test sections.
if test fails, try to find out and solve the problem.
if the all tests pass, you are ready to go.

Sources:

Contact me:

Github Repo: https://github.com/pandashavenobugs/testing-mongoose-with-tsjest-blogpost

Mutual TLS(mTLS) with Nginx and NodeJs

BERAT DİNÇKAN — Fri, 11 Mar 2022 13:08:13 +0000

Mutual TLS a.k.a mTLS is a method for mutual authentication between server and client . It is also used to secure microservices. In this tutorial, I make a basic mTSL example using nginx and node.

Github Code

You can read the project belonging to this tutorial by getting the link below.

https://github.com/pandashavenobugs/mTLS-with-node-blogpost

First of all the client connects to the server then the server presents its own TLS certificate. Unlike the TLS encryption protocol, in mTLS both server and client have a certificate and the server verifies the certificate of the client. The client presents its own certificate. The server grants access to the client depending on whether the client of the certificate is verified or not.

What does the nginx server in this situation?

The nginx server presents the TLS certificate and verifies the certificate of the client. if there is a certificate of the client and the nginx server can verify it the nginx server grants the client access and passes the client request to node js express server using reverse proxy. If not, the nginx server rejects the client and sends an error message.

Creating certificates

Creating the certs folder and navigating to it.



mkdir certs
cd certs

Creating server certificate and key.

note After creating the certificate and key, we could see some questions about certificates such as country, locality name and email address, etc. You can pass all of the questions by pressing enter. In this tutorial, I'm passing all of these questions because I use the IP address instead of the hostname. In addition, you can change the rsa and days. It depends on what you want.



openssl req -x509 -nodes -days 9999 -newkey rsa:2048 -keyout server.key -out server.crt

Creating client certificate and key.



openssl req -x509 -nodes -days 9999 -newkey rsa:2048 -keyout client.key -out client.crt

Navigating back and copy-pasting the certs folder to /etc/nginx/ directory.



cd ..
sudo cp -r certs /etc/nginx/

nginx configuration

Heading to the /etc/nginx directory.



cd /etc/nginx/sites-available

Creating and editingthe nginx server configuration which is named myapp.

The myapp file



server{
    listen 443 ssl;
    ssl_certificate /etc/nginx/certs/server.crt;
    ssl_certificate_key /etc/nginx/certs/server.key;
    ssl_client_certificate /etc/nginx/certs/client.crt;
    ssl_verify_client on;
    location /api/ {
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }

}

Including the myapp file to /etc/nginx/nginx.conf.



http{
    .
    .
    .
    include /etc/nginx/conf.d/*.conf;
    #include /etc/nginx/sites-enabled/*;
    include /etc/nginx/sites-available/myapp;
}

I comment out the sites-enabled side because the myapp can be affected in a negative way.

In this example, using "ssl_client_certificate", the nginx server verifies the certificate which is claimed by the client. The client makes a request to the "https://{serverIP}/api/" host with the client certificate and client key. If the nginx server verifies the certificate, passes this request to "http://127.0.0.1:3000".

Note: Usually ssl certificates are used in the /etc/ssl directory but in this tutorial, I get the certificates from the certs folder in the /etc/nginx directory.

Creating node express server and client.



mkdir mtls_node
cp -r certs mtls_node
cd mtls_node
yarn init -y
yarn add express cors axios
yarn add @types/node @types/express @types/cors typescript ts-node -D
tsc -init
mkdir src
touch src/app.ts
touch src/client.ts

I created the mtls_node folder but you can name it whatever you want. I copy-pasted the certs folder to the mtls_node.

tsconfig.json file



{
  "compilerOptions": {
    /* Visit https://aka.ms/tsconfig.json to read more about this file */

    /* Projects */
    // "incremental": true,                              /* Enable incremental compilation */
    // "composite": true,                                /* Enable constraints that allow a TypeScript project to be used with project references. */
    // "tsBuildInfoFile": "./",                          /* Specify the folder for .tsbuildinfo incremental compilation files. */
    // "disableSourceOfProjectReferenceRedirect": true,  /* Disable preferring source files instead of declaration files when referencing composite projects */
    // "disableSolutionSearching": true,                 /* Opt a project out of multi-project reference checking when editing. */
    // "disableReferencedProjectLoad": true,             /* Reduce the number of projects loaded automatically by TypeScript. */

    /* Language and Environment */
    "target": "es2016",                                  /* Set the JavaScript language version for emitted JavaScript and include compatible library declarations. */
    // "lib": [],                                        /* Specify a set of bundled library declaration files that describe the target runtime environment. */
    // "jsx": "preserve",                                /* Specify what JSX code is generated. */
    // "experimentalDecorators": true,                   /* Enable experimental support for TC39 stage 2 draft decorators. */
    // "emitDecoratorMetadata": true,                    /* Emit design-type metadata for decorated declarations in source files. */
    // "jsxFactory": "",                                 /* Specify the JSX factory function used when targeting React JSX emit, e.g. 'React.createElement' or 'h' */
    // "jsxFragmentFactory": "",                         /* Specify the JSX Fragment reference used for fragments when targeting React JSX emit e.g. 'React.Fragment' or 'Fragment'. */
    // "jsxImportSource": "",                            /* Specify module specifier used to import the JSX factory functions when using `jsx: react-jsx*`.` */
    // "reactNamespace": "",                             /* Specify the object invoked for `createElement`. This only applies when targeting `react` JSX emit. */
    // "noLib": true,                                    /* Disable including any library files, including the default lib.d.ts. */
    // "useDefineForClassFields": true,                  /* Emit ECMAScript-standard-compliant class fields. */

    /* Modules */
    "module": "commonjs",                                /* Specify what module code is generated. */
    "rootDir": "./src",                                  /* Specify the root folder within your source files. */
    "moduleResolution": "node",                       /* Specify how TypeScript looks up a file from a given module specifier. */
    "baseUrl": ".",                                  /* Specify the base directory to resolve non-relative module names. */
    // "paths": {},                                      /* Specify a set of entries that re-map imports to additional lookup locations. */
    // "rootDirs": [],                                   /* Allow multiple folders to be treated as one when resolving modules. */
    // "typeRoots": [],                                  /* Specify multiple folders that act like `./node_modules/@types`. */
    // "types": [],                                      /* Specify type package names to be included without being referenced in a source file. */
    // "allowUmdGlobalAccess": true,                     /* Allow accessing UMD globals from modules. */
    // "resolveJsonModule": true,                        /* Enable importing .json files */
    // "noResolve": true,                                /* Disallow `import`s, `require`s or `<reference>`s from expanding the number of files TypeScript should add to a project. */

    /* JavaScript Support */
    // "allowJs": true,                                  /* Allow JavaScript files to be a part of your program. Use the `checkJS` option to get errors from these files. */
    // "checkJs": true,                                  /* Enable error reporting in type-checked JavaScript files. */
    // "maxNodeModuleJsDepth": 1,                        /* Specify the maximum folder depth used for checking JavaScript files from `node_modules`. Only applicable with `allowJs`. */

    /* Emit */
    // "declaration": true,                              /* Generate .d.ts files from TypeScript and JavaScript files in your project. */
    // "declarationMap": true,                           /* Create sourcemaps for d.ts files. */
    // "emitDeclarationOnly": true,                      /* Only output d.ts files and not JavaScript files. */
    // "sourceMap": true,                                /* Create source map files for emitted JavaScript files. */
    // "outFile": "./",                                  /* Specify a file that bundles all outputs into one JavaScript file. If `declaration` is true, also designates a file that bundles all .d.ts output. */
    "outDir": "./build",                                   /* Specify an output folder for all emitted files. */
    // "removeComments": true,                           /* Disable emitting comments. */
    // "noEmit": true,                                   /* Disable emitting files from a compilation. */
    // "importHelpers": true,                            /* Allow importing helper functions from tslib once per project, instead of including them per-file. */
    // "importsNotUsedAsValues": "remove",               /* Specify emit/checking behavior for imports that are only used for types */
    // "downlevelIteration": true,                       /* Emit more compliant, but verbose and less performant JavaScript for iteration. */
    // "sourceRoot": "",                                 /* Specify the root path for debuggers to find the reference source code. */
    // "mapRoot": "",                                    /* Specify the location where debugger should locate map files instead of generated locations. */
    // "inlineSourceMap": true,                          /* Include sourcemap files inside the emitted JavaScript. */
    // "inlineSources": true,                            /* Include source code in the sourcemaps inside the emitted JavaScript. */
    // "emitBOM": true,                                  /* Emit a UTF-8 Byte Order Mark (BOM) in the beginning of output files. */
    // "newLine": "crlf",                                /* Set the newline character for emitting files. */
    // "stripInternal": true,                            /* Disable emitting declarations that have `@internal` in their JSDoc comments. */
    // "noEmitHelpers": true,                            /* Disable generating custom helper functions like `__extends` in compiled output. */
    // "noEmitOnError": true,                            /* Disable emitting files if any type checking errors are reported. */
    // "preserveConstEnums": true,                       /* Disable erasing `const enum` declarations in generated code. */
    // "declarationDir": "./",                           /* Specify the output directory for generated declaration files. */
    // "preserveValueImports": true,                     /* Preserve unused imported values in the JavaScript output that would otherwise be removed. */

    /* Interop Constraints */
    // "isolatedModules": true,                          /* Ensure that each file can be safely transpiled without relying on other imports. */
    // "allowSyntheticDefaultImports": true,             /* Allow 'import x from y' when a module doesn't have a default export. */
    "esModuleInterop": true,                             /* Emit additional JavaScript to ease support for importing CommonJS modules. This enables `allowSyntheticDefaultImports` for type compatibility. */
    // "preserveSymlinks": true,                         /* Disable resolving symlinks to their realpath. This correlates to the same flag in node. */
    "forceConsistentCasingInFileNames": true,            /* Ensure that casing is correct in imports. */

    /* Type Checking */
    "strict": true,                                      /* Enable all strict type-checking options. */
    // "noImplicitAny": true,                            /* Enable error reporting for expressions and declarations with an implied `any` type.. */
    // "strictNullChecks": true,                         /* When type checking, take into account `null` and `undefined`. */
    // "strictFunctionTypes": true,                      /* When assigning functions, check to ensure parameters and the return values are subtype-compatible. */
    // "strictBindCallApply": true,                      /* Check that the arguments for `bind`, `call`, and `apply` methods match the original function. */
    // "strictPropertyInitialization": true,             /* Check for class properties that are declared but not set in the constructor. */
    // "noImplicitThis": true,                           /* Enable error reporting when `this` is given the type `any`. */
    // "useUnknownInCatchVariables": true,               /* Type catch clause variables as 'unknown' instead of 'any'. */
    // "alwaysStrict": true,                             /* Ensure 'use strict' is always emitted. */
    // "noUnusedLocals": true,                           /* Enable error reporting when a local variables aren't read. */
    // "noUnusedParameters": true,                       /* Raise an error when a function parameter isn't read */
    // "exactOptionalPropertyTypes": true,               /* Interpret optional property types as written, rather than adding 'undefined'. */
    // "noImplicitReturns": true,                        /* Enable error reporting for codepaths that do not explicitly return in a function. */
    // "noFallthroughCasesInSwitch": true,               /* Enable error reporting for fallthrough cases in switch statements. */
    // "noUncheckedIndexedAccess": true,                 /* Include 'undefined' in index signature results */
    // "noImplicitOverride": true,                       /* Ensure overriding members in derived classes are marked with an override modifier. */
    // "noPropertyAccessFromIndexSignature": true,       /* Enforces using indexed accessors for keys declared using an indexed type */
    // "allowUnusedLabels": true,                        /* Disable error reporting for unused labels. */
    // "allowUnreachableCode": true,                     /* Disable error reporting for unreachable code. */

    /* Completeness */
    // "skipDefaultLibCheck": true,                      /* Skip type checking .d.ts files that are included with TypeScript. */
    "skipLibCheck": true                                 /* Skip type checking all .d.ts files. */
  },
  "include": ["src/**/*.ts"],
  "exclude": ["node_modules"]
}

```

### src/app.ts

```typescript
import express, { NextFunction, Request, Response } from "express";
import cors from "cors";
const port = 3000;
const host = "127.0.0.1";
const app = express();

app.use(cors());

app.get("/api/test", (req: Request, res: Response, next: NextFunction) => {
  res.status(200).json({
    message: "certificate verified succesfully",
  });
});

app.listen(port, host, () => {
  console.log("im listening");
});
```

When the nginx server passes the get request to the express server that is running on "127.0.0.1:3000" the express server sends the client a message with a 200 status code.

### src/client.ts

```typescript
import axios, { AxiosError } from "axios";
import https from "https";
import fs from "fs";

const getRequestWithCertificate = async () => {
  try {
    const cert = fs.readFileSync("certs/client.crt");
    const key = fs.readFileSync("certs/client.key");
    const hostName = "192.168.0.20";
    const httpsAgent = new https.Agent({
      cert,
      key,
      rejectUnauthorized: false,
    });

    const response = await axios.get(`https://${hostName}/api/test`, {
      httpsAgent,
    });
    console.log(response.data);
  } catch (e: any) {
    const error = e as Error | AxiosError;
    if (!axios.isAxiosError(error)) {
      console.log("native error");
      // when it throws native error
      console.log(error);
    } else {
      // when it throws axios error
      if (error.request) {
        console.log("request error");
        console.log(error.request);
        //when requested but there is no response from server
      }
      if (error.response) {
        console.log("response error");
        // the request was made and server responsed tiwh a status code
        console.log(error.response.data);
        console.log(error.response.status);
        console.log(error.response.headers);
      }
    }
  }
};

setTimeout(() => {
  getRequestWithCertificate();
}, 1000);
```

While creating the client, the important thing is the "rejectUnauthorized: false" side. If the rejectUnauthorized is true, the axios throws the 'DEPTH_ZERO_SELF_SIGNED_CERT' error. If you are working with a server with its own self-signed certificate set the rejectUnauthorized false. It can cause some problems later.

# Compiling and running

Compiling the project.

```bash
tsc
```

Running the app.

```bash
node build/app.js
```

Running the client.

```bash
node build/client.js
```

When we run the build/client.js we should see the console log like this.

```js
{
  message: "certificate verified succesfully";
}
```

Thanks for reading.

Sources:

- https://www.cloudflare.com/learning/access-management/what-is-mutual-tls/
- https://www.youtube.com/watch?v=UzEzjon3IAo&t=171s
- https://medium.com/geekculture/mtls-with-nginx-and-nodejs-e3d0980ed950

Contact me:

- [Linkedin](https://www.linkedin.com/in/cengiz-berat-din%C3%A7kan-ab4208128/)

- [twitter](https://twitter.com/dinckan_berat)

- [github](https://github.com/pandashavenobugs)

The Solution of "curl: (6) Could not resolve host" Error with "nginx server_name"

BERAT DİNÇKAN — Wed, 09 Feb 2022 12:02:08 +0000

When I tried to reach out to the server that I named in nginx.conf, curl threw me an error:

curl myServerName.test:80

And then

"curl: (6) Could not resolve host: "

My nginx.conf

http{

     server {

        listen 80;
        server_name myServerName.test;


        return 200 "hello world!\n";
    }
}

The curl could not reach the host. I had to edit the /etc/hosts.

sudo nano /etc/hosts

and then /etc/hosts appeared like below

127.0.0.1       localhost

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

And added the myServerName.test after a space tab near the localhost. the last state of /etc/hosts :

127.0.0.1       localhost  myServerName.test

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

Tried the curl again

curl myServerName.test:80

Then

hello world!

Contact me

MQTT Response Pattern in Node.js

BERAT DİNÇKAN — Tue, 01 Feb 2022 17:36:15 +0000

Github Code

First thing first you can read the project belonging to this tutorial by getting the link below.

Github code: https://github.com/pandashavenobugs/MqttResponsePatternBlogPost

Purpose

Mqtt is a client server publish/subscribe messaging transport protocol and so popular in IoT projects. In this blog post I am going to get into MQTT response pattern not the basic. To checkout the MQTT essentials please get to link.

Warning: MQTT version 5 is required to use MQTT response pattern in this tutorial

Why Do We Need MQTT Response Pattern ?

MQTT is usually used for getting data from devices. Sometimes we may need to send data to devices, services, or people. Publishing the message(payload as Buffer) is enough to send subscribers data but there are some issues. For instance, we have a device that has a 3.3V relay and this device subscribes to the "device_1/relay_1". When the message ("1" or "0") is published to the "device_1/relay_1" topic the state of the relay changes depending on the message.

publishing message

So far so good but there are some issues with it. Is data received by the subscriber? What is the latest situation of the subscriber? The publisher is not sure if the subscriber receives the message. The publisher needs the response coming from the subscriber to be sure, however, only can make certain that the message is delivered to the broker by changing the level of the QoS. At this point the MQTT response pattern becomes necessary.

basic_mqtt_response_pattern_logic

In the MQTT response pattern we work on 2 main topics, request and response topics. The publisher subscribes to the response topic and the subscriber subscribes to the request topic. Note that! Response and Request topics have to be unique otherwise it could cause confusion. When the subscriber receives the message coming from the request topic immediately it creates and publishes a new response message to the response topic. The response message coming from the response topic is received by the publisher. So the publisher is sure that the subscriber has received the message. This is why we need MQTT response pattern.

Let's Practice

In this tutorial we're going to use just the mqtt package. The other packages are relevant to typescript.

Creating package.json



npm init -y

Installing mqtt package



npm install mqtt

Packges for typescript. Note that: mqtt npm package doesn't need @types/mqtt. So we dont intsall @types/mqtt.



npm install @types/node ts-node typescript -D

Creating tsconfig.json



tsc --init

Editing tscofnig.json. tsconfig.json in this project is like below.



{
  "compilerOptions": {
    /* Visit https://aka.ms/tsconfig.json to read more about this file */

    /* Projects */
    // "incremental": true,                              /* Enable incremental compilation */
    // "composite": true,                                /* Enable constraints that allow a TypeScript project to be used with project references. */
    // "tsBuildInfoFile": "./",                          /* Specify the folder for .tsbuildinfo incremental compilation files. */
    // "disableSourceOfProjectReferenceRedirect": true,  /* Disable preferring source files instead of declaration files when referencing composite projects */
    // "disableSolutionSearching": true,                 /* Opt a project out of multi-project reference checking when editing. */
    // "disableReferencedProjectLoad": true,             /* Reduce the number of projects loaded automatically by TypeScript. */

    /* Language and Environment */
    "target": "es6" /* Set the JavaScript language version for emitted JavaScript and include compatible library declarations. */,
    // "lib": [],                                        /* Specify a set of bundled library declaration files that describe the target runtime environment. */
    // "jsx": "preserve",                                /* Specify what JSX code is generated. */
    // "experimentalDecorators": true,                   /* Enable experimental support for TC39 stage 2 draft decorators. */
    // "emitDecoratorMetadata": true,                    /* Emit design-type metadata for decorated declarations in source files. */
    // "jsxFactory": "",                                 /* Specify the JSX factory function used when targeting React JSX emit, e.g. 'React.createElement' or 'h' */
    // "jsxFragmentFactory": "",                         /* Specify the JSX Fragment reference used for fragments when targeting React JSX emit e.g. 'React.Fragment' or 'Fragment'. */
    // "jsxImportSource": "",                            /* Specify module specifier used to import the JSX factory functions when using `jsx: react-jsx*`.` */
    // "reactNamespace": "",                             /* Specify the object invoked for `createElement`. This only applies when targeting `react` JSX emit. */
    // "noLib": true,                                    /* Disable including any library files, including the default lib.d.ts. */
    // "useDefineForClassFields": true,                  /* Emit ECMAScript-standard-compliant class fields. */

    /* Modules */
    "module": "commonjs" /* Specify what module code is generated. */,
    // "rootDir": "./",                                  /* Specify the root folder within your source files. */
    "moduleResolution": "node" /* Specify how TypeScript looks up a file from a given module specifier. */,
    "baseUrl": "." /* Specify the base directory to resolve non-relative module names. */,
    // "paths": {},                                      /* Specify a set of entries that re-map imports to additional lookup locations. */
    // "rootDirs": [],                                   /* Allow multiple folders to be treated as one when resolving modules. */
    // "typeRoots": [],                                  /* Specify multiple folders that act like `./node_modules/@types`. */
    // "types": [],                                      /* Specify type package names to be included without being referenced in a source file. */
    // "allowUmdGlobalAccess": true,                     /* Allow accessing UMD globals from modules. */
    // "resolveJsonModule": true,                        /* Enable importing .json files */
    // "noResolve": true,                                /* Disallow `import`s, `require`s or `<reference>`s from expanding the number of files TypeScript should add to a project. */

    /* JavaScript Support */
    // "allowJs": true,                                  /* Allow JavaScript files to be a part of your program. Use the `checkJS` option to get errors from these files. */
    // "checkJs": true,                                  /* Enable error reporting in type-checked JavaScript files. */
    // "maxNodeModuleJsDepth": 1,                        /* Specify the maximum folder depth used for checking JavaScript files from `node_modules`. Only applicable with `allowJs`. */

    /* Emit */
    // "declaration": true,                              /* Generate .d.ts files from TypeScript and JavaScript files in your project. */
    // "declarationMap": true,                           /* Create sourcemaps for d.ts files. */
    // "emitDeclarationOnly": true,                      /* Only output d.ts files and not JavaScript files. */
    "sourceMap": true /* Create source map files for emitted JavaScript files. */,
    // "outFile": "./",                                  /* Specify a file that bundles all outputs into one JavaScript file. If `declaration` is true, also designates a file that bundles all .d.ts output. */
    "outDir": "./build" /* Specify an output folder for all emitted files. */,
    // "removeComments": true,                           /* Disable emitting comments. */
    // "noEmit": true,                                   /* Disable emitting files from a compilation. */
    // "importHelpers": true,                            /* Allow importing helper functions from tslib once per project, instead of including them per-file. */
    // "importsNotUsedAsValues": "remove",               /* Specify emit/checking behavior for imports that are only used for types */
    // "downlevelIteration": true,                       /* Emit more compliant, but verbose and less performant JavaScript for iteration. */
    // "sourceRoot": "",                                 /* Specify the root path for debuggers to find the reference source code. */
    // "mapRoot": "",                                    /* Specify the location where debugger should locate map files instead of generated locations. */
    // "inlineSourceMap": true,                          /* Include sourcemap files inside the emitted JavaScript. */
    // "inlineSources": true,                            /* Include source code in the sourcemaps inside the emitted JavaScript. */
    // "emitBOM": true,                                  /* Emit a UTF-8 Byte Order Mark (BOM) in the beginning of output files. */
    // "newLine": "crlf",                                /* Set the newline character for emitting files. */
    // "stripInternal": true,                            /* Disable emitting declarations that have `@internal` in their JSDoc comments. */
    // "noEmitHelpers": true,                            /* Disable generating custom helper functions like `__extends` in compiled output. */
    // "noEmitOnError": true,                            /* Disable emitting files if any type checking errors are reported. */
    // "preserveConstEnums": true,                       /* Disable erasing `const enum` declarations in generated code. */
    // "declarationDir": "./",                           /* Specify the output directory for generated declaration files. */

    /* Interop Constraints */
    // "isolatedModules": true,                          /* Ensure that each file can be safely transpiled without relying on other imports. */
    // "allowSyntheticDefaultImports": true,             /* Allow 'import x from y' when a module doesn't have a default export. */
    "esModuleInterop": true /* Emit additional JavaScript to ease support for importing CommonJS modules. This enables `allowSyntheticDefaultImports` for type compatibility. */,
    // "preserveSymlinks": true,                         /* Disable resolving symlinks to their realpath. This correlates to the same flag in node. */
    "forceConsistentCasingInFileNames": true /* Ensure that casing is correct in imports. */,

    /* Type Checking */
    "strict": true /* Enable all strict type-checking options. */,
    // "noImplicitAny": true,                            /* Enable error reporting for expressions and declarations with an implied `any` type.. */
    // "strictNullChecks": true,                         /* When type checking, take into account `null` and `undefined`. */
    // "strictFunctionTypes": true,                      /* When assigning functions, check to ensure parameters and the return values are subtype-compatible. */
    // "strictBindCallApply": true,                      /* Check that the arguments for `bind`, `call`, and `apply` methods match the original function. */
    // "strictPropertyInitialization": true,             /* Check for class properties that are declared but not set in the constructor. */
    // "noImplicitThis": true,                           /* Enable error reporting when `this` is given the type `any`. */
    // "useUnknownInCatchVariables": true,               /* Type catch clause variables as 'unknown' instead of 'any'. */
    // "alwaysStrict": true,                             /* Ensure 'use strict' is always emitted. */
    // "noUnusedLocals": true,                           /* Enable error reporting when a local variables aren't read. */
    // "noUnusedParameters": true,                       /* Raise an error when a function parameter isn't read */
    // "exactOptionalPropertyTypes": true,               /* Interpret optional property types as written, rather than adding 'undefined'. */
    // "noImplicitReturns": true,                        /* Enable error reporting for codepaths that do not explicitly return in a function. */
    // "noFallthroughCasesInSwitch": true,               /* Enable error reporting for fallthrough cases in switch statements. */
    // "noUncheckedIndexedAccess": true,                 /* Include 'undefined' in index signature results */
    // "noImplicitOverride": true,                       /* Ensure overriding members in derived classes are marked with an override modifier. */
    // "noPropertyAccessFromIndexSignature": true,       /* Enforces using indexed accessors for keys declared using an indexed type */
    // "allowUnusedLabels": true,                        /* Disable error reporting for unused labels. */
    // "allowUnreachableCode": true,                     /* Disable error reporting for unreachable code. */

    /* Completeness */
    // "skipDefaultLibCheck": true,                      /* Skip type checking .d.ts files that are included with TypeScript. */
    "skipLibCheck": true /* Skip type checking all .d.ts files. */
  },
  "include": ["src/**/*.ts"],
  "exclude": ["node_modules"]
}
```

Creating utils/connectMqtt.ts

```ts
// utils/connectMqtt.ts file
import mqtt, { IClientOptions, MqttClient } from "mqtt";

const options: IClientOptions = {
  port: 1883,
  protocolVersion: 5,
  keepalive: 60,
  properties: {
    requestResponseInformation: true,
    requestProblemInformation: true,
  },
};

const mqttServerClient: MqttClient = mqtt.connect("mqtt:/127.0.0.1", options);

mqttServerClient.on("connect", () => {
  console.log(`connected to mqtt  ${new Date()}`);
});

mqttServerClient.on("error", (err) => {
  console.log(err);
});

export default mqttServerClient;
```

The properties object in option is supported by just MQTT 5.0. So we set the protocolVersion 5 to use properties object. At this point we have 2 necessary options, requestResponseInformation and requestProblemInformation. The [RequestResponseInformation](https://www.hivemq.com/blog/mqtt5-essentials-part9-request-response-pattern/#:~:text=sender%20and%20receiver.-,Response%20Information,-In%20the%20spirit) is used for the broker sending the clients the response information in the CONNACK packet. The requestProblemInformation is used by clients to indicate any failures.

Creating interfaces/relay.interface.ts

```typescript
export interface RelayResponseMessage {
  error: boolean;
  message: string;
}

export interface RelayRequestMessage {
  relayState: number;
}
```

Creating helpers/mqtt-async.helper.ts

```ts
import { IClientPublishOptions, MqttClient } from "mqtt";
import {
  RelayRequestMessage,
  RelayResponseMessage,
} from "../interfaces/relay.interface";

export function publishWithResponseBasic({
  client,
  message,
  publishOptions,
  requestTopic,
  responseTopic,
}: {
  client: MqttClient;
  message: 0 | 1;
  publishOptions: IClientPublishOptions;
  requestTopic: string;
  responseTopic: string;
}): Promise<string> {
  return new Promise((resolve, reject) => {
    const relayRequestMessage: RelayRequestMessage = {
      relayState: message,
    };
    client.subscribe(responseTopic);
    client.once("message", (topic, payload) => {
      client.unsubscribe(responseTopic);
      try {
        const relayResponseMessage: RelayResponseMessage = JSON.parse(
          payload.toString()
        );
        relayResponseMessage.error
          ? reject(relayResponseMessage.message)
          : resolve(relayResponseMessage.message);
      } catch (error) {
        resolve("JsonConvertError");
      }
    });
    client.publish(
      requestTopic,
      JSON.stringify(relayRequestMessage),
      publishOptions
    );
  });
}
```

mqtt-async.helpers.ts is a wrapper that turns the mqttClient callbacks into the async await function. It is required to subscribe to the response topic before publishing the message to the request topic. So that the publisher can receive the response data from the subscriber. In this situation the subscriber has to publish the response message to the response topic, otherwise, the publishWithResponseBasic function will not be closed and it will cause errors, memory leaks, etc. This is the weakness of this wrapper.

Creating requester_basic.ts

```ts
import { IClientPublishOptions } from "mqtt";
import { publishWithResponseBasic } from "./helpers/mqtt-async.helper";
import mqttServerClient from "./utils/connectMqtt";

const deviceName: string = "device_1";
const relayName: string = "relay_1";

setTimeout(() => {
  startSystem();
}, 1000);

const startSystem = () => {
  startResponsePatternExample();
};

const startResponsePatternExample = async () => {
  const responseTopic = `response/${deviceName}/${relayName}`;
  const requestTopic = `request/${deviceName}/${relayName}`;
  const publishOptions: IClientPublishOptions = {
    qos: 1,
    properties: {
      responseTopic,
      correlationData: Buffer.from("secret", "utf-8"),
    },
  };
  try {
    const responseMessage = await publishWithResponseBasic({
      client: mqttServerClient,
      publishOptions,
      requestTopic,
      responseTopic,
      message: 1,
    });
    console.log(responseMessage);
  } catch (error) {
    console.log(error);
  }
};
```

We have two optional parameters,[responseTopic](https://www.hivemq.com/blog/mqtt5-essentials-part9-request-response-pattern/#:~:text=a%20concrete%20example.-,Response%20Topic,-A%20response%20topic) and [correlationData](https://www.hivemq.com/blog/mqtt5-essentials-part9-request-response-pattern/#:~:text=out%20the%20request.-,Correlation%20Data,-Correlation%20data%20is). The responseTopic option represents the topic that the subscriber uses to publish the response message. The correlationData is a buffer data in nodeJS and used for additional information about the request message or a specific request. Also, it can be used for identifying by using a secret word as buffer data.

Creating subscriber_basic.ts

```ts
import { IClientSubscribeOptions } from "mqtt";
import mqttServerClient from "./utils/connectMqtt";

const opts: IClientSubscribeOptions = {
  qos: 1,
};
mqttServerClient.subscribe("request/+/+", opts);

mqttServerClient.on("message", (topic, payload, packet) => {
  console.log(packet);
  const { relayState } = JSON.parse(payload.toString());
  console.log(payload.toString());
  if (
    packet.properties &&
    packet.properties.responseTopic &&
    packet.properties.correlationData &&
    packet.properties.correlationData.toString() === "secret"
  ) {
    const responseData = {
      error: false,
      message: `${relayState === 1 ? "relay opened" : "relay closed"}`,
    };
    mqttServerClient.publish(
      packet.properties.responseTopic,
      JSON.stringify(responseData)
    );
  }
});
```

We focus on the packet parameter that contains the responseTopic and the correlationData. I use the correlationData as a secret code. The response message is published if the converted correlationData is 'secret'.

Compiling the typescript codes.

```bash
tsc
```

Compiled project is in build folder.

First of all we start the subscriber.

```bash
node build/subscriber_basic.js
```

Then we start the requester.

```bash
node build/requester_basic.js
```

The console log of the subscriber_basic.js.

```bash
Packet {
  cmd: 'publish',
  retain: false,
  qos: 1,
  dup: false,
  length: 82,
  topic: 'request/device_1/relay_1',
  payload: <Buffer 7b 22 72 65 6c 61 79 53 74 61 74 65 22 3a 31 7d>,
  messageId: 1,
  properties: {
    responseTopic: 'response/device_1/relay_1',
    correlationData: <Buffer 73 65 63 72 65 74>
  }
}
```

As you can see the subscriber receives the responseTopic and correlationData in the properties object in the packet. The subscriber can use these informations to publish the response message. This example works well within certain limits. We have one topic to request and response. What would happen if the publisher subscribed to multiple response topics or other topics ? We can not listen to the specific topics while using nodeJS MQTT package. The message coming from the subscribed topics can trigger the listener in the publishWithResponseBasic function. We need a solution more complex than before.

### Solution

For complex MQTT subscribe structures in nodeJS projects we take advanced of the nodeJS events. So we could pick out the specific topic by using event listeners with unique event names.

Creating relayResponseEvent.helper.ts

```ts
import EventEmitter from "events";
import { RelayResponseMessage } from "src/interfaces/relay.interface";

export function relayResponseEvent({
  eventEmitter,
  deviceName,
  relayName,
  payload,
}: {
  eventEmitter: EventEmitter;
  deviceName: string;
  relayName: string;
  payload: Buffer;
}) {
  const eventName = `responseEvent/${deviceName}/${relayName}`;
  const eventData: RelayResponseMessage = JSON.parse(payload.toString());
  return eventEmitter.emit(eventName, eventData);
}
```

The function that emits the eventData to the eventName.

Creating the publishWithResponse function in mqtt-async.helper.ts

```ts
export function publishWithResponse({
  client,
  data,
  publishOptions,
  responseEventName,
  requestTopic,
  eventEmitter,
}: {
  client: MqttClient;
  data: 0 | 1;
  publishOptions: IClientPublishOptions;
  requestTopic: string;
  responseEventName: string;
  eventEmitter: EventEmitter;
}): Promise<RelayResponseMessage> {
  return new Promise((resolve, reject) => {
    const checkTimeOut = setTimeout(() => {
      const relayResponseMessage: RelayResponseMessage = {
        error: true,
        message: "timeOut",
      };

      eventEmitter.emit(responseEventName, relayResponseMessage);
    }, 5000);
    eventEmitter.once(
      responseEventName,
      (relayResponseMessage: RelayResponseMessage) => {
        clearTimeout(checkTimeOut);
        relayResponseMessage.error
          ? reject(relayResponseMessage.message)
          : resolve(relayResponseMessage);
      }
    );

    const payload = { relayState: data };
    client.publish(requestTopic, JSON.stringify(payload), publishOptions);
  });
}
```

In this function we have 2 sections. The checkTimeOut is the setTimeout timer that will pass the relayResponseMessage to responseEventName after 5 seconds. The eventEmitter.once listens to the responseEventName. When the eventEmitter.once listener is invoked the checkTimeOut is cleared even if eventEmitter.once listener is called by the checkTimeOut. Why do we need the checkTimeOut timer? Actually the eventEmitter.once listener is invoked when the response message is published by the subscriber. The response message is passed to a specific responseEvent listener As soon as it is received by the publisher but somehow the subscriber may not publish the response message to the response topic. After waiting 5 seconds, if the eventEmitter.once listener with specific eventName is not invoked it means that something has been missing or gone wrong. So we can easily understand whether the subscriber publishes the response data or not.

Creating requester.ts

```ts
import EventEmitter from "events";
import { IClientPublishOptions, MqttClient } from "mqtt";
import { publishWithResponse } from "./helpers/mqtt-async.helper";
import { relayResponseEvent } from "./helpers/relayResponseEvent.helper";
import mqttServerClient from "./utils/connectMqtt";

const eventEmitter = new EventEmitter();

// ==> response/deviceName/relayName
mqttServerClient.subscribe("response/+/+");
mqttServerClient.subscribe("otherTopics/#");
mqttServerClient.on("message", (topic, payload) => {
  const topicArr = topic.split("/"); //spliting the topic ==> [response,deviceName,relayName]
  switch (topicArr[0]) {
    case "response":
      return relayResponseEvent({
        eventEmitter,
        deviceName: topicArr[1],
        relayName: topicArr[2],
        payload,
      });
    case "otherTopics":
      console.log("other topics");
      return;
    default:
      return console.log("can not find anything");
  }
});

const startSystem = () => {
  startResponsePatternExample({
    deviceName: "device_1",
    relayName: "relay_1",
    message: 1,
  });
  startResponsePatternExample({
    deviceName: "device_2",
    relayName: "relay_1",
    message: 1,
  });
};

const startResponsePatternExample = async ({
  deviceName,
  relayName,
  message,
}: {
  deviceName: string;
  relayName: string;
  message: 1 | 0;
}) => {
  try {
    const responseTopic = `response/${deviceName}/${relayName}`;
    const requestTopic = `request/${deviceName}/${relayName}`;
    const responseEventName = `responseEvent/${deviceName}/${relayName}`;
    const publishOptions: IClientPublishOptions = {
      qos: 1,
      properties: {
        responseTopic,
        correlationData: Buffer.from("secret", "utf-8"),
      },
    };
    const responseMessage = await publishWithResponse({
      client: mqttServerClient,
      data: message,
      publishOptions,
      requestTopic,
      responseEventName,
      eventEmitter,
    });
    console.log(`${deviceName}/${relayName} : ${responseMessage.message}`);
  } catch (error) {
    console.log(`${deviceName}/${relayName} : ${error}`);
  }
};
setTimeout(() => {
  startSystem();
}, 1000);
```

The requester subscribed to the response and other topics. When a message is received by the requester the topic gets split and converted to a string array(topicArr). If the first item of this string array is 'response' the payload is passed to the responseEventName listener by using the deviceName and relayName.

Compile and run

```bash
tsc
```

First of all we start the subscriber.

```bash
node build/subscriber_basic.js
```

Then we start the requester.

```bash
node build/requester.js
```

The console log of the requester.js.

```bash
device_1/relay_1 : relay opened
device_2/relay_1 : relay opened
```

We can pick out the response topics by using events. This example is not the best solution but I hope this tutorial gives you an idea about the MQTT response pattern.

Thanks for reading.

Contact me

- [Linkedin](https://www.linkedin.com/in/cengiz-berat-din%C3%A7kan-ab4208128/)

- [twitter](https://twitter.com/dinckan_berat)

- [github](https://github.com/pandashavenobugs)

DEV Community: BERAT DİNÇKAN

Building Multi-Agent Systems: @openai/agents Handoffs with NestJS and TypeScript

🤖 AI Agent Development with NestJS

📖 About This Project

🔧 Implementation

Step 1: Install @openai/agents

Step 2: Create Agent Module

Step 3: Create Agent Service

📁 Project Structure

Step 4: Initialize Agent Service

Step 5: Configure Environment

🚀 Getting Started

Agent Configuration Explained

Why Use Prompt Arrays?

Step 6: Implementing the Harmful Agent to the Agent Service

Understanding the checkThePlantIsHarmful Function

📁 Updated Project Structure

⚠️ IMPORTANT: Response Format Issue

🚨 Current Problem:

🎯 Why This Matters:

🔧 What We Need to Fix:

🔍 Solution: Structured Response Formatting

🎯 What is outputType?

🔧 How outputType Works:

📋 Benefits of Using outputType:

🚀 Implementation Steps:

🌿 Building the Poisonous Plant Analysis System

🔄 Step-by-Step Development Process

📝 Step 1: Create the Poisonous Plant Prompt

🎯 Why This Prompt Structure Works:

🏗️ Step 2: Create the Poisonous Plant Schema

🔍 Schema Design Principles:

🤖 Step 3: Create the Poisonous Plant Agent

🔄 Agent Handoffs: The Solution

🎯 How Agent Handoffs Work

🏗️ Implementing Agent Handoffs with Cat-Safe Agent

🧠 Create the Cat-Safe Prompt with Handoff Logic

🤖 Create the Cat-Safe Agent

⚠️ IMPORTANT: Agent.create vs new Agent()

🔧 Update Agent Service with Location Integration

🎮 Update Cat-Safe Service with Handoff Implementation

📍 Adding Location Information to the Controller

🔧 Why Location Information is Needed:

🎯 Controller Updates Required:

📝 Updated Controller Implementation:

🚀 How Handoffs Improve the Experience

Before (Multiple Requests):

After (Single Request with Handoffs):

🧠 Handoff Decision Logic

🎯 Benefits of Agent Handoffs

🎯 Results: Single Request, Multiple Output Types

📡 Single API Endpoint:

🚀 Key Achievements:

🎭 The Magic of Agent Handoffs:

📋 Summary

🎯 What We Built:

🚀 Key Technical Achievements:

🔗 Connect & Collaborate

👨‍💻 Find Me Online:

📂 Code Repository:

🤝 Get Involved:

📧 Contact:

ESP32 to AWS: Complete IoT Solution with IoT Core, DynamoDB, and Lambda Functions in Golang

Introduction

Technologies and Techniques Used

Hardware Side:

Cloud and Backend Side:

Creating a PlatformIO Project: Your First Step

Adding Libraries and Board Informations to PlatformIO

Installing Packages

note:

Setting Up Your First 'Thing' in AWS IoT Core with SSL Certificates

1. Log in to AWS IoT Core:

2. Create a Policy:

3. Create a 'Thing':

4. Download SSL Certificates:

Configuring ESP32 Filesystem for AWS IoT Core Connection

1. Create a 'data' Folder:

Writing the ESP32 Code for AWS IoT Core Connectivity

Bringing It All Together in main.cpp: Finalizing the ESP32-AWS Connection

Understanding the `checkThePlantIsHarmful` Function

🎯 What is `outputType`?

🔧 How `outputType` Works:

📋 Benefits of Using `outputType`: