DEV Community: Noni Gopal Sutradhar Rinku

Building a Smart Recommendation System: Leveraging BERT for Skill Extraction

Noni Gopal Sutradhar Rinku — Fri, 14 Nov 2025 10:25:36 +0000

As developers, we're constantly looking for ways to build more intelligent and user-friendly applications. Recommendation systems are a prime example, powering everything from e-commerce suggestions to content discovery. In this post, I'll walk you through how I built an online recommendation system, highlighting the crucial role of AI and machine learning, particularly with the power of BERT for extracting valuable insights from resumes.

The Vision: A Personalized Skill-Based Recommender
My goal was to create a system that could intelligently recommend relevant opportunities or connections based on a user's skills. Imagine a platform where your unique skillset, gleaned directly from your CV, could instantly connect you with the perfect job, mentor, or project.

The Architecture: A Full-Stack Approach
To bring this vision to life, I opted for a robust full-stack architecture:
• Frontend (React): A dynamic and responsive user interface built with React to provide a seamless experience.
• Backend (Express.js): A flexible and powerful Node.js backend using Express.js to handle API requests, database interactions, and orchestrate the machine learning processes.
• Database: (You might want to mention which database you used here, e.g., MongoDB, PostgreSQL, etc.) To store user profiles, CV data, and recommendation results.
The AI Magic: Extracting Skills with BERT
The core intelligence of this recommendation system lies in its ability to accurately understand and extract skillsets from unstructured text, specifically CVs. This is where BERT (Bidirectional Encoder Representations from Transformers) comes into play.

Why BERT?
Traditional keyword matching can be brittle and often misses the nuanced meaning of language. BERT, on the other hand, is a pre-trained transformer-based model that excels at understanding context. It can process words in relation to all other words in a sentence, leading to a much deeper comprehension of the text.
Here's a simplified look at how BERT helps:
1. Contextual Understanding: When BERT reads a CV, it doesn't just look for "Python." It understands "proficient in Python development" as a strong indicator of Python skills.
2. Entity Recognition (and Adaptation): While BERT isn't natively a Named Entity Recognition (NER) model for skills, it provides powerful embeddings that can be fine-tuned or used in conjunction with other techniques (like rule-based methods or further machine learning models) to identify and categorize specific skills.

My Implementation Strategy:
• Data Preprocessing: Before feeding CVs to BERT, I performed standard NLP preprocessing steps: cleaning text, tokenization, and handling special characters.
• Skill Identification: (Example: "I leveraged BERT's contextual embeddings to identify potential skill phrases. These embeddings were then passed to a custom classification model trained on a dataset of common technical skills, allowing the system to accurately tag and extract relevant proficiencies.")
• Vector Representation: Each CV, once processed by BERT, was transformed into a rich vector representation of its contained skills.
The Recommendation Engine
Once the skills were extracted and represented numerically, the next step was to build the actual recommendation logic.
• User Profiles: Each user's skill vector formed the basis of their profile.
• Similarity Matching: When a user requested recommendations, the system would compare their skill vector with those of available opportunities or other users (depending on the recommendation type). Common similarity metrics like cosine similarity were used to find the closest matches.
• Ranking and Filtering: Recommendations were then ranked by similarity score and potentially filtered based on other criteria (e.g., location, experience level).
Lessons Learned and Future Enhancements
Building this system was a fantastic journey, and I learned a lot along the way:
• Data Quality is King: The accuracy of skill extraction heavily relies on the quality and diversity of the training data (if you fine-tuned BERT) and the cleanliness of the input CVs.
• BERT's Power: **BERT is incredibly powerful, but understanding its nuances and how to best integrate it into a pipeline is key.
• **Iterative Refinement: Recommendation systems are never truly "finished." Continuous monitoring, feedback loops, and model retraining are essential for maintaining relevance.
Looking ahead, I'm excited to explore:
• More Advanced NLP: Integrating other transformer models or exploring few-shot learning techniques for even better skill extraction.
• Hybrid Recommendation Approaches: Combining content-based recommendations (like skills) with collaborative filtering (based on user interactions) for even richer suggestions.
• Explainable AI: Providing users with clear reasons why a particular recommendation was made.
Conclusion
Building an AI-powered recommendation system is a challenging yet incredibly rewarding endeavor. By combining robust backend and frontend technologies with cutting-edge NLP models like BERT, we can create intelligent applications that truly understand and serve their users.
I encourage you to explore the world of NLP and machine learning in your next project. The possibilities are endless!
How does that sound? Would you like me to make any adjustments or add more technical details in specific sections?

Secure Service-to-Service Auth: Verifying Laravel JWTs in a NestJS Microservice

Noni Gopal Sutradhar Rinku — Tue, 11 Nov 2025 23:52:32 +0000

In a microservices world, your Laravel application often handles user authentication (via Sanctum or Passport) and issues a JWT (JSON Web Token). The challenge is securely allowing a separate, downstream service (like a NestJS data processor) to verify that token without needing to access Laravel's database on every single request.

The solution is to use the JWT verification public key, a pattern known as JWKS (JSON Web Key Set).

1. The Core Security Principle

Instead of the NestJS service asking Laravel: "Is this token valid?", the NestJS service asks: "Does the cryptographic signature of this token match the public key provided by Laravel?"

Laravel (Issuer): Uses its private key to sign the JWT, ensuring integrity.

NestJS (Verifier): Uses Laravel's public key (usually exposed via a public endpoint) to cryptographically verify the token's signature.

This process is fast, stateless, and secure.

2. Laravel Setup (The Issuer)

If you are using Laravel Passport, it already exposes the necessary public key. If you are using Sanctum, you may need a small package to expose the JWKS endpoint, but the principle is the same: the public key must be accessible.

The key file is typically found at: storage/oauth-public.key.

3. NestJS Setup (The Verifier)

We will use NestJS's standard @nestjs/passport and @nestjs/jwt strategy. The key is configuring the JwtModule to use the public key for verification.

A. The JWT Verification Strategy

We define a simple Passport Strategy to pull the token from the header and verify it using the public key.

// src/auth/jwt.strategy.ts

import { Injectable, UnauthorizedException } from '@nestjs/common';
import { PassportStrategy } from '@nestjs/passport';
import { ExtractJwt, Strategy } from 'passport-jwt';
import * as fs from 'fs';
import * as path from 'path';

// Load the public key from the Laravel API environment
// NOTE: In production, load this key via an environment variable or KMS.
const pathToPublicKey = path.join(__dirname, '..', '..', 'config', 'laravel_public.key');
const publicKey = fs.readFileSync(pathToPublicKey, 'utf8');

@Injectable()
export class JwtStrategy extends PassportStrategy(Strategy) {
  constructor() {
    super({
      jwtFromRequest: ExtractJwt.fromAuthHeaderAsBearerToken(),
      ignoreExpiration: false, // Always respect token expiration
      secretOrKey: publicKey,  // <-- CRITICAL: Use the Laravel Public Key here
      algorithms: ['RS256'],   // Use the signing algorithm Laravel Passport uses
    });
  }

  // The payload contains the user information embedded by Laravel
  async validate(payload: any) {
    if (!payload.user_id) {
        throw new UnauthorizedException('Token payload is missing user ID.');
    }
    // You can now access the user ID via req.user.user_id in your controllers
    return payload; 
  }
}

B. The Auth Module

We create an AuthModule to register the strategy and make it available application-wide.

// src/auth/auth.module.ts

import { Module } from '@nestjs/common';
import { PassportModule } from '@nestjs/passport';
import { JwtStrategy } from './jwt.strategy';

@Module({
  imports: [PassportModule.register({ defaultStrategy: 'jwt' })],
  providers: [JwtStrategy],
  exports: [PassportModule],
})
export class AuthModule {}

4. Final Step: Securing Controllers

In any NestJS controller that requires a valid, verified token issued by your Laravel API, simply apply the AuthGuard.

// src/user-data/user-data.controller.ts

import { Controller, Get, UseGuards, Req } from '@nestjs/common';
import { AuthGuard } from '@nestjs/passport';

@Controller('user-data')
export class UserDataController {

  // Use the default 'jwt' strategy defined in AuthModule
  @UseGuards(AuthGuard('jwt')) 
  @Get('profile-status')
  getProfileStatus(@Req() req) {
    // If execution reaches here, the token signature was verified by the public key.
    const userId = req.user.user_id; // Access the user ID from the validated payload

    return { status: `Data for user ${userId}` };
  }
}

Let me know your suggestions about this post.

Clean Architecture: Building Type-Safe AWS Service Wrappers in NestJS

Noni Gopal Sutradhar Rinku — Sun, 09 Nov 2025 00:10:20 +0000

If you've worked with the AWS SDK in a large Node.js application, you know it can quickly lead to "SDK spaghetti"—direct imports of bulky services, messy configuration, and difficult unit testing.

NestJS gives us the perfect tools (Modules and Providers) to enforce clean architecture by building lightweight, type-safe wrappers around external tools like the AWS SDK.

Here is the step-by-step guide to encapsulating the S3 SDK into a reusable, testable NestJS service.

1. Why We Need a Wrapper for the AWS SDK

Directly importing and using new S3Client(...) inside your controllers or business services creates problems:

Poor Testability: You can't mock or spy on the native SDK client easily.

Lack of Type Safety: Relying on the SDK's internal return types can make refactoring difficult.

Tight Coupling: Your business logic is directly coupled to AWS implementation details.

The solution is to define a clear TypeScript interface and inject a dedicated wrapper service.

2. Step 1: Define the TypeScript Interface

Start by defining exactly what your business application needs. We don't want to expose all 100+ S3 methods, only the ones we use (e.g., upload and getSignedUrl).

// src/aws/s3/s3.interface.ts

export interface S3File {
  key: string;
  url: string;
}

export interface IS3Service {
  /** Uploads a file buffer to S3. */
  uploadFile(
    fileName: string, 
    fileBuffer: Buffer, 
    mimeType: string
  ): Promise<S3File>;

  /** Gets a temporary signed URL for viewing a private file. */
  getTemporaryUrl(key: string): Promise<string>;
}

// Token for Dependency Injection (ensures type safety during injection)
export const S3_SERVICE_TOKEN = 'IS3Service';

3. Step 2: Implement the S3 Wrapper Service

Now, we create the NestJS Provider that implements our interface and handles the direct communication with the native AWS SDK client.

// src/aws/s3/s3.service.ts

import { Injectable } from '@nestjs/common';
import { 
  S3Client, 
  PutObjectCommand, 
  GetObjectCommand 
} from '@aws-sdk/client-s3';
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";

import { IS3Service, S3File } from './s3.interface';

@Injectable()
export class S3Service implements IS3Service {
  private s3Client: S3Client;
  private readonly bucketName = process.env.AWS_S3_BUCKET_NAME;
  private readonly region = process.env.AWS_REGION;

  constructor() {
    this.s3Client = new S3Client({ region: this.region });
  }

  async uploadFile(fileName: string, fileBuffer: Buffer, mimeType: string): Promise<S3File> {
    const key = `uploads/${Date.now()}-${fileName}`;

    const command = new PutObjectCommand({
      Bucket: this.bucketName,
      Key: key,
      Body: fileBuffer,
      ContentType: mimeType,
    });

    try {
      await this.s3Client.send(command);
      const url = `https://${this.bucketName}.s3.${this.region}.amazonaws.com/${key}`;
      return { key, url };
    } catch (error) {
      console.error("S3 Upload Error:", error);
      throw new Error('Failed to upload file to S3.');
    }
  }

  async getTemporaryUrl(key: string): Promise<string> {
    const command = new GetObjectCommand({
      Bucket: this.bucketName,
      Key: key,
    });

    return getSignedUrl(this.s3Client, command, { expiresIn: 3600 }); // 1 hour expiration
  }
}

4. Step 3: Create a Dedicated NestJS Module

Encapsulate the service into a dedicated module to manage configuration and ensure reusability. We use the S3_SERVICE_TOKEN for the export, ensuring we inject against the interface, not the concrete class.

// src/aws/s3/s3.module.ts

import { Module } from '@nestjs/common';
import { S3Service } from './s3.service';
import { S3_SERVICE_TOKEN } from './s3.interface';

@Module({
  providers: [
    {
      // Use the token for injection
      provide: S3_SERVICE_TOKEN,
      useClass: S3Service,
    },
  ],
  exports: [S3_SERVICE_TOKEN], // Export only the token
})
export class S3Module {}

5. Step 4: Using the Type-Safe Service

Now, any other module (like UsersModule or ProductsModule) that needs S3 access simply imports S3Module and injects the service using the token. This is clean and powerful.

// src/products/products.service.ts

import { Injectable, Inject } from '@nestjs/common';
import { IS3Service, S3_SERVICE_TOKEN } from '../aws/s3/s3.interface';

@Injectable()
export class ProductsService {
  // Inject against the Interface Token!
  constructor(
    @Inject(S3_SERVICE_TOKEN) private readonly s3Service: IS3Service,
  ) {}

  async createProductAttachment(fileBuffer: Buffer, fileName: string) {
    // Business logic...

    // Clean, type-safe call to the wrapper
    const s3Result = await this.s3Service.uploadFile(
      fileName, 
      fileBuffer, 
      'image/jpeg'
    );

    // Save s3Result.url to your database
    return { url: s3Result.url, message: 'File uploaded successfully.' };
  }
}

Conclusion: Testability and Maintainability Gains

By using this wrapper pattern, you have decoupled your application from the AWS SDK.

Testability: When testing ProductsService, you can easily provide a MockS3Service that implements IS3Service but returns fake data, never touching the network or the real AWS client.

Maintainability: If you ever switch from S3 to Google Cloud Storage or another provider, you only need to create one new GCSProvider class that implements IS3Service—your entire application remains untouched.

This is how professional NestJS applications handle external services: clean, encapsulated, and fully type-safe. Feel free to share your thoughts.

Scaling Laravel Queues: A Practical Guide to AWS SQS

Noni Gopal Sutradhar Rinku — Thu, 06 Nov 2025 21:46:48 +0000

When your Laravel application moves from MVP to a production system handling thousands of jobs per hour, the default database or Redis queue can quickly become a bottleneck. The solution for high availability and elastic scaling is almost always AWS SQS (Simple Queue Service).

Here's how to integrate SQS effectively to handle burst traffic and improve your application's reliability.

1. Why SQS Over Redis or Database Queues?

Feature	Redis/Database	AWS SQS
Durability	Data loss possible if the server crashes.	High durability, messages are replicated across multiple availability zones.
Elasticity	Scaling requires scaling up the host server.	Automatically scales to handle millions of messages per second.
Visibility Timeout	Basic control, sometimes buggy.	Robust control, ensuring workers don't process the same job simultaneously.

2. Setting up AWS IAM and SQS

Before touching Laravel, you need a queue in AWS and credentials to access it.

Create an SQS Queue: Choose a Standard Queue. Name it something logical (e.g., production-default-queue).

IAM User: Create a dedicated IAM user with the minimal necessary policy (Least Privilege Principle). This user needs:

sqs:SendMessage

sqs:ReceiveMessage

sqs:DeleteMessage

sqs:GetQueueAttributes

Credentials: Get the Access Key ID and Secret Access Key for this IAM user.

3. Laravel Configuration

Laravel makes the switch simple in your environment files.

Update your .env file with the SQS credentials and configuration:

# .env File Changes
QUEUE_CONNECTION=sqs

AWS_ACCESS_KEY_ID="your_iam_access_key"
AWS_SECRET_ACCESS_KEY="your_iam_secret_key"
AWS_DEFAULT_REGION="your_aws_region" # e.g., ap-southeast-2
AWS_QUEUE="[https://sqs.ap-southeast-2.amazonaws.com/123456789012/production-default-queue](https://sqs.ap-southeast-2.amazonaws.com/123456789012/production-default-queue)" 
AWS_QUEUE_PREFIX="${AWS_QUEUE}/" # SQS uses the full URL as the queue name

Next, ensure your config/queue.php file is correctly configured under the 'sqs' connection:

//config/queue.php
'sqs' => [
    'driver' => 'sqs',
    'key' => env('AWS_ACCESS_KEY_ID'),
    'secret' => env('AWS_SECRET_ACCESS_KEY'),
    'prefix' => env('AWS_QUEUE_PREFIX', '[https://sqs.us-east-1.amazonaws.com/](https://sqs.us-east-1.amazonaws.com/)'), // Ensure this is set!
    'queue' => env('AWS_QUEUE', 'default'),
    'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
]

4. Tuning Workers and Visibility

This is where performance is won or lost. SQS uses a Visibility Timeout—the time during which a message is hidden from other workers after being picked up.

Queue Setting (AWS Console): Set the SQS queue's Visibility Timeout to match the maximum runtime of your longest job (e.g., 300 seconds for 5 minutes).

Worker Timeout (Supervisor/Horizon): Set your worker process timeout to match, or slightly exceed, the SQS timeout. If the worker crashes, SQS will re-release the message after the timeout.

Running the Workers

If you use Supervisor to manage your workers, ensure the SQS connection is specified:

[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/html/artisan queue:work sqs --daemon --tries=3 --timeout=300
numprocs=5 # Start with 5 processes and scale up

Conclusion

Migrating to SQS immediately gives your application a durable, infinitely scalable message broker. It moves the responsibility of message handling and retry logic to a cloud service, freeing up your Laravel application to focus solely on processing business logic. This is the cornerstone of any highly available, modern Laravel deployment on AWS. Feel free to share your valuable suggestions.