DEV Community: Gokula Krishna

From benchmarking to fine-tuning: what I learned about small decision models

Gokula Krishna — Wed, 30 Sep 2026 05:47:25 +0000

I tested two small decision models. First I tested TypeSafe AI's hosted model, Jev. Then I tested Laya, an independent open-source alternative. I started out benchmarking and ended up fine-tuning.

Here is what I found, in the order I found it.

Part 1: Benchmarking Jev

TypeSafe AI launched Jev on September 15, 2026. It does not write text. You give it a message and a list of possible answers. It tells you how confident it is in each one.

I skipped the launch demos and ran my own evals on public Kaggle datasets:

31,369 test samples
12 classification tasks
Three areas: anti-money laundering, customer support intent, and malicious network traffic
0.3 s median response time, calling from Singapore to the US

The main lesson: accuracy depends on the task, and even more on how you set the task up.

Describe what the data shows, not what the criminal wants

My first anti-money laundering pattern descriptions said what the criminal was trying to achieve. I rewrote them to describe what the transactions look like.

Accuracy went from 65% to 77%. The model was the same. Only the descriptions changed.

The model sees transactions, not motives. So describe the labels using what's in the input.

Remove hidden bias from the setup

In account-level laundering detection, I switched to using the same time window for every account. That removed a hidden bias in the setup. Accuracy went from 63% to 75%.

Some tasks have no signal

Not everything worked. Classifying a single transaction as laundering or not gave 54%, which is about chance. One transaction on its own doesn't carry enough signal. This is a limit of the task, not of the model.

Confidence you can trust

For customer support intent, Jev picked the right answer 92.3% of the time. On banking intents it was 79.8%.

The confidence score is honest. When Jev said it was 90% sure, it was right about 90% of the time.

That makes thresholds useful. If you accept only predictions above 90% confidence, accuracy rises to 97.6%, and that still covers 82% of cases. The other 18% go to a fallback: a bigger model, a person, or a request for more information.

Ask the question directly

To catch off-topic messages, I first used a low top score as the signal. That gave 91.7%.

Then I added "is this off-topic?" as a separate yes/no question. That gave 97.7%. It costs no extra time, because everything is answered in one pass.

If you care about a decision, ask for it directly. Don't infer it from another score.

The dataset can be wrong too

On malicious network traffic, Jev scored 78% against the dataset labels as given.

One capture had about 400 windows labeled "benign". They were unanswered scans of thousands of hosts. Jev flagged them as malicious with 80% to 90% confidence. Without that capture, the score was 98%.

I report both numbers. The 98% is a filtered result. But the lesson holds: when the model disagrees with the ground truth, look at the data before you count it as a model error.

Part 2: Laya, the open-source alternative

Less than a week later, an open-source alternative appeared. Laya is a separate project, not a release of Jev's weights. It claims performance equal to Jev.

I ran it on 11 of the same tasks, using the same harness I built for Jev.

Accuracy

Task	Jev	Laya (untuned)
Help-desk requests	92.3%	49.7%
Banking questions	79.8%	39.4%
Money-laundering patterns	77.3%	13.5% (no better than guessing)

Across 11 tasks and 3 setups, Laya never beat Jev. On binary (yes/no) decisions it came much closer. On choosing between many answers it fell well behind.

Confidence was the real problem

The accuracy gap was not the biggest issue. The confidence scores were.

Help-desk task: 4,173 of 4,500 answers were at 99% confidence, and about half of them were right.
Banking task: Laya claimed about 97% confidence on almost everything, and got more than half wrong.

Calibration error measures how far confidence is from actual accuracy, where 0 is perfect. Jev scored 0.013. Laya scored 0.486.

That breaks confidence-based routing. With Jev, keeping only answers at 90%+ confidence raised accuracy to 97.6%. With Laya, the same filter dropped 7% of questions and gained only 2 points, to 51.7%.

It also breaks refusals. Of the off-topic questions Laya should have refused, it still answered 88% with 90%+ confidence. Jev did that on 14%.

My conclusion then: I wouldn't use untuned Laya to route between many options.

Part 3: Fine-tuning changed my mind

The next day I fine-tuned Laya on the banking task. It took about 140 minutes on my laptop.

Accuracy went from 39.4% to 78.3%, just 1.5 points below Jev's 79.8%. The task has 77 possible answers, so random guessing would get 1.3%.

The confidence fix mattered more. After fine-tuning, when Laya said it was 97% confident, it was right 96% of the time.

That is one confidence group, not a full calibration curve. But it's a big change from "99% sure, right half the time."

	Jev	Laya (fine-tuned)
Banking accuracy	79.8%	78.3%
Latency	~309 ms (hosted)	~90 ms (local)
Inference cost	~$0.07 per 1,000 requests	$0 (runs on my laptop)
Setup	none	fine-tune per task

The catch is that you need to fine-tune for each task, and that takes task-specific data. I have not yet tested off-topic detection on the fine-tuned model.

Part 4: Laya experts

So I built Laya experts: Laya models fine-tuned for specific tasks. Each one is lightweight, runs on a single machine, and makes a decision in about 80 ms.

One expert detects personally identifiable information (PII). On entity F1, the untuned Laya scored 0.156, Jev 0.736, and the Laya PII expert 0.967. A check like this could run at each stage of a data pipeline to support compliance.

Laya experts is published here: Laya experts.

What I'd tell someone trying these models

These models look like a real "System 1" decision layer. They handle fast classification, routing and other bounded decisions, so an LLM or agent doesn't have to call a generative model for everything.

To get good results, spend less time on the model and more on the setup around it:

Descriptions. Describe labels using what's visible in the input.
Ask directly. Make important decisions their own question.
Check confidence. Test whether "90% sure" really means right 90% of the time. If it doesn't, thresholds won't work.
Set thresholds and a fallback. Decide what happens to the cases the model isn't sure about.
Inspect disagreements. Sometimes the label is wrong, not the model.
Fine-tune when needed. An untuned open-source model can be far off. A short fine-tune on task data can close most of the gap.

Benchmark note: all figures are my own results on public benchmarks, with thresholds tuned per task. They are not vendor claims. Jev 1.13.0 (hosted). Laya 421M via laya-mlx, run locally on an Apple M3. The untuned Laya evaluation covered 23 runs and 65,078 requests.

Originally published at gokulakrishna.co.

I built a Digital Human you can talk to

Gokula Krishna — Mon, 03 Aug 2026 09:07:58 +0000

Most digital-human demos start with a prepared script or audio file. I wanted a live conversation.

A user had to speak through the browser. The system had to understand the question, retrieve relevant information, generate a response, synthesize speech, animate a face, and stream the result back as video.

The notes from our 21 June 2024 demo recorded about five seconds to stream video for a short, 20-token response. We ran that demo on an AWS G5.2xlarge instance using Groq, Chroma, AWS Polly, GeneFace++, FastAPI, and WebRTC.

A separate end-to-end test on the AWS A10G measured an 18-second minimum and a 25-second average for a ten-word response. The experiment log does not contain enough detail to reconcile the difference. I treat the five-second figure as a demo observation and the end-to-end table as the repeatable benchmark.

The five-second result came from a pipeline of separate systems. Each system had its own latency, output format, failure modes, and hardware requirements.

Defining the conversation

The proof of concept supported a turn-based, two-way loop:

The user spoke to the avatar through a browser.
The browser sent the audio to the backend.
The backend transcribed the speech and generated an answer.
The avatar spoke the answer and streamed video back to the browser.

Full-duplex barge-in remained outside the PoC. The notes do not show a user interrupting the avatar mid-response and forcing the system to cancel queued speech and video frames. That requires a different session state machine and cancellation path.

The architecture

Browser microphone → FastAPI session → Deepgram VAD + STT → Groq + Chroma → AWS Polly → GeneFace++ → WebRTC video → Browser

The system processed one turn through eight stages:

Capture: React captured the user’s microphone input in the browser.
Session handling: FastAPI accepted the audio stream and maintained the conversation session.
Turn detection: Deepgram’s voice activity detection identified speech boundaries.
Transcription: Deepgram converted the recorded turn into text.
Retrieval and generation: Chroma retrieved relevant document chunks, and Groq generated the response.
Chunking: The backend split the response into units that the speech and animation stages could process without waiting for the full answer.
Speech and animation: AWS Polly produced audio. GeneFace++ used that audio and the trained avatar data to generate video frames.
Delivery: aiortc sent the frames to the browser over WebRTC.

Chroma held indexed reference material for retrieval. It did not train the language model. That distinction matters because teams often label any data connected to an LLM as training data.

The latency budget

Tvisible ≈ Tendpoint + Tretrieval + Tfirst token + Tspeech chunk + Tavatar chunk + Ttransport

The individual benchmark values below cannot be added into one exact end-to-end total. We tested some components with different input sizes, GPUs, cloud regions, and levels of overlap. The equation shows where the delay came from and where we had to measure it.

The avatar renderer consumed much of the budget. GeneFace++ generated ten seconds of video in about five seconds on an RTX 4070. A one-second LLM response still felt slow if the system waited for a complete audio track and complete video before sending the first frame.

Text chunking gave the pipeline a way to overlap work. The speech and animation stages could process the first chunk while the LLM produced later chunks. Chunk size created a trade-off: small chunks reduced the wait but could damage prosody and create visible transitions; large chunks produced smoother output at the cost of a longer pause.

Technical specification

Layer	PoC choice	Role
Compute	AWS G5.2xlarge with NVIDIA A10G	Hosted the API and GPU avatar workload
Backend	Python 3.8 and FastAPI	Managed sessions and pipeline orchestration
Voice activity and STT	Deepgram	Detected turns and converted speech to text
LLM	Groq, with Mixtral 8x7B in the RAG tests	Generated the response
Retrieval	Local Chroma DB	Retrieved document context without a managed search service
Text-to-speech	AWS Polly	Generated the response audio
Avatar renderer	GeneFace++	Generated lip-synced video frames for the demo
Video transport	WebRTC through Python `aiortc`	Streamed low-latency video to the browser
Frontend	TypeScript and React with chat UI components	Captured audio and displayed the avatar conversation

The proposed architecture kept the avatar engine replaceable. We also evaluated SyncTalk and several other lip-sync systems. GeneFace++ powered the measured demo stack.

The source notes did not record the microphone codec, sample rate, chunk duration, VAD thresholds, WebRTC topology, or concurrent-session capacity. Those values belong in a production specification before another team tries to reproduce the system.

LLM and retrieval benchmarks

We averaged each result across ten runs. These numbers describe this PoC environment rather than a general ranking of the services.

Without a knowledge base

Setup	Throughput	Average latency	Observation
Groq	107 tokens/s	1.77 s	Best response time in this test; API variance remained a dependency
AWS Bedrock with Titan Express in Singapore	9.2 tokens/s	2.12 s	Easier managed setup; answer quality failed some basic test questions

With retrieval

Setup	Throughput	Average latency	Observation
Groq, Mixtral 8x7B, local Chroma	168.1 tokens/s	1.00 s	Fastest RAG configuration in the test
Groq, Mixtral 8x7B, AWS OpenSearch	9 tokens/s	11.00 s	Managed retrieval added a large delay in this setup
AWS Bedrock Claude Haiku and AWS OpenSearch in Oregon	10.46 tokens/s	10.22 s	Slowest managed configuration tested

The Chroma and OpenSearch rows include more than database time. Region, model endpoint, networking, and implementation choices also changed between configurations. The results support a decision about this PoC stack; they do not isolate the retrieval engine as the sole cause.

Text-to-speech benchmarks

Engine	Generation latency	Test input	Generated audio	Notes recorded during the evaluation
AWS Polly	1.25 s	337 characters	17.1 s	Fast managed API; limited custom-voice options for our use case
ElevenLabs	4.73 s	337 characters	17.1 s	Custom voice support; slower in this test
Picovoice	2.00 s	337 characters	22 s	Strong inference speed; custom voice required vendor coordination
Speechify	1.52 s	337 characters	26 s	Custom voice and good perceived quality
Deepgram	1.41 s	337 characters	17.1 s	Custom voice options, pause control, and local-hosting potential

AWS Polly gave us the clearest path to the demo because it generated a 17.1-second clip in 1.25 seconds and required no voice-model training.

Lip-sync benchmarks

We ran these tests on an RTX 4070 and measured the time required to generate ten seconds of video.

Framework	Generation time	Evaluation note
Wav2Lip	4 s	Fast setup, lower resolution, and no head movement in our test
GeneFace++	5 s	Fast inference and better lip movement; model training and environment setup took more work
SyncTalk	10 s	Best lip movement and image quality in our evaluation
Real3DPortrait	29 s	Incorrect head, lip, and torso movement in our test
SadTalker	45 s	Slow inference and weak lip sync for this use case
Hallo	More than 10 min	Natural body movement, but the inference time ruled it out for conversation

The evaluation notes also recorded commercial-use concerns for some frameworks. Licences and repository terms can change, so any production team should verify the current terms before selecting an engine.

End-to-end GPU results

We used Groq, AWS Polly, and GeneFace++ for the end-to-end GPU comparison. Prices show the hourly figures recorded during the 2024 evaluation.

Provider and GPU	Recorded price/hour	Minimum response time for 10 words	Inference speed	Average response time
AWS G5.2xlarge, A10G, Sydney	$1.212	18 s	45 it/s	25 s
Runpod, RTX 4090, Slovakia	$0.74	4.6 s	130 it/s	11.26 s
Runpod, RTX 4090, Romania	$0.74	5.6 s	123 it/s	11.44 s
Runpod, H100, Netherlands	$4.96	6 s	86 it/s	9.14 s

The RTX 4090 produced the shortest minimum response and the highest iteration rate in this workload. The H100 delivered the lowest average response time. Its recorded hourly price exceeded the 4090 instances by more than six times. Model architecture, clock speed, software optimisation, and provider variance affected the outcome. The GPU name alone did not predict the best user experience.

Why WebRTC

The avatar needed a browser-native path with low delay. HLS introduced segmenting and caching delays. RTMP lacked direct browser playback. Downloading complete files produced broken transitions between responses.

WebRTC handled media delivery in the browser and supported a future path toward two-way media. Python’s aiortc let us keep the media server in the same language as the AI pipeline, though it added setup work around tracks, timing, and connection state.

Production changes I would make

The PoC proved the conversation loop. A production version needs more control around cancellation, load, and measurement.

Add a session state machine

Each session should move through LISTENING, TRANSCRIBING, THINKING, and SPEAKING. A new user turn should invalidate stale LLM, TTS, and frame-generation jobs.

Introduce bounded queues and backpressure

The LLM can generate text faster than the avatar can render it. Unbounded audio and frame queues increase memory use and make interruptions feel broken.

Measure first-frame latency

Total response time hides the pause the user feels. I would record speech-end to transcript, first token, first audio chunk, first video frame, and playback start, with P50 and P95 values.

Separate GPU workers from the API

FastAPI should manage sessions while dedicated workers handle avatar inference. That separation makes GPU scheduling and horizontal scaling easier to control.

Design for barge-in

Full-duplex conversation needs interruption detection, job cancellation, audio ducking, and a clean return to the listening state. The session controller must drop stale frames from the previous response after the user starts a new turn.

Synchronise with timestamps

The media layer should align audio samples and video presentation timestamps. Queue length alone cannot preserve lip sync under network jitter or variable inference time.

What I learned

End-to-end latency followed the slowest media stage. The one-second RAG result helped, while the user still waited for speech generation, avatar inference, and the first playable WebRTC frame.

Streaming changed the useful metric. The system did not need to finish the whole response before it could begin playback. It needed to deliver the first coherent audio-and-video chunk, then keep producing chunks faster than the browser consumed them.

The hardware tests also challenged a common assumption. The most expensive GPU did not produce the shortest first response in our stack. Testing the complete pipeline gave us a better answer than choosing hardware from model specifications.

The PoC gave us a working digital-human conversation and a clear list of the remaining engineering work. The next version would focus on barge-in, cancellation, percentile latency, and concurrent sessions.

Demo

June 2024 digital-human proof of concept

Benchmark figures in this post come from the PoC tests recorded in June 2024. Cloud pricing, model endpoints, framework licences, and service performance may have changed since the evaluation.

How I Migrated a 350,000-Line Java/JSP Application to TypeScript in Five Days

Gokula Krishna — Fri, 31 Jul 2026 06:30:06 +0000

I recently migrated a 350,000-line Java/JSP application to TypeScript using AI agents.

Our original estimate for a conventional migration was three engineers and close to a year. Instead, one engineer orchestrating pools of up to 20 sub-agents at each stage got the application running in a development environment in five days.

The headline sounds like a story about faster code generation. It is not.

The decisive factor was verification. Once the agents had a reliable way to compare the old and new systems, parallelism became useful rather than chaotic.

350,000

lines of Java/JSP

5 days

to a running development environment

20

sub-agents in each stage

Why parallel agents were only half the solution

Jarred Sumner’s account of the Bun rewrite gave me useful ideas for running agents in parallel. Bun had an important advantage: a language-independent TypeScript test suite. The implementation could change while the expected behaviour remained measurable.

Our application had no equivalent safety net.

Only about half of the existing tests worked. Many had weak or missing assertions, and others depended directly on infrastructure. A large agent pool could generate code quickly, but without an independent oracle it could also produce 20 plausible versions of the wrong system.

We needed a way to answer a simple question at every stage: does the new application behave like the one already running?

The migration sequence

Database
APIs
UI
Business logic
Improvements

That order was deliberate. Each stage created a more stable foundation for the next, and it limited the number of variables that could change at once.

1. Reconstructing the database contract

The legacy application did not have a reliable database schema we could treat as authoritative.

We reconstructed the application-facing Oracle schema from the SQL queries embedded throughout the codebase. The agents identified tables, columns, joins, constraints and data assumptions from the queries, then we validated the reconstructed contract against the running system.

This was more than a database exercise. The schema defined the vocabulary shared by every API, page and business rule that followed. Starting anywhere else would have forced the agents to build on guesses.

2. Rebuilding the APIs

Once the data contract was stable, agents could migrate API components in parallel. Each work item was narrow: rebuild a component, exercise it against both systems, compare the result and resolve the discrepancy.

This gave the UI migration a dependable backend target instead of asking agents to infer behaviour simultaneously at every layer.

3. Migrating the UI

The UI required more than checking HTTP status codes. We needed to know whether the same data appeared, whether the same elements were present and whether the same interactions produced the same outcomes.

This is where the parity harness became the centre of the migration.

Building a parity harness where tests were missing

The parity harness ran the legacy and TypeScript applications side by side and compared them at three levels:

API responses and resulting database state
Page elements and screenshots
User interactions in both applications

The parity harness compared observable behaviour across the legacy and TypeScript systems.

The harness did not need to understand the intended design of the whole system. It needed to expose observable differences.

That distinction mattered. It turned the running legacy application into an executable specification, including behaviours that were not documented and edge cases that the surviving tests did not cover.

Parity before correctness

During the migration, we treated existing bugs as expected behaviour.

That may sound counterintuitive, but fixing bugs while changing languages, frameworks and architecture would have made failures ambiguous. When a result differed, we would not know whether we had introduced a regression or intentionally changed behaviour.

First reproduce the system. Then improve it.

We recorded known defects and fixed them only after achieving parity. This separated migration risk from product change and made discrepancies much easier to diagnose.

How the agent pools worked

At each migration stage, a pool of up to 20 sub-agents handled bounded pieces of work. Their jobs generally fell into three categories:

Rebuild a component in TypeScript.
Compare the old and new implementations through the parity harness.
Investigate and fix the differences.

The primary engineering task shifted from writing every line to designing work packets, maintaining dependency order, interpreting failures and improving the verification loop.

Parallelism helped because the work was decomposable. The harness helped because every agent could check its own result against the same external standard.

Without that feedback loop, adding more agents would have increased review load. With it, discrepancies became actionable inputs for the next iteration.

The cost

The migration consumed approximately 100 million output tokens.

At Fable 5 output-token pricing, that was roughly US$5,000 , excluding input tokens and other usage costs.

That number needs context. It is not the total cost of productionizing a modernized core system, and getting the application running in development is not the same as completing security review, performance testing, deployment and operational handover.

But it changes the economics of the first major migration milestone. A project estimated at three engineers for close to a year reached a working development environment with one engineer in five days.

What made the five-day migration possible

The result did not come from asking a model to “rewrite the application in TypeScript.”

It came from combining five practices:

Establish an observable source of truth before scaling code generation.
Migrate in dependency order so every stage has a stable foundation.
Give agents bounded work with clear inputs and outputs.
Preserve legacy behaviour until parity removes ambiguity.
Make comparison and repair part of every agent’s loop.

The parity harness was the force multiplier. It reduced the amount of trust required, made parallel work measurable and let agents detect many of their own mistakes before human review.

AI agents supplied the throughput. Verification converted that throughput into progress.

What this means for legacy modernization

The biggest shift may not be that AI writes code faster. It is that migration work can be reorganized around machine-executable feedback.

Systems once considered too large, too poorly documented or too risky to touch may become practical candidates when their observable behaviour can be captured and compared incrementally.

That does not remove the need for engineering judgment. It makes judgment more leveraged. The engineer still decides the sequence, defines the boundaries, builds the oracle and determines when parity is good enough to move forward.

As migration time and cost continue to fall, the question for many organizations may change from “Can we afford to modernize?” to “What would we need to verify it safely?”

Would you consider modernizing your core systems?

Get in touch if you are exploring an AI-assisted migration or need help designing a verification strategy.

Accelerate product development and marketing using these AI Agents

Gokula Krishna — Thu, 16 Jan 2025 12:00:00 +0000

2024 was the year of AI Chatbots, and 2025 will be the year of AI Agents. AI agents can act on your behalf and not just generate content. Here is the list of tools you can implement in your company to reap the benefits of AI agents.

1. Create a full-stack prototype using Replit Agent

Replit Agent is a great way to create full-stack applications using prompts. It helps with both the development and deployment of your complete application in the cloud with a predictable cost.

2. Rapid front-end prototyping with Locofy

Locofy is a Figma plugin that helps convert Figma designs into production-ready front-end code using its proprietary Large Design Model. The designs can be exported to front-end frameworks like NextJS, Vue, or Angular, and they directly help deploy to hosting providers such as Vercel.

Note: Locofy expects the Figma prototype in a particular format; you might want to change some processes in your design.

3. Automate report generation using Vanna.AI

Vanna.ai helps generate reports using natural language. It provides an interface to extract your DB schema and train the LLM. Once you have done that, it becomes easier to integrate their agent into Slack, have them start asking questions regarding your company data, and generate relevant reports.

4. Personalized marketing using Sync

Digital avatars allow you to hyper-personalize the message for users. For example, a company’s CEO can send personalized videos to millions of customers using the LipSync framework. This can be done with an open-source lip sync generator like SyncLabs.

5. Establish brand presence using ComfyUI

Tools like ComfyUI will help you create a content-generation pipeline using no-code tools. Here is an example pipeline to extract an anime character and place it on another background.

ComfyUI plugins can automatically generate content and post it on your social media.

Conclusion

As AI agents become more widespread, it is crucial to adopt these tools to stay competitive. It is also vital to have systems to verify the output generated by these tools that cause undesirable results.

Are you interested in integrating AI agents into your company? Feel free to get in touch.

Top 5 Startup Security Checklist

Gokula Krishna — Sun, 29 Dec 2024 12:00:00 +0000

In 2024, there were over 2,741 publicly disclosed data breaches in the U.S. alone, affecting 1.35 billion individuals. Small businesses were especially vulnerable, with 43% of cyberattacks targeting small businesses, yet only 14% are prepared to defend against them.

If you are a startup founder, the security of your application should not be an afterthought. In this post, I will share the top 5 essential security measures that every startup should implement to protect their application and their users. These recommendations are based on my experience as a CTO scaling an insurance tech platform across 10+ markets.

1. Conduct Penetration Testing

Penetration testing simulates real-world attacks to find vulnerabilities before malicious actors do. You don’t need an expensive security firm to get started.

Tools to get started:

OWASP ZAP — Free, open-source web app security scanner. Great for automated and manual testing.
Burp Suite Community Edition — Industry-standard tool for web vulnerability assessment.
Nmap — Network discovery and security auditing.

When to do it: Before every major release and at least quarterly. Consider hiring a professional penetration testing firm annually.

2. Enable Web Application Firewall (WAF)

A WAF protects your application from common web exploits like SQL injection, XSS, and DDoS attacks by filtering and monitoring HTTP traffic.

Recommended: Cloudflare WAF — offers a generous free tier with:

DDoS protection
Bot management
Rate limiting
IP reputation filtering
Custom firewall rules

Quick setup: Point your DNS to Cloudflare, enable WAF rules, and configure rate limiting for your API endpoints.

3. Implement Rate Limiting

Rate limiting prevents abuse by restricting the number of requests a user can make in a given time period. This protects against brute force attacks, credential stuffing, and API abuse.

Implementation approaches:

Application-level: Use middleware like express-rate-limit (Node.js) or django-ratelimit (Python)
Infrastructure-level: Configure rate limits in your CDN/WAF (Cloudflare, AWS WAF)
API Gateway: Use built-in rate limiting in API gateways

4. Block High-Risk Countries

If your startup operates in specific regions, consider geo-blocking traffic from countries where you don’t have users. This significantly reduces your attack surface.

How to implement:

Use Cloudflare’s IP Geolocation to identify visitor countries
Create firewall rules to block or challenge traffic from high-risk regions
Whitelist countries where your legitimate users are located
Consider using CAPTCHA challenges instead of outright blocks for borderline regions

Important: Always ensure geo-blocking doesn’t impact legitimate users, especially if you have a distributed team or use VPNs.

5. Secure Your Dependencies

Supply chain attacks through compromised dependencies are increasingly common. Regularly audit and update your dependencies.

Tools and practices:

npm audit / yarn audit — Built-in vulnerability scanning for Node.js projects
Dependabot / Renovate — Automated dependency update PRs
Snyk — Comprehensive vulnerability database and monitoring
Pin dependency versions in production
Review changelogs before updating major versions
Use lock files (package-lock.json, yarn.lock) and commit them

Conclusion

Security is not a one-time setup but an ongoing process. Start with these five fundamentals, and as your startup grows, invest in more comprehensive security measures like SOC 2 compliance, bug bounty programs, and dedicated security teams.

The cost of implementing these measures early is a fraction of the cost of a data breach. Protect your users, protect your business.

How to Delete Background from Videos and Add a New Background via CLI

Gokula Krishna — Mon, 23 Sep 2024 12:00:00 +0000

Recently, I was looking for tools to delete the background from a video I needed for a presentation. Unfortunately, the quality of the free online tools I tried wasn’t up to par. After some exploration, I found some open-source alternatives that delivered much better results. In this post, I’ll walk you through the process I followed and what I learned along the way.

I’ll introduce an open-source library called Rembg and demonstrate how to delete the background from a single image. Next, I’ll explain how to extract frames from a video and apply Rembg for background removal. Finally, I’ll show you how to add a new background to those images and combine them back into a video.

Prerequisites

Before we start, let’s make sure we have the following tools installed:

Python : For running AI models.
FFmpeg : A powerful tool for video processing. We will use it to extract video frames and combine images back to video.
ImageMagick : Tool for manipulating images. We will use it to overlay the foreground on a new background.
Rembg : An AI-powered tool to remove image backgrounds.

Ubuntu installation:

# Ubuntu procedure
sudo apt update

# Install Python
sudo apt install python3

# Install FFmpeg
sudo apt-get install ffmpeg

# Install rembg Python library
pip install rembg[gpu,cli]

# Install ImageMagick
sudo apt-get install imagemagick

Mac installation:

# Mac procedure
brew install python3
brew install ffmpeg
pip install rembg[gpu,cli]
brew install imagemagick

Step 1: Background Removal from a Single Image

Let’s start with removing the background from a single image using Rembg:

rembg i input.png output.png

You can also choose different AI models. I prefer the isnet-anime model for anime-style images:

rembg i -m isnet-anime anime.png output.png

Step 2: Extracting Frames from Video

To process a video, we first need to extract individual frames using FFmpeg:

mkdir frames
ffmpeg -i input_video.mp4 frames/frame_%04d.png

Step 3: Remove Background from All Frames

Now apply Rembg to all extracted frames:

mkdir frames_no_bg
for f in frames/*.png; do
  rembg i -m u2net "$f" "frames_no_bg/$(basename $f)"
done

Performance comparison:

CPU processing: ~12.7 seconds per frame
GPU processing: ~3.6 seconds per frame

Using a GPU dramatically speeds up the process.

Step 4: Add New Background and Combine

Use ImageMagick to overlay each frame on a new background:

mkdir frames_final
for f in frames_no_bg/*.png; do
  convert background.png "$f" -gravity center -composite "frames_final/$(basename $f)"
done

Finally, combine the frames back into a video:

ffmpeg -framerate 30 -i frames_final/frame_%04d.png -c:v libx264 -pix_fmt yuv420p output_video.mp4

Conclusion

Using open-source tools like Rembg, FFmpeg, and ImageMagick, you can achieve professional-quality background removal from videos entirely via the command line. The GPU acceleration makes it practical even for longer videos.

My journey and insights in building a chatbot for eCommerce claims handling

Gokula Krishna — Tue, 23 Apr 2024 12:00:00 +0000

Introduction

In this article, I will share my journey and insights while building a chatbot for automating eCommerce claims handling. The chatbot was designed to handle customer claims, guide users through the claims process, and integrate with existing systems for seamless resolution.

The Problem

In eCommerce, handling customer claims (returns, refunds, damaged goods) is a significant operational burden. Most companies rely on manual processing, which is slow, inconsistent, and expensive to scale. The goal was to automate this process using a conversational AI chatbot that could:

Understand customer claims in natural language
Guide customers through the required steps
Collect necessary information (order details, photos, descriptions)
Make decisions based on company policies
Escalate complex cases to human agents

The Plan

The chatbot needed to handle multiple claim types with different workflows. I broke the problem down into:

Intent recognition — Understanding what the customer wants
Entity extraction — Pulling out order IDs, product names, issue descriptions
Dialog management — Managing multi-turn conversations
Policy engine — Applying business rules for auto-resolution
Handoff — Smooth transition to human agents when needed

Evaluating the Tools

I evaluated several tools and frameworks:

Amazon Lex — AWS’s conversational AI service. Good integration with AWS ecosystem.
Dialogflow — Google’s NLU platform. Strong entity recognition.
Rasa — Open-source, highly customizable. Steeper learning curve.
Custom LLM-based — Using GPT/Claude for more flexible conversations.

I chose Amazon Lex for its integration with our existing AWS infrastructure and its built-in support for multi-turn conversations.

The Development Stack

Amazon Lex for conversational AI
AWS Lambda for fulfillment logic
DynamoDB for session and claims data
AWS Lex Web UI (github.com/aws-samples/aws-lex-web-ui) for the frontend
S3 for storing claim attachments

Operations

Running the chatbot in production taught me several lessons about operations:

Monitor confidence scores — Low confidence often indicates new intent patterns
Log everything — Every conversation turn should be logged for training data
A/B test responses — Small wording changes can dramatically affect resolution rates
Build escalation paths early — Not every case can be automated

Insights

Start with the most common claim types first (80/20 rule)
Users prefer guided flows over open-ended conversation for claims
Photo upload capability dramatically reduces back-and-forth
Clear expectations about what the bot can/cannot do reduces frustration
Regular retraining with real conversation data is essential
The biggest ROI came from reducing first-response time, not full automation

Building a claims chatbot is as much about understanding business processes as it is about AI technology. The key is starting simple, measuring everything, and iterating based on real user interactions.

Build and deploy your own Shopify checkout extension: A step-by-step guide

Gokula Krishna — Wed, 01 Mar 2023 12:00:00 +0000

I came across the Shopify checkout extension and decided to do a proof of concept to demo it to a client. Since this is a new feature with limited preview to only Shopify Plus store merchants, I couldn’t find any straightforward tutorial on deployment. There are good tutorials on YouTube on how to build it in development, but I couldn’t find solutions for deploying it on a live server.

In this tutorial, I will assume that you are new to Shopify development and break down the concepts.

Prerequisites

Familiarity with React development using TypeScript
A Shopify Partner account
Familiar with a reverse proxy such as Nginx
Ngrok setup
A new Shopify demo store

Initializing the Project

Create a new Shopify app with the following command:

npm init @shopify/app@latest

After that, change to the checkout-demo directory and run npm run dev. You will be prompted to log in to Shopify.

Once logged in via browser, go back to the terminal and choose your organization, app name, and the development store. A ngrok URL will be generated. Use it to install your app.

Creating the Checkout Extension

To add a new extension to this app, run:

npm run shopify app generate extension

From the wizard, choose checkout UI, give it a suitable name, and select TypeScript React. Then run npm run dev again. You will see a link to the dev console.

Product Recommendation

Now I’ll use a fake API to get a product recommendation. Change the directory to extensions/checkout-demo-ext/src.

Add network permission by opening shopify.ui.extension.toml and adding:

[capabilities]
network_access = true

Copy and paste the following code in extensions/checkout-demo-ext/src/index.ts:

import { useState, useEffect } from 'react';
import {
  render,
  Banner,
} from '@shopify/checkout-ui-extensions-react';

render('Checkout::Dynamic::Render', () => <App />);

function App() {
  const [productPlacement, setProductPlacement] = useState("Loading product recommendation...");

  useEffect(() => {
    fetch('https://fakestoreapi.com/products/9')
      .then(res => res.json())
      .then(json => setProductPlacement(`${json.title} ($${json.price})`));
  })
  return (
    <Banner title="checkout-demo-ext">
      {productPlacement}
    </Banner>
  );
}

Preparing for Deployment

Copy the below environment variables to a new file and get the relevant keys:

SHOPIFY_API_KEY=<Client ID from Shopify Partner > Apps > Client Credentials>
SHOPIFY_API_SECRET=<Client Secret>
HOST=https://your-domain.com
SCOPES=write_products
PORT=8081

Copy the below nginx config to /etc/nginx/sites-enabled/your-domain.com:

server {
    listen 80;
    server_name your-domain.com;

    location / {
        proxy_pass http://localhost:8081/;
    }
}

Web Setup

We need a CLI token for production deployment. Go to Shopify Partners Settings > CLI Token > Manage Token. Generate and copy the SHOPIFY_CLI_PARTNERS_TOKEN.

Go to the web/ directory and install dependencies:

cd web/
npm install dotenv
npm install -g pm2

Create a .env file under web and place all the environment variables. Then open web/shopify.js and add:

import * as dotenv from 'dotenv';
dotenv.config()

Start the web server:

pm2 start index.js --name 'Checkout UI'

Deployment

Build the app:

SHOPIFY_API_KEY=<Client ID> SHOPIFY_API_SECRET=<Client Secret> SCOPES=write_products HOST=https://your-domain.com PORT=8081 npx shopify app build

Deploy:

SHOPIFY_CLI_PARTNERS_TOKEN=<CLI token> SHOPIFY_API_KEY=<Client ID> SHOPIFY_API_SECRET=<Client Secret> HOST=https://your-domain.com PORT=8081 npx shopify app deploy

Note: Your previous development deployment will still be on the website and may throw errors during testing. I suggest creating different stores for dev/prod testing.

Activating the Plugin

On the bottom left, check the “Add an app” folder and you should see your plugin there.

Encrypting traffic between Cloudflare and Kubernetes cluster

Gokula Krishna — Sat, 10 Jul 2021 14:00:00 +0000

Objective

In this post, I’ll explain how to encrypt the traffic between Cloudflare and your Kubernetes cluster using TLS certificates. This ensures end-to-end encryption from the user’s browser to your application pods.

The Setup

The architecture involves:

Cloudflare as the CDN and DNS provider (handles browser-to-Cloudflare TLS)
Traefik as the Kubernetes ingress controller
TLS certificates for Cloudflare-to-cluster encryption

Steps

Step 1: In your Cloudflare dashboard, go to SSL/TLS and set the encryption mode to Full (Strict).

Step 2: Generate an Origin Certificate in Cloudflare. Go to SSL/TLS > Origin Server > Create Certificate. Download both the certificate and private key.

Step 3: Create a Kubernetes TLS secret with the Cloudflare origin certificate:

kubectl create secret tls cloudflare-origin-cert \
  --cert=origin-cert.pem \
  --key=origin-key.pem \
  -n default

Step 4: Configure Traefik IngressRoute to use the TLS certificate:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: site-a-ingress
  namespace: default
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`your-domain.com`)
      kind: Rule
      services:
        - name: site-a-service
          port: 80
  tls:
    secretName: cloudflare-origin-cert

Extending to Multiple Domains

To add another domain, repeat the process:

Generate a new origin certificate for the additional domain in Cloudflare
Create a new TLS secret in Kubernetes
Add a new IngressRoute pointing to the appropriate service

With this setup, all traffic between Cloudflare and your Kubernetes cluster is encrypted, ensuring end-to-end security for your applications.

Caesar’s Encryption in C++

Gokula Krishna — Sat, 10 Jul 2021 13:00:00 +0000

The Caesar cipher (also known as ROT13 for a shift of 13) is one of the simplest and most widely known encryption techniques. It is a type of substitution cipher where each letter is replaced by a letter a fixed number of positions down the alphabet.

How It Works

In ROT13, each letter is shifted by 13 positions. Since the English alphabet has 26 letters, applying ROT13 twice returns the original text. This makes it both an encryption and decryption function.

For example, the letter A becomes N, B becomes O, and so on.

The C++ Implementation

Here is the complete C++ program for ROT13 encryption:

#include <iostream>
#include <string>
using namespace std;

string rot13(string input) {
    string output = input;
    for (int i = 0; i < output.length(); i++) {
        char c = output[i];
        if ((c >= 'a' && c <= 'z')) {
            output[i] = ((c - 'a' + 13) % 26) + 'a';
        } else if ((c >= 'A' && c <= 'Z')) {
            output[i] = ((c - 'A' + 13) % 26) + 'A';
        }
    }
    return output;
}

int main() {
    string message;
    cout << "Enter message: ";
    getline(cin, message);

    string encrypted = rot13(message);
    cout << "Encrypted: " << encrypted << endl;

    string decrypted = rot13(encrypted);
    cout << "Decrypted: " << decrypted << endl;

    system("pause");
    return 0;
}

Testing It Out

When we encode a message:

And when we decode it back:

Since ROT13 shifts by exactly half the alphabet, the same function works for both encryption and decryption. This makes it elegantly simple, though not suitable for real security purposes.

Computing Prime numbers faster with C++

Gokula Krishna — Sat, 10 Jul 2021 12:00:00 +0000

Introduction

“Prime numbers are numbers which are divisible by 1 and itself” by definition. So why do we need to know about prime numbers? How do we find them using a computer program? How do we generate them? If generated, is there a faster method? This post answers these questions.

Application of Prime Numbers

Prime numbers have a wide range of applications in cryptography and pseudo random number generators. In cryptography, prime numbers are used to generate keys for public key techniques. The rand() function in C++ is also based on prime number generation. Let us create the best algorithm to generate prime numbers from scratch, analyze and improvise them.

Use of Modulo Operation

The modulo operator % gives the remainder when two numbers are divided:

2 % 2 = 0 // No remainder
5 % 4 = 1 // Remainder is 1

2 is divisible by 2 hence there is no remainder, but when 5 is divided by 4 the remainder is 1.

Let’s Code It!

The definition says “numbers which are divisible by 1 and itself.” So when a number is given, we check by dividing all numbers from 1 up to the given number. If we find that it has only 2 divisors, then it is prime.

On a computer, we count the number of modulo operations that result in zero:

The above gives us an idea of finding a pattern. See the zero counts of prime numbers and non-prime numbers. For example, 4 has 3 zeros as a result of modulo operation.

The left operand is constant (our variable n) and the right operand increments by one (variable i), so we use a loop. A counter variable increments when the modulus equals zero:

#include <iostream>
using namespace std;
int main(){
    int count=0;
    for (int i=1;i<=5;i++){
        cout << 5 << "%" << i << "= " << 5%i << endl;
        if (5%i==0){
            count++;
        }
    }
    if (count==2){
        cout << "Prime" << endl;
    }else{
        cout << "Not Prime" << endl;
    }
    system("pause");
    return 0;
}

Now the algorithm is made interactive by getting user input and checking whether it is prime.

Optimization: Checking up to n/2

As a programmer, we should improve the algorithm to take fewer steps. Basic algebra shows that it is enough to check from 1 to half of the number. Since 4×2 = 8, no factors will be greater than n/2. This saves computation time.

Here is a program demonstrating the performance difference when generating primes from 1 to 100 using both approaches (1 to n/2 vs. 1 to n):

The n/2 version requires significantly fewer operations than the traditional approach. This optimization becomes more impactful as the numbers get larger.