DEV Community: Sergey Laptick

The Hidden Cost of "Fast Development" Nobody Talks About

Sergey Laptick — Fri, 07 Aug 2026 08:51:08 +0000

The tech industry is currently obsessed with a single metric: how fast AI can write code. We celebrate closed tickets, generated lines of code, and feature delivery times cut in half. However, there are some hidden costs that come with AI coding. For instance, accelerated debt accumulation.

The biggest danger of AI isn't bad code or syntax hallucinations. AI makes it absurdly easy to create technical debt faster than organizations can even recognize it exists. The bottleneck in software engineering has officially shifted from writing code to understanding the consequences of shipping it.

Technical Debt Used to Grow Slowly

Before LLMs took over our IDEs, technical debt had a natural speed limit.

Accumulating architectural mess took manual effort. A developer who wanted to introduce a poorly designed abstraction layer had to sit down, type out every interface, write the boilerplate, set up the dependency injection, and explain the implementation during a design discussion or code review. Typing speed, mental stamina, and review cycles acted as physical friction.

That delay gave teams a window to ask questions: Do we actually need another microservice for this? Is this dependency necessary? Couldn't we solve this with a simple function? You couldn't easily build five unnecessary services in an afternoon because your hands and your calendar simply wouldn't let you.

Today, as the Google Cloud 2025 DORA Report points out, AI acts as an amplifier. When you remove physical friction, you accelerate the rate at which downstream problems compound in your architecture, testing suites, and long-term maintenance.

AI as a Decision Multiplier

Every time you hit tab or accept a suggestion, you are accepting a series of design choices made by a probabilistic model:

A new abstraction layer;
A bespoke helper utility;
An extra API endpoint;
Another third-party dependency.

For example, data from GitClear's 2025 research shows that in AI-assisted repos duplicated and repeated logic rose 8x and copy-pasted code blocks rose from 8.3% to 12.3%. When the cost of adding complexity approaches zero, human nature dictates that we will add more complexity. But while generating software has become less costly, maintaining it remains painfully expensive.

Why Technical Debt Compounds Faster Than Ever

The acceleration of technical debt happens through a few distinct mechanisms that are quietly playing out across dev teams.

1. More Code Ships

When developers produce three times more code, they also ship three times as many hidden assumptions, edge cases, and subtle failure modes. Code generated by LLMs often contains security vulnerabilities out of the box. Add to that what the Stack Overflow 2025 Developer Survey calls the "Almost Right" tax: 45% of engineers identify "nearly correct" code as their biggest headache, with 66% reporting that they waste significant hours debugging logic that appeared sound at a glance.

2. Less Architectural Thinking

When solutions appear in your editor instantly, teams stop asking “Should we build this?” and start asking “Can AI build this?”

Those two questions lead to wildly different codebases. The first encourages minimalism and domain modeling. The second encourages feature creep, bloated interfaces, and solving simple problems with heavy infrastructure just because the prompt made it easy to spit out.

3. Nobody Fully Understands the System

Imagine a standard workflow today:

Developer A generates a module using Copilot;
Developer B generates a complementary feature using Cursor;
Developer C uses Claude to refactor the interaction between the two.

Six months later, a production incident occurs. The on-call engineer opens the file and realizes that nobody on the team actually designed the mental model behind the code. Understanding becomes the new bottleneck. This dynamic has triggered a trust crash. According to Stack Overflow's 2025 analytics, developer trust in AI-generated code dropped to 29%.

4. Review Becomes Impossible

Pull requests are growing larger while human attention spans remain fixed.

When a PR contains 800 lines of AI-generated boilerplate and helper methods, true peer review collapses under the sheer volume. As discussed in the article from XB Software's COO about AI impact on software engineering, AI changes how engineering work is evaluated: polished output no longer guarantees deep understanding. The same dynamic affects code reviews, where approvals can become a judgment of appearance rather than architecture.

Fast Development Creates Slow Organizations

This leads us to a strange paradox. Engineering managers look at their dashboards and celebrate. Ticket velocity is up! Commits are multiplying! Sprint burn-down charts look incredible!

Yet, on an organizational level, everything feels stickier:

Lead time for complex changes drifts upward;
Bug fixes take longer because tracing execution paths requires extra effort;
Onboarding a new developer takes months because the codebase lacks a coherent design narrative;
The team develops a deep fear of touching core modules.

Deployment gets faster, but fundamental change becomes slower.

The Honeycomb DORA 2025 report frames this as a systems problem. Optimizing local velocity (how fast an individual engineer pushes code out of their IDE) without an overarching design framework eventually degrades organizational velocity. Your mean time to recovery (MTTR) rises because when things break, nobody has the system-level intuition required to fix them quickly.

AI Didn't Create Technical Debt. It Removed Its Speed Limit

Software engineering was never just about typing speed. Now that those barriers are gone, only deliberate engineering discipline stands between a clean architecture and complete entropy. The solution to AI-driven debt is to put structure in place before the prompts start running.

A great real-world example of this shift comes from a practical experiment conducted by XB Software engineers on a mature, four-year-old legacy codebase. Rather than letting AI assistants generate code directly from loose user stories, they tested a spec-driven development approach.

Using tools like GitHub Spec Kit paired with Jira via Model Context Protocol (MCP), our team shifted AI's role to the planning phase. The AI was first tasked with analyzing existing legacy behavior and generating functional specifications before any implementation began. By establishing a solid specification as the source of truth, the team reduced ambiguity, avoided speculative architecture decisions, and proved that AI yields the highest return when used to enforce engineering structure rather than bypass it.

How Good Teams Prevent "AI Debt"

If you want your team to ship fast without drowning in maintenance costs, skip the generic productivity advice and focus on structural engineering habits:

Build less. Not every feature or helper module suggested by an LLM deserves to exist. Treat code as a liability, not an asset.
Delete aggressively. Celebrate PRs that net negative lines of code. If AI makes writing replacement code trivial, make code removal a core productivity metric;
Review architecture, not syntax. LLMs can handle syntax and formatting. Human code reviews should focus exclusively on boundary design, data flow, and architectural decisions;
Enforce design reviews before prompting. Require written design docs, type definitions, or OpenAPI specifications before developers generate feature implementations;
Track maintainability metrics. Stop measuring commits or lines of code. Watch your deployment frequency, rollback rates, MTTR, cycle time, and developer onboarding times. Those metrics reveal the true cost of hidden debt.

Teams that successfully navigate AI-assisted software development establish explicit architecture guardrails and review standards that keep high execution speed from turning into unsustainable long-term maintenance costs.

Conclusion

Fast development can be dangerous because organizations can now accumulate years of structural technical debt in a matter of weeks without realizing it. The bottleneck in modern software engineering is comprehension. The engineering organizations that thrive over the next decade will be the ones that build the strongest engineering discipline around the code they choose to keep.

Manual vs. Automated Software Testing With AI Supercharges

Sergey Laptick — Fri, 31 Jul 2026 08:20:36 +0000

With applications growing ever more intricate and release cadences tightening, development houses are constantly revisiting their quality assurance practices. The long-running discussion around hand-performed versus scripted checks has now gained a fresh dimension: AI-powered testing.

While automation has traditionally been the means to accelerate repetitive verification, the real advance is the shift away from rigid, predefined scripts toward flexible, context-sensitive approaches. Throughout this article, we’ll explore both manual and automation testing, and illustrate how Playwright MCP and AI agents are redefining QA.

What Manual Testing Entails and Where It Still Proves Essential

Manual testing consists of a human examiner stepping through test scenarios without any automation aid, checking whether the software’s functions, usability, and overall response match expectations. The tester clicks through the interface, fills in fields, and monitors outcomes, just as a real user would operate the application.

Far from being obsolete, this practice continues to be crucial in many contexts. Manual effort excels when human discernment, observation, and spontaneity are indispensable. It becomes especially powerful when requirements shift frequently or when the user interface and experience demand direct, subjective assessment. Likewise, it remains the preferred route for exploratory, ad‑hoc, and usability testing, and situations where a prewritten script cannot foresee all the ways a person might interact with the system.

Skilled testers lean on their intuition, reshape their approach as the product evolves, and often catch defects early in the development of web-based applications. They log manual procedures and any flaws uncovered during free-form exploration, which feeds into adaptive planning for upcoming iterations. Building consistent feedback channels with developers helps resolve reported issues quickly, lifting the overall level of software quality.

Key Weaknesses of Manual Testing

For all its advantages, it has issues as well:

Manual verification is slow;
Susceptible to oversight;
Tough to scale under rapid delivery schedules.

These limitations are what have driven many teams to adopt automation for the bulk of repetitive work.

Automated Testing and Its Core Strengths

Automated testing runs pre-authored scripts at high velocity through dedicated tools and frameworks, validating application functionality without ongoing human intervention. It forms the engine of contemporary CI/CD pipelines, letting QA teams execute thousands of checks within minutes with exceptional precision.

The main advantage lies in handling large-scale and recurrent activities effectively. Automation boosts correctness, cuts down on human mistakes, and slots neatly into continuous integration and continuous testing workflows. The approach delivers high dependability for tests that are stable and repeatable, and scales readily as features adjust or the system expands. This makes it a natural fit for regression packs, performance evaluations (both load and stress scenarios), and bulk execution of extensive test suites.

“New or fast-changing functionality is often best validated manually first. Once a scenario becomes stable, repeatable, and business-critical, it becomes a strong candidate for automation.”

Denis Matsiush, QA Engineer

Automated Software Testing with Playwright

To ground the discussion, consider a real illustration using Playwright, a browser automation framework that works across Chromium, Firefox, and WebKit. The script snippet below replicates how a customer might search for an item on an online store:

test('should filter products when searching', async ({ page }) => {
  // 1. Locate the element using its ID
  const searchInput = page.locator('#search-input');

  // 2. Simulate user action: Typing "keyboard" into the field
  await searchInput.fill('Keyboard');

  // 3. Perform action: Click the search button
  await page.locator('#search-button').click();

  // 4. Verification: Expect the results to show exactly one product
  const products = page.locator('.product-card');
  await expect(products).toHaveCount(1);

  // 5. Verification: Ensure the product contains the correct text
  await expect(page.locator('.product-info h3')).toContainText('Keyboard');
});

This script identifies the search input using a CSS selector #search-input. It then uses the .fill() method to type text and .click() to submit. Finally, it uses assertions (expect) to verify that the application responded correctly by showing the expected item. This level of detail and precision makes scripts invaluable for repetitive validation.

What Are the Drawbacks of Automated Testing?

Automation is not a magic wand:

It takes time to set up;
Requires skilled resources;
It's not ideal for usability or exploratory testing.

Moreover, scripts can be brittle: any change in the UI’s structure often breaks the test, requiring manual updates.

Manual vs. Automated Software Testing: A Head‑to‑Head Comparison

Let’s summarize the key differences between manual and automated approaches across accuracy, speed, ideal use cases, and cost:

How AI Expands the Boundaries of Test Automation

Modern software testing taps into large language models to produce checks that are not only automated but also context‑aware, resilient, and capable of self‑repair. A key enabler in this transformation is the Model Context Protocol (MCP) that establishes a structured, bidirectional link between AI models and external applications.

It powers Playwright MCP and forms a browser automation system in which agents perceive web pages through organized accessibility snapshots. As a result, the AI can interpret the layout, locate elements by their semantic roles, and flexibly respond to interface modifications without the tests falling apart.

Inside an AI‑Driven Test Flow with Playwright MCP

A typical AI‑powered testing cycle unfolds in several stages:

Setup. The MCP server is deployed and configured to translate incoming instructions into browser operations;
Capability Discovery. The client asks the server what actions are available (e.g., navigation, clicks, text entry, screen capture);
Command Generation. The model outputs JSON‑formatted commands that correspond to a given test scenario;
Browser Execution. The MCP server receives the commands and uses Playwright to execute them in a real web browser, engaging with the real user interface and recording the resulting state;
Contextual Feedback. The server sends back accessibility snapshots and operation logs, which the AI uses to plan its next moves and correct course.

This iterative loop enables the AI to mimic genuine user journeys, surface edge cases that static scripts might overlook, and automatically recalibrate test steps when the UI evolves.

What AI Brings to Software Testing

Self‑healing tests. AI‑powered models locate and work with web elements by understanding their context, drastically reducing failures caused by minor layout tweaks;
Intelligent test creation. Leveraging LLMs, teams can auto‑generate test cases from application data, boosting coverage without writing every script by hand;
Edge case surfacing. Autonomous agents uncover unexpected behaviors that conventional, rigid tests often miss;
Self‑maintaining suites. The entire test portfolio adapts as the product changes, reconfiguring flows on the fly;
Parallel and distributed execution. Multiple browser instances can run concurrently, slashing total execution time and raising throughput.

The Growing Ecosystem of AI Testing Tools

Beyond Playwright MCP, several other tools are making testing more accessible and practical for teams of all sizes:

KaneAI is an AI‑native automation solution that allows users to plan, author, and evolve end‑to‑end tests using natural language. It supports web applications as well as Android and iOS native apps, and integrates with over 120 tools, including Jira for defect tracking;
Autify offers an AI‑agent‑powered test automation platform. Autify Nexus features a chat‑based agent that generates scenarios from natural language and automatically repairs broken tests. Autify Genesis uses generative AI to design test cases from specifications or even uploaded PDF documents;
BetterBugs focuses on the bug reporting side of the process. It allows testers to record a session (screen recording, console logs, network requests) and then pass that session, along with a prompt, to Claude for automated root cause analysis and proposed fixes;
Testmo has introduced AI Test Case Generation, which generates structured, reviewable test cases directly from requirements. Teams can provide clear requirements, let the system propose an initial set of test cases, and then review and refine them before generating real ones that live in the repository. This approach maintains full traceability and integrates seamlessly with existing management workflows.

COO's Insights: AI Has Broken an Unspoken Rule of IT. And That’s a Problem We Haven’t Fully Understood Yet

Conclusions

Manual approach remains the irreplaceable human lens for exploratory sessions, usability critiques, and rapidly changing features where intuition leads the way. Automated scripts bring relentless speed and consistency to regression suites and performance checks, forming the backbone of continuous delivery pipelines. AI‑enhanced tools add a layer of intelligence: tests that heal themselves, adapt to shifting interfaces, and uncover subtle defects that static approaches overlook.

How to Ensure Application Compatibility When Modernizing a Web Application

Sergey Laptick — Thu, 30 Jul 2026 13:18:16 +0000

Application compatibility, in the context of software modernization, means many things. For instance, preserving seamless data exchange, ensuring API compatibility, and maintaining user experience (UX) continuity. Integrating a modern frontend with a legacy application that wasn't built for it or overhauling a UI without alienating users requires deliberate technical and human-centered strategies.

This article explores a dual-pronged strategy. First, we examine the architectural patterns that ensure technical integration. Second, we dive into UX methodologies that prevent user resistance.

Architectural Patterns for Technical Compatibility

When modernizing a web application, the greatest risk often lies in the backend. Legacy databases, outdated APIs, and codebases were not designed to interface with modern services, including cloud-based platforms. To ensure technical compatibility, developers turn to proven incremental patterns.

The Strangler Fig Pattern for Gradual Replacement

The Strangler Fig pattern (a term introduced by Martin Fowler) helps ensure app compatibility by enabling a controlled, piece-by-piece migration from a legacy system. Instead of replacing everything at once, you gradually substitute old components with new applications and services. Over time, the new application absorbs all the valuable functionality of the old one, allowing you to retire the original application completely.

How the pattern works:

Step 1. Set up a routing layer. Place a proxy (or facade) between client applications, the legacy system, and the new environment. Initially, the proxy directs nearly all traffic to the old system, keeping operations running without interruption;
Step 2. Shift incrementally. With each development cycle, you move more features to the new application. The proxy gradually reroutes requests away from the legacy app, steadily reducing its role;
Step 3. Retire the legacy system. Once every function has been migrated, the proxy sends all requests to the new system. You then remove the proxy, and clients connect directly to the modernized application.

When to use. Large, complex applications where a full-scale rewrite would be too risky. However, watch out for potential bottlenecks. The proxy must be designed for high availability and performance, as it becomes a critical traffic hub.

The Strangler Fig pattern fits naturally into any app modernization strategy because it lets teams progress at a sustainable speed. You keep your existing system productive while gradually introducing improvements, prioritizing high-value replacements first. This incremental method ensures compatibility, reduces disruption, and avoids the classic "big bang" failure where a single migration attempt breaks everything at once.

The Anti-Corruption Layer (ACL): Shielding the Modern System

While the Strangler Fig pattern handles incremental replacement, the Anti-Corruption Layer pattern addresses the semantic mismatch problem. Old applications often have convoluted data schemas, obsolete APIs (e.g., SOAP instead of REST), or business logic that doesn’t align with modern architecture and breaks compatibility. If your new application communicates directly with the legacy system, you risk "corrupting" your clean design with legacy quirks. The ACL acts as a translation layer between the two systems:

Translates Requests. Your modern application talks to the ACL using your clean, sensible API. The ACL performs all translation, transforming requests into the format the old system expects;
Shields Your Codebase. The ACL quarantines all legacy-specific logic. Your domain models, APIs, and services remain pristine and uninfluenced by outdated protocols;
Preserves Architectural Integrity. The old system continues to function, receiving the data it needs, while your new system maintains its modern technological approach.

When to use. The ACL approach is especially valuable when your modernization project requires app compatibility during gradual migrations where new features must interoperate with existing resources. It allows different subsystems with different semantics to communicate without either side compromising its design.

The ACL adds another service to manage and monitor, and it introduces some latency. But the trade-off is worth it. Your app modernization efforts remain clean, maintainable, and future-proof, and don’t become entangled in legacy complexity.

Business and Product Methodologies for UX Continuity

The way people work today has evolved far beyond what software from ten years ago was designed to support. Over time, old applications collect countless small inefficiencies, such as outdated workflows, clumsy workarounds, and interface quirks. Each of them adds to the daily burden on users. Yet moving from an old interface to a modernized one comes with its own challenge, including user resistance.

Even when a new design delivers clear improvements, people tend to push back against anything that disrupts their established habits.

The Incremental UX Redesign Strategy

For better application compatibility, rather than ripping off the bandage, smart modernization teams adopt an evolutionary UX approach. This involves updating interface elements section by section while preserving core architecture. You update navigation patterns in one module, gather feedback for two weeks, then move to the next section. This minimizes disruption and allows teams to validate impact and adjust iteratively.

Practical techniques for UX continuity include:

Progressive Disclosure. Introduce new features gradually, allowing users to adopt them at their own pace and not overwhelming them with a completely new interface;
Keep existing workflow structures intact. Don't alter the step-by-step processes your team relies on every day. Stick with terms users already know when those terms still carry the right meaning;
Provide a switch to move between old and new interfaces. Give people the option to flip back and forth from the original application to the updated version. This approach works especially well for business-to-business setups, where keeping operations running without interruption is paramount.

A fresh design will often be perceived as a worse design simply because it is new and breaks user habits. A better strategy is to play up familiarity and build on users’ existing knowledge of how an application works. This means avoiding complete overhauls in favor of modular, testable changes.

Role-Based Interfaces and AI-Embedded UX

UI/UX modernization that makes usual things smarter also helps to avoid rejection of the new. Modern ccompanies that provide app modernization services incorporate two major trends:

Role-Based Dashboards. Stop giving every employee the same dashboard. A manager needs different data than a floor worker. Context-aware design shows users only what they need for their specific job, reducing clutter and speeding up decision-making. Role-based design should also govern data access, ensuring that sensitive information is only available to authorized roles;
AI-Powered Interfaces. Features like predictive inputs, natural language processing, and intelligent data analysis can improve outdated workflows. Users can type or speak commands to retrieve data without complex queries. AI can suggest the next best actions based on patterns and past behavior, helping users stay on track with minimal input.

AI in app modernization helps automate code analysis and accelerate migration while also transforming the end-user experience.

XB Software’s Application Compatibility Modernization Approach

At XB Software, we recognize that legacy modernization services must address both technical integration and user adoption simultaneously. Over the years of delivering web app modernization projects, we have developed a systematic approach that preserves what works while evolving what doesn’t.

For instance, during SaaS API integration, we modernize outdated applications by integrating them with cloud-native SaaS platforms and APIs. Our approach includes designing multi-tenant architectures, implementing secure and scalable APIs, and automating updates. We maintain strict API compatibility through versioning strategies and consumer regression tests.

From a UI/UX perspective, rather than forcing users into an entirely unfamiliar interface overnight, we prioritize incremental, human-centered UX updates. A proven technique from our practice, demonstrated in the UI/UX modernization project, is the old/new UI toggle, that allows employees to switch between the old interface and the modernized version at will. This eliminates fear, builds confidence gradually, and results in zero downtime and minimal resistance.

Conclusions

Modernizing a web application while maintaining compatibility is a continuous balancing act between technical integrity and user trust. For organizations seeking legacy modernization solutions, the path forward is clear: they need to adopt a phased, modular approach that prioritizes compatibility at every layer. It’s also important to consider the human factor as seriously as the technical one, because an application that users refuse to adopt delivers no ROI.

React State Management Is No Longer About Redux vs Zustand

Sergey Laptick — Tue, 28 Jul 2026 11:09:20 +0000

For years, Redux shaped how React developers approached state management. A single global store became the default solution for everything, from business data to temporary UI state.

Today, however, the React ecosystem looks very different. Instead of relying on a single dominant library, developers can choose from Redux, Zustand, MobX, Jotai, XState, Valtio, and many other tools. So what changed?

Why React State Management Changed

React provides built-in tools for state management, such as useState, useReducer (since v16.8) , and Context (since v16.3). And, perhaps surprisingly, they are enough for many applications. Introducing an external React state management library where it isn't needed often adds complexity instead of solving problems.

However, once an application needs to:

Share data across many components;
Isolate business logic from the UI;
Prevent unnecessary re-renders;
Keep increasingly complex state predictable;

a dedicated state management solution becomes difficult to avoid.

The key realization is that not all states are the same. Different applications contain different types of state, each with its own lifecycle, complexity, and architectural requirements. This is exactly why the React ecosystem evolved from a single Redux-centric architecture to several specialized state management approaches.

Redux still solves many of these problems extremely well through a centralized store and an explicit data flow. It remains the standard architecture for many enterprise React applications. But it is no longer considered the universal answer for every kind of application state.

Think about these components:

Cross-widget dashboards with decoupled components;
Figma-style interactive canvases;
A multi-step approval workflow.

Should all of them be managed in exactly the same way? Would they really benefit from the same architecture?

This question fundamentally changed how developers think about React state management. Instead of searching for the "best" state management library, modern React applications start by identifying what kind of state they need to manage, and only then choose the approach that fits that problem.

This shift has transformed the ecosystem. Rather than competing over a single "best" solution, today's state management libraries are built around different philosophies of what application state is and how it should be managed.

Source: 2025 State of React survey

Four Ways to Think About State

Let's look at the four major state management philosophies in React and see which libraries represent each mental model.

Approach	How It Works	Examples
Centralized stores	Treat application state as a single, authoritative database	Redux/RTK, Zustand
Reactive systems	You mutate proxies directly, and the library handles derivation and propagation under the hood	MobX, Valtio
Atomic state management	Breaks the global store into independent particles	Jotai
State machines	UI exists in exactly one predefined state at a time, and transitions follow a defined graph	XState

The Philosophy of Centralized Stores: Redux and Zustand

The centralized store philosophy treats your application state as a single source of truth. Redux popularized this architecture in React by introducing explicit state updates through actions and reducers. Later, Zustand emerged as a lighter alternative built on the same philosophy but with far less ceremony.

Redux stores the entire application state in a single global store. Components never modify that state directly. Instead, they dispatch actions describing what happened, reducers calculate the next state, and selectors allow components to subscribe only to the slices of data they actually need.

Today, most projects use Redux Toolkit, which simplifies the original architecture with slices, Immer-powered immutable updates, and additional utilities such as RTK Query.

🎯Best for: enterprise applications that prioritize predictability, explicit data flow, and powerful debugging.

Zustand keeps the same centralized-store philosophy while removing much of Redux's ceremony. The store contains both the state itself and the functions for updating it. Components directly call the create function and update the state via the set function. The store hook returns both data and update functions. The result is a much smaller API, without reducers or action types, and without a dependency on React Context, which also helps avoid unnecessary re-renders.

🎯Best for: modern React apps that need shared global state without the architectural overhead of Redux.

The Philosophy of Reactive Systems: MobX and Valtio

Unlike centralized stores, reactive libraries automatically detect which pieces of state each component actually uses. Instead of manually writing selectors, developers mutate plain JavaScript objects while the library tracks dependencies and re-renders only the affected components.

MobX is the most well-known implementation of the reactive store philosophy. It uses observables that components subscribe to via the observer wrapper. Stores are typically written as JavaScript classes using makeAutoObservable, and state is updated via direct mutations. While the developer modifies the state, MobX automatically tracks dependencies in the background, making it particularly effective for applications with complex business logic and deeply interconnected domain models.

🎯Best for: data-intensive applications and teams familiar with object-oriented programming.

Valtio follows the same reactive approach, but without the object-oriented layer. It wraps plain JavaScript objects with ES6 Proxies, requiring developers to directly modify state (userStore.data = newData). Components subscribe through the useSnapshot hook and re-render only when the accessed property changes. With its small API and hook-oriented architecture, Valtio is one of the simplest proxy-based reactivity solutions.

🎯Best for: lightweight applications with frequently changing UI state, such as child panels, editors, and design tools.

The Philosophy of Atomic State Management: Jotai

Think of an application not as a single entity with a shared state, but as a collection of small, independent particles (atoms), each with its own state and the ability to freely combine with others. This is the philosophy of atomicity.

Jotai uses a bottom-up architecture, where primitive atoms (numbers, strings, objects, etc.) store values, and derived atoms read those values and derive new states from them. The resulting dependency graph is updated automatically whenever primitive atoms change. Components interact with atoms via useAtom, useAtomValue, or useSetAtom. Because atoms are independent, updating one atom does not affect components subscribed to another. This minimizes unnecessary re-renders without selectors or memoization.

🎯Best for: simple interfaces with a small number of external UI elements, such as dashboards and visual editors.

The Philosophy of State Machines: XState

State machines focus on application behavior rather than data storage. Instead of asking what data changed, they define which states are possible and which transitions are allowed.

XState, a JS library, models your application as a state machine. By creating a machine with createMachine, you define the initial state, all possible states, the events that trigger transitions, and any additional workflow context. In React, components are connected via the useMachine hook, which provides the current state along with a send function for dispatching events. By making each transition explicit, XState prevents incorrect UI states and significantly simplifies understanding complex workflows. For example: Draft → Review → Approved → Published.

🎯Best for: multi-step processes such as authentication, checkout flows, approval pipelines, and complex user flows.

Closing Thoughts

Modern React isn't moving away from Redux. It's moving away from the idea that one architecture should solve every state problem.

Centralized stores, reactive systems, atomic state, and state machines each solve different classes of problems. The right choice depends less on which library is trending and more on the type of state your application needs to manage.

Need	Good choice
Enterprise architecture	Redux
Lightweight global state	Zustand
Complex domain models	MobX
Proxy-based reactivity	Valtio
Granular UI state	Jotai
Multi-step workflows	XState

If you're integrating complex UI components such as SVAR React Gantt chart, you shouldn't have to redesign your application's architecture. That's why we've published integration guides for Redux, Zustand, MobX, Valtio, Jotai, and XState, so you can use the state management approach that already fits your project.

How We Automated a Production-Grade Traccar Deployment

Sergey Laptick — Mon, 20 Jul 2026 14:28:48 +0000

GPS tracking systems have become essential infrastructure for fleet management, logistics, delivery services, and field operations. Companies that manage vehicles, assets, or mobile workers need reliable visibility into where things are and where they have been.

Traccar is a mature open-source platform that handles such functionality well. The challenge is how to get it running in a production environment with monitoring, security, and the ability to customize it without spending weeks on setup.

We built an automated deployment script to solve this. We use it to deploy production-ready Traccar environments, then customize them according to our clients’ needs. This article explains how it works and what we deliver.

What Available Traccar Deployment Options Lack

Traccar is a system for tracking anything that moves: couriers, drivers, fleets, assets, etc. It includes a server, a frontend interface, and mobile applications that can function as GPS trackers themselves. The open-source version is available on GitHub, and anyone can download and try it. However, launching it and running it in production for years are two different things.

Most companies that need GPS tracking software do not want to spend time figuring out which database works best with Traccar, how to set up production environment monitoring so they know when something breaks, or how to configure a firewall to prevent data leaks. They want a system that works, stays running, and alerts them before it breaks down.

The default deployment options available today have gaps:

AWS Marketplace offers a basic Traccar instance at about $0.03 per hour. You get the software running, but the monitoring is whatever AWS provides by default. Limited styling, no tailored alerts, no ability to change how the system behaves;
Traccar's own hosted server for 50 devices costs around $49.95 per month in the US. It includes some branding options (logo, name and colors) with no UX or functional changes available. The server runs in Traccar's environment, not on the client's infrastructure. All data lives on Traccar's servers. For companies with a preference to keep their operational data on their own hardware, this is a non-starter;
Deploying Traccar yourself on your infrastructure from the open-source repository means your company owns and manages the entire stack. This option includes figuring out deployment, database configuration, reverse proxy setup, monitoring, alerting, and security. It is doable, but it takes time and expertise that most operations teams do not have to spare.

Here is how the available options compare and where we bring our expertise to the table:

What We Built: Automated Production Environment Deployment

When a client came to us needing a production-ready Traccar deployment without the usual headaches, we delivered it quickly and reliably. Here is what we built and how it works.

Instead of treating Traccar as a standalone application to be installed manually, XB Software built an automated deployment script that creates a complete production environment around it. The script runs on a Linux machine and handles everything needed to get Traccar running in a production-ready state.

Here is what the deployment includes:

Traccar backend and frontend;
PostgreSQL database;
Prometheus for metrics collection;
Alertmanager for sending notifications when things go wrong;
Nginx as a reverse proxy for monitoring interfaces;
Node exporter for Linux machine metrics;
PostgreSQL exporter for database metrics;
Firewall configuration using UFW/iptables to expose only necessary ports.

The script checks environment variables, installs Docker and Docker Compose, and generates configuration files from templates. It then sets up the firewall, clones the frontend repository (either standard Traccar frontend or a custom one), builds the stack, and starts everything. The logic strictly separates application, production environment monitoring, and data storage layers. When we update the Traccar interface or Alertmanager settings, the script performs smart, targeted restarts and rebuilds only the affected containers while leaving others untouched. This protects data from corruption, preserves active GPS tracker sessions, and maintains database uptime.

The deployment process with the script is fully idempotent. It never damages the system or creates duplicate resources. All original configuration templates remain on the server.

If a client needs to change monitoring passwords, update email alert settings, or modify any other configuration, there is no need for developers to edit complex files manually. They can update the corresponding variable in the centralized .env file and rerun the script. The infrastructure applies the changes without risking production stability.

For production-level reliability, we separated deployment logic from service management. Systemd handles stack availability, ensuring all containers restart automatically after a server reboot, while build and update tasks remain isolated within the deployment script.

System Architecture: Simple, Flexible, and Production-Capable

All the components listed above work together in a simple but production-capable architecture. Everything runs in Docker containers on a single server. For small to medium deployments this is more than sufficient. If tracking volume grows, the client can simply move the deployment to a more powerful server.

The Traccar backend and frontend are packaged in a single container. This is how Traccar's official Docker image works, and it keeps deployment straightforward. The frontend is written in React, and the backend runs on a custom Java server. The mobile applications (Traccar Client for drivers and Traccar Manager for fleet supervisors) are available from the official app stores.

For the database, we went with PostgreSQL. Traccar supports multiple databases, but PostgreSQL handles geospatial queries better than MySQL and manages high loads more effectively. For most fleets, the difference between databases is not dramatic, but PostgreSQL gives more headroom if tracking volume grows.

Prometheus and Alertmanager run in their own container. Prometheus scrapes data from two exporters. The Node exporter exposes such Linux machine metrics as CPU, memory, disk, network (these are the basic ones that can be extended if needed). The PostgreSQL exporter exposes database metrics. Together, they provide visibility into the health of the entire stack.

Nginx sits in front of everything. It routes traffic to Traccar, Prometheus, and Alertmanager. We added basic authentication to the monitoring interfaces so that metrics are not publicly accessible.

Why Traccar Integration Matters for Fleet Operations

Companies don’t buy tracking software and logistics solutions because they enjoy looking at maps. They buy them because they need to know where their vehicles are, how long drivers spend on routes, whether deliveries are on time, and whether assets are being used efficiently.

When the tracking system goes down, operations suffer. Dispatchers cannot assign jobs. Customers cannot get delivery updates. Managers cannot verify that routes were followed. The business loses visibility, and visibility is the whole point.

Our deployment approach addresses this directly.

Monitoring and Alerting: Knowing Before It Breaks

We built a deployment that not only works today but also tells you when it might stop working tomorrow. More vehicles get added. More trips get tracked. The database grows. Memory usage creeps up. CPU spikes during peak hours. Without monitoring, the first sign of trouble is often the system going down.

Our observability stack provides early warning. The alerts we configured track:

Host health: CPU, memory, disk usage, network activity;
Database health: PostgreSQL connections, query performance, storage;
Service availability: whether Traccar and its components are responding.

When metrics cross predefined thresholds, Alertmanager sends notifications. The client can see exactly what is happening on dashboards and plan upgrades before the system fails.

Security: Basic But Solid

Security follows the same principle as the rest of the stack: simple and effective. The script configures iptables (a traditional utility used to configure and manage IP packet filtering and NAT rules in the Linux kernel) to block all ports except those needed for the application to function.

This is essentially the same approach major cloud providers use with their security groups, just without the fancy UI. All internal traffic (database queries, exporter metrics collection) is fully isolated within Docker's private virtual networks and physically hidden from the outside world.

Customization: Frontend and Backend Modifications

A production environment also has to reflect the client's needs. One of the reasons companies choose Traccar is that they can customize it. In our experience, clients typically prioritize UX improvements. So we built flexibility into the deployment script accordingly. The repositories it uses are defined in environment variables. The frontend repo variable can point to the standard Traccar frontend on GitHub, a private repository with custom styles, or even a local directory.

This means we can:

Apply branding: replace the Traccar logo with the client's logo, adjust color palettes;
Modify UI elements: add, remove, or rearrange features in the frontend;
Update the frontend: pull new versions from the repository without changing the deployment script.

Most clients do not ask for backend changes. But for those who do need backend modifications (custom business logic, integrations with existing systems, specialized tracking features) we can handle those as well.

Support: Ongoing Maintenance, Adjustments, and Troubleshooting

Deploying the system is only the first step. When something drifts out of spec (memory spikes, database connection bloat, or unusual latency) we can diagnose it, and adjust the server or Traccar configuration before it becomes a problem.

Future Enhancements for Traccar Production Deployments

The deployment script is a foundation. As we work with more clients, we will refine the monitoring rules, expand the customization options, and improve the deployment process.

One area we are exploring is using AI-assisted development to accelerate frontend customization. Instead of manually modifying React components for each client, we could generate custom interfaces based on client requirements.

This is still in the exploration phase, but the potential is significant.

We are also considering how to handle larger deployments. Docker Swarm or Kubernetes would provide better scaling for fleet management software that works with hundreds or thousands of vehicles. The current script is a starting point, and we can build more complex architectures as client needs grow.

Conclusions

Traccar is a powerful GPS tracking system. But power without reliability is useless, and reliability without visibility is dangerous. Our automated deployment creates a production environment that combines Traccar's capabilities with the monitoring, security, and customization that businesses actually need.

Senior Developers Don’t Write More Code. They Prevent Disasters

Sergey Laptick — Thu, 09 Jul 2026 13:10:47 +0000

Imagine this. A team celebrates a massive release with dozens of features, thousands of commits, and months of late nights. Then, six months later, that same system is a maintenance nightmare. Bugs crop up everywhere. Onboarding new engineers takes weeks. Every small change breaks something else.

The problem is that many companies still measure senior engineers the same way they measure juniors: by output (more features delivered, more code in the repository, etc.). But that misses the real layer of seniority and results in production systems that fail from decisions that were never challenged.

Hidden Work of Senior Engineers

A senior developer’s day rarely looks like a productivity dashboard. They’re not constantly pushing commits or closing tickets. Instead, they’re doing things that look like they’re doing nothing:

Sitting in architecture discussions;
Mapping out failure modes;
Pushing back on vague requirements;
Asking “why” five times before anyone writes a spec.

None of this “ships.” But everything becomes safer because of it.

What actually separates a mid-level from a senior engineer in 2026 isn’t syntax mastery or knowing the latest build tool. It’s strategic thinking, technical influence, and the ability to balance trade-offs between speed, scalability, and maintainability. You can teach someone React in a week. Teaching them to see around corners takes years.

This invisible work is why senior engineers prevent problems that junior engineers cannot yet see:

Scaling bottlenecks before they exist;
Data consistency issues that only appear under load;
Dependency risks in third-party integrations;
Over-engineering disguised as “future-proofing”;
Under-engineering disguised as “MVP speed”.

A production outage is rarely a surprise. It’s a delayed decision that a senior engineer might have spotted six months earlier if anyone had listened.

The Paradox of Deleting Code or Why More Code Isn't Always Better

Here’s a counterintuitive truth that keeps coming up in engineering circles: your value as a developer becomes inversely proportional to the amount of code you write. Here’s how this so-called Deletion Paradox works. Junior engineers are busy adding features while senior engineers are quietly removing thousands of lines that never should have existed.

Why does that matter? Because every line of code is a future maintenance liability. It adds to the cognitive load, increases the surface area for bugs, and creates hidden coupling that will snap under pressure.

In mature systems, more code rarely means more value. It often means more operational complexity, more maintenance overhead, and more places where things can go wrong. The best way to improve a system is often to remove code, because you understand what happens after it ships.

Outcome Vs. Output: The Real Measuring Stick

This is where the shift from output to outcomes becomes concrete:

Output is what your team produces: a shipped feature, a release, a line of code. It’s easy to count, easy to celebrate, and easy to fake as progress;
Outcome is the measurable change in user behavior that creates business value: higher activation, better retention, more expansion revenue.

There’s this concept of “Feature Factories”, describing a situation where teams that celebrate activity and features delivered instead of business value and changed system behavior. The issue is that building a feature without an expected outcome is just a guess shipped at full cost.

Senior developers look at the backlog through the lens of hypotheses rather than blind commitments. They ask: “What are we trying to learn?” “What will change if this works?” “What’s the metric that moves if we get this right?”

This is why their most valuable contribution is often saying “don’t build this yet.” Not because they’re lazy or risk-averse, but because they can see that a feature will create long-term maintenance cost, or that an abstraction will break under real usage patterns, or that a shortcut will become technical debt faster than anyone expects.

At XB Software, we’ve seen this play out repeatedly in legacy app modernization projects. Sometimes the most valuable thing you can do is to know which parts of an old system to preserve and which to retire.

A UI redesign that strips familiar workflows can break user habits that took years to build. A senior engineer’s job is to see that coming and push back before anyone writes a line of code. That’s why we emphasize careful planning. The absence of a bad redesign often delivers more value than a dozen new screens.

What Changes When AI Generates Half Your Code

The Sonar 2026 State of Code Developer Survey found that 42% of production code is now AI-generated or assisted. 72% of developers who’ve tried AI coding tools now use them every day. That changes what “coding” means.

Anyone can generate massive amounts of code now. However, the hard part is knowing what not to generate, what to delete, and what to constrain.

As the VibeCoder Engineering Blog points out, a senior developer’s value in the AI era is building the guardrails. They write .cursorrules or CLAUDE.md files that encode project conventions, architecture patterns, and forbidden anti-patterns. The AI then generates code that follows their standards instead of generic best practices.

This is a form of systemic decision-making that happens before AI-generated code exists. AI cannot reliably make engineering trade-offs under business constraints. It doesn’t know when to choose simplicity over software scalability, or when to exchange speed for correctness. Those are still human judgments, and they’re exactly what senior engineers get paid for.

So while AI makes code cheaper, it makes engineering decision-making more valuable. The bottleneck here is editing, deleting, and deciding what not to build. The senior engineer’s job has shifted from writing code to writing rules that constrain the machines.

So What Does Seniority Actually Look Like?

Junior engineers optimize for “correct implementation.” They want to build the thing right. Senior engineers optimize for “sustainable system behavior.” They want to build the right thing, and sometimes that means not building anything at all.

The most important contribution a senior engineer makes is often invisible: shaping the system before it becomes expensive to fix. That’s the work that doesn’t show up in commit logs. But it’s the work that separates a stable product from a sinking one.

Senior developers don’t “do more.” They make sure the system doesn’t accumulate problems faster than the team can understand them. And in production, that is what defines success.

AI Can Write Code, But It Can't Own Production Systems

Sergey Laptick — Thu, 02 Jul 2026 13:36:27 +0000

AI is getting scary good at writing code. Drop a prompt into Lovable or Claude Code, and minutes later you have a working prototype with a modern tech stack. It feels like we solved all big problems in AI-assisted software development.

Then production happens. And production does not care how clean your code looks. It cares whether it survives reality. Can it handle traffic spikes or recover when a dependency fails at 2 AM? Can a person who inherits it six months from now actually understand what it does?

The uncomfortable truth is AI can generate code at scale, but it cannot own what happens after you deploy it.

The Illusion: "Perfect" Code from AI

What AI actually produces is impressive. Clean structure. Readable functions. Fast prototypes. Even decent architecture suggestions if you prompt it right. From the outside, it looks like senior-level output.

But here is the thing. AI does not understand business risk. It does not know what happens when your database connection pool runs out under load. It cannot tell you which of two equally valid solutions will survive a 10x traffic spike. It generates what looks right based on what it has seen before, not what is right for your specific system with its specific constraints.

Andy Anderson, a researcher at IBM, spent four months building a Kubernetes dashboard from scratch using Claude Code. No team. Just him and the AI. The first two weeks were exhilarating. Code poured out at a pace he had never experienced. Features that would normally take days appeared in hours. It felt like having a tireless junior developer who typed at the speed of thought.

Then the limitations hit. All at once: broken builds, wrong architectural patterns, scope creep, the AI trying to modify files Andy did not ask it to touch, etc. The problems were cascading: fix one thing, three others break. The researcher ended up spending more time reviewing and reverting than he would have spent writing the code himself.

We saw the same pattern with a scheduling app we built recently. Lovable generated a working prototype in hours with React frontend, Node.js backend, and PostgreSQL database. The calendar looked fine in the demo. But when we tested it, the AI-generated scheduler could not handle recurring events or drag-and-drop reliably. It was a concept car made of clay. So we swapped that component for DHTMLX Scheduler, a production-grade library, and rebuilt the backend for real-world scale. The prototype was useful, but the production system needed actual engineering.

This pattern shows up everywhere. The tools are great at generating code but terrible at understanding what the code is for.

Where Things Break: Production Reality

In production, systems behave differently. Traffic is unpredictable. Dependencies fail. Data grows. Users do things you never expected. Integrations become fragile. And suddenly, that “perfect” AI-generated code becomes just one layer in a very messy environment. The problem is not the code but everything around it

A large-scale study analyzing 302,600 verified AI-authored commits across 6,299 GitHub repositories found something sobering. AI-generated code introduces real issues, such as code smells, correctness problems, or security vulnerabilities, and 22.7% of those issues survive long-term, silently accumulating as technical debt in production codebases. More than 15% of commits from every AI coding assistant introduce at least one issue. Code that looks fine in a PR can quietly rot in production for years.

Stéphane Maes, who writes about software engineering, calls this the “Great Toil Shift.” The time you save generating code gets entirely consumed by the downstream work. This includes architectural review, security auditing, code understanding, documentation, and ongoing maintenance. The efficiency gains are not what they seem.

This is a common thing for teams that go all in on AI-generated code. The prototypes ship fast. Everyone is excited. Then the first production incident hits. The AI-generated code does something unexpected under load. Nobody fully understands the system because nobody wrote it. The team spends days debugging something that would have taken hours if they had built it themselves.

The Missing Piece: Engineering Judgment

Here is where the conversation needs to shift. What AI does is options generation. It gives you suggestions, often good ones. But someone still has to decide which suggestion to take. Someone still has to understand the trade-offs.

And they are the hard part:

Architecture trade-offs: Do we use a message queue or direct calls?
Failure handling: What happens when this service goes down?
Performance constraints: How many requests per second can this handle?
Security decisions: Is this endpoint properly authenticated?
Long-term maintainability: Will someone understand this in two years?

This is where experience matters more than output. Two solutions can both "work" in AI terms. But only one survives production load.

Andy Anderson's central finding from his experience report is worth quoting directly: "The intelligence of an AI-driven development system resides not in the AI model itself, but in the infrastructure of instructions, tests, metrics, and feedback loops that surround it."

In other words, AI is just a tool. The real system is everything you build around it. The tests. The review process. The deployment pipeline. The monitoring. The on-call rotation. The institutional knowledge about why things are the way they are.

In production environments, organizations fundamentally need what they call a "throat to choke," a human engineer who holds ultimate accountability, who can assess blame, who possesses the systemic intuition to troubleshoot complex, real-world failures. A machine cannot sign off on a Service Level Agreement (SLA). It cannot face a compliance audit. It cannot absorb liability for downtime.

This is not a philosophical point. Researchers who analyzed the terms of service for nine AI coding assistants found a consistent pattern: while users own the generated output, full liability, responsibility for correctness, and downstream production risk rests on the human user. The providers explicitly disclaim warranties and allocate responsibility to the developer.

If your AI-generated code causes a production outage, the AI company is not getting the pager duty alert. You are.

What This Means for How We Build Software

None of this means the tools are useless. They are incredibly useful. They can generate boilerplate, write tests, refactor code, and help you explore solutions faster than ever before. But the tools do not replace judgment. They do not eliminate the need for testing. They do not make architecture decisions for you. And they definitely do not own the systems they help you build.

The real skill in modern AI-assisted development is learning how to integrate generated code into a system that you fully understand, can debug, and evolve over time. It is knowing when to accept a suggestion and when to rewrite it.

Some teams are figuring this out. They use AI for rapid prototyping, like the Lovable-generated scheduler that we built, which validated the concept before replacing the AI-generated components with a production-ready solution. The prototype was useful for testing and validation. But the production system required real engineering: understanding the domain, handling edge cases, and ensuring the component could actually survive real-world use.

AI changes how fast we build software. But it has not changed what it means to be responsible for it. And in production, responsibility is still the hardest part.

How to Build a Local LLM Agent to Automate Work List Generation from Monthly Reports (With Jira Integration)

Sergey Laptick — Thu, 21 May 2026 11:39:00 +0000

Our management team spent hours manually extracting work items (“bug fix”, “released version 1”, etc.) from dozens of developer reports. The task was repetitive, error‑prone, and a security risk when using cloud‑based AI tools, since it means exposing internal activity to external servers.

To solve this, we built a local LLM‑powered agent that runs entirely on our own servers, normalizes chaotic report data, filters out useless noise, enriches descriptions from Jira, and generates a clean list of actual accomplishments. In this article, we break down the architecture and explain why a CPU‑only, on‑premise approach is practical for enterprise clients who prioritize data privacy.

The Problem: Manual Work List Generation Is Slow, Inconsistent, and Insecure

Usually, our managers followed the same routine: collect a month’s worth of developer reports, manually scan through hundreds of entries, and pick out the items that actually represented completed work. This process was straightforward but flawed.

The first issue was data quality. Developers write reports in wildly different formats. Some include detailed Jira ticket IDs and descriptions, others are cryptic one‑liners like “fixed issue”. When a manager who wasn’t deeply involved in the project later reviews these reports, the meaning is often lost. What does “adjusted header” refer to? Which feature did “refactored code” touch? What we really needed was an AI-powered task management approach that could process this unstructured data automatically.

The second issue was duplicate work. Managers would occasionally include tasks that had already been declared in previous months, creating overlaps. Another example is a task that spans several days. In this case, the same activity could be logged repeatedly, producing many near-identical entries. There was no automated way to compare new reports against historical data.

The third issue was security. Initially, we experimented with feeding entire monthly reports into ChatGPT, asking it to clean up the data and suggest a final list. It worked reasonably well, but we were handing over a full month of internal project activity to a cloud service. For many enterprise businesses, especially those in finance or healthcare, that level of exposure is unacceptable.

The Solution: A Secure, On‑Premise AI Agent for Task Extraction from Reports

Our approach was to implement a console‑based application that converts reports into tasks automatically. It runs on our internal server, triggered by a cron job (or an optional API call) at the end of each monthly reporting cycle. The AI agent processes raw reports for each active project, applies a series of transformations, and outputs a polished list of work items.

The entire pipeline runs on a CPU‑only server using Ollama to serve a local instance of the Gemma 4 E2B model. For embedding generation (used in duplicate detection), we use the tiny nomic‑embed‑text model, which is only a few megabytes in size. Here’s a high‑level view of the process flow:

Let’s walk through each stage in detail.

1. Normalization: Making Chaos Readable

A single project might receive 80+ individual reports per month with varying levels of detail. The first task for our AI agent was to normalize these disparate inputs into a consistent, machine‑readable format. This step alone turns a jumble of free‑form text into structured data that the rest of the pipeline can reliably process.

2. Chunking: Working Within Token Limits

This is where we hit our first major technical constraint. Running on CPU via Ollama, our Gemma 4 model is limited to a context window of 4,096 tokens. That’s not a lot. A single month of reports from a busy project can easily exceed that.

We solved this by chunking. The AI system calculates the approximate token count of the combined report text and splits it into batches of about 20 reports each. This ensures that the LLM never runs out of context space and that each chunk receives full attention.

Within each chunk, we also further split entries that contain multiple tasks in a single line (e.g., “Did A, did B, did C”). After this splitting, 22 raw reports became 94 individual work items in one of our test runs.

3. Jira Enrichment: Adding Missing Context

One of the most valuable features of our AI agent is its ability to automatically fetch additional context from Jira. When the system detects a Jira ticket ID in a report, it calls the Jira API to retrieve the ticket description.

Developers often write terse reports assuming the ticket ID is enough. But when that report later appears as “AAA‑123 – done”, it tells nothing. By pulling the full, manager‑written description from Jira, our AI agent replaces the vague entry with a clear, professional summary of what was actually accomplished.

4. Filtering Out the Noise

Not every report entry is worth including. Generic statements like “working on…” or “following up” don’t convey meaningful work. We built a bad‑word filter, one of key components of our intelligent document processing (IDP) pipeline. It flags entries containing these vague phrases.

The LLM processes each chunk and identifies data that match our exclusion list. In our test, this filter removed 69.1% of entries and only 29 items out of 94 survived the cut. What remained were concrete, specific descriptions of completed tasks.

5. Selecting the Best Candidates

Once we have a clean set of candidates, we need to choose the top N entries to present. The number N varies by project and is stored in our internal reporting database. To account for further filtering in the next step, we typically select a larger pool, say, 80 items.

6. Vector Duplicate Detection: Ensuring We Never Repeat Ourselves

This is the secret sauce that prevents duplicate entries. Before finalizing the list, the AI agent compares each candidate against a historical database of all work items we’ve ever submitted for that project. Here’s how it works:

Embedding Generation. Each work item is converted into a vector (a list of numbers) using the nomic‑embed‑text model. This vector captures the semantic meaning of the text;
Similarity Calculation. The system compares the new candidate’s vector against the vectors of all previously stored data for that project;
Threshold Decision. If the similarity score exceeds 0.85 (85%), the candidate is flagged as a duplicate and removed. This threshold catches not just exact matches but also near‑duplicates where the phrasing or word order has changed while the underlying idea remains the same.

The historical data is stored in a lightweight PostgreSQL table with just a few fields: project_id, text (the final description), embedding (the vector), and created_at (date of creation).

After duplicate removal, we’re left with a set of truly unique, high‑quality work items. These are then formatted for final delivery to the project manager.

Real‑World Performance: What Test Run Tells Us

Let’s walk through an actual test run to see the numbers in action. These test run results demonstrate how an AI report analysis tool can summarize reports into tasks even with noisy, inconsistent input.

Technical Deep Dive: Why CPU‑Only Deployment Works

One of the most common objections to running local LLMs is the perceived need for expensive GPU hardware. We deliberately chose a CPU‑only deployment to keep costs manageable and to prove that on‑premise AI doesn’t require significant infrastructure investments.

Model Selection: Gemma 4 E2B

We evaluated several local models and settled on Gemma 4 E2B. Here’s why:

Size. At 5 billion parameters, it fits comfortably in RAM without needing a GPU. Our server has extra memory allocated specifically for the model;
Performance. It’s fast enough for batch processing;
Quality. The model handles JSON output reliably, and follows detailed prompts with minimal hallucination.

NOTE: If you work with a multilingual team, make sure that the model you use understands target languages natively.

Proper Model Settings and Prompt Engineering for Consistency

Each pipeline stage has its own carefully crafted prompt that includes:

A clear role definition (e.g., “You are a specialized Data Parsing Engine”);
Good examples and bad examples of expected output;
Explicit formatting rules (JSON structure, field names);
Instructions to avoid creativity (temperature set to 0).

For the bad‑word filter, we provide a list of prohibited terms and their synonyms: “working on,” “following up,” “in progress,” “discussed,” etc. The LLM simply acts as a pattern matcher with semantic understanding. It can recognize that “still working on the header” is conceptually similar to “in progress” and flag it accordingly.

Also, for data‑processing tasks like this, we always disable “thinking” or “chain‑of‑thought” modes. Those are useful for complex reasoning but introduce unnecessary variability and output length in structured extraction tasks.

Extra Challenges We Overcame

Challenge 1: LLM Unpredictability. Even with temperature set to 0, LLMs can occasionally produce unexpected output. We added timeout limits to prevent the model from getting stuck in a loop, and we structured our prompts to request strictly formatted JSON that is easy to validate programmatically.All LLMs glitch. We can never fully trust them. They help, but everything still needs to be reviewed and rephrased. They assist, but they don’t solve the task 100%.

Challenge 2: CPU Processing Speed. Processing 94 items across multiple LLM calls takes time. We solved this by running the AI agent as an overnight cron job, so speed is never a bottleneck. The manager arrives in the morning to a ready‑to‑review list.

Why This Approach Matters for Enterprise Clients

1. Complete Data Sovereignty

When you use on-premise Artificial Intelligence solutions, no data ever leaves your infrastructure. The LLM runs locally, the embedding model runs locally, and the historical database resides on your own PostgreSQL server.

2. No Vendor Lock‑In

Cloud AI services change their pricing, deprecate models, or alter their APIs without notice. By using local AI agents and Ollama, you retain full control over the entire stack. Need to switch to a different model tomorrow? Just pull a new one and update the configuration.

3. Predictable Costs

The only ongoing cost is the electricity to run the server. There are no per‑token API fees, no monthly subscriptions, and no surprise bills after a particularly busy month of processing. For organizations that process thousands of reports annually, the savings are substantial.

4. Customizable to Your Workflow

Because we own the code, we can adapt the pipeline to fit your specific reporting format, integrate with your existing project management tools, and fine‑tune the prompts to match your industry’s terminology. This enables using AI for business process automation across diverse sectors, from construction to healthcare.

From Manual Chore to Automated Precision

Before, turning chaotic developer notes into clean reports meant choosing between tedious manual work or exposing sensitive data to cloud AI. Our private AI agent for document analysis offers a third way. Namely, secure, on‑premise automation.

By combining Gemma 4 on standard CPU hardware with vector‑based duplicate detection and direct Jira enrichment, we’ve turned hours of monthly review into a hands‑off process. The system normalizes vague entries, filters out noise, and guarantees you never repeat a task description.

How Spec-Driven Development Brings Structure to AI-Assisted Engineering and How We Put It to the Test

Sergey Laptick — Mon, 02 Mar 2026 13:13:18 +0000

AI coding assistants have made developers incredibly fast since the start of the AI boom, but this new speed often comes at a hidden cost. The IT industry is realizing that generating code is the easy part. The real challenge is building systems that are coherent, maintainable, and actually do what they were supposed to do.

This is where Spec-Driven Development (SDD) comes in. This methodology shifts the focus from vibe coding to following the general intent, using specifications as the new source of truth for AI-assisted engineering.

Why Vibe Coding Isn’t Built to Last

The term “vibe coding” perfectly captures the current experimental phase of AI-assisted development. You describe what you want, get a block of code back, and if it looks right and seems to work, you move on. This approach is undeniably powerful for prototypes and small scripts, allowing for unprecedented velocity. However, when applied to serious, mission-critical applications, the cracks begin to show:

The code might compile and even function, but the underlying architecture becomes an afterthought;
New features create unexpected conflicts;
Documentation is sparse or non-existent;
The codebase transforms into a collection of disjointed components that are hard to maintain, debug, and evolve.

The problem isn’t the AI’s coding ability. It’s rather the workflow where developers treat AI like a search engine when they should be treating it like a literal-minded, but exceptionally talented, pair programmer who needs unambiguous instructions.

What Is Spec-Driven Development

Spec-Driven Development is the practice of writing clear, structured, and testable specifications before a single line of code is generated. In the context of AI-assisted development, SDD provides the blueprint that guides AI agents to generate code that is consistent, architecturally sound, and perfectly aligned with business goals.

Unlike traditional waterfall requirements that gather dust, an SDD spec is a living, executable artifact. It becomes the shared source of truth for both humans and AI, driving development, software testing, validation, and even documentation. By moving architectural decisions, constraints, and clarity upstream, SDD directly addresses the shortcomings of vibe coding. It replaces guesswork with a clear contract for how your app should behave.

How Spec-Driven Development Works: A Step-by-Step Guide

The SDD process is structured into distinct, sequential phases. Each phase produces a key artifact that feeds into the next, ensuring a clear, traceable path from a high-level idea to production-ready code.

Specify (The “What” and “Why”). You start with a high-level description of what you’re building and why. The AI then generates a detailed functional specification. This phase is purely about business intent: user journeys, success criteria, and edge cases. It explicitly excludes technical details, forcing clarity on the problem before jumping to a solution;
Plan (The “How”). With the functional spec locked in, you provide the AI with your desired stack, architecture, and constraints. The AI generates a comprehensive technical plan, including technology choices, system design, integration patterns, and security considerations. This ensures the new code feels native to your project and aligns with your technical strategy;
Tasks (The Breakdown). The AI takes the spec and plan and breaks them down into small, reviewable, and actionable tasks. Each task is specific enough to be implemented and tested in isolation, like “create a user registration endpoint that validates email format.” This decomposition prevents the “big bang” coding approach that overwhelms both AI and reviewers;
Implement (The Execution). Finally, the AI tackles the tasks one by one. Instead of reviewing thousand-line code dumps, developers review focused changes that solve specific problems, verifying that the implementation matches the specification.

A Quick Look at the Tools Enabling SDD

The SDD ecosystem is maturing rapidly. While GitHub’s Spec Kit is a powerful open-source example, other platforms offer different interpretations of the model:

Spec Kit

An open-source CLI and template-based toolkit that integrates with your existing AI assistants like Copilot, Claude Code, and Gemini CLI. It introduces the concept of a constitution.md, a file that encodes your project’s immutable principles, such as stack versions, naming conventions, and architectural patterns.

Kiro

An agentic AI with an IDE and CLI that add structure to an existing editor. Kiro has SDD built directly into its core. When starting a new feature, its agents automatically generate requirements, design documents, and create task lists, guiding the developer through an opinionated workflow. It’s designed for developers who want a deeply integrated, automated, and context-aware environment for moving from concept to code.

BMAD Method

An open-source framework that simulates an entire agile team using specialized AI agents. With over 12 distinct agent personas, including a Product Manager, Architect, Scrum Master, etc., it manages the entire project lifecycle.

Putting Spec Kit to the Test

Theory is one thing, but practice is where the real lessons are learned. Our team recently dove into GitHub’s Spec Kit to understand its practical applications and limitations. Here’s what we found trying to recreate the app shown below. It’s a small fleet management dashboard with a scheduler, a map, and vehicle tables:

How Spec Kit Works (main commands)

/constitution:
- When to use: At the very beginning of a project.
- Purpose: Establishes your project’s foundational rules, defining the tech stack, architectural patterns, and coding conventions that AI-generated code must follow.
/specify:
- When to use: After the constitution is set.
- Purpose: Takes your high-level, plain-language description of a feature (the “what” and “why”) and expands it into a detailed functional specification.
/plan:
- When to use: After the specification is reviewed and approved.
- Purpose: Generates a technical implementation plan based on the spec and the project’s constitution, defines frameworks, libraries, etc.
/tasks:
- When to use: After the plan is finalized.
- Purpose: Breaks down the specification and plan into a list of small, concrete, and actionable tasks. Each task is designed to be implemented and tested independently.
/implement:
- When to use: After the tasks are defined.
- Purpose: Instructs the AI agent to start writing the actual code, working through the generated task list one by one.

Our web development team started by testing the limits of the process. The first attempt was to generate a complete demo application, in one go. The high-level description was fed into the /specify command, and the results quickly revealed the AI’s contextual limitations. The application generated by Spec Kit was a mess: the scheduler rendered strangely, filters didn’t work, and the statistics pulled random data not connected to the users or vehicles:

This shows that feeding too much information at once simply overwhelms the model. It gets confused and starts losing track of early requirements halfway through implementation. You shouldn’t feed a neural network a very large amount of data at once because it will get confused

The failure illustrated a core principle of SDD: decomposition is not optional. The AI’s context window, while large, has finite capacity. Handling a complex, multi-page application in one go leads to forgotten requirements and inconsistent results.

A better strategy is to build the project feature by feature. On the second attempt, our team started with the core layout (the header and collapsible menu) with a highly detailed spec that included exact styles and components:

Next came the “vehicles” page, specified down to the placement of inputs, instructing the AI to match the layout of a provided demo. While it wasn’t perfect, and the AI still decided to paint a button a different color, it was a manageable, high-quality chunk of work that could be easily corrected with a follow-up prompt or a quick manual tweak.

The final step was a tiny, well-scoped feature: adding sorting to two columns in an existing table. The simple command create client-sorting for type and year columns on Vehicle Stable proved perfectly suited for the SDD workflow, demonstrating that the methodology’s value extends to changes of any size, ensuring even small updates are implemented correctly and consistently:

Our experiments with Spec Kit revealed that successful AI-assisted development hinges on proper decomposition. Attempting to generate an entire application in one pass overwhelmed the model and produced unusable results, while breaking the project into small, well-scoped features consistently delivered high-quality, reviewable code. The key insight: specifications must match the size of the task, ensuring AI can maintain focus and consistency throughout implementation.

When SDD Makes the Most Sense

It is not a silver bullet for every coding task, but it provides immense value in specific scenarios:

Enterprise & Production Systems. For long-lived applications where maintainability, stability, and compliance are critical;
Complex Architectures. For projects with multiple services, APIs, and integration points where a lack of clarity can lead to catastrophic failure;
Team Development. When multiple developers (and AI agents) need to collaborate on a shared codebase, a central source of truth is invaluable;
Legacy Modernization. When rebuilding an old system, you can use SDD to capture the essential business logic in a modern spec before letting AI regenerate a clean, new implementation.

Conclusion: From Code-Centric to Spec-Centric Thinking

We are moving from an era where “code is the source of truth” to one where “intent is the source of truth.” AI is making specifications executable, turning our documented intent directly into working software. Spec-Driven Development allows small teams to build robust systems and large organizations to move with coherence and speed.

How AI Search Solves the Problem of Working with Unstructured Data

Sergey Laptick — Mon, 27 Oct 2025 11:57:03 +0000

Are you struggling with unstructured data, like support tickets, employee feedback, and documents? Many businesses face this challenge, leading to wasted time and missed insights. Unstructured datasets contain up to 90% of all enterprise-generated data, yet most systems are optimized for structured, field-based records. AI-powered search can interpret intent and context, find conceptually similar content, and improve results over time based on user behavior. Today, we’ll explore how AI search can transform the way you interact with data.

What’s Unstructured Data and Why It’s Hard to Work With It

Unstructured data refers to any information that doesn’t have a predefined format and does not conform to fixed schemas of databases. Common examples in enterprise environments include:

Text-heavy documents. Emails, meeting notes, support tickets, contracts, and project briefs;
Files and reports. PDFs, Word documents, PowerPoint presentations, and scanned forms;
Multimedia content. Customer call recordings, interview videos, and voice memos;
Collaboration tools. Slack/Teams messages, comment threads, and internal wikis.

Unlike structured data, such as a CRM system’s "Customer Name" field, unstructured content doesn’t have fixed labels. As a result, traditional database queries can’t easily retrieve it, which leads to multiple difficulties:

Keyword Search Falls Short. Users must guess the exact phrasing (e.g., searching for "order delay" won’t surface an email saying "shipment stuck in customs");
No Consistent Formatting. A contract clause might appear in a PDF, an email attachment, or a scanned handwritten note, each requiring different extraction methods;
Lack of Metadata and Context. Without AI, a customer complaint buried in a support ticket won’t be tagged unless manually labeled;
Scalability Issues. Manual categorization and tagging become slow at large scale. Imagine HR reviewing hundreds of resumes, or legal teams parsing years of contracts.

When employees can’t quickly find the information they need, businesses suffer:

Slower decision-making. Sales reps waste time digging for client details instead of closing deals;
Increased errors. Misinterpreted notes or overlooked contract terms lead to compliance risks;
Frustration and inefficiency. 30% of an employee’s time is spent searching for information;
Missed opportunities. Hidden insights in customer feedback or employee surveys go unused.

AI to the Rescue: Solving Issues with Unstructured Data

When traditional search methods become increasingly ineffective, AI-powered search changes the game. It can understand context, intent, and relationships within data. Let’s explore how it works and why it outperforms conventional approaches.

How AI Search Processes Unstructured Data

AI-driven search relies on several advanced technologies to make sense of unstructured content:

Natural Language Processing helps understand user intent. For example, a user's query "Find customer complaints about late deliveries" retrieves emails, support tickets, and notes even without exact keywords;
Vector Embeddings and Semantic Search. An AI-based system converts text from unstructured datasets into numerical vectors, which allows capturing meaning rather than just rely on word matching. AI search enables similarity-based retrieval to find documents with related ideas, even if phrasing differs ("order delay" vs. "shipment backlog");
Machine Learning for Continuous Improvement. AI software can learn from user interactions to prioritize relevant results over time. It automatically tags and categorizes documents, for example, to group contracts by clause type;
Multimodal Processing. AI software enables extracting text from images, PDFs, and scanned docs (OCR) and transcribe and index audio/video files for searchability.

Transforming Business Outcomes with Server-Side AI Search

Integrating a semantic search engine on the server side allows extracting value from unstructured data. By moving beyond literal keyword matching to a deeper conceptual understanding, businesses can convert unstructured text into a dynamic, intelligent asset that drives both growth and operational efficiency.

The real estate sector offers a compelling example of this transformation. Let’s see how a modern property platform uses server-side AI semantic search to make property discovery more intuitive and aligned with user intent.

From Query to Match: How Our AI Search Validates Results

Vector-Based Matching

Converts user queries and property descriptions into mathematical vectors;
Finds conceptual matches even without exact keyword overlap;
Example: Searching "education" returns properties mentioning "family seeking home near top-rated schools" and "Investor purchasing a property for student housing".

Intelligent Threshold Filtering

Applies similarity threshold to ensure relevant results;
Automatically excludes matches below confidence level;
Example: Query "quiet neighborhood" triggers matches for:"Retired teacher moving to a quiet suburb" (84% match)"Empty nester looking for a quiet neighborhood with walking trails" (77% match)"Single parent looking for a safe neighborhood and good schools" (76% match)

Contextual Understanding

Interprets related concepts through embedding relationships;
Recognizes "kids" → family amenities, playgrounds, safety features;
Identifies "investment potential" → rental yields, appreciation areas.

How Semantic Search Powers Property Discovery

This AI-powered property search directly addresses critical pain points for real estate businesses:

Capturing Buyer Intent Beyond Keywords. Many homebuyers use vague phrases like "good schools" or "quiet area" that traditional searches miss. The AI solution matches "family-friendly neighborhood" to listings mentioning "top-rated elementary school" or "playground access" even when exact terms differ. Business Impact: 30% more relevant leads by understanding implicit requirements;
Reducing Missed Opportunities. Valuable properties often buried due to keyword mismatches (e.g., "serene" vs "quiet"). AI solution surfaces conceptually aligned listings, which increases agent productivity by eliminating manual matchmaking;
Converting Ambiguous Queries. Another problem with traditional systems is when investors search for things like "high-growth areas" but listings describe "appreciation hotspots" or "development zones". Vector relationships can help by connecting investment terms across descriptions, leading to faster deal flow through precision matching.

AI Search Query Examples Across Industries

Customer Support

AI search uncovers recurring support issues by recognizing patterns across various channels, regardless of how customers describe their problems. This enables faster troubleshooting and better product insights.

Query: "Find tickets mentioning API integration issues"

What AI Search Does:

Retrieves support tickets, forum posts, and chat logs;
Understands related terms like "API connection errors," "webhook failures," or "authentication problems";
Groups similar cases for trend analysis.

Traditional Search Limitation: Fails if the ticket says "system won’t connect" without the exact term "API."

Legal Department

Legal teams save time reviewing contracts and compliance docs by having AI identify relevant clauses and terms, even in PDFs or scanned documents. No need to manually tag or search exact legal phrasing.

Query: "Show contracts discussing 2024 delivery terms"

What AI Search Does:

Analyzes PDFs, scanned agreements, and email attachments;
Identifies clauses about "delivery timelines," "shipment deadlines," or "FOB terms";
Filters by year without manual metadata.

Traditional Search Limitation: Can’t parse handwritten edits or complex legal phrasing without exact matches.

Sales & Marketing

Sales and marketing teams can get a clearer view of customer sentiment around campaigns and launches by analyzing unstructured feedback in real time. AI search connects surveys, emails, and social mentions to reveal trends and pain points.

Query: "Find customer feedback about the new service launch in April"

What AI Search Does:

Aggregates survey responses, social media mentions, and support tickets;
Detects sentiment (e.g., "loved the update" vs. "had issues with the rollout");
Focuses on April-timeframe discussions, even if undated (using contextual clues).

Traditional Search Limitation: Requires pre-tagged categories and dates, missing informal feedback.

Enterprise-Wide Real-World Example

A travel-fintech firm, Super.com, integrated an AI-powered enterprise search (powered by Glean) across Slack, Confluence, GitLab, and Google Drive to manage its remote-first workflow. The solution:

Saved over 1,500 employee hours per month;
Reduced onboarding effort by 20%;
Supported AI-driven, personalized task lists and email drafts;
Ensured rigorous access controls to maintain security boundaries.

AI-Powered Search: Advantages and Limitations

Advantages of Adopting AI-Based Search for Unstructured Data

Employee Time Savings. AI search enables instant, context-aware retrieval that eliminates manual digging through emails, documents, and legacy systems. Automated tagging and categorization reduce administrative overhead, while unified search across CRM, HRM, and ERP systems cuts time wasted switching between apps;
Improved Customer Experience. Frustrated clients often repeat themselves when agents can't locate previous interactions. By linking all support tickets, calls, and emails to customer profiles, Artificial Intelligence creates a complete interaction history. Companies using AI see first-contact resolution rates increase significantly;
Reduced Risk of Data Loss. AI software penetrates deep into attachments, images, and audio files to surface buried content. The system preserves decades of knowledge through intelligent organization and retrieval. Organizations using AI maintain permanent access to historical IP, R&D notes from years past remain instantly discoverable.

Potential Challenges and Limitations of AI-Powered Search

Data Quality Requirements. Common issues include poor OCR (Optical Character Recognition) accuracy in scanned documents, inconsistent file formats (e.g., handwritten notes vs. digital PDFs), and duplicate or outdated versions of documents. Solutions include pre-implementation data cleansing, use of OCR enhancement tools, and metadata enrichment that helps supplement raw content with organizational context;
Security and Privacy Considerations. Accidental exposure of confidential information can occur through broad search results. Also, there’s a risk of over-permissioned access when searching across sensitive documents. Mitigation strategies may include implementing role-based access controls or using redaction capabilities for sensitive content;
User Training Requirements. Employees accustomed to keyword searches may struggle with formulating natural language queries or understanding semantic search capabilities. For example, a search for "drilling" could return unrelated results from oil industry reports and dental clinic records. You can plan a phased rollout and use interactive tutorials showing query examples (instead of: "invoice 2023" try: "Show me unpaid vendor invoices from Q1 2023");
Additional Limitations. Language support limitations for multilingual organizations; Computational costs for processing large document volumes; Integration complexity with legacy systems.

Unlocking Tomorrow’s Insights Today: The AI Search Revolution

As Artificial intelligence models grow more sophisticated, we’ll see systems that don’t just find documents but anticipate user needs. They’ll be able to surface relevant contracts before negotiations begin or highlight customer pain points before they escalate. Multimodal capabilities will bridge text, voice, and even visual data. For businesses ready to begin this transformation, success lies in phased adoption paired with strategic data preparation. Companies can focus first on cleaning and connecting core data sources, then expand as users adapt to conversational queries and confidence-based results.

Data Migration in Software Modernization. Balancing Automation and Developer’s Expertise

Sergey Laptick — Wed, 30 Jul 2025 12:40:37 +0000

When business owners think about modernizing a legacy application, they often focus on the most visible part: a sleek new user interface. However, the real challenge often lies beneath the surface. It’s data migration strategy. Moving data from an outdated system isn’t just a simple copy-paste job. It requires deep planning and expert execution. While automated data migration tools promise speed and cost savings, they are not a silver bullet. In this article, we’ll explore why automated tools alone aren’t enough, when developer expertise remains irreplaceable, and how a hybrid approach can save time and money.

How Databases Evolve During Legacy Software Modernization

When we talk about how data usually changes in the context of modernizing legacy software, we typically mean the following key processes.

Schema Migration includes incremental changes to the database structure itself, such as:

Adding, modifying, or dropping tables, columns, indexes, or constraints;
Adjusting data types or relationships between tables;
Essentially, this is about evolving the database design while keeping the same underlying system.

Data Migration as a process of moving data from one database system to another, such as:

Extracting data from an old database;
Transfering to a different DBMS (database management system), a new version of the same DBMS, migrating from on-premise to cloud database, etc.

A Real-World Challenge: From One Table to Multiple Tables

Legacy system modernization often means completely restructuring how data is stored. Let’s consider a real-world example of legacy system migration. Take a 20-year-old system, for example, where all user information was crammed into a single users table, which is pretty typical for old software. We need to deliver a new software version, and the new system usually provides much more detailed functionality. Therefore, that same data might be spread across multiple normalized tables, with relationships, constraints, and additional fields that didn’t exist before.

For example, In a legacy application, user data can be stored in a single table called users, which includes the following fields:

id
email
nickname
username
Several fixed columns representing the user’s interests (e.g., interest_1, interest_2, interest_3)

This structure is common in older applications that treated interests as fixed, hardcoded fields in the main user table. Such a table can be useful for e-commerce and marketplace platforms, for example to recommend products, collections, or promotions. A user who marks "home decor" and "kitchen gadgets" as interests may receive targeted recommendations and offers. This model is also common in eLearning platforms and EdTech apps to recommend relevant courses or learning materials based on interests. When user interested in "machine learning" and "data visualization" will see courses, webinars, or learning paths tailored to those topics.

Why Dividing User-related Data Into Multiple Tables

Dividing user-related data into multiple tables, such as separating interests, profile data, and relationships, introduces structural and functional advantages to a modern application. This approach follows the principles of relational database normalization, and it offers both technical benefits and business value. Here's what it enables:

Scalability and Flexibility. Instead of being limited to a fixed number of interest fields (e.g., interest_1, interest_2), a separate interests table allows for unlimited, dynamically added interests. Proper data migration strategy helps support future features, like tagging, user-generated topics, or hierarchical categorization, without redesigning the schema;
Efficient Queries and Personalization. With a proper schema, you can quickly query for all users interested in a specific topic or all interests of a given user. It enables fast personalization features like interest-based recommendations, user segmentation, and content filtering;
Localization and Internationalization. With a separate interests table, it's easy to localize interest names into different languages;
Better Maintainability. Adding a new feature (e.g., allowing users to follow interests or subscribe to updates) only requires changes in the interests or user_interests tables;
Rich Metadata and Behavior Modeling. You can add attributes to interests, like popularity, category, or trendiness, or track user behavior (e.g., last_engaged_at). This feature improves user experience through trend-based suggestions and behavioral analytics.

Modernized Schema

To reap the described benefits, In the new system, user interests can be treated as a flexible, scalable relationship:

A separate interests table holds all possible interest categories;
A junction table user_interests maps users to their selected interests.

Therefore, in a modernized app, we’ll have three tables instead of a single users table from the outdated app. Additionally, the new system no longer stores a single username field. Instead, it splits this into:

first_name
last_name
gender (to support features like salutation: Mr., Ms., in an AI chatbot, for instance)

However, the original system does not store gender. Therefore, it must be inferred based on the user's name if no other information is available.

The Cost of Fully Manual Data Migration

The process of data migration is often far more labor-intensive than expected. We've only described a few basic features, and even implementing this little set requires splitting a single legacy table into three normalized tables. In real-world scenarios, the number of such transformations is often significantly higher.

Additionally, consider the volume of data handled by applications that have been on the market for decades. Migrating such data structures is a major task. The amount of custom logic a developer must implement to ensure data integrity and correct representation can be substantial.

This naturally raises the question:

Is there a way to reduce the workload, cut modernization costs, and accelerate the release of the new software version?

The answer is yes, and this is exactly where automated data migration tools come into play.

How Hybrid Data Migration Help Developers Modernize Legacy Systems

How Automated Tools Help Migrate Data

While complex transformations, such as splitting one legacy table into multiple modern relational tables, require custom logic and developer-written scripts, automated data migration tools can still significantly accelerate the overall process when applied strategically. Here are some examples:

Schema Discovery and Mapping. Automated data migration tools, such as Talend, can inspect and map source schemas to target schemas, highlighting structural differences. This is especially useful during initial planning, since it helps developers understand which parts of the schema need custom transformation and which can be migrated directly;
Bulk Data Transfer. Once transformation logic is implemented, automated tools, such as Fivetran, can execute bulk data loads to transfer millions of records reliably and with built-in error handling, logging, and retry mechanisms. This reduces human error and dramatically shortens the time required to move large data sets;
Validation and Monitoring. Automated systems like Datafold can verify that row counts and data types are consistent between source and target. Such data validation tools free developers from writing boilerplate verification logic and allows them to focus on business-critical transformation code.

By offloading tasks like extraction, loading, and validation, automated tools can save developers hours or even days on each test iteration of the migration process. For business owners modernizing decades-old software, this hybrid approach offers:

Lower development costs, as automation reduces the number of custom scripts required;
Faster iteration, making it easier to test, refine, and re-run migrations before final deployment;
Reduced risk, thanks to automated validation and rollback capabilities.

How Developers Handle Tasks That Are Beyond the Capabilities of Automated Tools

Automated data migration tools can help developers migrate to a different database management systems or to a new version of the DBMS in use, applying the required data manipulations to ensure accurate representation. Also, they can copy the id, email, and nickname fields with little trouble. Possibly, there will be no issues with replicating the old users table into a staging environment.

Automated data migration tools can’t successfully perform tasks required for the use case we described earlier. For instance, infer gender from names (e.g., determine "Sarah" is female, "John" is male), or populate the interests table dynamically from user-provided values. Also, there could be issues with deduplicating shared interests across users (e.g., don’t insert "kitchen gadgets" twice) or creating the correct many-to-many relationships in user_interests.

To address these limitations and handle the data migration process, developers must first copy to a new system all transferable data “as-is,” without any changes. This can be done using SQL queries or automated migration tools. Then, for data requiring additional processing, developers write custom migration scripts that:

Iterates through each row in the legacy users table;
Splits the username into first and last names;
Uses predefined logic or an external service to infer gender based on the first name;
Normalizes user interests. It includes adding new interests to the interests table (if not already present) and adding corresponding entries in user_interests linking users to their interests.

Example:

User 1: Sarah Jones → first_name = Sarah, inferred gender = Female
- Interests: A, B, C → Insert A, B, C into interests (if new), and link Sarah to them in user_interests
User 2: John Chivers → first_name = John, inferred gender = Male
- Interests: B, C, D → B and C already exist; insert D; create mappings for all three in user_interests

The Human-Automation Partnership in Data Migration

While automated tools bring undeniable speed and reliability to repetitive migration tasks, they cannot fully replace the insight, logic, and domain expertise. Effective data migration requires a hybrid approach. Automation can handle the heavy lifting (bulk transfers, change tracking, and validation) while developers focus on high-value tasks like transforming data models, preserving business logic, and ensuring consistency across evolving schemas.