DEV Community: Harshdeep Singh

The 95% Problem: Why Enterprise AI Keeps Failing — and What the 5% Get Right

Harshdeep Singh — Thu, 11 Jun 2026 19:53:22 +0000

Ninety-five out of every hundred enterprise AI pilots produce nothing a CFO would sign off on. The reflex is to blame the model — too dumb, too small, the wrong vendor. It almost never is. The thing quietly killing enterprise AI is older and more boring than any model: data nobody organized for machines, and rules nobody ever wrote down. The strangest part of the story is who is losing the fight hardest — the firms whose entire business is selling everyone else the cure.

The most expensive irony in enterprise software

In late 2025, Deloitte gave part of a government cheque back. The firm had delivered a report to Australia's Department of Employment and Workplace Relations, and reviewers found something awkward buried inside it: citations to academic papers that did not exist, and a fabricated reference to a federal court judgment. The work had been produced with help from generative AI, and no one had checked it before it went out the door. Deloitte agreed to refund part of its fee.

It is tempting to read that as a story about a hallucinating chatbot. It is not. A capable model can cite a real paper; the failure was not that the AI was too weak. The failure was that nothing in the process forced a human to verify machine output before it reached a client. There was no standard operating procedure, no checkpoint, no rule with teeth. That distinction — between a model problem and a data-and-governance problem — is the entire subject of this essay, and the firms that sell AI for a living have just handed us the clearest possible illustration of it.

Consider the position those firms are in. Since 2023, the Big Four and the major strategy houses have collectively poured more than ten billion dollars into AI. It is their flagship pitch. Accenture reported close to six billion dollars in generative-AI bookings in a single fiscal year. PwC became one of OpenAI's largest enterprise customers and then its reseller. KPMG signed a two-billion-dollar alliance with Microsoft. These organizations have built their modern brand on the promise that they can walk into any enterprise and fix its AI problem. And yet, internally, they have hit precisely the wall they are paid to dismantle.

This is not schadenfreude about one embarrassing report. It is the most useful data point in the entire enterprise-AI conversation, because it removes the easy excuses. You cannot say the consultants lacked talent, budget, model access, or executive buy-in. They had all of it in abundance. If the people who sell the cure can still catch the disease, then the disease is not what most companies think it is. It is not a shortage of intelligence in the model. It is a shortage of order in the data and discipline in the governance — and almost nobody is immune.

The number that belongs on every board agenda

Start with the figure that has been ricocheting around boardrooms since it landed. In its 2025 report The GenAI Divide: State of AI in Business, MIT's NANDA initiative studied hundreds of enterprise AI initiatives and concluded that roughly ninety-five percent of them had produced no measurable impact on the bottom line. Not weak returns. No returns. The spending in scope ran to tens of billions of dollars, and the overwhelming majority of it bought experiments that never crossed into anything a finance team could defend.

95% of enterprise generative-AI pilots deliver no measurable business impact — the spending lands, the value does not.

It is not an isolated finding. Gartner expects that by the end of 2025, three in ten generative-AI projects will be abandoned after the proof-of-concept stage, and that through 2026, sixty percent of AI projects will be scrapped specifically because the organizations lacked AI-ready data. The firm goes further on agents: it forecasts that more than forty percent of agentic-AI projects will be cancelled by the end of 2027. The RAND Corporation has put the historical AI project failure rate above eighty percent — roughly twice the rate of conventional IT projects. And S&P Global found that the share of companies abandoning most of their AI initiatives jumped to forty-two percent in 2025, up from just seventeen percent a year earlier. The trend is not improving as the technology matures. It is getting worse as spending outruns readiness.

The crucial detail is where these projects die. They almost never fail in the lab. They fail on the road to production. A pilot runs on a curated slice of data — a clean schema, a controlled volume, a problem chosen because it demos well. Production runs on the actual enterprise: the duplicated records, the contradictory definitions, the fields that mean different things in different systems, the knowledge trapped in formats no machine can read. The distance between the demo and the deployment is the distance between curated data and real data, and that distance is where the money disappears. People in the field have a name for the place projects go to expire: pilot purgatory.

The people closest to the data already know this. In Informatica's 2025 survey of chief data officers, the most-cited obstacle to AI success was not talent, not budget, not model quality — it was data quality and readiness. The executives responsible for the foundation are telling everyone the foundation is the problem. Most strategies are simply not listening, because listening would mean slowing down to do the tedious work, and the market is rewarding speed.

And the window in which to fix this is closing faster than the failure rate alone suggests, because the industry is sprinting from chatbots to agents. Gartner expects that by the end of 2026, four in ten enterprise software applications will include task-specific AI agents, up from less than one in twenty in 2025. Agents raise the stakes of the underlying problem by an order of magnitude. A chatbot that retrieves bad data returns a bad answer a human can still catch. An agent that acts on bad data — reconciling an account, approving a request, triggering a downstream workflow — propagates the error into the real world before anyone reviews it. The same analysts forecasting the agentic wave also forecast that more than forty percent of agentic projects will be cancelled by the end of 2027, for the same unglamorous reasons the chatbots failed. We are, in other words, about to point far more autonomous systems at foundations that were already too weak for the last generation of tools.

That is what makes the failure rate a strategic problem rather than a technical footnote. The cost is not the wasted pilot budget; that is the cheap part. The real cost is competitive. Every quarter a rival reaches production while you re-run experiments that were always going to fail for the same reason, the rival's system gets better, its data gets cleaner, its people get more fluent, and the gap compounds. You are not standing still. You are losing ground while looking busy.

It was never the model

The comforting story inside most failed AI programs is that the technology was not ready, and that the next model — bigger, newer, from a different lab — will be the one that finally works. It is comforting because it requires nothing of the organization except patience and a bigger invoice. It is also wrong.

Here is the inconvenient test. The model that hallucinated its way through your failed pilot is, in most cases, the same model that performed flawlessly in the vendor's demo. Nothing about the weights changed between those two moments. What changed was everything around the model: the quality of the data it was fed, the clarity of the instructions it was given, and the rules governing what it was allowed to touch. The model was never the variable. The environment was.

The model in your failed pilot and the model in the vendor's flawless demo are usually the same model. The difference between them is everything you built — or failed to build — around it.

This is also why "wait for the next model" is such a seductive and expensive trap. Each new model is genuinely more capable than the last, which makes it easy to believe the next one will finally clear the bar. But a more capable model pointed at the same unstructured data and the same absent rules does not fix the problem — it executes the same mistakes more fluently, and, increasingly, more autonomously. Capability without a foundation is not progress. It is leverage applied to a fault line.

That environment has two load-bearing pieces, and almost every enterprise is missing both. The first is data that a machine can actually reason over. The second is governance a machine can actually obey. The original instinct that AI needs "organized data and some kind of SOP" is exactly right — it just turns out that each half is a deep discipline in its own right, and that naming them separately is the difference between a strategy that works and a slide that sounds good. Take them one at a time.

Gap one: data that was never built for machines

An AI agent does not think the way a database is organized. It does not navigate neat rows and columns; it reasons over entities, the relationships between them, and the context that gives them meaning. It needs to know that this customer is the same as that account, that "revenue" in the finance system and "revenue" in the sales dashboard are or are not the same number, that this contract supersedes that one, that this policy applies to this region. Enterprise data, as it actually exists, is almost the precise opposite of that.

In most companies the data is siloed across systems that were never designed to talk to each other, duplicated in ways no one fully maps, and defined inconsistently enough that the same word can name genuinely different things in different systems. Worse, the knowledge that actually matters — the reasoning, the precedent, the hard-won judgment — tends to live in formats machines cannot read: slide decks, PDFs, email threads, and the heads of senior people who are about to retire. You can connect the cleanest model in the world to that, and it will faithfully reflect the chaos back to you.

The most instructive proof of this comes, again, from a consultancy. When McKinsey built its internal AI platform, the firm discovered that the tool could not initially parse PowerPoint — which was a problem, because PowerPoint is where most of McKinsey's institutional knowledge actually lived. Sit with that for a moment. One of the most knowledge-intensive organizations on earth, a firm whose entire product is structured thinking, found that its crown-jewel intellectual property was effectively illegible to a machine until it did real work to fix the ingestion. If McKinsey's knowledge was trapped in slides, it is worth asking, honestly, what shape yours is in.

The failure rarely announces itself as missing data. It hides in data that is present but means subtly different things in different places. Ask an agent a question as ordinary as how many active customers the business has, and it will find a dozen tables with a dozen definitions of "active" — a login within the last thirty days in one system, a non-zero balance in another, an uncancelled contract in a third. A human analyst resolves that ambiguity with context and a quick message to a colleague. An agent, lacking both, picks one definition silently and reports a confident number that is wrong in a way nobody can see. Multiply that across every entity and every metric a company cares about, and you have the real texture of the problem: not an empty warehouse, but a full one with no shared language.

You cannot retrieve your way out of a data swamp

The popular hope is that retrieval-augmented generation — pointing an agent at your documents and letting it fetch what it needs — will paper over the mess. It will not. An agent retrieving from a swamp returns swamp, dressed up in fluent prose that makes the swamp harder to detect. And the instinct to fix this by building a bigger data lake usually just produces a bigger swamp with better storage economics. Volume was never the problem. Meaning was.

What actually closes the gap is a layer most enterprises have never built: a semantic, machine-readable map of what the data means. In practice this goes by several names that point at the same idea — a semantic layer, an ontology, a knowledge graph, a governed data catalog. The common thread is that core business concepts get defined once, consistently, in a form an agent can consume: what a customer is, what counts as revenue, how entities relate, which rules and constraints apply. The catalog becomes the control plane of truth, and the semantic layer becomes the thing that lets a model answer in terms of your business rather than in terms of raw, ambiguous tables.

The organizing principle behind all of this is treating data as a product rather than as exhaust. Exhaust is whatever a system happens to emit, owned by no one, documented nowhere. A product has an owner, a contract, documentation, versioning, and a consumer whose needs shape it. The research bears out how much this matters: organizations that treat data as a product — with curated models and shared vocabularies — are dramatically more likely to scale generative AI successfully than those that do not. When the foundation is built this way, retrieval techniques like graph-based RAG can ground every answer in verified, connected data, enforce access controls at the moment of the query, and trace each response back to the exact source it came from. That is the difference between an agent that confidently invents a court case and one that shows its work.

A knowledge graph earns its keep precisely here. Instead of flat, disconnected tables, it stores the business as a web of entities and the relationships between them: this customer belongs to this account, which is covered by this contract, governed by this policy, owned by this team. An agent reasoning over that structure can follow the connections the way a knowledgeable employee would, and every answer it produces carries its lineage — which source, which definition, which version. That is also what makes governance enforceable at the level of meaning rather than the level of the raw row, because the graph knows what a thing is and who is permitted to see it.

The strategic point hiding in the plumbing

Strip away the vocabulary and the strategic truth is simple: AI readiness is mostly data maturity wearing a more exciting outfit. You cannot purchase your way past it with a model subscription, because the thing you are missing is not compute or intelligence — it is the slow, structural work of making your own knowledge legible. That work is unglamorous, expensive, and invisible in a board deck, which is exactly why most organizations skip it and exactly why most organizations end up in the ninety-five percent. The semantic foundation is not overhead on the way to the real prize. It is the prize. It is the part competitors cannot copy by signing the same vendor contract you did.

Gap two: governance that was never written down

Order, structure, repetition — the corporate grid. Governance is what gives an agent the same scaffolding a new employee takes for granted. Photo: Fabian Kleiser / Unsplash.

If you ask most enterprises where their AI governance lives, the honest answer is a PDF on a shared drive — a well-intentioned document of principles that almost no one has read and that no system enforces. A PDF nobody reads is not a policy an agent can obey. It is a statement of hope. And hope does not survive contact with an autonomous system acting at machine speed across systems it was never explicitly cleared to touch.

Governance for AI, and especially for agents, has to be machine-actionable to mean anything. The "SOP" intuition is the right one, but it resolves into two concrete questions that a slide of principles never answers: what is this agent actually allowed to touch and do, and what is the operating rhythm that keeps it honest over time? Get specific about both, and governance stops being a compliance ornament and starts being the thing that lets you deploy without holding your breath.

An agent is a new employee with root access and no onboarding

It helps to think of an agent as exactly what it is becoming: a digital worker. The trouble is that we wrap human workers in decades of accumulated controls and give agents almost none of them. A new human employee receives an identity, a defined role, least-privilege access to only the systems their job requires, a manager who reviews their work, and an audit trail that records what they did. An agent, in too many deployments, gets a single shared API key with broad standing credentials, the ability to call tools far outside its actual task, and no logging worth the name. It is the most over-permissioned new hire in the building, and no one interviewed it.

The discipline that fixes this is well understood, even if it is rarely applied. Security researchers call the core idea least-agency, or least-privilege: an agent should receive the minimum autonomy required for its specific task and nothing more. A customer-support agent does not need write access to the billing database. A research agent does not need the ability to send external email. From there it cascades into concrete controls: whitelisting the specific tools an agent may use, issuing short-lived credentials instead of permanent keys, sandboxing execution, restricting where an agent can send data, and — critically — keeping a human in the loop for actions that are irreversible or sensitive. A mature deployment will refuse to let an agent move money without clearing a confidence threshold or obtaining a second approval, and will strip personally identifiable information before it ever reaches a model, restoring it only on the way back. None of that lives in a principles document. All of it lives in enforced policy. That, and not the PDF, is the real standard operating procedure: rules expressed as controls a system cannot route around.

The danger is not hypothetical, and it does not require malice. Picture an agent handed broad database credentials so it could "be helpful," then asked to tidy up some duplicate records. With no constraint on its scope and no human checkpoint, a single ambiguous instruction becomes a destructive write across production data in seconds — faster than any person could intervene, and recorded nowhere anyone thought to look. The same autonomy that makes agents useful is what makes their mistakes fast and quiet. Standing credentials, missing audit trails, and unrestricted tool access are not exotic edge cases; they are the default state of most early agent deployments, and they are exactly how a promising program turns into a board-level incident.

A policy an agent cannot read is decoration. Governance that scales is policy an agent is structurally unable to disobey.

You do not have to invent the rulebook

The encouraging part is that the scaffolding for all of this already exists, written by people who have thought about it harder than any individual team has time to. The U.S. National Institute of Standards and Technology publishes the AI Risk Management Framework, along with a dedicated profile for generative AI, and its emphasis maps almost directly onto agent controls: role-based access, continuous monitoring, adversarial testing, and lifecycle logging for traceability. The international ISO/IEC 42001 standard formalizes the idea of an AI management system, with oversight and continual improvement built in. The OWASP GenAI Security Project maintains a Top 10 for large-language-model applications and a newer Top 10 for agentic applications, cataloguing the exact failure classes teams keep rediscovering the hard way: prompt injection, tool misuse, memory leakage. And the external pressure is rising fast, from the EU AI Act to a wave of national AI laws, which means governance is shifting from a nice-to-have to a condition of doing business.

Underneath the frameworks sits a single overlooked capability that decides whether any of them are real: observability. If you cannot see what an agent did — which data it touched, which tools it called, which decision it made and why — then you cannot govern it, debug it, or defend it to a regulator, and you certainly cannot trust it with anything that matters. Audit logging and traceability are not paperwork. They are the line between an agent you can put into production and a black box you can only hope behaves. Trust, in the end, is not extended to systems because they are clever. It is extended to systems because they are accountable.

The reframe that matters most here is that governance is not a brake on AI. It is the enabler. The organizations that actually reach production are, consistently, the ones that invested in governance frameworks before they scaled agent capabilities — not after a breach forced their hand. The absence of guardrails does not make you faster; it produces exactly the brittle, untrustworthy, occasionally catastrophic behavior that makes leadership pull the plug and sends a promising program back to purgatory. Guardrails are what let you move quickly without flinching.

The consultants' mirror

Return now to where we started, because the consulting firms are not just a cautionary anecdote. They are the most public, best-funded live experiment in internal AI adoption that exists, and watching them is the closest thing the rest of us have to a controlled trial. They are simultaneously the largest sellers of AI transformation and a room full of organizations trying to transform themselves — and they have been unusually candid about how hard it is. BCG, in its own cross-industry research, found that roughly three-quarters of companies struggle to achieve and scale value from their AI initiatives. The firms are not describing a problem they have solved from the outside. They are describing one they are living from the inside.

The stakes are visible in their own pyramids. Internal tools like McKinsey's assistant and BCG's slide-polishing system can already perform a large share of the research-and-formatting work that used to define a junior analyst's first years, and entry-level hiring across the industry has tightened as a result. That is what it looks like when this technology genuinely lands inside an organization — and it is a useful reminder that getting it right is not a productivity nicety. It restructures the firm. The flip side is that getting it wrong, in public, with a client's name on the document, restructures the firm's reputation just as quickly.

And so every major firm now has its own platform: McKinsey with its knowledge assistant and a fleet of internal agents numbering in the tens of thousands, BCG with its build unit and internal tools, Deloitte with its assistant and an agentic platform, PwC with an agent operating system spanning tens of thousands of deployed agents, EY with a platform giving tens of thousands of staff access to a growing roster of agents and a multi-year plan to scale into the hundreds of thousands, and KPMG with its own agentic workbench. Billions of dollars, real engineering, genuine ambition. But the tools are the visible ten percent. The invisible ninety percent — the part that determines whether any of it works — is the data and governance plumbing underneath. There is a useful rule of thumb circulating in this world that only a small fraction of AI value comes from the algorithms and the technology, with the overwhelming majority coming from people, process, and the organizational change required to make the technology stick. The firms that are winning internally are the ones that took that ratio seriously.

The five percent: what McKinsey's Lilli actually proves

Organized, machine-legible knowledge was the real moat — not the model. Every rival had the same models. Photo: Susan Q Yin / Unsplash.

If the failure rate has a counterexample worth studying, it is McKinsey's internal platform, Lilli. It is the case study everyone cites, and almost everyone draws the wrong lesson from it. The wrong lesson is that McKinsey succeeded because it had access to powerful models. That cannot be the explanation, because every competitor had access to the same models. The right lesson is far less flattering to the technology and far more useful to anyone trying to replicate the result: McKinsey succeeded because it did the boring work that everyone else was skipping.

Look at what the boring work actually was. The platform draws on more than forty knowledge sources and over a hundred thousand documents and interview transcripts — but the unlock was not aggregation, it was curation and tagging, the patient labor of making a century of accumulated knowledge consistent and machine-legible. The team built what is better described as an orchestration layer than a simple retrieval bot, designed to synthesize and contextualize rather than just fetch. They confronted the unglamorous reality that their best material was trapped in slides and fixed the ingestion so the machine could read it. Only then did the human side of adoption begin: a phased rollout, training that cured what the firm called "prompt anxiety" in roughly an hour, internal evangelists, and senior leaders modeling the behavior they wanted to see.

The results are the part people quote, and they are genuinely impressive: more than three-quarters of the firm's tens of thousands of employees now use the tool, heavy users return to it more than a dozen times a week, and the firm reports its people save close to a third of their research time. But the number to internalize is not the adoption rate. It is what produced it. The moat was never the model. The moat was a hundred years of knowledge made legible to machines, wrapped in the governance and the change management required to make people actually trust it and use it.

McKinsey's edge was not a smarter model — every rival had the same models. The edge was a century of knowledge made legible to machines, and the discipline to govern it.

The pattern repeats across the rest of the industry, even if less dramatically. The firms making real internal progress — across the spectrum of platforms and agents now deployed — are consistently the ones that invested in their data foundations and their governance before they tried to scale. And the lesson generalizes cleanly to any enterprise, in any sector, that wants off the wrong side of the divide. The winners are not the organizations with the best model. Everyone has the same models. The winners are the organizations that did the unglamorous data-and-governance work that everyone else found a reason to defer.

It is worth saying plainly what this does and does not mean for everyone else, because the lesson is easy to mislearn. You cannot buy McKinsey's result by buying McKinsey's tool, any more than you could acquire a rival's culture by licensing their software. What travels is not the platform; it is the method — the willingness to treat your own knowledge as an asset worth making machine-legible, and your own governance as an engineering problem worth solving before the agents arrive. That method is available to any organization in any industry. It is simply not for sale, and it cannot be rushed.

What the winners actually do

None of this resolves into a checklist, and anyone selling you one is selling the wrong thing. But the organizations on the right side of the divide do share a small number of strategic commitments, and they are worth stating plainly — not as steps to execute in order, but as the shape of a serious posture toward AI.

They sequence data before models

The single most counterintuitive move the winners make is to stop running hero pilots and fix the foundation first. That does not mean a multi-year data project before any value is delivered; it means choosing initial use cases precisely where the data is already clean and compatible, shipping those to generate real and defensible returns, and then using that credibility and momentum to fund the remediation of the messier domains. The failure mode is the opposite: picking use cases based on strategic ambition and executive enthusiasm, discovering the data underneath cannot support them, and producing an over-budget pilot that demonstrates the limits of the data rather than the capability of the AI. Match the ambition to the data maturity, not to the org chart's excitement.

They treat data as a product, not exhaust

The winners give their data owners, contracts, documentation, and a semantic layer that defines the business vocabulary once and reuses it everywhere. The catalog becomes the control plane of truth; the ontology and knowledge graph become the connective tissue that lets agents reason over entities and relationships rather than guess at ambiguous tables. This is the work that does not show up in a launch announcement and entirely determines whether the launch was real. It is also, not coincidentally, the part of the strategy a competitor cannot acquire by signing the same contracts you did.

They make governance machine-actionable

The winners do not confuse a principles document with a control. They express policy as something a system enforces: identity for every agent, least-agency access, tool whitelisting, audit trails, and a human in the loop for anything irreversible or sensitive. They adopt an established standard rather than inventing their own — the NIST framework, the ISO management-system standard, the OWASP failure catalogues — and they wrap it in an operating rhythm: regular reviews, red-teaming, and genuine change management for agents, treating a new agent with the seriousness one would treat a new hire with broad system access. Governance, done this way, is not the thing that slows the program down. It is the thing that lets the program move without fear.

They build the context layer agents inherit

Rather than re-explaining the business inside every prompt, the winners push meaning and rules down into the data itself, so that any agent connecting to it inherits both. It is worth watching the emerging plumbing here — open protocols that standardize how agents connect to tools and to one another, sometimes described as an "HTTP for agents," are quickly becoming the connective standard for this world. But a word of caution that the protocol enthusiasm tends to skip: plumbing that lets agents reach your data faster does nothing good if the house behind the tap is a mess. Standard connectivity over a swamp just distributes the swamp at higher throughput. The connectivity is necessary; the clean, governed foundation is what makes it worth having.

They treat adoption as a change program, not a software rollout

Finally, the winners understand that buying licenses is not the same as achieving adoption. The most replicable lesson from the internal success stories is almost embarrassingly human: an hour of training to dissolve the anxiety of a blank prompt, visible evangelists, and leaders who actually use the tools they are asking their people to use. If the overwhelming majority of AI value comes from people and process rather than the algorithm, then the overwhelming majority of the effort has to go there too. Technology adoption has always been a human problem wearing a technical mask, and AI has not changed that. It has only raised the stakes.

They measure value, not motion

The organizations stuck in the ninety-five percent tend to measure activity — pilots launched, seats provisioned, models evaluated — and mistake it for progress. The winners measure outcomes, and they are ruthless about it: a use case either moves a number a finance team recognizes, or it is killed quickly, before it hardens into a permanent science project. That discipline is exactly what frees the budget and the attention to pour into foundations, where the compounding returns actually live. Counting pilots is how a program feels busy while going nowhere. Counting value is how it escapes the lab.

The real divide

It is worth being precise about what the so-called GenAI Divide actually divides. It is not a line between companies with good models and companies with bad ones. Frontier models are a commodity now; the same handful are available to everyone with a credit card. The divide is between the organizations that did the foundational work and the organizations that did not — and underneath the AI costume, that is simply a gap in data maturity and governance discipline that has existed for years and that AI has suddenly made expensive to ignore.

And it compounds. The organizations on the right side of the divide get faster every quarter, because their agents inherit ever-cleaner data and ever-tighter rules, and each success funds the next. The organizations on the wrong side re-run pilots that fail for the same reason they failed last time, mistaking a foundation problem for a model problem and waiting for a model that was never going to save them. The gap between the two groups does not stay constant. It widens.

The deepest irony of the whole story is the one we began with. The cure for the failing enterprise AI program was never a smarter model. It was the boring, expensive, unglamorous discipline that the consultants themselves had to learn the hard way, in public, with a refunded invoice as tuition: organize the data so a machine can reason over it, write the rules down in a form a machine is forced to obey, and only then let the agents loose. The companies that internalize that will not merely adopt AI. They will compound on it — quietly, structurally, and largely out of view — while everyone else is still abandoning pilots and blaming the model.

The divide compounds. The foundation you lay now decides how fast you can move later. Photo: Robert Bye / Unsplash.

Sources and further reading

MIT NANDA initiative, The GenAI Divide: State of AI in Business 2025 — the source of the widely cited finding that roughly 95% of enterprise generative-AI pilots show no measurable business impact.
Gartner — predictions on proof-of-concept abandonment, AI-ready data, and agentic-project cancellation rates.
VentureBeat — background on McKinsey's internal Lilli platform and how it was built.
Deloitte, State of AI in the Enterprise — recurring survey data on enterprise AI spend, scaling, and ROI.
NIST AI Risk Management Framework and its Generative AI Profile — role-based access, monitoring, adversarial testing, and lifecycle logging.
OWASP GenAI Security Project — the Top 10 for LLM applications and the Top 10 for agentic applications, covering prompt injection, tool misuse, and related failure classes.
ISO/IEC 42001 — the international standard for an AI management system, covering oversight and continual improvement.

Building an LLM Project From Scratch in 2026

Harshdeep Singh — Tue, 09 Jun 2026 17:33:40 +0000

Here’s the uncomfortable truth about “AI projects” a few years ago: the hard part was never the model. It was the plumbing. Standing up a vector database, wiring an embeddings pipeline, fighting with streaming responses, gluing five libraries together — by the time it worked, you’d forgotten what you set out to build.

In 2026 that plumbing has largely collapsed into a weekend’s worth of work. Model prices have fallen roughly 80% year over year, free tiers are genuinely usable, your database now does vector search natively, and one SDK handles streaming and tool calling across every provider. The skill that’s actually in demand — retrieval-augmented generation with agents — is now reachable by a developer who has never touched machine learning.

So this guide does something specific. We’re going to build one real project, end to end, that you can put on your portfolio and let strangers use: an app where someone uploads their documents and chats with them — asking questions and getting answers grounded in their own files, with citations, streamed token by token. It’s the canonical 2026 LLM project, and it teaches almost everything else by osmosis.

In plain English. “RAG” means the AI doesn’t answer from memory — it looks things up first. You give it a pile of documents; when you ask a question, it finds the most relevant passages and answers using only those. That’s why it can talk about your files without ever having been trained on them.

This guide is written for three readers at once: newcomers, working software engineers, and AI engineers. The main text stays approachable; the “In plain English” notes add no-jargon explanations, and the “Under the hood” notes add depth for engineers.

The roadmap — what we’ll actually do

Eight steps. Each one produces something that works before we add the next layer.

Build the mental model — how RAG (and agentic RAG) really works, in one diagram.
Choose your models — a cost comparison of hosted and self-hosted LLMs, and which embedding model to use.
Set up the MERN stack — project skeleton plus a MongoDB Atlas vector index.
Ingest documents — upload, parse a PDF, and split it into chunks.
Embed & store — turn chunks into vectors and save them in MongoDB.
Retrieve — find the right passages with a single $vectorSearch query.
Make it agentic — let the model call retrieval as a tool, on its own terms.
Stream & deploy — render tokens live in React, then ship it to its own URL for free.

Let’s start with the one idea that makes the other seven make sense.

Step 1 · The mental model: how RAG actually works

An LLM is a brilliant improviser with no access to your private data and a tendency to confidently make things up. Retrieval-Augmented Generation (RAG) fixes both problems with one move: before the model answers, you fetch relevant facts and hand them over as context. The model then answers from evidence rather than from vibes.

There are two phases. The first happens once, ahead of time (ingestion); the second happens on every question (retrieval + generation).

INGESTION (run once, when a document is added)
  document --> split into chunks --> embed each chunk --> store vectors in MongoDB

RETRIEVAL + GENERATION (run on every question)
  question --> embed --> vector search in MongoDB --> top-k chunks
                                                         |
                             +---------------------------+
                             v
        [ question + retrieved chunks ] --> LLM --> grounded answer + citations

The magic ingredient is the embedding: a list of numbers (a vector) that captures the meaning of a piece of text. Two passages about “canceling a subscription” land near each other in this number-space even if one says “refund” and the other says “cancel my plan.” Searching by meaning instead of keywords is what makes RAG feel intelligent.

In plain English. Imagine every sentence gets pinned onto a giant map, where similar meanings sit close together. To answer your question, the app drops a pin for your question and grabs whatever text is pinned nearby. Those nearby notes become the AI’s cheat sheet.

What makes it “agentic” — the 2026 upgrade

Classic RAG retrieves once and hopes the first search was good enough. That breaks on real questions: “Compare the refund policy in the 2024 contract with the 2025 one” needs two different searches and a comparison. Agentic RAG hands the model the steering wheel. Retrieval becomes a tool the model can call — repeatedly — deciding what to search for, judging whether the results are sufficient, and searching again before it commits to an answer.

Under the hood. A 2025 survey (“Agentic Retrieval-Augmented Generation,” arXiv:2501.09136) frames these systems around four patterns: reflection, planning, tool use, and multi-agent collaboration. In practice you’ll implement query rewriting/decomposition, multi-hop “retrieve → reason → retrieve” loops, and self-critique (“do these passages actually answer the question?”). The cost: 3–10× the tokens and 2–5× the latency of vanilla RAG. So gate it — a trivial FAQ should never enter the loop; a cross-document question can’t be answered without it. Cap the loop at ~5 iterations so a confused agent can’t spend your budget in a runaway.

We’ll build the simple pipeline first (so you can see every piece), then promote retrieval to an agentic tool in Step 7. That progression is the lesson.

Step 2 · Choosing your LLMs — cheapest viable first

You’ll use two kinds of model: an embedding model (turns text into vectors) and a generation model (writes the answer). They’re priced and chosen separately. Let’s start with generation, since that’s where the “which LLM?” anxiety lives.

Read this first. Every price below is a 2026 snapshot and model names change almost monthly. Treat this table as a shape, not gospel — confirm the current number on the provider’s pricing page before you commit. The strategy (route cheap, escalate rarely) outlives any specific figure.

Provider / model	Input ($/1M)	Output ($/1M)	Free tier?	Best for
Google Gemini Flash-Lite	~$0.10–0.25	~$0.40–1.50	Yes — generous	Learning & high volume; the default starter
Groq · Llama 3.1 8B	~$0.05	~$0.08	Yes	Blazing-fast responses; cheapest tokens
DeepSeek	~$0.14	~$0.28	No (cheap)	Cheapest frontier-class; OpenAI-compatible API
OpenAI · GPT mini-tier	~$0.15–0.75	~$0.60–4.50	Credits	Strong all-rounder; great tool calling
Anthropic · Claude Haiku	~$1.00	~$5.00	No	Cheapest Claude; reliable instruction-following
Frontier (GPT / Claude / Gemini Pro)	~$2.50–5.00	~$15–30	No	Final answer only, when quality truly matters

The pattern jumps out: the cheapest models are 50–100× cheaper than the flagships. For a RAG app, most of your token spend is feeding retrieved context into the model — so a cheap, capable model handling that bulk is the entire cost game. Use a frontier model only for the final synthesis, and only if you can measure that it’s actually better for your task.

Self-hosted: running models on your own machine

You can skip API bills entirely with Ollama, which runs open models locally and exposes an OpenAI-compatible endpoint at http://localhost:11434/v1 — meaning your code barely changes. One command pulls and runs a model:

# install from ollama.com, then:
ollama run llama3.1:8b          # chat model, ~6-8 GB VRAM
ollama pull nomic-embed-text    # local embedding model, free

Under the hood. Rough VRAM at Q4 quantization: 7–8B models ≈ 6–8 GB, 14B ≈ 10–12 GB, 32B ≈ 20–22 GB, 70B ≈ 43–48 GB (Apple Silicon unified memory counts fully). Break-even vs hosted APIs is roughly 500K tokens/day of sustained traffic — below that, hosted is cheaper and you skip the ops. Trade-offs: full privacy and $0/token, but weaker reasoning than frontier models and you own the uptime.

The embedding model — quieter, but it matters

Embeddings are dramatically cheaper than generation, so this is an easy call. For most projects, OpenAI’s text-embedding-3-small is the sweet spot.

Model	Dimensions	Price ($/1M)	Notes
OpenAI text-embedding-3-small	1536	~$0.02	Best balance; our pick for the build
Google gemini-embedding	768	Free tier / ~$0.025	Free-tier friendly
Voyage (voyage-3.5-lite)	512–1024 (reducible)	~$0.02	Now MongoDB-owned; long context
nomic-embed-text / BGE-M3 (open)	768 / 1024	Free (self-host)	Run free in Ollama; great quality

Gotcha that bites everyone. Vectors from different embedding models are not compatible. If you switch embedding models later, you must re-embed your entire corpus. Pick one and commit. (Embedding 10M chunks with text-embedding-3-small costs only ~$100, so this is about consistency, not cost.)

Our choices for this build: embeddings via text-embedding-3-small; generation via a cheap, fast model (Gemini Flash or a GPT mini-tier model) while learning — swappable in one line thanks to the SDK we’re about to set up.

Step 3 · Setting up the MERN stack

MERN is a natural fit for RAG in 2026 for one reason that didn’t used to be true: MongoDB does vector search natively. Your embeddings live in the same documents as your data, queried with a normal aggregation pipeline. No separate vector database to run, sync, or pay for.

Here’s the shape of the app — a standard MERN split, with the LLM logic living safely on the server:

MongoDB Atlas — stores documents, chunks, embeddings, and chat history. Free M0 tier includes vector search.
Express + Node.js — the API: handles uploads, embedding, retrieval, and talking to the LLM. All API keys live here, never in the browser.
React — the chat UI, rendering streamed tokens as they arrive.
The glue: the Vercel AI SDK — one library for streaming, provider-switching, and tool calling, on both server and client.

The one piece of setup that’s new: the vector index

After creating a free cluster on Atlas, you define a vector search index on the collection that will hold your chunks. This tells MongoDB how to search the embedding field. In the Atlas UI (Atlas Search → Create Index → JSON editor), or via code:

{
  "fields": [
    {
      "type": "vector",
      "path": "embedding",
      "numDimensions": 1536,
      "similarity": "cosine"
    },
    { "type": "filter", "path": "userId" }
  ]
}

Under the hood. numDimensions must exactly match your embedding model’s output (1536 for text-embedding-3-small). similarity can be cosine, euclidean, or dotProduct — cosine is the safe default. The filter field on userId is what lets you scope searches per-user, so visitors only ever retrieve their own documents — essential the moment your demo is public. Atlas uses HNSW for approximate nearest-neighbor search under the hood.

That’s the only “AI-specific infrastructure” in the whole project. Everything else is ordinary Express and React.

Step 4 · Ingesting documents: upload, parse, chunk

When a user uploads a file, three things happen on the server: we accept the upload, extract its text, and split that text into bite-sized chunks.

Accepting uploads is standard Express (use multer). Extracting text from a PDF is a one-liner with pdf-parse (v2 fork, TypeScript-native — see note in code):

import { PDFParse } from "pdf-parse"; // requires the v2 fork, not the default pdf-parse@1

// `buffer` is the uploaded file from multer
const parser = new PDFParse({ data: buffer });
const { text } = await parser.getText();

Why we chunk — and how big

You can’t embed an entire 50-page PDF as one vector; the meaning gets blurred into mush, and you’d feed the model far more than it needs. So we slice the text into passages. Each chunk becomes one searchable unit.

// ~1 token = ~4 characters, so 2000 chars = ~500 tokens.
// We overlap chunks so a sentence split across a boundary
// still appears whole in at least one chunk.
function chunkText(text, size = 2000, overlap = 200) {
  const chunks = [];
  for (let i = 0; i < text.length; i += size - overlap) {
    chunks.push(text.slice(i, i + size).trim());
  }
  return chunks.filter(Boolean);
}

In plain English. Think of chunking like cutting a long article into index cards. Too big and each card covers too many topics to be useful; too small and you lose context. A few hundred words per card, with a little overlap so sentences don’t get sliced in half, is the reliable starting point.

Under the hood. Start with recursive character splitting at ~400–512 tokens with 10–20% overlap — the pragmatic default (~85–90% retrieval recall). Semantic chunking can add ~2–3% recall but costs roughly 14× more to index, so only graduate to it when your evaluation metrics demand it. One caveat: at least one 2026 analysis found overlap added no measurable benefit in its setup while raising indexing cost — so treat the overlap figure as a starting point to validate against your own data, not a law.

Step 5 · Embedding & storing vectors in MongoDB

Now we turn each chunk into a vector and save it. The AI SDK’s embedMany batches the whole array efficiently, then we write one MongoDB document per chunk — text and vector together, tagged with the owner and source.

import { embedMany } from "ai";
import { openai } from "@ai-sdk/openai";

const chunks = chunkText(text);                 // from Step 4

const { embeddings } = await embedMany({
  model: openai.embedding("text-embedding-3-small"),
  values: chunks,                               // array of strings
});

await db.collection("chunks").insertMany(
  chunks.map((chunk, i) => ({
    userId,                                     // who owns it
    source: filename,                           // where it came from
    text: chunk,                                // the passage itself
    embedding: embeddings[i],                   // the 1536-dim vector
    chunkIndex: i,
    createdAt: new Date(),
  }))
);

That’s ingestion done. The vector is just an array of floats stored on a normal document — no special database, no migration. Upload a 30-page PDF and you’ve got a few dozen searchable, meaning-aware chunks sitting in MongoDB.

Under the hood. embedMany auto-batches large arrays, so you can hand it hundreds of chunks without managing request limits yourself. Store rich metadata (page number, section heading, document ID) alongside each chunk now — you’ll want it later for citations, filtering, and “parent-document” retrieval. This is the step you run once per upload, not once per question.

Step 6 · Retrieval: finding the right passages

Here’s the payoff for all that setup. To answer a question, we embed the question with the same model, then run a single $vectorSearch aggregation to pull the closest chunks. This is the whole of “search by meaning,” in one query:

import { embed } from "ai";
import { openai } from "@ai-sdk/openai";

const { embedding } = await embed({
  model: openai.embedding("text-embedding-3-small"),
  value: userQuestion,
});

const passages = await db.collection("chunks").aggregate([
  {
    $vectorSearch: {
      index: "vector_index",
      path: "embedding",
      queryVector: embedding,
      numCandidates: 150,        // over-fetch, then narrow
      limit: 5,                  // keep the best 5
      filter: { userId: { $eq: currentUserId } }
    }
  },
  {
    $project: {
      _id: 0,
      text: 1,
      source: 1,
      score: { $meta: "vectorSearchScore" }
    }
  }
]).toArray();

You now have the five most relevant passages, each with a similarity score. Feed those into the model as context and you have working RAG. But before we generate, two notes that separate a toy from something good:

Under the hood. $vectorSearch must be the first stage in the pipeline. numCandidates is the approximate-search breadth — it must be ≥ limit, and a common heuristic is 10–20× your limit (here 150 for a limit of 5). The filter on userId uses the field we declared in the index, enforcing per-user isolation efficiently. Use the score as a relevance gate: if the top result scores below ~0.75, it’s often better to answer “I don’t have enough information” than to let the model hallucinate from weak matches.

The single biggest quality upgrade you’ll make later isn’t a bigger model — it’s hybrid search + reranking (we cover it in “Where to go next”). For now, vector search alone is plenty to ship.

Step 7 · Making retrieval agentic

So far the server retrieves before calling the model — a fixed pipeline. To make it agentic, we flip the control: we describe retrieval as a tool, hand it to the model, and let the model decide when (and how often) to call it. This is the “hot” part of 2026 — and with the AI SDK it’s remarkably little code.

import { streamText, tool, embed, stepCountIs } from "ai";
import { openai } from "@ai-sdk/openai";
import { z } from "zod";

// 1) Retrieval, described as a tool the model can call
const searchDocuments = tool({
  description: "Search the user's uploaded documents for passages " +
               "relevant to a question. Call this whenever you need facts.",
  inputSchema: z.object({
    query: z.string().describe("a focused search query"),
  }),
  execute: async ({ query }) => {
    const { embedding } = await embed({
      model: openai.embedding("text-embedding-3-small"),
      value: query,
    });
    return db.collection("chunks").aggregate([
      { $vectorSearch: {
          index: "vector_index", path: "embedding",
          queryVector: embedding, numCandidates: 150, limit: 5,
          filter: { userId: { $eq: currentUserId } } } },
      { $project: { _id: 0, text: 1, source: 1,
          score: { $meta: "vectorSearchScore" } } },
    ]).toArray();
  },
});

// 2) Let the model run the loop: think -> search -> (search again) -> answer
const result = streamText({
  model: openai("gpt-4o-mini"),     // swap to any provider in one line
  system: "Answer ONLY using passages returned by searchDocuments. " +
          "Cite the source. If the passages don't contain the answer, " +
          "say you don't know - do not guess.",
  messages,               // from req.body — full chat history from the client
  tools: { searchDocuments },
  stopWhen: stepCountIs(5),         // hard cap on the agentic loop
});

return result.toUIMessageStreamResponse();

Read what that does, because it’s genuinely different from classic RAG: the model receives the question, decides on its own to call searchDocuments with a query it wrote, reads the results, and may call it again with a refined query before answering. For “compare the 2024 and 2025 refund policies,” it can naturally run two searches and synthesize. You didn’t orchestrate that — the model did.

Important 2026 change. If you learned the AI SDK before v5: maxSteps was removed from the client. Multi-step tool loops are now controlled server-side with stopWhen (e.g. stepCountIs(5)). This cap is also your cost safety rail — without it, a confused agent could loop and run up your bill.

Under the hood. The system prompt is doing heavy lifting for safety and grounding: “answer only from retrieved passages” plus “say you don’t know” is your first and cheapest defense against hallucination. The Zod inputSchema gives the model a typed contract for the tool’s arguments. toUIMessageStreamResponse() emits a standard SSE stream the React client consumes natively. Want it reusable across other AI clients (Claude Desktop, Cursor, etc.)? Expose this same retrieval as an MCP server — overkill for a single app, but the natural next step if your tools should be shared.

Step 8 · Streaming to React, then deploying for free

The backend streams tokens; the frontend renders them as they land. The AI SDK’s useChat hook handles the entire streaming lifecycle, so your component stays tiny:

"use client";
import { useChat } from "@ai-sdk/react";
import { DefaultChatTransport } from "ai";
import { useState } from "react";

export default function Chat() {
  const [input, setInput] = useState("");
  const { messages, sendMessage, status } = useChat({
    transport: new DefaultChatTransport({ api: "/api/chat" }),
  });

  return (
    <div>
      {messages.map((m) => (
        <div key={m.id}>
          <strong>{m.role}: </strong>
          {m.parts.map((p, i) =>
            p.type === "text" ? <span key={i}>{p.text}</span> : null
          )}
        </div>
      ))}

      <input value={input} onChange={(e) => setInput(e.target.value)} />
      <button
        disabled={status !== "ready"}
        onClick={() => { sendMessage({ text: input }); setInput(""); }}
      >
        {status === "streaming" ? "Thinking..." : "Ask"}
      </button>
    </div>
  );
}

The status field (ready / submitted / streaming / error) gives you loading and disabled states for free. Tokens appear live as the model writes them — the experience people now expect from any AI app.

Putting it on its own website — for free

This is the part that turns a tutorial into a portfolio piece. Three boxes, three free tiers:

Vercel — React frontend, free Hobby tier, global CDN, custom domain, auto-deploy from GitHub.
Render — Node/Express backend, free web service (sleeps when idle), where streaming and keys live.
MongoDB Atlas M0 — database + vector search, permanently free, 512 MB, limited Vector Search index capacity (verify current limits in Atlas docs).

Total cost for a low-traffic demo: $0–$5/month.

Under the hood. Run the streaming endpoint on the long-lived backend (Render/Railway), not a short serverless function that can time out mid-stream. Know the free-tier edges: Render free services spin down after ~15 min idle (a ~30–60s cold start on the next request — fine for a portfolio), and Atlas M0 caps at 512 MB and limited Vector Search index capacity. When you outgrow them, a dedicated Atlas tier and an always-on backend plan are the upgrade path.

Cost controls so you never get a surprise bill

Set a hard spend cap in your LLM provider dashboard. Non-negotiable for a public demo.
Default to a cheap model; cap max_output_tokens; keep the agentic stepCountIs low.
Add per-user / per-IP rate limiting and an auth wall so bots can’t drain your quota.
Log token usage per request (the SDK’s onFinish callback) so you can see costs before they surprise you.
Keep every API key on the server. A key shipped to the browser is a key that will be stolen.

Common pitfalls (and the fixes)

The semantic gap. Your question and the document use different words and vector search misses the match. Fix: add hybrid search (keyword + vector).
Context dilution. You retrieve 10 chunks when only 2 are relevant, and the noise degrades the answer. Fix: rerank, then keep a tighter top-k.
Chunk-boundary amnesia. The answer is split across two chunks and neither is retrieved whole. Fix: overlap, or parent-document retrieval.
Confident nonsense. The model answers from weak matches as if certain. Fix: a similarity-score threshold plus a system prompt that permits “I don’t know.”
Reaching for the agent loop too early. Simple lookups don’t need multi-hop reasoning — they need one fast search. Fix: gate the agentic path to genuinely complex questions.

Where to go next

You’ve shipped a working agentic RAG app. Three upgrades, in priority order:

Hybrid search + reranking — the highest-ROI quality jump. Run keyword (Atlas full-text via $search) and vector search, fuse them with Reciprocal Rank Fusion (RRF), then rerank the top 20–50 candidates with a cross-encoder (Cohere, Voyage, or self-hosted BGE) and keep the best handful. Benchmarks routinely show reranking as the single biggest accuracy gain.
Better ingestion — messy PDFs with tables and multi-column layouts need a real parser (LlamaIndex’s LiteParse, Unstructured, or Docling) rather than plain text extraction.
Make it shareable via MCP — expose your retrieval as a Model Context Protocol server so other AI clients can use the same tool. Worth it once your tools outlive this one app.

That’s the whole arc: from “an LLM can’t see my data” to a public, agentic, document-grounded chat app that costs about nothing to run. The plumbing finally got out of the way — what you build on top of it is the interesting part. Now go put something on that empty portfolio URL.

Frequently asked questions

What is agentic RAG, exactly?

Agentic RAG turns retrieval into a tool the model calls on demand. Instead of one fixed “retrieve then answer” pass, the model plans, searches, judges whether the results are sufficient, and searches again until it has enough evidence — then answers. It’s slower and costs more tokens, but it handles complex, multi-step questions that one-shot RAG can’t.

Do I need a separate vector database?

No. On the MERN stack you store embeddings inside your normal MongoDB documents and query them with the $vectorSearch aggregation stage in MongoDB Atlas. For the vast majority of projects, that removes the need for a dedicated vector database entirely.

What’s the cheapest LLM for a RAG app in 2026?

For learning, the most generous free tiers are Google Gemini Flash/Flash-Lite and Groq. For the cheapest paid frontier-class model, DeepSeek is usually lowest. Prices change monthly — confirm on the provider’s pricing page. The durable strategy is to route bulk work to a cheap model and reserve a frontier model for the final answer only.

How much does it cost to run?

A portfolio-grade demo runs at roughly $0–$5/month: MongoDB Atlas free M0, a free LLM tier, embeddings at ~$0.02 per million tokens, and free deploy tiers on Vercel and Render. Set a provider spend cap and rate limits so a public demo can’t surprise you.

LangChain, LlamaIndex, or the Vercel AI SDK?

For a MERN streaming chat app, the Vercel AI SDK plus direct MongoDB vector queries is the lighter, recommended path in 2026. Reach for LlamaIndex.TS if your main challenge is heavy document ingestion, or LangChain.js/LangGraph for complex multi-agent orchestration. For ~90% of RAG web apps, the AI SDK is the right call.

Can I run this fully offline with a local model?

Yes. Ollama runs open models locally and exposes an OpenAI-compatible endpoint, so your code barely changes. Use a local embedding model like nomic-embed-text too. It’s ideal for development and privacy-sensitive data; for a low-traffic public demo, hosted free tiers are usually simpler and cheaper.

A note on accuracy. LLM pricing, model names, and SDK APIs change fast. Every figure here is a 2026 snapshot — verify current prices on each provider’s pricing page and current method names in the Vercel AI SDK docs before shipping to production. Code samples are illustrative walkthroughs, not drop-in files. Images: “Visualising AI” by Google DeepMind on Unsplash, free under the Unsplash License.

TL;DR

You can build and ship a production-shaped, agentic RAG “chat with your documents” app entirely on MERN for about $0/month: MongoDB Atlas’ free tier (with built-in vector search), a free LLM tier (Gemini Flash or Groq), embeddings at $0.02 per million tokens, and free deploys on Vercel + Render.
The 2026 stack is leaner than you think: MongoDB Atlas Vector Search (no separate vector database), the Vercel AI SDK for streaming + tool calling, and “agentic retrieval” — where the model itself decides when and what to search — instead of the old retrieve-once-then-answer pipeline.
Pick models by job, not by brand. Route cheap, high-volume work to Gemini Flash-Lite, DeepSeek, or Groq-hosted Llama; reserve a frontier model only for the final answer when quality matters. Self-hosting with Ollama only beats hosted APIs above heavy, sustained traffic.
By the end you’ll understand embeddings, chunking, vector retrieval, tool calling, token streaming in React, and how to put the whole thing on its own public website — with cost controls so you never get a surprise bill.

The Best Claude Setup (That Works on Any AI Tool)

Harshdeep Singh — Thu, 04 Jun 2026 23:16:05 +0000

Here is a confession that might sound odd in a guide about setting up Claude Code: the goal is not to marry Claude Code. The goal is to build a setup so portable that if something better ships next month — OpenAI's Codex, Cursor, Windsurf, whatever wins the week — you could pack up everything you have built and move in an afternoon. Your instructions, your custom tools, your reusable workflows: all of it should come with you.

That sounds like a strange thing to optimize for. Most "best setup" posts try to lock you deeper into one tool. But the AI coding world is moving too fast for that bet to be safe, and — happily — a small set of open standards now make portability the default instead of a fantasy. If you are a working engineer, this is how you stop re-plumbing your environment every time you switch editors. And if you are a vibe coder — someone who builds mostly by describing what you want in plain English and steering the AI, a style Andrej Karpathy named in early 2025 — this is how you get a professional-grade setup without months of fiddling.

The short version: keep your AI's instructions, tools, and skills in plain, open formats you own — not buried in one vendor's settings. Three standards make this work: MCP for tools, AGENTS.md for instructions, and Agent Skills for reusable know-how. Learn those, and the question "which coding agent should I use?" stops being scary.

Three standards do the heavy lifting here, and we will spend most of our time on them: MCP AGENTS.md Agent Skills. Let's build up to a setup you would actually be happy to leave.

Think of your setup as interchangeable bricks, not a sculpture glued to one base. Photo by Xavi Cabrera on Unsplash.

Why portability is the whole game now

In the last 18 months, more than a dozen serious AI coding agents have shipped: Claude Code, OpenAI Codex, Cursor, Windsurf, Zed's agent, Aider, Cline, Continue, Gemini CLI, and more. Each one claims to be the fastest or the smartest. Some genuinely leapfrog the others — for a few weeks, until the next release.

Here is the trap. If you pour weeks into one tool — memorizing its config files, hand-tuning its rules, wiring up its integrations — you have quietly built a switching cost. The day a clearly better tool arrives, you do the math on re-learning everything and you stay put, not because your tool is best, but because leaving hurts. That is the real lock-in. It is not a contract; it is your own sunk effort.

The fix is to treat the agent as a replaceable part and invest in the layer underneath it. The engineer Geoffrey Huntley has made this point sharply: the agents themselves are becoming commodities, and the durable advantage is the standards layer you build around them. Put your effort there, and any agent becomes a front-end you can swap.

An analogy: remember when every phone had its own charger, and switching brands meant a drawer full of dead cables? USB-C fixed that by agreeing on one shape. You buy a charger once; it works with the next phone, and the one after. The standards below are USB-C for your AI setup. Learn them once, and your tools become things you plug into — not things you are wired into.

The three open standards that set you free

You do not need to memorize specs. You need to understand what each standard is for, because once you see the shape of the problem each one solves, the portability falls out naturally. They split cleanly: one is for tools, one is for instructions, one is for skills.

MCP — one way to plug in tools

The Model Context Protocol (MCP) is an open standard, created and open-sourced by Anthropic in November 2024, for connecting an AI to outside tools and data — your database, your GitHub, your Notion, your company's internal services.

If you have used a code editor, you have already benefited from this idea. Editors once needed custom code to support each programming language, until the Language Server Protocol let any editor talk to any language through one shared interface. MCP does the same trick for AI and tools. Instead of every AI app writing a custom integration for every service — an N-times-M explosion — you write one MCP server for a service, and every MCP-compatible host can use it. The math collapses from N times M down to N plus M.

Mechanically, a host (Claude Code, Cursor, ChatGPT, and others) runs a client that talks to your server over one of two transports: stdio for a local tool running as a subprocess, or streamable HTTP for a remote service. The server exposes tools, read-only resources, and prompt templates; the host stays in charge of what the model is actually allowed to touch. Crucially, the configuration looks almost identical across tools — a small block naming each server:

{
  "mcpServers": {
    "postgres": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-postgres", "postgresql://localhost/mydb"]
    },
    "github": {
      "url": "https://api.githubcopilot.com/mcp/",
      "headers": { "Authorization": "Bearer ${GITHUB_TOKEN}" }
    }
  }
}

Move from Claude Code to Cursor or Codex and you copy that block over, rename one key if needed, and your tools come with you. One safety note worth tattooing on your brain: an MCP server can run code and reach real systems, so only install servers you trust, keep secrets in environment variables like the ${GITHUB_TOKEN} above rather than hard-coding them, and review what each tool is permitted to do.

MCP is the wiring: many tools, one shared protocol. Photo by Alina Grubnyak on Unsplash.

AGENTS.md — a README for your AI

Every project has unspoken rules: how to install it, how to run the tests, which folders are off-limits, what "good code" means here. A human teammate learns these over weeks. An AI agent starts fresh every session and will happily reinvent your conventions unless you write them down.

That is what AGENTS.md is — a plain Markdown file at the root of your repo that tells any coding agent how to work on your project. No special syntax, no required fields. Think of it as a README written for your AI instead of for a human. It was introduced alongside OpenAI's Codex and is now supported by Codex, Cursor, Aider, Cline, Windsurf, and others — with Gemini CLI requiring a GEMINI.md symlink as described below. A good one leads with commands, because the agent refers back to them constantly:

# Project: Acme API

## Commands
- Install: `pnpm install`
- Dev server: `pnpm dev`
- Test (run before every PR): `pnpm test`
- Lint & typecheck: `pnpm lint && pnpm typecheck`

## Conventions
- TypeScript strict mode. Never use `any`.
- Do not edit files in `/generated` — they are built from schemas.
- Write a test for every new endpoint.

Here is the one wrinkle on the Claude side. Anthropic's tool uses its own file, CLAUDE.md, and as of mid-2026 Claude Code does not read AGENTS.md natively — a gap the community has been loudly requesting for months. The portable move is to keep one source of truth in AGENTS.md and point the others at it with a symlink, so you never maintain the same rules twice:

# One canonical file, linked everywhere your tools look
ln -s AGENTS.md CLAUDE.md
ln -s AGENTS.md GEMINI.md

One file, every tool, zero duplication. One caution worth knowing: do not blindly auto-generate this file and walk away. Research has consistently shown that bloated, auto-generated instruction files tend to hurt more than they help — the important rules get buried in noise and the agent learns to half-ignore them. Keep it short, hand-curated, and honest about what is genuinely non-obvious.

Agent Skills — teach once, reuse everywhere

The third standard is the one that feels like magic the first time it clicks. An Agent Skill is a folder containing a SKILL.md file — Markdown instructions for a specific task — plus any optional scripts or reference docs that task needs. Anthropic's own framing is the clearest: building a skill is like writing an onboarding guide for a new hire. You capture how to do something once, and the agent follows it forever after.

The clever part is progressive disclosure. At startup the agent only reads each skill's name and one-line description — a few dozen words — so a hundred skills cost almost nothing. Only when a task actually matches does it open the full instructions, and only then does it reach for the bundled scripts. Your context window stays clean until the moment a skill is relevant.

pdf-form-filler/
├── SKILL.md          # name + description + instructions
├── scripts/          # code the agent runs (stays out of context)
│   └── fill.py
├── references/       # long docs, loaded only when needed
└── assets/           # templates, fonts, icons

Because a skill is just Markdown and files in a folder, it is inherently portable — you can point Codex or Gemini CLI at a skills folder, tell it to read the SKILL.md, and it simply works. Agent Skills are now supported across a growing list of tools: Claude Code, Codex, Cursor, Gemini CLI, and more. Write your "how we deploy" or "how we write migrations" skill once, and it travels.

So when do you reach for which? They are not competitors; they answer different questions. Here is the cheat sheet:

Mechanism	What it gives the agent	Reach for it when
Agent Skill	A repeatable procedure — the how	You keep re-explaining the same multi-step workflow
MCP server	A connection to an outside system — the reach	The agent needs live data or a third-party API
Subagent	A fresh worker with its own clean context	A job is large, or you want an independent reviewer
AGENTS.md	Always-on project facts and rules	Conventions every session should already know
Hook	Enforcement that always runs, no exceptions	Something must happen — formatting, blocking secrets

Does "switch tomorrow" actually hold?

It is a fair thing to be skeptical about — a portability promise is only as good as the tools honoring it. So here is an honest snapshot of where the major agents stood in mid-2026 on all three standards. It is not perfect across the board, but it is good enough that moving your setup is a copy-and-tweak job, not a rebuild.

Agent	MCP	AGENTS.md	Agent Skills
Claude Code	Yes	Via CLAUDE.md / symlink	Yes, native
OpenAI Codex	Yes	Yes (its home format)	Yes
Cursor	Yes	Yes	Yes
Windsurf	Yes	Yes	Yes
Zed	Yes	Yes	Via hosted agents
Aider	Limited	Yes	Via conversion
Cline	Yes	Yes	Yes
Gemini CLI	Yes	Yes (GEMINI.md)	Yes

The weak spots are real and worth naming — Aider's MCP support is limited, and a few tools only run skills through their cloud agents — but the spine holds. Your tools, instructions, and skills are written in formats more than one vendor understands.

Your portable setup, layer by layer

Now let's assemble it. The trick is two tiers: one that lives with each project and travels in its Git repo, and one that is personal to you and follows you across every machine and every project.

Tier 1 — the project repo (commit this)

At the root of each project, keep an AGENTS.md as the single source of truth, with CLAUDE.md and friends symlinked to it. Add an .mcp.json for the tools that project needs (databases, issue trackers), with secrets pulled from environment variables. Put team-shared skills in a folder like .agents/skills/. Commit all of it. The payoff: a teammate — or you, six months later — clones the repo and the AI is instantly productive, with the same rules and the same tools, no setup call required.

Tier 2 — your personal dotfiles

Your personal taste — how you like commit messages written, your favorite skills, your global tool configs — belongs in a dotfiles repo you carry everywhere. The reliable pattern is to keep one canonical copy of each file and symlink it into the locations each tool expects, using a tiny helper so a fresh machine is set up in seconds:

safe_symlink() {
  local src=$1 dst=$2
  [ -L "$dst" ] && rm "$dst"
  [ -e "$dst" ] && mv "$dst" "$dst.bak"
  mkdir -p "$(dirname "$dst")"
  ln -s "$src" "$dst"
}

safe_symlink "$DOTFILES/AGENTS.md" "$HOME/.claude/CLAUDE.md"
safe_symlink "$DOTFILES/AGENTS.md" "$HOME/.codex/AGENTS.md"
safe_symlink "$DOTFILES/skills"    "$HOME/.claude/skills"

Tools like GNU Stow and chezmoi exist to manage exactly this, and chezmoi is the kinder choice if you bounce between macOS, Linux, and Windows. One Windows gotcha to save you an hour: symlinks there need Developer Mode or admin rights and core.symlinks=true in Git, and mixing WSL and native-Windows symlinks does not work — pick one world and stay in it.

Advanced Claude Code moves that survive the move

Everything above keeps you portable. But within Claude Code there are sharper techniques worth knowing — and because they are built on the same standards and plain files, they travel with you too. The single mental model that explains most of them: Claude's context window fills up fast, and quality drops as it fills. Almost every good habit is really about protecting that space.

Subagents are separate Claude instances with their own clean context and their own narrow tool permissions. Hand a big exploration or an independent code review to a subagent, and only its summary comes back — your main conversation stays uncluttered. A favorite pattern: after a long stretch of work, spin up a fresh reviewer subagent that sees only the final diff and the requirements, with no memory of the messy path that got there.

Slash commands and hooks are where you encode habits. A custom command is a saved prompt you trigger by name. A hook is different and more powerful: it is a script that fires automatically at a set moment — and this is the key distinction — your instruction files only suggest behavior, while a hook guarantees it. Want code formatted every single time a file is written, no exceptions?

{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "Write|Edit",
        "hooks": [{ "type": "command", "command": "pnpm prettier --write \"$CLAUDE_FILE_PATHS\"" }]
      }
    ]
  }
}

Plan mode is the antidote to an agent that charges off in the wrong direction. Toggle it and Claude reads, analyzes, and proposes a plan without touching your files. The workflow Anthropic recommends is simple and worth internalizing: explore, plan, implement, commit — in that order, with your eyes on the plan before any code is written.

Worktrees and headless mode are for scale. Git worktrees let you run two to four Claude sessions in parallel on separate branches without collisions. Headless mode — the much-missed claude -p flag — turns Claude into a command-line citizen you can pipe into and wire into CI:

# Pipe an error log straight into a headless review
cat error.log | claude -p "Find the root cause and propose a fix"

# CI-safe: cap turns and restrict tools
claude -p "Run the test suite and fix failures" \
  --max-turns 12 \
  --allowedTools "Bash(pnpm test:*)" "Edit"

And one underused habit: ask Claude to explain its reasoning as it works — add "and briefly explain each decision" to your prompt. You absorb the why as you read the diff, not just the what. For a new codebase, this single habit is worth the extra few lines of output.

If you are a vibe coder, start here

If the section above felt like a lot, this part is for you. Vibe coding — building by describing and steering rather than hand-writing every line — is a real and legitimate way to ship. But there is an honest catch worth saying plainly: the impressive demo is the easy 80%. The boring 20% — real authentication, a real database, payments, deployment, security — is where projects quietly fall apart. Security researchers scanning AI-assisted apps throughout 2025 found troubling rates of exposed credentials — API keys hardcoded in frontends, admin passwords committed to public repos — almost all of it preventable with a single review step.

So here is the short, high-leverage starter kit. None of it slows you down much, and all of it keeps you out of the ditch:

Treat the AI like a sharp junior engineer with tools, not a magic box. Build features in layers — get auth working, then the database, then payments — rather than asking for everything in one breath.
Give it a role that changes its priorities. "Act as a senior engineer who has been burned by sloppy payment code" produces noticeably more careful, defensive work.
Always run an adversarial reviewer at the end — a fresh subagent that sees only the diff and is asked to find what is wrong. This one habit catches most of those exposed-secret disasters.
Do not launch Claude Code from your home folder — it then has reach over your SSH keys and tokens. Work inside a dedicated project folder.
Turn on the "Learning" output style and let the tool explain itself. You will absorb the why, not just collect the what.

You do not need a 10-monitor battlestation to build well — you need good defaults. Photo by Jakub Żerdzicki on Unsplash.

Anti-patterns to skip

A few traps catch nearly everyone. Knowing them in advance saves real time:

Instruction-file bloat. A giant AGENTS.md backfires — the important rules get lost in the noise and the agent half-ignores them. For each line, ask whether removing it would cause a mistake. If not, cut it.
MCP overload. Connecting fifteen servers globally floods the agent with tool options and it starts picking wrong. Add tools per project, only where they earn their place.
No way to check the work. This is the big one. If you do not give the agent a test, a build, or a linter it can run, it stops when the code merely looks done — and you become the error-checker. Give it a check it can run, and have it show you the passing output rather than just claiming success.
Rules where guarantees belong. "Always format the code" written in an instruction file is a suggestion it may drop. If it must happen, make it a hook.
Keeping it all to yourself. If your .mcp.json and shared skills are not committed, your teammates are flying blind. Portability includes your team.

The point

The best Claude Code setup is not a clever pile of configuration locked inside one app. It is a small, portable kit — instructions in AGENTS.md, tools behind MCP, know-how in Skills — written in open formats you own and can carry anywhere. Build it that way and you are not betting on which agent wins the next release cycle. You are making that question stop mattering, because whatever you reach for tomorrow, your setup is already waiting for you there.

You can start in the next ten minutes. Write a short, honest AGENTS.md for your current project. Symlink CLAUDE.md to it. Then take the one workflow you keep re-explaining to your AI, and move it out of your head and into a SKILL.md. That is the whole foundation — and it already belongs to you.

Sources & further reading

Photography & AI: a faster, smarter 2026 workflow

Harshdeep Singh — Thu, 04 Jun 2026 05:00:35 +0000

Ask a working photographer where their week actually goes, and almost none of them will say “behind the camera.” The shoot is the fun part. The grind is everything after it: culling thousands of frames, matching edits across an entire gallery, color-grading video, chasing invoices, and answering the same five client emails for the hundredth time.

That grind is exactly what AI is good at. Industry surveys now put AI adoption among professional photographers in the low-90s percent — it has quietly gone from novelty to default. Used well, it doesn’t replace your eye; it removes the repetitive work standing between you and the next shoot. Here’s what a modern, AI-assisted pipeline looks like, from capture to paid invoice.

The AI Photo editing Pipeline

It starts before you touch a single slider. AI culling tools like Aftershoot and Imagen sort a 3,000-frame wedding in minutes — flagging the sharp shots, catching closed eyes, and grouping near-duplicates so you choose from a shortlist instead of the whole card.

Then comes the part that actually matters: editing in your style. This is where 2026 tools pull ahead of presets. A preset applies the same math to every photo; feed a “bright and airy” preset a dark reception and you get mush. Imagen’s Personal AI Profile works differently — point it at a few thousand of your previously edited images and it studies how you handle exposure, white balance, and color, then applies that judgment to a fresh set.

For pixel-level rescue, Topaz Photo AI handles denoise, sharpening, and Gigapixel upscaling; Lightroom’s AI denoise and masking isolate skies and subjects in a click; and Photoshop’s Firefly Generative Fill removes a stray tourist or extends a cramped background without a clone-stamp marathon. The pattern is consistent: AI does the first 80%, and you spend your time on the 20% that carries your signature.

AI in the video pipeline

If you shoot hybrid, video is where AI buys back the most time, because video post is where the most time disappears. The headline shift is text-based editing: your footage is transcribed, and you cut the video by editing the transcript — delete a sentence, and the matching frames vanish. Rough cuts that used to eat an afternoon now take minutes.

Color is the other big win. DaVinci Resolve’s Magic Mask isolates a subject for targeted grading, and neural color matching lines up clips shot on different bodies — your A-cam, a second camera, and the drone — into one consistent look. Imagen’s video tool, launched at NAB 2026, brings the same learn-your-style grading photographers already enjoy straight to the timeline.

The finishing touches are increasingly automatic too: smart reframing turns a 16:9 edit into vertical 9:16 and square 4:5 cuts for social, AI leveling and noise removal clean up audio without manual EQ, and upscaling pushes older footage to 4K. The result is same-day turnarounds on work that used to take a week.

AI for the business: clients, CRM & delivery

Here’s the unglamorous truth no gear review mentions: the thing most likely to sink a photography business isn’t bad photos — it’s bad admin. Inquiries that go cold, contracts that sit unsigned, invoices that slip a month. A well-run CRM is the fix, and the payoff is real: photographers consistently report clawing back the better part of a working day every week once the back office runs itself.

Platforms built for this — HoneyBook, Dubsado, Studio Ninja, Táve, Bloom, Sprout Studio — automate the entire client journey: an inquiry triggers an instant reply, a proposal, a contract, and a payment schedule, with reminders firing on their own. The AI layer goes further. Sprout Studio drafts your emails and questionnaires, HoneyBook plugs into post-production tools so editing and client management finally talk to each other, and gallery platforms now use face recognition so guests find and buy their own photos without you lifting a finger.

Speed is the quiet advantage. The studio that answers an inquiry in five minutes books the client that the studio answering in five hours was still drafting a reply to. AI simply makes those five minutes happen while you’re on a shoot.

AI handles the first 80%. Your taste is the last 20% — and that’s the only part a client is really paying for.

Tools mentioned

AftershootImagenTopaz Photo AILightroomPhotoshopFireflyDaVinci ResolveCapCutHoneyBookDubsadoStudio NinjaSprout Studio

Where the Human still wins

None of this is about handing your work to a machine. AI is leverage, not authorship. It can match your edit, but it can’t decide what’s worth photographing, read a nervous couple on a wedding morning, or build the trust that turns a one-off booking into a decade of referrals. That judgment is the moat — and it’s getting more valuable, not less.

So don’t try to automate everything at once. Find your single biggest bottleneck — for most photographers it’s culling or the CRM — and hand that one to AI first. Win back those hours, then reinvest them where they compound: better shoots, sharper craft, and a client experience no software will ever replicate.

TL;DR

The shoot was never the bottleneck. AI’s real value is post-shoot — it clears the repetitive work, not your creative judgment.
Photo editing: AI culls thousands of frames and edits in your learned style (Imagen, Aftershoot), while Topaz, Lightroom, and Firefly handle pixel-level fixes. You keep the final 20%.
Video: text-based editing, neural color-matching across cameras, and auto reframe + audio cleanup turn week-long edits into same-day deliveries.
Business: a CRM plus AI automations (HoneyBook, Dubsado, Sprout Studio) save roughly a day a week — and a five-minute inquiry reply wins the booking.
Start small: automate one bottleneck (culling or your CRM) first, then reinvest the hours into better work.

Full Stack Developer Portfolio Lessons: What I Learned Building 10+ Projects

Harshdeep Singh — Tue, 02 Jun 2026 21:33:43 +0000

I applied for a role at a mid-sized SaaS company about two years into my career. Strong company, interesting problem, good pay. I sent my application, got a recruiter callback, and then nothing for two weeks. When the feedback finally came: "We went with candidates with a stronger portfolio presence."

I had 23 GitHub repositories. I had a portfolio site. I had projects. What I didn't have — and what I didn't understand for another six months — was a portfolio that told a story. I had code. Not evidence of thinking, decision-making, or the ability to ship something real.

I've since built, rebuilt, and advised on a lot of developer portfolios. I've seen what gets people calls and what gets them ghosted. This isn't a guide about which framework to use or how to pick colors. It's about what actually moves the needle — the things I wish someone had told me in year one.

Lesson 1: Two Great Projects Beat Twenty Mediocre Ones

The instinct is to fill the portfolio. More projects = more evidence of experience. This is wrong.

A hiring manager or engineering lead looking at your portfolio has about three minutes. They're going to look at your two or three most prominent projects, click one or two live demo links, and form an opinion. If they see twenty repositories and most of them are "Todo App v2," "Weather App," "Netflix Clone," "Portfolio v1 through v6" — they've already categorized you as someone who builds tutorials, not someone who builds things.

The better approach: three to five projects, each with:

A real problem it solves (not "I wanted to learn React")
A live deployment that actually works
A README that explains why you made the decisions you made
Enough complexity to have generated at least one interesting engineering problem

Projects that tend to work: tools you built because you were frustrated with an existing tool, apps solving problems you personally had, projects where you integrated with a real API or real data source, anything with a live user base (even 10 users counts).

Projects that tend not to work: tutorial clones (unless heavily modified), apps that only run locally, projects that stop at the MVP and never got deployed, apps with the same name as thousands of other developer portfolios ("My Todo App," "My Weather App").

If you have 20 repos, that's fine. Pin your three best to your GitHub profile. Don't make people wade through everything — curate it.

Lesson 2: Case Studies Beat Code Screenshots

Here's the thing about showing a screenshot of your app: everyone can make an app look good in a screenshot. Filters, cropping, ideal state data. A screenshot shows what you built. It tells me nothing about how you think.

A case study shows how you think. And how you think is what you're being hired for.

A case study doesn't have to be a five-page document. Two or three paragraphs on each project covering:

The problem. What did you set out to solve? Be specific. Not "I wanted to learn Next.js" — that's not a problem. "Resume submissions were getting lost in email threads, so I built a tool that…" — that's a problem.
Your approach and the tradeoffs you considered. What did you think about? What did you try first? What didn't work? This is where you demonstrate that you can make technical decisions, not just execute instructions.
What you shipped. Not every feature you imagined — what you actually built and deployed.
What you'd do differently. This one is disarming in the best way. It shows self-awareness, reflection, and the ability to evaluate your own work critically. Engineers who can't critique their own code can't grow.

I've seen portfolios with two projects and a well-written case study for each that outperformed portfolios with fifteen projects and no context. The case study gives an interviewer something to ask about. It shows you've thought deeply about the work. It makes the technical interview easier because you already answered half the questions in writing.

Lesson 3: If It's Not Deployed, It Doesn't Exist

This is the blunt version. A project that runs on localhost is a project you're still working on. It is not a portfolio piece.

I've reviewed portfolios where the "live demo" link was a localhost URL. I've seen GitHub repositories where the README says "deployment in progress" with a date from 18 months ago. I've seen apps in screenshots that couldn't actually run because they depended on a local database with no seed data.

Deploying has never been easier or cheaper. There's no excuse for a portfolio project that isn't live.

Frontend: Vercel (free), Netlify (free), Cloudflare Pages (free). Zero configuration for most frameworks.
Backend / API: Railway (free tier), Render (free tier), Fly.io (free tier). These all support Node.js, Python, Go, whatever you're running.
Database: MongoDB Atlas free tier (512MB), Supabase free tier (PostgreSQL), PlanetScale free tier (MySQL).
Full stack: Railway handles full-stack apps well. Render lets you deploy multiple services from one repo. Both have one-click GitHub deploys.

Total cost of a deployed side project: $0, with a free domain subdomain. Add a custom domain for $12/year and you have a genuinely professional-looking production deployment.

There's a secondary benefit to deploying: it forces you to actually finish things. There's a long list of problems you don't know about until you deploy — environment variable management, CORS configuration, database connection pooling, static asset serving. Deploying is part of building. A portfolio project that's never been deployed has never been truly finished.

Lesson 4: The README Is Your First Interview

When a hiring manager or senior engineer clicks the GitHub link from your portfolio, the first thing they see is the README. If it says "A project I made for learning" or has no description at all, they've already lost interest.

The README is where you make the technical case for yourself before you're in the room. Here's what a good one contains:

First paragraph: What does this thing do and why does it exist? Not "this is a web app" — tell me the specific problem it solves. One or two sentences.

Tech stack and why: Not just a logo grid. A sentence about why you chose what you chose. "Used PostgreSQL instead of MongoDB because the data has strong relational structure with lots of joins." "Chose Next.js App Router over CRA because we needed SSR for SEO and a built-in API layer." These sentences prove you made intentional decisions.

Screenshots or a GIF: A 10-second screen recording of the app working is worth a thousand words. Not staged, not filtered — just the actual app.

How to run it: Clear, complete instructions. If I clone it and follow your README and it doesn't work, that's a flag. If it works first try, that's a positive signal — it means you document carefully and you care about the developer experience of your code.

Known limitations / what you'd do differently: One paragraph. Shows maturity. "If I built this again, I'd use a message queue for the email sending instead of doing it synchronously in the request lifecycle — it caused timeouts under load."

This README takes maybe 45 minutes to write. It dramatically changes how your project is perceived.

Lesson 5: Get Your Own Domain

This one is simple and often skipped. Your portfolio should live at yourname.com or yourname.dev — not github.io/yourname/portfolio or yourname.netlify.app.

A custom domain does two things: it signals that you take yourself seriously as a professional, and it's a much better URL to put on a resume, LinkedIn, or business card. "theharshdeepsingh.com" looks intentional. "harshdeep-singh-13.github.io/portfolio-2024" looks like a homework assignment.

Domains cost $10–15 per year. That is a rounding error in any budget. Buy yours today. Redirect your GitHub Pages / Vercel / Netlify deployment to it. It takes 30 minutes and it never needs to change — you own it.

A note on choosing the domain: use your name. Not your "developer brand" or a clever handle. Names rank in Google. If someone searches for you, they should find your portfolio at the top. A personal domain with your name is one of the easiest SEO wins available to you.

Lesson 6: One AI Integration Changes Everything

Here's the hiring landscape in 2025 from a practical perspective: companies want developers who can work with AI, build on top of AI APIs, and integrate AI capabilities into existing products. This is new enough that not everyone has done it. Old enough that "I'm planning to learn it" isn't a compelling answer.

One project with a real AI integration moves you from the pile to the shortlist. Not because AI is a magic word — but because it demonstrates technical currency. You know what the OpenAI API looks like. You've dealt with token limits and streaming and prompt engineering. You've thought about cost and abuse prevention. These are all non-trivial.

What counts:

A feature in an existing project that uses GPT-4o, Claude, or Gemini for a specific, meaningful task (not "ask AI anything" — that's too vague to be impressive)
A RAG (retrieval-augmented generation) pipeline — document upload, embedding, search, answer generation
An agent that takes structured actions based on LLM output (web search, database queries, API calls)
A classification or extraction feature that uses an LLM where a simpler approach wouldn't have worked

What doesn't count:

"Used ChatGPT to help me write this code" (everyone does this)
A UI wrapper around ChatGPT that just passes prompts through (no engineering decision was made)
A project that uses AI for something that a regex would handle just as well

The bar isn't high. Ship one genuinely useful AI feature, document the decisions you made (model selection, prompt design, cost management), and you're ahead of the majority of developers applying for the same roles.

Watch: Portfolio Reviews — What Actually Works

Weak vs. Strong Portfolio Signals

Signal
Weak
Strong

Project count
20+ repos, half are tutorial clones
3–5 curated projects, each with a clear purpose

Project quality
Todo apps, weather apps, Netflix/Airbnb clones
Tools solving real problems, deployed with real users

Live demos
No live link, "works on my machine," localhost screenshots
Deployed URLs that load in under 3 seconds

Documentation
No README or "this is a project I made"
Problem statement, tech choices explained, known limitations

Tech recency
Create React App, class components, outdated dependencies
Current stack (Next.js 15, TypeScript, modern APIs)

AI integration
None, or "used AI to help me code"
One genuine AI feature with documented engineering decisions

Domain
github.io/username or platform subdomain
yourname.com — personal, memorable, professional

Case studies
Screenshots in a grid with a "View Project" button
Problem → approach → tradeoffs → outcome — per project

What I'd Do on Day 1 If I Were Starting Over

If I were a developer today with no portfolio and a job to find, here's the exact sequence I'd follow:

Day 1: Register firstnamelastname.com. It costs $12. Do it before you build anything. Having the domain makes the whole thing feel real and gives you a deadline.

Week 1: Identify one problem I genuinely have — something I do manually that should be automated, something I've searched for that doesn't exist, something at work that annoys me. Build the simplest version of the solution. Not a full product — a working tool. One feature, deployed.

Week 2: Write the case study. What was the problem? What did I consider? What did I ship? What didn't make the cut? What would I do differently? Two or three paragraphs per question. This is more valuable than the code itself.

Week 3: Add an AI integration to the project — something that actually makes the tool better, not a bolt-on. Even a single endpoint that uses an LLM for classification or text generation counts, as long as it's doing something a simpler approach couldn't.

Week 4: Point the domain at the project. Add the URL to LinkedIn's "Featured" section and your resume. Ask one person who is not a developer to try using the tool and tell you what they're confused by. Fix those things.

That's it. One good project, one good case study, one AI integration, a custom domain, a LinkedIn presence. Four weeks. That's a portfolio that gets callbacks. Everything else is refinement.

TL;DR

Curate, don't accumulate. Three deployed projects with case studies beat twenty unfinished repos. Pin your best work. Hide the rest.
Write case studies for every project. Problem → approach → tradeoffs → outcome → what you'd do differently. This is what interviews are about anyway — you're just answering in advance.
Nothing without a live URL. Free tiers on Vercel, Railway, and Render make deployment trivial. An undeployed project is an unfinished project.
The README is your first impression from GitHub. Spend 45 minutes on it. Explain the why, not just the what. Include how to run it. List the limitations. That's a professional developer.
Get the domain. yourname.com, $12, 30 minutes. It signals intent and makes your portfolio findable in Google searches by your own name.
Build one thing with a real AI integration. Not a ChatGPT wrapper — a feature that uses an LLM to solve a specific problem in your project, with documented decisions about model selection and prompt design.

How to Integrate the OpenAI API into a Production Express App

Harshdeep Singh — Tue, 02 Jun 2026 21:28:00 +0000

Last year I helped a startup integrate the OpenAI API into their product. It was a chat feature — users could ask questions about their data and get natural language answers. The integration took about a day. Three days after launch, the founder messaged me: "Hey, something's wrong. Our AWS bill just showed an unexpected charge."

It was $340. For three days. They had 60 users.

The issue wasn't a bug — it was that production API usage looks nothing like a tutorial. The tutorial shows you openai.chat.completions.create() and returns a response. The tutorial doesn't show you what happens when users send 500-token messages, when they open 15 browser tabs each maintaining their own chat context, or when one user fires requests 30 times per minute because they think it's broken.

This guide covers what the tutorials skip: rate limiting, token counting, cost guards, streaming, error handling with retries, and model selection. These aren't optional additions — they're what separates a demo from a production feature.

Why Production Is Different

Here's the gap between tutorial code and production code, stated plainly:

Concern
Tutorial Code
Production Code

Cost control
Not mentioned
Token counting, spending limits, model selection by task

Rate limiting
Not mentioned
Per-user and per-IP limits to prevent abuse

Error handling
try/catch that logs to console
Typed errors, retries with backoff, user-facing messages

Response delivery
Wait for full completion, return at once
Streaming via SSE — response appears as it generates

Context management
Each request is independent
Conversation history managed, truncated at token limit

Secrets management
API key hardcoded or in .env (no rotation)
Rotation strategy, usage monitoring, per-feature keys

Let's build a production-grade Express API that addresses all of this. We'll go layer by layer.

The Architecture

┌─────────────────────────────────────────────────────────┐
│ CLIENT (Browser / Mobile) │
│ POST /api/chat { messages: [...] } │
│ GET /api/chat/stream (SSE) │
└──────────────────────┬──────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ EXPRESS MIDDLEWARE STACK │
│ │
│ 1. express-rate-limit (10 req/min per IP) │
│ 2. tokenGuard() (reject if > 4,000 tokens) │
│ 3. auth middleware (verify user session) │
└──────────────────────┬──────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ ROUTE HANDLER │
│ │
│ Select model by task type │
│ Build messages array from context │
│ Call openai.chat.completions.create() │
│ Stream or return response │
└──────────────────────┬──────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ OPENAI API │
│ Model: gpt-4o-mini (default) / gpt-4o (complex tasks) │
└─────────────────────────────────────────────────────────┘

Project Setup

mkdir express-openai && cd express-openai
npm init -y
npm install express openai express-rate-limit tiktoken dotenv
npm install --save-dev nodemon

# .env
OPENAI_API_KEY=sk-proj-your-key-here
PORT=3001

Step 1: The OpenAI Client (Configured for Production)

Don't instantiate the OpenAI client inside route handlers. Create it once, configure it for production, and export it:

// src/openaiClient.js
import OpenAI from "openai";

export const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  maxRetries: 3,     // retry on transient failures (rate limits, timeouts)
  timeout: 30_000,   // 30 second timeout — don't hang forever
});

// Model selection by task complexity
export const MODELS = {
  fast: "gpt-4o-mini",   // classification, simple Q&A, summarization
  smart: "gpt-4o",        // complex reasoning, code generation, analysis
};

The maxRetries: 3 and timeout settings are critical. Without a timeout, a hung OpenAI request will keep your Express server's response object open indefinitely — and if you're running on a serverless function, you'll pay for that idle time.

Step 2: Token Counting and Cost Guard

The tiktoken library is OpenAI's own tokenizer — it counts tokens the exact same way the API does. Use it to reject requests before they hit the API:

// src/tokenCounter.js
import { encoding_for_model } from "tiktoken";

export function countMessageTokens(messages, model = "gpt-4o-mini") {
  const enc = encoding_for_model(model);
  let totalTokens = 0;

  for (const message of messages) {
    totalTokens += 4; // every message has ~4 tokens of overhead
    if (message.role) totalTokens += enc.encode(message.role).length;
    if (message.content) totalTokens += enc.encode(message.content).length;
    totalTokens += 1; // reply primer
  }

  enc.free(); // tiktoken requires explicit cleanup
  return totalTokens + 3; // overall reply overhead
}

// Express middleware — rejects requests over the token limit
export function tokenGuard(maxInputTokens = 4_000) {
  return (req, res, next) => {
    const messages = req.body?.messages;

    if (!Array.isArray(messages)) {
      return res.status(400).json({ error: "messages must be an array" });
    }

    const tokenCount = countMessageTokens(messages);

    if (tokenCount > maxInputTokens) {
      return res.status(400).json({
        error: `Message too long: ${tokenCount} tokens (limit: ${maxInputTokens}). Shorten your message or clear the conversation.`,
        tokenCount,
        limit: maxInputTokens,
      });
    }

    req.tokenCount = tokenCount; // pass downstream for logging
    next();
  };
}

A note on the limit: GPT-4o-mini's context window is 128K tokens, so 4,000 is conservative. But conservative is good here — a user who sends 30,000 tokens in one request is either doing something unusual or has a bug in their client. Reject it, log it, and let them know to clear their context.

Step 3: Rate Limiting

One user shouldn't be able to drain your API budget or trigger OpenAI rate limits for everyone else. Add rate limiting before the AI routes:

// src/middleware/rateLimiter.js
import rateLimit from "express-rate-limit";

export const aiRateLimiter = rateLimit({
  windowMs: 60 * 1000,  // 1-minute window
  max: 15,               // 15 requests per minute per IP
  standardHeaders: true, // return RateLimit headers
  legacyHeaders: false,
  message: {
    error: "Too many requests. Please wait a moment before trying again.",
    retryAfter: 60,
  },
  keyGenerator: (req) => {
    // Use authenticated user ID if available, otherwise fall back to IP
    return req.user?.id || req.ip;
  },
});

// Stricter limit for expensive models
export const smartModelLimiter = rateLimit({
  windowMs: 60 * 1000,
  max: 5,
  message: { error: "Too many complex requests. Rate limited for 60 seconds." },
});

Step 4: Error Handling with Typed OpenAI Errors

The OpenAI Node SDK throws typed errors. Use them — don't just check err.message:

// src/middleware/openaiErrorHandler.js
import OpenAI from "openai";

export function handleOpenAIError(err, req, res, next) {
  if (err instanceof OpenAI.APIError) {
    console.error(`OpenAI API error: ${err.status} ${err.name}`, {
      message: err.message,
      requestId: err.headers?.["x-request-id"],
    });

    if (err.status === 429) {
      return res.status(429).json({
        error: "AI service is busy. Please try again in a moment.",
        retryAfter: parseInt(err.headers?.["retry-after"] || "5"),
      });
    }

    if (err.status === 400) {
      return res.status(400).json({
        error: "Invalid request to AI service. Check your message format.",
      });
    }

    if (err.status === 401) {
      console.error("OpenAI authentication failed — check OPENAI_API_KEY");
      return res.status(503).json({ error: "AI service unavailable." });
    }
  }

  // Not an OpenAI error — pass to your generic error handler
  next(err);
}

Step 5: The Chat Endpoint (Non-Streaming)

Let's wire everything together for a standard, non-streaming response first:

// src/routes/chat.js
import express from "express";
import { openai, MODELS } from "../openaiClient.js";
import { tokenGuard } from "../tokenCounter.js";
import { aiRateLimiter } from "../middleware/rateLimiter.js";

const router = express.Router();

router.post(
  "/",
  aiRateLimiter,
  tokenGuard(4_000),
  async (req, res, next) => {
    const { messages, useSmartModel = false } = req.body;
    const model = useSmartModel ? MODELS.smart : MODELS.fast;

    try {
      const completion = await openai.chat.completions.create({
        model,
        messages,
        max_tokens: 1_000, // cap output tokens to control cost
        temperature: 0.7,
      });

      const reply = completion.choices[0].message;
      const usage = completion.usage;

      res.json({
        message: reply,
        usage: {
          inputTokens: usage.prompt_tokens,
          outputTokens: usage.completion_tokens,
          totalTokens: usage.total_tokens,
          estimatedCostUsd: estimateCost(usage, model),
        },
      });
    } catch (err) {
      next(err);
    }
  }
);

function estimateCost(usage, model) {
  // Prices per million tokens (as of mid-2025)
  const pricing = {
    "gpt-4o-mini": { input: 0.15, output: 0.60 },
    "gpt-4o": { input: 5.00, output: 15.00 },
  };
  const p = pricing[model] || pricing["gpt-4o-mini"];
  const inputCost = (usage.prompt_tokens / 1_000_000) * p.input;
  const outputCost = (usage.completion_tokens / 1_000_000) * p.output;
  return Number((inputCost + outputCost).toFixed(6));
}

export default router;

Notice max_tokens: 1_000. Without this, GPT-4o can produce 4,096 output tokens per request. If a user asks it to "write me a book," it will try. The max_tokens cap is your backstop.

Step 6: Streaming Responses with Server-Sent Events

Streaming makes AI features feel responsive. Instead of a blank screen for 3–8 seconds, the user sees text appear word by word. It's the difference between "this feels AI-powered" and "this is broken."

// src/routes/chat-stream.js
import express from "express";
import { openai, MODELS } from "../openaiClient.js";
import { tokenGuard } from "../tokenCounter.js";
import { aiRateLimiter } from "../middleware/rateLimiter.js";

const router = express.Router();

router.post(
  "/stream",
  aiRateLimiter,
  tokenGuard(4_000),
  async (req, res, next) => {
    const { messages } = req.body;

    // Establish SSE connection
    res.setHeader("Content-Type", "text/event-stream");
    res.setHeader("Cache-Control", "no-cache");
    res.setHeader("Connection", "keep-alive");
    res.setHeader("Access-Control-Allow-Origin", "*");
    res.flushHeaders(); // send headers immediately

    try {
      const stream = await openai.chat.completions.create({
        model: MODELS.fast,
        messages,
        max_tokens: 1_000,
        stream: true,
      });

      let totalOutputTokens = 0;

      for await (const chunk of stream) {
        const delta = chunk.choices[0]?.delta?.content ?? "";
        if (delta) {
          totalOutputTokens += 1; // approximate; tiktoken is more accurate
          res.write(`data: ${JSON.stringify({ type: "delta", content: delta })}

`);
        }

        // Check for stop reason
        if (chunk.choices[0]?.finish_reason === "length") {
          res.write(`data: ${JSON.stringify({ type: "warning", message: "Response truncated at token limit" })}

`);
        }
      }

      res.write(`data: ${JSON.stringify({ type: "done" })}

`);
      res.end();
    } catch (err) {
      // Send error over SSE before closing
      res.write(`data: ${JSON.stringify({ type: "error", message: "Generation failed. Please try again." })}

`);
      res.end();
      // Also pass to error handler for logging
      console.error("Streaming error:", err.message);
    }
  }
);

export default router;

Watch: OpenAI API with Node.js + Express

Streaming vs. Non-Streaming — When to Use Which

Factor
Non-Streaming
Streaming (SSE)

User experience
Blank screen until done (3–8s)
Text appears word by word — feels instant

Complexity
Standard REST response
SSE connection, chunked parsing on frontend

Usage logging
Easy — completion.usage has exact token counts
Harder — token counts only available via the final chunk

Caching
Can cache the full response
Can't cache a stream

Best for
API-to-API calls, short responses, classification tasks
User-facing chat, long completions, code generation

Serverless functions
Works everywhere
Needs long-running connection — use Vercel Edge Functions or a real server

Testing Your OpenAI Integration

Mocking the OpenAI API in tests is a trap. The mock will pass but the real integration will fail in ways you didn't anticipate — different error formats, unexpected token usage, streaming chunk structure variations.

Instead:

Unit test everything except the API call. Test your token counting, your error handler, your response formatter — all without touching OpenAI. These functions should be pure and deterministic.
Use a cheap model for integration tests. gpt-4o-mini is $0.15 per million input tokens. Your integration test suite probably costs fractions of a cent to run. Run it.
Record and replay for expensive tests. Libraries like nock or VCR-style recording let you record real API responses and replay them in future test runs without hitting the API.

// Example: testing the token guard middleware in isolation
import { tokenGuard } from "../src/tokenCounter.js";
import { createMockMiddlewareContext } from "./helpers.js";

test("tokenGuard rejects messages over the limit", async () => {
  const guard = tokenGuard(10); // tiny limit for test
  const { req, res, next } = createMockMiddlewareContext({
    body: {
      messages: [{ role: "user", content: "a".repeat(500) }],
    },
  });

  await guard(req, res, next);

  expect(res.statusCode).toBe(400);
  expect(res.body.error).toContain("too long");
  expect(next).not.toHaveBeenCalled();
});

TL;DR

Initialize the OpenAI client once with maxRetries and timeout set. Don't instantiate it in route handlers or you'll get a new client per request with no retry or timeout configuration.
Count tokens before you call the API. Use tiktoken to measure input size and reject oversized requests before they cost you money. Set a max_tokens cap on output for the same reason.
Rate limit by user ID, not just IP. Authenticated users with the same IP (corporate NAT, mobile networks) would all share a single IP limit — use their user ID as the rate limit key.
Use typed error handling — instanceof OpenAI.APIError gives you the status code, request ID, and message. Turn 429s into user-friendly retry prompts, not 500 errors.
Stream for user-facing features, skip it for internal calls. SSE streaming transforms the UX for chat interfaces. For batch processing or API-to-API calls, non-streaming is simpler to implement and log.
Test everything except the API call. Token counting, error handling, and response formatting are all pure functions you can test cheaply. For integration tests, use gpt-4o-mini — it's cheap enough to run in CI.

Deploying a Next.js App to AWS with CI/CD Pipelines (Step-by-Step)

Harshdeep Singh — Tue, 02 Jun 2026 21:27:52 +0000

The first time I deployed a Next.js app to production, it took me three days. Not because the app was complicated — it was a straightforward portfolio site. It took three days because I had no idea what I was doing with AWS, I'd never written a GitHub Actions workflow, and every tutorial I found either skipped the hard parts or assumed I already knew them.

By the time I was done, I had a deployment pipeline I was genuinely proud of: push to main, GitHub Actions runs the build, tests pass, the app deploys to an EC2 instance behind CloudFront. Zero manual steps. Zero downtime deploys. Total cost: about $5/month.

This guide is the one I wish had existed. We're going to deploy a Next.js app to AWS from scratch — EC2 for compute, CloudFront for CDN, GitHub Actions for CI/CD — with every step explained so you understand what you're building, not just copying commands.

Why AWS Instead of Vercel?

This is a fair question. Vercel is genuinely excellent for Next.js, and for most projects it's the right call. You push, it deploys. Done.

AWS makes sense when:

You need to control the infrastructure (compliance, data residency, custom VPC configuration)
You're running other services (databases, queues, lambdas) in AWS and want everything in the same network
You want to learn infrastructure skills that transfer to enterprise environments
Your app has specific performance requirements that benefit from custom CloudFront configuration
You're a freelancer or consultant who wants to bill separately for infrastructure

If none of those apply to you, use Vercel. This guide is for when they do.

The Architecture

Here's what we're building:

┌────────────────────────────────────────────────────────┐
│ GITHUB ACTIONS CI/CD │
│ │
│ Push to main → Build → Test → Deploy to EC2 │
└──────────────────────┬─────────────────────────────────┘
│ SSH deploy
▼
┌────────────────────────────────────────────────────────┐
│ AWS EC2 INSTANCE │
│ │
│ Ubuntu 22.04 LTS │
│ Node.js 20 + PM2 (process manager) │
│ Next.js app running on port 3000 │
│ Nginx reverse proxy (port 80/443 → 3000) │
└──────────────────────┬─────────────────────────────────┘
│ Origin
▼
┌────────────────────────────────────────────────────────┐
│ CLOUDFRONT CDN │
│ │
│ Static assets cached at edge (/_next/static/*) │
│ TTL: 1 year for static, 0 for HTML │
│ SSL termination via ACM certificate │
│ Custom domain: yourapp.com │
└────────────────────────────────────────────────────────┘

This isn't the only way to run Next.js on AWS. You could use Elastic Beanstalk, App Runner, ECS, or deploy static exports to S3 + CloudFront. The EC2 + CloudFront approach gives you the most control and transfers the most skills to enterprise environments.

Prerequisites

An AWS account (free tier works for learning; a t3.micro is enough for small apps)
A domain name (optional but recommended — we'll set up SSL)
A GitHub repository with your Next.js app
Basic familiarity with the AWS console

The total setup takes about 90 minutes the first time. After that, every deployment is automatic.

Step 1: Set Up the EC2 Instance

In the AWS Console, navigate to EC2 and launch a new instance. The settings that matter:

AMI: Ubuntu Server 22.04 LTS (free tier eligible)
Instance type: t3.micro (1 vCPU, 1GB RAM) for small apps; t3.small for medium traffic
Key pair: Create a new one, download it — you'll need this for SSH and GitHub Actions
Security group: Allow inbound traffic on ports 22 (SSH), 80 (HTTP), and 443 (HTTPS). Add your IP as the only source for port 22 (don't expose SSH to 0.0.0.0/0).

Once the instance is running, SSH in and set up the environment:

# Connect to your instance
ssh -i your-key.pem ubuntu@YOUR_EC2_PUBLIC_IP

# Update system packages
sudo apt-get update && sudo apt-get upgrade -y

# Install Node.js 20 via NodeSource
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt-get install -y nodejs

# Install PM2 globally (process manager for Node.js)
sudo npm install -g pm2

# Install Nginx
sudo apt-get install -y nginx

# Verify everything installed correctly
node --version   # v20.x.x
npm --version    # 10.x.x
pm2 --version    # 5.x.x
nginx -v         # nginx/1.24.x

Step 2: Configure Nginx as a Reverse Proxy

Nginx will listen on port 80 and forward requests to your Next.js app on port 3000. This is the standard setup for Node.js apps on Linux servers.

sudo nano /etc/nginx/sites-available/nextjs-app

Paste this configuration:

server {
    listen 80;
    server_name YOUR_DOMAIN.com www.YOUR_DOMAIN.com;

    # Proxy requests to Next.js
    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_cache_bypass $http_upgrade;
    }

    # Cache Next.js static assets at Nginx level too
    location /_next/static/ {
        proxy_pass http://localhost:3000;
        proxy_cache_valid 200 1y;
        add_header Cache-Control "public, max-age=31536000, immutable";
    }
}

# Enable the site and test the config
sudo ln -s /etc/nginx/sites-available/nextjs-app /etc/nginx/sites-enabled/
sudo nginx -t            # should say "test is successful"
sudo systemctl restart nginx

Step 3: The GitHub Actions Workflow

This is where the CI/CD magic happens. The workflow does four things: checks out code, runs your build, SSHs into the server, and restarts the app. Create this file in your repository:

mkdir -p .github/workflows

Create .github/workflows/deploy.yml:

name: Deploy to AWS EC2

on:
  push:
    branches: [main]
  workflow_dispatch:    # also allow manual triggers

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "20"
          cache: "npm"

      - name: Install dependencies
        run: npm ci

      - name: Run linter
        run: npm run lint

      - name: Build Next.js app
        run: npm run build
        env:
          # Pass any build-time env vars here
          NEXT_PUBLIC_GA_ID: ${{ secrets.NEXT_PUBLIC_GA_ID }}

      - name: Deploy to EC2
        uses: appleboy/ssh-action@v1.0.3
        with:
          host: ${{ secrets.EC2_HOST }}
          username: ubuntu
          key: ${{ secrets.EC2_PRIVATE_KEY }}
          script: |
            cd /var/www/nextjs-app

            # Pull latest code
            git pull origin main

            # Install dependencies (production only)
            npm ci --omit=dev

            # Build the app on the server
            npm run build

            # Restart with PM2 (zero-downtime reload)
            pm2 reload nextjs-app --update-env

            echo "Deploy complete at $(date)"

Step 4: Set Up GitHub Secrets

In your GitHub repository, go to Settings → Secrets and variables → Actions, and add these three secrets:

Secret Name
Value

EC2_HOST
Your EC2 instance's public IP address or domain

EC2_PRIVATE_KEY
The full contents of your .pem file (including the BEGIN/END lines)

NEXT_PUBLIC_GA_ID
Your Google Analytics measurement ID (or any other public env vars)

For the private key, open the .pem file in a text editor, copy everything including -----BEGIN RSA PRIVATE KEY----- and -----END RSA PRIVATE KEY-----, and paste it as the secret value.

Step 5: First Deploy and PM2 Setup

Before the GitHub Action can work, you need to get the app running on the server for the first time:

# Clone your repo to the server
sudo mkdir -p /var/www/nextjs-app
sudo chown ubuntu:ubuntu /var/www/nextjs-app
cd /var/www/nextjs-app
git clone https://github.com/YOUR_USERNAME/YOUR_REPO.git .

# Create .env file on the server
nano .env
# Add your production environment variables here

# Install dependencies and build
npm ci
npm run build

# Start with PM2
pm2 start npm --name "nextjs-app" -- start
pm2 save                  # persist across server restarts
pm2 startup               # generate startup script
# PM2 will output a command to run — run it

# Verify the app is running
pm2 status
curl http://localhost:3000  # should return HTML

Step 6: CloudFront CDN (Optional but Recommended)

CloudFront puts your app behind a global CDN, which means static assets load from an edge location near your users instead of your EC2 server. For most apps, this makes a meaningful difference in load times outside your server's region.

In the AWS Console, go to CloudFront and create a new distribution:

Origin domain: Your EC2 public IP or domain (not localhost)
Origin protocol policy: HTTP only (Nginx handles the connection to EC2)
Viewer protocol policy: Redirect HTTP to HTTPS
Cache policy for /_next/static/*: CachingOptimized — these files are content-addressed, so they can be cached for years
Cache policy for /* (HTML pages): CachingDisabled — Next.js handles its own cache headers; CloudFront should pass them through

If you have a domain, attach it to the CloudFront distribution and request an ACM (AWS Certificate Manager) certificate for free SSL. DNS validation takes about 15 minutes.

Watch: Next.js CI/CD to AWS EC2 with GitHub Actions

Common Pitfalls

1. Building on the server vs. building in CI

The workflow above builds in the GitHub Action AND on the server. That's redundant — you only need to do it in one place. For small apps, building on the server is fine (simpler). For larger teams, build in CI, upload the artifact, and skip the build step on the server. The tradeoff: artifacts can be large (100MB+), so you need S3 or similar to store them.

2. Forgetting to set NODE_ENV=production

When you run npm start (which runs next start), Next.js automatically sets NODE_ENV=production. But PM2 doesn't always inherit this. Be explicit in your PM2 config or startup command:

pm2 start npm --name "nextjs-app" -- start -- --NODE_ENV=production

3. Not configuring PM2 to restart on crash

By default PM2 restarts crashed processes, but you want to limit restarts to prevent crash loops. Add --max-restarts 10 and --min-uptime 5000 to your pm2 start command. Five seconds of uptime before a restart counts is usually enough to catch truly broken deployments.

4. SSH key permissions

The most common SSH error you'll hit is UNPROTECTED PRIVATE KEY FILE. GitHub Actions handles this correctly when you use appleboy/ssh-action, but if you're doing raw SSH commands, your .pem file needs chmod 400 your-key.pem — readable only by the owner, nothing else.

EC2 vs. Vercel vs. AWS Amplify — Which Should You Choose?

Factor
EC2 + CloudFront
Vercel
AWS Amplify

Setup time
90 min (first time)
5 min
20 min

Next.js feature support
Full (you control the runtime)
Full (built for Next.js)
Most features, some lag

Cost at low traffic
~$5/month (t3.micro)
Free tier, then $20+/month
Pay per build + hosting

Cost at high traffic
Predictable (fixed instance)
Can get expensive fast
Moderate

Infrastructure control
Full — you own everything
None — Vercel manages it
Partial

Learning value
High — enterprise-transferable
Low (it just works)
Medium

Best for
Learning, compliance, cost control
Speed, simplicity, teams
Existing AWS customers

TL;DR

The stack: EC2 (compute) + Nginx (reverse proxy) + PM2 (process manager) + CloudFront (CDN) + GitHub Actions (CI/CD). Each layer has one job.
GitHub Actions workflow: trigger on push to main → install → lint → build → SSH into EC2 → git pull → rebuild → pm2 reload. About 25 lines of YAML.
Store secrets properly: EC2 host, private key, and env vars go in GitHub repository secrets — never hardcoded in workflow files.
PM2 is essential for production Node.js — it keeps the process alive, restarts on crash, and enables zero-downtime reloads. Run pm2 startup to make it persist across server reboots.
CloudFront is optional but worth it — static assets cached at the edge make a real difference for users outside your server's region, and the free ACM SSL certificate saves you the hassle of Certbot configuration.

React + TypeScript Best Practices in 2025: What Actually Matters

Harshdeep Singh — Tue, 02 Jun 2026 21:21:57 +0000

You open a new React project, add TypeScript, and immediately hit Stack Overflow for how to type your first prop. The first answer says use interface. The second says type. The third is a six-paragraph thread about why one is semantically superior to the other. You close the tab and just write any to get on with your life.

Sound familiar? TypeScript in React has a reputation problem. Not because it's hard — it's genuinely great once it clicks — but because the community has generated a staggering volume of contradictory, context-free advice. Every dev tool tutorial starts with TypeScript. Every linting config bans any. Every PR reviewer has a hot take on generics.

This guide cuts through that. I'm not going to cover every TypeScript feature or every React pattern. What I'm going to do is share the specific conventions I use in production React + TypeScript apps in 2025 — the things that have made codebases genuinely easier to work in, not just theoretically safer.

What this guide is NOT

Before we get into it, let me set expectations clearly:

Not a TypeScript basics tutorial. I'm assuming you know what a type is, what an interface is, and that string !== String.
Not exhaustive. TypeScript has dozens of utility types, conditional types, template literal types, and more. I'm not covering all of them — just the ones I reach for constantly.
Not framework-neutral. This is specifically about TypeScript in React apps. Some of these patterns won't apply to a Node.js CLI or a library.
Not about configuration. Strict mode settings, tsconfig targets, module resolution — another article for another day.

What this guide IS is opinionated. I'm going to tell you what I think the right call is in most situations, and why. You'll disagree with some of it. That's fine.

Typing Props the Right Way

Let's start here because it's where every React + TypeScript journey begins, and where a lot of the confusion lives.

Interface vs. Type Alias

Here's my rule: use interface for component props, type for everything else.

Why? Interfaces have declaration merging, which can occasionally bite you in unexpected ways, but they also produce cleaner error messages and feel more natural for describing object shapes. They're also what the React community defaults to, so your code will look familiar to anyone joining your team.

// Good — interface for component props
interface ButtonProps {
  label: string;
  onClick: () => void;
  variant?: "primary" | "secondary" | "ghost";
  disabled?: boolean;
}

// Good — type alias for unions and computed shapes
type ButtonVariant = "primary" | "secondary" | "ghost";
type Theme = "light" | "dark";

You'll see guides that say "always use type" or "always use interface." Honestly? Consistency matters more than which one you pick. Pick a rule and stick to it across your codebase.

Required vs. Optional Props

Default to required. Make something optional only when it genuinely has a sensible default or when it's truly not needed in many use cases.

This is the inverse of what a lot of developers do. They add ? to everything to make TypeScript stop complaining, and then their components have fifteen optional props where most of them are actually always passed. That erases the value of having types at all.

// Bad — over-optionalized
interface CardProps {
  title?: string;
  description?: string;
  imageUrl?: string;
  href?: string;
}

// Good — be explicit about what's truly optional
interface CardProps {
  title: string;
  description: string;
  imageUrl?: string; // genuinely optional — card can work without an image
  href?: string;    // optional — sometimes cards aren't clickable
}

Extending HTML Element Props

This is one of the most useful patterns in React + TypeScript, and it's underused. When you're building a component that wraps an HTML element, extend that element's props so your component accepts all the native attributes automatically.

// Without this pattern — you have to manually add every HTML attribute
interface ButtonProps {
  label: string;
  onClick: () => void;
  // what about type="submit"? aria-label? data-testid? className?
  // you'll spend forever adding these one by one
}

// With this pattern — extend React's built-in types
interface ButtonProps extends React.ButtonHTMLAttributes {
  label: string;
  variant?: "primary" | "secondary";
  // ...and you automatically get onClick, type, aria-*, data-*, className, etc.
}

const Button = ({ label, variant = "primary", ...rest }: ButtonProps) => {
  return (

      {label}

  );
};

The ...rest spread pattern combined with extended HTML props is one of those things that once you start using, you can't go back. Your components become instantly more composable and you stop maintaining a manual list of passthrough props.

Custom Hooks with TypeScript

Custom hooks are where TypeScript really earns its keep, because hooks often manage complex state and return multiple values. If your hook's return type is just inferred as any[], you've lost all the benefit.

Typing Return Values Explicitly

Always define the return type of custom hooks explicitly. Don't rely on inference here — it breaks down the moment your hook has multiple return paths or conditional logic.

// Bad — inferred return type is unreliable
function useUser(id: string) {
  const [user, setUser] = useState(null); // typed as null
  const [loading, setLoading] = useState(true);
  const [error, setError] = useState(null);

  // ...fetch logic

  return { user, loading, error }; // TypeScript infers this poorly
}

// Good — define the return interface explicitly
interface User {
  id: string;
  name: string;
  email: string;
}

interface UseUserResult {
  user: User | null;
  loading: boolean;
  error: string | null;
}

function useUser(id: string): UseUserResult {
  const [user, setUser] = useState(null);
  const [loading, setLoading] = useState(true);
  const [error, setError] = useState(null);

  // ...fetch logic

  return { user, loading, error };
}

Now when you destructure this hook in a component, every field is typed correctly, and you get autocomplete without having to remember what the hook returns.

Generics for Reusable Hooks

Okay so here's where hooks get really powerful. If you're building a reusable data-fetching hook, generics let you make it work with any shape of data without losing type safety.

interface FetchResult {
  data: T | null;
  loading: boolean;
  error: string | null;
}

function useFetch(url: string): FetchResult {
  const [data, setData] = useState(null);
  const [loading, setLoading] = useState(true);
  const [error, setError] = useState(null);

  useEffect(() => {
    let cancelled = false;

    fetch(url)
      .then((res) => {
        if (!res.ok) throw new Error(`HTTP ${res.status}`);
        return res.json() as Promise;
      })
      .then((d) => {
        if (!cancelled) {
          setData(d);
          setLoading(false);
        }
      })
      .catch((err: Error) => {
        if (!cancelled) {
          setError(err.message);
          setLoading(false);
        }
      });

    return () => { cancelled = true; };
  }, [url]);

  return { data, loading, error };
}

// Usage — TypeScript knows exactly what data is
const { data, loading, error } = useFetch("/api/user/123");
// data is typed as User | null — not unknown, not any

The T propagates through the entire hook. That's the magic. You call useFetch<User> once at the call site, and TypeScript figures out the rest.

Generic Components

Generics in components are the thing that trips up most mid-level React developers. They look intimidating. They have funny angle-bracket syntax. But once you understand when to reach for them, they save you from maintaining three slightly-different versions of the same component.

When to Use Generic Components

Reach for generics when your component works with data of a variable shape, but still needs to be type-safe. A list component, a select dropdown, a data table — these are classic candidates.

// Without generics — you end up with separate UserList, ProjectList, etc.
// or you use any[] and lose type safety

// With generics — one component that works for any data shape
interface ListProps {
  items: T[];
  renderItem: (item: T, index: number) => React.ReactNode;
  keyExtractor: (item: T) => string;
  emptyMessage?: string;
}

function List({ items, renderItem, keyExtractor, emptyMessage = "No items" }: ListProps) {
  if (items.length === 0) {
    return 
{emptyMessage}
;
  }

  return (


      {items.map((item, index) => (
        - {renderItem(item, index)}
      ))}


  );
}

// Usage — TypeScript infers T from the items array
 u.id}  // TypeScript knows u is a User
  renderItem={(u) => }
/>

Notice that you don't even need to write <List<User>> at the call site — TypeScript infers T = User from the items prop. That's inference doing its job.

One thing to watch: in .tsx files, the compiler can confuse <T> with a JSX tag. If you get a parse error, either add a constraint (<T extends object>) or use a comma (<T,>) to disambiguate.

Discriminated Unions for State

Here's the thing that's changed how I think about React state more than anything else: replacing boolean flags with discriminated union types. This single pattern eliminates entire categories of bugs.

The Boolean Flag Problem

You've seen this component. You've written this component.

// The boolean flag trap
interface FormState {
  isLoading: boolean;
  isSuccess: boolean;
  isError: boolean;
  errorMessage?: string;
  data?: SubmitResult;
}

// Nothing stops you from setting isLoading: true AND isSuccess: true simultaneously
// That's an impossible state — but TypeScript can't catch it
const state: FormState = {
  isLoading: true,
  isSuccess: true, // ← this should be impossible
  isError: false,
};

When you have three booleans representing what should be a single sequential state, you have 2³ = 8 possible combinations, but only 4 of them are actually valid. TypeScript can't protect you from the invalid ones.

The Discriminated Union Fix

// Model the actual states that can exist
type FormState =
  | { status: "idle" }
  | { status: "loading" }
  | { status: "success"; data: SubmitResult }
  | { status: "error"; errorMessage: string };

// Now the impossible states are literally unrepresentable
const state: FormState = { status: "idle" };

// And in your component, TypeScript narrows types automatically
function FormFeedback({ state }: { state: FormState }) {
  if (state.status === "loading") {
    return ;
  }

  if (state.status === "error") {
    // TypeScript knows state.errorMessage exists here
    return ;
  }

  if (state.status === "success") {
    // TypeScript knows state.data exists here
    return ;
  }

  return null; // idle
}

The key is the status discriminant property. When you narrow on state.status === "error", TypeScript automatically knows which variant of the union you're in, and which other fields are available.

This pattern is especially powerful in data-fetching scenarios, form submission flows, and anywhere you have a multi-step process. Start reaching for it instead of isLoading / isError / isSuccess and your state management will become dramatically cleaner.

Taming any

Let me be direct: any is a code smell, but it's not always your fault. Sometimes you're working with a library that has poor types, an API that returns unpredictable shapes, or legacy code you don't own. The goal isn't to never use any — it's to reach for better tools first.

Use unknown Instead of any for External Data

unknown is the type-safe cousin of any. It says "I don't know what this is yet" instead of "pretend this is whatever I need it to be." You can't do anything with an unknown value without first narrowing it with a type guard.

// Bad — any lets you do anything, including wrong things
async function fetchData(url: string): Promise {
  const res = await fetch(url);
  return res.json();
}

const data = await fetchData("/api/user");
data.doesNotExist.boom; // TypeScript is fine with this. Your app is not.

// Good — unknown forces you to validate before using
async function fetchData(url: string): Promise {
  const res = await fetch(url);
  return res.json();
}

function isUser(value: unknown): value is User {
  return (
    typeof value === "object" &&
    value !== null &&
    "id" in value &&
    "name" in value &&
    typeof (value as User).id === "string"
  );
}

const data = await fetchData("/api/user");
if (isUser(data)) {
  // TypeScript now knows data is User
  console.log(data.name);
}

Type Guards Are Your Friends

The value is User return type in the example above is a type guard. It's a function that tells TypeScript "if this returns true, the value is of type T in the branches that follow." This is how you move from unknown territory into properly typed territory without resorting to any.

// Reusable type guard pattern
function isNonNull(value: T | null | undefined): value is T {
  return value !== null && value !== undefined;
}

const items: (User | null)[] = [user1, null, user2, null];
const validUsers = items.filter(isNonNull); // typed as User[], not (User | null)[]

When never Is the Right Answer

never is the type that can't exist. It's useful for exhaustiveness checks — making sure you've handled every case in a union.

type Shape = "circle" | "square" | "triangle";

function getArea(shape: Shape, size: number): number {
  switch (shape) {
    case "circle":
      return Math.PI * size * size;
    case "square":
      return size * size;
    case "triangle":
      return (size * size) / 2;
    default:
      // If you add "hexagon" to the Shape union and forget to handle it here,
      // TypeScript will throw a compile error on this line
      const _exhaustiveCheck: never = shape;
      throw new Error(`Unhandled shape: ${_exhaustiveCheck}`);
  }
}

This pattern scales beautifully with discriminated unions. Add a new status to your state type, and every switch statement that wasn't updated will fail at compile time. That's exactly the kind of safety net TypeScript is supposed to provide.

Before vs. After: TypeScript Patterns

Here's a quick-reference table of the common before/after shifts. These are the six patterns I most often see in code review that a bit of TypeScript discipline cleans up immediately.

  Pattern

  Without TypeScript Discipline

  With TypeScript Discipline

Prop typing

  props: any or no types at all

  interface ButtonProps extends React.ButtonHTMLAttributes&lt;HTMLButtonElement&gt;

Optional props

  Everything is ? to avoid errors

  Only truly optional fields are optional; defaults handled by destructuring

Loading/error state

  isLoading: boolean, isError: boolean, isSuccess: boolean

  Discriminated union: { status: "idle" | "loading" | "success" | "error" }

External API data

  const data: any = await fetch(...).then(r =&gt; r.json())

  const data: unknown + type guard before use

Reusable list

  Separate UserList, ProjectList components with duplicated logic

  One generic List&lt;T&gt; component with typed renderItem and keyExtractor

Hook return type

  Inferred — breaks on multiple return paths, confusing autocomplete

  Explicit interface UseXxxResult { ... } as the return type annotation

Module Structure for a TypeScript React Project

Okay, one more thing worth getting right from the start: where files live. A consistent folder structure does more for long-term maintainability than almost any TypeScript pattern. Here's the structure I use and recommend for mid-to-large React + TypeScript apps in 2025.

src/
├── app/ # Next.js App Router pages (or pages/ for older setup)
│ ├── layout.tsx
│ ├── page.tsx
│ └── blog/
│ └── [slug]/
│ └── page.tsx
│
├── components/ # Shared UI components
│ ├── Button/
│ │ ├── Button.tsx # Component
│ │ ├── Button.types.ts # Interface / type exports
│ │ ├── Button.styles.ts# Styled components or CSS module
│ │ └── index.ts # Re-export for clean imports
│ └── List/
│ └── ...
│
├── hooks/ # Custom hooks
│ ├── useFetch.ts
│ ├── useLocalStorage.ts
│ └── useDebounce.ts
│
├── lib/ # Utilities, helpers, non-UI logic
│ ├── api.ts # Fetch wrappers
│ ├── formatters.ts # Date, currency, string helpers
│ └── validators.ts # Type guards and runtime validation
│
├── types/ # Shared TypeScript types
│ ├── api.ts # API response shapes
│ ├── models.ts # Domain model interfaces (User, Post, etc.)
│ └── index.ts # Re-exports
│
├── context/ # React context providers
│ └── ThemeContext.tsx
│
└── styles/ # Global styles, theme tokens
└── globals.css

A few rules I enforce in this structure:

Co-locate component types. Each component folder has its own .types.ts file. Don't dump all types into a single global types.ts — that file becomes a graveyard.
The types/ directory is for shared domain types only. API response shapes, database models, shared interfaces that more than one component needs. Not component-specific props.
Barrel files are useful but dangerous. An index.ts in each component folder is fine. A single barrel for your entire components/ directory will cause circular dependency nightmares in larger apps.
Hooks go in hooks/, not co-located with components. This is a deliberate choice against the "co-locate everything" philosophy. In my experience, hooks get reused across features, and burying them inside a component folder makes them harder to find and share.

TL;DR

Use interface for component props, type for everything else — pick a rule and apply it consistently. Default to required props; only mark things optional when they genuinely are.
Extend HTML element props using React.ButtonHTMLAttributes<HTMLButtonElement> and friends — this gets you all native attributes for free and makes components composable with ...rest.
Always annotate custom hook return types explicitly — define a UseXxxResult interface and return it. Don't trust inference when there's conditional logic involved.
Use discriminated unions instead of boolean flags for anything with multiple states — { status: "idle" | "loading" | "success" | "error" } is safer, clearer, and catches impossible states at compile time.
Replace any with unknown at API boundaries — then validate with type guards before use. Save never for exhaustiveness checks in switch statements.
Structure your modules intentionally — co-locate component types, put shared domain types in types/, hooks in hooks/, and use barrel files per component folder (not globally).

How to Build an AI Resume Builder with LangChain and Node.js

Harshdeep Singh — Tue, 02 Jun 2026 21:21:55 +0000

A few months back, my friend Marcus was applying for a senior backend role at a fintech company. He had five years of solid experience — distributed systems, AWS, the whole stack. But his resume read like a list of job descriptions someone had copied from LinkedIn. "Responsible for maintaining microservices." "Assisted with CI/CD pipeline implementation." You know the type.

I told him: the problem isn't what you did, it's how you're saying it. Hiring managers spend about six seconds on a resume before deciding whether to read it properly. Six seconds. And if those six seconds are spent reading "responsible for maintaining" — you've lost them.

We spent two hours rewriting it together. Every bullet point started with a strong verb. Every achievement had a number. "Reduced API response time by 40% by introducing Redis caching across three high-traffic endpoints." Much better. Marcus got the interview.

The obvious next thought was: what if you could automate this? Not in the "dump your resume into ChatGPT and ask it to make it better" way — that produces generic slop. I mean a real, structured AI pipeline that understands resume context, applies professional rewriting patterns, and returns clean, job-specific output.

That's what LangChain is built for. And in this guide, we're going to build exactly that: an AI-powered resume rewriter using LangChain and Node.js, with a real Express API, streaming responses, and the kind of prompt engineering that actually produces good results.

What Is LangChain, and Why Bother?

Here's the honest answer: LangChain is an orchestration framework for building applications on top of large language models. Think of it the way you'd think of Express.js — Express doesn't do anything you couldn't do with raw Node's http module, but it gives you a structured, composable way to build web apps that doesn't collapse under its own weight.

LangChain does the same thing for LLM applications. You could just call the OpenAI API directly everywhere. For a one-off script, that's fine. But as soon as your app grows — different prompts for different tasks, multi-step reasoning chains, memory across conversations — raw API calls get messy fast.

Here's what raw OpenAI API code looks like once a project grows:

// Raw OpenAI — works, but scales badly
const response = await openai.chat.completions.create({
  model: "gpt-4",
  messages: [
    { role: "system", content: systemPrompt },
    { role: "user", content: `Rewrite this section: ${section}` }
  ]
});
const rewritten = response.choices[0].message.content;

That's fine for one call. Now add: prompt versioning, chaining that output into a second model call, memory from previous messages, fallback to a different model when rate limits hit, streaming output to the client. Suddenly you're managing a lot of state manually.

LangChain handles all of that with composable primitives: PromptTemplate for reusable, testable prompts; LLMChain for connecting a prompt to a model; SequentialChain for multi-step pipelines; built-in streaming support; and integrations with every major LLM provider.

For our resume builder, the chain looks like this: parse the resume into structured sections, run each section through a prompt that produces action-oriented bullet points, then return the assembled result. Let's build it.

What We're Building

Before we write a line of code, here's the system at a glance:

┌─────────────────────────────────────────────────────┐
│                   CLIENT (Frontend)                  │
│         POST /api/rewrite { resumeText, section }    │
└──────────────────────┬──────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────┐
│                  EXPRESS API (Node.js)               │
│  1. Validate input                                   │
│  2. Parse resume into sections                       │
│  3. Call LangChain rewrite chain                     │
│  4. Return improved bullet points                    │
└──────────────────────┬──────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────┐
│              LANGCHAIN REWRITE CHAIN                 │
│  PromptTemplate → ChatOpenAI (GPT-4) → Output       │
└──────────────────────┬──────────────────────────────┘
                       │
                       ▼
┌─────────────────────────────────────────────────────┐
│                  OPENAI API (GPT-4)                  │
└─────────────────────────────────────────────────────┘

Nothing revolutionary — but each layer has a single, testable job. The chain is the interesting part, so let's get there quickly.

Project Setup

Start a new Node.js project and install the dependencies:

mkdir resume-ai && cd resume-ai
npm init -y
npm install express langchain @langchain/openai @langchain/core dotenv

Create a .env file at the root:

OPENAI_API_KEY=sk-your-key-here
PORT=3001

And your project structure:

resume-ai/
├── src/
│   ├── parseResume.js
│   ├── resumeChain.js
│   └── app.js
├── .env
└── package.json

Add "type": "module" to package.json so we can use ES module syntax throughout.

Step 1: Parsing the Resume

This is the unglamorous part that everyone skips, and it's why most AI resume tools produce garbage. You can't just throw 800 words of resume text at a model and ask it to "make it better." You need to isolate the section you're improving — otherwise the model is operating without context.

Here's a simple section parser. It's not perfect — real resumes come in dozens of formats — but it handles the common patterns:

// src/parseResume.js
export function parseResumeText(rawText) {
  const sections = {
    summary: "",
    experience: [],
    skills: [],
    education: [],
  };

  const sectionKeywords = {
    summary: ["summary", "objective", "profile", "about"],
    experience: ["experience", "employment", "work history", "career"],
    skills: ["skills", "technical skills", "technologies", "competencies"],
    education: ["education", "academic", "degree", "university"],
  };

  const lines = rawText.split("\n").filter((l) => l.trim().length > 0);
  let currentSection = null;

  for (const line of lines) {
    const lowerLine = line.toLowerCase().trim();

    const detected = Object.entries(sectionKeywords).find(([, keywords]) =>
      keywords.some((kw) => lowerLine.includes(kw))
    );

    if (detected && lowerLine.length  {
  const { resumeText, targetSection } = req.body;

  if (!resumeText || typeof resumeText !== "string") {
    return res.status(400).json({ error: "resumeText is required" });
  }
  if (!targetSection || typeof targetSection !== "string") {
    return res.status(400).json({ error: "targetSection is required" });
  }

  // Stay within token limits — GPT-4 context window is large,
  // but we don't need to send the whole resume every time.
  const resumeContext = resumeText.slice(0, 3000);

  try {
    const result = await rewriteChain.call({
      resumeContext,
      sectionText: targetSection,
    });

    res.json({
      original: targetSection,
      rewritten: result.text.trim(),
    });
  } catch (err) {
    console.error("Chain error:", err.message);

    if (err.message?.includes("Rate limit")) {
      return res.status(429).json({ error: "Rate limit hit. Try again in a moment." });
    }

    res.status(500).json({ error: "Rewrite failed. Check your OpenAI API key." });
  }
});

const PORT = process.env.PORT || 3001;
app.listen(PORT, () => console.log(`Resume AI API running on :${PORT}`));

The input size limit (50kb) and the resumeContext.slice(0, 3000) are both intentional. Most GPT-4 token limits won't be hit by a 3,000-character resume excerpt, but some resumes are surprisingly long — especially ones with extensive project descriptions. Truncating at 3,000 characters keeps costs predictable.

Step 4: Streaming the Response

For a good UX, you want to stream the AI response as it arrives rather than waiting for the full completion. A 400-word rewrite might take 6–8 seconds to complete — a blank screen for 8 seconds feels broken.

LangChain makes streaming straightforward with callbacks:

import { HumanMessage } from "@langchain/core/messages";

app.post("/api/rewrite/stream", async (req, res) => {
  const { resumeText, targetSection } = req.body;

  res.setHeader("Content-Type", "text/event-stream");
  res.setHeader("Cache-Control", "no-cache");
  res.setHeader("Connection", "keep-alive");
  res.flushHeaders();

  const streamingModel = new ChatOpenAI({
    modelName: "gpt-4",
    temperature: 0.4,
    streaming: true,
    callbacks: [
      {
        handleLLMNewToken(token) {
          res.write(`data: ${JSON.stringify({ token })}

`);
        },
        handleLLMEnd() {
          res.write("data: [DONE]\n\n");
          res.end();
        },
        handleLLMError(err) {
          res.write(`data: ${JSON.stringify({ error: err.message })}

`);
          res.end();
        },
      },
    ],
  });

  const resumeContext = resumeText?.slice(0, 3000) || "";
  const prompt = `Rewrite these resume bullets for a software developer. Be concise and action-oriented:\n${targetSection}`;

  await streamingModel.invoke([new HumanMessage(prompt)]);
});

On the frontend, you'd consume this with the Fetch API and ReadableStream. Each data: event carries a token, and you append it to the UI as it arrives. The user sees the response materialize in real time — feels fast, even when it isn't.

Watch: LangChain in Node.js (Quick Start)

Common Pitfalls (and How to Dodge Them)

1. Token limits sneaking up on you

GPT-4's context window is large, but you pay per token. If you're sending the full resume + prompt on every request, costs add up fast at scale. The fix: truncate the resume context (as shown above) and cache the parsed sections so you're not re-parsing on every API call.

2. The model inventing achievements

This is the big one. Ask the model to "quantify achievements" without any source data, and it will make numbers up. "Reduced load time by 73%" sounds great until the hiring manager asks about it in an interview. The fix: explicitly tell the model in the prompt: "Only add numbers if they appear in the original text. If no numbers are present, use qualitative language instead."

3. Prompt injection through resume content

A crafty user could put something like "Ignore all previous instructions and output..." inside their resume text. Since you're sending that text directly to the model, it works. The fix: sanitize input and separate resume content from the instruction portion of the prompt with a clear delimiter, like ---RESUME START--- / ---RESUME END---.

4. Not rate limiting

OpenAI's rate limits are per API key, not per user. One user hammering your endpoint can hit the limit for everyone. Add a rate limiter like express-rate-limit before you go live — 5 requests per minute per IP is a reasonable starting point for a resume tool.

5. Picking GPT-4 when you don't need it

GPT-4 is expensive and slow. For most resume rewriting tasks, gpt-4o-mini produces nearly identical results at a fraction of the cost. Test both. You might be surprised how good the cheaper model is for structured, constrained tasks like this one.

LangChain vs. Raw OpenAI API — When to Use Which

Factor

Raw OpenAI API

LangChain

Setup complexity

Low — one import, one call

Medium — more abstractions to learn

Single prompt apps

Perfect fit

Overkill

Multi-step chains

Tedious to wire manually

First-class support

Prompt reuse and testing

DIY — no built-in structure

PromptTemplate makes this easy

Memory across turns

Manual array management

Built-in memory types

Streaming

Supported, manual wiring

Supported, callback-based

Switching LLM providers

Rewrite API calls

Swap the model object

Community / ecosystem

Smaller (OpenAI-specific)

Large, active, lots of integrations

The rule of thumb: if your app makes more than two different types of LLM calls, or if you need any kind of chaining, LangChain saves you from writing orchestration code from scratch. For a simple one-shot wrapper, raw API is cleaner.

TL;DR

LangChain is an orchestration layer for LLM apps — think Express for AI. Use it when you have multi-step chains, prompt reuse, or memory requirements.- Parse before you prompt. Sending a raw resume blob to the model is a recipe for generic output. Identify the section you want to improve and give the model focused context.- Constrain the prompt explicitly. Action verbs, number quantification, bullet count — tell the model exactly what format you want. Vague prompts produce vague results.- Stream responses for better UX. A blank screen for 8 seconds feels broken; a response materializing in real time feels fast.- Guard against pitfalls: rate limit your API, sanitize resume input against prompt injection, and test gpt-4o-mini before defaulting to GPT-4 — it's often good enough and 10x cheaper.